2. Tutorials
  3. hapi — Zero-Downtime Deployments Using PM2

hapi — Zero-Downtime Deployments Using PM2

The previous tutorial on how to correctly stop your hapi server already teasered another tutorial on the same topic with more focus on your production environment.

We at Future Studio are huge fans of hapi, which you can see at all the tutorials. Also, we love the awesome Node.js process manager PM2. In production, we completely rely on PM2 to run our hapi platform and both technologies work great in combination.

This tutorial shows you how to run and deploy your hapi application without downtime in production using PM2.

hapi Series Overview

  1. Get Your Server Up and Running
  2. v17 Upgrade Guide (Your Move to async/await)
  3. Become a Better Developer in 2018
  4. Become a Better Developer in 2017
  1. What You’ll Build
  2. Prepare Your Project: Stack & Structure
  3. Environment Variables and Storing Secrets
  4. Set Up MongoDB and Connect With Mongoose
  5. Sending Emails in Node.js
  6. Load the User’s Profile Picture From Gravatar Using Virtuals in Mongoose
  7. Implement a User Profile Editing Screen
  8. Generate a Username in Mongoose Middleware
  9. Displaying Seasons and Episodes for TV Shows with Mongoose Relationship Population
  10. Implementing Pagination for Movies
  11. Implement a Watchlist
  12. Create a Full Text Search with MongoDB
  1. Create a REST API with JSON Endpoints
  2. Update Mongoose Models for JSON Responses
  3. API Pagination for TV Shows
  4. Customize API Endpoints with Query Parameters
  5. Always Throw and Handle API Validation Errors
  6. Advanced API Validation With Custom Errors
  7. Create an API Documentation with Swagger
  8. Customize Your Swagger API Documentation URL
  9. Describe Endpoint Details in Your Swagger API Documentation
  10. 10 Tips on API Testing With Postman
  11. JWT Authentication in Swagger API Documentation
  12. API Versioning with Request Headers
  1. IP-Based Rate Limits (Part 1 of 7)
  2. Rate Limits for Routes (Part 2 of 7)
  3. Dynamic Rate Limits (Part 3 of 7)
  4. Render a “Rate Limit Exceeded” View (Part 4 of 7)
  5. Show “Rate Limit Exceeded” Error on Login (Part 5 of 7)
  6. Disable Rate Limiting (Part 6 of 7)
  7. Rate Limiter Refactoring & Cleanup (Part 7 of 7)
  1. API Login With Username and Password to Generate a JWT
  2. JWT Authentication and Private API Endpoints
  3. Refresh Tokens With JWT Authentication
  4. Create a JWT Utility
  5. JWT Refresh Token for Multiple Devices
  6. Check Refresh Token in Authentication Strategy
  7. Rate Limit Your Refresh Token API Endpoint
  8. How to Revoke a JWT
  9. Invalidate JWTs With Blacklists
  10. JWT Logout (Part 1/2)
  11. JWT “Immediate” Logout (Part 2/2)
  12. A Better Place to Invalidate Tokens
  13. How to Switch the JWT Signing Algorithm
  14. Roll Your Own Refresh Token Authentication Scheme
  15. JWT Claims 101
  16. Use JWT With Asymmetric Signatures (RS256 & Co.)
  17. Encrypt the JWT Payload (The Simple Way)
  18. Increase JWT Security Beyond the Signature
  19. Unsigned JSON Web Tokens (Unsecured JWS)
  20. JWK and JWKS Overview
  21. Provide a JWKS API Endpoint
  22. Create a JWK from a Shared Secret
  23. JWT Verification via JWKS API Endpoint
  24. What is JOSE in JWT
  25. Encrypt a JWT (the JWE Way)
  26. Authenticate Encrypted JWTs (JWE)
  27. Encrypted and Signed JWT (Nested JWT)
  28. Bringing Back JWT Decoding and Authentication
  29. Bringing Back JWT Claims in the JWT Payload
  1. How to Run Separate Frontend and Backend Servers Within a Single Project
  2. How to Use Server Labels
  3. How to Correctly Stop Your Server and Close Existing Connections
  1. Basic Authentication With Username and Password
  2. Authentication and Remember Me Using Cookies
  3. How to Set a Default Authentication Strategy
  4. Define Multiple Authentication Strategies for a Route
  5. Restrict User Access With Scopes
  6. Show „Insufficient Scope“ View for Routes With Restricted Access
  7. Access Restriction With Dynamic and Advanced Scopes
  8. hapi - How to Fix „unknown authentication strategy“
  9. Authenticate with GitHub And Remember the Login
  10. Authenticate with GitLab And Remember the User
  11. How to Combine Bell With Another Authentication Strategy
  12. Custom OAuth Bell Strategy to Connect With any Server
  13. Redirect to Previous Page After Login
  14. How to Implement a Complete Sign Up Flow With Email and Password
  15. How to Implement a Complete Login Flow
  16. Implement a Password-Reset Flow
  1. Views in hapi 9 (and above)
  2. How to Render and Reply Views
  3. How to Reply and Render Pug Views (Using Pug 2.0)
  4. How to Create a Dynamic Handlebars Layout Template
  5. Create and Use Handlebars Partial Views
  6. Create and Use Custom Handlebars Helpers
  7. Specify a Different Handlebars Layout for a Specific View
  8. How to Create Jade-Like Layout Blocks in Handlebars
  9. Use Vue.js Mustache Tags in Handlebars Templates
  10. How to Use Multiple Handlebars Layouts
  1. Extend Your Server Functionality With Plugins
  2. Create Your Own Custom Plugin
  3. How to Register Plugins for a Selected Server Instance
  4. hapi Plugin for Client Geo Location (by Future Studio 🚀)
  5. Increase Development Speed With Errors in the Browser or as JSON Response
  1. Route Handling and Drive Traffic to Your Server
  2. How to Serve Static Files (Images, JS, CSS, etc.)
  3. How to Use Query Parameters
  4. Optional Path Parameters
  5. Multi-Segment/Wildcard Path Parameters
  6. Ignore Trailing Slashes on Route Paths
  7. How to Fix “Unsupported Media Type”
  1. How to Access and Handle Request Payload
  2. Access Request Headers
  3. How to Manage Cookies and HTTP States Across Requests
  4. Detect and Get the Client IP Address
  5. How to Upload Files
  6. Quick Access to Logged In User in Route Handlers
  7. How to Fix “handler method did not return a value, a promise, or throw an error”
  8. How to Fix “X must return an error, a takeover response, or a continue signal”
  1. How to Reply a JSON Response
  2. How to Set Response Status Code
  3. How to Handle 404 Responses for Missing Routes
  1. Query Parameter Validation With Joi
  2. Path Parameter Validation With Joi
  3. Request Payload Validation With Joi
  4. Validate Query and Path Parameters, Payload and Headers All at Once on Your Routes
  5. Validate Request Headers With Joi
  6. Reply Custom View for Failed Validations
  7. Handle Failed Validations and Show Errors Details at Inputs
  8. How to Fix AssertionError, Cannot validate HEAD or GET requests
  1. Add CSRF Protection on Forms and API Endpoints
  1. Report Errors to Sentry
  2. Don’t Report Errors to Sentry in Development Mode
  3. Logging Fundamentals
  4. Log to Console With Good
  5. Log to File With Good
  1. Getting Started With Testing Using Lab and Code
  2. Run Tests with Assertions using Code
  3. Test Route Handlers by Injecting Requests
  4. Inject Request Payload, Headers and Parameters While Testing Route Handlers
  1. Zero-Downtime Deployments Using PM2

Correctly Shut Down Your Hapi Server

The mentioned tutorial on shutting down your hapi server correctly illustrates the process with just a single hapi server instance. You might wonder how it’s possible to interact with client requests and process them correctly while restarting the application? Actually, you’re right; it’s impossible to have a Schrödinger’s Server: being online and offline at the same time.

What you want is the process to finish existing requests first, response them appropriately and simultaneously don’t accept new connections so you can restart your hapi app once every open connection got closed.

So what’s the trick to restart your hapi app without downtime? If you already guessed that you’d at least require two instances of your hapi server: you’re completely correct! You need some kind of cluster where at least one instance is always able to handle requests and serve them correctly.

Prepare your Hapi Server for Zero Downtime Restarts

Before diving into the part of starting multiple hapi instances with PM2, you need to prepare your hapi server to actually be ready for zero-downtime restarts, also called graceful restarts. When receiving a signal to stop your hapi server, you need to properly close existing connections. Graceful restarts of your processes are an elegant way to keep your services available while updating the current deployment.

Therefore, install the hapi-graceful-pm2 plugin which handles the correct teardown and handling of existing requests.

server.register({  
  register: require('hapi-graceful-pm2'),
  options: {
    // the "listen_timeout" value within the PM2 process file 
    // needs to be greater or equal to this one here!
    timeout: 8000
  }
})

There’s only a single option to customize the behavior: timeout. The timeout defines the interval (in milliseconds) before hapi finally stops the server instance and cuts of all connections. If your server does some heavy lifting and executes long running tasks, make sure to increase the timeout value. If every connection is closed before the timeout, your server stops earlier and doesn’t wait for the timeout interval to expire.

Start Your hapi Application in Cluster Mode

Zero-downtime deployments require at least two instances of your applications so that in case of a restart you can shut down one after another and at each time there’s at least an instance able to accept and handle requests. You could build the cluster functionality for multiple instances yourself, but that’s not what you want. Trust me.

With PM2, there’s a great process manager doing a solid job providing the required features to manage your applications without headaches. Head over to our PM2 series to get more details. There’s a dedicated tutorial on PM2’s cluster mode and zero-downtime restarts that outlines PM2’s viewpoint on this topic. Within this tutorial, you’ll read about the hapi view of things :)

At this point, you should have installed PM2 on your server (or local machine). If you didn’t have the tool installed yet, just do it now and proceed from here.

To run your hapi app in cluster mode with at least two instances, you can either use the pm2 command line utility or define a process file. We prefer the process file because it’s easily extendable and you don’t need to remember the actual command that would be executed on the command line. You can’t forget what you configured :)

The following snippet defines a simple process file that allows you to restart your hapi app without downtime.

app.json

{
  "apps" : [{
    "name"            : "fs-platform",
    "script"          : "server.js",
    "instances"       : 2,
    "exec_mode"       : "cluster",
    "listen_timeout"  : 8000
  }]
}

The app.json includes the necessary configuration to run 2 instances in cluster mode of your application that gets kicked off with the server.js entry point. Running more than a single application instance will always result in PM2’s cluster mode.

Up to this point, you’ve prepared your hapi app for zero-downtime restarts with the hapi-graceful-pm2 plugin. Further, the PM2 process file defines two instances in cluster mode. Alright, you’re good to go.

Use the pm2 command line tool and start the app.json.

$ pm2 start pm2.json --env dev
[PM2][WARN] Applications fs-platform not running, starting...
[PM2] App [fs-platform] launched (2 instances)
┌─────────────┬────┬─────────┬───────┬────────┬─────────┬────────┬─────┬───────────┬──────────┐
│ App name    │ id │ mode    │ pid   │ status │ restart │ uptime │ cpu │ mem       │ watching │
├─────────────┼────┼─────────┼───────┼────────┼─────────┼────────┼─────┼───────────┼──────────┤
│ fs-platform │ 0  │ cluster │ 15394 │ online │ 0       │ 0s     │ 50% │ 32.9 MB   │ disabled │
│ fs-platform │ 1  │ cluster │ 15395 │ online │ 0       │ 0s     │ 40% │ 23.8 MB   │ disabled │
└─────────────┴────┴─────────┴───────┴────────┴─────────┴────────┴─────┴───────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app

The code snippet above illustrates the command line output of PM2’s start procedure. You can immediately spot the two instances of your application that run in cluster mode. That’s exactly what you wanted the outcome to be. You can now access your application within the browser via domain or IP address or whatever your setup looks like.

Graceful Restarts with Zero Downtime

Within the previous steps of this guide, you’ve prepared your hapi application to run in cluster mode with PM2 and already started two processes. Assuming that you’ve added new features to your application, you want to deploy them and therefore need to restart your instances.

Actually, what you want is a graceful restart without downtime.

Pull the updates for your application and ultimately use PM2’s reload command instead of restart. With reload, PM2 will restart your processes one after another and wait until at least a single instance is ready to handle requests before initiating the next restart.

$ pm2 reload fs-platform
Reloads are now immutable, to update environment or conf use --update-env  
[PM2] Applying action reloadProcessId on app [fs-platform](ids: 0,1)
[PM2] [fs-platform](0) ✓
[PM2] [fs-platform](1) ✓
┌─────────────┬────┬─────────┬───────┬────────┬─────────┬────────┬─────┬────────────┬──────────┐
│ App name    │ id │ mode    │ pid   │ status │ restart │ uptime │ cpu │ mem        │ watching │
├─────────────┼────┼─────────┼───────┼────────┼─────────┼────────┼─────┼────────────┼──────────┤
│ fs-platform │ 0  │ cluster │ 17485 │ online │ 1       │ 2s     │ 78% │ 121.0 MB   │ disabled │
│ fs-platform │ 1  │ cluster │ 17486 │ online │ 1       │ 2s     │ 85% │ 121.5 MB   │ disabled │
└─────────────┴────┴─────────┴───────┴────────┴─────────┴────────┴─────┴────────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app

The reload procedure will take longer, because of the waiting time for each instance being up again. Be patient while all instances of your application get restarted. Users requesting your application won’t notice the reload process going on behind the curtain. They’ll notice the deployed update in case there are visual changes on your websites.

Awesome! You did it! From now on, you don’t need to worry about downtimes during deployments. \o/

The Curse and Blessing of Timeouts

Throughout this tutorial, you’ve seen multiple timeouts. At first, while preparing your hapi server with the hapi-graceful-pm2 plugin, you had the option to define a timeout interval that hapi waits at most before shutting down.

Then, within the PM2 process file app.json, you’ve seen a listen_timeout which is the interval PM2 waits for a listen signal that indicates your server is back up „listening“ for connections. If you exceed the listen_timeout, PM2 will restart your process the hard way to get it back up.

We didn’t show you within this tutorial, but there’s an equivalent environment variable to listen_timeout called PM2_GRACEFUL_LISTEN_TIMEOUT. If you used the environment variable before and want to switch to the definition within the process file, make sure to clean up properly so you don’t get caught into strange behavior due to different values for listen_timeout and PM2_GRACEFUL_LISTEN_TIMEOUT.

The curse of all the timeouts is that you need to keep them either in sync or at least set the timeout for the hapi-graceful-pm2 plugin to a greater or equal value compared to the listen_timeout. Because both timeouts are used by another process and they shouldn’t interfere each other.

The hapi-graceful-pm2 timeout is used by hapi itself to wait at most the defined timeout before shutting down the hard way. The listen_timeout is used by PM2 to wait at most the defined time within that your hapi server should listen for connections again.

Outlook

Zero-downtime deployments are an essential part of your deployment pipeline because you skip all concerns that affect active users on your page. With the help of PM2, you can easily deploy your hapi application and gracefully restart the related processes.

Prepare your hapi server with the hapi-graceful-pm2 plugin to shut down your server correctly. Using PM2’s reload command sends a signal to stop your server (which closes existing connections) and also restarts your server processes sequentially. And the combination of your correct shutdown and the sequential restart results in always having a hapi instance available for client requests.

This extensive guide may arise various questions. Please use the comments below or shoot us a tweet @futurestud_io if you want to know more details about zero downtime deployments in hapi using PM2.

Make it rock & enjoy coding!

Marcus Pöhls

Marcus is a fullstack JS developer. He’s passionate about the hapi framework for Node.js and loves to build web apps and APIs. Creator of Futureflix and the “learn hapi” learning path.

Explore the Library

Find interesting tutorials and solutions for your problems.

Android

Node.js

Server

Miscellaneous