Migrating an Application to the Cloud: Node.js Graceful Shutdown

When you're migrating an application to the cloud and every deploy causes a short burst of 502s, the fix is usually two small changes. Give your Node.js service honest health check endpoints, and make it shut down gracefully when it gets SIGTERM. Together they make the load balancer stop sending traffic to an instance before that instance stops answering.

Maybe you just containerised a service that ran fine on one VM for years, and now your error graph jumps for half a minute every time you ship. If so, this is almost certainly the cause. Your business logic is fine. The service has simply never had to shut down cleanly before.

Why do rolling deploys drop requests?

A rolling deploy starts new instances and then stops the old ones. The trouble is timing. When the orchestrator (Kubernetes, ECS, Nomad) stops an old container, it sends SIGTERM and, at about the same moment, tells the load balancer to remove that instance. These two signals travel separately, and the load balancer often lags a few seconds behind.

By default, a Node.js process exits as soon as it receives SIGTERM. So for those few seconds the load balancer keeps sending requests to a process that no longer exists. Any requests already in progress are lost along with it.

Startup is the other half. Without a readiness check, a new container starts getting traffic as soon as its port opens. That can happen before the database pool has connected, so the first requests fail too.

Then there's keep-alive. Node's default keepAliveTimeout is 5 seconds, while an AWS Application Load Balancer holds idle connections open for 60. The balancer reuses a socket that Node has already closed, and the user gets a 502 even when no deploy is running.

What does getting this wrong cost when migrating an application to the cloud?

The errors are brief, so teams learn to tolerate them. That's where the real cost starts. Engineers deploy less often and bundle more changes into each release, because every release means a minute of failed checkouts or broken API calls. Bigger releases are riskier and slower to roll back.

Clients calling your API see random failures and add retries. If your endpoints aren't idempotent, those retries can turn a small blip into duplicate orders.

Real incidents also get lost in the usual deploy noise, so alerts get loosened until they stop catching anything.

Who can skip most of this?

If your code runs as functions on Lambda or a similar platform, the platform manages each request's lifecycle and you can ignore the draining logic. You can also skip it if you run a single instance, deploy in an agreed maintenance window, and nobody minds a short outage. If you run more than one long-lived container behind a load balancer, you need it.

Step 1: What is the difference between liveness and readiness?

Liveness asks whether the process is stuck and should be restarted. Readiness asks whether this instance should receive traffic right now. Mixing the two up is the most common mistake in newly migrated services.

const express = require('express')
const { Pool } = require('pg')

const app = express()
const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 10 })
let shuttingDown = false

const withTimeout = (promise, ms) =>
  Promise.race([
    promise,
    new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), ms)),
  ])

// Liveness: is the event loop responding? Nothing else.
app.get('/healthz', (req, res) => res.status(200).send('ok'))

// Readiness: should the load balancer send traffic here?
app.get('/readyz', async (req, res) => {
  if (shuttingDown) return res.status(503).send('draining')
  try {
    await withTimeout(pool.query('SELECT 1'), 1000)
    res.status(200).send('ready')
  } catch (err) {
    res.status(503).send('not ready')
  }
})

Two rules matter here. First, liveness must never check the database. If the database has a bad minute and liveness fails, the orchestrator restarts every instance at once, and a slow database becomes a full outage. Second, readiness can check dependencies, but only the ones you truly can't serve without. Give that check a short timeout so a slow query can't make the probe hang.

Wire both into your deployment:

spec:
  terminationGracePeriodSeconds: 30
  containers:
    - name: orders-api
      readinessProbe:
        httpGet: { path: /readyz, port: 3000 }
        periodSeconds: 5
        failureThreshold: 2
      livenessProbe:
        httpGet: { path: /healthz, port: 3000 }
        periodSeconds: 10
        failureThreshold: 3

Step 2: How do you drain connections on SIGTERM?

Make readiness fail, wait for the load balancer to notice, stop accepting new connections, let in-flight requests finish, then exit. That order is the whole technique.

const server = app.listen(process.env.PORT || 3000)

// Must exceed the load balancer idle timeout (60s on AWS ALB)
server.keepAliveTimeout = 65000
server.headersTimeout = 66000

process.on('SIGTERM', () => {
  console.log('SIGTERM received, draining')
  shuttingDown = true // readiness now returns 503

  // Hard stop before the orchestrator sends SIGKILL
  setTimeout(() => process.exit(1), 25000).unref()

  // Give the load balancer time to stop routing here
  setTimeout(() => {
    server.close(async () => {
      await pool.end()
      console.log('clean shutdown')
      process.exit(0)
    })
    server.closeIdleConnections() // Node 18.2+
  }, 10000)
})

The 10 second wait can look like wasted time. It isn't. It covers the gap between SIGTERM and the load balancer actually removing the instance. A reasonable starting value is your readiness period multiplied by the failure threshold, plus a few seconds for the change to propagate. On ECS, also check the target group's deregistration delay.

The hard exit timer is your safety net. It has to fire before terminationGracePeriodSeconds runs out, or the orchestrator will SIGKILL the process in the middle of cleanup.

Step 3: Close the database pool last

Order matters here too. If you call pool.end() before the HTTP server has finished, in-flight requests try to query a closed pool. The errors they throw look like database problems, which sends whoever is debugging in the wrong direction.

Close the server, wait for its callback, then end the pool. Do the same for Redis clients, queue consumers and anything else that holds a socket open. If the same process runs background jobs, stop pulling new jobs first and let the current job finish within the grace period.

Is your container even receiving SIGTERM?

Often it isn't. With CMD npm start in your Dockerfile, npm runs as PID 1, and npm does not reliably pass signals on to your Node process. Your handler never runs. The orchestrator waits out the full grace period and then kills everything.

CMD ["node", "server.js"]

Use the exec form shown above, or run with docker run --init (or set tini as your entrypoint) so a proper init process forwards signals.

How do you prove it works?

Don't rely on code review. Put load on the service while you restart it:

hey -z 60s -c 20 https://staging.example.com/api/orders &
kubectl rollout restart deployment/orders-api

A healthy result has zero non-2xx responses across the whole restart. If you still see 502s, check three things in this order. Is SIGTERM reaching Node (add a log line and look for it)? Is the drain delay longer than the balancer's deregistration lag? Is keepAliveTimeout higher than the balancer's idle timeout? Any one of these on its own causes the same symptom, which is why guessing can waste days.

Health checks and graceful shutdown are one small part of a cloud move. Deciding which services move first and how to cut over without downtime is covered in this guide to sequencing a cloud migration plan without downtime. If your deploys still drop requests after these changes, Geminate Solutions will review your setup for free and reply within 24 hours.