Your API works perfectly on your laptop. The endpoints return the expected responses, database queries succeed, and your frontend communicates with the backend without any issues.
Then you deploy.
Suddenly, requests return 500 Internal Server Error. Authentication stops working. Database connections fail. Some endpoints work in Postman but fail in the browser. Other requests remain pending until they time out.
The frustrating part is that you did not intentionally change the application logic.
This is one of the most challenging parts of software development: an application that works in a development environment does not necessarily behave the same way in production.
The solution is not to randomly change configuration values or rewrite working code. It is to identify what changed between the two environments and isolate the failure systematically.
In this guide, we will examine the most common causes of production API failures, practical debugging techniques, and a repeatable workflow developers can use to diagnose problems without wasting hours.
1. Start by Identifying Where the Request Fails
Before changing code, determine which part of the request lifecycle is failing.
A typical API request passes through several stages:
- The client sends an HTTP request.
- DNS resolves the API domain.
- A network connection is established.
- A reverse proxy or load balancer forwards the request.
- The application receives and processes it.
- Authentication and validation run.
- The application accesses a database or external service.
- The server returns a response.
- The client receives and processes that response.
A failure at any stage can make the API appear broken.
For example, a 502 Bad Gateway may indicate that a proxy cannot communicate successfully with the upstream application. A 401 Unauthorized usually points toward authentication, while a 500 Internal Server Error indicates an unexpected server-side failure.
These status codes are clues, not definitive diagnoses.
Start with the browser's Network tab, your API client, and the server logs. Record the failing endpoint, HTTP method, status code, response body, and approximate timestamp.
Then compare a successful local request with the failing production request.
The objective is simple: identify the earliest stage where their behavior differs.
2. Missing or Incorrect Environment Variables
Environment configuration is one of the first things to investigate when an application works locally but fails after deployment.
Your local machine may contain a .env file with values such as:
PORT=3000
DATABASE_URL=postgresql://localhost:5432/myapp
JWT_SECRET=local-development-secret
API_BASE_URL=http://localhost:3000
However, your production server may use a completely different configuration system.
The deployment platform might not have all the required variables. A variable name may be incorrect, or the application may be reading a development URL instead of the production URL.
For example:
const databaseUrl = process.env.DATABASE_URL;
if (!databaseUrl) {
throw new Error("DATABASE_URL is not configured");
}
Failing early with a clear error is better than allowing the application to start and fail unpredictably when the first database request arrives.
You can also validate configuration during startup:
const requiredVariables = [
"DATABASE_URL",
"JWT_SECRET",
"API_BASE_URL"
];
for (const key of requiredVariables) {
if (!process.env[key]) {
throw new Error(
`Missing required environment variable: ${key}`
);
}
}
Do not print secret values to the logs. Log whether a required variable exists, not its contents.
Also remember that frontend environment variables and backend environment variables are not necessarily interchangeable. Some frontend build systems embed configuration into the generated bundle during the build process. Changing a server variable after deployment may not update an already-built frontend.
Debugging tip: Compare configuration names and non-sensitive settings between environments. Check whether the application was rebuilt or restarted after configuration changes.
Reference: Node.js documentation on environment variables.
3. CORS Errors That Appear Only in Production
Cross-Origin Resource Sharing, or CORS, is another common source of confusion.
Imagine your frontend runs locally at:
http://localhost:5173
Your backend runs at:
http://localhost:3000
During development, your backend might explicitly allow the local frontend origin.
After deployment, the frontend moves to:
https://app.example.com
If the backend does not allow that production origin, browser requests can fail even though the API itself is running correctly.
For an Express application, a basic configuration might look like this:
import express from "express";
import cors from "cors";
const app = express();
const allowedOrigins = [
"http://localhost:5173",
"https://app.example.com"
];
app.use(cors({
origin: allowedOrigins,
methods: ["GET", "POST", "PUT", "PATCH", "DELETE"],
allowedHeaders: ["Content-Type", "Authorization"]
}));
This is only an example. Your allowed origins, methods, and headers should match your application's actual requirements.
If your application uses cookies or other credentialed requests, additional configuration is needed. Credentialed CORS requests cannot use a wildcard origin.
Another important distinction: CORS is enforced by browsers. A request succeeding in Postman does not prove that a browser request will succeed.
https://goodoff.co/
Check the browser console and inspect the response headers. Also verify that the server handles preflight OPTIONS requests correctly.
Avoid fixing CORS by allowing every origin without understanding the security implications.
Reference: MDN: Cross-Origin Resource Sharing.
4. Database Connections That Work Locally but Fail After Deployment
Local development often uses a database running on the same machine. Production databases typically have different network boundaries, authentication rules, connection limits, and security requirements.
A connection string that works locally may point to the wrong host in production.
Other common causes include:
- The production database does not allow connections from the application.
- Credentials are incorrect or expired.
- The database requires TLS.
- The application exceeds its connection pool limit.
- Database migrations have not been applied.
- The database hostname cannot be resolved.
- The database is temporarily unavailable.
Start by checking the exact database error in the server logs.
A connection timeout suggests a different problem from an authentication failure or a missing table.
For example, if your application reports that a relation does not exist, repeatedly changing firewall rules will not solve the issue. You need to investigate schema migrations and database selection.
A useful production health check can verify database connectivity:
app.get("/health", async (req, res) => {
try {
await db.query("SELECT 1");
res.status(200).json({
status: "ok"
});
} catch (error) {
console.error("Database health check failed", {
name: error.name,
code: error.code
});
res.status(503).json({
status: "unavailable"
});
}
});
Here, db.query() represents your database client's query method. Adapt it to your database library.
Keep health checks separate from detailed diagnostic endpoints. Public health endpoints should not expose connection strings, internal hostnames, stack traces, or other sensitive infrastructure details.
5. Reverse Proxies, Ports, and Deployment Configuration
Your application may listen on the correct port locally but fail behind a production proxy.
Consider this code:
app.listen(3000, () => {
console.log("Server started");
});
This can work locally, but some hosting environments expect the application to use a port supplied through an environment variable.
A more flexible configuration is:
const port = Number(process.env.PORT) || 3000;
app.listen(port, "0.0.0.0", () => {
console.log(`Server listening on port ${port}`);
});
Whether you need to bind to 0.0.0.0 depends on your hosting environment. Some platforms manage networking differently, so follow their deployment requirements.
You should also investigate reverse proxy configuration.
For example, the proxy might forward /api/users to an incorrect upstream address. It may also remove a URL prefix, impose a request body size limit, or terminate a connection before the application finishes processing.
When investigating these problems, check:
- Is the application process running?
- Is it listening on the expected port?
- Can the proxy reach the application?
- Are the forwarded paths correct?
- Are proxy and application timeout settings compatible?
- Do the proxy logs show an upstream connection error?
A 502 or 504 should lead you to investigate the proxy and upstream service rather than immediately changing application business logic.
6. Production Builds Behave Differently
Development and production builds may execute different code paths.
A development server can provide hot reloading, local proxies, detailed error messages, and permissive settings. A production build may use optimized assets, different environment variables, stricter security rules, or a different startup command.
For frontend applications, a development proxy may forward requests from /api to your local backend. That proxy may not exist after deployment.
For backend applications, a missing dependency, incorrect module format, or incompatible runtime version can prevent the application from starting.
Reproduce the production build as closely as possible before deploying:
npm ci
npm run build
npm start
These commands are examples. Use the scripts defined by your project.
Also verify the runtime version:
node --version
npm --version
Compare the results with your production runtime. If your application depends on a particular Node.js version, declare that requirement and configure the deployment environment accordingly.
For containerized applications, test the actual production container image rather than relying exclusively on your local development environment.
7. Authentication and HTTPS Problems
An endpoint may work locally because your development environment uses different authentication settings.
Production introduces additional variables, including HTTPS, cookie security, OAuth redirect URLs, token expiration, and secret rotation.
Suppose your frontend sends a bearer token:
const response = await fetch(
"https://api.example.com/profile",
{
headers: {
Authorization: `Bearer ${token}`
}
}
);
If the request returns 401, investigate whether the token is present, valid, unexpired, and signed with the expected key.
If the server returns 403, the identity may be valid but lack the necessary permission. However, applications can use these status codes differently, so inspect your authentication middleware.
For cookie-based authentication, verify the cookie's domain, Secure, SameSite, and path settings. Cross-site cookie behavior can differ significantly between localhost and production.
Never solve a TLS certificate problem by disabling certificate verification. Fix the certificate chain, hostname, trust configuration, or server certificate instead.
Useful checks include:
- Confirming that the correct production identity provider is configured.
- Verifying allowed OAuth redirect URLs.
- Checking token issuer and audience validation.
- Confirming that cookies are actually sent by the browser.
- Ensuring that secret rotation has been applied consistently.
8. Missing Logs Turn Small Problems Into Long Investigations
When production fails, a developer needs evidence.
A generic message such as Something went wrong does not tell you whether the failure came from a database query, external API, validation rule, or application bug.
Structured logging makes requests easier to trace:
console.error("Request failed", {
requestId,
method: req.method,
path: req.path,
errorName: error.name,
errorCode: error.code
});
In a real application, use a logging library and middleware to generate or propagate request IDs.
A request ID allows you to connect the same request across application logs, proxy logs, and downstream service logs.
Be careful about logging request bodies, authorization headers, passwords, session cookies, and personal information. Diagnostic data should help you investigate the failure without creating a security incident.
Production logging should capture enough information to answer four questions:
- What operation failed?
- When did it fail?
- Which dependency or component was involved?
- What error category explains the failure?
For larger systems, distributed tracing can help identify where latency accumulates across multiple services.
References: Express debugging guide and NestJS deployment and observability guidance.
9. A Systematic Debugging Workflow
Instead of trying fixes randomly, follow a consistent process.
Step 1: Reproduce the failure
Record the endpoint, HTTP method, request payload, response status, and timestamp. Use a test account and non-sensitive data whenever possible.
Step 2: Compare environments
Compare runtime versions, environment variable names, API base URLs, database configuration, network access, and build commands.
Step 3: Inspect the response
Look at response headers, response bodies, browser console messages, and network timing. Determine whether the request reached the application.
Step 4: Check server-side evidence
Search logs using the request timestamp or request ID. Identify the first meaningful error rather than focusing only on the final HTTP status.
Step 5: Test dependencies independently
Verify database connectivity, external API availability, DNS resolution, authentication, and network access from the production environment.
Step 6: Change one variable at a time
Apply the smallest reasonable fix. Changing several settings simultaneously makes it harder to identify the actual cause.
Step 7: Verify the fix
Repeat the original failing request. Test nearby cases, inspect logs, and confirm that the change has not introduced a new problem.
Step 8: Prevent recurrence
Add regression tests, configuration validation, monitoring, or deployment checks that catch the same failure earlier next time.
This process may feel slower than immediately changing code, but it usually reduces wasted debugging time.
10. Build a Production Readiness Checklist
Before deploying an API, verify the following:
- [ ] Required environment variables are validated at startup.
- [ ] Production secrets are configured securely.
- [ ] The application uses the expected runtime version.
- [ ] Database connectivity and migrations are verified.
- [ ] CORS allows only the required origins.
- [ ] Authentication and authorization work in the production environment.
- [ ] Reverse proxy routes and timeout settings are correct.
- [ ] Logs include request IDs and useful error context.
- [ ] Health checks distinguish application availability from dependency health.
- [ ] External services have sensible timeout and retry policies.
- [ ] Production builds are tested before release.
- [ ] A rollback procedure exists for failed deployments.
Not every application needs the same infrastructure. A small API may not require distributed tracing or a complex monitoring platform. The important thing is to introduce controls that match your application's risks and operational requirements.
Conclusion
When an API works locally but fails in production, the problem is not necessarily the code itself. It may be configuration, networking, database access, authentication, deployment behavior, or an interaction between several components.
The most effective approach is to isolate the failing stage, compare the two environments, collect evidence, and test one hypothesis at a time.
Developers who treat production debugging as a repeatable engineering process spend less time guessing and more time fixing the actual cause.
The goal is not just to make the API work again. It is to understand why it failed and make the next failure easier to diagnose.
Top comments (0)