A service can be successfully deployed and still be unprepared for production.
The application may be running, health checks may be passing, and the deployment pipeline may report success. But what happens when a dependency becomes unavailable? Who responds when an alert fires? Can the team restore the service without relying on the person who originally built it?
These are operational questions, and they are often left unanswered until the first serious incident.
An operational readiness review helps identify these gaps before they affect users. The goal is not to create another approval checklist. It is to establish whether the team can operate, troubleshoot, recover, and maintain the system after it goes live.
Why Traditional Readiness Checklists Fail
Many readiness reviews focus heavily on infrastructure and deployment requirements. They ask whether the service has been deployed, whether monitoring is enabled, whether backups are configured, and whether documentation exists.
These checks are useful, but they do not necessarily demonstrate operational readiness.
For example, having a backup configuration does not prove that the team can restore the service. Having alerts does not prove that someone owns them. Having a runbook does not prove that the instructions work during an incident.
A useful readiness review should examine operational evidence rather than simply confirm that a configuration exists.
1. Establish Clear Operational Ownership
Before reviewing infrastructure, establish who owns operating the service. Ownership should cover more than the development team that built the application. It should include responsibility for incidents, routine maintenance, dependencies, security updates, and operational decisions.
The review should answer questions such as:
- Which team owns the service in production?
- Who is responsible for responding to critical alerts?
- Who can approve or coordinate an emergency change?
- Which team owns each important external dependency?
- How are incidents escalated when the primary owner is unavailable?
Ownership also needs to be visible in the tools engineers already use. A service catalogue, incident management platform, or internal documentation system can help make responsibility discoverable.
If an alert fires and the team cannot identify who should respond, the service has an ownership gap regardless of how well it was deployed.
2. Check Whether Telemetry Helps Engineers Diagnose Problems
Monitoring should help engineers understand what is happening, not simply indicate that something is wrong. A readiness review should examine whether the service produces enough useful telemetry to support investigation.
Metrics help identify trends and abnormal behaviour. Logs provide context about application events and failures. Traces help engineers understand how requests move across services and where latency or errors are introduced.
These signals become more useful when they can be correlated. For example, imagine a request that starts failing after an application deployment. An engineer should be able to identify the affected service, inspect relevant logs, follow the request across dependencies, and compare behaviour before and after the change.
Review questions should include:
- Can engineers identify the affected service from an alert?
- Are important errors recorded with enough context to investigate?
- Can requests be traced across relevant service boundaries?
- Are latency, availability, and error trends visible?
- Are alerts actionable, or do they generate noise without helping diagnosis?
Telemetry should also have clear ownership. An alert that nobody understands or maintains can become another source of operational risk.
3. Review the Incident Response Process
Incident response should not depend on engineers improvising under pressure. The team should understand how incidents are detected, investigated, escalated, and communicated.
A useful review examines whether alerts reach the right people and whether responders have enough information to make decisions. It should also establish what happens when the first responder cannot resolve the issue.
Runbooks and playbooks should describe the expected outcome, the steps involved, the tools and permissions required, possible errors, and escalation paths. However, documentation alone is not sufficient. Someone other than the original author should be able to follow the instructions and understand what to do.
Consider a service that depends on an external payment provider. If requests begin timing out, the response process should help engineers distinguish between an application issue, a network problem, and a provider outage.
The team should know whether to retry requests, activate a fallback, pause affected operations, or escalate to another owner. The correct response depends on the system, but it should be understood before an incident occurs.
4. Demonstrate Recovery Instead of Assuming It Works
Recovery is one of the most important parts of operational readiness. A configured backup does not guarantee that data can be restored. A rollback mechanism does not guarantee that a previous application version will work with the current database schema.
The review should examine how the system behaves when recovery is necessary.
Important questions include:
- Has the team tested restoring data from backup?
- Can the application be rolled back safely?
- What happens if a critical dependency becomes unavailable?
- Are recovery objectives documented and realistic?
- Can the team verify data integrity after recovery?
- Are recovery procedures accessible to the people expected to use them?
Testing should include realistic failure scenarios where practical. For example, a team might simulate an unavailable dependency or perform a controlled restore in a non-production environment. The purpose is to identify missing permissions, undocumented steps, unexpected dependencies, and assumptions that would otherwise appear during a real incident.
Recovery procedures should be treated as operational capabilities that need regular validation.
5. Examine Deployment and Change Safety
Production systems continue to change after their initial release. New application versions, configuration updates, infrastructure changes, and dependency upgrades can all introduce risk.
A readiness review should establish whether the team can make changes safely and identify problems quickly.
This includes reviewing:
- Deployment and rollback procedures.
- Health checks and release validation.
- Configuration and secret management.
- Database migration strategies.
- Feature flags or other controlled rollout mechanisms.
- Approval and escalation processes for high-risk changes.
Database changes deserve particular attention. An application rollback may not reverse a database migration, especially when the new version has already written data using a different schema.
The team should understand how application versions, database compatibility, and rollback procedures interact. A deployment process is not operationally safe simply because it is automated. It is safe when the team understands the consequences of a change and has a practical way to respond when it fails.
6. Review Dependencies and Operational Capacity
Most production services rely on more than their own application code. They may depend on databases, queues, cloud services, identity providers, third-party APIs, and internal platforms.
A readiness review should identify important dependencies and examine how failures affect the service.
For each critical dependency, engineers should understand:
- What functionality depends on it?
- What happens when it becomes slow or unavailable?
- Are timeouts, retries, and circuit breakers configured appropriately?
- Can the application degrade gracefully?
- Is there a fallback or recovery procedure?
- Who is responsible for resolving dependency-related problems?
Retries deserve special attention. Uncontrolled retries can increase load on an already unhealthy dependency and make an incident worse. Idempotency, bounded retry policies, timeouts, and backoff strategies should be considered where appropriate.
Operational capacity also matters. A service may have reliable infrastructure but still be difficult to operate if only one engineer understands its architecture or recovery process.
The review should identify knowledge gaps, access constraints, and responsibilities that could prevent the team from responding effectively.
7. Make the Readiness Decision Evidence-Based
The outcome of an operational readiness review should not be a vague statement that everything looks good. A practical approach is to use three possible outcomes.
Ready: The required operational capabilities have been demonstrated, and no unresolved critical gaps prevent production operation.
Ready with accepted risk: Some gaps remain, but they have been documented, assigned to an owner, and explicitly accepted by the appropriate decision-maker.
Not ready: Critical operational requirements are missing, untested, or unsupported by sufficient evidence.
Every identified gap should have an owner and a clear next action. Where appropriate, it should also have a target date and a defined acceptance authority. This prevents the review from becoming a collection of observations that nobody is responsible for resolving.
A Practical Review Structure
Instead of maintaining a long checklist that teams complete without discussion, organise the review around a few operational areas:
Service ownership: Confirm the accountable team, incident contacts, escalation paths, and dependency owners.
Observability: Review service metrics, logs, traces, alert quality, and the ability to diagnose failures.
Incident response: Validate runbooks, response procedures, escalation, and communication responsibilities.
Recovery: Examine backup restoration, rollback, dependency failure handling, and data integrity.
Change safety: Review deployment validation, configuration management, migrations, and release controls.
Operational capacity: Identify access requirements, knowledge gaps, maintenance responsibilities, and support constraints.
For each area, record the evidence reviewed, the gaps identified, the person responsible, and the resulting decision. This keeps the review focused on whether the service can actually be operated rather than whether a document has been completed.
Keep Readiness Relevant After Go-Live
Operational readiness is not a one-time approval. Services change. Dependencies are replaced, teams reorganise, infrastructure evolves, and incident patterns reveal weaknesses that were not obvious during the initial review.
The review should therefore be revisited after significant architectural changes, major incidents, changes in ownership, and the introduction of critical dependencies.
Teams should also use incident findings to improve their operational procedures. If an engineer had to discover a recovery step during an outage, that knowledge should be incorporated into the appropriate runbook or automation.
A readiness review is valuable when it reflects how the system operates today, not how it was originally designed.
Conclusion
A production deployment confirms that software has been released. It does not confirm that the organisation is prepared to operate it.
A useful operational readiness review examines ownership, observability, incident response, recovery, change safety, and dependencies. More importantly, it asks teams to demonstrate these capabilities rather than relying on configuration screenshots or completed checklists.
The real test is straightforward: when the system behaves unexpectedly, can the responsible team understand what is happening, make a safe decision, recover the service, and maintain it afterwards?
Top comments (0)