A green pipeline isn't the same thing as a safe deploy

Cyber Technology — Wed, 12 Aug 2026 16:39:39 +0000

Every team eventually builds the same mental shortcut: green pipeline means it's safe to ship. Tests passed, the build compiled, the checks are all checkmarks. Merge, deploy, move on.

That shortcut is wrong more often than most postmortems admit, and it's wrong in a specific, repeatable way: CI/CD pipelines are very good at verifying that code does what it's supposed to do, and almost silent about whether the environment it's about to land in still matches the one it was tested against.

Three places this gap actually bites, in order of how often I've seen each one cause a real incident.

1. The test suite passed against a database that no longer looks like production

Migrations are the classic case. A pipeline runs unit and integration tests against a freshly-migrated test database, everything's green, and the deploy goes out. What the pipeline didn't check: whether that migration is actually safe to run against a production table with real row counts, real lock contention, and real traffic hitting it mid-migration.

A migration that adds a NOT NULL column with no default locks the table for the duration of the backfill on most engines. On a test database with a thousand rows, that's invisible — it finishes before you'd notice. On a production table with tens of millions of rows, it can hold a lock long enough to back up every write for minutes, which looks nothing like "the tests failed" and everything like "the site is down."

The fix isn't a smarter test — it's a different question in the pipeline: not "does this migration produce the right schema," but "is this migration safe to run online, at this table's actual size, without an explicit maintenance window." That's a check most CI setups never encode, because it requires knowing something about production the pipeline usually isn't told.

2. The deploy succeeded, but nobody ever tested the rollback

Almost every deploy pipeline is optimized to answer one question: did the new version start successfully? Very few are set up to actually exercise the answer to "if this version is bad, can we get back to the old one without a human improvising at 2am?"

Rollback usually gets tested for the first time during an actual incident, which is the worst possible time to discover that the previous image was already garbage-collected off the registry, or that a schema migration in the new release isn't backward-compatible with the old code, or that a feature flag the new version depends on doesn't exist in the old one's code path.

If a pipeline can deploy a version, it should also be able to prove it can un-deploy that version, on a schedule, not just in theory. That's a genuinely different pipeline stage than "ship the new thing" — most teams have the first and skip the second.

3. Staging is green because staging quietly drifted from what production actually runs

This is the slow one. Staging starts as an accurate mirror of production. Six months later, staging is running a slightly older version of a managed service, a smaller instance class, a different set of feature flags, or a mocked-out third-party dependency that production talks to for real. Every pipeline run against staging is honestly green — the code really does work, in an environment that no longer represents what it's about to be deployed into.

Nobody decides to let this happen. It accumulates one small, individually-reasonable shortcut at a time: "let's just mock that API in staging for now," "we'll upgrade staging's database version next sprint." Each shortcut is fine in isolation. The sum of them is a green pipeline that's answering a question nobody's actually asking anymore.

The only real defense is treating environment parity as something that gets checked, not assumed — a periodic, automated diff between staging's actual configuration and production's, not a one-time setup step everyone trusts forever after.

The pattern underneath all three

None of these are exotic failures. They're the ordinary, boring result of a pipeline that was built to answer "does the code work" being trusted to also answer "is it safe to put this in front of real traffic right now" — a question it was never actually wired to check.

Closing that gap isn't about adding more tests in the traditional sense. It's about adding a small number of specific, deliberate checks that ask about the deploy itself, not just the code: is this migration safe at production scale, is rollback actually exercised, does staging still resemble what's about to receive this change. Each one is cheap to build once someone decides it's a gap worth closing — the hard part is noticing the gap exists before an incident points it out.

I do this kind of infrastructure/CI-CD hardening work professionally through CyberTechnology — happy to talk through a specific pipeline if any of this sounds familiar.

O padrão que aparece em quase todo produto construído rápido com IA

Cyber Technology — Tue, 11 Aug 2026 04:05:27 +0000

Um padrão que aparece com frequência em produtos construídos rápido com ferramentas de IA (Lovable, Cursor, Claude Code, N8N, Supabase, entre outras): múltiplos provedores de LLM — Claude, GPT, Gemini — por trás de uma camada de integração fina, sem observabilidade real de custo, latência ou taxa de erro por provedor.

Funciona perfeitamente até o dia em que:

um provider degrada e ninguém percebe na hora;
o custo por chamada dobra silenciosamente;
um bug de prompt sutil começa a gerar respostas erradas para uma fração dos usuários, e não existe log estruturado para achar isso rápido.

Nada disso é sobre trocar de stack ou "fazer certo desde o início". É sobre colocar instrumentação, circuit breakers e rastreabilidade onde hoje só existe "chama a API e reza" — depois que o produto já está em produção, com clientes de verdade usando.

Por que isso é comum, não uma exceção

Ferramentas de IA aceleram muito bem a parte de "fazer funcionar". Elas não respondem — e não deveriam responder — a perguntas como:

o que acontece quando esse provider está fora do ar?
como eu sei que uma resposta está errada antes do cliente perceber?
quanto isso vai custar em escala, e onde está o log que me diz isso?

Essas são perguntas de arquitetura, não de velocidade de entrega. E é exatamente aí que a maioria dos produtos vibe-coded para de crescer sem dor.

O que muda na prática

Não é reescrever do zero. É adicionar, em cima do que já existe:

Observabilidade por provedor — custo, latência e taxa de erro separados por Claude/GPT/Gemini/etc, não agregados.
Circuit breakers reais — fallback automático quando um provider degrada, não descoberto às 3h da manhã por um cliente reclamando.
Log estruturado de prompt/resposta — para conseguir investigar um bug silencioso em minutos, não em dias.

Se o seu produto tem uma camada de IA que ninguém revisou desde que "funcionou na v1", essa é a conversa que vale ter antes que o próximo cliente grande apareça.

Trabalho com arquitetura de cloud, DevOps e produtização de sistemas de IA na CyberTechnology. Se esse padrão descreve onde seu produto está agora, marca 20 min.

DEV Community: Cyber Technology