Your CI/CD pipeline is green. That doesn't guarantee any reliability.
Most engineers write their first pipeline to make a status check pass, — not to buy confidence. It works, right up until you actually depend on it. Then the gaps show up: a 12-minute run triggered by a one-line README edit. Two runs racing each other, and the older one wins the deploy. An image that shipped to production without anyone scanning it, or even starting a container from it.
The pipeline is running. But it’s useless.
Here's why this happens on the first attempt almost every time: CI/CD has a huge surface area, triggers, permissions, environments, caching, matrix builds, OIDC, concurrency groups, artifact promotion — a first-timer can lose a week reading docs and marketplace actions before writing a single meaningful line, and still ship something full of dangerous flaws.
You don’t need to know everything to build something complete, you just need to know the basics that make you stand on a solid ground, things that make you actually ship something secure and durable.
A few concepts to carry with you in the pipeline:
1. Treat every action as a real cost, an action that isn't buying you information or testing a legitimate suite of tests is just slower feedback and wasted compute. A docs-only change shouldn't run the same pipeline as a source change, Scope your triggers with path filters to reduce the cost and save time.
2. Editing the workflow file should still fire the pipeline — and it should run the version you just wrote, not a stale one. Synchronize and reopen events on a PR should trigger it too. A pipeline that only runs on "opened" is giving you false confidence on every commit after the first.
3. 2 runs of the same workflow running concurrently at the same branch will bring up a lot of conflicts to the table. Use concurrency controls, with cancel-in-progress scoped to the branch so a stale run can't finish after a newer one and wins the deploy race. Pair that with required status checks. And when GitHub's branch protection genuinely can't express what you need, enforce it inside the workflow itself — CI/CD platforms usually offer a lot of functionality that you can adapt to configure your desired policies.
4. It’s not a real CI if it’s not testing anything. Unit and integration tests on the source code are fundamental. Build the Docker artifact, scan it properly, tag it, and let that same tag drives your docker-compose file — so you're spinning up the image you actually built, not rebuilding it with slightly different inputs. Delegate environment variables through a secrets management mechanism. Then smoke test the artifact, at minimum, before it goes near a real environment.
5. Promote what you tested. Never deploy what you didn't. This one rule eliminates most of "worked in CI, broke in prod" stories. The artifact that reaches production is the exact same artifact that passed all tests and got staged, always: “Build once, promote everywhere”.
None of this requires mastering CI/CD end to end, it only requires understanding the purpose of the pipeline, and then building something that really buys you confidence.
What's the CI/CD mistake that took you longest to resolve?
Top comments (0)