Your application worked yesterday. You deployed today. Everything is broken. And the most frustrating part? You didn't change your application code. 💀
Imagine this conversation between a developer and a DevOps engineer.
👨💻 Developer: "I didn't change anything. Why is production broken?"
🛠️ DevOps Engineer: "What Docker image did you deploy?"
👨💻 Developer: "my-app:latest."
🛠️ DevOps Engineer: "And what was inside latest yesterday?"
👨💻 Developer: "I don't know."
🛠️ DevOps Engineer: "Congratulations. We've found our first problem." 😭
Sound familiar?
If you've ever worked with Docker, Kubernetes, or CI/CD pipelines, you've probably seen the latest tag.
It looks harmless.
It's convenient.
It's easy to type.
But treating latest as a reliable version identifier can turn a simple deployment into a production nightmare.
Let's understand why—and how to fix it like a real DevOps engineer.
🤔 1. What Does latest Actually Mean?
When you run:
docker pull nginx:latest
Docker pulls the image referenced by the latest tag from the registry.
But here's the catch:
latest does not automatically mean the newest version, the most stable version, or the version you deployed yesterday.
It's simply a tag.
A registry publisher can move that tag to point to a different image.
For example, imagine your team has this image:
my-app:latest
On Monday, it points to build A.
On Tuesday, someone pushes build B using the same tag.
Now the tag points to build B.
MONDAY
my-app:latest
|
v
Build A ✅
TUESDAY
my-app:latest
|
v
Build B 💀
Your deployment configuration hasn't changed, but the image referenced by the tag may have.
That's the problem.
A mutable tag is not a reliable record of exactly which build you intended to deploy.
💣 2. The Real Problem: You Cannot Easily Reproduce Your Deployment
Imagine your production environment is running:
containers:
- name: app
image: my-app:latest
A new image gets pushed to the registry.
Later, you create another Pod.
Depending on the image pull policy, node cache, and registry state, the new Pod may run a different image from an existing Pod.
Now your environment could look like this:
Kubernetes Deployment
|
| image: my-app:latest
|
+------------------+
| |
v v
Pod A Pod B
Build A Build B
This is an illustrative scenario: whether the Pods actually run different builds depends on when and where the image was pulled.
But the underlying risk is real.
If you cannot identify the exact image behind each deployment, troubleshooting and rollback become harder.
Your team starts asking questions:
- Which version is running in production?
- Which commit introduced the bug?
- Can we reproduce the failure locally?
- Which image should we roll back to?
- Are all replicas running the same build?
If your answer to every question is "Let me check," you're missing an important part of release management.
🔥 3. The Fix: Use Versioned Docker Image Tags
Instead of relying on:
my-app:latest
Use a meaningful version:
my-app:1.0.0
my-app:1.1.0
my-app:1.2.0
Or tag images with the Git commit SHA:
my-app:a1b2c3d
my-app:d4e5f6a
my-app:8f9a0bc
Now every deployment can refer to a specific build identifier.
For example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
replicas: 3
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: my-app
image: registry.example.com/my-app:a1b2c3d
ports:
- containerPort: 8080
Here, a1b2c3d represents a particular Git commit identifier in our example.
Your CI pipeline should generate and publish the corresponding image consistently.
Important: Unique tags improve traceability, but tags can still be overwritten unless your registry enforces immutability. For stronger guarantees, deploy images by digest.
🧠 4. Want Stronger Guarantees? Use Image Digests
A Docker image digest identifies image content using a cryptographic hash.
An image reference might look like this:
registry.example.com/my-app@sha256:abcdef123456...
The digest is content-addressed, so it provides a stronger way to identify the exact image than a mutable tag.
A production deployment might use:
containers:
- name: my-app
image: registry.example.com/my-app@sha256:YOUR_IMAGE_DIGEST
Replace the placeholder with the real digest produced by your image registry.
Why is this useful?
Because you can connect your deployment to a specific image artifact rather than relying on a tag that might later move.
For stronger supply-chain security, teams can also verify image signatures and provenance before deployment.
This is especially valuable when working with Kubernetes, automated release pipelines, and regulated environments.
⚙️ 5. Build a Better Docker CI/CD Pipeline
Let's look at a simplified workflow.
Developer pushes code
|
v
GitHub Repo
|
v
CI Pipeline Starts
|
v
Run Tests
|
v
Build Docker Image
|
v
Tag with Git Commit SHA
|
v
Scan the Image
|
v
Push Image to Registry
|
v
Deploy Exact Image
|
v
Verify Application Health
The key idea is simple:
Build once, identify the artifact, and promote that same artifact through your environments.
Avoid rebuilding an image independently for staging and production if you want to guarantee that both environments receive the same artifact.
Here's an example GitHub Actions workflow:
name: Build and Publish Docker Image
on:
push:
branches:
- main
permissions:
contents: read
packages: write
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout source
uses: actions/checkout@v4
- name: Set lowercase image name
id: image
run: |
echo "name=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
- name: Log in to GitHub Container Registry
run: |
echo "${{ secrets.GITHUB_TOKEN }}" |
docker login ghcr.io \
-u "${{ github.actor }}" \
--password-stdin
- name: Build image
run: |
docker build \
-t "${{ steps.image.outputs.name }}:${GITHUB_SHA}" .
- name: Push image
run: |
docker push \
"${{ steps.image.outputs.name }}:${GITHUB_SHA}"
This example builds and publishes an image tagged with the full Git commit SHA.
Before using it in production, pin third-party actions to verified full commit SHAs, configure registry access and package permissions appropriately, and add the tests, security checks, and deployment approvals required by your team.
Also, remember that a successful image build does not automatically mean the application is secure or production-ready.
☸️ 6. What About Kubernetes Deployments?
Once your image is versioned, deploying a specific release becomes much easier.
For example:
kubectl set image deployment/my-app \
my-app=registry.example.com/my-app:a1b2c3d
Check the rollout:
kubectl rollout status deployment/my-app
Inspect the rollout history:
kubectl rollout history deployment/my-app
If the previous Deployment revision is suitable for recovery, you can roll back:
kubectl rollout undo deployment/my-app
Then verify the result:
kubectl get pods
kubectl describe deployment my-app
One important detail: Kubernetes rollback restores a previous Deployment revision. Make sure that revision references the intended image and that the image remains available in the registry.
Also, application and database changes may require additional recovery steps. Rolling back a container image alone does not reverse database migrations or external side effects.
🛡️ 7. Production-Ready Docker Deployment Checklist
Before deploying an application, ask yourself:
- [ ] Does the image have a unique release identifier?
- [ ] Can I trace the image back to a Git commit?
- [ ] Is the image stored in a trusted registry?
- [ ] Are image tags protected against accidental overwrites?
- [ ] Can I deploy by digest where appropriate?
- [ ] Have automated tests passed?
- [ ] Have I scanned the image for known vulnerabilities?
- [ ] Do I have a rollback plan?
- [ ] Can I identify the image actually running in production?
- [ ] Have I verified application health after deployment?
You don't need to implement every advanced security feature on day one.
Start with unique image tags, reliable release records, and a tested rollback process.
Then improve your pipeline as your system grows.
🧪 8. Your Weekend Challenge: Break and Fix a Docker Deployment
Want to understand this concept instead of merely reading about it?
Build a small local experiment.
Step 1: Create a simple application
Use any small web application that returns its version in an HTTP response.
For example:
Hello from build A
Build and tag its Docker image:
docker build -t demo-app:latest .
Run it locally and verify its response.
Step 2: Change the application
Update the response:
Hello from build B
Build a new image using the same latest tag.
Step 3: Compare the results
Inspect the image IDs:
docker image inspect demo-app:latest \
--format '{{.Id}}'
If you kept the original image under a different tag, inspect that image too.
Notice that a tag can refer to different image content at different times.
Step 4: Repeat with unique tags
Build separate images:
demo-app:1.0.0
demo-app:2.0.0
Compare their behavior and inspect their metadata.
Step 5: Write a short postmortem
Document:
- What changed?
- Why was the mutable tag confusing?
- How did unique tags improve traceability?
- How would you deploy the exact same artifact to staging and production?
Put the experiment in your GitHub portfolio.
A small project that demonstrates a real engineering lesson can be more useful in an interview than a repository full of unexplained commands.
🎯 Final Thoughts
Using latest is not automatically a disaster. It can be useful for experiments, tutorials, and certain development workflows.
The problem is treating a mutable tag as if it guarantees a reproducible production release.
Good DevOps practices make deployments traceable, repeatable, and recoverable.
You should be able to answer three questions at any time:
- What exactly are we running?
- Where did this artifact come from?
- How do we safely recover if this release fails?
If you can answer those questions, you're building a much stronger deployment process.
One question for the community 👇
What's your team's strategy for Docker image versioning?
Do you use semantic versions, Git commit SHAs, image digests, or a combination of these?
Share your approach in the comments. I'd love to hear how other engineers handle this in production!
📚 Want to Level Up Your DevOps Skills?
If you're learning these technologies, check out my guides:
🏗️ Terraform Associate (003) Exam Crash Course
Don't just learn commands. Learn how to build systems you can trust.
Top comments (0)