DEV Community: Vivian Chiamaka Okose

I Destroyed My Entire Cloud Environment on Purpose. One Command Brought It Back.

Vivian Chiamaka Okose — Fri, 09 Oct 2026 07:05:47 +0000

I ran one command and deleted my whole AWS environment, the network, the servers, the
database, the storage, a running Kubernetes cluster, all of it. Then I ran one more command
and it came back. Same network, same servers, same security rules, identical.

That round trip is the entire point of this module, and it is the test of whether your
infrastructure genuinely lives in code or just mostly does.

The problem

A company I was modelling this on built their AWS environment by clicking through the console.
It worked, until a new engineer joined and spent two days trying to recreate the dev
environment from screenshots and a colleague's memory. Nobody could reproduce it, nobody could
review a change before it happened, and tearing it down was a hand-written list of steps you
hoped was in the right order.

I had felt every piece of that myself. An earlier module where my teardown instructions failed
because I had written the delete order from memory. Another where a whole cluster lived in a
pile of commands that only existed in my terminal history.

The fix: describe it, do not click it

Terraform lets you describe infrastructure in files, then creates, changes, or destroys it to
match. You do not click "make a VPC." You write "there should be a VPC with these subnets,"
run one command, and it builds all of it in the right order.

If that sounds like Kubernetes, it is exactly the same idea, declare the desired state and let
the tool reconcile reality to match it, pointed one level down at the cloud itself instead of
at containers.

The daily rhythm has one step that makes it genuinely safer than the console:

terraform plan shows you precisely what it would do before it does anything:

Plan: 18 to add, 0 to change, 0 to destroy.

Every risky change I ever made blind in a console, Terraform previews first. That one feature
is most of the argument.

State, and why it goes in S3

Terraform remembers what it built, in a file called the state. It is how it tells "create new"
from "change existing." That state is precious, it holds the record of everything you own, and
it is also where secrets end up: the database password sits in it in plain text.

So the first thing I built, before any infrastructure, was a remote home for it: an S3 bucket,
versioned and encrypted and locked down, with a DynamoDB table holding a lock so two people
cannot apply at once and corrupt it.

The state in S3, not on my laptop. The empty local listing is the point.

The architecture, as modules

I split the infrastructure into four modules, networking, compute, database, storage, each a
folder with its own inputs and outputs. The networking module hands its subnet and
security-group IDs to the others. Terraform reads those references and figures out the build
order itself; I never sequenced anything.

Eighteen resources, an entire production-shaped architecture, from one command.

The gotcha the lab did not mention

I scaled the server count from two to three just to watch the plan, and Terraform stopped me
cold: the third server tried to use a third subnet that did not exist, and plan caught it
before building anything. A real limitation of counting resources by index, and a console
would have let me walk straight into it.

A whole EKS cluster from one block

The part that made me laugh. An earlier module took me a long saga to stand up one EKS cluster
with a command-line tool, through two failures. Here it was one module block from the community
registry, fifty-two resources from about forty lines of my own.

A full managed Kubernetes cluster, nodes ready, from a module someone else already tested.

Catching reality drifting from code

Two things that make Terraform trustworthy in a real, messy account. import adopts a
resource that already exists into management, without recreating it. And drift, someone
changing a resource by hand, gets caught by plan, which shows you the gap between your code
and what is actually there.

A manual change, surfaced. The real fix is stopping manual changes, not just re-applying.

CI that reviews the infrastructure, not just the app

Here is a part I did not expect to enjoy. Every pull request that touches a Terraform file now
runs five checks before anything merges: formatting, syntax validation, a linter, and two
security scanners (tfsec and checkov).

The first time it ran, it failed. Not on anything clever, on formatting. The scanner found my
files were not laid out in Terraform's canonical style and stopped the merge with a plain exit
code. One command fixed it (terraform fmt -recursive), but the lesson stuck: the pipeline
holds a line I would have let slide by hand.

Then the security scanners spoke up, and this is the interesting bit. They flagged ten things.
Not failures, findings, reported but not blocking. Instance metadata still allowing the older,
weaker version. A disk not explicitly set to encrypt. A community module pinned to a version
tag instead of a commit hash. I read each one and made a call: some are real hardening I would
do before this carried traffic, some are not worth it on a server that lives for an hour.

That triage is the actual skill. Running a scanner is easy. Knowing which of its findings
matter for the situation in front of you is the part that takes understanding, and it is the
difference between a report nobody reads and a decision you can defend.

Green pipeline, security findings posted as notes. Both true at once, and that is the healthy state.

The round trip

And then the headline. Destroy everything, apply it back, get an identical environment. If it
survives that, your infrastructure truly is code. If it does not, you have drift, unmanaged
resources, or a dependency your files do not capture, things you have been pretending do not
exist.

Mine came back identical. That is the whole module in one gesture.

One last honest note, on teardown

Cleaning up had its own small trap. The state bucket had versioning switched on, which I had
set on purpose so I could recover old state. When I went to delete it, the normal "force empty
and remove" command refused, the bucket still had every old version and a pile of delete
markers underneath. Versioning keeps everything, including the things you thought you deleted.
I had to clear the versions explicitly before the bucket would go. A good reminder that the
feature protecting you is the same feature standing in your way at cleanup, and that teardown
deserves as much care as the build.

Full build, every module, the remote-state setup, the EKS block, the CI workflow, and the
discipline runbook:
https://github.com/vivianokose/nexaops-operations-lab/tree/main/12-terraform

My Pods Got AWS Permissions From Thin Air. No Keys Anywhere.

Vivian Chiamaka Okose — Wed, 23 Sep 2026 22:25:38 +0000

I opened a shell inside one of my pods and asked AWS who it was.

It answered with an IAM role and a set of temporary credentials. No access keys anywhere. Not in an environment variable, not in a Kubernetes Secret, not in git.

The pod had real AWS permissions, and there was nothing I could point to and call a password.

An assumed role ARN, not a node role, not a static key. The pod proving its own identity.

That is IRSA, and it's the idea this whole build is anchored on.

The setup

ClearOps, my three-service app (frontend, API, Redis), was running on a local cluster while I proved out the Kubernetes migration.

This build takes it to production: Amazon EKS, AWS's managed Kubernetes.

The surprising part is how little changed. My Deployments, Services, ConfigMap, and autoscaler moved over almost untouched. EKS just means AWS runs the Kubernetes control plane, the API server, etcd, the scheduler, so I don't have to.

I keep the worker nodes and my manifests. Same Kubernetes, different management underneath. Same logic as using a managed database instead of running your own.

Images move to a real registry

The local cluster built images straight into its own daemon.

EKS nodes are EC2 instances that can be replaced at any time, so they pull images from a registry instead. ECR is AWS's private registry for this, the same idea as Docker Hub or Nexus, with vulnerability scanning on push turned on.

Every finding sits in the Debian base image, not my own code. The same base-image CVE pattern I'd already hardened against earlier in this build.

The one genuinely new idea

Before this, pods never needed AWS permissions, because there was no AWS in the picture.

On EKS, that becomes the central question. How does a pod get permission to do AWS things without a key baked into it somewhere?

The wrong answer is keys sitting in a Secret, a mistake I'd already caught and fixed once before. The too broad answer is a role attached to the node, because then every pod on that node inherits it, whether it needs it or not.

IRSA, IAM Roles for Service Accounts, is the actual answer. A pod proves its Kubernetes identity through the cluster's OIDC provider, and gets back temporary, auto rotating credentials scoped to just that pod. No keys stored anywhere.

An injected role ARN and a web identity token. That's the pod's entire AWS identity, and there's no key in sight.

This same pattern does the real work three separate times in this build: the load balancer controller, the app's own S3 access, and External Secrets. Each one is a pod that needs AWS permission, and each gets it the same keyless way.

Real load balancing from three lines

I'd built an Application Load Balancer by hand before. Created it, made a target group, set health checks, wired up security groups. An afternoon of console work.

The AWS Load Balancer Controller turns all of that into three lines of annotation on an Ingress. It's a pod that watches for Ingress objects and creates real ALBs to match, using its own IRSA identity for permission.

The app reachable through a real ALB, provisioned straight from my Ingress. An afternoon of manual console work, reduced to a manifest.

Secrets that never touch git

Kubernetes Secrets are only base64 encoded, not encrypted.

External Secrets Operator keeps the real value in AWS Secrets Manager and syncs it into the cluster on demand. What lives in git is only a pointer to the secret, never the value itself.

The value pulled live from Secrets Manager into a Kubernetes Secret. Nothing sensitive sitting in any manifest.

Pods with no nodes

A Fargate profile runs pods with no node of mine involved at all, billed per pod per minute. Good for short or spiky work that would otherwise mean paying for a node that mostly sits idle.

I ran a batch Job and watched it land on infrastructure that didn't exist a moment earlier.

A Job running on a Fargate node, not on my regular node group.

The part that actually needs attention: cost

EKS bills by the hour. The control plane has no free tier and no pause button, only delete.

My first cluster build failed outright, because the account was restricted to free tier instance types, and the size I picked wasn't one of them. The CloudFormation error said so plainly. Switching to an eligible size fixed it in one line.

So the whole build runs spin up and tear down in a single session. Budget alarm first, before anything else exists.

Teardown happens in strict order: the app and its Ingress first, so the controller removes the ALB on its own, confirmed gone, then the cluster, then the leftovers. Get that order wrong and the ALB orphans itself, billing on with nothing left to remove it.

A full session cost me under a dollar. A cluster left running by accident is where the scary numbers live.

The thread running through all of it

It's Kubernetes I already knew, wrapped in AWS plumbing, with IRSA as the identity glue and cost discipline as the constant.

Same manifests. Managed underneath. Keyless pod permissions. A meter you never take your eye off.

Full build, the eksctl config, the manifests, and the teardown runbook: github.com/vivianokose/nexaops-operations-lab/tree/main/11-eks

I Deleted the Same Pod Twice. Only One of Them Remembered.

Vivian Chiamaka Okose — Fri, 18 Sep 2026 15:17:11 +0000

I wrote a value to Redis, deleted its pod, and it was gone. Did the same to a different Redis, deleted the pod, and it was still there. Same command, opposite result, and the difference explains what Kubernetes is actually doing.

The shift: describe, don't instruct

Every tool before this took commands that run once. Kubernetes takes a description of what should be true and keeps reality matching it. You say "I want two API pods," and the control plane makes and keeps that true.

The catch: you cannot fix things by hand. Delete a pod, it comes back, because the description still says two. To change what runs, change the description.

Self-healing, proven

Delete a pod and a replacement is scheduled before the old one finishes terminating.
~12 seconds to serving traffic. Nothing responded to the delete specifically, the control loop just saw 1 where it wanted 2.

The pieces

Pods wrap containers and are disposable. Deployments hold the replica count and manage pods.
Services give a stable address in front of changing pods, reached by DNS name (redis, not an IP). Labels and selectors are the glue, a Service fronts "anything labelled app: redis", and a label/selector mismatch is the classic beginner bug.

Autoscaling

CPU hit 92% under load, pods went 2 to 4 in seconds. Needs metrics-server, and needs pods to declare CPU requests or there's no baseline to measure.

The two Redis pods

The API is stateless, interchangeable, perfect for a Deployment. Redis holds data. A plain Deployment loses it on restart. A StatefulSet gives a stable identity (redis-store-0) and a PersistentVolumeClaim that outlives the pod, so the value survives.

Caveat: real replicated databases in Kubernetes are hard (leader election, failover). Most teams use a managed database.

Config, updates, isolation

ConfigMap injects settings as env vars. Rolling updates replace pods gradually, gated by the readiness probe, so a broken version stops the rollout instead of taking the service down.

Rollback adds a revision rather than deleting history, and history is finite. The manifest in git is the real source of truth.

By default every pod can reach every other pod. A NetworkPolicy fixes it:

Unrelated pod: timed out. API with the allowed label: connects. Same chained isolation as AWS security groups, by labels not IPs. The moment a policy selects a pod it goes deny-by-default, so forgetting to allow needed traffic breaks your own app.

Helm vs Kustomize

Helm: templated manifests with variables, versioned releases, one-command rollback, but the files aren't valid YAML alone. Kustomize: plain YAML with per-environment patches, no release history, built into kubectl. Helm for third-party software, Kustomize for your own apps. Many teams use both.

Debugging

STATUS gives the category, describe Events give the cause, logs only help once a container starts.

A deliberate bad-image break. Events name it in one line. Reading Events first is the habit beginners skip.

The one idea

Declare the desired state, a control loop keeps reality matching it. Self-healing, autoscaling, rolling updates, rollback, reboot recovery, all the same loop closing a gap.

Full build: https://github.com/vivianokose/nexaops-operations-lab/tree/main/10-kubernetes

My Server Had Zero Inbound Ports. I Still Got a Shell.

Vivian Chiamaka Okose — Wed, 02 Sep 2026 16:44:46 +0000

A shell prompt on a server with no SSH key, no port 22, and no inbound firewall rules at all.
From it, a query against a database my own laptop cannot reach, not because of a password,
but because no route to it exists.

That is the end state. Here is the build.

The problem

A SaaS company, 500 paying customers, everything on one server: web app, MySQL, and file
uploads on one box with one public IP. It went down for eleven minutes when the server
restarted. Worse, the logs showed thousands of automated login attempts nightly, and the
database sat at the same address as the app.

The fix is not a better password. It is making the database unreachable.

What actually makes a subnet public

A VPC with four subnets across two availability zones. Public for things that face customers,
private for things that must not.

The precise version, and it is a common interview question: a subnet is public because its
route table sends 0.0.0.0/0 to an internet gateway. Private subnets have no such route.
Not a checkbox, just the presence or absence of a line.

Public points at the IGW. Private points at a NAT gateway, outbound only.

Security groups referencing security groups

Three groups, chained: the ALB accepts HTTP from anywhere, app servers accept 3000 only from
the ALB's security group, the database accepts 3306 only from the app servers' group.

The rules name groups, not IPs. Replace a server tomorrow with a different address and
the rule still works, because it grants access to anything wearing that badge.

Hitting an app server directly on 3000 from a browser timed out. Not refused. Timeout
means packets silently dropped, usually a firewall or missing route. Refused means you
reached the machine but nothing was listening. Useful distinction when debugging.

Redundancy plus health checks

Two app servers in two availability zones, physically separate data centres. An ALB in front
spreads traffic and, more importantly, requests /health every thirty seconds. Three
consecutive failures and that server stops receiving traffic. The failure still happens; it
stops being an outage.

Same URL, different servers, neither reachable directly.

IAM roles, not access keys

Access keys in a config file end up on disk, in the repo, on every laptop that cloned it, and
in backups. They do not expire. Bots scan public repos for them constantly and find them
within minutes.

A role attached to the instance gets temporary credentials, rotated automatically. Nothing to
leak, nothing to rotate, and it cannot be lifted off the machine the way a stolen key can.

The least-privilege policy needs two statements, which is where most first attempts fail:

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::my-bucket" },
    { "Effect": "Allow", "Action": ["s3:GetObject","s3:PutObject","s3:DeleteObject"],
      "Resource": "arn:aws:s3:::my-bucket/*" }
  ]
}

Bucket actions take the bucket ARN. Object actions take /*. Grant one without the other
and you get confusing half-failures.

After swapping the broad managed policy for that one, uploads still worked and aws s3 ls
returned AccessDenied. The denial is the win. A compromised server now reaches one bucket
and nothing else.

RDS: no shell, and that is the point

AWS handles backups, patching, storage, hardware. There is no SSH into an RDS instance,
because the shell existed to do maintenance that is no longer yours. You get a stable
endpoint that survives hardware replacement and failover.

Private subnets, public access off, app servers only.

Session Manager instead of SSH

No key pair, no SSH rule. An agent on the instance holds an outbound connection to AWS, and
sessions come back through it.

No port 22 to scan. No key to store, copy, lose, or rotate. Authentication through your IAM
identity, so removing someone's AWS access locks them out of every server at once. Every
session logged in CloudTrail under a real name. And it reaches private subnets, so no bastion
host.

Trap worth knowing: Amazon Linux ships the MariaDB client, which uses
--ssl-verify-server-cert, not MySQL's --ssl-mode=VERIFY_IDENTITY.

Cost, and the runbook that failed

Cloud bills for existing, not for using. An idle NAT gateway is the same price as a busy one,
roughly a dollar a day.

Budget alarm before any resource existed. Tags on creation. Teardown runbook written up front,
then tested, and testing it found a real gap: deleting the VPC failed because the NAT gateway
and its network interface had to go first, individually. That is exactly how orphaned
resources end up billing for months.

On method

I zoned out on the videos for this module, repeatedly. So I changed approach: every concept
as a specific problem first, named plainly, then built. Ten concepts before a single
resource. By the time I was in the console I was confirming something I understood rather
than following steps I did not.

If videos are not landing for you, that is worth knowing about yourself early.

Full build, screenshots, diagram, retro, and the tested teardown runbook:
https://github.com/vivianokose/nexaops-operations-lab/tree/main/09-cloud-fundamentals

I Was Scared of Jenkins. Then I Built a Full CI/CD Pipeline From One git push

Vivian Chiamaka Okose — Thu, 27 Aug 2026 14:43:02 +0000

I watched an entire Jenkins video series and understood almost none of it. The moment
Groovy appeared, I zoned out, convinced I needed to learn a programming language to do this
job. I was wrong, and figuring out why unlocked everything.

By the end I had a pipeline that, from one git push, computes the version, lints, tests,
builds a Docker image, pushes it to a registry, deploys it, verifies it, and tags the
release. Automatically.

The fear-buster

A Jenkinsfile is a fill-in-the-blanks form, not a language. The structure
pipeline { stages { stage { steps { } } } } is fixed. You fill in stage names and shell
commands you already know. That is it.

Two servers

A Jenkins server (the factory, runs the pipeline) and an app server (the shop, runs the
app). Jenkins builds and ships; it never runs the app.

Jenkins on its own server.

Secrets in a vault

Every secret (SSH key, server IP, Docker Hub token, GitHub token) lives in Jenkins'
credential store with a label. The Jenkinsfile references the label, never the secret.

Secrets referenced by label.

Eight stages

Checkout, Compute Version, Lint, Test, Build and Push, Deploy, Verify, Tag Release. Runs
top to bottom, stops on any failure so broken code never deploys.

A full run, every stage green.

The webhook makes it automatic

A doorbell: GitHub pings Jenkins on every push, so the pipeline runs on its own. I opened
Jenkins only to GitHub's webhook IP ranges, not the whole internet.

A build that started itself after a push.

Multibranch and PR checks

Every branch and pull request builds automatically, and the PR shows a pass/fail check on
GitHub. Deploy runs only on main, so feature branches are tested but never shipped.

The check stamped on the pull request.

Shared library

The build-and-push logic lives in a separate repo. The Jenkinsfile calls it in one line.
Fix once, every project gets the fix.

Calling the shared function.

Dynamic versioning

A feat: commit bumps the minor version, a fix: bumps the patch. The pipeline reads
the commit and tags the release. No human picks a version.

feat produced v0.1.0, fix produced v0.1.1.

The bugs that taught me most

Cloud account suspended mid-build (rebuilt on a new provider, faster the second time).
Deleted my SSH keys with a careless command. Jenkins refusing to start on the wrong Java
version. A stage hanging because my lint command started a never-ending server. A first run
failing on a missing tool. Every one was a real lesson.

Takeaway

Jenkins is a form, not a language. The full code and screenshots:
https://github.com/vivianokose/nexaops-operations-lab/tree/main/08-jenkins

I Shrank My Docker Image From 1.11GB to 133MB, and That Was the Easy Part

Vivian Chiamaka Okose — Thu, 13 Aug 2026 18:45:39 +0000

My first Docker image was 1.11GB. The app inside it was a tiny delivery tracker that could
have fit in a fraction of that. By the end of this module the image was 133MB, signed,
scanned, non-root, and sitting in two registries. Here is the whole journey, including the
parts that fought me.

I gave myself four questions, borrowed from how real teams judge a production image. A
naive image fails all four. A good one passes all four.

How small is it?
What is in it?
Who built it?
Will it run as root?

The app

SwiftMove Logistics: a Node.js server reading deliveries from MySQL. Small enough to
understand, real enough to need a database, a private network, volumes, and health checks.

Live data from the MySQL container beside it.

Question 1: How small is it?

I built the worst reasonable Dockerfile first, on purpose, to see the cost.

1.11GB. A whole build toolchain that the running app never touches.

The fix is a multi-stage build. Ship the sandwich, not the whole kitchen: do the messy
building in a throwaway stage, copy only the finished app into a tiny clean stage.

1.11GB to 133MB.

99 percent efficiency confirmed with dive.

Question 4: Will it run as root?

No. A non-root user, created in the Dockerfile and switched to, so the app has almost no
power inside its own container.

Running as UID 10001.

A clean Dockerfile lint.

One command for the whole stack

App plus database, one command.

MySQL has no port exposed to the host. Secure by not opening a door.

The first run showed "table doesn't exist", the app started before MySQL was ready. The
fix was a retry loop so the app waits patiently for the database. That bug taught me more
than a clean run would have.

Data survived a full container destroy and recreate, because it lives in a volume.

Question 2: What is in it?

The first scan. Dozens of findings from the base image.

Most shared one root cause: outdated OpenSSL. I bumped the base image and rescanned.

OS findings dropped from 50 to 2. My own dependencies scan clean.

The goal is not zero. It is: know, fix what you can, document the rest in a SECURITY.md.

Question 3: Who built it?

Signed and verified with cosign. A wax seal proving origin and integrity.

Two registries

Public on Docker Hub.

The Nexus push threw a 403 first: my least-privilege deploy user could not create a new
repo, exactly as designed. Least privilege, felt from the other side.

The same image in my private Nexus registry.

The takeaway

The naive image took minutes. The production one took hours. That gap is the whole module,
and the bugs along the way were the best teachers.

Full code and screenshots: https://github.com/vivianokose/nexaops-operations-lab/tree/main/07-docker

I Built a Private Software Registry, and Four Different Tools Now Publish to It

Vivian Chiamaka Okose — Wed, 05 Aug 2026 08:15:17 +0000

I will be honest about where this one started. I watched the videos for this module, and
at the end I was completely lost. Not "need to review" lost. "The words make sense one at
a time but the whole picture will not form" lost.

So I threw the videos out and rebuilt it from a single idea. Once that idea landed,
everything else fell into place. Here is the whole thing, the way it finally made sense to
me.

The one idea: a registry is a pantry for software

When developers build software, their code needs ingredients: libraries other people
wrote. Normally every build runs out to the public internet to fetch those ingredients
fresh, every single time. That is slow, and if the public site is down or rate-limits you,
your build breaks.

No real kitchen works that way. Kitchens have a pantry. Ingredients get stocked once and
kept on a shelf, and cooks grab what they need from there.

An artifact registry is a pantry for software. It stores copies of the ingredients your
builds use, and it stores your own finished artifacts too. Nexus is one popular brand of
this pantry. That is the whole concept. Everything else is detail.

Three kinds of shelf

A registry stores things three ways:

Hosted: your own artifacts. The jam you made yourself.
Proxy: cached copies of public libraries. Shop flour kept on your shelf, so you do not run to the shop every time, and you still have some when the shop is closed.
Group: one address that serves from both, so a build only needs one URL.

The shelves in my registry. You can see the types right there: hosted, proxy, group.

Running it as a limited user

Before publishing anything, the most important security decision. Nexus runs on a Linux
machine, and every program runs as some user. That user decides what the program can touch.

You never run something like this as root, the all-powerful user. You make a limited user,
literally called nexus, that can only touch its own files. If the application is ever
broken into, the attacker is trapped in that small world instead of owning the whole
machine. It is the difference between giving a cleaner the key to one room versus the
master key to the building.

Proof it runs as nexus, not root. That first column is the entire security lesson.

The part where it fought me

Real infrastructure work is never the happy path. The lab specified an old version of
Nexus that wanted an ancient Java, but I had a modern one. Then the download links kept
returning 404, because the vendor had rotated the versions and even changed the filename
format since the lab was written.

I learned to stop trusting a hardcoded version number and go to the official download page
for the current link. The newer version shipped with its own Java bundled inside, which
made the whole conflict disappear. Frustrating in the moment, but this is the actual job:
read the error, adjust, keep going.

Publishing four kinds of artifact

Here is the part that surprised me most. I published four completely different artifact
types, and they all work the same way underneath.

Java, with Maven:

Pushing my Java JAR to the registry.

Node, with npm. Python, with a tool called twine, and for that one I built a Python
package from scratch, something I had never done: a wheel file, described by a
pyproject.toml, built and uploaded.

My first Python package, sealed into a wheel.

And Docker, an image pushed to the registry:

A Docker image, the fourth and final type.

Once I had done the first one, the other three were variations on a pattern I already
knew: point the tool at Nexus, give it a credential, run a publish command. Four different
ecosystems, one shape. That was the moment it stopped being four things to memorise and
became one idea.

Least privilege, and the errors that taught it

I published everything as admin at first, which is fine for setup and wrong for a real
build pipeline. A pipeline's credentials leak constantly; you never hand it the keys to
everything.

So I made a deploy user that can only push, no admin, no delete. And setting it up, I kept
hitting "403 Forbidden" errors. One for the jar, another for the metadata. Each one taught
me that access control is iterative: grant the minimum, test, find you are one privilege
short, grant exactly that, and no more. The errors were the lesson.

Logged in as the deploy user: the admin controls are simply gone. It can do its job, and
only its job.

Automating the inventory

Clicking through a UI is fine for a human. Real operations automate. So I wrote a small
bash script that asks the registry's API what it holds and prints an inventory.

One command, and the whole registry reports itself: every artifact, across all four
types.

That is the leap from using a tool to automating infrastructure, and it is the difference
between clicking buttons and doing the job.

What I actually learned

Before this, if you had asked me how teams share build artifacts, I would have said they upload them to a repository like Nexus or JFrog for distribution and versioning, and for
control over what stays in the repository. That was right. But building it taught me the part I had not appreciated: how much of a registry is about control and safety, not just storage. The limited user. Anonymous access off. A deploy user that can push but not administer. A record of who uploaded what.

A registry is not a shared folder. It is a controlled, auditable gate that everything passes through. And a topic that completely lost me at the start became something I can now build with my own hands and explain to someone else. That is the part I am proud of.

I Locked Myself Out of My Own Cloud Server. Here Is How I Got Back In.

Vivian Chiamaka Okose — Wed, 29 Jul 2026 14:35:00 +0000

I did something this week that sounds like a mistake and was actually the most useful thing I have learned in months.

I locked myself out of my own server. On purpose. A server that was running fine, serving a live website, with no way back in. No SSH, no password, and no keyboard to plug in, because it lives in an Amazon data centre I will never enter.

Then I recovered it. And now I understand something most engineers only read about.

Logged back into the server that had locked me out.

Let me walk through the whole thing, because the recovery only makes sense once you see how the server was built.

Renting a computer by the hour

The cloud is someone else's computer that you rent by the hour. You get a slice of a machine in a warehouse, use it, and hand it back when you are done. Amazon calls this service EC2, and one rented machine is an "instance."

The mental shift that matters: you pay for the machine existing, not for using it. A server you forgot about costs the same as one serving thousands of people. That single fact shapes every decision in this module, and it is why cost discipline shows up at the end.

Locking the front door before building anything

The very first step had nothing to do with servers. It was locking down the account itself.

When you sign up for AWS, you get a "root" account that can do anything, including run up unlimited charges if the credentials leak. There is a whole economy of bots scanning the internet for leaked AWS keys, and they spin up expensive servers within minutes of finding one.

So the first move is to lock root away behind multi-factor authentication and create a weaker, everyday identity to actually work with. That everyday identity is an IAM user. Root is the master key you lock in a drawer; the IAM user is the normal key you carry.

Root account secured with multi-factor authentication.

An everyday IAM user, in an Administrators group, so I never work as root.

The server that would not let me in

With the account secured, I launched my first instance. And immediately hit a wall.

SSH timed out. Not "permission denied," which would mean I reached the server but was not allowed in. A timeout, which means the knock got no answer at all.

My security group, AWS's firewall, was correct: SSH open to my IP, web traffic open to everyone. So why the silence?

I worked through it one layer at a time. Was my key wrong? A wrong key gets rejected, not ignored. Had my home IP changed? It matched. Was the subnet missing its route to the internet? The gateway was there.

Then I found it. A network ACL, a second firewall sitting above the security group, set to deny all traffic. Someone had configured this account's network in a non-standard way long before I arrived. My rules were perfect; the layer above them blocked everything.

I made a call that felt like the professional one: stop patching someone else's confusing setup, and build my own clean network from scratch. A fresh VPC, one public subnet, a proper gateway, all wired correctly. I launched into that, and SSH worked instantly.

The instance running in my own clean network, finally reachable.

The lesson stuck harder than any happy-path tutorial could have taught it. A timeout is not one problem, it is a checklist: security group, your IP, the route, the network ACL. And when you inherit a mess, rebuilding clean often beats untangling.

Hardening, against real threats this time

I had hardened Linux servers before, but always on my own laptop where the threats were hypothetical. This one had a public address, which means bots start probing it within minutes of it going live.

Same baseline, real stakes: patch everything first, put up a firewall that denies everything by default and opens only SSH and the web ports, and run fail2ban to jail anything that hammers the login repeatedly.

The firewall: everything denied by default, only SSH and the web ports open.

fail2ban standing guard, ready to jail repeat login offenders.

Giving it a name and a padlock

A raw IP address is no way to reach a website. So I pointed a subdomain of my own domain, app.viviancloud.site, at the server, while carefully leaving my main domain on its existing site. One domain, different subdomains for different things, which is exactly how real companies organise this.

Then HTTPS. For years, certificates cost money and were fiddly. A nonprofit called Let's Encrypt made them free and automatic. One tool, one command, and my site loaded with a padlock and a certificate that renews itself. The plain-HTTP postcard became a sealed envelope.

My own page, served from the EC2 instance.

Reachable by name at app.viviancloud.site.

The padlock: HTTPS live, certificate valid and auto-renewing.

Data on its own disk

I attached a second disk, separate from the one the operating system lives on. The reason is practical: keep your data apart from your OS, and you can rebuild the OS whenever you like without touching the data. Format it, mount it, and add it to the system's startup config so the mount survives a reboot. I proved it by actually rebooting and watching the disk come back on its own.

The separate data disk, still mounted after a full reboot.

The lockout, and the way back

Now the part I opened with.

To learn recovery for real, I created the disaster. I emptied the file that lists which keys are allowed to log in, disconnected, and confirmed I was locked out. Permission denied, on my own running server.

But before breaking anything, I took a snapshot: a full backup of the disk, my undo button.

A snapshot first. Never break something risky without a backup.

Here is the recovery, and the single idea that makes it possible: the file that decides who can log in lives on the disk, and a disk is a movable object.

So I stopped the server, detached its disk, and attached that disk to a second "rescue" instance as a spare drive. From the rescue instance, I could reach into the locked server's files, and I wrote my key back onto its login list, with the exact ownership and permissions SSH insists on. Then I detached the disk, reattached it to the original server as its boot drive, and started it up.

I logged in. The same key that got "permission denied" ten minutes earlier now worked.

It is exactly like being locked out of your house, unbolting the front door, carrying it to a locksmith who fits a new lock you have a key for, and rehanging it. The house never changed. You fixed the lock by taking the door somewhere you could work on it.

The deeper realisation: a server accepts any key whose public half is written in its login file. So a lost key is never a dead end. You do not recover the old key, you generate a new one and install it. The lock is always replaceable.

There are faster recovery routes when you set them up in advance, but the disk-swap always works, with no agent and no preparation. It is the universal fallback, which is why it is the one worth learning first.

Building carefully, tearing down completely

The module ended where cloud work always should: accounting for the cost and knowing how to remove everything.

Total spend for all of this: zero dollars, by staying on free-tier resources and stopping the instance between sessions. I set a billing alarm as a tripwire, and wrote a teardown runbook listing every resource and how to remove it in order, because a resource you forget is a resource you keep paying for.

A billing alarm: emails me if estimated charges ever cross five dollars.

What I actually learned

Not just the steps. The principles underneath them.

That cloud bills on existence, not use. That you default everything to closed and open only what each service needs. That a lost key is a lock to replace, not a catastrophe. And that knowing how to cleanly destroy infrastructure matters as much as knowing how to build it.

I locked myself out of a server and got back in by hand. I will not forget how that works.

Your Build Output Is Not an Artifact Yet. Here Are the Four Things It Needs.

Vivian Chiamaka Okose — Thu, 23 Jul 2026 12:24:56 +0000

In December 2021, a flaw was found in a Java logging library called Log4j. Not a famous library. The boring thing that writes messages to a log file.

Then every company on earth asked the same question at once: are we using it?

Most couldn't answer. Not out of carelessness, but because nobody keeps a list of what is actually inside their software. Their app pulled in a library, which pulled in another, which quietly pulled in Log4j four levels down. Teams spent entire weekends grepping through servers trying to work out whether they were exposed.

The companies that answered in ten minutes had one thing the others didn't. A list.

That list is called an SBOM, and it's one of four things I learned to attach to my own artifacts this week.

First, what a build actually does

Source code is text. UserService.java is a file full of words like public class and return. A processor cannot run that. It speaks something much lower level.

Building is what closes that gap. It's the cooking, and source code is the recipe.

Nobody serves a recipe card to a customer. They serve the dish. Nobody deploys source code to a production server either; they deploy the built artifact.

I built two of them. A Java service with Maven, which produced a 4.8K JAR file. A Node service with npm, which produced a 1.7K .tgz. Different languages, same idea: source in, shippable thing out.

Every build tool is the same tool wearing different clothes

Java uses Maven. Node uses npm. Python uses pip. Three tools, three sets of commands, and it looks like three separate things to learn.

It isn't. They all answer the same five questions:

A config file. What is this project and what does it need? (pom.xml, package.json, requirements.txt)
A lock file. Which exact versions did we actually install? (package-lock.json, pinned versions)
A registry. Where do the libraries come from? (Maven Central, npmjs.com, PyPI)
A cache. Where do downloaded libraries live locally? (~/.m2, node_modules/)
An artifact. What comes out at the end? (.jar, .tgz, .whl)

Config, lock, registry, cache, artifact. Once you see those five boxes, learning the third tool takes minutes, because it's just the first tool with different filenames.

One genuine difference worth knowing. Java compiles everything upfront, translating the whole book before anyone reads it. Node and Python translate live, line by line, as the program runs. That's why my first Maven build took 64 seconds and npm install took 4.

Four questions that turn a build output into an artifact

Here's the part most tutorials skip. A file sitting in a target folder isn't ready to ship. A real artifact answers four questions.

What's in it?

An SBOM, a Software Bill of Materials, is a complete itemised list of every component inside your artifact. It's the ingredients label on food packaging.

I generated mine with a tool called Syft. My Node service listed two entries: my own code, and one library called dotenv.

Small, but the version was interesting. My config file asked for ^16.4.5, meaning "16.4.5 or anything compatible." What actually got installed was 16.6.1. Fuzzy request, specific reality. That gap is precisely why lock files exist.

Real applications run this and get three hundred entries. Most of which nobody on the team chose.

Is it safe?

Knowing what's inside is step one. Step two is checking whether any of it is known to be broken. Security researchers maintain public databases of flaws found in software libraries, and a tool called Grype cross-checks your ingredients list against them.

It's a food safety inspection. You have the ingredients; now someone checks them against the recall list.

Both my artifacts came back clean, which was true and slightly boring. So I deliberately installed an old version of a popular library called lodash to see what a real finding looks like.

Six findings from one library. Three rated High. Read one row as a sentence: you have lodash 4.17.15, it has a high severity flaw, version 4.17.21 fixes it.

Two columns earn their place. EPSS estimates how likely the flaw is to be exploited in the wild, and the top result sat at 22.4%, the 97th percentile. That's not theoretical; attackers are actively using it. RISK blends severity with that likelihood so you know what to fix on Monday morning.

And every single one had a fix already available. Most vulnerabilities are not unsolvable mysteries. They're someone forgetting to update a library for two years.

Is it the one I built?

In 2016, attackers broke into the servers of Linux Mint and swapped the download file for their own version with a backdoor in it. Same filename, same page, same site. Anyone downloading that day got malware.

Underneath the download link sat a line of gibberish characters. Anyone who ran one command against their file and compared it would have known instantly.

That line is a checksum: a fingerprint calculated from a file's exact contents. It's the tamper-evident seal on a medicine bottle.

I made one for my JAR, verified it passed, then appended a single character to the file and checked again.

One character. FAILED, immediately. There is no such thing as small tampering; any change at all breaks the seal, which is exactly what makes it worth publishing.

Who made it?

A checksum proves the file didn't change. It doesn't prove who made it. An attacker who controls the download page can replace both the file and the checksum, and everything matches perfectly. It's just their seal now.

Signing closes that gap. A signed release carries cryptographic proof it came from a specific person, and it cannot be forged without their private key.

I signed my release tag with the same SSH key I set up months ago for signed commits, and got back: Good "git" signature for vivian-signing-2026. On GitHub it shows a green Verified badge.

A checksum is a seal anyone can apply. A signature is your handwriting, notarised.

Then I stopped relying on my memory

Producing all of that took eight separate commands in the right order with the right flags. Which is fine on a Tuesday morning and useless at 5pm on a Friday when you need to ship a fix and the scan feels optional.

So I wrapped the whole path in one script. Build, SBOM, scan, checksum, in that order, stopping dead if anything fails.

![The full build and verify run]

One command, four steps, three files at the end: the artifact, its ingredients list, and its seal. You can't skip a step because there are no steps to skip.

That's also the bridge to what comes next. A CI pipeline is essentially this script running on a server, triggered by a push. Write it now, hand it to a robot later.

Three things that caught me

A command finishing without error is not proof it did the right thing. My Node tests ran green while testing nothing at all, because I'd pointed the test runner at the wrong path. Five tests written, one thing ran. I only caught it because the count said 1 and I expected 5.

Node's package doesn't contain its dependencies. Scanning the .tgz found zero packages, which looked broken. It wasn't. npm deliberately excludes node_modules, so the box holds the recipe and the ingredients get fetched on arrival. Java bakes them in; Node doesn't.

Build output does not belong in git. My first commit attempt swept in compiled classes, test reports, and the JAR itself. All of it reproducible from source with one command. Commit the recipe, not the meal.

The question I can answer now

If someone hands me a JAR with no SBOM, what should I assume about it?

That nobody knows what's inside it. Including the person who built it.

It might be perfectly fine. But there's no way to answer the Log4j question about it, no way to know whether it's carrying something with a public exploit, and no way to prove it's the file they think it is.

That's not ready to ship. It's just bytes.

I Deleted My Database on Purpose. It Was Back in 47 Seconds.

Vivian Chiamaka Okose — Fri, 17 Jul 2026 06:11:01 +0000

There's a sentence I can't stop thinking about:

A backup you have never restored is a wish, not a backup.

Most engineers set up backups and never test them. The backup job runs every night, everyone feels safe, and then one day a real disaster hits, they reach for the backup, and discover it's been silently broken for months. Now the data is genuinely gone, at the worst possible moment.

So this week I stopped wishing. I destroyed my own database on purpose and proved I could bring it back.

First, I learned before I built

I did something different this module: I asked to understand the concepts before touching a single command. What a database actually is. Why SQL and NoSQL both exist. What containers really are. What migrations do.

It changed everything. Instead of copying steps and hoping, I knew why each command mattered. If you're learning DevOps and you find yourself typing commands you don't understand, stop and go learn the idea underneath. The labs land completely differently after that.

The stack, from one command

I built a four-container stack that starts with a single command: Postgres (the strict, organized store for important data), Redis (the fast, flexible one), Adminer (a web view of the data), and pgBouncer (connection pooling).

The idea that made containers click for me: a container is a sealed lunchbox holding software plus everything it needs to run. It works identically on my laptop, your laptop, or a server in a data center. No more "but it works on my machine."

And one crucial piece: a volume. Containers are disposable, you throw them away and make new ones. A volume is the storage box that survives that. Without one, your data dies with the container. That distinction matters more than anything else in this setup.

Version control, but for your database's shape

Your database's structure changes constantly as an app grows. Add a column, add a table, add an index. The tempting way is to just log in and type the change. The problem: nobody has a record of what changed or when, and now your database, your teammate's, and production are all subtly different.

Migrations fix that. They're numbered, recorded files, V1, V2, V3, applied in order, with the tool keeping a history of exactly what ran and when.

They are, quite literally, git for your database structure. Which is why the concept felt instantly familiar after last month's Git module.

One detail worth stealing: I added a column in V2, and backfilled its values in V3, deliberately two separate migrations, not one. Because if an app is live while you change the database, adding and filling in one step means a moment where the app and database disagree. Additive first, destructive later. That staging is how you change a database without breaking the thing using it.

The refusal that impressed me most

Here's my favourite moment. In the real world, you almost never get to start a database from scratch. You inherit one that's been running for years, full of customer data you cannot wipe. So I simulated that: an existing database, real rows, no migration history.

I pointed the migration tool at it, and it refused:

Found non-empty schema but no schema history table.

It was saying: there's data here I didn't create, I don't know what version this is, and I won't guess and risk breaking it. That refusal is a feature. The tool protecting two years of data from a stranger's assumptions.

The fix is called baselining: you tell it "whatever exists now, call that version 1, count my changes from version 2." Like joining a company mid-project and saying "I'm not recreating what happened before I arrived. Today is my starting line."

That's the version of this skill you'll actually use at work.

Then I destroyed everything

The main event. Here's the drill:

Count the rows: 3 authors, 3 posts
Take a backup (8.2K file)
Drop the entire database
Restore it
Verify the counts match
Time it

Step 3 feels wrong to type. You're deleting the thing. And then:

FATAL: database "lumberyard_dev" does not exist

That's the message on someone's screen at 3am when a room goes quiet. My data was genuinely gone.

Then I restored it from that 8.2K file.

3 authors, 3 posts. Exact match. Nothing lost. 47 seconds.

That number matters more than it looks. In a real incident, everyone asks "how long until we're back?" Most people guess. I can answer, because I measured it.

What I'd tell past me

Three things I actually learned, beyond the commands:

Check the file size. A zero-byte backup file is a silent catastrophe that only surfaces when you desperately need it. Always look.

"No errors" is not "it worked." Verify the row counts. A restore that finishes quietly but incompletely is worse than one that fails loudly.

Backups don't belong in git. I committed my dump file, then realized: backups contain real data, they bloat the repo forever, and in a real job that's publishing customer data. They belong in object storage. I removed it and gitignored *.dump. A small fix, but the instinct matters.

The real takeaway

Code is replaceable. You lose it, you redeploy from git. Data is not. Lose it and it may be gone forever.

Which is why the most valuable thing I did this week wasn't building the stack or writing migrations. It was deleting everything and proving I could get it back. Most junior engineers have never restored a backup. They've just trusted one exists.

Don't trust. Drill.

Next up: containers, properly.

The Week Git Stopped Being Scary: Secrets, Conflicts, and Recovery

Vivian Chiamaka Okose — Mon, 13 Jul 2026 06:41:37 +0000

Most people learn just enough Git to add, commit, and push. I used to be one of them. This module was about crossing the gap between "I use Git" and "I trust Git with my work", the gap that, on most teams, separates a junior from a mid-level engineer. Here is everything I built, including the part where I got completely lost and how I found my way back.

Making it impossible to commit a secret

The single most valuable thing I set up: a pre-commit hook chain that scans every commit before it happens. I tested it by deliberately trying to commit a fake AWS key.

The commit never happened. The scanner (gitleaks) spotted the key, pointed to the exact file and line, and blocked it cold. Nothing leaked. Nothing to rotate.

This matters because leaked cloud keys get found by attackers within seconds of hitting a public repo, and the cleanup is genuinely painful. The whole philosophy here: turn discipline into automation. Not "remember to check for secrets," but "make it physically impossible to commit one." The same chain also enforces good commit messages, so a lazy "fixed stuff" gets rejected at the door.

Surviving my first merge conflict

I used to think merge conflicts were a disaster. They're not. A conflict is just Git saying "two people changed the same spot, you decide which wins."

I created one on purpose in a throwaway sandbox: changed the same line on two branches, then tried to combine them. Git stopped and marked the clash with <<<<<<< and >>>>>>> markers.

Resolving it was simpler than the reputation suggests: edit the file to what I actually want, delete the marker lines, tell Git it's resolved. That's the entire skill. Once I'd done it once, conflicts stopped being frightening.

The concept that unlocked everything

Halfway through, I got completely lost. Moving a commit between branches made no sense to me. I stopped and admitted I didn't understand, rather than pretending.

The fix was reframing what a branch actually is. A branch is not a copy of your code. It's a sticky note stuck on one commit. "Moving a commit to another branch" is really just sliding sticky notes around while the commit itself stays put. The moment that clicked, branching, resetting, and recovery all suddenly made sense.

Admitting I was lost was the best decision I made all module.

Recovering deleted work

Then I broke things on purpose. I ran git reset --hard and wiped three commits, including a merge. They vanished. In a real panic, this is the stomach-drop moment.

Except they weren't gone. Git keeps a diary called the reflog, a record of everywhere I've been. My "deleted" commits were still in it. One command, and everything came back.

This is the calm that experienced engineers have and beginners don't. git log only shows my current path; git reflog remembers the paths I left, including the lost work. Knowing that safety net exists changes how it feels to make a mistake.

Locking down main like a real team

Finally, I protected my main branch: no direct pushes, changes must come through a pull request, commits must be signed, history must stay linear. I proved it worked by trying to push straight to main, and watching GitHub reject it.

Then I did the full professional flow: made a branch, pushed it, opened a pull request, watched my automated checks run on GitHub's servers, and merged through the proper door.

That flow, branch, push, PR, pass checks, merge, is exactly what I'll do every day in a DevOps job. Building it on my own repo, with no pressure, is the best practice I could ask for.

What I actually learned

The commands matter, but the mindset matters more. Git isn't a thing to fear; it's a safety net that remembers almost everything. The real skills are: make good habits automatic so they don't depend on memory, and know your recovery moves before you need them.

And the most important lesson had nothing to do with Git: when I was lost, saying so out loud, instead of nodding along, was what let me actually learn. That's a habit worth keeping.

Next up: databases.

From Blank VM to Bastion: Everything I Learned Hardening Linux

Vivian Chiamaka Okose — Thu, 09 Jul 2026 08:49:25 +0000

A full walkthrough of a hands-on Linux module: hardening a server, building a multi-user permission system, writing a diagnostic toolkit, and setting up a real jump host. Plus the seven times the docs lied and the delete command that scared me.

Every server I will ever touch in DevOps runs Linux. Every container is a Linux process. Every CI runner shells out to bash. So I spent a module getting genuinely fluent at the command line, not copy-pasting my way through, but understanding every step. Here is everything I built, and more importantly, everything that went wrong on the way, because that is where the real learning was.

What I built

A hardened Ubuntu 24.04 server. Starting from a blank VM, I brought it to a production baseline: a deny-by-default firewall (UFW) with only SSH allowed, fail2ban guarding against brute-force login attempts, automatic security updates, a correct timezone, and SSH locked down to key-only auth with no root login.

The order matters more than it looks. Enable the firewall before allowing SSH, and you lock yourself out of your own server. Allow SSH first, then enable. That sequence has saved more careers than any clever command.

A multi-tenant permission system. Four users, each with a different level of access: two full developers with sudo constrained to specific commands, a read-only user who can look but not touch, and a contractor sealed into her own sandbox with an account set to auto-expire in 30 days. Then I tested every boundary by becoming each user and confirming the walls held.

This is where Linux permissions stopped being abstract. Watching a read-only user get "Permission denied" when they tried to write, and watching a "restricted" admin get blocked from reading the password file, made access control real in a way no diagram could.

A four-tool diagnostic toolkit. I got tired of typing the same six commands every time something felt slow, so I wrote tools that do it for me:

sysnap: a one-screen snapshot of a server's health (load, memory, top processes, disk, open ports, failed logins)
logscan: search the system logs for a pattern, with context, scoped to a time or service
diskmap: find what's actually eating the disk, biggest-first
netcheck: confirm I'm really online and reaching the things I depend on

Each is 30 to 45 lines of bash, all shellcheck-clean, all covered by a smoke test.

A real jump host (bastion). Production servers don't sit on the open internet. They hide behind a gateway you have to hop through. I built that with three VMs: a client, a bastion, and a private server with no direct route from outside. With one line of SSH config (ProxyJump) and one command, ssh bedrock-private, SSH tunneled through the bastion to reach the hidden box.

The proof it worked: SSH printed <no hostip for proxy command> during the handshake, its way of saying "I reached this through a proxy, not directly." That phrase is the fingerprint of a working jump host.

The seven times the docs lied

Here is what the tutorials never prepare you for: the documentation and my actual system disagreed seven separate times. Every time, the system was right. Learning to read what the machine tells me, instead of trusting what the guide claims, was the single most valuable skill of the module.

The OS version was wrong. I launched "22.04" and got 24.04. lsb_release -a is the source of truth, not the launch command.
A find-and-replace silently did nothing. My sed pattern didn't match, so it changed nothing, with no error. I only caught it by verifying with grep afterward. Silent no-ops are the most dangerous kind.
A command flag didn't exist. The guide used adduser --expiredate; my version had no such flag. The fix was a different tool (chage -E).
A missing package. setfacl didn't exist until I installed acl. "Command not found" is not always a typo.
My restricted user wasn't restricted. A user I'd limited to three commands still had full admin power, because sudo group membership silently overrode my rule. Checking effective permissions, not intended ones, revealed it.
Mutually exclusive flags. df -T and df --output couldn't be combined on my build.
grep broke my test. My smoke test failed because grep returns a failure exit code when it finds zero matches, which my set -euo pipefail correctly caught. Zero failed logins is good news, but grep calls it a failure. Guard it with || true.

The mistake that taught me the most

Near the end, I ran a recursive delete with a folder path accidentally on the list, and wiped my entire screenshots folder. No recycle bin. No undo.

Except the work survived, because it lived in two other places: committed in git, and backed up separately. One git checkout and a re-copy, and everything was back in under a minute.

That scare taught me more than any smooth step. Three habits I now hold without exception:

Commit often. Recovery took a minute instead of hours because the work was already saved somewhere safe.
ls before you rm. Look at exactly what you are about to delete.
Verify, don't assume. A command running without error does not mean it did what you intended.

The real takeaway

The gap between "works in the tutorial" and "works on my actual machine" is where the real skill lives. Run the command, read the error, understand what the system is telling you, adapt, repeat. Tutorials cannot teach that loop, because tutorials never fail. Your real system will, and that is a gift.

Next up: version control done properly. On to Git.