Halfway. And today's AWS task is the last one in the 50-day AWS track, so half of the series is finished. More on that at the end.
The two tasks themselves were both about layers that look like one thing. A resource spec where a single value changes how Kubernetes treats the whole pod. A disk that has to be made bigger three separate times before the operating system notices.
One Kubernetes task, one AWS task. Set resource requests and limits on a pod, then expand an EC2 root volume without stopping the instance. The tasks come from the KodeKloud Engineer platform.
Requests, limits, and the one number
resources:
requests:
memory: "15Mi"
cpu: "100m"
limits:
memory: "<memory limit>"
cpu: "100m"
A request is what the scheduler reserves when it places the pod. A limit is the ceiling once it is running. And the two limits are enforced in very different ways. Kubernetes documents CPU limits as enforced by throttling, so a container over its CPU limit just runs slower. Memory limits are enforced by the kernel with out-of-memory kills, so a container over its memory limit may be terminated, though only when the kernel detects memory pressure, not necessarily at once. CPU over the limit is slow. Memory over the limit is dead.
The units deserve a second look too. 100m is a tenth of a CPU, one hundred millicpu. 15Mi is 15 mebibytes. And Kubernetes' own warning about case is worth quoting in spirit: 400m of memory is a request for 0.4 bytes, when whoever typed it almost certainly meant 400Mi.
Now the one number. I wrote the memory limit down as 15Mi, the same as the request, but I have not confirmed that was the figure the task gave, and it matters more than any other value in the file.
Kubernetes assigns each pod a QoS class from its requests and limits. A pod is Guaranteed only if every container has both CPU and memory requests and limits, all above zero, with each limit equal to its request. It is Burstable if it misses that but has at least one request or limit, and BestEffort if it has none at all.
The CPU request and limit here are both 100m. So if the memory limit is 15Mi, every limit equals its request, and the pod is Guaranteed. If the memory limit is anything higher, the same pod is Burstable. One value, two classes.
And the class has consequences. The documentation is explicit: when a node runs out of resources, Kubernetes evicts BestEffort pods first, then Burstable, and Guaranteed last. You can read the result with kubectl describe pod, on the QoS Class line.
One small thing I appreciated. My first draft of this manifest had a typo, rrequests. kubectl's --validate flag defaults to strict, which rejects unknown fields rather than dropping them, so that typo fails at apply instead of producing a pod with no requests and a very confusing QoS class.
A bigger disk is three jobs
The AWS task was to grow a root volume from 8 GiB to 12 GiB and have the instance see the space, without disrupting it.
| Layer | Tool | Effect |
|---|---|---|
| EBS volume | aws ec2 modify-volume |
The virtual disk is bigger |
| Partition | growpart |
The partition uses the new space |
| Filesystem |
xfs_growfs or resize2fs
|
The filesystem uses the bigger partition |
Each one is separate, and AWS's procedure says so: before you can extend a filesystem, you must extend the partition, if the volume has one. Do only the first, and you have a 12 GiB disk holding an 8 GiB partition holding an 8 GiB filesystem, and df still says 8G.
I could see the middle state directly, which is what convinced me the layers are real. After growpart, lsblk showed the partition at 12G while df still showed the filesystem at 8G. Only xfs_growfs closed the gap.
Three details from that sequence worth keeping.
You do not have to wait for the modification to finish. AWS documents that size increases take effect once the modification reaches the optimizing state, usually within seconds, and that you can extend the partition and filesystem as soon as it does. Waiting for completed burns time for nothing.
growpart takes two arguments. The AWS procedure asks you to note the space between the device and the partition number: growpart /dev/xvda 1, not growpart /dev/xvda1.
And the two filesystem tools take different arguments. xfs_growfs wants the mount point, resize2fs wants the partition device. Check df -hT first, because Amazon Linux 2023 is xfs, Ubuntu is ext4, and using the wrong one produces an error that reads like corruption.
The rule I had wrong
My notes said a volume cannot be modified again for six hours after a change. That rule is out of date.
AWS's current documentation says you must wait for a modification to reach completed before starting another on the same volume, and that you can modify a volume at most four times in a rolling 24-hour period. The six hours seems to come from a different sentence on the same page, that a 1 TiB volume can typically take up to six hours to modify.
The practical point survives, just for a different reason. EBS volumes cannot be shrunk at all, and XFS cannot be shrunk either. And you cannot iterate freely: the next change waits for the last to finish, which on a large volume can be hours, and you only get four a day. Pick the size carefully the first time.
This is exactly why I now check every note against the current documentation before it goes out. The rule I had was plausible, specific, and wrong.
Halfway, and the end of the AWS track
Fifty tasks of AWS are done, and the cloud half of this series moves to Azure from tomorrow, under the same series name.
Looking back across the fifty, three habits came up often enough to be worth naming.
Verify the deliverable, not a status field. An ECS task can read RUNNING and be unreachable. A route can look right and read blackhole. A modification can be optimising while the filesystem is still 8G. The check that means something is the one that exercises the actual promise: the curl, the cmp, the df.
The console does things the CLI makes you do yourself. A DB subnet group, an instance profile, a listener, a resource-based permission on a Lambda. Each was invisible in the console and a separate, silent failure on the CLI.
Errors name the call that failed, not the cause. A missing zip reported three commands after a broken heredoc. AccessDenied that meant a public access block. Unable to validate the following destination configurations meant a missing Lambda permission. The fix was rarely where the message pointed.
None of that is specific to AWS, which is a good sign for the Azure half.
So here is the Day 50 question. What is a rule you are confident about that you have not checked against the documentation since you learned it?
Day 50 down. Fifty to go.
Top comments (1)
Great write-up, Nnamdi — and congrats on reaching Day 50! 👏
The part that stood out to me most wasn’t actually Kubernetes or EBS, but this:
“The rule I had was plausible, specific, and wrong.”
That is one of the most dangerous situations in infrastructure work. Completely wrong information is often easy to challenge. Outdated information that used to be correct is much harder, because experience itself makes us trust it.
I’ve run into this many times over the years. You remember a limit, a default, or a recommended practice with absolute confidence, only to discover that the platform quietly changed underneath you.
Your EBS example is a perfect demonstration of why checking current documentation matters even when we think we already know the answer.
I also liked the “three separate jobs” explanation. Seeing:
disk → partition → filesystem
as separate layers is much more useful than simply memorizing three commands. The same principle applies all over infrastructure: understanding where one abstraction ends and another begins usually makes troubleshooting much easier.
As for your Day 50 question: these days, the rules I’m most suspicious of are the ones I can recite from memory without checking. 😄
Great milestone. Looking forward to seeing what changes — and what assumptions get challenged — when the series moves into Azure.