Day 54 of DevOps, Day 4 of Azure. Today was about where things belong, and the answer was not the obvious container in either case.
A file written in one container and read from another turned out to belong to neither of them. And a virtual network in West US sat inside a resource group in East US, which turns out to be completely normal.
One Kubernetes task, one Azure task. Share a volume between two containers in one pod, then create a virtual network in a named region. The tasks come from the KodeKloud Engineer platform.
One volume, two paths
volumes:
- name: volume-share
emptyDir: {}
containers:
- name: volume-container-devops-1
image: debian:latest
command: ["sleep", "3600"]
volumeMounts:
- name: volume-share
mountPath: /tmp/official
- name: volume-container-devops-2
image: debian:latest
command: ["sleep", "3600"]
volumeMounts:
- name: volume-share
mountPath: /tmp/apps
Write a file through the first container, read it through the second:
# inside volume-container-devops-1
echo "Welcome to xFusionCorp Industries" > /tmp/official/official.txt
# from outside, through volume-container-devops-2
kubectl exec volume-share-devops -c volume-container-devops-2 -- cat /tmp/apps/official.txt
The same file is /tmp/official/official.txt in one container and /tmp/apps/official.txt in the other. The Kubernetes docs say an emptyDir "can be mounted at the same or different paths in each container". Where a file appears is a property of the mount, not of the file.
Different paths were fine here, and yesterday they broke a website. The difference is whether a path travels. Yesterday nginx sent PHP-FPM a file path, so the two containers had to agree on it. Here neither container tells the other where anything is, so they do not have to.
It belongs to the pod
The volume's lifetime follows the pod, not the containers. It is created when the pod is assigned to a node, and it starts empty. The docs say its data "is safe across container crashes", and that when the pod is removed from the node, the data "is deleted permanently". It is for scratch space and sharing, not for keeping anything.
The sleep 3600 shows that difference in practice. The debian image's default command is bash, and with no terminal and no script, a shell has nothing to do and exits. A pod's restartPolicy defaults to Always, so without sleep the containers would just restart over and over. With it, they run for an hour, then exit and restart, and the file is still there afterwards, because it never belonged to the container.
A small fix to my own notes: I had cat official.txt inside the first container, which only works if the shell happens to be sitting in /tmp/official. It is not, by default, so the full path is the version that works every time.
A VNet in a different region from its resource group
The Azure task was one virtual network, devops-vnet, in West US.
RG kml_rg_main-bfe226f9972c4173 eastus
VNet devops-vnet westus
The resource group is in East US. The VNet inside it is in West US. Both are correct, and this is not a workaround.
Microsoft describes a resource group's location as where the group's metadata is stored, and says the resources in a group "can be located in different regions than the resource group, but we recommend that you use the same location."
The trap is the default. az vm create documents that it falls back to the resource group's location when you leave out --location, which is how yesterday's VM landed in East US without being told to. The az network vnet create reference documents no default at all. Either way, the rule is the same: when a task names a region, pass --location. A default that happens to match is luck, not correctness.
Coming from AWS, there is nothing to confuse, because nothing sits between the account and the resource. The region comes from your CLI configuration or --region.
The rule in my notes did not survive the docs
My notes said that if the resource group's region had an outage, you might be unable to create, change or delete resources in the group, even ones running healthily somewhere else.
Microsoft's current documentation says otherwise: "If a resource group's region is temporarily unavailable, Azure Resource Manager automatically reroutes ARM control plane requests to a backup region." That covers the management calls, not the resources themselves. Requests sent straight to a resource's own endpoints are not affected by the resource group's location, and if the region a resource runs in is down, that resource is affected wherever its group lives.
So the metadata location matters less than I had written. It is the same pattern as the EBS rule on Day 50: specific, plausible, and not what the current documentation says.
"Any CIDR" is only true on day one
The task allowed any IPv4 range, and I used 10.0.0.0/16. That is also exactly what az network vnet create uses when you give it no prefix at all; the reference lists it as the default. It is how an organisation ends up with three VNets on the same range without anyone choosing it.
It matters the day two of them need to talk. Microsoft's rule for peering is that "The virtual networks you peer must have nonoverlapping IP address spaces", the same rule as AWS VPC peering on Day 49. You can add address ranges to a VNet later, which solves running out of space. It does not solve an overlap, because the new range cannot overlap a peer's either. Azure's newer subnet peering narrows the problem by peering chosen subnets instead of whole VNets, but it does not remove it.
In a lab, any range will do. Anywhere else, it should come from a plan.
Where things belong
A file belongs to the volume, and the volume belongs to the pod. A VNet belongs to a region, and its resource group only keeps the record. In both cases the thing you see holding it is not the thing that decides how long it lives or where it runs.
So here is the Day 54 question. If your team created three new networks tomorrow without a plan, what address range would each of them get?
Day 54 down. Forty-six to go.
Top comments (1)
The contrast with Day 53 is the interesting part for me.
Yesterday, different mount paths were a bug because the path itself crossed the boundary between nginx and PHP-FPM. Today, different mount paths are perfectly valid because only the underlying data is shared.
Same Kubernetes feature, opposite outcome depending on what is actually part of the contract between the containers. That’s a useful distinction.
And for your network question: if my team created three networks tomorrow with no address plan, I’d stop them before the third one existed. 😄
“Just use 10.0.0.0/16” is harmless right up until somebody asks for peering, VPN connectivity, a cluster spanning networks, or connectivity back to an existing corporate network. Then yesterday’s convenient default becomes tomorrow’s migration project.
I’d rather allocate from an organisation-wide IPAM plan from day one, with non-overlapping blocks reserved by environment/region/site and enough space left for growth.
CIDR planning is boring when you don’t need it.
When you finally need it, it’s suddenly very expensive. 😂
Day 54 down. 👊