DEV Community

Cover image for Why Backend Engineers Should Learn Infrastructure
Tobi Ogundiyan
Tobi Ogundiyan

Posted on

Why Backend Engineers Should Learn Infrastructure

A customer clicks Create just as you deploy a new version of your API. The old process writes the record to the database, then shuts down before it can send a response. The client sees an error and tries again. Now the customer has two records.

Where do you fix that? You can change how the deployment drains requests before stopping the old process. You can also make the API recognise a retry and return the result it already committed. I’d look at both, because the customer experienced one failure, even though it crossed two parts of the system.

That’s why I think backend engineers should deploy and operate some of the software they write. You learn things about your own code that are hard to see from your laptop.

Put a service on a VPS at least once

Vercel, Railway, and platforms like them have made it easy to get an application online. I like that. When I’m trying to find out whether somebody wants to use what I’m building, getting it in front of them quickly is worth a lot.

A VPS is useful for a different reason. You have to answer the questions a managed platform usually answers for you. What starts your application after the server reboots? Which user does it run as? Where do the logs go? You might get the service running over SSH and then realise you still need a way for someone on the internet to reach it over HTTPS. By the time you’ve set up a reverse proxy and deployed a second version, you have a much clearer picture of the path a request takes to reach your code.

The second deployment is where I’d spend the time. Make a request that takes a few seconds and deploy while it’s running. Watch what happens to the request and check the database afterward. Then try rolling back. If your new version changed the schema, see whether the previous binary still works. That one exercise will change how you think about writing migrations and handling retries.

A cluster gives you more to learn once you’ve done that on one machine. Run two instances and replace one while requests are still coming in. You’ll see why a process can be running before it’s ready for traffic, and why a deployment needs to give the old process time to finish its work. Those ideas are easier to learn when you can see the failure they’re meant to prevent.

AI makes the next layer easier to learn

The gap AI has closed for me is the time between encountering something unfamiliar and being able to ask a useful question about it. If I understand my API but have never written a systemd unit, I can ask an assistant to walk through the file with me. What starts the process? Which signal does it get when I restart it? What happens if it’s still handling a request when the shutdown deadline expires?

Then I can try it. If the assistant says the request will finish gracefully, I can send a slow one and restart the service. The answer becomes something I can observe. I get a working deployment and an explanation I can carry into the next project.

Infrastructure and backend work meet in the behaviour of the application. A platform engineer can tell me how traffic moves during a deployment. I can tell them what my service does after it commits a transaction. Knowing enough about each other’s work lets us figure out why a customer ended up with two records and fix the conditions that made it possible.

If you write backend software and have only run it locally or pushed it through a managed platform, take a small service you already understand and put it on a VPS. Deploy it. Break a dependency. Roll it back. The next time a production request fails, you’ll have a better idea where to look.

Top comments (0)