In the previous article, I shared how I optimized the cloud bill for the company. But in reality, there were still many things that needed improvement.
Pipeline to Deploy Code
Our company uses Jenkins to run pipelines.
Of course, I have worked with GitLab CI/CD before, but for me they are all just tools.
Argo CD, GitHub Actions, or AWS CodePipeline — in the end they are simply tools to build and deploy code.
At the beginning, our pipeline only had a build stage for backend projects. Deployments were still done mostly by hand.
So I added new steps to automatically deploy to staging and production.
Now everything is fully automated after a simple git push.
Pretty nice 😄
At the moment, all the projects I work on already have automatic deployment pipelines. My job now is simply to change the code and push it to git.
Working with Git
I have to admit something.
Even though I had used git for a long time, I never really had an effective workflow.
In a team with many developers, things could easily become messy and hard to manage.
Every time we wanted to test a new feature, we had to ask questions like:
- Can we deploy now?
- Will it overwrite someone else’s code?
- Is another feature still being tested?
- Is it my turn to test?
The main problem was simple:
we only had one dev environment.
At that time, my understanding of an effective git workflow was still very vague.
Then one day, while walking home from work along the river, without looking at my phone, just enjoying the fresh air and thinking randomly about git workflows… suddenly an idea came to me. Haha.
Later I found a blog post online that described almost exactly the same workflow.
That’s when I realized:
it wasn’t a new idea — I just hadn’t been exposed to it before.
In the end, everything is about what works best for your team.
My principles are very simple:
- The
masterbranch is always the most stable and correct version of the system. - All new features and bug fixes must start from this branch.
- The
developbranch is used for testing. - Feature branches and bug-fix branches must merge into
developto be deployed to the testing environment. - After testing is successful on
develop, the feature branch can then be merged intomasterand deployed to production.
With this workflow, our team has been working very smoothly so far.
If problems appear later…
maybe I will just go walk along the river again to think about it. 😄
Infrastructure (Infrastructure as Code)
The benefits of Infrastructure as Code are well known, so I probably don’t need to explain them much. You can easily find plenty of information online or just ask AI.
After finishing the deployment of the backend infrastructure on Amazon Web Services, the next thing I did was write IaC for all the projects I worked on.
The idea was simple.
One day, if I am no longer working here, at least the people who stay — or new engineers joining the company — will have something that helps them understand the infrastructure that was built.
And if something goes wrong, they can quickly rebuild it.
I use OpenTofu, which is a fork of Terraform. Everything is basically the same.
Sorry AWS, but I’m not a big fan of AWS CloudFormation.
Terraform code just looks much nicer to me 😄
Monitoring
I will probably write a separate article about this topic.
For me, after one year at this company, the thing I’m most proud of is not AWS infrastructure, but building a monitoring platform for the company.
At my previous company, I worked with Datadog, but at that time my understanding of monitoring was still very basic. I mostly just used it to read logs.
Back then I was just a developer, and honestly no one really guided me to explore the system deeply.
When I joined this company, I was surprised to see that there was no monitoring platform at all.
If developers wanted to read logs, they had to SSH into the server.
At that moment I felt that monitoring was absolutely necessary.
Without it, debugging production systems feels like fighting enemies with bare hands.
Today, monitoring takes a significant part of my daily work. But the value it brings has helped me learn many new things.
Backup
Do you know what the most valuable asset of a company is?
For me, it’s data.
As long as the data still exists, the company can survive.
If the data disappears, the company may disappear too.
There was a company that lost all its data after hackers gained root access and deleted everything. They asked AWS for help to recover it, but it wasn’t possible. Eventually the company had to shut down.
That story made me think a lot about how to protect our data.
Our company uses Amazon RDS with PostgreSQL.
But what happens if someone gains root access?
Everything could disappear instantly.
Eventually I learned about AWS Backup. With this service, backups are protected and cannot easily be deleted — even with root access.
At least, within my current understanding, this feels safer.
Unless the entire AWS infrastructure collapses… which hopefully is very unlikely.
Cron Jobs
This is something almost every company needs — processing large datasets periodically.
In our company, we originally used the Spring Boot scheduled annotation to run these jobs.
It worked well.
But there were two problems:
- Debugging these jobs could be difficult.
- Horizontal scaling becomes problematic. If multiple servers run at the same time, the same job might execute twice.
My solution was quite simple.
Instead of refactoring everything, I moved job scheduling to:
- AWS Lambda
- Amazon EventBridge
EventBridge triggers Lambda, which then calls a HTTP API endpoint in our Spring Boot service. The controller routes the request to the appropriate job based on parameters.
Everything became much easier to manage.
Conclusion
I’m always thinking about ways to improve the systems I work on.
Maybe some of these solutions look simple to others.
But for me, every time I find a solution, it brings a small sense of joy.
And honestly, I’m always a little proud of that. 😉
(And yes, I still take walks by the river to brainstorm!) 😊
(If you enjoy these kinds of engineering stories, you can subscribe or visit my blog to receive the next ones.)
Connect me on LinkedIn :D
Top comments (0)