Lanre Awe

Posted on Jun 15

From Learning DevOps to Deploying a Production Application: My DMI Cohort 2 Experience

Introduction

One of the biggest milestones in my DevOps journey was participating in the deployment of a production-ready microservices application as part of the DevOps Mentorship Initiative (DMI) Cohort 2.

Together with a team of talented engineers, we successfully deployed the Spring Petclinic Microservices application to Azure Kubernetes Service (AKS). This was not a simple demo project—it involved real-world tools, cloud infrastructure, CI/CD pipelines, observability, AI integration, and production deployment practices.

The experience challenged me technically and personally, but it also showed me what working on a real DevOps project looks like.

About the Project

Spring Petclinic is a cloud-native microservices application built with Spring Boot. The application consists of multiple services that communicate with each other and are deployed as containers.

Our technology stack included:

Spring Boot
Spring AI
Docker
Terraform
Azure Kubernetes Service (AKS)
Helm
Azure Pipelines
Azure OpenAI
Grafana

As part of the project, I worked as the GenAI Engineer, responsible for integrating the AI chatbot using Spring AI and Azure OpenAI.

My Role: Integrating AI into the Platform

My responsibility was to make the chatbot feature work seamlessly within the microservices environment.

This involved:

Connecting the application to Azure OpenAI
Integrating Spring AI into the backend services
Ensuring the chatbot worked correctly within a reactive Spring WebFlux environment
Troubleshooting deployment and runtime issues

Although it was a single project ticket, it required understanding multiple layers of the architecture—from application code to cloud infrastructure.

Challenges We Faced

No real-world deployment is without problems, and this project was no exception.

1. Docker Hub Access Issues

At one point, Docker Hub access became unreliable due to ISP restrictions.

To solve this, we imported images directly into Azure Container Registry (ACR) using:

az acr import

This removed our dependency on Docker Hub and improved reliability.

2. Azure OpenAI Quota Problems

Having Azure credits did not automatically mean we had access to AI model quotas.

We had to upgrade the subscription and request the required quota before deploying the chatbot successfully.

3. Spring WebFlux Blocking Calls

The AI service initially caused threading issues because the AI call was blocking.

The solution was to wrap the operation using:

Mono.fromCallable(...)
    .subscribeOn(Schedulers.boundedElastic())

Once implemented, the chatbot responded correctly without blocking the reactive application.

4. Secrets Accidentally Committed to Git

One of the most important lessons came from discovering that an API key had been committed to the repository.

The team immediately:

Rotated the compromised key
Removed it from Git history
Improved secret management practices

It was a valuable reminder that security starts from day one.

5. Empty Vector Store

The chatbot initially returned poor results because the vector store had been built with incompatible embeddings.

We regenerated the embeddings using the correct Azure OpenAI model and rebuilt the vector store using production data.

What I Learned

This project taught me lessons that go beyond certifications and tutorials.

Cloud Solves Real Problems

Some issues that are difficult to handle locally become much easier when using managed cloud services.

Reactive Programming Requires Discipline

Spring WebFlux works extremely well, but blocking operations can quickly cause problems if not handled correctly.

Secrets Management Is Critical

Security cannot be treated as an afterthought. Proper handling of credentials and sensitive information must be part of the development process from the beginning.

Deployment Order Matters

In a microservices environment, services depend on one another. Starting services in the wrong order can prevent the entire application from functioning correctly.

The Team Behind the Project

One of the most rewarding parts of this experience was working alongside an amazing team.

Special appreciation to:

Michael Ikedimma
Benjamin Akinteye
Gift Ukporo
Duru Juliet Chinenye
Pradeep Neelaboyina
Angela Chibuike
Oladayo Aremu
Ubani OnU Chukwu
Osman Farah Ali Farah
Kolawole Yinusa

Everyone brought unique skills and expertise that helped make the project successful.

Final Thoughts

Looking back, this project was one of the most impactful experiences of my career so far.

Beyond deploying applications and solving technical problems, I learned how real engineering teams collaborate, troubleshoot, communicate, and deliver production systems.

The journey was challenging, but every obstacle helped me grow as a DevOps engineer.

If you're looking for a practical way to learn DevOps by working on real-world projects, I highly recommend the DevOps Mentorship Initiative (DMI).

DMI Cohort 3 starts on 27 June.

Apply here:

https://docs.google.com/forms/d/e/1FAIpQLSel7ai7nyb0P1qLW4vEyfB_nEsD4lUF1XG88vmAaFGBOb6hPA/viewform