I thought building a RAG application would be straightforward.
Scrape the data. Generate embeddings. Build a FAISS index. Connect an LLM. Expose an API.
That's what most tutorials make it look like.
Reality was very different.
Every optimization solved one problem and uncovered another.
Even after switching to a lightweight language model, every response still took several seconds.
That's when I realized the bottleneck wasn't the LLM. It was the architecture itself.
That realization completely changed the direction of this project. Instead of building just another chatbot, I decided to design a production-oriented GenAI platform on Oracle Cloud Infrastructure (OCI), focusing on data engineering, retrieval quality, inference optimization, deployment, observability, and LLMOps.
Over the past few weeks, I built this platform while exploring Oracle Cloud Infrastructure, documenting every architectural decision, optimization, and failure along the way.
Some Problems I Didn't Expect
What started as a simple RAG prototype quickly turned into an engineering exercise.
Along the way, I encountered challenges I hadn't anticipated:
- Large Docker images and slow deployments
- Infrastructure limitations on OCI Free Tier
- Container architecture mismatches (ARM vs. AMD)
- Retrieval and inference latency bottlenecks
- IAM and networking complexities during deployment
Every challenge forced a new architectural decision and those decisions shaped this entire series.
Why Another RAG Series?
If you've searched for RAG tutorials, you've probably noticed a common pattern.
Most articles demonstrate how to connect an LLM to a vector database and answer questions from a PDF. While that's a great way to understand the basics, it leaves out many of the challenges you'll encounter when building a real-world system.
Along the way, I found myself asking questions like:
- Where should the data actually live?
- How do you keep it synchronized?
- Why is retrieval slower than expected?
- Should you use FAISS or a managed vector database?
- How do you deploy and scale inference?
- What happens when multiple users hit the API simultaneously?
Those were exactly the questions I started asking while building this project.
The Goal
I wanted to understand why production systems look so different from GitHub demos.
The objective wasn't simply to make an LLM answer questions, it was to understand the complete lifecycle of a production GenAI application.
That meant learning how to:
- Build scalable data pipelines
- Design an efficient retrieval layer
- Optimize inference latency
- Containerize and deploy services
- Integrate with enterprise applications
- Apply MLOps and LLMOps principles
- Keep operational costs under control
Why Oracle Cloud Infrastructure(OCI)?
Most tutorials use AWS or Azure.
I intentionally chose Oracle Cloud Infrastructure (OCI) because I wanted to explore its AI ecosystem and understand how enterprise GenAI applications could be built using Oracle's cloud services.
The platform combines OCI Compute for ingestion, Object Storage as a data lake, Data Science for experimentation, Model Deployment for inference, and Oracle APEX as the application layer.
Choosing OCI also introduced a unique set of challenges around resource limits, deployment strategies, container architectures, and cloud-native design decisions many of which I'll cover throughout this series.
What We'll Build
By the end of this series, we'll have built an enterprise-oriented RAG platform capable of:
- Automated data ingestion
- Cloud-native data lake
- Semantic search using embeddings
- Hybrid Retrieval (FAISS + BM25)
- Cross-Encoder Re-ranking
- Prompt Engineering
- Streaming responses
- Guardrails
- Dockerized deployment
- OCI Model Deployment
- Oracle APEX integration
- CI/CD pipeline
- LLMOps best practices
More importantly, we'll discuss why each architectural decision was made, along with the trade-offs involved.
Note:
This isn't another "build a chatbot in 15 minutes" tutorial. It's an engineering journal documenting the architectural decisions, trade-offs, failures, and lessons learned while building an enterprise GenAI platform on OCI.
If you're looking for a copy-and-paste RAG tutorial, this series probably isn't for you.
But if you're interested in understanding how enterprise GenAI systems are designed from data ingestion and retrieval to deployment, observability, and optimization. I hope you'll find this journey worthwhile.
Enterprise AI systems aren't built by choosing the largest model.
They're built through hundreds of engineering decisions.
This series is about those decisions.
See you in Part 1.
Series Roadmap
📚 This Series (10 Parts)
Before we dive into the next article, I'd love to hear from you:
- Have you built a RAG application beyond a simple proof of concept?
- What was the biggest challenge you encountered—retrieval quality, latency, deployment, or something else?
- Are you using OCI, AWS, Azure, or another cloud platform for GenAI workloads?
- Is there a specific topic you'd like me to cover in more detail as this series progresses?
Feel free to share your experiences in the comments. I'm always interested in learning how others are approaching these problems.
If you'd like to follow my work or connect, you can find me here:
Top comments (1)
Informative!