Hey Devs! 👋
I’m currently working on an exciting hands-on project that blends the power of LLMOps and DevOps: an AI-powered Anime Recommender System.
Though the project is still a work in progress, I wanted to share what I’m exploring:
- The architecture and toolchain I'm using 🧱
- The technologies I’m learning along the way 🧰
- And why this is such an exciting learning opportunity 💡
I believe in learning in public, so here’s a peek behind the scenes!
🎯 Project Goal
The goal is to build a recommender system that
- Uses Groq LLM APIs and HuggingFace embeddings
- Stores vector data in ChromaDB
- Communicates with LLMs using LangChain
- Has an interactive UI built in Streamlit
- Is deployed on GCP VM via Docker + Kubernetes
- And monitored with Grafana Cloud
🛠️ Tech Stack
| Category | Tools/Technologies |
|---|---|
| LLM & Embeddings | Groq API, HuggingFace |
| GenAI Framework | LangChain |
| Vector Store | ChromaDB |
| Frontend/UI | Streamlit |
| Containerization | Docker |
| Orchestration | Kubernetes (via Minikube) |
| Deployment Platform | GCP VM |
| Monitoring | Grafana Cloud |
| CLI | Kubectl |
| Version Control & CI/CD | GitHub |
🔄 Architecture (Work in Progress)
_Here’s the architectural flow I’m working
_
Project Setup
- Groq API, HuggingFace API
- Logging, Custom Exception Handling
- Virtual Environment, Project Structure
Core Engine
- Configuration and Data Loading
- Embedding storage using ChromaDB
- Prompt templating
- Recommender Class logic
- Training & Recommendation process
UI and Deployment
- Streamlit App as frontend
- Dockerizing the app
- Deploying on Minikube within a GCP VM
- Monitoring Kubernetes metrics with Grafana Cloud
📚 What I’m Learning
✅ Integrating Groq and HuggingFace for LLM-based workflows
✅ Creating prompt pipelines with LangChain
✅ Managing embeddings using ChromaDB
✅ Deploying containerized apps using Docker on Kubernetes
✅ Running K8s inside GCP VM with Minikube
✅ Setting up Grafana Cloud dashboards for cluster observability
✅ CI/CD and GitHub-based DevOps flows for MLOps/LLMOps
📌 What’s Next
- Finalizing the embedding + recommender pipeline
- Adding GitHub Actions CI/CD
- Creating APIs for interaction
- Hosting a public demo
- Publishing full documentation and source code
💡 Why Share This Now?
This isn’t a finished project. I’m posting while I explore and build — because
- Feedback makes me better
- Others might benefit from this stack
- It keeps me accountable and motivated
💬 Stay Tuned for Part 2
In the next post, I’ll share
- Source code and repo walkthrough
- API endpoints for recommendations
- LLM prompt tuning strategies
- Live monitoring via Grafana Cloud
Thanks for reading! 🙏
If you’re working on something similar or want to explore LLMOps, feel free to drop your thoughts and ideas in the comments!
Let’s connect 🚀
Top comments (0)