GitHub now hosts millions of AI-related repositories, and new frameworks seem to trend every other week. For a developer trying to actually build production AI systems not just chase whatever is viral on a given Tuesday it helps to have a curated map of what actually matters. Here's the list I keep coming back to, organized by what problem each category solves.
1. Orchestration Frameworks (Tying It All Together)
LangChain — Still the default starting point for chaining LLM calls, retrievers, tools, and memory into a working pipeline. Its ecosystem (LangSmith for tracing, LangGraph for stateful agent flows) has matured into a full toolkit rather than just a library.
LlamaIndex — Where LangChain is general-purpose, LlamaIndex is more specialized for data ingestion and retrieval parsing documents, building indices, and connecting structured/unstructured data sources to an LLM.
Dify and Langflow — Visual, no-code/low-code builders for LLM apps and agent workflows. Useful for prototyping fast or handing off simpler use cases to non-engineers on your team.
2. Local Inference & Self-Hosted Models
Running models on your own hardware has gone from a niche hobby to a serious production strategy, mostly driven by cost and data-privacy concerns.
Ollama — A lightweight Go-based tool for downloading and serving open-weight models (Llama, Mistral, Gemma, DeepSeek, and others) locally with a couple of terminal commands. Pairs well with a self-hosted chat UI for a fully private setup.
llama.cpp — The engine underneath a lot of local-inference tooling. Lets large models run efficiently on CPUs, which matters a lot if you don't want to depend on expensive GPU instances for every experiment.
vLLM — For when you do have GPUs and need high-throughput serving in production, with efficient memory management for concurrent requests.
3. Vector Databases (The Backbone of RAG)
If you're building anything retrieval-based, you'll end up choosing one of these:
Qdrant — Rust-based, fast, with strong metadata filtering. A common pick for performance-sensitive, self-hosted setups.
Weaviate — Go-based with a richer schema model than most alternatives it stores objects and vectors together and supports hybrid (keyword + vector) search out of the box.
Milvus — Built for billion-scale datasets with a cloud-native, horizontally scalable architecture — the choice when you're operating at real enterprise scale.
Chroma — The simplest to get started with. Great for local development and prototyping before you commit to a heavier production database.
pgvector — If your team already runs PostgreSQL, this extension lets you add vector search without introducing a whole new piece of infrastructure.
4. Multi-Agent & Autonomous Agent Frameworks
Agent frameworks have been the fastest-growing category on GitHub over the past year, with visual builders like Langflow and Dify leading in adoption alongside code-first frameworks for more custom multi-agent orchestration, task delegation, and tool use. This space is also where the Model Context Protocol (MCP) ecosystem has taken off a growing set of MCP servers that let agents plug into external tools and data sources through a standard interface, rather than every framework inventing its own integration layer.
5. Fine-Tuning & Model Training
Hugging Face Transformers — Still the standard library for working with pretrained models, tokenizers, and training pipelines across virtually every modality.
PEFT — Parameter-efficient fine-tuning (LoRA and friends) so you can adapt large models without needing a full retraining budget.
6. Repos Worth Following Just to Learn
nanochat (Andrej Karpathy) — A minimal, educational implementation for understanding LLM training fundamentals from scratch. Not production-ready, but excellent for building real intuition instead of just calling an API.
awesome-chatgpt-prompts and similar "awesome" lists — Not code, but genuinely useful curated references when you need a starting point for prompt design or want to see what a category of tools looks like at a glance.
How to Actually Use This List
Bookmarking a repo does nothing on its own. What's worked for me:
Read the source of at least one framework you use daily. You'll understand its failure modes far better than any documentation will tell you.
Star selectively, not everything. A curated list of 15–20 repos you actually revisit beats 200 stars you'll never open again.
Contribute small fixes where you can. Even a documentation fix on a popular repo teaches you more about real-world codebases than most tutorials.
I write about RAG systems, AI agents, and backend engineering. You can check out my work at Portfolio or connect with me on LinkedIn / GitHub.
Top comments (0)