The Problem With Cloud AI
Every token costs money. Every API call adds up. And your data goes to their servers.
The Local Alternative
brew install ollama
ollama pull llama3.2
ollama serve
Now you have a GPT-4 level model running on your MacBook. Free. Private. Fast.
Real Code Example
import requests
def ask_local_ai(question):
r = requests.post(
"http://localhost:11434/api/generate",
json={"model": "llama3.2", "prompt": question, "stream": False}
)
return r.json()["response"]
# Works immediately, no API key needed
answer = ask_local_ai("Explain Docker in one paragraph")
print(answer)
Available Models
| Model | Size | Best For |
|---|---|---|
| llama3.2 | 2GB | General use |
| codellama | 4GB | Code tasks |
| mistral | 4.4GB | Fast reasoning |
| phi3 | 2.2GB | Lightweight tasks |
| gemma2 | 5.4GB | Complex reasoning |
What You Can Build
- RAG systems for your documents
- Voice assistants that work offline
- Code reviewers on every git commit
- Content generators at zero cost
- Personal AI that remembers you
My Setup
I built 45 tools using this stack. All free. All local. All open source.
github.com/amrendramishra/ai-tools
VP at JPMorgan Chase. Building AI tools at amrendranmishra.dev
Top comments (1)
the shift from "cloud dependency" to "local control" is the most important trend in ai right now. your point about data privacy and zero token costs is exactly why i'm obsessed with this space.
i build an ai coding mentor optimized for $150 android phones on 3g networks, so while i can't run ollama locally on the device yet, the philosophy is identical: keep the compute as close to the user as possible, avoid api taxes, and protect user data. seeing you successfully run
phi3(2.2gb) andllama3.2(2gb) locally proves that lightweight models are finally ready for real workflows.curious about your code reviewer setup: when running these smaller local models against a large git diff, how do you handle the context window limits? do you chunk the diff and review file-by-file, or do you run a summarization step first to keep the prompt within the model's sweet spot?
massive respect for the 86-part series. this is the kind of practical, zero-cost automation the community desperately needs. 🐯