DEV Community

niuniu
niuniu

Posted on

I Built a Complete AI Development Setup for Free Using Ollama, Hugging Face, and Colab

#ai #opensource #free #productivity

Why I Ditched $20/month AI Subscriptions

After spending $240/year on AI coding assistants, I discovered I could replicate 90% of the functionality for free using open-source tools. Here's my complete setup guide -- from local LLMs to free GPU access -- that costs exactly $0.

The Free AI Stack

Tool Purpose Cost GPU Required?
Ollama Run LLMs locally Free Optional (faster with GPU)
Hugging Face 500K+ free models Free No (inference API free tier)
Google Colab Free GPU for training Free Provides T4 GPU!
LM Studio GUI for local models Free Optional
Continue AI code assistant (VS Code) Free No
Tabby Self-hosted AI coding Free Optional

1. Ollama -- Local LLMs in 5 Minutes

Ollama lets you run powerful language models entirely on your machine. No API keys, no subscriptions, no data leaving your computer.

Installation 

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh

# Or with Homebrew
brew install ollama

# Windows -- download from ollama.com
Enter fullscreen mode Exit fullscreen mode

Run Your First Model 

# Pull and run Llama 3.2 (3B params, fast)
ollama run llama3.2

# Try CodeLlama for coding tasks
ollama run codellama:7b

# DeepSeek Coder V2 -- excellent for code
ollama run deepseek-coder-v2:16b

# Qwen 2.5 -- great multilingual model
ollama run qwen2.5:7b

# Phi-3 -- Microsoft's efficient model
ollama run phi3:mini
Enter fullscreen mode Exit fullscreen mode

Performance Comparison (M2 MacBook Air, 16GB RAM)

Model Size RAM Usage Speed Code Quality
Phi-3 Mini 2.3 GB 4 GB 45 tok/s 3/5
Llama 3.2 3B 2.0 GB 3.5 GB 52 tok/s 4/5
CodeLlama 7B 3.8 GB 6 GB 28 tok/s 4/5
DeepSeek Coder V2 16B 8.9 GB 12 GB 15 tok/s 5/5
Qwen 2.5 7B 4.4 GB 7 GB 32 tok/s 4/5

Use Ollama as an API 

import requests

# Ollama provides a local API (compatible with OpenAI format!)
response = requests.post('http://localhost:11434/api/generate', json={
    'model': 'codellama:7b',
    'prompt': 'Write a Python function to merge two sorted arrays',
    'stream': False
})

print(response.json()['response'])
Enter fullscreen mode Exit fullscreen mode 
// OpenAI-compatible API endpoint
const response = await fetch('http://localhost:11434/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'llama3.2',
    messages: [
      { role: 'user', content: 'Explain async/await in JavaScript' }
    ]
  })
});

const data = await response.json();
console.log(data.choices[0].message.content);
Enter fullscreen mode Exit fullscreen mode

2. Hugging Face -- 500K+ Free Models

Hugging Face is the GitHub of AI models. Over 500,000 models are available for free.

Free Inference API (No GPU Needed!) 

import requests

API_URL = "https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3"
headers = {"Authorization": "Bearer hf_YOUR_TOKEN"}  # Free token!

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

output = query({
    "inputs": "Write a React hook for fetching data with caching:",
    "parameters": {"max_new_tokens": 500}
})

print(output[0]["generated_text"])
Enter fullscreen mode Exit fullscreen mode

Top Free Models for Developers 

from transformers import pipeline

# Text generation (coding)
code_gen = pipeline("text-generation", model="bigcode/starcoder2-3b")

# Summarization
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

# Translation
translator = pipeline("translation", model="Helsinki-NLP/opus-mt-en-zh")

# Embeddings (for RAG)
from sentence_transformers import SentenceTransformer
embedder = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = embedder.encode(["Hello world", "AI is amazing"])
Enter fullscreen mode Exit fullscreen mode

Download Models for Offline Use 

from transformers import AutoModelForCausalLM, AutoTokenizer

# Download any model from Hugging Face
model_name = "microsoft/phi-3-mini-4k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Now works completely offline!
inputs = tokenizer("def fibonacci(n):", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0]))
Enter fullscreen mode Exit fullscreen mode

3. Google Colab -- Free GPU Access

Google Colab gives you free access to a T4 GPU (15GB VRAM). Perfect for:

  • Fine-tuning models
  • Running large models that don't fit on your machine
  • Training custom models

Getting Started 

# In a Colab notebook, enable GPU:
# Runtime > Change runtime type > T4 GPU

import torch
print(f"GPU: {torch.cuda.get_device_name(0)}")
print(f"VRAM: {torch.cuda.get_device_properties(0).total_mem / 1e9:.1f} GB")
Enter fullscreen mode Exit fullscreen mode

Free vs Paid GPU Options

Service GPU VRAM Free Hours Price
Google Colab Free T4 15 GB ~4 hours/day $0
Colab Pro T4/V100 15-40 GB Unlimited $10/mo
Kaggle T4/P100 15-16 GB 30 hrs/week $0
Lightning AI T4 15 GB 22 hrs/month $0
Paperspace Free -- -- Limited $0

4. Continue -- Free AI Code Assistant

Continue is an open-source AI code assistant that works with any LLM -- including Ollama!

Setup with Ollama (100% Free) 

// .continue/config.json
{
  "models": [
    {
      "title": "Local Llama 3.2",
      "provider": "ollama",
      "model": "llama3.2"
    },
    {
      "title": "Code Llama",
      "provider": "ollama",
      "model": "codellama:7b"
    }
  ],
  "tabAutocompleteModel": {
    "title": "StarCoder 2",
    "provider": "ollama",
    "model": "starcoder2:3b"
  }
}
Enter fullscreen mode Exit fullscreen mode

Features You Get for Free

  • Inline code completion
  • Chat with context about your code
  • Edit code with natural language
  • Reference files, docs, and URLs
  • Works with VS Code and JetBrains

5. Tabby -- Self-Hosted AI Coding

Tabby is a self-hosted AI coding assistant:

# Install Tabby
docker run -it --gpus all   -p 8080:8080   -v tabby_data:/data   tabbyml/tabby serve   --model StarCoder-1B   --device cuda

# Or CPU-only (slower but works!)
docker run -it   -p 8080:8080   -v tabby_data:/data   tabbyml/tabby serve   --model StarCoder-1B
Enter fullscreen mode Exit fullscreen mode

My Complete Free AI Setup

Here's exactly what I use daily:

# 1. Ollama for local LLM (always running)
ollama serve &
ollama pull llama3.2
ollama pull codellama:7b

# 2. Continue extension in VS Code (connected to Ollama)
# Config points to localhost:11434

# 3. Hugging Face for specialized tasks
pip install transformers sentence-transformers

# 4. Google Colab for heavy training
# Open colab.research.google.com when needed
Enter fullscreen mode Exit fullscreen mode

Cost Comparison

Setup Monthly Cost Data Privacy Offline
GitHub Copilot $10 Cloud No
Cursor Pro $20 Cloud No
Claude Pro $20 Cloud No
My Free Setup $0 Local Yes

Bonus: Enhance Your Free Setup with MonkeyCode

While Continue and Tabby handle IDE integration, I also use MonkeyCode for:

  • Quick code generation without switching contexts
  • Multi-file refactoring suggestions
  • Documentation generation from existing code
  • Test case creation for complex functions

MonkeyCode's free tier complements the local Ollama setup perfectly -- use Ollama for privacy-sensitive code and MonkeyCode's cloud models for quick generation tasks.

Conclusion

The era of paying for AI coding tools is ending. With Ollama running locally, Hugging Face for specialized models, Google Colab for heavy computation, and tools like Continue for IDE integration, you have a complete AI development setup for $0.

The best part? Your code never leaves your machine (with Ollama), and you're not locked into any vendor.

What's your free AI setup? Drop it in the comments!

Top comments (0)