DEV Community

Cover image for Building Andrej Karpathy's LLM Council but with Gaia Nodes: A Privacy-First Alternative to Traditional LLM APIs
Harish Kotra (he/him)
Harish Kotra (he/him)

Posted on

Building Andrej Karpathy's LLM Council but with Gaia Nodes: A Privacy-First Alternative to Traditional LLM APIs

#programming #javascript #ai #beginners

Imagine having a team of AI experts sitting around a table, each bringing their unique perspective to solve your problems. That's exactly what I built, but with a privacy-focused twist using Gaia Nodes instead of traditional cloud-based AI APIs.

This example is fully inspired by Andrej Karpathy's LLM Council.

The What

I created an AI Council application where multiple AI models work together to answer your questions. Think of it like a brainstorming session where:

  1. Each AI model gives their individual opinion
  2. They review and rank each other's answers (anonymously!)
  3. A "chairman" synthesizes everything into a final, comprehensive response

The cool part? Instead of relying on companies like OpenAI or Google, we use Gaia Nodes - decentralized, privacy-focused AI nodes that you can run yourself.

The Why

🕵️ Privacy First

  • Your conversations stay private - no data sent to big tech companies
  • You control where your data goes
  • No tracking or data mining

🔧 Full Control

  • Choose which AI models to use
  • Run everything locally or on your own servers
  • Customize the council behavior

💰 Cost Effective

  • No API fees per request
  • Predictable costs - just run your Gaia nodes
  • Scalable - add more nodes as needed

The How

Let's break down the magic with a simple example. When you ask: "What are the best practices for learning programming?"

Stage 1: Individual Opinions 📝

Each Gaia node gives its own answer:

// This happens behind the scenes:
const responses = await Promise.all([
  gaiaNode1.chat("What are the best practices for learning programming?"),
  gaiaNode2.chat("What are the best practices for learning programming?"),
  gaiaNode3.chat("What are the best practices for learning programming?"),
  gaiaNode4.chat("What are the best practices for learning programming?")
]);
Enter fullscreen mode Exit fullscreen mode

Node 1 might say: "Start with Python, focus on fundamentals..."
Node 2 might say: "Build projects, learn by doing..."
Node 3 might say: "Join communities, get mentorship..."
Node 4 might say: "Practice daily, use online resources..."

Stage 2: Peer Review 🔍

Now comes the interesting part - each node anonymously reviews the others:

// Each node gets anonymized responses to rank
const reviewPrompt = `
Here are 4 responses to "What are the best practices for learning programming?":
Response A: [Node 1's answer]
Response B: [Node 2's answer] 
Response C: [Node 3's answer]
Response D: [Node 4's answer]

Please rank them from best to worst and explain why.
`;
Enter fullscreen mode Exit fullscreen mode

The nodes don't know who wrote what - they just evaluate based on quality!

Stage 3: Final Synthesis 🎯

Finally, our "chairman" node combines everything:

const finalPrompt = `
User question: "What are the best practices for learning programming?"

All responses: [Individual answers]
All rankings: [Peer reviews]
Aggregate scores: [Combined rankings]

Create the best possible answer considering all perspectives.
`;
Enter fullscreen mode Exit fullscreen mode

The Technical Setup

Backend (Node.js + Express) 

// Simple council orchestration
export const runFullCouncil = async (userQuery) => {
  // Stage 1: Get individual responses
  const stage1Results = await stage1CollectResponses(clients, userQuery);

  // Stage 2: Get peer rankings  
  const { stage2Results, labelToNode } = await stage2CollectRankings(
    clients, userQuery, stage1Results
  );

  // Stage 3: Synthesize final answer
  const stage3Result = await stage3SynthesizeFinal(
    clients, userQuery, stage1Results, stage2Results
  );

  return { stage1: stage1Results, stage2: stage2Results, stage3: stage3Result };
};
Enter fullscreen mode Exit fullscreen mode

Frontend (React + Vite) 

// Clean, modern React components
function ChatInterface({ conversation, onSendMessage, loading }) {
  const [currentStage, setCurrentStage] = useState('stage1');

  return (
    <div className="flex flex-col h-full">
      {/* Messages display */}
      <div className="flex-1 overflow-y-auto p-6">
        {/* Council results with tabs for each stage */}
        <div className="flex gap-2 mb-4">
          <button onClick={() => setCurrentStage('stage1')}>
            Individual Responses
          </button>
          <button onClick={() => setCurrentStage('stage2')}>
            Peer Rankings  
          </button>
          <button onClick={() => setCurrentStage('stage3')}>
            Final Synthesis
          </button>
        </div>
      </div>
    </div>
  );
}
Enter fullscreen mode Exit fullscreen mode

Setting Up Your Own AI Council

Step 1: Start Your Gaia Nodes

First, you need Gaia nodes running. Each node can have different AI models:

# Example: Start 4 Gaia nodes on different ports
gaia node --port 8080 --model llama-2
gaia node --port 8081 --model codellama  
gaia node --port 8082 --model mistral
gaia node --port 8083 --model neural-chat
Enter fullscreen mode Exit fullscreen mode

Step 2: Configure Your Environment

Create a .env file with your node URLs:

# Backend configuration
GAIA_NODE_1_URL=https://your-node-id.gaia.domains/v1
GAIA_NODE_1_API_KEY=your-key-1
GAIA_NODE_2_URL=https://your-node-id.gaia.domains/v1
GAIA_NODE_2_API_KEY=your-key-2
GAIA_NODE_3_URL=https://your-node-id.gaia.domains/v1
GAIA_NODE_3_API_KEY=your-key-3
GAIA_NODE_4_URL=https://your-node-id.gaia.domains/v1
GAIA_NODE_4_API_KEY=your-key-4

CHAIRMAN_NODE=GAIA_NODE_1
Enter fullscreen mode Exit fullscreen mode

Step 3: Run the Application 

# Start backend
cd backend && npm run dev

# Start frontend  
cd frontend && npm run dev
Enter fullscreen mode Exit fullscreen mode

Visit http://localhost:5173 and start chatting with your AI council!

Real-World Use Cases

🏢 Business Decisions

  • Get multiple perspectives on strategic decisions
  • Reduce bias in important choices
  • Have AIs debate pros and cons

🎓 Learning & Research

  • Compare different explanations of complex topics
  • Get comprehensive answers from multiple angles
  • Learn through AI peer discussions

🔒 Private Consultations

  • Medical questions (with proper disclaimers)
  • Legal research (with professional verification)
  • Personal development advice

Common Questions

Q: Do I need powerful hardware?
A: Not necessarily! You can:

  • Run smaller models on regular computers
  • Use cloud servers for bigger models
  • Mix local and remote Gaia nodes

Q: Is it slower than regular ChatGPT?
A: It's parallel, so often faster! All nodes respond simultaneously, not sequentially.

Q: Can I use different models?
A: Absolutely! Each Gaia node can run any compatible model - Llama, Mistral, CodeLlama, etc.

Q: What if one node fails?
A: The council continues! We built in graceful degradation - failed nodes don't break the system.

This project shows how AI can work together collaboratively rather than competitively. Instead of choosing "the best" AI model, we get the best of all models working together.

By using Gaia Nodes, we're not just building better AI interactions - we're building private, decentralized, and user-controlled AI systems that respect your data and autonomy.

Example UI & Output

Try It Yourself!

The complete code is available, and you can:

  • ✅ Run it locally today
  • ✅ Customize it for your needs
  • ✅ Add your own Gaia nodes
  • ✅ Modify the council logic
  • ✅ Build on top of it

Top comments (0)