I built a 17-agent AI swarm on my phone — here's how

I built a 17-agent AI swarm on my phone — here's how

Okay, buckle up. This is going to be a bit of a deep dive. For the last few weeks, I’ve been obsessed with running a multi-agent system (a "swarm") directly on my phone. Not just accessing it remotely, but actually running the processing locally. And not just a few simple agents, but 17 interacting, goal-oriented AI entities. Why? Because it's cool, challenging, and I believe local AI processing is the future. Here's how I did it.

The Vision: Decentralized Intelligence in Your Pocket

I've been following the advancements in Large Language Models (LLMs) and agent frameworks with excitement. The idea of autonomous agents tackling complex tasks is incredibly powerful. However, much of the focus is on cloud-based solutions. What if you could have a mini-swarm operating on-device, handling personal tasks, learning from your behavior, and providing insights without relying on a network connection? That's the goal.

The use case I initially targeted was a personalized "research assistant." Imagine having agents dedicated to summarizing articles, finding relevant data, generating creative content based on your preferences, and even formulating questions you hadn't thought to ask. Each agent would have a specific role, and they’d collaborate to achieve a common objective.

The Tech Stack: A Surprisingly Viable Combination

Running 17 agents on a phone isn't trivial. Computational resources are limited, and battery life is a constant concern. Here’s the stack I landed on:

• Language Model: This was the biggest hurdle. Full-sized LLMs like GPT-4 are obviously out of the question. I opted for TinyLlama-1.1B-Chat-v1.0, a surprisingly capable 1.1 billion parameter LLM. It’s small enough to run reasonably well on a modern smartphone. I quantized it down to 4-bit using llama.cpp which significantly reduced its memory footprint.
• Agent Framework: I chose BabyAGI. While simple, it provides a solid foundation for managing agent tasks and memory. The core loop of BabyAGI – task creation, prioritization, and execution – maps well to my desired swarm behavior.
• Orchestration & Persistence: SQLite. This is where things got interesting. Instead of relying on a vector database like ChromaDB (too resource-intensive for a phone), I leveraged SQLite to store agent memories, tasks, and results. This allows for persistent state even when the app is closed.
• Development Environment: Python (using Kivy for a basic UI). Kivy allows for cross-platform development, making it easier to test on different Android devices. I utilized termux on my phone for initial testing and debugging, accessing the Python environment via SSH.
• Hardware: Google Pixel 7 Pro. It’s got decent RAM (12GB) and a capable processor, but this wouldn't run smoothly on a lower-end device.

The Architecture: A Swarm of Specialists

The "swarm" consists of 17 agents, each designed for a specific function. Here's a breakdown of some key roles:

• Task Master (1 Agent): Manages the overall task queue and prioritizes tasks.
• Summarizer (2 Agents): Takes long-form text and generates concise summaries. I split this into two to handle concurrent summarization of different sources.
• Fact Checker (2 Agents): Verifies information from summaries or external sources.
• Question Generator (3 Agents): Formulates relevant questions based on the current task and available information.
• Content Creator (4 Agents): Writes articles, summaries, or other creative content. I experimented with different "personalities" for these agents (e.g., formal, informal, technical).
• Web Scraper (2 Agents): Retrieves information from the web based on specific keywords.
• Memory Curator (2 Agents): Processes and stores relevant information into the SQLite database for future use.
• Output Formatter (1 Agent): Presents the final results in a user-friendly format.

Code Snippets: The Heart of the Swarm

Here's a simplified example of the agent task execution loop in Python:

import llama_cpp
import sqlite3
import os

# Load the LLM (quantized version)
llm = llama_cpp.Llama(model_path="./tinyllama-1.1b-chat-v1.0.Q4_0.gguf", n_ctx=2048)

# Connect to the SQLite database
conn = sqlite3.connect('swarm_memory.db')
cursor = conn.cursor()

def execute_task(task_description):
    """Executes a task using the LLM and stores the result in the database."""
    prompt = f"You are a helpful agent. Your task is: {task_description}\n\nResponse:"
    output = llm(prompt, max_tokens=256, stop=["\n"], echo=False)
    result = output["choices"][0]["text"]

    # Store the task and result in the database
    cursor.execute("INSERT INTO tasks (description, result) VALUES (?, ?)", (task_description, result))
    conn.commit()

    return result

# Example Usage:
task = "Summarize the key points of the latest news on Artificial Intelligence."
result = execute_task(task)
print(f"Task: {task}\nResult: {result}")

conn.close()

This is a simplified example, but it illustrates the core principle. Each agent would have a slightly modified prompt tailored to its specific role. For instance, the Fact Checker agent would receive a prompt emphasizing the need for accurate verification. The Web Scraper agent would use a prompt designed to generate search queries.

Challenges & Optimizations

This wasn't without its challenges.

• Performance: TinyLlama, while impressive, is still slow on a phone. I optimized by limiting the context window (the amount of text the model considers at once) and using batch processing where possible.
• Memory Management: SQLite is efficient, but the database can still grow quickly. I implemented a “garbage collection” system to periodically remove old or irrelevant data.
• Prompt Engineering: Getting the agents to cooperate effectively required a lot of experimentation with prompts. Clear, concise prompts are crucial for guiding the LLM's behavior.
• UI Responsiveness: Kivy wasn't the fastest UI framework, but it was the most straightforward option for quickly building a prototype.

Results & Future Directions

The current system is functional, albeit a bit sluggish. It can successfully summarize articles, answer questions, and generate