I built a 17-agent AI swarm on my phone — here's how

I built a 17-agent AI swarm on my phone — here's how

Okay, buckle up. This isn't your average "hello world" tutorial. I've spent the last few weeks building a surprisingly capable AI swarm that runs entirely on my Android phone. And no, I'm not talking about running pre-trained models – I'm talking about 17 independent agents, each with their own goal, memory, and reasoning capabilities, collaborating (and sometimes colliding) to solve problems.

Why? Well, partly because I could. Partly because the advancements in on-device machine learning are genuinely astounding. And partly because I wanted to see just how far I could push the limits of what's possible with limited resources.

The Core Idea: Agent-Based Modeling & LangChain

The architecture is based on agent-based modeling (ABM). Imagine a bunch of digital ants, each following simple rules, but collectively able to achieve complex tasks. Each 'agent' is an independent AI entity, using Large Language Models (LLMs) as its reasoning engine.

I leveraged LangChain, a framework for developing applications powered by language models, to streamline the process. Specifically, I used LangChain's Agents module. This allows me to define agents with specific tools, memory, and roles.

The Tech Stack: Surprisingly Mobile-Friendly

This is where it gets interesting. Running 17 LLMs on a phone sounds… ambitious. Here’s what I used:

• Android Phone: A moderately powerful phone – Pixel 7 Pro. RAM is crucial (8GB+ recommended).
• llama.cpp: This is the hero. llama.cpp allows running Large Language Models (LLMs) quantized to incredibly small sizes directly on CPU and GPU, including on mobile. We’re talking models that fit within the phone's memory.
• Ollama: Makes managing and running llama.cpp models much easier. It handles model downloads, setup, and provides an API.
• Python (via Termux): The coding language. Termux provides a Linux environment on Android, enabling Python execution.
• LangChain: For orchestrating the agents.
• SQLite: A lightweight database to store agent memories and task history.
• GPT-2 (774M parameters, quantized): The LLM. I opted for GPT-2 because it’s relatively small and performs well when quantized. Larger models (like Llama 2) are possible with more RAM.

The Agents: Roles and Responsibilities

Each agent has a specific role. Here are a few examples:

• The Researcher: Responsible for gathering information from a predefined set of sources (mostly text files stored on the phone).
• The Summarizer: Takes long pieces of text and condenses them into key takeaways.
• The Critic: Analyzes the output of other agents, looking for inconsistencies or inaccuracies. A vital role!
• The Planner: Breaks down complex tasks into smaller, manageable steps.
• The Executor: Carries out the plans generated by the Planner.
• The Memory Keeper: Manages long-term memory for the entire swarm, summarizing past interactions and key findings.
• The Coordinator: Acts as a central hub, delegating tasks and coordinating agent actions.

Code Snippet: Agent Creation (Simplified)

This shows a simplified example of creating a Researcher agent:

from langchain.agents import initialize_agent, Tool
from langchain.llms import Ollama
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

# Define the tool - in this case, accessing files
def read_file(filename):
    with open(filename, "r") as f:
        return f.read()

tools = [
    Tool(
        name="Read File",
        func=read_file,
        description="Useful for reading the contents of a file. Input should be the filename."
    )
]

# Instantiate the LLM (using Ollama)
llm = Ollama(model='gpt2')

# Define the prompt
template = """You are a researcher AI agent. Your goal is to answer questions by consulting files. 
If you don't know the answer, say "I do not have the information needed to answer".

Here are your available tools: {tools}

Question: {input}
"""
prompt = PromptTemplate(
    input_variables=["input", "tools"],
    template=template
)

# Create the LLMChain
chain = LLMChain(llm=llm, prompt=prompt)

# Initialize the agent
agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)

# Now 'agent' can be used to interact with the environment and tools

This snippet shows how to:

1. Define a Tool: read_file allows the agent to access data.
2. Instantiate the LLM: Using Ollama pointing to the gpt2 model.
3. Create a Prompt: The prompt instructs the agent on its role.
4. Initialize the Agent: initialize_agent sets up the agent with its tools and LLM.

The Challenges (and Workarounds)

This project wasn’t without its hurdles:

• Memory Management: Running 17 LLMs simultaneously eats RAM. Quantization (reducing the precision of the model's weights) was essential. I used 4-bit quantization to significantly reduce the model size. Strategic garbage collection within the Python script also helped.
• Speed: Inference on a phone is slow compared to a dedicated GPU. I optimized prompts to be concise and avoided unnecessary complexity.
• Context Length: GPT-2 has a limited context length. I implemented a hierarchical memory system, where the Memory Keeper agent summarizes past conversations, providing condensed context to other agents.
• Agent Collisions: Multiple agents attempting to access the same resource simultaneously caused issues. I implemented a simple locking mechanism using Python's threading library to manage resource access.
• Termux Limitations: Termux doesn't have full Linux compatibility. Finding workarounds for certain dependencies was necessary.

How It Works: A Simple Task Example

Let’s say the goal is to "Summarize the key arguments for and against remote work." Here’s how the swarm tackles it:

1. Coordinator: The Coordinator breaks down the task: "Find relevant information," "Summarize the pros," "Summarize the cons," "Critique the summaries," "Combine the results."
2. Researcher: The Researcher retrieves articles on remote work from local files (loaded onto the phone).
3. Summarizer (x2): Two Summarizer agents independently create summaries of the pros and cons.