I built a 17-agent AI swarm on my phone — here's how

I built a 17-agent AI swarm on my phone — here's how

Okay, buckle up. This is going to be a bit of a deep dive. For the past few weeks, I’ve been obsessed with pushing the boundaries of what's possible on mobile AI. Specifically, I wanted to see if I could run a genuinely complex multi-agent system – a swarm of interacting AI agents – directly on my phone. Not just a demo, but something that felt…alive. The result? A 17-agent AI swarm I’m calling “Nexus”, running entirely locally on an Android device. Here’s how I did it.

Why a Swarm? And Why on a Phone?

Multi-agent systems (MAS) are fascinating. They allow you to model complex behaviors by letting simple agents interact. Think flocking birds, ant colonies, or even market dynamics. Each agent follows a set of rules, and the emergent behavior of the swarm is often far more sophisticated than the rules themselves.

The ‘why phone’ part is a challenge I enjoy. Everyone's talking about cloud-based AI, which is fantastic, but what about offline capability? What about privacy? What about leveraging the processing power we already carry in our pockets? Modern smartphones are seriously powerful. Ignoring that feels like leaving a lot on the table. My goal was to create something demonstrable that proved a complex AI system isn’t limited to server racks.

The Core Tech Stack

This project leans heavily on a few key technologies:

• Python (via Pydroid 3): Android isn’t exactly known for its native AI development tools. Pydroid 3 provides a surprisingly capable Python environment for Android. It's not perfect, but it’s a lifesaver.
• TensorFlow Lite: This is crucial. TensorFlow Lite is Google’s framework for running TensorFlow models on mobile and embedded devices. It’s optimized for size and speed.
• NumPy: The backbone for numerical operations in Python. Essential for handling the agent states and interactions.
• A (very) simple custom communication protocol: The agents needed to talk to each other. More on that later.
• A touch of threading: To parallelize some of the agent computations.

The Agents: Simple Minds, Complex Interactions

Each agent in Nexus is conceptually simple. They operate within a 2D space (think a virtual room) and have these core attributes:

• Position (x, y): Their coordinates.
• Velocity (vx, vy): How fast they’re moving, and in what direction.
• Energy: A value representing their “health” or activity level.
• Perception Radius: How far they can "see" other agents.
• Behavioral Weights: These dictate how they respond to different stimuli (more on this below).

The agents’ behavior is governed by three primary rules, implemented as functions:

1. Cohesion: Move towards the average position of nearby agents.
2. Separation: Avoid colliding with nearby agents.
3. Alignment: Match the average velocity of nearby agents.

These rules are combined using weighted sums. Each agent has weights assigned to each rule (cohesionWeight, separationWeight, alignmentWeight). Tweaking these weights dramatically changes the swarm’s behavior. This is where the fun begins.

Here’s a simplified Python snippet showing the cohesion calculation:

import numpy as np

def cohesion(agent, agents, perception_radius, cohesion_weight):
    nearby_agents = [a for a in agents if np.linalg.norm(np.array(agent['position']) - np.array(a['position'])) < perception_radius and a != agent]

    if not nearby_agents:
        return np.array([0.0, 0.0])

    avg_pos = np.mean([np.array(a['position']) for a in nearby_agents], axis=0)
    direction = avg_pos - np.array(agent['position'])
    magnitude = np.linalg.norm(direction)

    if magnitude > 0:
        direction = direction / magnitude * cohesion_weight
    else:
        direction = np.array([0.0, 0.0])

    return direction

This calculates the direction an agent should move to be closer to its neighbors. Similar functions exist for separation and alignment.

The Swarm Dynamics & Communication

Each agent’s movement is updated in each frame (roughly 30 frames per second). The total movement is the sum of the forces from cohesion, separation, and alignment, scaled by their respective weights.

The tricky part was inter-agent communication. I didn't want to rely on complex messaging queues. Instead, I opted for a simplified approach:

• Shared Memory: I created a global list (agents) that all agents can access.
• Direct Reading: Each agent simply reads the positions and velocities of nearby agents from this list.
• Limited Range: The perception_radius ensures agents only consider nearby neighbors, minimizing the computational load.

This system isn't perfect – it's prone to race conditions if not carefully managed (threading helps mitigate this), but it's remarkably effective for this relatively small swarm.

Optimization for Mobile: The TensorFlow Lite Angle

Running 17 agents, each with calculations for cohesion, separation, and alignment, 30 times a second can quickly bog down a phone. Here's where TensorFlow Lite comes in.

Initially, I was using pure Python for all calculations. It was slow. Really slow. I identified the distance calculations (using np.linalg.norm) as the biggest bottleneck. Instead of doing these calculations in Python, I trained a simple TensorFlow model to approximate the distance between two points.

The model takes the (x, y) coordinates of two agents as input and outputs a single value representing the distance. It’s a simple fully-connected network, trained on a dataset of randomly generated points.

python
import tensorflow as tf

model = tf.keras.Sequential([
  tf.keras.layers.Dense(16, activation='relu', input_shape=(4,)),
  tf.keras.layers.Dense(8, activation='relu'),
  tf.keras.layers.Dense(1)
])

model.compile(optimizer='adam', loss='mse')

# Training data (example)
x_train = np.random.rand(1000, 4)  # 1000 samples, 4 features (x1, y1, x2, y2)
y_train = np.linalg.norm(x_train[:, :2] - x_train[:, 2:], axis=1, keepdims=True)

model.fit(x_train, y_train, epochs