Running an Autonomous AI Agent Stack on Android with Termux — No Cloud, Just a Phone

I’ve spent the last few months trying to turn an old Android phone into a self‑contained AI agent that can listen, reason, and act without ever reaching for a cloud service. The goal was simple: keep the bill at zero, rely only on what fits inside Termux, and learn where the real bottlenecks hide. What follows is a honest walk‑through of the architecture, the things that broke, and the tiny tricks that kept the whole thing alive.

Why Termux?

Termux gives you a Linux‑like environment without root. You can install packages with pkg, run Node.js, Python, or even compile C binaries. For a phone‑only stack it’s the only realistic way to get a full‑featured shell, a package manager, and persistent storage that survives reboots. The downside? You’re stuck with the phone’s RAM (usually 2‑4 GB) and no /tmp directory that behaves like a typical Linux tmpfs. Anything you write to /data/data/com.termux/files/usr/tmp is still counted against the app’s private storage, which can fill up quickly if you’re not careful.

Core Components

1. Speech‑to‑Text – I used Vosk’s offline model (English, ~50 MB) wrapped in a small Node.js script that streams audio from the mic via arecord.
2. LLM Reasoning – A quantized Llama‑2‑7B model running through llama.cpp. The binary is about 1.4 GB; I store it on the external SD card to avoid eating internal storage.
3. Action Executor – A JavaScript bridge that interprets the LLM’s JSON output and triggers Termux APIs (to send SMS, toggle Wi‑Fi, launch apps, etc.).
4. Watchdog – A Bash loop that restarts any crashed component and logs OOM events.

The data flow is: mic → Vosk → text → LLM → JSON command → executor → action → feedback (spoken back via espeak-ng).

The Real Problems

OOM Kills

The first time I launched the Llama binary, the phone killed it after ~12 seconds with “Killed”. Android’s low‑memory killer (LMK) treats any process that exceeds a certain fraction of total RAM as a candidate. With 3 GB RAM, the Llama process plus the Node.js wrapper and Vosk exceeded the limit.

Fix: I swapped the model for a 3‑B parameter quantized version (≈800 MB) and pinned the Llama process to a specific CPU core using taskset. I also lowered the JVM heap for Node (node --max-old-space-size=256). After those tweaks the LMK left us alone.

Heredoc Bugs in Termux

Termux’s default shell is bash, but it behaves oddly with quoted heredocs when the delimiter contains spaces. I tried to embed a multi‑line prompt for the LLM like this:

LLM_INPUT=$(cat <<'EOF'
You are a helpful agent. Respond with JSON.
User said: "$(cat /data/data/com.termux/files/home/last_speech.txt)"
EOF
)

The script would hang waiting for the delimiter because Termux’s bash strips the newline before the closing EOF when the heredoc is quoted. The workaround was to avoid quoting the delimiter and instead escape any $ inside the block:

LLM_INPUT=$(cat <<EOF
You are a helpful agent. Respond with JSON.
User said: \"$(cat /data/data/com.termux/files/home/last_speech.txt)\"
EOF
)

That tiny change eliminated the hang.

No Usable /tmp

Because Termux doesn’t mount a real tmpfs, writing large temporary files to /tmp quickly exhausts the app’s private storage. I initially dumped the Vosk audio chunks there, leading to “No space left on device” errors after a few minutes.

Fix: I redirected all temporary files to a folder on the external SD card (/storage/XXXX-XXXX/tmp) and added a periodic cleanup cron job (*/5 * * * * rm -rf /storage/XXXX-XXXX/tmp/*). The external storage is slower but far more capacious.

The Watchdog Loop

Below is the Bash watchdog I run in a separate Termux session. It checks each critical process every 15 seconds and restarts it if it’s missing or has exceeded a memory threshold (using pmap to gauge RSS).

#!/data/data/com.termux/files/usr/bin/bash
while true; do
  # Check Vosk listener
  if ! pgrep -f "vosk_listener.js" >/dev/null; then
    echo "$(date): Vosk listener dead, restarting"
    node /data/data/com.termux/files/home/vosk_listener.js &
  fi

  # Check LLM backend
  if ! pgrep -f "llama.cpp" >/dev/null; then
    echo "$(date): LLM backend dead, restarting"
    taskset -c 0-3 /data/data/com.termux/files/home/llama.cpp/main -m /sdcard/models/llama-2-7b-q4.bin -n 128 &
  fi

  # Simple OOM guard: if RSS > 800M, kill and respawn
  for pid in $(pgrep -f "llama.cpp"); do
    rss=$(pmap $pid | tail -1 | awk '{print $2}' | sed 's/K$//')
    if [ $rss -gt 800000 ]; then
      echo "$(date): LLM RSS $rss KB > limit, killing $pid"
      kill $pid
    fi
  done

  sleep 15
done

Save this as watchdog.sh, make it executable (chmod +x watchdog.sh), and launch it with ./watchdog.sh &. The loop writes timestamps to the terminal, which you can pipe to a log file if you like.

Quoted Heredoc Example (Prompt Builder)

Here’s the quoted heredoc I use to build the prompt that gets fed to the LLM. It pulls the latest transcription, inserts it into a static instruction block, and then pipes the result to the Llama binary.

bash
PROMPT=$(cat <<'EOF'
You are an AI agent running on a phone. 
When the user speaks, output a JSON object with keys "action" and "params".
Valid actions: "send_sms", "toggle_wifi", "open_app", "speak".
Only output JSON, no extra text.

User said: "$(cat /data/data/com.termux/files/home/last_speech.txt)"
EOF

---
The toolkit from this journey: https://samhiotisiddn-jpg.github.io/ironvision-store/