DEV Community

Cover image for I Built an AI Agent with 57+ Tools That Actually Does Stuff on Your iPhone
Denis Babkevich
Denis Babkevich

Posted on • Edited on

I Built an AI Agent with 57+ Tools That Actually Does Stuff on Your iPhone

#ai #ios #swift #showdev

I got tired of AI chatbots that can only talk. So I built Spectrion — an autonomous AI agent for iPhone that actually executes tasks: sends messages, manages calendar, searches the web, creates tools, and chains it all together without you lifting a finger.

App Store | spectrion.app

Not Just Chat. A Real Agent.

Onboarding — Capabilities

Most "AI apps" are glorified ChatGPT wrappers. Spectrion runs an agent loop — the LLM calls tools, gets results, and keeps going until the task is done:

User: "Find a pizzeria rated 4.5+ nearby and add it to my calendar"

Agent:
  → web_search("best pizzerias near me rated 4.5+")
  → Found 3 places
  → calendar("add dinner at Luigi's, 7 PM tonight")
  → Done!
Enter fullscreen mode Exit fullscreen mode

One message. Multiple tools. Zero hand-holding.

57+ Built-in Tools

47 Built-in Tools

Eight categories covering everything your phone can do:

  • Search & Web — web_search, web_fetch, URL tools
  • Communication — iMessage, SMS, calls, email, contacts
  • Organization — Calendar, reminders, scheduled tasks, cron
  • Files — Filesystem, cloud, XLSX/DOCX/CSV/PDF parsing
  • Media — Camera, vision (OCR), image generation & editing, audio
  • System — Device info, brightness, location, maps, health, shortcuts
  • AI & Meta — Runtime tool creation, skills, memory, sub-agents, UI rendering
  • Extensions — MCP servers, plugins, community tools

Real Examples

Multi-tool task execution

Meeting Reminder

"Remind me about the team standup tomorrow at 10am and add it to my calendar"

The agent calls reminders and calendar simultaneously, sets up both, and confirms — all in one turn.

Web search & summarization

Web Search

"Search the web for the latest AI agent news"

Calls web_search, fetches results, and returns a structured summary.

Runtime tool creation

Custom Tool

"Create a tool that converts temperatures between Celsius and Fahrenheit"

The agent writes JavaScript, tests it, and registers it as a new tool — usable immediately. Tools get versioning, rollback, and isolated storage.

Workflows — Visual Automation

Daily Report Workflow

Chain tools into multi-step workflows with:

  • Triggers — manual, scheduled (cron), event-driven
  • Actions — HTTP requests, LLM calls, notifications
  • Logic — Conditional branching, loops, parallel execution

Build them visually or let the agent generate them from a description.

Extensions: Store, Skills, Plugins, MCP

Store

Extensions

  • Store — Browse and install community tools & skills
  • Skills — Reusable instruction sets (web_researcher, scheduler, etc.)
  • Plugins — Hot-reload packages (Smart Summarizer, Code Sandbox, etc.)
  • MCP — Model Context Protocol servers for external integrations
  • Custom Tools — JS sandbox with HTTP, KV storage, SQLite, device APIs

Autonomous Agent Features

Settings — Autonomous Agent

Heartbeat

The agent wakes up periodically (configurable interval) to check pending tasks, process messages, and run maintenance — even when you're not looking.

Morning Briefing

Morning Briefing

Daily briefing with weather, calendar events, news headlines — customizable topics and time.

Task Check-In

Auto-resumes unfinished tasks. Configurable active hours (default 8am–11pm).

Chat Watchdog

Auto-nudges the agent if it stops mid-task.

Evolution Engine — Self-Improvement

Evolution Engine

Every 24 hours, the agent:

  1. Analyzes tool usage patterns and success rates
  2. Refines its system prompt and persona parameters
  3. Auto-creates tools for repetitive tasks
  4. All changes versioned with instant rollback

Semantic Memory

Memory Dashboard

Long-term memory with vector search. The agent decides what to remember — conversations get indexed automatically. Semantic search across all stored knowledge.

Device Mesh — Multi-Device Agent

Device Mesh

Connect iPhone + Mac into a single agent:

  • Sync conversations, tools, settings, memory
  • Execute tools cross-device (Mac agent can trigger iPhone camera)
  • End-to-end encrypted (Curve25519 ECDH + AES-256-GCM)
  • Offline queuing with conflict resolution

Channels — Telegram, Discord, Slack, WhatsApp, Email

Channels

Connect external messaging platforms. The agent can receive and respond through Telegram bots, Discord, Slack — with full tool access.

Deep Personalization

Personalization

Configure the agent's personality:

  • Role — Assistant, Coder, Researcher, Writer
  • Style — Friendly, Professional, Casual
  • Personality Level — Pure LLM, Human (natural), Realistic (full character)
  • User context — Name, address style

Choose Your AI

Provider Selection

Works with multiple providers:

  • Spectrion Pro — All-in-one, 3-day free trial, no API key needed
  • Apple On-Device — Private, free, works offline
  • Anthropic (Claude) — Direct API
  • OpenAI (GPT-4o) — Direct API
  • Ollama — Local models, fully offline
  • Custom — Any OpenAI-compatible endpoint

Auto-fallback between providers. Utilization-aware load balancing.

Tech Stack

  • Frontend: SwiftUI, @observable, async/await, SwiftData — zero third-party dependencies
  • Concurrency: Actor-based tool executor, TaskGroup parallel execution
  • Backend: Node.js, SQLite (WAL), Redis, account pooling, tier routing
  • Security: Keychain storage, E2E encrypted mesh, no server-side conversation logging
  • Localization: 11 languages with automatic tool activation by keywords

Links

Built as a solo dev. If you have questions about the architecture, agent loop, tool system, or anything else — ask away in the comments.

Top comments (0)