DEV Community

Doni Leong
Doni Leong

Posted on Edited on

Controlling My Local AI Coding Agent (Google Antigravity) from Telegram with Voice Notes

#productivity

I use Google's Antigravity (a local AI coding environment) every day to write tests, fix bugs, and refactor code. But there was one problem: I had to sit in front of my computer the whole time.

When I take a walk, ride the train, or make dinner, I often think of a bug fix or a task I want to test. Typing long prompts on a phone keyboard is slow and annoying. Sending a quick 20-second voice note is much easier.

I built a tool called agy-pilot to connect Telegram with my local Antigravity environment. Now, I can send a voice message from my phone, let the agent work on my machine, and approve risky actions with one tap.

Here is how the setup works and how I built it.

System Architecture

The whole setup runs locally on my Mac. I do not open any ports on my home router. The bridge uses Telegram's long-polling method, so all network requests go outbound over HTTPS.

[ Telegram App on Phone ]
       |
       | (Send voice note or text)
       v
[ Telegram Bot API (Cloud) ]
       |
       | (Outbound HTTPS polling)
       v
[ Local Daemon on My Mac (agy-pilot) ]
       |-- 1. Checks my Telegram User ID (blocks everyone else)
       |-- 2. Downloads the .ogg voice note to disk
       |-- 3. Hands the file path to Antigravity
       v
[ Google Antigravity (Local Session) ]
       |-- Transcribes / reads audio
       |-- Runs shell commands, edits code
       v
[ Progress & Approval Buttons sent back to Telegram ]
Enter fullscreen mode Exit fullscreen mode

Step 1: Receiving and Saving Voice Messages

Telegram saves voice notes as .ogg files (Opus audio). Antigravity can read audio files directly, so I do not need a third-party speech-to-text API. I only need to download the file and store it where the agent can find it.

Here is the Python code using aiogram:

from pathlib import Path
from aiogram import Bot
from aiogram.types import Voice

async def download_voice_note(bot: Bot, voice: Voice, message_id: int, workspace_dir: Path) -> Path:
    # Save the file into the project folder
    upload_dir = workspace_dir / ".user_uploads"
    upload_dir.mkdir(parents=True, exist_ok=True)

    filename = f"voice_{message_id}_{voice.file_unique_id}.ogg"
    destination = upload_dir / filename

    # Get file details from Telegram and download
    file_info = await bot.get_file(voice.file_id)
    if not file_info.file_path:
        raise RuntimeError("Could not find file path from Telegram")

    await bot.download_file(file_info.file_path, destination=destination)
    return destination.resolve()
Enter fullscreen mode Exit fullscreen mode

Once the file is saved, the daemon tells Antigravity:

audio_path = await download_voice_note(bot, message.voice, message.message_id, conv_dir)

prompt = (
    f"The user sent a voice message saved at: {audio_path}.\n"
    "Please listen to this audio file, follow the instructions, and reply."
)
Enter fullscreen mode Exit fullscreen mode

Antigravity processes the file directly, plans the task, and starts editing code.

Step 2: Showing Progress Without Hitting Rate Limits

Coding agents take multiple steps. They think, read files, run terminal commands, and check test results.

If you send a new Telegram message every time the agent does something, two things happen:

  1. Your chat fills up with hundreds of notifications.
  2. Telegram blocks you with an HTTP 429 error (Too Many Requests).

To fix this, I update a single "status message" instead of sending new ones:

  1. State Tracking: The daemon tracks what Antigravity is doing (Thinking, Running pytest, Editing src/main.py).
  2. Throttled Updates: It edits the Telegram message at most once every 1.5 seconds. If the agent changes status three times in one second, only the latest status is sent.

This keeps the chat clean and shows real-time progress without hitting Telegram limits.

Step 3: Security and Action Approvals

Running shell commands remotely on your main computer can be dangerous. You do not want anyone else using your bot, and you do not want the agent running commands like git push --force or rm -rf by accident.

I added two layers of protection:

1. User ID Whitelist

Telegram assigns a unique number to every user account. The bot checks this ID before doing anything:

ALLOWED_USER_IDS = {123456789}  # Replace with your real Telegram user ID

@router.message()
async def handle_message(message: types.Message):
    if message.from_user.id not in ALLOWED_USER_IDS:
        # Ignore messages from anyone else
        return
    ...
Enter fullscreen mode Exit fullscreen mode

2. Inline Approval Buttons

When Antigravity wants to run a risky tool, it pauses and waits for user confirmation. The daemon sends an inline keyboard to Telegram:

Agent wants to run:
rm -rf dist/ && python -m build

[ Allow Once ]    [ Reject ]
Enter fullscreen mode Exit fullscreen mode

When I tap Allow Once, Telegram sends a callback back to my Mac, and Antigravity continues. If I tap Reject, it stops.

Step 4: Running Automatically on macOS

I do not want to keep a terminal window open all day. I use macOS launchd to keep agy-pilot running in the background.

I created a file at ~/Library/LaunchAgents/com.ii2d.agy-pilot.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.ii2d.agy-pilot</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/python3</string>
        <string>-m</string>
        <string>agy_pilot</string>
        <string>start</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/tmp/agy-pilot.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/agy-pilot.err.log</string>
</dict>
</plist>
Enter fullscreen mode Exit fullscreen mode

Start the service:

launchctl load ~/Library/LaunchAgents/com.ii2d.agy-pilot.plist
Enter fullscreen mode Exit fullscreen mode

Now it runs on startup and restarts automatically if it ever crashes.

What I Learned

  • Voice is much faster: Explaining a problem out loud gives the agent more context than typing short sentences on a phone.
  • Keep it local: By pulling messages from Telegram with long-polling, I did not have to set up webhooks, open router ports, or pay for cloud servers.
  • Always keep a confirmation step: Having mobile approval buttons makes using local agents away from my desk much safer.

Note: This setup is an independent personal bridge for running local Antigravity sessions and is not affiliated with or endorsed by Google.

Top comments (0)