DEV Community

Chandrani Mukherjee
Chandrani Mukherjee

Posted on

Build AI Agents with Twilio: SMS, Voice & WhatsApp Automation

#python #aws #twilio

AI agents are reshaping how applications interact with the world—performing tasks, scheduling actions, retrieving information, and responding intelligently to users. Pairing AI agents with Twilio unlocks real-time communication capabilities across SMS, Voice, and WhatsApp. In this article, we’ll build a Twilio-powered Python AI agent that can reason, plan, and act.

Why AI Agents + Twilio?

An AI agent becomes far more useful when it can:

  • Receive instructions from users by SMS/WhatsApp
  • Take actions (search, fetch data, schedule reminders)
  • Trigger workflows or APIs
  • Provide reasoning back to the user
  • Handle voice calls and respond dynamically

Twilio acts as the communication gateway, while the AI model provides intelligence and decision-making.

Prerequisites

  • Python 3.10+
  • Twilio account + SMS-enabled phone number
  • AI model API (OpenAI, Groq, Anthropic, or local LLM)
  • Libraries: 
pip install twilio flask openai requests
Enter fullscreen mode Exit fullscreen mode

Architecture

  1. User sends SMS/WhatsApp → Twilio Webhook
  2. Flask endpoint receives message
  3. Python AI Agent interprets task
  4. Agent executes tools (APIs, searches, actions)
  5. Sends response back via Twilio

Example: Python AI Agent

Below is a minimal agent that can:

  • Search the web
  • Look up weather
  • Set reminders
  • Respond conversationally

agent.py 

import requests
from datetime import datetime, timedelta

class AIAgent:

    def search_web(self, query):
        # Dummy search
        return f"Search results for: {query}"

    def get_weather(self, city):
        return f"The weather in {city} is sunny and 72°F."

    def plan(self, user_input):
        user_input = user_input.lower()

        if "search" in user_input:
            query = user_input.replace("search", "").strip()
            return self.search_web(query)

        if "weather" in user_input:
            city = user_input.replace("weather", "").strip()
            return self.get_weather(city)

        if "remind" in user_input:
            return "Reminder set! (demo version)"

        return "I can help with search, weather, reminders, or questions!"
Enter fullscreen mode Exit fullscreen mode

Twilio + Flask AI Agent Endpoint

app.py 

from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse
from agent import AIAgent

app = Flask(__name__)
agent = AIAgent()

@app.route("/sms", methods=["POST"])
def sms_reply():
    user_message = request.form["Body"]
    result = agent.plan(user_message)

    resp = MessagingResponse()
    resp.message(result)

    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)
Enter fullscreen mode Exit fullscreen mode

Connecting Twilio Webhook

In Twilio Console → Phone Numbers → Messaging

Set the webhook:

https://your-server.ngrok.io/sms
Enter fullscreen mode Exit fullscreen mode

Now your number behaves like an AI agent!

Extending the Agent

You can add:

  • Calendar and task automation
  • Database lookups
  • Document RAG
  • LLM-based reasoning
  • Multi-step planning & tool execution
  • WhatsApp support

Conclusion

Twilio gives AI agents the ability to interact with users in real time across SMS, Voice, and WhatsApp. With just a few lines of Python, you can build intelligent assistants that perform tasks, answer questions, and automate workflows—all from a phone.

Top comments (0)