DEV Community

Qasim Muhammad
Qasim Muhammad

Posted on • Originally published at cli.nylas.com

Connect Voice Agents to Email and Calendar with Python

#python #webdev #ai #tutorial

Voice agents can talk, but they can't email. They handle phone calls, take orders, and answer questions — then the conversation ends and the context disappears. No follow-up email. No calendar invite. No CRM update.

Bridging voice to email and calendar is a common integration challenge. You need to capture the voice agent's output, format it for email, authenticate with the right provider, and handle the send — all in real time while the caller is still on the line.

Nylas CLI makes this a subprocess call. Your voice agent framework (LiveKit, Vapi, Retell, or custom) captures the intent, and the CLI handles email delivery and calendar creation. No API client libraries, no OAuth plumbing, no SMTP config.

Architecture 

┌──────────────┐     ┌──────────────┐     ┌─────────────┐
│  Voice Agent  │────▶│   Python     │────▶│  Nylas CLI   │
│ (LiveKit/Vapi)│◀────│  Bridge      │     │ (email/cal)  │
└──────────────┘     └──────────────┘     └─────────────┘
                           │
                      ┌────▼────┐
                      │   LLM   │
                      │(compose)│
                      └─────────┘
Enter fullscreen mode Exit fullscreen mode

Install Nylas CLI 

brew install nylas/nylas-cli/nylas
nylas auth login
Enter fullscreen mode Exit fullscreen mode

Full setup: Getting Started with Nylas CLI

Send a follow-up email after a voice call

The simplest integration. After your voice agent finishes a call, send a summary email:

import subprocess

def send_followup(to_email, caller_name, summary):
    """Send a follow-up email after a voice call."""
    subject = f"Follow-up: Call with {caller_name}"
    body = f"Hi {caller_name},\n\nThanks for calling. Here's a summary:\n\n{summary}\n\nBest regards"

    subprocess.run([
        "nylas", "email", "send",
        "--to", to_email,
        "--subject", subject,
        "--body", body,
        "--yes"
    ])

# After voice call ends
send_followup(
    "alice@company.com",
    "Alice",
    "- Discussed Q2 timeline\n- Agreed on April 15 deadline\n- You'll send the spec by Friday"
)
Enter fullscreen mode Exit fullscreen mode

Full email sending guide: Send Email from the Command Line

Schedule a meeting during a voice call

Caller says "Let's book a meeting for next Tuesday at 2pm." Your agent creates it:

def schedule_meeting(title, when, duration, participants):
    """Create a calendar event from voice agent context."""
    cmd = [
        "nylas", "calendar", "create",
        "--title", title,
        "--when", when,
        "--duration", duration,
        "--participants", ",".join(participants),
        "--yes"
    ]
    result = subprocess.run(cmd, capture_output=True, text=True)
    return result.returncode == 0

# Voice agent captures: "Book a meeting with Bob next Tuesday at 2pm for 30 minutes"
schedule_meeting(
    "Call follow-up with Bob",
    "next tuesday 2pm",
    "30m",
    ["bob@company.com"]
)
Enter fullscreen mode Exit fullscreen mode

Calendar guides:

Check availability by voice

"Am I free tomorrow afternoon?" Your agent checks and responds:

import json

def check_availability(from_time, to_time):
    """Check calendar availability."""
    result = subprocess.run(
        ["nylas", "calendar", "list",
         "--from", from_time, "--to", to_time, "--json"],
        capture_output=True, text=True
    )
    events = json.loads(result.stdout) if result.returncode == 0 else []
    return events

# Voice agent asks about tomorrow
events = check_availability("tomorrow 12pm", "tomorrow 6pm")
if not events:
    response = "You're free tomorrow afternoon."
else:
    titles = [e["title"] for e in events]
    response = f"You have {len(events)} events: {', '.join(titles)}"
Enter fullscreen mode Exit fullscreen mode

LiveKit integration example

A complete LiveKit voice agent that can email and schedule:

from livekit.agents import AutoSubscribe, JobContext, WorkerOptions, cli
from livekit.agents.voice_assistant import VoiceAssistant
import subprocess
import json

async def entrypoint(ctx: JobContext):
    assistant = VoiceAssistant(
        # ... your LLM and TTS config
    )

    @assistant.on("function_call")
    async def on_function_call(call):
        if call.name == "send_email":
            subprocess.run([
                "nylas", "email", "send",
                "--to", call.args["to"],
                "--subject", call.args["subject"],
                "--body", call.args["body"],
                "--yes"
            ])
            return "Email sent."

        elif call.name == "schedule_meeting":
            subprocess.run([
                "nylas", "calendar", "create",
                "--title", call.args["title"],
                "--when", call.args["when"],
                "--duration", call.args.get("duration", "30m"),
                "--yes"
            ])
            return "Meeting scheduled."

        elif call.name == "check_inbox":
            result = subprocess.run(
                ["nylas", "email", "list", "--unread", "--limit", "5", "--json"],
                capture_output=True, text=True
            )
            emails = json.loads(result.stdout)
            summaries = [f"{e['from'][0]['email']}: {e['subject']}" for e in emails]
            return "Your unread emails: " + "; ".join(summaries)
Enter fullscreen mode Exit fullscreen mode

Read inbox by voice

"Do I have any unread emails?" or "What did Alice send me today?"

def get_unread_summary(limit=5):
    """Get a voice-friendly summary of unread emails."""
    result = subprocess.run(
        ["nylas", "email", "list", "--unread", "--limit", str(limit), "--json"],
        capture_output=True, text=True
    )
    emails = json.loads(result.stdout) if result.returncode == 0 else []

    if not emails:
        return "No unread emails."

    summaries = []
    for e in emails:
        sender = e["from"][0].get("name", e["from"][0]["email"])
        summaries.append(f"{sender} sent: {e['subject']}")

    return f"You have {len(emails)} unread emails. " + ". ".join(summaries)
Enter fullscreen mode Exit fullscreen mode

For provider-specific email reading:

Extract OTP codes by voice

Voice agents handling account verification can pull OTP codes:

def get_otp():
    """Get the latest OTP code for voice readback."""
    result = subprocess.run(
        ["nylas", "otp", "get", "--raw"],
        capture_output=True, text=True
    )
    if result.returncode == 0:
        code = result.stdout.strip()
        return f"Your verification code is {' '.join(code)}"  # spell out digits
    return "No verification code found."
Enter fullscreen mode Exit fullscreen mode

Full OTP guide: Extract OTP Codes from Email

Record the call itself

Use the built-in notetaker if the voice call happens over Zoom/Meet/Teams:

nylas notetaker send --meeting-link "https://zoom.us/j/123456789"
Enter fullscreen mode Exit fullscreen mode

Full guide: Record Zoom, Meet, and Teams from the CLI

Audit trail

Every action your voice agent takes through the CLI is logged:

nylas audit list --limit 20
# Shows: who did what, when, from which agent
Enter fullscreen mode Exit fullscreen mode

Full guide: AI Agent Audit Logs

Full guide with Vapi, Retell, and custom framework examples: Connect Voice Agents to Email and Calendar

Related guides:

All guides: cli.nylas.com/guides

Top comments (0)