DEV Community

Qasim
Qasim

Posted on

Ingest invoices and receipts into an agent mailbox

#ai #email #api #productivity

Most "AI accounts payable" demos point a model at someone's personal inbox and ask it to find the invoices. That works in a screenshot. It falls apart the moment you want the thing to be an account — a real, addressable mailbox that vendors can send to, that doesn't share a thread with your CFO's lunch plans, and that you can wipe and re-provision without anyone losing their email.

The cleaner shape is to give the pipeline its own inbox. Vendors email invoices@yourcompany.com. A webhook fires the second a message lands. Your code pulls the message, walks its attachments, downloads the PDFs, and hands them to whatever extraction logic you've got — an LLM, an OCR service, a regex over the email body for the small stuff. The parsed records go into your own database, keyed however your AP system wants them.

That dedicated inbox is a Nylas Agent Account. I work on the Nylas CLI, so the terminal commands below are the exact ones I reach for when I'm wiring this up; the curl calls next to them are what the CLI is doing under the hood, so you can drop either into your stack.

What an Agent Account actually is

Here's the part worth internalizing before you write a line of code: an Agent Account is just a grant. Same grant_id, same /v3/grants/{grant_id}/* endpoints, same Authorization: Bearer <NYLAS_API_KEY> header as any connected Gmail or Microsoft mailbox. There's no separate SDK, no special document-ingestion API, no new auth dance. If you've ever listed messages or downloaded an attachment from a Nylas grant, you already know the data plane for this entire post.

What's different is the control plane. There's no human OAuth, no refresh token to babysit — you provision the mailbox yourself on a domain you own (or a *.nylas.email trial subdomain), and Nylas hosts it. For an AP pipeline that's exactly what you want: an inbox that belongs to the system, not to a person who might leave the company and take their refresh token with them.

The honest tradeoff to flag up front: Agent Accounts don't support custom metadata on messages yet. You can't tag a message with { "invoice_id": "INV-204" } and filter on it later. That's fine for document ingestion — the parsed invoice belongs in your accounting database anyway, not bolted onto an email object. We'll lean on that below.

Why route invoices here instead of scraping a shared inbox

A few reasons this beats pointing a parser at a human's mailbox:

  • One sender intent per inbox. Everything that lands in invoices@ is, by construction, an invoice or a receipt. You're not classifying "is this even AP-relevant" against a stream of newsletters and calendar invites.
  • Clean retention and blast radius. You can delete and re-provision the account without touching anyone's real email. Free-plan accounts retain the inbox for 30 days, which is plenty for an ingestion buffer — you're moving the durable record into your own DB, not relying on the mailbox as storage.
  • Policies and rules at the door. Agent Accounts let you attach a policy that caps inbound attachment size, count, and allowed MIME types. You can reject a 90 MB ZIP before it ever hits your parser.
  • Deliverability signals built in. Beyond the standard inbound message.created webhook, Agent Accounts emit message.delivered, message.bounced, and message.complaint — useful when your pipeline also sends (payment confirmations, "we received your invoice" auto-replies).

If all you ever need is one Gmail mailbox and you're happy owning its OAuth lifecycle, the regular connected-grant flow is right there. The Agent Account earns its keep when the inbox is infrastructure.

Before you begin

You'll need:

  • A Nylas API key from the dashboard, exported as NYLAS_API_KEY.
  • A registered domain for the mailbox, or a Nylas trial subdomain. New domains warm over roughly four weeks, so don't expect day-one volume — but for inbound AP ingestion, warming mostly matters for the mail you send back out.
  • The Nylas CLI if you want the terminal path: nylas init once to store your key.
  • A public HTTPS endpoint for the webhook. During development, a tunnel (ngrok or similar) is fine.

:::info
New to Agent Accounts? Start with What are Agent Accounts for the product overview, then come back here.
:::

Provision the AP mailbox

Create the grant first. The API call is POST /v3/connect/custom with provider: "nylas" and the email on your domain. The optional name sets the display name.

curl --request POST \
  --url 'https://api.us.nylas.com/v3/connect/custom' \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "provider": "nylas",
    "name": "AP Inbox",
    "settings": { "email": "invoices@yourcompany.com" }
  }'
Enter fullscreen mode Exit fullscreen mode

The response includes the grant_id — that's the only identifier you carry forward. Nylas also auto-creates a default workspace and policy for the account.

The CLI collapses the connector setup, the grant creation, and the default workspace into one command:

nylas agent account create invoices@yourcompany.com --name "AP Inbox"
Enter fullscreen mode Exit fullscreen mode

If the nylas connector doesn't exist in your app yet, this creates it first, then the grant. Add --json to capture the grant_id for a script.

Want to clamp inbound attachments before anything reaches your parser? The default policy is attached to the auto-created workspace; update it to point at a custom policy. The API call is PATCH /v3/workspaces/{workspace_id}:

curl --request PATCH \
  --url "https://api.us.nylas.com/v3/workspaces/$WORKSPACE_ID" \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{ "policy_id": "<policy-id>" }'
Enter fullscreen mode Exit fullscreen mode

CLI:

nylas workspace update <workspace-id> --policy-id <policy-id>
Enter fullscreen mode Exit fullscreen mode

There's no --workspace flag on account create — the workspace is created for you, and you attach a custom policy afterward. Tune limit_attachment_size_limit, limit_attachment_count_limit, and limit_attachment_allowed_types on the policy so a 200-page scanned PDF or a disallowed file type gets rejected at the edge. See Policies, Rules, and Lists for the full shape.

Subscribe to inbound mail

Inbound mail fires the standard message.created webhook — nothing Agent-Account-specific to learn. Subscribe to it.

curl --request POST \
  --url 'https://api.us.nylas.com/v3/webhooks' \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "trigger_types": ["message.created"],
    "webhook_url": "https://ap.yourcompany.com/webhooks/nylas",
    "description": "AP invoice ingestion"
  }'
Enter fullscreen mode Exit fullscreen mode

CLI:

nylas webhook create \
  --url https://ap.yourcompany.com/webhooks/nylas \
  --triggers message.created \
  --description "AP invoice ingestion"
Enter fullscreen mode Exit fullscreen mode

This post is scoped to inbound mail: you care about message.created for invoices that vendors send to the account. If your pipeline also auto-replies (a "we received your invoice" confirmation), guard your handler so it only acts on mail addressed to the account — filter on the sender at the top so you never try to parse your own outbound confirmation as an invoice.

Here's the skeleton handler. Ack fast, do the work off the request path:

// Node.js / Express
app.post("/webhooks/nylas", async (req, res) => {
  // Verify X-Nylas-Signature first -- see /docs/v3/notifications/.
  res.status(200).end();

  const event = req.body;
  if (event.type !== "message.created") return;

  const msg = event.data.object;
  if (msg.grant_id !== AP_GRANT_ID) return;

  // Only act on inbound mail -- skip anything from the account's own address.
  if (msg.from?.[0]?.email === "invoices@yourcompany.com") return;

  await ingestInvoice(msg);
});
Enter fullscreen mode Exit fullscreen mode

Read the message and find its attachments

The webhook payload is a summary — subject, from, snippet, thread_id, and a list of attachment stubs. It deliberately doesn't carry the full body or the attachment bytes. (And if the body tops ~1 MB, the trigger becomes message.created.truncated and the body is dropped entirely.) So step one is to fetch the full message.

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/$AP_GRANT_ID/messages/$MESSAGE_ID" \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY"
Enter fullscreen mode Exit fullscreen mode

The message object includes an attachments array. Each entry carries everything you need to decide what to download:

{
  "id": "msg_abc123",
  "subject": "Invoice INV-204 — Acme Supplies",
  "from": [{ "name": "Acme Supplies", "email": "billing@acme.example" }],
  "attachments": [
    {
      "id": "att_9xkz",
      "filename": "INV-204.pdf",
      "content_type": "application/pdf",
      "size": 48213,
      "is_inline": false
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

content_type and filename are your filter. For AP you typically want application/pdf and the spreadsheet types, skipping is_inline: true entries — those are logos and signature images embedded in the HTML, not documents.

The CLI gives you the same picture two ways. To read the full message:

nylas email read <message-id> [grant-id]
Enter fullscreen mode Exit fullscreen mode

To list just the attachments on that message:

nylas email attachments list <message-id> [grant-id]
Enter fullscreen mode Exit fullscreen mode

That second command is the one I lean on when I'm eyeballing what a vendor actually sent — it prints the attachment IDs, file names, and types in a table, which is exactly the input the download step needs. Pass --json and you've got machine-readable stubs to iterate over.

If you'd rather not run a webhook at all, polling works too. Listing messages on a cadence is a perfectly reasonable choice for a batch AP run that processes the day's invoices every evening:

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/$AP_GRANT_ID/messages?in=INBOX&limit=50" \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY"
Enter fullscreen mode Exit fullscreen mode

CLI:

nylas email list --folder INBOX --limit 50 --json
Enter fullscreen mode Exit fullscreen mode

Webhooks win for near-real-time ("the invoice hit the inbox, kick off approval now"); polling wins for tidy nightly batches. Both are first-class.

Download the attachment bytes

This is the heart of document ingestion, and the one endpoint with a non-obvious shape. The download path requires the message_id as a query parameter — the attachment ID alone isn't enough to locate the bytes:

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/$AP_GRANT_ID/attachments/$ATTACHMENT_ID/download?message_id=$MESSAGE_ID" \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY" \
  --output INV-204.pdf
Enter fullscreen mode Exit fullscreen mode

The response streams the raw file bytes — no base64 wrapper, no JSON envelope. Pipe it straight to disk, an object store, or an in-memory buffer for your parser.

The CLI mirrors the same two-identifier requirement. Note the argument order — attachment ID first, then message ID:

nylas email attachments download <attachment-id> <message-id> [grant-id] -o INV-204.pdf
Enter fullscreen mode Exit fullscreen mode

By default it writes to the attachment's original file name; -o overrides the output path. This is genuinely the same call as the curl above — the CLI is just URL-encoding the IDs and handling the stream for you, which matters because attachment IDs sometimes contain characters (#, /) that break an un-encoded URL.

There's also a metadata-only endpoint if you want the size and type before committing to a download — handy when a policy didn't pre-filter and you want to bail on anything over, say, 10 MB. Like the download call, it requires the message_id query parameter:

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/$AP_GRANT_ID/attachments/$ATTACHMENT_ID?message_id=$MESSAGE_ID" \
  --header 'Authorization: Bearer '"$NYLAS_API_KEY"
Enter fullscreen mode Exit fullscreen mode

The CLI takes the same two positionals — attachment ID first, then message ID:

nylas email attachments show <attachment-id> <message-id> [grant-id]
Enter fullscreen mode Exit fullscreen mode

Extract amount, date, and vendor — honestly

Here's where I'll be straight with you: Nylas hands you the document, not the parsed fields. There is no "extract invoice total" endpoint, and you shouldn't expect one. The amount/date/vendor extraction is your application code, and it generally looks like one of:

  • An LLM with the PDF or its extracted text. Pass the downloaded bytes (or text you've pulled with a PDF library) to a model and ask for structured JSON: { "vendor", "invoice_number", "amount", "currency", "due_date" }. This handles the messy long tail of vendor layouts that rules never will.
  • OCR for scanned/image PDFs. Plenty of invoices are photographed receipts. Run them through an OCR service first, then your LLM or regex over the text.
  • Cheap heuristics for the easy stuff. Some vendors put the total right in the email body. The full message you already fetched includes body — a regex for a currency pattern can catch the obvious cases before you spend a model call.

A reasonable pipeline composes all three: try the body regex, fall back to text extraction plus an LLM, fall back to OCR for image-only PDFs. Whatever you build, the Nylas side ends at the downloaded bytes. Frame your expectations there and you won't be surprised.

async function ingestInvoice(msg) {
  const message = await getMessage(AP_GRANT_ID, msg.id);

  const documents = message.attachments.filter(
    (a) => !a.is_inline && a.content_type?.startsWith("application/pdf")
  );

  for (const att of documents) {
    const bytes = await downloadAttachment(AP_GRANT_ID, att.id, msg.id);

    // Your code: LLM / OCR / regex over `bytes` and `message.body`.
    const parsed = await extractInvoiceFields(bytes, message.body);

    // Store in YOUR database -- metadata isn't supported on the message.
    await db.invoices.insert({
      sourceMessageId: msg.id,
      sender: message.from?.[0]?.email,
      filename: att.filename,
      ...parsed,
      ingestedAt: new Date(),
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

Where the parsed records live

This is the design decision that trips people up coming from email-tagging workflows: you can't store the parsed invoice on the message. Agent Accounts don't support custom metadata yet, so there's no field on the Nylas object to stash invoice_id or approved: true.

That's not a limitation for AP — it's the right boundary. The durable record of an invoice belongs in your accounting database, keyed by your invoice number, joined to your vendor table, audited by your finance team. Keep the message.created payload's id as a back-reference (so you can re-download the original document later), and let your DB own everything else. The mailbox is the intake; your database is the system of record.

If you do need lightweight in-mailbox organization — say, moving processed invoices out of the inbox — that's what folders and rules are for. You can move a message to an archive folder once ingested, or write a rule that auto-files mail from known vendor domains. But the data lives in your DB.

Guardrails worth setting before you ship

  • Dedup on the message ID. Webhook redelivery and concurrent workers will both re-trigger your handler. A unique constraint on sourceMessageId (or an idempotency check before insert) stops you from booking the same invoice twice — the kind of bug finance will notice.
  • Filter inline attachments. is_inline: true entries are logos and signatures. Send them to your parser and you'll waste model calls on a vendor's email banner.
  • Cap attachment size and type at the policy. Reject the giant scan or the unexpected .exe at the door rather than in your parser.
  • Mind free-plan limits. 200 messages per account per day and 3 GB of org storage on the free tier. For high-volume AP you'll want a paid plan, but the limits are generous enough to build and prove the pipeline.
  • Verify the webhook signature. Check X-Nylas-Signature before trusting a payload. An unauthenticated /webhooks endpoint is an invitation to inject fake invoices into your AP flow.

What's next

AI-answer pages for agents

When this post is published, link AI agents and crawlers to the retrieval-ready version on cli.nylas.com:

Top comments (0)