DEV Community

Georg Piwonka
Georg Piwonka

Posted on

Building SmartReceiptOrganizer: An AI-Powered Email Receipt Processing System

#devchallenge #postmarkchallenge #webdev #api

This is a submission for the Postmark Challenge: Inbox Innovators.

What I Built

I created SmartReceiptOrganizer - an automated receipt processing system that turns your email inbox into a smart financial database. The system receives receipt emails via Postmark's inbound webhooks, automatically detects PDF attachments, and uses AI to extract key financial data like amounts, currencies, dates, and merchant names.

Think of it as your personal AI accountant that never sleeps! 🤖💰

Key Features:

  • 📧 Automatic Email Processing: Receives receipts via Postmark webhooks
  • 🔍 Smart PDF Detection: Handles various content types (even when PDFs are sent as application/octet-stream)
  • 🤖 AI Data Extraction: Uses Mindee AI to extract amounts, currencies, dates, and merchants
  • 💾 Intelligent Storage: Saves everything in a searchable database
  • 🛡️ Robust Error Handling: Bulletproof DateTime parsing and duplicate detection
  • 📊 REST API: Full CRUD operations and statistics endpoints

Demo

Live API Endpoints:

  • GET http://postmandevto.runasp.net/api/receipts - View all processed receipts
  • GET http://postmandevto.runasp.net/api/receipts/stats - Financial statistics
  • GET http://postmandevto.runasp.net/api/mindee-debug/config - System health check

How to Test:

  1. Send an email with a PDF receipt to any of these addresses:
    • devto_postmark_invoice@piwonka.cc (special address for dev.to community testing!)
  2. Check the processed data via the API endpoints above
  3. The system automatically extracts financial data from the PDF and responds within seconds

Sample API Response:

{
  "success": true,
  "receiptsCount": 5,
  "receipts": [
    {
      "id": 3,
      "merchant": "MTEL",
      "amount": 29.99,
      "currency": "EUR",
      "transactionDate": "2025-06-01T00:00:00Z",
      "originalEmailSubject": "WG: MTEL Monatsrechnung",
      "attachmentCount": 1,
      "hasMindeeData": true
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Code Repository

The complete source code is available on GitHub: SmartReceiptOrganizer

Key files to explore:

How I Built It

Tech Stack

  • Backend: ASP.NET Core 8 (C#)
  • Database: SQL Server with Entity Framework Core
  • Email Processing: Postmark Inbound Webhooks
  • AI: Mindee Receipt API for data extraction
  • Hosting: monsterasp.net (Free Tier)

The Journey

Challenge #1: Postmark DateTime Formats 🕒
Postmark sends dates in RFC 2822 format ("Tue, 3 Jun 2025 21:46:05 +0000"), but .NET's JsonSerializer expects ISO 8601. My solution was elegantly simple:

[JsonPropertyName("Date")]
public string? DateString { get; set; } // Parse as string first!

public void ParseDate()
{
    // Multiple format parsing with fallbacks
    var formats = new[]
    {
        "ddd, d MMM yyyy HH:mm:ss zzz",
        "ddd, dd MMM yyyy HH:mm:ss zzz",
        // ... more formats
    };
    // Robust parsing logic
}
Enter fullscreen mode Exit fullscreen mode

Challenge #2: PDF Content Type Detection 📄
Discovered that Postmark sometimes sends PDFs as application/octet-stream instead of application/pdf. Built smart detection:

private static bool IsPdfAttachment(PostmarkAttachment attachment)
{
    // Check ContentType
    if (attachment.ContentType?.Contains("pdf", StringComparison.OrdinalIgnoreCase) == true)
        return true;

    // Fallback: check filename extension
    if (!string.IsNullOrEmpty(attachment.Name) && 
        attachment.Name.ToLowerInvariant().EndsWith(".pdf"))
        return true;

    return false;
}
Enter fullscreen mode Exit fullscreen mode

Challenge #3: AI Integration 🤖
Integrating Mindee's Receipt API was surprisingly smooth. The trickiest part was handling the multipart form data properly:

using var formContent = new MultipartFormDataContent();
var fileContent = new ByteArrayContent(Convert.FromBase64String(attachment.Content));
fileContent.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
formContent.Add(fileContent, "document", attachment.Name ?? "receipt.pdf");

_httpClient.DefaultRequestHeaders.Add("Authorization", $"Token {mindeeApiKey}");
var response = await _httpClient.PostAsync(url, formContent);
Enter fullscreen mode Exit fullscreen mode

Challenge #4: Free Hosting Constraints 💰
Running on free tier means no HTTPS, but Postmark's flexible webhook configuration made this a non-issue. The HTTP endpoint works perfectly for webhook delivery!

Postmark Experience

Working with Postmark was a fantastic experience! The inbound webhook system is incredibly reliable and well-documented. What impressed me most:

  1. Flexible Webhook Configuration: Works seamlessly with HTTP endpoints (perfect for free hosting!)
  2. Consistent JSON Structure: The webhook payload is always predictable
  3. Rich Attachment Support: Base64 encoding makes file handling straightforward
  4. Excellent Documentation: Clear examples and troubleshooting guides
  5. Reliable Delivery: Never missed a webhook during development
  6. Multiple Email Address Support: Easy to set up multiple inbound addresses for different use cases

The only "gotcha" was the DateTime format difference, but that's more about different standards than a Postmark issue.

Architecture Highlights

  • Idempotent Processing: Duplicate detection prevents reprocessing the same email
  • Graceful Degradation: System works even if AI extraction fails
  • Comprehensive Logging: Detailed request tracking for debugging
  • RESTful API: Clean endpoints for integration with other systems
  • Multi-Address Support: devto_postmark_invoice@piwonka.cc are processed

Real-World Testing

The system is actively processing receipts! Here's what happens when you send a PDF receipt:

  1. Email arrives at either address
  2. Postmark webhook fires within seconds
  3. PDF is extracted and saved to database
  4. Mindee AI analyzes the receipt content
  5. Structured data (amount, merchant, date) is extracted
  6. Everything is stored and immediately queryable via API

This project showcases how Postmark's reliable email infrastructure combined with modern AI services can create powerful automation solutions. It's been an absolute joy to build!

The system is now processing real receipts and saving me hours of manual data entry.

Try it yourself by sending a PDF receipt to devto_postmark_invoice@piwonka.cc and watch the magic happen!

Built with ❤️ using Postmark, ASP.NET Core, and a healthy dose of debugging coffee ☕

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.