Turn 'Hello' Emails into Beautiful Artist Profiles with AI (and Postmark!)

#devchallenge #postmarkchallenge #webdev #api

This is a submission for the Postmark Challenge: Inbox Innovators.

What I Built

I built what I'm calling the "Artist Draft" feature for our online art marketplace (Otterkin). This allows interested prospective artists to easily create a stunning interactive draft profile so they can get excited about signing up when we launch.

Our goal at Otterkin is to build a vibrant community of talented artists and we realised that exchanging email addresses is the most natural way for artists to connect with us. So, we leveraged this interaction to create a more engaging onboarding experience. When an artist sends an introductory email to introduction@inbound.otterkin.co.uk, our system springs into action.

It all begins when Postmark delivers the email contents to our Rails Action Mailbox webhook. From there, a new ArtistDraft model is created. Crucially, we use Google's Gemini AI and its structured output feature to intelligently extract relevant information from the email into JSON and populate the required ArtistDraft fields. Given it's meant to be a draft, the LLM is instructed to make up reasonable sounding guesses for anything not obvious in the email, and so this is a perfect use case for generative AI!

This ArtistDraft then powers a dynamic, glossy "fake" Artist Profile page - think an Airbnb listing but for artists. A link to this page is sent back to the artist so they can instantly visualize what their public profile on the Otterkin platform could look like, making the onboarding process much more tangible and exciting than just mockups or screenshots. At the moment the main call to action is for the artist to join our newsletter or "waitlist", but after launch this could easily be the start of the fully fledged onboarding process to making a real artist profile.

Demo

Try sending an email to introduction@inbound.otterkin.co.uk! I've had far too much fun coming up with the most unhinged artist introductions imaginable and seeing the end result cracks me up. Unless you are serious artist with serious artist thoughts and serious artist places to be, I'd so encourage you to send something slightly bizarre. Here's an example of something I concocted earlier:

I hope you have as much fun as I do! There's definitely scope for ratcheting up the silliness by using Gemini to create some fake artwork to put in the gallery but I've not implemented that yet to save on API costs. I wonder what my email inspired art (from the screenshots above) would look like? Sadly we may never know.

Code Repository

I have forked and gutted our rails app leaving not much beyond anything relevant to our inbound feature Here, the README should help you take a tour of the highlights!

How I Built It

The app is built with Rails which is always such a treat to work with. The steps I took (roughly in order) to build this were:

Implement and write tests for the GeminiClient, which had to be handmade as I couldn't find a good library that supported structured output.
Make all the ArtistDraft resource stuff (controllers, model, view etc) where the model is responsible for calling the Gemini client (and handling all the errors my word).
Writing the ArtistDraftMailer and making the Postmark templates I needed.
Writing the IntroductionsMailbox which handles the incoming email from Postmark.

Originally, as I was learning how the inbound email API worked, I didn't know about Rails' Action Mailbox. I used a normal #create action on my ArtistDraftsController to use as a webhook and I set about trying to make it work. I spent ages creating mock json files for various types of inbound email to use in testing then wrote a fair amount of code for parsing it properly. This all felt like a giant faff. I then read in the Postmark docs about how it would be a good idea to implement basic auth for the webhook and I did a bit of basic googling about how I'd implement that when, that fateful evening, I discovered the ActionMailbox!

I'd been quietly worried about what would happen if I wanted to add another inbound feature and how I'd route different emails around my app with my existing controller-based approach and so to find out that was a solved problem felt incredible. I may have wept. Scrolling through the Action Mailbox Guide was another incredible experience when I saw that Postmark integration was already built and wonderful and so all that frustration of my previous integration was soon overcome by Rails-y joy and it only took me an evening to replace everything with a mailbox. Behold:

class IntroductionsMailbox < ApplicationMailbox
  def process
    sender_email = mail.from.first
    text_body = extract_email_body

    return if handle_existing(sender_email)

    @artist_draft = ArtistDraft.new(email_address: sender_email)
    @artist_draft.extract_from_email_data(text_body: text_body, subject: mail.subject)

    return notify_gemini_error(sender_email) if gemini_processing_failed?

    return notify_not_intro(sender_email) unless @artist_draft.introduction

    attach_artworks_to_draft

    if @artist_draft.save
      ArtistDraftMailer.with(artist_draft: @artist_draft).notify.deliver_later
    else
      logger.error "Failed to create ArtistDraft for #{sender_email}. Errors: #{@artist_draft.errors.full_messages.to_sentence}"
    end
  end

  private

  ...
end