One of the things I've learned from running a SaaS is that a feature can be technically easy and still be a difficult production problem.
Form submissions are a good example. My product already knew the moment a submission arrived. The harder problem was deciding what deserved a business owner's attention. A genuine customer asking for a quote, a spam bot, an SEO sales pitch, and a support request could all arrive in the inbox looking identical, same subject line, same formatting, same priority: none.
I wanted the system to figure out what each submission actually was before the business owner opened the email.
So I built an AI layer that classifies every submission, assigns a priority, generates a summary, and puts that information directly into the notification email and the dashboard.
The interesting part wasn't calling an LLM. It was making the feature cheap enough to run on every single submission, fast enough that it couldn't affect form delivery, and resilient enough that an AI failure could never cause a lost lead.
Here's how I built it.
Why a simple LLM call wasn't enough
The naive version of this feature is easy to picture: submission comes in, call an LLM, wait for the response, attach the result to the notification. That works fine in a demo. It falls apart under a few real constraints:
Latency. This runs on every submission, on every form, for every account on the platform. If the LLM call sits in the request path, a slow model response means a slow form submission, and a visitor filling out a contact form has no idea or reason to care that AI is running behind the scenes. Their form just needs to submit.
Reliability. LLM APIs fail, time out, or rate-limit. If classification is a required step before a submission is accepted, then an AI outage becomes a lost lead, which is the one outcome this feature was never allowed to cause.
Cost at scale. This isn't a one-off classification task; it's every submission, indefinitely. A model chosen because it produced impressive answers in testing can turn into a real cost problem once it's running continuously in production.
Structured, consistent output. I needed a category from a fixed list, a priority level, and a summary, every time, in a format I could parse reliably. Not a flexible chat response I'd have to coerce into structure after the fact.
Each of those ruled out the "just call an LLM inline" version of the feature.
The architecture
The fix was to take the AI entirely off the critical path. The visitor's submission gets accepted and confirmed immediately. The classification happens afterward, as a background step that the request itself doesn't wait on.
┌───────────────┐
│ Form visitor │
└───────┬───────┘
│
▼
┌───────────────┐
│ Formgrid API │
└───────┬───────┘
│
persist submission
│
▼
┌───────────────┐
│ HTTP response │ ← visitor sees success instantly
└───────────────┘
│
background job
▼
┌───────────────┐
│ Gemini Flash │
└───────┬───────┘
│
classification + priority
+ summary
│
▼
┌───────────────┐
│ Update lead │
└───────┬───────┘
│
▼
┌───────────────┐
│ Notification │
│ email │
└───────────────┘
The visitor never waits on the AI step; they never even know it exists. If the classification job fails or times out, the submission has already been saved, and the business owner still gets notified, just without a category attached. The AI layer can degrade without ever taking the core product down with it.
The lead detail view for the first example below, with category, priority, and summary generated automatically.
Choosing the model
I didn't benchmark models based on which one produced the most impressive-sounding answer. I benchmarked against the actual task: a six-way classification, a priority level, and a two-to-three sentence summary, run on every submission across every account on the platform.
That reframing changed the decision. A larger, more capable model would have added cost and latency without adding accuracy that mattered for a fairly constrained classification problem. I went with Google Gemini Flash; it gave me the speed and the economics the task actually needed, while being more than capable of handling six categories and a summary consistently.
The six categories I landed on, after looking at what actually shows up in a real form inbox:
- Potential Customer: Genuine buying or engagement intent
- Likely Spam: Automated or irrelevant submissions
- Sales Pitch: Someone trying to sell to the business, not buy from it
- Support Request: An existing customer with a question or issue
- Job Application: Someone applying for a role
- General Enquiry: A real message that doesn't fit anywhere else
Each submission also gets a priority, High, Medium, or Low, based on urgency and specificity rather than category alone. A potential customer asking a vague question is Medium. A potential customer asking about pricing for 50 seats by Friday is High.
Handling failure without losing a lead
The rule I held myself to: an AI failure can never cause a lost lead. If Gemini times out, errors, or the account hits its analysis limit for the billing period, the submission still saves, the business owner still gets notified, and it arrives without a category badge instead of breaking anything downstream. The AI layer only ever adds information. It's never allowed to be a dependency the core product relies on to function.
That constraint is also why this couldn't be an inline, request-blocking call in the first place. Once the submission is treated as complete the moment it's persisted, an AI failure downstream literally cannot touch it.
What the classification actually looks like
Abstract categories are easier to trust with real examples. Here's what four different submissions to the same contact form produce:
| Submission | Category | Priority | AI summary |
|---|---|---|---|
| "Hi, we're a 40-person agency looking for a form tool with an API. Can you send pricing for the Business plan? We'd want to migrate by the end of the month." | Potential Customer | High | Agency evaluating Business plan for a 40-person team, wants to migrate by end of month, time-sensitive purchase intent. |
| "I'm an existing customer, my Google Sheets sync stopped working yesterday, can someone help?" | Support Request | High | Existing customer reporting broken Google Sheets sync since yesterday, needs troubleshooting. |
| "Do you offer a student discount?" | General Enquiry | Low | Prospective user asking about pricing discounts, no urgency indicated. |
| "hey i can get you to page 1 of google in 30 days guaranteed, reply for more info" | Sales Pitch | Low | Unsolicited SEO service pitch, not a product enquiry. |
| "asdkjf83 !!! CLICK HERE now4free-followers.xyz buy cheap instagram followers 100% guaranteed !!!" | Likely Spam | Low | Automated or bot-generated submission with no coherent message, promotional link pattern typical of spam. |
Same form, same inbox, five completely different submissions, and now five completely different subject lines instead of one identical "New submission" notification for all of them. That's the whole point of the feature: the business owner never has to open the spam or the SEO pitch to know they don't matter, and never risks missing the agency lead because it's buried between them.
What I haven't measured yet
I'd like to be honest about where the data stands rather than make this sound more finished than it is: I don't yet have hard numbers on what percentage of submissions get classified as spam versus genuine leads across accounts, or the exact inference cost per 1,000 submissions at current volume. I'm instrumenting both now, cost per classification and category distribution across real accounts, and I plan to publish those numbers once I have a few weeks of real data behind them rather than guess at them here.
Where it shows up
Once a submission is classified, the result appears in three places:
-
The email subject line, as a prefix before the form name,
[Hot Lead],[Likely Spam],[Sales Pitch],[Support Request], so it's readable without opening the email at all. - The email body, with the priority, category, and full summary.
- The dashboard, as a colored badge on every lead row, with a filter to pull up just the high-priority leads or exclude spam entirely.
The clearest way to see it in one place is the leads list itself, all five example submissions from the table above, each carrying its own category badge:
Caption: All example submissions in the leads list, filterable by category.
The form tool behind this
The product this shipped on is Formgrid, an open-source form backend and lead management platform I've been building and running on my own since September 2025, alongside a full-time contract role. Every form submission on Formgrid becomes a tracked lead automatically, and this AI layer, which I've been calling AI Smart Inbox, is the newest piece of that pipeline.
If you want to see it running on a real submission, every account gets 10 free AI analyses to try it with no setup required.
The broader lesson, for me at least: the hard part of shipping AI in a real product usually isn't the model call. It's everywhere around it, keeping it off the critical path, deciding what happens when it fails, and picking a model sized to the actual task instead of the most impressive one available.
Top comments (2)
Your approach to decouple the AI classification from the submission process is commendable, especially considering the constraints of latency and reliability. By offloading the classification to a background job, you've ensured that user experience remains smooth while still leveraging AI for prioritization. I wonder if caching strategies could further optimize the classification speed for repeated submissions, especially for common queries. If you're looking for help enhancing this part of the project or exploring additional optimizations, I'd be happy to discuss a paid collaboration.
This is a really sensible way to add AI to a production workflow. I like that the design starts with the invariant that a submission must never depend on the model succeeding.
Taking classification out of the request path is the biggest win here. Persist first, respond, then enrich the lead asynchronously. That keeps the user-facing SLA independent from Gemini’s latency or availability.
I also think choosing the model based on the actual classification task rather than benchmark hype is the right approach. With only six categories and a constrained output, a fast model is much easier to justify economically.
The next thing I’d be interested in seeing is how you measure classification quality over time. A simple human-correction loop could be useful: when someone changes an AI category or priority, that becomes a labeled example for evaluating prompts/models and detecting drift.
The architecture is already clean; adding that feedback/evaluation layer could make the Smart Inbox considerably more valuable as the dataset grows.