Since 2021, our team at Autor has shipped over 50 AI products across healthcare, fintech, logistics, and SaaS. Some of them are running in production right now, handling thousands of automated calls per month. Others failed spectacularly — and those are the ones that taught us the most.
Where This Comes From
I started Autor in Toronto as a one-person AI development shop. The original thesis was simple: companies needed custom AI but couldn't hire fast enough to build it themselves. Four years and 50+ products later, we're a senior-only studio with a production voice AI platform (Loquent) serving healthcare and dental clients 24/7. Along the way, we've impacted over 5 million users, helped clients raise more than $10 million in funding, and shipped across 10+ countries.
This isn't a highlight reel. This is the unvarnished list of things I got wrong, figured out the hard way, or wish someone had told me before I wrote my first line of production AI code.
1. Your First AI Product Should Be Boring
Our first few products were ambitious. Multi-modal pipelines, complex reasoning chains, novel architectures. Most of them took twice as long as estimated and required constant babysitting in production.
The products that actually made money and kept clients happy? A straightforward document classifier. A simple intent router. A basic FAQ bot with good fallback logic.
I used to think "boring" meant "not innovative." Now I know boring means "reliable enough that I don't get paged at 3am." Our most successful product, Loquent, handles healthcare scheduling calls. It's not doing anything architecturally exotic. It picks up the phone, understands what the caller needs, books or reschedules an appointment, and hangs up. The magic isn't in the model — it's in the 200+ edge cases we've handled around it.
If you're building your first AI product, pick the most boring version of your idea and ship that. You can add complexity later. You cannot add reliability later.
2. Prompt Engineering Is Real Engineering
Early on, I treated prompts like configuration — something you'd tweak until it worked, then move on. That was a $40,000 mistake across three client projects.
Prompts in production need version control, testing, rollback plans, and monitoring. We learned this when a Claude model update changed how our voice AI interpreted scheduling instructions. Calls that used to resolve correctly started booking appointments on wrong days. The model hadn't broken — our prompts were too tightly coupled to specific model behaviors instead of being robust against model variations.
Today, every prompt at Autor goes through the same review process as application code. We maintain prompt test suites with 50+ test cases per critical path. When we onboard a new healthcare client, we run their specific terminology through our prompt regression suite before going live.
The companies that treat prompt engineering as "just writing" are the ones calling us at midnight asking why their AI started hallucinating appointment times.
3. The Model Is 20% of Your Production System
This is the hardest lesson for technical founders to internalize. When I started, I spent 80% of my time on model selection, fine-tuning, and evaluation. That ratio was exactly backwards.
In a production AI system, the model is a component. The other 80% is:
Monitoring and observability — knowing when something goes wrong before your users tell you. We log every Loquent call: latency per turn, confidence scores, fallback triggers, transfer reasons. That's how we discovered that 18% of calls were being transferred to humans and could systematically fix the common failure patterns.
Integration plumbing — connecting to Twilio for telephony, to clinic management systems for real-time appointment data, to HubSpot and Salesforce for CRM updates. Each integration has its own failure modes, retry logic, and authentication quirks.
Edge case handling — what happens when the caller speaks a language the system wasn't configured for? When the clinic's API is down? When someone calls to report a medical emergency? These aren't model problems. They're systems engineering problems.
Deployment infrastructure — zero-downtime deployments, canary releases, A/B testing different prompt versions. We use Docker on AWS, and the CI/CD pipeline for Loquent is more complex than the AI logic itself.
If you're spending more than 20% of your time on the model, you're probably not ready for production.
4. Ship Weekly or You'll Ship Never
At product number 15 or so, I noticed a pattern. Projects with two-week sprint cycles shipped. Projects with "we'll ship when it's ready" timelines didn't.
The reason is subtle. AI systems have a special failure mode where you can always make them "a little better." Run one more evaluation. Try one more prompt variant. Fine-tune on one more dataset. Each improvement is real but marginal, and the compound effect is that you never ship.
We now enforce a hard rule at Autor: something goes to a real user every week. Not a demo. Not a staging environment. A real user, using it for a real task. Loquent was built in 8 weeks, and it was talking to real callers by week 3. Those early calls were rough — the AI mispronounced the clinic name, got confused by call-waiting beeps, and once tried to book an appointment at 2am. But each of those failures generated specific, actionable fixes that no amount of internal testing would have caught.
The team that ships a mediocre v1 in week 3 will have a solid v4 by week 12. The team perfecting their v1 will still be in staging.
5. Charge What Production AI Actually Costs
Our first three clients got significant discounts because I was terrified of sticker shock. The result: we couldn't afford to invest in proper monitoring, the systems degraded, and two of those three clients churned anyway.
Production AI isn't a commodity. The model API costs are the smallest line item. The real costs are: senior engineers maintaining the system (we're a senior-only team for a reason), infrastructure running 24/7, ongoing prompt optimization as models update, and being available when something breaks at 3am on a Saturday.
We charge $150/hour and we're transparent about why. A junior team charging $60/hour will cost you more in the long run because they'll build something that works in the demo and breaks in production. We know because we've been hired to replace those systems more than once.
The clients who balk at $150/hour aren't our clients. The ones who've had a production AI fail on them understand exactly why it costs what it does.
6. Pick Your Stack and Commit
Over 50 products, we've tried almost everything. LangChain, LlamaIndex, raw API calls, various vector databases, different embedding models, multiple TTS and STT providers. Here's what we actually use in production today:
OpenAI and Anthropic Claude for language models — we run both because each has strengths for different use cases. Deepgram for speech-to-text because their latency is unmatched for real-time voice. ElevenLabs for text-to-speech. Twilio for telephony. Pinecone for vector search when we need it (which is less often than you'd think — RAG is genuinely overkill for most voice AI). PostgreSQL with Prisma, not some exotic vector-native database. Next.js and NestJS for the application layers. AWS and Docker for infrastructure.
We stopped experimenting with our production stack around product number 30. Every new tool has a learning curve, integration cost, and risk of undiscovered failure modes. The marginal improvement from switching to the latest vector database isn't worth the three weeks of integration work and six months of discovering its production quirks.
Pick tools that are boring, well-documented, and have active communities. Your future self will thank you.
Key Takeaways
Start boring, add complexity later. The most successful AI products solve mundane problems reliably. Pick the simplest version of your idea and ship it.
Treat prompts as production code. Version control, test suites, regression testing, and monitoring. Prompts are not configuration — they're the core logic of your system.
Spend 80% of your time on everything except the model. Monitoring, integrations, edge cases, and deployment infrastructure are where production AI lives or dies.
Enforce weekly shipping to real users. AI projects have a unique "just a little better" trap. The antidote is putting imperfect work in front of real users every week.
Charge what production AI costs, and be transparent about it. Underpricing leads to underinvestment, degradation, and churn. The clients who understand this are the ones worth keeping.
If you're building something similar, we'd love to hear about it. Reach out at hello@autor.ca or visit autor.ca.
Top comments (0)