In August 2019, a Wall Street Journal reporter asked an uncomfortable question about a London startup called Engineer.ai, and it took the market six more years and $445 million of investor capital to hear the answer. The company, later renamed Builder.ai, finally collapsed into bankruptcy in 2025 after auditors found its revenue inflated by roughly 300 percent, and a detailed postmortem published as The Credibility Tax pairs that collapse with a fresher one — Cluely, whose 21-year-old founder admitted in March 2026 that the $7 million ARR figure he'd bragged about on the TechCrunch Disrupt stage was, in his own words, blatantly dishonest. Venture capitalists lost money on these companies. But engineers lost something harder to recover: months of integration work, migration plans, and production systems built on vendors that were quietly hollow. This article is about the audit you can run yourself, from the outside, before you sign anything.
The Anatomy of a Hollow Vendor
Builder.ai is worth dissecting because it wasn't subtle — it was systematic. The company sold an AI assistant named Natasha that supposedly assembled software the way a kitchen assembles pizza. Behind the interface, roughly 700 engineers in India wrote the code by hand, timed their updates to UK business hours, and were reportedly instructed to avoid phrasing that might reveal a human was typing. Bloomberg's deeply reported feature on how a $1.5 billion unicorn unraveled traces how the fiction survived Microsoft's investment, a Qatar sovereign wealth fund round, and years of press coverage — right up until a lender seized $37 million from the company's accounts and the whole structure buckled in weeks.
Here is the detail every engineer should sit with: the deception was invisible in the demo and obvious in the telemetry. Human-written code arrives at human speed. It carries human working hours, human sick days, human timezone signatures. Anyone who had instrumented Builder.ai's delivery latency against its "AI generates 80% of your app" claim would have seen the impossibility in a spreadsheet. Nobody was looking at latency. Everybody was looking at the deck.
Regulators Caught Up. Your Procurement Process Should Too
The legal system has now given this pattern a name. In March 2024, the U.S. Securities and Exchange Commission brought its first enforcement actions over what it calls "AI washing," announcing charges against two investment advisers that had claimed machine-learning capabilities they simply did not possess — one had marketed itself as the "first regulated AI financial advisor" while running nothing of the sort. The penalties were modest, but the precedent matters: misrepresenting AI capability is no longer a marketing foul, it's securities fraud. The Builder.ai case escalated the stakes further, drawing a federal investigation out of the Southern District of New York.
For engineering teams, the practical takeaway is that regulators are running the same audit you can run — they're just running it after the collapse. You get to run it before.
The Outside-In Audit
None of the following requires NDA access, a data room, or the vendor's cooperation. It requires an afternoon and a healthy disregard for adjectives.
- Latency forensics. Ask for a live task during the call, not a recorded demo, and time it. Then ask for the same task at 3 a.m. your vendor's local time. Genuine model inference doesn't sleep; a mechanical turk does. Variance between those two runs is your single highest-signal data point.
- The determinism probe. Submit the identical prompt or job twice. Real systems produce characteristically similar outputs with model-shaped variation. Two "AI-generated" deliverables with completely different structure, style, and error patterns suggest two different humans.
- Headcount arithmetic. Pull the vendor's job postings and public team size. A company claiming full automation while hiring hundreds of "delivery specialists," "productologists," or "solution engineers" in low-cost regions is telling you its unit economics in public. Builder.ai's job boards were legible for years.
- Restated numbers and quiet edits. Search for revenue figures the company has publicly walked back, changelog entries that vanished, or benchmark pages edited after criticism. One restatement is an accident; a pattern is a business model.
- Failure-mode candor. Ask the sales engineer, on the record, what the product is bad at. A real engineering organization answers instantly, because its team lives inside those limitations daily. A narrative-first organization stalls, deflects, or promises a roadmap.
Why This Is Now a Core Engineering Skill
There's a tempting way to read the Cluely and Builder.ai stories: founders lied, investors got burned, nothing to do with us. That reading misses where the damage actually landed. Builder.ai's bankruptcy stranded paying customers mid-project, left apps without a maintainer, and, according to court filings, left tens of millions in unpaid cloud bills to Amazon and Microsoft — infrastructure debt that translated directly into service instability for the people who had built on top. When a hollow vendor dies, the outage happens in your stack.
The skill of technically auditing a vendor's claims used to belong to due-diligence consultants and short sellers. In a market where the marginal cost of a confident claim has fallen to zero, it now belongs to whoever is about to type npm install. The engineers who ran informal versions of the audit above — and there were some, posting their skepticism in forums years before the collapses — were dismissed as cynics. They turned out to be the only people in the room doing engineering.
The loudest companies of this cycle are being repriced by exactly this kind of scrutiny, one seized bank account at a time. Run the audit early, write down what you find, and let the vendors who can't survive an afternoon of measurement select themselves out of your architecture. The quiet ones who pass it are the ones still standing when the grand jury convenes for everyone else.
Top comments (0)