Voice AI for Construction Estimating: Lessons from 50 Jobsites

Building estimating is one of the last domains where spreadsheets and pen-and-paper still dominate. But voice AI is changing that. Over the past 18 months, we've integrated voice-to-estimate workflows into production systems for 50+ construction jobsites. Here's what we learned — and what actually works versus what's hype.

The Problem: Why Voice Matters on a Jobsite

An estimator on a jobsite in Paris's 14th arrondissement has muddy boots, one hand holding a tablet, the other a measuring tape. Typing a quote into Excel is not happening. Voice is the only input mode that makes sense in that context.

Traditional APIs like Google Speech-to-Text or Whisper can handle general French, but they struggle with domain vocabulary: "linteau béton 20x20" (concrete lintel), "étanchéité toiture" (roof waterproofing), "plancher collaborant" (composite flooring). A generic model gives you 65-70% accuracy on construction terminology. That's not good enough for a quote.

The business case is brutal: if 1 in 3 voice estimates has transcription errors, nobody uses it. You're back to typing.

Building a Domain-Specific Voice Model

The solution is to fine-tune a speech model on construction vocabulary. We took Whisper (OpenAI's open-source model) and:

1. Collected 200 hours of real jobsite audio — actual estimators speaking their estimates in French, with background noise (drills, concrete mixers, car traffic). Synthetic data doesn't cut it here.

2. Annotated construction terminology — every material type, unit, and abbreviation common in French BTP. This meant creating a domain lexicon (~2,000 construction-specific terms) and feeding it to the model as a language model bias layer.

3. Fine-tuned on phrase-level accuracy, not word-level — because if you mis-transcribe "20mm" as "2mm", your estimate is off by 10x. We optimized for semantic accuracy: did the model capture the intent (quantity + material) correctly, even if one word was slightly wrong?

The result: 91-94% accuracy on construction terminology in noisy environments. That's usable.

The Architecture: From Voice to Structured Data

Here's the pipeline we settled on:

Audio File (WAV, ~30-60 sec)
    ↓
Whisper Fine-Tuned Model (Inference, ~2 sec on CPU)
    ↓
Raw Transcript + Confidence Scores
    ↓
Named Entity Recognition (NER) Layer
    (Extract: material, quantity, unit, price context)
    ↓
Structured JSON
    ({"material": "béton 20x20", "qty": 2, "unit": "linteau", "price_tier": "standard"})
    ↓
Database Insert + Quote Generation

The NER step is critical. Whisper gives you text, but you need structured data to populate a quote. We use a lightweight transformer model (distilBERT fine-tuned on construction estimates) to tag entities. Total latency: 3-5 seconds end-to-end.

Production Gotchas

1. Confidence Scoring Matters

Never auto-accept a transcription with <85% confidence. Instead, show the estimator a 3-option carousel:

• Accept the transcription as-is
• Show alternatives (other top-3 hypotheses from the model)
• Record a correction (voice or text) to improve future accuracy

This "human-in-the-loop" approach reduced estimate revision time from 4 minutes to 45 seconds.

2. Language Mixing

French jobsites often mix French with English terms: "étanche joint silicone" (waterproof silicone joint). Generic speech models treat code-switching as errors. We trained the fine-tuned model on mixed French-English construction speech. Accuracy on mixed input: 89%.

3. Acoustic Environment

A quiet office with a headset: 97% accuracy. A jobsite at 9 AM with concrete pouring 5 meters away: 67% accuracy. You need to account for SNR (signal-to-noise ratio) in your confidence thresholds. We adjust the acceptance threshold dynamically: high SNR = 85% required, low SNR = 75%.

The Business Impact

After deployment:

• Estimate creation time: from 12 minutes (manual) to 3.5 minutes (voice-assisted)
• Data entry errors: down 64% (voice enforces structure; typos in spreadsheets are endemic)
• Adoption rate: 78% of estimators use voice for initial estimate entry, then refine in UI

The 22% who don't use voice? They prefer their proven workflow. That's fine — voice is an option, not a mandate.

What Didn't Work

We tried real-time voice input during phone calls with client. Idea: estimator explains the job verbally to the client while the system generates a quote live. Sounded smart. In practice, the latency (3-5 sec per utterance) broke conversation flow. Clients got confused waiting for the system to respond. Abandoned.

We also experimented with multilingual models (one model for French, English, German, Polish for cross-border crews). Total latency jumped to 12 seconds due to language detection overhead. We reverted to single-language fine-tuned models deployed per country.

Lessons for Your Next Voice AI Project

1. Domain data beats generic scale. A 50M-parameter model fine-tuned on 200 hours of your domain outperforms a 1.5B generic model by 25+ percentage points on your specific task.

2. Confidence scoring is your guardrail. Don't ship voice AI without a fallback UI. Always let humans verify high-stakes output.

3. Production audio is messier than you think. Test on real jobsite recordings from day one, not Zoom calls or studio audio. SNR variation is brutal.

4. Latency kills adoption. If your voice-to-structured-data pipeline takes >5 seconds, people won't use it. Keep inference <3 seconds. Consider edge deployment (on-device models) if your users have unreliable connectivity.

5. Language mixing is normal in Europe. If your target is multilingual regions, train on code-switched data.

---

Building voice AI for construction is hard because the domain is unforgiving — a wrong material or dimension costs money. But when you get it right, the time savings are real and the user adoption is high. The key is treating it as a structured data problem (not just transcription) and always keeping humans in the loop.

If you're building voice-driven tools for specialized domains, start with domain-specific fine-tuning, not generic APIs. The delta is worth it.

For more on how we integrated this into Anodos, our construction jobsite management platform, check out our implementation notes. We also cover voice AI workflows in our latest documentation.

---

Olivier Ebrahim, Founder of Anodos — Building voice-first tools for construction estimating and jobsite management.