DEV Community: Dharmesh_bizz

Odoo 20: 7 Technical Changes Developers Should Review

Dharmesh_bizz — Tue, 06 Oct 2026 14:10:31 +0000

An Odoo upgrade is not only a version change. If an implementation includes custom modules, external integrations, automated workflows, or website customizations, developers need to understand how the new version affects the existing environment.

Odoo 20 introduces changes across AI, APIs, inventory, manufacturing, accounting, website, eCommerce, and other areas.

Rather than covering every new feature, this article focuses on the areas that developers and technical teams should review when working with Odoo 20.

## 1. AI Is Moving Closer to Actual ERP Actions

Odoo 20 expands the role of AI in business workflows.

AI agents can work with supported Odoo actions, including reading and modifying data when the relevant tools are exposed. Odoo also provides an MCP server that allows compatible external AI agents to interact with an Odoo database.

For developers, this raises a more practical question than simply asking whether Odoo has AI:

Which actions should an AI system be allowed to perform?

Odoo's MCP documentation shows that tools can be exposed through Server Actions, and tools can also be marked as read-only. This makes permissions and tool exposure an important part of the implementation.

When introducing AI into an Odoo environment, developers should therefore consider:

Which data the AI can access
Which operations it can perform
Which actions require approval
Whether tools should be read-only
How AI actions fit into existing business workflows

The technical challenge is not simply connecting an AI model to Odoo. It is controlling what that connection is allowed to do.

## 2. Review Existing External API Integrations

The Odoo 20 API changes deserve particular attention from developers maintaining integrations.

Odoo's documentation states that the XML-RPC and JSON-RPC APIs are deprecated. In Odoo 20, the legacy db service has been removed, while the common and object services remain available for now and are scheduled for removal in Odoo 22. Odoo's External JSON-2 API is the newer API developers should evaluate for future integrations.

This does not mean every existing XML-RPC or JSON-RPC integration stops working with Odoo 20.

Instead, developers should identify exactly which services their integrations use.

A useful inventory can include:

External applications
Custom Python scripts
Middleware
Scheduled synchronization jobs
Third-party connectors
Data migration scripts
Custom integrations

The JSON-2 API uses a different authentication and request structure from the older object service, so moving an integration should be treated as a technical migration rather than simply changing an endpoint.

## 3. Custom Modules Need Regression Testing

Custom modules are one of the first areas to review during a major Odoo upgrade.

A module may depend on specific models, fields, views, access rules, automated actions, scheduled jobs, or other modules.

Before moving to production, developers should identify the customizations that support critical business processes and test them in the Odoo 20 environment.

Areas worth reviewing include:

Custom models and fields
XML views
Access rights and record rules
Automated actions
Scheduled actions
Custom reports
Module dependencies
External API calls

The objective is not simply to confirm that a module installs successfully.

It is to confirm that the business process depending on that module still works as expected.

## 4. Inventory and Manufacturing Changes Can Affect Custom Logic

Odoo 20 includes changes in inventory and manufacturing workflows.

Inventory improvements include suggested reordering rules and an allocation report that helps users work with orders waiting for incoming stock.

Manufacturing also includes capabilities such as continuous production, Bill of Materials comparison, and work-order planning through Kanban and Gantt views.

For developers, these changes are relevant when custom code interacts with:

Stock availability
Reordering
Procurement
Manufacturing orders
Bills of Materials
Work orders
Product quantities

If a customization relies on a particular workflow or data structure, that dependency should be tested rather than assumed to behave exactly as before.

## 5. Localization Changes Can Affect Customizations

Indian implementations should also review the Odoo 20 localization changes.

The Odoo 20 feature set includes changes involving Bill of Entry, TDS, Composition Taxpayer workflows such as CMP-08 and GSTR-4, Schedule III financial statements, and payroll-related functionality.

For developers maintaining India-specific customizations, this creates an important review point:

Does existing custom code still need to handle a process that Odoo now supports through standard functionality?

A major version upgrade can therefore be an opportunity to review customizations instead of automatically carrying every customization forward.

The right decision depends on the actual implementation and business requirements.

## 6. Website and eCommerce Customizations Should Be Rechecked

Odoo 20 also brings changes to Website and eCommerce.

The AI Website Assistant can help with tasks such as creating pages, rewriting content, translating pages, and suggesting SEO titles and descriptions.

There are also improvements involving structured data, sitemaps, responsive images, product discovery, Click & Collect, cross-selling, customer reviews, and GA4 eCommerce tracking.

For developers working on customized Odoo websites, the useful question is:

Does a custom solution still need to exist when standard Odoo functionality has expanded?

This is particularly relevant during an upgrade because maintaining unnecessary custom code adds testing and maintenance work.

## 7. Test Complete Workflows, Not Just Individual Modules

One of the easiest mistakes during an ERP upgrade is testing applications separately without testing the complete workflow.

Consider a manufacturing business:

Quotation → Sales Order → Inventory → Procurement → Manufacturing → Delivery → Invoice → Payment

A customization may work correctly inside one application while still causing problems somewhere later in the process.

That is why upgrade testing should include complete business workflows.

A practical technical checklist could include:

Custom modules
API integrations
Automated actions
Scheduled jobs
Custom reports
Accounting workflows
Inventory workflows
Manufacturing workflows
Website and eCommerce flows
Critical user journeys

Testing should happen in a controlled environment before the production upgrade.

## A Practical Odoo 20 Developer Checklist

Before upgrading an Odoo environment, ask:

Which custom modules are business-critical?
Which external systems communicate with Odoo?
Do any integrations depend on the legacy RPC services?
Which automated and scheduled actions need regression testing?
Can any existing customization now be replaced with standard functionality?
Have critical workflows been tested end-to-end?
Has the deployment and rollback process been reviewed?

The important part of an Odoo upgrade is not simply knowing what changed in Odoo 20.

It is understanding which of those changes affect your particular implementation.

For a broader overview of Odoo 20 features, including AI, accounting, India localization, eCommerce, Field Service, API changes, and upgrade considerations, see the Odoo 20 feature guide.

If you work with custom Odoo modules or integrations, what is the first thing you check before a major Odoo upgrade?

How Real-Time Speech Translation Works: The Engineering Behind Live Conversations

Dharmesh_bizz — Wed, 30 Sep 2026 14:06:29 +0000

Real-time speech translation sounds straightforward: someone speaks in one language, and another person hears the message in a different language.

The interesting part is everything that happens between those two points.

Unlike a document, live speech arrives continuously. A system has to process audio, recognize what was said, translate it, and produce an understandable result while the conversation is still moving.

A simplified model looks like this:

Speech → Audio Processing → Speech Recognition → Translation → Output

The exact architecture can vary, but each stage can influence the quality and responsiveness of the overall experience.

Start With the Audio

Every speech translation workflow begins with an audio source, such as a microphone or another supported audio stream.

The quality of that input matters.

A person speaking in a quiet room with a good microphone provides very different input from someone speaking in a busy conference hall, a vehicle, or a room with several people talking.

Background noise, echoes, microphone quality, speaking volume, and overlapping speech can all make the incoming audio harder to process.

This means real-time translation is not only a language problem. It is also an audio-processing problem.

Turning Speech Into Something a System Can Process

The next stage commonly involves automatic speech recognition (ASR).

ASR systems process spoken audio and produce text or another representation that downstream systems can work with.

Recognition can be affected by accents, pronunciation, speaking speed, background noise, and specialized vocabulary.

Consider a technical meeting where participants use product names, abbreviations, or industry terminology. A recognition error at this stage can affect everything that follows.

That is why speech recognition deserves attention when evaluating a real-time translation workflow. Translation quality alone does not describe the complete experience.

Translation Happens Inside a Moving Conversation

Once speech has been recognized, a machine translation system can generate content in the target language.

This is different from translating a finished document.

A document provides a complete source that can be processed as a whole. During a conversation, speech arrives incrementally, and the system has to work with the information available at that point.

Context also matters.

A phrase can have different meanings depending on the subject, previous statements, terminology, and the situation in which it is being used.

For live translation, the system therefore has to balance the available context with the need to produce an output without unnecessary delay.

Latency Becomes Part of the Experience

Latency is easy to overlook when thinking about translation.

For a document, receiving the translation a little later may not matter. During a live conversation, timing can directly affect the interaction.

If a translated response arrives after the discussion has already moved to another point, users may find it harder to follow the exchange.

A real-time system therefore has to consider several factors together:

Speech recognition accuracy
Translation quality
Processing time
Audio quality
Language support
Network conditions
Output generation

There is no single setting that solves all of these problems. Improving one part of the pipeline does not automatically improve the complete experience.

Speech in the Real World Is Messy

People do not speak like perfectly prepared text.

They pause, restart sentences, change their wording, interrupt one another, use informal expressions, and refer to information that depends on the surrounding conversation.

Audio conditions can make things even more difficult.

A live system may encounter:

Background noise
Multiple speakers
Different accents
Overlapping speech
Poor microphone placement
Unclear audio
Specialized terminology

These factors can influence recognition and, consequently, the translated result.

For that reason, testing a real-time speech translation system in a controlled environment may not tell the whole story. The conditions in which people actually communicate matter.

Where Does Real-Time Translation Make Sense?

The technology is most relevant when people need to understand spoken communication while an interaction is happening.

An international meeting is a simple example.

A translated agenda or presentation can help participants prepare before the meeting. But it does not solve the problem when someone asks an unexpected question or introduces a new topic during the discussion.

Similar situations include:

Multilingual business meetings
International conferences
Customer interactions
Training sessions
Travel and hospitality
Cross-border collaboration
On-site communication
Multilingual events

The requirement in these situations is different from translating a piece of finished content.

The goal is to help participants understand one another while the conversation continues.

Why Real-Time Translation Does Not Replace Prepared Translation

It is easy to think of real-time translation as simply a faster version of traditional translation.

That is not quite the right way to look at it.

Prepared translation remains useful when the final content needs review, consistency, formatting, or approval. Policies, reports, product documentation, and other formal materials can benefit from a controlled workflow.

Real-time translation addresses another problem: supporting communication while people are speaking.

The choice between manual and real-time translation therefore depends on the communication workflow and what the translated content needs to accomplish.

In practice, organizations may use both. Prepared translation can support content that needs to be reviewed and retained, while real-time translation can support meetings, conversations, events, and other live interactions.

What Developers Should Think About

From an engineering perspective, real-time speech translation is not a single feature. It is a chain of connected processing stages.

A simplified flow is:

Audio → Speech Recognition → Translation → Output

The implementation behind that flow can differ considerably between systems.

Developers may need to consider:

How audio is captured and processed
How speech recognition handles imperfect input
How translation is performed
How intermediate results are handled
How errors are managed
How latency affects interaction
How the system behaves under different network conditions
Where processing takes place

Deployment and privacy requirements can also influence architectural decisions, particularly when an organization needs greater control over its processing environment.

The important point is that optimizing one component in isolation does not necessarily produce a better end-to-end experience.

From Translation Software to a Communication Layer

This is what makes real-time speech translation an interesting engineering problem.

A conventional translation workflow generally starts with existing content and produces a translated version.

A live speech translation workflow operates during the communication itself.

The outcome is therefore not only a translated sentence. The system also needs to help people follow the conversation and continue the interaction.

That makes factors such as timing, audio quality, recognition, translation, and output part of the same user experience.

Final Thought

Real-time speech translation brings together several areas of technology, including audio processing, automatic speech recognition, machine translation, and, in speech-to-speech systems, speech synthesis.

The individual components matter, but the overall experience depends on how well they work together under real communication conditions.

For developers and businesses evaluating this technology, a useful question is not only:

"How accurate is the translation?"

It is also:

"Does the system provide useful translated communication while the conversation is taking place?"

That is the engineering challenge behind making speech translation useful in real-world conversations.

Building Real-Time Audio Translation: What Happens Between Speech and Translation

Dharmesh_bizz — Thu, 24 Sep 2026 13:45:28 +0000

Real-time translation looks simple from the outside.

Someone speaks in one language, and another person hears the same message in a different language a moment later.

But if you've ever worked with streaming audio, you know the difficult part isn't simply calling a translation model.

The real challenge is keeping the entire pipeline responsive while speech is still arriving.

A typical real-time speech translation pipeline looks something like this:

Live Audio
↓
Audio Capture
↓
Speech Recognition
↓
Translation
↓
Speech Synthesis
↓
Translated Audio

Every stage affects the experience.

Real-Time Translation Is a Streaming Problem

The first important distinction is between batch processing and streaming.

With a recorded file, you can wait until the entire audio is available before processing it.

Live audio doesn't give you that luxury.

A speaker may still be talking when your system needs to decide whether it has enough information to process the current segment.

That creates a constant trade-off:

Process earlier → lower latency, less context

Wait longer → more context, higher latency

This trade-off is one of the reasons real-time translation is harder than simply combining speech recognition, translation, and text-to-speech APIs.

1. Capture and Stream the Audio

Everything starts with the audio source.

It might be:

A microphone
A video call
Browser or tab audio
A live stream
A presentation
Another supported audio stream

The application needs to capture that audio continuously and move it through the pipeline without introducing unnecessary buffering.

For developers, this is where the architecture begins to matter.

Audio chunk size, transport method, buffering, network conditions, and playback strategy can all influence perceived latency.

Recent real-time translation implementations commonly use streaming connections and carefully chosen audio frame sizes for this reason.

2. Speech Recognition Has to Work Incrementally

The next stage is automatic speech recognition (ASR):

Audio → ASR → Partial/Final Transcript

With live speech, the transcript may change while the speaker continues talking.

For example, the recognizer might initially hear:

"We need to review the..."

A moment later, it becomes:

"We need to review the production schedule."

The translation system therefore needs to handle partial results without producing a confusing stream of constantly changing output.

This is one reason when to finalize a speech segment becomes an important engineering decision.

Voice activity detection, punctuation, pauses, and streaming ASR can all play a role.

3. Translation Needs Context

Once speech has been recognized, the text can be translated.

But sending every tiny fragment directly to a translation model isn't necessarily the best approach.

A fragment may not contain enough context to determine the intended meaning.

Imagine receiving:

"We're going to..."

There isn't much to translate yet.

Waiting for:

"We're going to move the deployment to Friday."

provides much more context—but also adds delay.

So the system needs a sensible segmentation strategy.

This is where latency, stability, and translation quality start competing with each other. Developers building real-time translation systems have reported this trade-off as one of the central challenges of the architecture.

4. Speech Synthesis Creates Another Challenge

If the application only needs translated text, the pipeline can stop here.

For speech-to-speech translation, however, the translated text needs to become audio:

Translated Text
↓
TTS
↓
Translated Audio

This sounds straightforward until you start streaming the result.

If you generate speech in many small chunks, the boundaries between those chunks can become audible.

One recent DEV case study found that small gaps between synthesized audio chunks were enough to make otherwise correct translation sound unnatural. The solution involved better buffering, silence handling, and text aggregation before synthesis.

That's a useful reminder:

In real-time audio, the bytes matter as much as the model.

5. Measure End-to-End Latency

A common mistake is to measure only the response time of the AI model.

The user experiences the entire chain:

Capture
+
ASR
+
Translation
+
TTS
+
Network
+
Playback

So the meaningful metric is the delay between the speaker saying something and the listener hearing the translation.

A fast translation model cannot compensate for inefficient buffering, slow audio transport, or delayed playback.

This is why real-time systems often require optimization across the entire pipeline rather than inside a single component.

Audio Quality Still Matters

Better models don't eliminate bad input.

Background noise, overlapping speakers, accents, poor microphones, and unstable audio can all make speech recognition harder.

For a production system, developers should think about:

Audio format and sample rate
Chunk size
Voice activity detection
Buffering
Network interruptions
Partial transcripts
Speaker overlap
TTS chunk boundaries
Playback underruns
Error recovery

These details may not appear in a product demo, but they can determine whether the final system feels smooth.

Where This Architecture Becomes Useful

The same basic architecture can support several applications:

Multilingual meetings — participants can follow spoken discussions across languages.

Browser audio — spoken content from supported videos, webinars, or presentations can be translated while playing.

Live events — presentations can be made more accessible to multilingual audiences.

Voice applications — developers can build multilingual assistants and communication tools.

The input and interface may change, but the underlying problem remains similar: process a continuous stream of speech while keeping the output useful and timely.

The Real Engineering Question

When building a real-time audio translator, the question isn't simply:

"Which translation model should I use?"

A better set of questions is:

Where does the audio come from?
How will it be streamed?
When is a speech segment ready to translate?
How much context should each segment contain?
How will partial results be handled?
How will translated audio be buffered?
What happens when the network becomes unstable?
What latency can users realistically tolerate?

Once you start looking at the system this way, real-time translation becomes less about one AI model and more about coordinating an entire streaming pipeline.

Final Thought

The interesting part of live audio translation isn't making each individual component work.

It's making them work together, continuously, and predictably.

A system can have excellent speech recognition and accurate translation and still feel frustrating if the output arrives too late or the generated audio sounds broken between chunks.

That's why building real-time speech translation is ultimately a systems problem involving audio processing, streaming infrastructure, AI models, latency management, and user experience.

If you're interested in the user-facing side of this pipeline, we've also put together a practical guide covering how to translate live audio from meetings, videos, webinars, browser audio, and other sources: How to Translate Live Audio in Real Time.

What would you optimize first in a real-time translation system: ASR, translation, TTS, or the streaming layer?

Your Browser Translates Text. What Happens to the Audio?

Dharmesh_bizz — Mon, 21 Sep 2026 13:28:56 +0000

Modern web applications are no longer just collections of HTML and text.

A single browser tab can contain a video, webinar, online class, meeting, presentation, or live stream. The page itself might be translated into your language, while the person speaking in the video continues speaking another language.

That raises an interesting technical question:

What happens when the information you need isn't text at all?

This is where the difference between browser translation and real-time audio translation becomes important.

Browser Translation Has a Clear Input

Traditional browser translation starts with text.

The browser identifies written content on a webpage and sends it through a translation process before displaying the translated result.

Conceptually:

Webpage text
↓
Translation
↓
Translated text

This works well for articles, documentation, product pages, navigation, and other text-based content.

But a browser page can contain information that isn't represented in its text layer.

Consider a webinar page. The HTML might contain the event title, description, and a few headings. The speaker's explanation, however, exists in the audio stream.

Translating the HTML doesn't translate the speaker.

Real-Time Audio Translation Starts Somewhere Else

Real-time audio translation works with spoken language instead of already-written text.

A typical pipeline looks like:

Audio source
↓
Speech recognition
↓
Translation
↓
Speech synthesis
↓
Translated audio

The first challenge is therefore speech recognition.

The system needs to determine what was said before it can translate it. That introduces additional processing compared with translating text that already exists.

The system then needs to produce the translated result quickly enough to remain useful while the original audio is still playing.

Why Latency Becomes a System Problem

For normal text translation, a few extra seconds may not matter much.

For live audio, they can change the entire experience.

Imagine a meeting where every translated sentence arrives several seconds after the speaker finishes. Participants may start waiting for translations, talking over each other, or losing track of the discussion.

Latency can come from multiple stages:

Audio capture
Speech recognition
Translation
Audio generation
Network communication
Playback

Reducing latency isn't simply about making one component faster. The whole pipeline has to work efficiently.

There is also a trade-off between context and responsiveness.

Processing a longer section of speech can provide more context for translation, but it may increase the delay. Processing smaller segments can reduce waiting time but may provide less context.

For real-time systems, that trade-off matters.

Browser Audio Is Different From Browser Text

This distinction becomes particularly useful when building browser-based applications.

A webpage may expose text through the DOM, but the audio playing inside the page is a different data source.

For example:

Webpage
├── HTML / visible text
└── Audio / video stream

A text translation feature operates on the first layer.

A browser audio translation workflow needs access to the second.

That means browser-tab audio capture, audio routing, speech recognition, translation, and playback can all become part of the architecture.

This is why translating a webpage and translating a presentation playing inside that webpage are two different engineering problems.

Where Does Real-Time Audio Translation Fit?

The use case isn't limited to two people having a conversation.

It can apply to browser-based meetings, webinars, training sessions, presentations, educational content, videos, and live streams.

The common factor is that the information is being delivered through speech.

For example, a developer attending a technical webinar in another language may be able to translate the webpage interface easily. But the actual technical explanation may only exist in the speaker's voice.

The translation problem has therefore moved from document processing to live audio processing.

What Developers Need to Consider

Building or integrating real-time audio translation involves more than choosing a translation model.

Audio Input

Where does the audio come from?

It could be a microphone, browser tab, media stream, meeting application, or another application.

The input method affects what the system can actually translate.

Speech Recognition

The system needs reliable speech recognition before translation can happen.

Accents, background noise, overlapping speakers, and poor microphones can all affect the recognized output.

Translation

Translation quality depends on language pairs, context, terminology, and the underlying translation technology.

Technical content can be particularly challenging because a small terminology error can change the meaning of an explanation.

Speech Generation

If the final output needs to be spoken rather than displayed as text, translated speech has to be generated and delivered quickly enough for the use case.

Privacy and Deployment

Audio can contain meetings, customer conversations, internal discussions, or other sensitive information.

That makes the architecture and data flow important considerations. Developers may need to understand where audio is processed, what external services receive it, whether data is retained, and whether a self-hosted deployment is required.

A Simple Architecture

At a high level, a real-time speech translation system can be thought of as a streaming pipeline:

Audio Input
↓
Audio Processing
↓
Speech Recognition
↓
Translation
↓
Speech Synthesis
↓
Audio Output

The interesting engineering work happens between these stages.

How much audio should be buffered?

When should a segment be considered complete?

How much context should be passed to translation?

How should partial results be handled?

What happens when the network becomes unstable?

These questions become increasingly important as the system moves from a demonstration to real-world usage.

The Key Difference

Browser translation and real-time audio translation solve different problems.

If the information exists as text on a webpage, text translation is usually enough.

If the important information is being spoken during a meeting, presentation, webinar, video, or live stream, the system needs to work with audio.

That distinction also changes the engineering architecture.

You're no longer translating a finished text string. You're processing a continuous stream of speech under timing constraints.

For a deeper look at the practical differences between these approaches, including browser-tab audio and live use cases, see Browser Translation vs. Real-Time Audio Translation.

The broader lesson is useful beyond translation:

When the information moves from text to a live stream, the engineering problem changes with it.

Building Real-Time Speech Translation for Customer Support

Dharmesh_bizz — Wed, 09 Sep 2026 13:34:14 +0000

Imagine debugging a customer's issue while communicating in different languages.

The technical problem might be straightforward. The difficult part is explaining it clearly, understanding the customer's response, and keeping the conversation moving.

For a global support team, this can turn a normal conversation into a transfer, a waiting period, or a search for another agent.

That raises an interesting engineering problem:

How can software translate a live conversation quickly enough that two people can keep talking naturally?

Real-time speech translation is essentially a pipeline that tries to solve that problem.

Start With the Pipeline

At a high level, the system looks something like this:

Audio → Speech Recognition → Language Detection → Translation → Speech Synthesis → Audio

It sounds straightforward.

In practice, every stage introduces its own problems.

1. Capturing the Audio

First, the system needs a reliable audio source.

For a phone conversation, this might come from a telephony system. For an online meeting, it could come from a browser or conferencing application.

The quality of this input matters.

Background noise, microphone quality, compression, overlapping speakers, and inconsistent volume can all affect the stages that follow.

Garbage in still tends to become garbage out.

2. Recognizing the Speech

The next step is speech recognition.

The system needs to determine what the speaker actually said before it can translate it.

This gets harder with:

Strong accents
Fast speech
Background noise
Technical terminology
Product names
People speaking over each other

A recognition error can propagate through the rest of the pipeline.

If the speech recognition layer gets an important technical term wrong, the translation layer may have no way of knowing that the original transcription was incorrect.

Translation Is Not Just Word Replacement

Once speech has been recognized, the system needs to translate it into the listener's language.

But customer conversations rarely consist of isolated sentences.

Consider:

"I tried that yesterday, but it still doesn't work."

What does "that" refer to?

The answer may have appeared several sentences earlier.

This is why conversational context matters.

A useful translation system may need to consider recent dialogue, terminology, session information, and other relevant context rather than treating every sentence as an independent translation request.

The challenge is finding the right amount of context.

Too little can produce vague or inconsistent translations. Too much irrelevant context can increase processing and introduce noise.

Latency Is Part of Translation Quality

For a live conversation, accuracy is only one part of the problem.

Imagine this:

Agent speaks
↓
Speech recognition
↓
Translation
↓
Speech synthesis
↓
Customer hears response

If every stage adds noticeable delay, the conversation quickly becomes awkward.

The participants start waiting for each other. People interrupt because they think the other person has finished. Responses become less natural.

This means a real-time system has to optimize the whole pipeline, not just the translation model.

Important factors include:

Recognition latency
Translation latency
Network latency
Speech synthesis latency
Audio buffering
Turn detection
Streaming behavior

A technically accurate system can still provide a poor user experience if the end-to-end delay is too high.

What About Browser Audio?

Not every support conversation happens over a traditional phone system.

Modern teams increasingly use browser-based tools for:

Customer onboarding
Product demonstrations
Troubleshooting
Technical training
Video meetings
Remote support

That makes browser audio an interesting input source.

Conceptually, the architecture can become:

Browser Tab → Shared Audio → Speech Recognition → Translation + Context → Translated Speech

The useful part is that the original application does not necessarily need to provide its own translation feature.

The audio already being played in the browser can become the input to another processing pipeline.

That opens up interesting possibilities beyond customer support, including multilingual training sessions, online courses, webinars, and technical demonstrations.

What Happens When the Customer Responds?

The pipeline has to work in both directions.

Agent Language → Translation → Customer Language

Customer Language → Translation → Agent Language

That creates another engineering challenge: turn-taking.

The system needs to distinguish between useful speech and things like background noise, short interruptions, or overlapping speakers.

In a real conversation, people do not always wait politely for one person to finish before responding.

A production system therefore needs to think about conversational behavior, not just individual audio segments.

Where Human Support Still Matters

Real-time speech translation does not eliminate the need for human expertise.

Some conversations require specialist knowledge, cultural understanding, or human judgment.

A technical support engineer still needs to understand the product.

An interpreter may still be the right choice for high-stakes conversations.

Translation solves a language problem. It does not automatically solve the underlying business, technical, or human problem.

That distinction is important when designing these systems.

The Interesting Engineering Problem

The most interesting part of real-time speech translation is not simply translating English into another language.

It is making the entire interaction work under real-world constraints.

You need to deal with:

Audio → Recognition → Context → Translation → Synthesis → Delivery

while keeping the system responsive enough for people to continue talking.

That brings together several areas of engineering:

Speech recognition
Machine translation
Audio processing
AI inference
Streaming systems
Networking
User experience

And customer support is only one application.

The same architecture can be used anywhere people need to communicate across language barriers in real time.

The bigger goal is simple:

Language should be a smaller engineering problem between two people who need to communicate—not the reason they cannot communicate at all.

How Real-Time Translation Can Make Global Education More Accessible

Dharmesh_bizz — Thu, 03 Sep 2026 12:31:06 +0000

The browser has become one of the world's biggest classrooms.

A student can attend a university lecture from another country, a developer can follow a technical workshop hosted overseas, and a researcher can watch a presentation from a team halfway around the world.

Access is no longer the biggest problem.

Understanding can be.

Language can still create friction even when the content is freely available online. For developers, this raises an interesting question:

How can audio playing in a browser be translated in real time without requiring the original platform to provide a translated version?

It turns out that the problem is less about translating a sentence and more about building a reliable streaming pipeline around speech, context, and latency.

Real-Time Translation Is More Than Text Translation

Text translation starts with something that already exists as text.

Speech is different.

It arrives continuously. Speakers pause, change direction, correct themselves, use abbreviations, and refer to things mentioned earlier. Technical and educational content makes this even harder because meaning often builds across several minutes of conversation.

Imagine an instructor saying:

"Now change this value in the configuration panel."

The sentence is easy to translate.

But which value?

That answer might depend on something the instructor explained earlier or something currently visible on screen.

This is why real-time speech translation is better treated as a streaming language-understanding problem rather than a sequence of independent translation requests.

A Simple Architecture

A browser-based translation workflow can be represented like this:

Browser Audio → Audio Capture → Speech Recognition → Language Detection → Context + Translation → Translated Text/Speech → Real-Time Delivery

Each stage solves a different problem.

Audio capture provides the spoken content being delivered through the browser.

Speech recognition converts the incoming speech into information the system can process.

Language detection identifies the source language when it isn't already known.

Context + translation combines the current speech with relevant information from the ongoing session before producing the translated result.

Real-time delivery gets that result back to the learner as text, speech, or both.

The interesting engineering challenge is keeping this pipeline moving continuously without allowing latency to become disruptive.

Why Context Matters

A translation system that only sees the current sentence can miss important relationships.

Consider a technical workshop. The instructor introduces an API, explains several endpoints, demonstrates a configuration, and then says:

"Now let's update the endpoint."

The word endpoint is easy to translate.

Knowing which endpoint the instructor means depends on the conversation that came before it.

Relevant context can include:

Recent conversation history
Session context
Previously introduced terminology
User instructions
Relevant information from shared content

The goal isn't to feed the system as much information as possible.

It's to provide the right context at the right time.

That distinction becomes especially important for technical training, research presentations, and online lectures.

Latency Is Part of Translation Quality

A translation can be linguistically accurate and still provide a poor real-time experience.

If a lecturer speaks for ten seconds and the translation arrives several seconds later, the learner has to constantly reconcile the original speech with delayed output.

A streaming architecture helps by processing incoming audio incrementally.

The goal isn't simply:

"Translate this sentence accurately."

It's closer to:

"Translate this ongoing stream accurately enough and quickly enough for the learner to keep following the explanation."

That makes latency an end-to-end concern, not just a property of the translation model.

Why Browser Audio Is Interesting

A lot of modern education and professional communication already happens inside browser tabs:

Online courses
University lectures
Technical workshops
Research presentations
Software tutorials
Webinars
Virtual conferences

The audio is already there.

That creates an opportunity to treat browser audio translation as a separate accessibility layer rather than requiring every content platform to build its own multilingual translation system.

In a simplified model:

Browser Content
↓
Available Audio
↓
Translation Pipeline
↓
Translated Experience

The original platform can continue delivering the content while another system works with the available audio as translation input.

Where This Becomes Useful

The technology becomes valuable when it removes a real communication barrier.

Online learning: Students can follow lectures and courses delivered in languages they aren't fully comfortable with.

Technical training: Translated speech can help learners follow specialized terminology and step-by-step demonstrations.

Research presentations: International teams can make research discussions easier to follow without waiting for a separate translated recording.

Webinars: Browser-based events can become more accessible to multilingual audiences.

Software tutorials: Learners can follow the spoken explanation while watching the actions taking place on screen.

This is where real-time translation for global education moves beyond a language feature and becomes an access-to-knowledge problem.

A Practical Example

This workflow is also relevant to tools such as PolyTalk's Share Audio capability, where audio from a shared browser tab can be used as translation input for lectures, technical training, research presentations, webinars, conferences, and software demonstrations.

For longer sessions, contextual information such as recent conversation history, session context, custom instructions, and relevant visual information can also contribute to the translation experience where available.

The important idea isn't the product itself.

It's the architecture: capture the audio, understand the speech, maintain relevant context, translate continuously, and deliver the result with low enough latency to remain useful.

The Engineering Challenge Ahead

Building real-time translation isn't simply a matter of choosing a capable AI model.

Developers also need to think about:

Latency: How quickly can audio move through the pipeline?
Context management: What information should be retained?
Terminology: How should APIs, acronyms, and domain-specific terms be handled?
Audio quality: How does the system handle noise and different speakers?
Language detection: When should the source language be detected automatically?
Scalability: Can the system maintain performance during long sessions?

These decisions are connected.

More context can improve interpretation but increase processing requirements. More aggressive streaming can reduce perceived latency but create additional complexity around incomplete speech. Better speech recognition doesn't automatically guarantee better translation.

That's why real-time translation is ultimately an end-to-end systems problem.

Making the Web Easier to Learn From

The bigger opportunity isn't simply translating more words.

It's making the knowledge already available online easier for more people to understand.

The web provides the distribution layer. AI can increasingly provide the language layer.

When browser audio, speech recognition, contextual processing, translation, and real-time delivery work together, a lecture created in one language can become accessible to learners somewhere else without requiring the entire learning experience to be rebuilt.

For developers, that makes real-time translation an interesting intersection of AI, speech processing, browser technology, and multilingual user experience.

And perhaps the most useful question isn't "Can we translate this?"

It's:

"Can we make someone feel like they never missed the explanation because of the language?"

Building Real-Time Speech Translation for Multilingual Events

Dharmesh_bizz — Mon, 31 Aug 2026 13:30:54 +0000

International events bring together people who may have completely different languages, backgrounds, and communication styles.

From a technology perspective, this creates an interesting problem.

How do you enable two people to have a natural conversation when they do not speak the same language?

At first, this sounds like a straightforward translation problem. But real-time communication makes it much more challenging.

A document can take a few seconds to translate. A live conversation cannot always afford that delay.

This is where real-time speech-to-speech translation becomes an interesting engineering and product problem.

**Translation Is Easy. Real-Time Conversation Is Harder.

**
A typical translation workflow is relatively simple:

Input → Translation → Output

Real-time speech translation involves a much longer pipeline:

Speech → Speech Recognition → Language Processing → Translation → Speech Output

Each stage introduces processing time.

And latency matters.

If someone says something and has to wait several seconds before the other person hears the translated response, the conversation quickly becomes unnatural. People may pause, interrupt, repeat themselves, or change how they communicate.

So building a useful real-time translation system is not just about getting the translation right.

It is about finding the right balance between accuracy, latency, reliability, and conversation flow.

**Why Events Are an Interesting Use Case

**
Events make this challenge especially visible.

At a conference or trade show, communication is rarely limited to scheduled presentations.

People are constantly having short, spontaneous conversations:

An attendee meets someone during networking.
A visitor asks an exhibitor about a product.
Two founders discuss a possible partnership.
A customer asks questions during a demonstration.
Professionals continue a discussion after a conference session.

These interactions are difficult to predict or plan for.

Professional interpreters are extremely useful for formal presentations, meetings, and high-stakes conversations. But it is not practical to provide an interpreter for every spontaneous interaction happening throughout a large event.

That creates an interesting space for real-time translation technology.

The Real Engineering Challenge: Latency

Imagine a two-person conversation.

Person A speaks.

The system needs to detect the speech, convert it into text or another internal representation, determine the meaning, translate it, and then deliver the result to Person B.

Then the process happens again in the opposite direction.

If every step waits for the previous one to completely finish, latency can quickly increase.

This is why real-time systems need to think carefully about how the pipeline is designed.

The goal is not necessarily to eliminate every millisecond of processing time. The goal is to keep the perceived delay low enough that people can maintain a natural conversation.

That makes low-latency speech translation an important part of the overall user experience.

Accuracy and Speed Are Both Important

There is an obvious trade-off here.

A system can take more time to process speech and potentially improve its understanding of the input. But a slower response can make a live conversation harder to follow.

On the other hand, optimizing heavily for speed can create problems if important meaning is lost.

For real-world multilingual communication, several factors need to work together:

Speech recognition quality
Translation accuracy
Response latency
Audio quality
Language coverage
System reliability
Privacy and data handling

A system can perform well in one area and still provide a poor overall experience if another part of the pipeline becomes a bottleneck.

Why Conversation Flow Matters

Consider how people actually talk.

They do not always wait for a perfectly finished sentence before responding. They pause, clarify, change direction, ask follow-up questions, and react to what they hear.

A translation system needs to work within that natural rhythm.

This is one reason speech-to-speech translation is particularly interesting.

The objective is not simply to produce translated text on a screen. It is to help the listener understand what was said and respond without turning the conversation into a sequence of manual translation steps.

The technology should become part of the communication layer rather than another task the user has to manage.

Where Real-Time Translation Can Be Used

Networking Events

Networking depends on spontaneous communication.

Real-time translation can help people start conversations across language barriers and interact with professionals they might otherwise avoid approaching.

Trade Shows and Exhibitions

Exhibitors often have only a few minutes to explain a product and answer questions.

Real-time translation can support conversations between exhibitors and international visitors without requiring a separate translation workflow for every interaction.

Conferences

Formal sessions may already have interpretation services.

But multilingual communication is also happening in breakout rooms, hallways, networking areas, and informal discussions.

Real-time speech translation can help extend communication beyond the main presentation.

International Business Events

Business discussions often require more than basic translation.

Participants may need to explain products, discuss requirements, ask detailed questions, and explore potential partnerships.

Reducing the language barrier can make it easier for those conversations to begin.

A Practical Example

Consider a founder from Japan meeting an investor from Brazil at a technology conference.

They discover a potential opportunity to work together, but neither person is comfortable discussing technical and business details in the other's language.

Without an effective translation option, they might keep the conversation short or rely on a shared language they are not comfortable using.

With real-time speech translation, each person can communicate in their preferred language while the system handles the translation between them.

The important part is not simply that the words are translated.

It is that the conversation can continue.

Designing for the User, Not the Translation Pipeline

For developers building real-time communication systems, this distinction is important.

It is easy to think about the individual components:

speech recognition → translation → speech synthesis

But users experience the entire system as one interaction.

A technically strong component does not automatically create a good product.

The overall experience depends on how quickly the system responds, how reliably it handles different speakers and environments, and how naturally the translated output fits into the conversation.

That means real-time translation is both an AI problem and a systems problem.

How PolyTalk Fits Into This Use Case

PolyTalk focuses on real-time speech-to-speech translation for multilingual communication.

For events, networking sessions, exhibitions, and international business interactions, the aim is to reduce the friction involved in communicating across languages.

The use cases include:

Real-time multilingual conversations
Cross-language networking
Exhibitor and visitor communication
Product demonstrations
International business discussions
Live communication across language barriers

The broader idea is simple: translation should help people communicate without becoming the focus of the interaction.

The Bigger Opportunity

Global events are becoming increasingly connected.

People travel across countries to meet customers, partners, investors, developers, and communities. Yet language can still determine who people talk to and which opportunities they discover.

Real-time translation will not replace human interpreters in every situation. Cultural context, specialized terminology, and high-stakes communication still require careful consideration.

But for spontaneous conversations, the technology can remove an important first barrier.

That makes multilingual events an interesting real-world test case for real-time AI systems.

The challenge is no longer just:

Can we translate this sentence?

It is:

Can we translate it quickly and accurately enough for two people to keep talking naturally?

That is the problem that makes real-time speech translation worth building—and worth exploring.

Building Better Restaurant Communication with Real-Time Speech Translation

Dharmesh_bizz — Mon, 24 Aug 2026 13:58:43 +0000

A restaurant may have great food, experienced staff, and a well-designed menu.

Communication can still break down when a guest and a staff member do not speak the same language.

Consider a simple interaction.

A guest wants to ask whether a dish contains dairy and whether it can be prepared with less spice. The server understands only part of the request.

The conversation may then involve a translation app, manual typing, gestures, or another staff member who understands the guest's language.

This is a practical example of where real-time speech translation in restaurants can help.

The challenge is not simply translating a sentence from one language to another. The real challenge is supporting a conversation with enough speed and accuracy that people can communicate naturally.

The Problem: Translation Is Easy. Conversations Are Harder.

Traditional translation tools work well for individual phrases.

But a restaurant conversation is rarely a single request.

A guest might ask:

Is this vegetarian?

Then:

Does it contain dairy?

Followed by:

Can it be made less spicy?

Each question adds another step to the interaction.

When translation requires manually entering text, waiting for a response, and repeating the process, the conversation becomes fragmented.

For a real-world communication system, the goal is different.

The system needs to support a continuous exchange between people who speak different languages.

That is where real-time speech-to-speech translation becomes interesting from a technical perspective.

How Real-Time Speech Translation in Restaurants Works

At a high level, a real-time speech translation pipeline can involve several components.

1. Speech Capture

The system first captures spoken audio from the user.

In a restaurant environment, this can introduce practical challenges.

Restaurants are noisy.

There may be background conversations, music, kitchen activity, and multiple people speaking at the same time.

The quality of the input directly affects everything that follows.

2. Speech-to-Text

The spoken audio is converted into text using automatic speech recognition.

The system needs to identify what was said accurately enough for the next stage to work.

This can become more challenging when users have different accents, speak quickly, or use local food names and regional terms.

3. Machine Translation

The recognized text is then translated into the target language.

Context matters here.

Restaurant conversations may include ingredient names, dish names, preparation methods, and special requests.

A literal translation is not always enough. The output needs to preserve the intended meaning of the request.

4. Text-to-Speech or Translated Text Output

The translated content can then be delivered as text or converted back into speech.

This allows the other participant to read or hear the translated message.

For a speech-to-speech experience, this final step helps make the interaction feel closer to a natural conversation.

Latency Is Part of the User Experience

A translation system can be accurate and still provide a poor experience if it is too slow.

Imagine waiting several seconds after every sentence.

The conversation quickly starts to feel unnatural.

This makes latency an important part of real-time translation system design.

The overall delay can come from multiple stages:

Audio capture
Speech recognition
Translation
Speech generation
Network communication

Reducing delay requires looking at the entire pipeline rather than optimizing only one component.

The technical challenge is finding the right balance between speed, accuracy, and resource usage.

Why Restaurants Are an Interesting Real-World Use Case

Restaurants provide a useful example because communication is both frequent and unpredictable.

The system cannot assume that users will follow a script.

Guests may ask about:

Ingredients
Dietary preferences
Allergies
Spice levels
Recommendations
Custom orders

The conversation can change direction at any moment.

This makes the use case more demanding than translating a static document or menu.

A translated menu solves one problem.

Real-time speech translation addresses the communication that happens after the guest starts asking questions.

Supporting Multilingual Restaurant Teams

The same problem can also exist between employees.

A restaurant may have servers, kitchen staff, managers, and support teams who are comfortable communicating in different languages.

Consider a request such as:

Table 12 needs this dish prepared without onions.

The message is simple, but accuracy matters.

Miscommunication can lead to incorrect orders, delays, and unnecessary rework.

A real-time voice translation system can act as an additional communication layer between multilingual teams.

The technology does not need to replace existing communication processes.

It can help make those processes more accessible across language differences.

Privacy and Deployment Considerations

Real-time translation systems also raise questions about deployment.

Many applications depend on cloud-based APIs for speech recognition, translation, or speech synthesis.

That approach can be practical, but it may not fit every organization.

Some businesses may want greater control over:

Where communication data is processed
Infrastructure configuration
System integration
Data handling policies

A self-hosted speech translation system provides another deployment option.

Depending on the architecture, more of the speech and translation pipeline can run within infrastructure controlled by the organization.

For developers and organizations building these systems, the choice between cloud, self-hosted, or hybrid deployment becomes an architectural decision rather than just a feature choice.

Where PolyTalk Fits Into This Architecture

PolyTalk is designed around the same core communication challenge: helping people communicate across languages through real-time speech-to-speech translation.

For restaurants and hospitality environments, potential use cases include:

Communication between international guests and staff
Menu questions and recommendations
Special requests
Multilingual team communication
Staff training
International events

Its privacy-first and self-hosted approach also introduces an interesting deployment model for organizations that want greater control over their translation infrastructure.

The broader idea is not limited to restaurants.

Restaurants simply provide an easy-to-understand example of a larger technical problem: enabling natural communication between people who do not share the same language.

Final Thoughts

Real-time speech translation sits at the intersection of several technologies.

Speech recognition.

Machine translation.

Text-to-speech.

Low-latency processing.

Infrastructure and deployment design.

The interesting part is what happens when these technologies are combined into a single communication experience.

Real-time speech translation in restaurants is one example of how that technology can solve a practical problem.

A guest should be able to ask a question in the language they are comfortable using.

A staff member should be able to understand and respond in theirs.

Building systems that make that interaction feel natural is where the real technical challenge begins.

The Infrastructure Decision That Changed Our AI Translation Platform

Dharmesh_bizz — Thu, 06 Aug 2026 13:20:47 +0000

When we started building PolyTalk, we assumed the hardest problem would be AI.

Speech recognition.

Translation quality.

Voice synthesis.

Model selection.

Like most teams building AI products, we spent a lot of time comparing models and measuring accuracy.

Then something unexpected happened.

The conversations we had with potential users weren't really about AI.

They were about infrastructure.

Questions like:

Where does conversation data actually go?
Can translation stay inside our own environment?
How does this fit with our existing security policies?
What happens if we don't want to depend on third-party APIs?
Can we deploy this inside a private network?

Those questions completely changed how we thought about the platform.

We stopped thinking of translation as an AI problem.

We started thinking of it as an infrastructure problem.

Cloud Translation Solves a Lot of Problems

There's a reason cloud translation has become the default.

The developer experience is excellent.

You authenticate with an API, send text or speech, receive translated output, and let the provider worry about infrastructure, scaling, monitoring, updates, and availability.

For many applications, that's exactly the right decision.

Typical examples include:

website localization
marketing content
documentation
customer support
internal collaboration

If your primary goal is shipping quickly, cloud translation is difficult to beat.

Then We Started Talking to Enterprise Teams

As we worked with organizations evaluating multilingual communication, the discussion changed.

Very few people asked,

"Which translation model are you using?"

Instead they asked:

Where is our voice data processed?
Can translation remain inside our private infrastructure?
How would this work under GDPR or HIPAA?
Can it integrate with our existing identity management?
Can we control where data is stored?
What happens if internet connectivity is unreliable?

None of those questions were about translation quality.

They were about architecture.

That was probably the biggest surprise during development.

The Trade-Off Isn't AI Quality

One misconception we encountered repeatedly was that self-hosted translation automatically produces better translations.

In reality, deployment and model quality are different decisions.

If both systems use the same speech recognition and language models, translation quality can be very similar.

The Trade-Off Isn't AI Quality

One misconception we kept hearing was that self-hosted translation automatically means better translation.

It doesn't.

If two platforms use the same AI models, the translation quality can be nearly identical. The real difference isn't the model—it's the deployment.

Cloud translation lets you move fast. The provider handles infrastructure, scaling, and updates, so your team can focus on building features.

Self-hosted translation gives you more control. You decide where data is processed, how it's secured, and how it integrates with your existing infrastructure.

Neither approach is objectively better. They simply solve different problems.

What surprised us was that most organizations weren't asking, "Which is better?" They were asking, "Which deployment model fits this workload?"

That shift changed how we approached the entire platform.

Things We Didn't Expect

Looking back, several lessons surprised us.

Infrastructure decisions mattered more than model selection.

We expected AI models to dominate every conversation. Instead, organizations spent more time discussing deployment, networking, and governance.

Data ownership often mattered more than translation accuracy.

Most modern AI models already produce strong results. The bigger concern was where conversations were processed and who controlled them.

Hybrid deployments were more common than we expected.

Many organizations didn't want to replace cloud services entirely.

Instead, they wanted cloud translation for websites, documentation, and public content, while keeping meetings, customer support, and sensitive conversations inside infrastructure they already trusted.

That wasn't the architecture we initially expected.

But it quickly became one of the most common deployment patterns we encountered.

The Architecture Decision We Made

Those conversations ultimately shaped PolyTalk's architecture.

Instead of building around external translation APIs, we designed the platform so organizations could deploy real-time speech translation inside infrastructure they already control.

PolyTalk combines:

Faster-Whisper for speech recognition
Ollama-compatible language models for translation
Piper for speech synthesis

Everything runs within the organization's own environment, whether that's on-premises, a private cloud, or another controlled deployment.

The goal wasn't simply to translate speech.

It was to give organizations a choice about where AI runs.

That architectural decision influenced almost every part of the platform.

One Lesson We'll Carry Into Every AI Project

Building PolyTalk changed the way we think about AI systems.

We still care about model quality.

We still benchmark latency.

We still optimize inference.

But we've learned that many organizations ask a different question first:

Where does the AI actually run?

For enterprise software, deployment has become part of the product.

Cloud translation remains the right choice for many workloads.

Self-hosted deployment isn't about replacing the cloud.

It's about giving organizations another option when privacy, compliance, infrastructure ownership, or integration become business requirements.

That was probably the biggest lesson we took away from building PolyTalk.

I'm Curious

If you were designing a real-time AI application today, where would you draw the line between cloud services and self-hosted infrastructure?

Would you optimize for:

Faster development?
Lower operational overhead?
Infrastructure ownership?
Privacy and compliance?
Something else entirely?

We're seeing more teams move toward hybrid deployments rather than treating cloud and self-hosted as competing approaches.

I'd be interested to hear whether you're seeing the same pattern in your own projects.

The Most Important Metric in Real-Time AI Isn't Accuracy

Dharmesh_bizz — Fri, 31 Jul 2026 12:27:41 +0000

Every AI benchmark told us to optimize for accuracy.

So that's exactly what we did.

We compared speech recognition models, evaluated translation quality, tested different text-to-speech engines, and spent weeks chasing better results.

On paper, everything looked promising.

Then we ran one of our first internal demos.

Two people started talking through the system.

The first person finished speaking.

Nothing happened for a moment.

The second person assumed the system had stopped listening and started talking.

A second later, the translation finally played.

The AI hadn't failed.

Our assumptions had.

That demo completely changed how we thought about building real-time AI.

We Were Measuring the Wrong Thing

At the beginning of the project, our only question was:

"How can we make the translation more accurate?"

It seemed like the obvious goal.

But after watching people use the application, we realized they weren't evaluating our AI models. They were evaluating the experience.

Nobody asked how accurate the translation was.

Instead, they asked:

"Why is it taking so long?"

"Is it still processing?"

"Can I start speaking again?"

Those few seconds of silence mattered far more than tiny improvements in translation quality.

That's when we realized users notice latency long before they notice accuracy.

Building a Real-Time System Is More Than Choosing the Right Model

Speech translation isn't just one AI model working in isolation.

Audio has to be captured, streamed, converted into text, translated, converted back into speech, and finally played to the listener.

Each stage adds a small amount of delay.

Individually, none of those delays looked concerning.

Together, they completely changed how natural the conversation felt.

That shifted our focus from improving individual models to improving the entire pipeline.

The Small Change That Made the Biggest Difference

Our first implementation waited until a speaker completed an entire sentence before sending it for translation.

From an engineering perspective, it made sense.

More context usually leads to better translations.

But conversations don't work like written paragraphs.

People pause.

They interrupt themselves.

They restart sentences.

They change their minds halfway through speaking.

Waiting for complete sentences created unnecessary silence.

We changed our approach and began translating partial transcripts as they arrived, continuously refining the output as more speech became available.

Translation quality occasionally became slightly less precise.

The conversations, however, became much smoother.

That was the first time we truly understood that a better user experience doesn't always come from a better AI model.

The Lesson We'll Carry Into Future Projects

Looking back, we spent too much time asking which model performed best.

Today, we'd start with different questions.

Where does the user actually experience delay?

Which part of the pipeline contributes the most latency?

What improvement will users actually notice?

Those questions changed the direction of our engineering work far more than another round of model benchmarking ever did.

Building real-time AI taught us that users don't experience individual models.

They experience the entire system.

Sometimes the biggest improvement isn't replacing the AI model at all.

It's improving everything around it.

These lessons came from building PolyTalk, an open-source, privacy-first platform for real-time speech-to-speech translation. Every iteration has challenged our assumptions about latency, streaming, and multilingual communication, and we're still learning with each new release.

If you've built applications involving streaming, voice AI, WebRTC, or other low-latency systems, I'd love to hear about the engineering trade-offs you've encountered. Have you ever optimized one metric, only to discover your users cared more about something else?

Resources

If you're interested in exploring PolyTalk or contributing to the project:

Try the Live App: https://app.polytalk.io/
Explore the Source Code: https://github.com/PolyTalkIO/polytalk
Learn More: https://www.polytalk.io/

Whether you have feedback, feature ideas, bug reports, or want to contribute, we'd love to hear from you. Every conversation and contribution helps us make PolyTalk better.

Building Multilingual Customer Support: Lessons from Real-Time Speech Translation

Dharmesh_bizz — Tue, 28 Jul 2026 13:32:40 +0000

When we started building real-time speech translation for PolyTalk, we thought the hardest part would be translation.

It wasn't.

The real challenge was keeping conversations natural.

Supporting multilingual customer support sounds simple at first. Convert speech into text, translate it, generate speech in another language, and play it back. Plenty of AI models can handle each of those tasks individually.

The difficult part is making all of them work together fast enough that two people can have a normal conversation without noticing the technology in between.

Here's what we learned while building it.

The First Prototype Looked Great (Until We Tried It)

Our initial pipeline was straightforward.

Microphone
↓
Speech Recognition
↓
Language Detection
↓
Translation
↓
Speech Synthesis
↓
Speaker

Everything worked.

Speech was recognised correctly.

Translation quality was good.

The generated voice sounded natural.

But conversations still felt... awkward.

There was a noticeable pause after almost every sentence. Technically the system worked, yet talking through it didn't feel natural.

That was our first lesson.

Building multilingual customer support isn't just about translation accuracy. It's about conversation flow.

Latency Matters More Than We Expected

Once people start talking, even small delays become obvious.

A pause of one or two seconds doesn't seem significant when translating a document. During a live conversation, though, it changes how people communicate.

They interrupt each other.

They repeat sentences.

They wonder whether the system stopped working.

We quickly realised that reducing latency often had a bigger impact on the overall experience than making the translation model slightly more accurate.

Every stage in the pipeline contributes to the delay:

Speech recognition
Language detection
Machine translation
Speech synthesis
Audio streaming

Optimising only one component doesn't solve the problem. The entire pipeline has to work efficiently.

Real Conversations Are Messier Than Test Data

Demo environments are clean.

Production environments aren't.

People switch languages halfway through a sentence. Background noise changes constantly. Microphone quality varies from one device to another. Different accents and speaking speeds introduce additional complexity.

None of these situations are unusual in customer support.

That changed how we approached the problem. Instead of optimising for perfect demo conditions, we focused on making the system reliable during everyday conversations.

Privacy Isn't Just a Feature

One thing that became clear early on was that many organisations couldn't rely entirely on cloud-based translation services.

Customer support conversations often include personal information, financial details, healthcare records, or confidential business discussions.

For many teams, privacy isn't simply another feature on a comparison page. It's an architectural requirement.

That was one of the reasons we chose a privacy-first, self-hosted approach while building PolyTalk. It gives organisations greater control over where customer conversations are processed and stored.

The Goal Isn't Better Translation

This was probably our biggest takeaway.

Customers don't judge a conversation by the BLEU score of a translation model or the accuracy of speech recognition.

They judge it by whether the conversation feels natural.

If they can explain a problem without repeating themselves and the support agent responds naturally, the technology has done its job.

That's the benchmark we kept returning to throughout development.

Looking Ahead

Real-time AI has made multilingual customer support far more practical than it was only a few years ago, but building a production-ready system still involves much more than connecting a few AI models together.

Latency, speech quality, reliability, privacy, and user experience all matter just as much as translation accuracy.

We're still learning as we continue building PolyTalk, and every iteration reinforces the same lesson:

The best translation system isn't the one with the most impressive model.

It's the one people stop noticing because the conversation simply flows.

Building a Privacy-First Speech Translation System (Without Sending Audio to the Cloud)

Dharmesh_bizz — Fri, 24 Jul 2026 13:01:34 +0000

When we first started building a real-time speech translation platform, we assumed translation quality would be the hardest problem.

It wasn't.

The real challenge was deciding where the conversation should be processed.

Most speech translation applications send audio to cloud APIs for speech recognition, translation, and speech synthesis. That approach is fast to build and works well for many consumer applications. But it raises an obvious question for enterprise software:

What if the conversation shouldn't leave the organization's infrastructure at all?

That question completely changed how we thought about system architecture.

Why Cloud APIs Aren't Always Enough

Cloud AI services are incredibly useful. They reduce operational complexity and let teams ship features quickly.

But once you're working with live conversations, a few trade-offs become difficult to ignore.

Audio containing confidential business information leaves your infrastructure.
Compliance requirements may restrict where data is processed.
Every network request adds latency to an already time-sensitive pipeline.
You're dependent on an external service for every conversation.

None of these make cloud APIs a bad choice. They simply mean they aren't the right choice for every application.

A Privacy-First Architecture Looks Different

A typical speech translation pipeline follows this sequence: Microphone → Speech Recognition → Language Detection → Machine Translation → Speech Synthesis → Audio Playback.

The difference is where those services run.

Instead of sending audio to multiple third-party providers, a privacy-first architecture keeps the pipeline inside infrastructure the organization already controls. That could be an on-premise server, a private cloud, or a dedicated enterprise deployment.

From the application's perspective, the workflow barely changes. From a security and governance perspective, everything changes.

The Hardest Part Isn't Translation

Real-time speech translation is fundamentally a streaming problem.

Unlike translating a document, you can't wait for the speaker to finish before processing the input.

The system has to continuously handle:

live audio streams
partial transcripts
speaker pauses
language detection
translation
speech synthesis

All while keeping latency low enough for the conversation to feel natural.

Every additional API call, network hop, or processing delay adds friction. Users don't usually notice whether translation takes 800 milliseconds or 1.2 seconds, but they immediately notice awkward pauses that interrupt the flow of a conversation.

That's why architecture decisions often matter as much as model quality.

Building for Modularity

One lesson we learned early was to avoid treating speech translation as a single service.

Keeping speech recognition, translation, and text-to-speech as independent components makes the system much easier to evolve.

It allows teams to:

upgrade individual models without rebuilding everything
replace providers when needed
optimize different stages independently
deploy components closer to users

This flexibility becomes especially valuable as speech AI models continue to improve.

Why Enterprises Ask About Self-Hosting

One interesting pattern we've seen is that enterprise conversations rarely begin with model accuracy.

Instead, the first questions are often:

Where is our audio processed?
Can we deploy it ourselves?
What happens to conversation data?
Does it integrate with our existing infrastructure?

Those questions aren't about AI. They're about operational trust.

For organizations working with healthcare data, legal discussions, internal strategy meetings, or regulated environments, deployment architecture matters just as much as translation quality.

What We Learned While Building PolyTalk

These challenges shaped many of the decisions behind PolyTalk.

Rather than assuming every conversation belongs in the cloud, we designed the platform to support self-hosted real-time speech translation, allowing organizations to keep multilingual conversations inside infrastructure they already manage.

The biggest takeaway wasn't that self-hosting is always better.

It was that deployment should be a design decision, not a limitation imposed by the technology.

Final Thoughts

AI models will continue to improve. Translation quality will keep getting better.

But building a production-ready speech translation system is about much more than choosing the latest model.

Latency, streaming architecture, deployment, privacy, and operational control all influence the user experience just as much as the AI itself.

If you're building real-time AI applications, it's worth treating privacy as part of the system architecture from day one—not something added after the product ships.

I'd be interested to hear how others are approaching this. If you've built streaming AI applications or self-hosted inference pipelines, what trade-offs surprised you the most?

https://github.com/PolyTalkIO/polytalk