Building Real-Time AI Suggestions Without a Meeting Bot

Most AI meeting assistants are built around the same idea: join the call, record what happens, and give you something useful afterwards.

A transcript. A summary. Action items. Maybe a list of decisions.

Those tools can be useful, but I became interested in a different problem:

Can AI help while the conversation is still happening?

Not five minutes later. Not in a follow-up email. Right at the moment when you are trying to understand a technical point, formulate an answer, remember something important, or find the right words in a language that is not your own.

That question led me to build LiveSuggest, a real-time meeting assistant that works without joining the meeting as a bot.

It turned out that calling an LLM was the easy part.

The difficult parts were everything around it: capturing the conversation, keeping latency low, deciding when the AI should speak, preserving enough context to be useful, and making sure the system helps without constantly interrupting the user.

Why avoid the meeting bot?

A common architecture for meeting assistants is straightforward: create a participant, join Zoom or Teams, receive the meeting audio, and process it on the server.

Technically, that solves a lot of problems.

From a user-experience perspective, however, it changes the meeting itself.

Everyone can see that a bot has joined. Some companies restrict them. Some participants are uncomfortable with them. And sometimes you simply want assistance for yourself without adding another participant to the call.

So I wanted LiveSuggest to work differently.

The application runs in the browser and captures audio that the user explicitly chooses to share. It does not need access to the meeting platform's API and does not need to know whether the user is on Meet, Teams, Zoom, or something else.

That sounds simple, but browser audio capture has some important limitations.

Capturing meeting audio from the browser

There are two useful sources of audio:

• the user's microphone;
• audio shared through the browser.

For microphone capture, the normal getUserMedia() API works well.

For meeting audio, things are more subtle.

On Chromium browsers, getDisplayMedia() can return an audio track alongside the shared display surface. We explicitly request system audio where the browser supports it:

navigator.mediaDevices.getDisplayMedia({
  video: true,
  audio: {
    systemAudio: "include"
  }
})

The exact behavior depends heavily on the operating system and on what the user shares.

On Windows with Chrome or Edge, sharing the entire screen can include system audio. This means audio from native desktop applications such as Teams or Zoom can be captured without integrating with those applications directly.

Browser-tab audio also works well when the meeting itself is running in a tab.

There is an important trap, though:

sharing an application window does not give you its audio.

A user can therefore share the Zoom window, see everything working visually, and still produce a completely silent audio stream.

Platform differences matter as well. Native browser-level system audio capture is available on Windows/ChromeOS Chromium setups, but not equivalently on macOS or Linux. On those platforms, sharing a browser tab is the reliable native option; capturing all desktop audio generally requires additional operating-system audio routing.

That limitation influenced the product UX as much as the code.

A technically valid feature is not useful if users cannot understand which button to click in the browser's share dialog.

Turning browser audio into a real-time stream

Once we have an audio stream, the next goal is to send it to speech-to-text with as little delay as possible.

In LiveSuggest, audio processing happens through an AudioWorklet.

The stream is converted to mono PCM16 at 16 kHz and divided into small chunks before being sent to the backend.

Conceptually, the pipeline looks like this:

Microphone / system audio
        ↓
Web Audio API
        ↓
AudioWorklet
        ↓
PCM16 audio chunks
        ↓
WebSocket
        ↓
Streaming speech-to-text
        ↓
Finalized utterances

Small chunks matter.

If you buffer several seconds of audio before sending anything, you have already lost the real-time experience before the AI pipeline has even started.

But going too far in the opposite direction also creates overhead: more WebSocket messages, more processing and more opportunities for synchronization problems.

This is one of the recurring patterns I encountered while building the product: real-time systems are mostly a sequence of latency trade-offs.

No individual delay looks particularly large. But audio buffering, network transport, speech recognition, suggestion triggering, LLM time-to-first-token and frontend rendering all add up.

The only latency that matters is the one the user experiences at the end.

A transcript is not yet a suggestion

The speech-to-text system produces finalized utterances continuously.

At first, I thought the obvious next step would be:

New transcript → call the LLM.

That turns out to be a bad strategy.

Conversation does not arrive in neat semantic units.

Someone might say:

"I think the main issue here is..."

and then continue the actual idea in the following sentence.

Calling an LLM immediately can generate a response to incomplete context. Calling it on every transcript also generates far too many suggestions.

So the system instead accumulates conversational context.

A generation is triggered only after enough new information has appeared since the previous suggestion.

The threshold is configurable. A user asking for more frequent assistance can get a lower threshold; someone who wants fewer interruptions can use a higher one.

We also treat the first suggestion slightly differently by allowing it to trigger earlier. The reason is not purely technical: an empty interface creates uncertainty.

When someone starts a session, they want confirmation that the system is actually understanding the conversation.

Reducing the latency of that first useful result has a disproportionate effect on how responsive the product feels.

Streaming the answer matters too

Suppose speech recognition takes a second, context preparation takes another fraction of a second, and the model takes a second or two to generate the complete answer.

Waiting for the entire suggestion before displaying anything makes the system feel much slower than necessary.

Instead, suggestions are streamed to the browser as they are generated.

The frontend receives roughly this sequence:

suggestion-start
suggestion-title
suggestion-delta
suggestion-delta
suggestion-delta
...
suggestion-end

The user can therefore begin reading before the model has finished generating the complete suggestion.

Again, the interesting metric is not just total generation time.

For interactive AI, time to first useful information is often more important.

A perfect answer that arrives after the conversation has moved on is not useful.

The AI also needs to know when not to help

Another lesson was that "more AI" does not necessarily make a better assistant.

LiveSuggest can generate several categories of suggestions, for example:

• clarification of a technical concept;
• meeting follow-up or something worth remembering;
• a new idea or angle;
• explanation of a foreign expression;
• a possible answer when someone asks the user a direct question.

But generating one of these every few seconds would quickly become distracting.

The system therefore has two filtering problems:

1. When is there enough new context to justify generating something?
2. Is there actually something useful to say?

The second question is much harder than the first.

A meeting assistant should be allowed to remain silent.

That sounds obvious, but it changes how you think about the system. The objective is not maximum token generation. It is maximum useful intervention.

Privacy changes the architecture

Real-time meeting assistance also creates an obvious privacy question: what happens to the conversation?

One design decision was to separate product analytics from conversation content.

We need operational signals to improve the system. For example:

• suggestion latency;
• time to first token;
• whether a suggestion received a thumbs-up or thumbs-down;
• whether the user copied it;
• whether the user asked for more detail.

Those signals help answer questions such as:

Are suggestions arriving quickly enough?

and:

Are users actually interacting with them?

But the content of the transcript and the suggestions themselves does not need to be persisted simply to measure those things.

That distinction is useful beyond meeting assistants.

AI applications often need telemetry, but telemetry does not automatically require storing the underlying user content.

What I learned

After building this pipeline, my biggest takeaway is that real-time AI products are not primarily LLM problems.

The model is one component of a much larger feedback loop:

conversation
    ↓
audio capture
    ↓
speech recognition
    ↓
context accumulation
    ↓
decision to generate
    ↓
LLM
    ↓
streaming UI
    ↓
human decision

Every stage affects whether the final suggestion arrives at the right moment.

And "the right moment" is a surprisingly narrow window.

Generate too early and the model lacks context.

Generate too late and the conversation has moved on.

Generate too often and the assistant becomes noise.

Generate too rarely and users forget it is there.

That balance has been much more interesting to work on than the basic LLM integration.

Where this is going

I started LiveSuggest because I wanted something that could help during a meeting rather than explain the meeting afterwards.

The current system listens to explicitly shared audio, transcribes the conversation in real time, accumulates context and produces suggestions while the discussion is still happening.

There are still many problems I want to explore: better timing, better relevance detection, improved handling of multilingual conversations, and finding better ways to measure whether a suggestion was genuinely useful rather than simply displayed.

But the underlying idea has become clearer:

AI does not necessarily need to join a conversation to participate in it.

Sometimes the better interface is simply a quiet layer beside the conversation, listening only when invited and offering help when there is actually something useful to say.

If you are interested in trying the approach, I am building it at LiveSuggest.