DEV Community: Dev Vishwas

Why is Gemini 3.5 Live Translate promising?

Dev Vishwas — Sat, 27 Jun 2026 03:37:19 +0000

Got to attend the Gemini 3.5 Live Translate AMA @ Discord. I summarised some points and want to discuss about it.

Google DeepMind ran an exclusive Discord AMA for Gemini 3.5 Live Translate with Anuda Weerasinghe (Associate PM) and Thor Schaeff (DevRel Engineer). 74 people in the room. I was one of them.

This wasn't a product demo or a launch recap. It was an hour of engineers actually explaining architecture decisions, tradeoffs, and constraints. That's rare. So here's a full technical breakdown of everything worth knowing from it.

First, what even is Gemini 3.5 Live Translate?

The short version: it's a streaming speech-to-speech translation model that supports 70+ languages and generates translated audio in near real-time, a few seconds behind the speaker.

The part people gloss over: it's built on Gemini 3 Pro, not a lightweight translation model. That matters because the quality ceiling is much higher than what you'd get from a dedicated translation pipeline slapped onto a TTS layer.

It rolled out on June 9, 2026 across three surfaces: the Gemini Live API for developers, Google Meet for enterprise (private preview), and Google Translate on Android and iOS for everyone else.

It's not text in the middle.

Most translation systems work in a chain: speech to text, text translated, text back to speech. That chain introduces latency at every step, and each conversion is a place where meaning can degrade.

Gemini 3.5 Live Translate skips the middle entirely. It's audio in, audio out. The model takes streamed audio and generates translated audio directly, without converting to text as an intermediate representation.

The API enforces this at the format level. Both input and output have to be raw 16-bit PCM data. No MP3, no compressed formats. They're deliberately keeping the pipeline lean for latency reasons, and if you're building on it, that's a constraint to know upfront. Your audio capture code needs to output raw PCM, and whatever you're playing back needs to handle it.

If you need text, you can still get it. But it comes from a separate transcription layer, not from the audio model itself. The native audio model only returns audio.

The thinkingLevel parameter is basically a latency knob.

The model has a thinkingLevel parameter that controls how much reasoning it does before responding. At minimal, it skips chain-of-thought entirely and outputs translated speech as fast as possible. Crank it up and you get more deliberate, accurate translations, but you pay for it in latency.

For live conversation, you almost always want minimal. The whole point of real-time translation is that the lag is small enough that conversation still feels natural. Adding deliberate reasoning time defeats that.

What I found interesting is how server events work. One event can contain audio, text, and metadata simultaneously. They arrive bundled, not sequentially. That means your client doesn't have to coordinate waiting for different pieces to come in one after another.

There's also a concept called send_client_content worth understanding. It lets you front-load conversation history at the start of a session. But once the session goes active, that door closes. You can't inject context mid-session. So if you're building something where prior conversation matters, set it up before the first turn starts.

I asked Anuda directly about context degradation.

This was the question I wanted answered. Long-running sessions are where translation systems quietly fall apart, and most product demos never show you an hour-long call.

My question to Anuda: does translation quality degrade over a long session as earlier context is lost? Is there a context window limit on the stream?

His answer: context is retained throughout the session and you shouldn't see quality drops. The live API has a 32k token context window. The native model gets 128k, but the live API variant runs at 32k. Sessions are capped at 10 minutes, but session management APIs let you extend beyond that. Voice consistency is still an area they're actively working on.

The 32k vs 128k split is the thing to note here. If you're building something like a multilingual lecture tool or a long customer support call, you're working within 32k tokens. For most live conversations that's probably fine, but it's worth understanding before you hit an edge case.

The language switching is actually clean.

This was one of the most practically impressive things they talked about. No flags, no session resets, no manual configuration. The model detects language shifts directly from the audio stream and follows along.

If someone switches from Tamil to English mid-sentence, the translation adapts. It works on both incoming and outgoing speech. 70+ languages supported, and the full list includes languages like Bengali, Malayalam, Marathi, Burmese, Basque, and Mongolian, not just the usual high-resource set.

2000+ language pair combinations is the claim. That number comes from the combinatorics of the supported languages, and whether all of those pairs are equally well-evaluated is a fair question to ask. But the mid-conversation switching working without any manual input is genuinely useful, especially for multilingual teams where people naturally shift languages mid-call.

The video input is not what you think.

The API accepts video, but it's not a video call in any conventional sense. Frames are sent as individual JPEGs or PNGs via real-time inputs, and you're capped at one frame per second. It's frame-by-frame visual context, not a video stream.

Think of it less like a webcam feed and more like periodically telling the model what the environment looks like. Useful for grounding the translation in visual context, but if you're expecting the model to track motion or read fast-moving text on screen, that's not what this is.

The architecture they walked through.

The reference implementation uses LiveKit, and the flow is clean enough to be worth understanding.

The organizer publishes audio to a LiveKit room. A TranslationBridge bot joins the room per target language, subscribes to the organizer's audio, and sends it to the Gemini Live API with a translationConfig specifying directional translation and a targetLanguageCode. The model returns translated audio, which the bridge publishes back into the LiveKit room. Attendees then subscribe to a translator-{lang} audio track for whatever language they need.

The nice thing about this design is the separation. Each language gets its own bot. The core call infrastructure doesn't need to know anything about translation. You're just routing audio tracks. It's composable and relatively easy to extend.

The Gemini 3.5 Live Translate Docs:

Docs

The AMA was one of the more technically honest ones I've attended. Anuda and Thor explained real constraints, real tradeoffs, and architecture that you can actually implement. The raw PCM requirement and the 10-minute session cap are the two things to design around first. Everything else flows from those.

If you're building on the Gemini Live API, the reference LiveKit architecture is a solid starting point. And if you're just following the space, the audio-to-audio shift without a text intermediate is the architectural detail that actually matters here.

[Boost]

Dev Vishwas — Tue, 16 Jun 2026 16:18:13 +0000

Thor 雷神 Schaeff for Google AI

Jun 9

Build a Realtime Translation App with Gemini Live API, LiveKit, & Google Cloud Run

#ai #gemini #nextjs #tutorial

5 min read

Moyacchi ; Your AI Eco Companion

Dev Vishwas — Sun, 19 Apr 2026 22:43:17 +0000

This is a submission for Weekend Challenge: Earth Day Edition

This is a submission for Weekend Challenge: Earth Day Edition

What I Built

Most people have no idea what their daily choices are actually doing to the environment. Not because they don’t care ; but because nothing makes it visible, personal, or worth coming back to.

Moyacchi is my answer to that. It is an AI-powered eco companion that tracks your daily habits across three dimensions ; transportation, food, and energy use ; analyses their cumulative environmental impact using Google Gemini, and reflects that progress through a companion that literally evolves with you. Seed. Sprout. Bloom. Glow.

The goal was never to build another carbon calculator. It was to build something that feels alive. Something you actually want to open tomorrow.

Beyond the core habit loop, Moyacchi includes Green-Sight ; a real-time OCR scanner that reads any product label, flags harmful ingredients, identifies packaging material and waste classification. A conversational AI eco coach. A live air quality dashboard for any city on earth. And a Journal that doubles as a personal environmental diary, with shareable impact cards for every entry.

Demo

🍏 Live Demo ; moyacchi.vercel.app

Code

radmm / Moyacchi

Moyacchi AI Eco Companion - Submission at Dev.to

Moyacchi ; Your AI Eco Companion

Demo: https://youtu.be/y_8WKch14kQ

Track your habits. Understand your impact. Grow together.

Features

Daily Stack Analysis Log your day across three dimensions ; transportation, food, and energy use. Moyacchi analyses the entire day at once using Google Gemini Flash and returns a score, performance grade, behavioral swaps, and a real-world impact metaphor.

Evolving Companion Moyacchi grows with your progress through four stages ; Seed, Sprout, Bloom, Glow. The better your habits, the more he evolves.

Green-Sight OCR Snap any product label. Moyacchi identifies harmful ingredients, packaging material, waste classification, and disposal guidance in real time.

AI Eco Coach A conversational AI coach powered by Gemini. Ask anything about sustainable living ; no lectures, no guilt.

Sky Dashboard Live air quality for any city on earth. AQI, six pollutants, pollen levels ; powered by Open-Meteo, completely free and open.

Journal A chronological record of your environmental footprint…

View on GitHub

How I Built It

The frontend is React 19 with TypeScript, styled with Tailwind CSS v4. Firebase handles Google authentication and real-time Firestore persistence, with security rules strictly scoping every read and write to the authenticated user. Guest users get full functionality through localStorage with seamless cloud sync on sign in.

All AI inference runs through Google Gemini Flash. Every model call is structured against a strict JSON schema ; habit analysis, image OCR, chat, and air quality interpretation all return typed, validated, reliable output. No hallucinated formats, no unparseable responses.

The Sky Dashboard pulls live AQI, pollutant, and pollen data from the Open-Meteo API ; completely free, no key required, globally accurate.

The mascot is a pure SVG rendered directly in JSX with path morphing and mood animations handled by Framer Motion. No image files. Just code.
The entire application was designed, built, and shipped in one day.

Prize Categories

Best Use of Google Gemini ; Gemini Flash powers every intelligent layer of Moyacchi. Habit analysis with structured JSON schemas, Green-Sight OCR for real-time product label scanning, a multi-turn conversational eco coach, and image-based habit identification from photos. Gemini is not a feature ; it is the backbone.

Built by Dev Vishwas ; @devishwas

Building a Quantum ESPRESSO GUI on a Tablet

Dev Vishwas — Fri, 20 Mar 2026 19:39:14 +0000

So here’s something I didn’t expect to be writing about this semester.

I’m a first-year CSE student. I do most of my coding practice on a tablet with Google Colab open in the browser. And somehow, over the past few weeks, I ended up building a working simulation of BURAI 3; the GUI for Quantum ESPRESSO, specifically for visualizing the band gap of MoSe2 (Molybdenum Diselenide). A real 2D material being studied for next-gen transistors and solar cells.

Full disclosure upfront: I referenced published papers, leaned on the Three.js documentation, used AI assistance, and took workflow inspiration from existing GitHub repos. But I understood what went into it, and that’s what I actually want to talk about.

Because here’s the thing - nobody tells you in first year. Understanding something and building it from scratch are two completely different skills, and both are valid.

Why This Project

My professor mentioned MoSe2 band structure project. Something about how it has a direct band gap in monolayer form but an indirect one in bulk. Didn’t fully get it at the time, but the phrase “indirect-to-direct transition” stuck with me.

I came back to my dorm, looked it up, fell down a rabbit hole, found Quantum ESPRESSO, then found BURAI 3. Grey interface, teal and yellow atoms, old-school panel on the left. It looked like something a researcher would actually use.

I thought: can I build something that looks and works like this, but runs in a single Colab cell on a tablet?
Turns out yes.

What I Actually Built

Before anything technical, let me be clear about what this is and what it isn’t.

What it is:

A browser-based simulation of the BURAI 3 interface
A 3D interactive crystal structure viewer using Three.js
A band structure plot with real published gap values for MoSe2
Physics controls that actually change the graph (Monolayer/Bulk, PBE/HSE06/GW, SOC on/off)
Runs entirely in one Google Colab cell

What it isn’t:

A real Quantum ESPRESSO installation running pw.x
A live DFT calculation (that needs a cluster and hours of runtime)
Something I could have built entirely on my own at this stage

That last point matters. I see posts where people oversell what they built and it always feels off. So: built with assistance, understood it, iterated on it, learned a lot.

The Physics

Band gap is the energy distance between where electrons normally sit (valence band) and where they need to go to conduct electricity (conduction band). MoSe2 sits around 1.44 to 1.55 eV, comfortably in semiconductor range.

The reason the method choice matters is that PBE (the standard DFT approximation) consistently underestimates band gaps by 10 to 30 percent. HSE06 corrects for this. GW gets closest to experiment but is expensive to run. The viewer lets you switch between all three and see the difference directly.

The Prototype - What It Actually Looks Like

When you run the cell, you land on the 3D structure view first. The BURAI orange top bar, the dark tab bar with “Mini Project” and “mose2_final” tabs, and the grey viewport with the crystal rotating in front of you. Mo atoms in teal, Se atoms in yellow, bonds connecting them.

The band structure view section; the plot sits in the viewport — red solid bands, orange dashed SOC-split bands, a cyan shaded region marking the gap, and an arrow annotation on the right showing the exact gap value. Below the plot, the k-point labels: Γ M K Γ A.

The right panel has all the controls. Change any of them and the plot redraws immediately:

Monolayer vs Bulk — the band shapes shift, the gap type flips between direct and indirect.

PBE / HSE06 / GW — the gap value changes and a note appears at the bottom explaining the accuracy tradeoff for that method.

SOC on/off — the top valence band visibly splits into two branches when you turn it on.

Default on load: Monolayer + HSE06 + SOC ON = 1.55 eV. That’s the closest match to the experimental optical gap for monolayer MoSe2.

Languages and Tools used

Each language has one job and sticks to it.
Python delivers the whole thing to Colab’s output. All it needs is one call. That’s its entire role here.

HTML is the structure - the window layout, the top bar, the panels, the tabs. Matching BURAI meant getting specific: #cccccc grey, #3a3a3a tab bar, the exact button shapes.

CSS is purely visual - the orange gradient, the old-school inset borders, the toggle states.

JavaScript is where the project actually lives. The 3D viewer, the band plot, all the toggle logic. Switch from PBE to HSE06 and JavaScript catches it, grabs the new value, wipes the canvas, and redraws everything. Feels instant.

Three.js handles the 3D crystal. It wraps WebGL (the browser’s 3D engine) into something actually approachable; so you’re placing atoms and setting up lights, along with drag interactions.

HTML Canvas API draws the band structure graph. No library, it’s the browser’s built-in 2D drawing tools. Each line, label, and shaded region on that plot is a direct canvas command.

A Bit of Three.js

Here’s what placing an atom in the 3D viewer actually looks like:

// Mo atom — teal sphere at its crystal position
const moGeometry = new THREE.SphereGeometry(0.42, 32, 20);
const moMaterial = new THREE.MeshPhongMaterial({
  color: 0x22aacc,     // teal — matches BURAI exactly
  shininess: 90,
  specular: 0x99ddee   // gives it that shiny lit look
});
const moAtom = new THREE.Mesh(moGeometry, moMaterial);
moAtom.position.set(x, y, z); // fractional crystal coordinates
scene.add(moAtom);

Honest Limitations

The gap values come from published literature, not a live calculation. The band shapes are drawn from manually placed coordinates with smoothing. The BURAI interface is a visual simulation.

The honest pitch: this is an interactive visualization of MoSe2 band structure physics using literature values, built to explore how dimensionality, functional, and SOC choices affect band gap results — in a UI that mirrors real BURAI output.

For a first-year project, that’s a pretty good starter.

What I Took Away

The band structure is literally just a plot of allowed electron energies against crystal momentum. Once that clicked, the whole subject stopped being intimidating. And HTML, CSS, and JavaScript can build things that look like serious scientific software if you care enough about the details.

Sources: Liu et al. PRB 88, 085433 (2013) · github.com · threejs.org docs etc

Thanks for reading. Corrections welcome.

Dev Vishwas | First-year B.Tech CSE