DEV Community: Felona Voice

Voice AI Latency & Cost: Felona Voice Cuts LLM Inference by 95% with JEV

Felona Voice — Sun, 27 Sep 2026 15:34:59 +0000

The promise of natural, low-latency voice AI has long been tantalizing, yet often constrained by the realities of cost and performance. Traditional voice agents, relying heavily on large language models (LLMs) for conversational intelligence, frequently fall short of real-time expectations and rack up significant infrastructure bills. But what if there was a better way? An open-source, TypeScript-native framework that delivers sub-10ms intent routing with zero token cost and zero hallucinations?

Enter Felona Voice, a groundbreaking open-source project that’s redefining the economics and performance of voice AI. At its core, Felona Voice leverages Joint Embedding Vectors (JEV) to achieve neural routing that’s not just fast, but virtually free for structured conversational turns. Let's dive deep into why this innovation is a game-changer for your next voice AI project.

The Latency & Cost Conundrum of Traditional LLM-First Voice Agents

Most modern voice agents follow a similar, expensive, and slow pipeline:

Speech-to-Text (STT): User's speech is converted to text (e.g., Deepgram, Whisper).
LLM Inference: The transcribed text is sent to an LLM (e.g., GPT-4o, Claude 3 Opus) to understand intent, generate a response, or decide the next action.
Text-to-Speech (TTS): The LLM's text output is converted back into natural-sounding speech (e.g., ElevenLabs, Cartesia).

While powerful for open-ended dialog, this LLM-centric approach introduces several critical bottlenecks:

High Latency: Each step involves network round-trips and significant processing time. STT, LLM inference (especially for larger models or longer responses), and TTS can easily accumulate to 500ms, 800ms, or even 1200ms+ per conversational turn. This makes for a frustrating, unnatural user experience.
Exorbitant Costs: LLM API calls are priced per token. A single conversational turn, including input and output tokens, can cost anywhere from $0.02 to $0.06 or more, depending on the model and token count. These costs scale linearly and rapidly become prohibitive for high-volume applications.
Hallucination Risk: LLMs are probabilistic text generators. While incredibly capable, they can

This article was originally published on felona-voice.mohitjoe.tech.

Unlock Sub-10ms Voice AI: Real-time Agents with TypeScript & Felona Voice

Felona Voice — Sun, 27 Sep 2026 15:32:24 +0000

The dream of truly real-time, human-like voice agents has long been hampered by one critical bottleneck: latency. Traditional voice AI systems, heavily reliant on large language models (LLMs) for conversational understanding and response generation, introduce significant delays. We're talking 500ms to 1200ms or more per conversational turn, creating an awkward, unnatural experience that frustrates users and limits adoption.

But what if you could eliminate that latency? What if your voice agent could understand intent and react in mere milliseconds, without sacrificing accuracy or breaking the bank? Enter Felona Voice – an innovative, open-source TypeScript framework designed to build ultra-low-latency voice agents that feel genuinely real-time.

The Latency & Cost Problem with Traditional LLM Voice Agents

Let's face it: LLMs are powerful, but they're not built for instant, sub-10ms decision-making in high-volume voice interactions. Here's why traditional approaches struggle:

High Latency: Every conversational turn often involves an API call to a remote LLM. This introduces network latency, processing time on the LLM server, and token generation time. The combined effect easily pushes response times into the hundreds of milliseconds, creating noticeable pauses.
Exorbitant Costs: Each LLM API call costs money, typically per token. At scale, with thousands or millions of interactions, these costs quickly skyrocket, making real-time voice AI a luxury few can afford.
Hallucination Risk: LLMs are probabilistic by nature. While incredibly creative, they can sometimes generate inaccurate or off-topic responses, leading to frustrating user experiences and a lack of control for developers.
Rigid Static Graphs: On the other end of the spectrum, hardcoded decision trees are fast and deterministic but too brittle. They break down the moment a user deviates slightly from the expected script, leading to dead ends and frustrated users.

Felona Voice: The Sub-10ms Revolution Powered by JEVs and VoiceGraph

Felona Voice tackles these challenges head-on with a revolutionary approach: Joint Embedding Vectors (JEV) combined with intelligent, stateful conversational transition graphs, which we call VoiceGraph.

Instead of sending every user utterance to an LLM for interpretation, Felona Voice leverages JEV similarity matching. Here's the magic:

Intent Mapping: Your agent's defined actions and their descriptions are transformed into high-dimensional embedding vectors (JEVs).
Real-time Matching: When a user speaks, their transcribed utterance is also converted into a JEV. Felona Voice then performs ultra-fast similarity matching between the user's utterance JEV and your agent's action JEVs.
Sub-10ms Decisions: This similarity matching operation is incredibly efficient, happening in sub-10ms (typically ~5ms). It's an in-memory, local computation that requires zero external API calls for intent decision-making.
Zero Hallucinations: Because decisions are based on deterministic similarity matching against predefined actions, there's no room for LLM hallucinations. Your agent will always follow the intended conversational flow.
Dynamic VoiceGraph: The VoiceGraph isn't a rigid, static decision tree. It's a stateful graph that uses JEV similarity to dynamically decide the next best action, allowing for natural transitions and graceful handling of off-script speech, far beyond what traditional static graphs can offer.

The Stark Reality: Traditional vs. Felona Voice

Let's put it into perspective with a direct comparison:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

Cutting Infrastructure Bills by 90-95%

The cost savings with Felona Voice are not just significant; they're transformative. Let's do the math for the intent decision component alone:

Consider an application scaling to 1,000,000 conversational turns per month. If each turn involves an LLM call for intent detection at a conservative average of $0.02 per turn:

Traditional LLM Agent: 1,000,000 turns * $0.02/turn = $20,000 per month for intent decision alone.
Felona Voice Agent: 1,000,000 turns * $0.00/turn = $0 per month for intent decision.

This translates to a 100% reduction in LLM costs specifically for intent detection. While you still need ASR (Speech-to-Text) and TTS (Text-to-Speech) services (which Felona Voice integrates seamlessly), by eliminating the most expensive and slowest part of the conversational loop – the LLM-based intent inference – you can realistically cut your overall voice AI infrastructure bills by 90-95% when scaling to thousands or millions of calls. This is a game-changer for businesses looking to deploy cost-effective, high-performance voice agents.

Developer Experience: Fluent and Powerful in TypeScript

Felona Voice is built with developers in mind, offering an incredibly fluent and intuitive API in TypeScript. You can define complex conversational flows with ease:

import { createAgent } from "felona-voice";

const agent = createAgent("ConciergeBot")
  .system("You are an intelligent voice concierge for a luxury hotel.")
  .action("book_room", "Book a hotel room", async (ctx) => {
    const roomType = ctx.get("roomType");
    if (roomType) {
      return `Booking a ${roomType} for you.`;
    } else {
      return "What type of room are you looking for?";
    }
  })
  .action("check_inquiry", "Check reservation status or details", async (ctx) => {
    const reservationId = ctx.get("reservationId");
    if (reservationId) {
      return `Checking details for reservation ${reservationId}.`;
    } else {
      return "Please provide your reservation ID.";
    }
  })
  .fallback("I'm sorry, I couldn't understand that. Can you please rephrase or ask about booking a room or checking a reservation?");

async function runAgent() {
  console.log("User: I want to book a suite.");
  let reply = await agent.interact("I want to book a suite.");
  console.log("Bot: ", reply); // Expected: "Booking a suite for you."

  console.log("User: What's my reservation status?");
  reply = await agent.interact("What's my reservation status?");
  console.log("Bot: ", reply); // Expected: "Please provide your reservation ID."

  console.log("User: My reservation ID is 12345.");
  reply = await agent.interact("My reservation ID is 12345.");
  console.log("Bot: ", reply); // Expected: "Checking details for reservation 12345."

  console.log("User: Tell me a joke.");
  reply = await agent.interact("Tell me a joke.");
  console.log("Bot: ", reply); // Expected: "I'm sorry, I couldn't understand that. Can you please rephrase or ask about booking a room or checking a reservation?"
}

runAgent();

Key developer benefits include:

Pluggable Audio Pipelines: Felona Voice is designed for flexibility, allowing you to integrate with various audio processing services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia, ensuring you can use your preferred ASR/TTS providers.
Zero External API Keys for Local Testing: Develop and test your agent locally without needing to hit costly external LLM APIs. Intent routing is deterministic and works entirely in-memory.
TypeScript Native: Leverage the power of TypeScript for type safety, better tooling, and improved maintainability.

Getting Started with Felona Voice

Ready to build your own ultra-low-latency voice agent? Getting started is incredibly simple:

First, install the package via npm:

npm install felona-voice

Then, create your first agent:

import { createAgent } from "felona-voice";

const simpleAgent = createAgent("GreetingBot")
  .system("You are a friendly bot that greets users.")
  .action("greet_user", "Say hello to the user", async (ctx) => {
    const name = ctx.get("name") || "there";
    return `Hello ${name}! How can I help you today?`;
  })
  .fallback("I didn't quite catch that. Could you say hello?");

async function testSimpleAgent() {
  console.log("User: Hi, my name is Alice.");
  let reply = await simpleAgent.interact("Hi, my name is Alice.");
  console.log("Bot: ", reply); // Expected: "Hello Alice! How can I help you today?"

  console.log("User: Just saying hi.");
  reply = await simpleAgent.interact("Just saying hi.");
  console.log("Bot: ", reply); // Expected: "Hello there! How can I help you today?"

  console.log("User: What's the weather?");
  reply = await simpleAgent.interact("What's the weather?");
  console.log("Bot: ", reply); // Expected: "I didn't quite catch that. Could you say hello?"
}

testSimpleAgent();

This basic example demonstrates how quickly you can define actions and their corresponding responses, setting up a robust conversational flow without the typical LLM overhead.

The Future of Real-Time Voice AI is Here

Felona Voice isn't just another framework; it's a paradigm shift. By moving intent decision-making from slow, expensive, and probabilistic LLM calls to ultra-fast, deterministic, in-memory JEV similarity matching, Felona Voice unlocks true real-time voice AI.

Imagine customer service bots that respond instantly, gaming NPCs that react without delay, or smart assistants that feel genuinely present. This is the future Felona Voice enables, today. It's about building production-ready voice AI that is not only performant and reliable but also incredibly cost-effective at scale.

Get Involved and Start Building!

Join the revolution in real-time voice AI. Felona Voice is open-source, community-driven, and ready for your contributions.

🌟 Star the repository on GitHub: https://github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: https://felona-voice.mohitjoe.tech/docs

Start building your sub-10ms voice agent today and experience the difference true real-time AI makes!

This article was originally published on felona-voice.mohitjoe.tech.

Deterministic Voice AI: Sub-10ms & Hallucination-Free with Felona Voice

Felona Voice — Sun, 27 Sep 2026 15:28:38 +0000

The promise of natural, real-time conversational AI has long been tempered by the realities of latency and unpredictable responses. Traditional voice bots, often reliant on large language models (LLMs) for decision-making, grapple with response times upwards of 500ms and the notorious issue of 'hallucinations' – generating plausible but incorrect information. Hardcoded conversational flows, while deterministic, are too rigid for the organic nature of human speech, breaking down at the slightest deviation. This is where Felona Voice steps in, offering a revolutionary approach to building ultra-low-latency, hallucination-free voice agents.

Felona Voice, an open-source TypeScript framework, redefines the landscape of voice AI by introducing VoiceGraph, a stateful conversational transition graph powered by Joint Embedding Vectors (JEV). This innovative combination allows for neural, yet deterministic, routing of conversational turns in sub-10ms, effectively eliminating the trade-off between flexibility and reliability.

The Unpredictability of Pure Prompt-Based Voice Bots

Before diving into Felona Voice's solution, let's understand the challenges with existing paradigms:

Pure LLM-Based Routing: While powerful for understanding context and generating nuanced responses, LLMs introduce significant latency. Each conversational turn requires processing, which can take 500ms to 1200ms or more. This delay creates an unnatural, disjointed user experience. Furthermore, LLMs are known to 'hallucinate,' fabricating answers or taking unexpected conversational paths, making them unsuitable for mission-critical applications requiring precision.
Hardcoded Static Graphs: These offer determinism and low latency, as every path is explicitly defined. However, they are inherently inflexible. Any deviation from the script, a common occurrence in natural human conversation, can break the flow, leading to frustrating dead ends or generic fallback responses. They lack the intelligence to interpret intent beyond exact keyword matches.

Both approaches fall short of delivering the seamless, reliable, and real-time voice interactions users expect.

Felona Voice's Innovation: VoiceGraph with Neural Transitions

Felona Voice addresses these limitations head-on with its core innovation: VoiceGraph driven by Joint Embedding Vectors (JEV). Imagine a sophisticated state machine where transitions between states aren't just hardcoded rules, but intelligent, neural decisions made in milliseconds.

Joint Embedding Vectors (JEV): The Brain Behind the Speed

At the heart of Felona Voice's ultra-low latency is the concept of Joint Embedding Vectors. Instead of relying on token-by-token LLM inference for every decision, Felona Voice pre-computes semantic embeddings for all possible actions and user intents defined within your agent. When a user speaks, their utterance is quickly embedded into the same vector space. Felona Voice then performs a lightning-fast similarity match between the user's intent embedding and the embeddings of all available actions.

This similarity matching allows Felona Voice to decide the next action in approximately 5ms. This is not just faster; it's a paradigm shift. Zero token latency means no waiting for LLMs to generate text or process complex prompts for routing decisions.

VoiceGraph: Stateful and Deterministic Conversations

VoiceGraph acts as the architectural blueprint for your conversational flow. It's a state machine where each state represents a point in the conversation, and transitions are governed by the JEV similarity matching. This provides the best of both worlds:

Determinism: By mapping user intent directly to predefined actions via JEV, Felona Voice ensures that the agent always responds predictably. No more hallucinations or unexpected tangents.
Flexibility: Unlike rigid static graphs, VoiceGraph's transitions are "neural." They understand semantic meaning, allowing for natural variations in user speech to trigger the correct action without explicit keyword matching.
Statefulness: The VoiceGraph maintains the conversational state, enabling complex multi-turn interactions while always knowing where it is in the flow.

Let's compare the approaches:

Feature	Pure LLM Routing	Hardcoded Static Graph	Felona Voice (JEV + VoiceGraph)
Decision Latency	High (500ms - 1200ms+)	Low (hardcoded lookups)	Ultra-low (~5ms)
Hallucinations	Frequent, unpredictable	None (if matched)	None (deterministic)
Flexibility	High (but unpredictable)	Low (rigid, breaks easily)	High (neural understanding)
Determinism	Low	High	High
Cost	High (token usage)	Low	Low (zero token latency)
Developer Effort	Prompt engineering	Extensive rule definition	Fluent API, intent-driven

Building Hallucination-Free Agents with Felona Voice's Fluent API

Felona Voice is designed with developer experience at its core. Its fluent builder API in TypeScript makes defining complex conversational agents intuitive and powerful. You can define system instructions, specific actions, and fallback behaviors with ease.

To get started, simply install the package:

npm install felona-voice

Here’s a practical example of building a simple concierge agent that can book a table or provide general assistance:

import { createAgent } from "felona-voice";

const agent = createAgent("ConciergeAgent")
  .system("You are an intelligent voice concierge for a high-end restaurant. Your primary role is to assist with reservations and answer common inquiries.")
  .action(
    "book_table",
    "Book a restaurant reservation for a specific time and number of guests. Examples: 'I want to book a table for two tonight', 'Can I get a reservation for 7 PM on Saturday for four people?'",
    async (ctx) => {
      // In a real application, you'd integrate with a booking system here.
      // ctx.transcript would contain the user's full utterance.
      console.log(`Booking request received: ${ctx.transcript}`);
      return "Certainly, I've noted your request to book a table. Please provide your preferred time and number of guests.";
    }
  )
  .action(
    "check_menu",
    "Inquire about the menu or specific dishes. Examples: 'What's on the menu?', 'Do you have vegetarian options?', 'Tell me about your specials.'",
    async (ctx) => {
      console.log(`Menu inquiry received: ${ctx.transcript}`);
      return "Our menu features a delightful selection of seasonal dishes. We offer vegetarian, vegan, and gluten-free options. Would you like to hear about today's specials?";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. Could you please rephrase your request, or are you looking to book a table or inquire about the menu?");

async function testAgent() {
  console.log("--- Testing Agent Interactions ---");

  // Test case 1: Booking intent
  let reply1 = await agent.interact("I'd like to book a table for two this evening.");
  console.log(`User: I'd like to book a table for two this evening.`);
  console.log(`Agent: ${reply1}`); // Expected: Reply from 'book_table' action

  // Test case 2: Menu inquiry intent
  let reply2 = await agent.interact("What kind of food do you serve?");
  console.log(`User: What kind of food do you serve?`);
  console.log(`Agent: ${reply2}`); // Expected: Reply from 'check_menu' action

  // Test case 3: Unhandled intent, triggers fallback
  let reply3 = await agent.interact("What's the weather like today?");
  console.log(`User: What's the weather like today?`);
  console.log(`Agent: ${reply3}`); // Expected: Fallback message

  // Test case 4: Another booking intent, slightly different phrasing
  let reply4 = await agent.interact("Can I reserve a spot for four on Friday?");
  console.log(`User: Can I reserve a spot for four on Friday?`);
  console.log(`Agent: ${reply4}`); // Expected: Reply from 'book_table' action
}

testAgent();

In this example:

createAgent("ConciergeAgent") initializes your voice agent with a unique name.
.system(...) provides the overarching context and persona for your agent.
.action("book_table", "description", async (ctx) => { ... }) defines a specific, executable action. The second argument is a detailed natural language description of what this action handles, which is crucial for JEV similarity matching. This description helps Felona Voice understand a wide range of user phrasing that maps to book_table.
.fallback(...) specifies the response when no defined action matches the user's intent, ensuring a graceful degradation rather than a confused silence.

Felona Voice's deterministic routing means that as long as the user's intent semantically aligns with an .action's description, the correct function will be invoked. This eliminates the unpredictability and hallucinations common in pure LLM-driven systems while retaining the flexibility to understand natural language.

Furthermore, Felona Voice supports pluggable audio pipelines for various services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia. For local testing and deterministic routing, you don't even need external API keys, making development and iteration incredibly fast.

The Impact of Sub-10ms Decision Making

The ability to make conversational decisions in sub-10ms fundamentally changes the user experience. It enables:

Truly Duplex Conversations: Users don't have to wait for the bot to finish processing before speaking again, mirroring human-to-human interaction.
Highly Responsive Interfaces: Applications can react instantly to user commands, making voice control feel natural and intuitive.
New Interaction Paradigms: Real-time gaming, high-stakes customer service, and assistive technologies can leverage this speed for experiences previously impossible.

Felona Voice isn't just about speed; it's about reliability. By combining the semantic understanding of neural networks (via JEV) with the predictability of state machines (VoiceGraph), it offers a robust solution for building the next generation of voice AI applications.

Join the Felona Voice Community!

We believe Felona Voice represents a significant leap forward in voice AI development, offering a powerful, open-source solution to long-standing challenges. We invite you to explore its capabilities and become a part of our growing community.

🌟 Star the repository on GitHub: https://github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: https://felona-voice.mohitjoe.tech/docs

Start building your ultra-low-latency, hallucination-free voice agents today with Felona Voice!

This article was originally published on felona-voice.mohitjoe.tech.

Why 500ms Latency Kills Voice AI: Inside Felona Voice's Sub-10ms Neural Routing Engine

Felona Voice — Sun, 27 Sep 2026 15:25:30 +0000

Imagine talking to a voice assistant, and after every sentence, there's a noticeable, awkward pause. That frustrating 500ms to 1200ms delay isn't just an inconvenience; it's a fundamental flaw that kills the natural flow of conversation, making voice AI feel clunky and unintelligent. This is the inherent challenge of relying solely on large language models (LLMs) for conversational turn-taking.

At the heart of this problem is the token generation loop. Each time a user speaks, the audio is transcribed, sent to an LLM, the LLM processes the prompt, generates a response (token by token), and then that response is converted back into audio. This entire cycle, while impressive for its intelligence, is a sequential bottleneck that introduces unacceptable latency for real-time voice interactions.

The LLM Latency Trap: Why Traditional Voice AI Stumbles

Traditional voice AI applications often follow a pattern:

Speech-to-Text (STT): User speaks, audio is converted to text.
LLM Inference: The text and conversation history are sent to an LLM.
LLM Response Generation: The LLM deliberates, token-by-token, to formulate a reply or decide on an action. This is the primary source of latency, often ranging from 500ms to well over a second for complex prompts.
Action/Response: Based on the LLM's output, an action is triggered, or a Text-to-Speech (TTS) engine generates an audio reply.

While LLMs excel at understanding nuance and generating creative text, their generative nature is a liability for real-time decision-making in voice. The milliseconds add up, creating a disjointed experience that feels more like a walkie-talkie conversation than a natural human interaction. Furthermore, relying on LLMs for routing can lead to unpredictable behavior and hallucinations, as their responses aren't always deterministic.

Enter Felona Voice: Sub-10ms Neural Routing with Joint Embedding Vectors (JEV)

This is where Felona Voice, an open-source, ultra-low-latency voice agent framework for TypeScript, dramatically changes the game. Felona Voice tackles the latency problem head-on by separating the routing (deciding what to do next) from the generative aspects of AI. Its core innovation lies in using Joint Embedding Vectors (JEV) with stateful conversational transition graphs, which we call VoiceGraph.

Instead of sending every utterance to an LLM for interpretation and decision-making, Felona Voice pre-computes semantic embeddings for all defined actions and fallback phrases. When a user speaks, the input utterance is also converted into an embedding. Felona Voice then performs a lightning-fast similarity matching against its pre-computed JEVs.

This JEV-powered routing allows Felona Voice to decide the next action in an astonishing sub-10ms (typically around 5ms). This is not just faster; it's an order of magnitude faster than traditional LLM-based routing, effectively eliminating token latency and the associated conversational pauses. Because the routing is based on semantic similarity to predefined actions, it also offers zero hallucinations for decision-making.

How JEV Transforms Voice UX:

Feature	Traditional LLM-based Routing	Felona Voice (JEV-powered)
Decision Latency	500ms - 1200ms+	~5ms (Sub-10ms)
Token Latency	High (waiting for token generation)	Zero (no tokens generated for routing)
Hallucinations	Possible (LLM may misinterpret/invent)	Zero (deterministic similarity match)
Determinism	Low (LLM can vary responses)	High (predictable routing)
Resource Usage	High (constant LLM inference)	Low (pre-computed embeddings, fast match)
Conversational Flow	Disjointed, unnatural pauses	Instant, natural, seamless

Developer Experience: Fluent and Powerful

Felona Voice is built with developers in mind, offering a fluent builder API in TypeScript that makes defining complex conversational flows intuitive. You define your agent's persona, its actions, and its fallback behaviors, and Felona Voice handles the ultra-fast routing.

Let's look at a simple example:

import { createAgent } from "felona-voice";

// 1. Define your agent with a system prompt and actions
const conciergeAgent = createAgent("Concierge")
  .system("You are an intelligent voice concierge for a luxury hotel. Your goal is to assist guests with bookings and information.")
  .action(
    "book_table",
    "Book a restaurant reservation for a guest. Keywords: restaurant, table, reservation, dine.",
    async (ctx) => {
      // In a real app, you'd integrate with a booking system
      console.log(`Booking request received: ${ctx.input}`);
      return "Certainly, I can book a table for you. What time and how many people?";
    }
  )
  .action(
    "check_in",
    "Assist a guest with checking into the hotel. Keywords: check in, arrival, room key.",
    async (ctx) => {
      console.log(`Check-in request received: ${ctx.input}`);
      return "Welcome! Do you have a reservation number or a name?";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. Could you please rephrase or ask about booking a table or checking in?");

// 2. Interact with the agent
(async () => {
  console.log("User: Can I book a table for two tonight?");
  let reply = await conciergeAgent.interact("Can I book a table for two tonight?");
  console.log(`Agent: ${reply}`); // Agent: Certainly, I can book a table for you. What time and how many people?

  console.log("User: I'd like to check into my room.");
  reply = await conciergeAgent.interact("I'd like to check into my room.");
  console.log(`Agent: ${reply}`); // Agent: Welcome! Do you have a reservation number or a name?

  console.log("User: What's the weather like?");
  reply = await conciergeAgent.interact("What's the weather like?");
  console.log(`Agent: ${reply}`); // Agent: I'm sorry, I didn't quite catch that...
})();

Notice how each action is defined with a descriptive string that Felona Voice uses to generate the JEV. This string, combined with the system prompt, allows the agent to semantically understand the user's intent with incredible speed and accuracy.

Pluggable Audio Pipelines & Local Testing

Felona Voice isn't just about routing; it's a complete framework. It offers pluggable audio pipelines, allowing you to integrate seamlessly with various services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia. This flexibility ensures you can build voice agents tailored to your specific needs and infrastructure.

Crucially, for local testing and deterministic routing, zero external API keys are needed. You can rapidly prototype and test your conversational flows without incurring costs or relying on external services for the core routing logic. This makes development faster, more reliable, and more privacy-respecting.

The Future of Voice AI is Instant

The era of clunky, delayed voice interactions is coming to an end. Felona Voice represents a significant leap forward, offering a robust, open-source solution that prioritizes a natural, instant conversational experience. By leveraging Joint Embedding Vectors and VoiceGraph, it bypasses the inherent latency of LLM token generation for routing, delivering sub-10ms decision times that feel truly instantaneous.

For developers building the next generation of voice assistants, customer service bots, or interactive voice experiences, Felona Voice provides the foundational technology to create applications that are not just smart, but also genuinely pleasant and efficient to use.

Ready to build ultra-low-latency voice agents?

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.