DEV Community: Felona Voice

The Engineering Imperative: Sustainable Voice AI at Scale with Felona Voice's JEV

Felona Voice — Thu, 15 Oct 2026 17:36:15 +0000

In the rapidly evolving landscape of conversational AI, voice agents are becoming non-negotiable for superior customer experience and operational efficiency. Yet, the journey from proof-of-concept to a truly production-ready, scalable, and cost-effective voice AI system is fraught with challenges. Traditional approaches, heavily reliant on large language models (LLMs) for every conversational turn, often hit critical roadblocks: prohibitive costs, unacceptable latency, and unpredictable behavior.

This is where Felona Voice (https://github.com/mohitjoer/felona_voice) emerges as a game-changer. An open-source, ultra-low-latency voice agent framework built for TypeScript, Felona Voice introduces a paradigm shift with its Joint Embedding Vectors (JEV) and stateful conversational transition graphs (VoiceGraph). It's designed not just to work, but to scale sustainably, reliably, and economically in real-world enterprise environments.

The Production Challenge of Traditional Voice AI

Imagine deploying a voice agent that handles hundreds of thousands, or even millions, of calls per month. With traditional LLM-centric architectures, each user utterance typically triggers a cascade of events:

Speech-to-Text (STT): Convert audio to text.
LLM Inference: Send text to a cloud LLM API for intent recognition and response generation.
Text-to-Speech (TTS): Convert the LLM's text response back to audio.

While powerful for generative tasks, the LLM inference step is the bottleneck. It introduces significant latency (often 500ms-1200ms+ per turn) and incurs a token-based cost. At scale, these delays degrade user experience, and the costs skyrocket, making the system unsustainable.

Furthermore, LLMs, by their probabilistic nature, can "hallucinate" or deviate from expected conversational paths. For critical enterprise applications like banking, healthcare, or customer support, deterministic and predictable behavior isn't just a preference; it's a regulatory and operational imperative.

Felona Voice's Paradigm Shift: JEV for Sustainable Scale

Felona Voice tackles these challenges head-on by fundamentally rethinking how intent is recognized and actions are triggered. Instead of relying on an LLM for real-time decision-making, it leverages Joint Embedding Vectors (JEV) combined with a VoiceGraph.

Here's the core innovation:

Zero Token Latency: Instead of generating tokens, Felona Voice uses pre-computed JEVs to perform similarity matching. When a user speaks, their utterance is converted into an embedding, which is then compared against a library of pre-defined JEVs associated with your agent's actions within a VoiceGraph. This matching happens in sub-10ms (~5ms).
Zero Hallucinations: The VoiceGraph defines explicit, deterministic paths. JEV similarity matching ensures that the agent follows these predefined paths based on the closest intent. This means predictable outcomes, every single time – crucial for compliance and reliability.
In-Memory Efficiency: JEV matching is often an in-memory operation, drastically reducing external API calls for intent resolution, especially during high-volume conversational turns.

Developer Experience: Building with Felona Voice

Felona Voice prioritizes developer productivity with a fluent, intuitive TypeScript API. Defining a sophisticated voice agent is straightforward and expressive:

import { createAgent } from "felona-voice";

// Define a simple Felona Voice agent
const supportAgent = createAgent("CustomerSupport")
  .system("You are a helpful customer support agent for a SaaS company. You can assist with billing, technical issues, and general inquiries.")
  .action("check_billing", "Check my billing statement or current charges", async (ctx) => {
    // In a real application, you'd integrate with a billing API here
    const userId = ctx.get("userId"); // Example of context access
    return `Sure, I can help you check your billing. What's your account number?`;
  })
  .action("report_issue", "Report a technical issue or bug", async (ctx) => {
    return "I understand you're having a technical issue. Could you please describe it in more detail?";
  })
  .action("general_inquiry", "Ask a general question about the service", async (ctx) => {
    return "I'm happy to answer any general questions you have. How can I help?";
  })
  .fallback("I'm sorry, I didn't understand that. Can you please rephrase?");

// Simulate an interaction
async function runAgent() {
  console.log("User: Can you tell me about my latest bill?");
  let reply = await supportAgent.interact("Can you tell me about my latest bill?");
  console.log("Agent: " + reply);

  console.log("User: My dashboard is not loading.");
  reply = await supportAgent.interact("My dashboard is not loading.");
  console.log("Agent: " + reply);

  console.log("User: What's your favorite color?"); // Fallback example
  reply = await supportAgent.interact("What's your favorite color?");
  console.log("Agent: " + reply);
}

runAgent();

This example demonstrates how easily you can define actions with clear descriptions, allowing Felona Voice to map user utterances to the correct intent using JEVs. The fallback mechanism ensures graceful handling of out-of-scope requests.

The Unbeatable ROI: Cost & Latency Breakdown

For enterprise-grade voice AI, the choice of architecture has profound implications for both user experience and the bottom line. Felona Voice's JEV-driven approach delivers a compelling return on investment:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

Mathematically Cutting Costs by 90-95% (or More!)

Let's quantify the financial impact. Consider an enterprise with a voice agent handling 100,000 conversational turns per month. Each turn with a traditional LLM incurs a cost (e.g., for gpt-3.5-turbo or similar models). Let's assume a conservative average of $0.04 per turn for LLM inference alone (this doesn't even include STT/TTS costs).

Traditional LLM Cost: 100,000 turns/month * $0.04/turn = $4,000 per month

Now, let's look at Felona Voice. For intent resolution and action triggering, Felona Voice uses Joint Embedding Vectors (JEV). Once these embeddings are generated (a one-time or infrequent process), the real-time matching is an in-memory computation. This means the inference cost per turn for intent decision is effectively $0.00.

Felona Voice JEV Cost: 100,000 turns/month * $0.00/turn = $0 per month

This represents a 100% saving on LLM inference for intent resolution. Even if you still use an LLM for generative responses in specific action handlers, the vast majority of turns (e.g., navigating menus, confirming details) can be handled by JEV, drastically reducing your LLM token consumption by 90-95%. For a pure deterministic flow, the savings are absolute.

Scaling this to 500,000 turns per month:

Traditional LLM Cost: 500,000 turns/month * $0.04/turn = $20,000 per month
Felona Voice JEV Cost: $0 per month for intent resolution.

The savings are undeniable and directly translate to a sustainable, viable business model for voice AI at scale.

Reliability and Compliance: The Zero-Hallucination Advantage

Beyond cost, the deterministic nature of Felona Voice's VoiceGraph with JEV matching is paramount for enterprise use cases. In sectors like finance or healthcare, a voice agent cannot afford to misinterpret a user's intent or generate an off-script response. Felona Voice's architecture guarantees that your agent will always follow the defined conversational logic, eliminating the risk of LLM hallucinations and ensuring compliance with strict industry regulations.

Beyond the Basics: Pluggable Audio Pipelines & Local Testing

Felona Voice isn't just about core logic; it's a complete framework for production environments. It offers pluggable audio pipelines, supporting WebSockets, WebRTC, and integrations with leading STT/TTS providers like Deepgram, Whisper, ElevenLabs, and Cartesia. Crucially, for development and local testing, you don't need any external API keys for deterministic routing, making the inner development loop incredibly fast and efficient.

Conclusion: Build Sustainable Voice AI Today

The future of scalable, cost-effective, and reliable voice AI is here. Felona Voice empowers developers and enterprises to move beyond the limitations of traditional LLM-centric architectures, offering an innovative JEV-powered approach that delivers sub-10ms latency, zero hallucinations, and dramatic cost reductions. It's the engineering imperative for any organization serious about deploying high-performance voice agents at scale.

Ready to build the next generation of voice experiences?

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.

Mastering Enterprise Voice AI: Deterministic Flows with Felona Voice & TypeScript

Felona Voice — Mon, 12 Oct 2026 21:33:46 +0000

Building voice AI agents for enterprise applications comes with a unique set of challenges. Organizations demand not just intelligence, but also unwavering predictability, minimal latency, and stringent cost control. Traditional approaches, often reliant on large language models (LLMs) for every conversational turn, frequently fall short, introducing unacceptable delays, unpredictable responses, and skyrocketing operational costs.

This is where Felona Voice steps in. As an open-source, ultra-low-latency voice agent framework for TypeScript, Felona Voice is engineered from the ground up to address these enterprise imperatives. It offers a revolutionary approach to voice AI, prioritizing deterministic control, sub-10ms intent resolution, and a developer experience that is both powerful and intuitive.

The Enterprise Voice AI Dilemma: Latency, Cost, and Hallucinations

Imagine a customer support agent, a banking assistant, or a healthcare concierge. These systems cannot afford to be slow, expensive, or, worst of all, make up information. Yet, this is the reality for many LLM-centric voice solutions:

High Latency: Each conversational turn requires sending user speech to an ASR (Automatic Speech Recognition) service, then to an LLM for processing, and finally to a TTS (Text-to-Speech) service. This round trip often incurs 800ms to 1800ms or more of delay, making natural, rapid-fire conversations impossible and hindering barge-in capabilities.
Unpredictable Costs: Every LLM interaction translates to token usage, which accumulates rapidly. For businesses handling thousands or hundreds of thousands of calls monthly, these costs can quickly spiral into five or even six figures.
Hallucination Risk: LLMs are designed for creative, probabilistic text generation. While powerful, this makes them inherently unreliable for structured, compliance-critical conversations where factual accuracy and deterministic outcomes are paramount.

These challenges highlight a fundamental mismatch: generative LLMs excel at open-ended creativity, but enterprise voice agents often require precise, stateful, and predictable execution of predefined tasks.

Felona Voice: A Paradigm Shift with JEV and VoiceGraph

Felona Voice redefines voice AI architecture by moving beyond the limitations of auto-regressive LLM loops for core intent resolution. Its innovation lies in the powerful combination of Joint Embedding Vectors (JEV) and stateful conversational transition graphs, known as VoiceGraph.

Joint Embedding Vectors (JEV) for Sub-10ms Intent Resolution

Instead of sending every user utterance to an LLM for natural language understanding (NLU), Felona Voice leverages JEV. Here's how it works:

Pre-computation: During agent definition, all possible user intents, actions, and fallback phrases are converted into numerical embedding vectors (JEVs).
Runtime Matching: When a user speaks, their utterance is quickly converted into an embedding vector. Felona Voice then performs a highly efficient similarity matching against the pre-computed JEVs.
Instant Decision: This vector similarity matching process happens in sub-10ms, typically around 5ms. Because it's an in-memory operation, it completely bypasses the network latency and processing time associated with external LLM APIs.

This approach eliminates token latency and the associated costs for intent resolution, making Felona Voice agents incredibly fast and efficient.

VoiceGraph for Deterministic Conversational Flows

While JEV handles rapid intent recognition, VoiceGraph provides the structure and predictability. It's a stateful conversational transition graph that dictates how your agent should respond based on the detected intent and the current conversational context.

Predictability: Unlike the probabilistic nature of LLMs, VoiceGraph ensures deterministic outcomes. If a specific intent is matched, a predefined action is triggered, guaranteeing compliance and consistent user experiences.
Flexibility: Despite being deterministic, VoiceGraph is not rigid like hardcoded static graphs. The dynamic nature of JEV similarity matching allows for natural variations in human speech to be correctly mapped to the appropriate state transition.
Zero Hallucinations: Since responses and actions are tied to explicit definitions within the VoiceGraph, there's no risk of the agent generating irrelevant or incorrect information.

Building Predictable Agents with Felona Voice: A Developer's Perspective

Felona Voice offers a fluent, TypeScript-native builder API that makes defining complex conversational flows surprisingly simple. Let's look at a practical example of a concierge agent:

import { createAgent } from "felona-voice";

const conciergeAgent = createAgent("ConciergeService")
  .system("You are an intelligent voice concierge, ready to assist with hotel amenities and bookings.")
  .action(
    "book_table",
    "Book a restaurant reservation for the guest. The user might say 'I want to book a table' or 'Reserve a spot at the restaurant'.",
    async (ctx) => {
      // In a real application, you'd integrate with a booking API here.
      console.log(`User requested to book a table. Context: ${JSON.stringify(ctx.slots)}`);
      return "Certainly, I can help with that. What's your preferred time and number of guests?";
    }
  )
  .action(
    "request_room_service",
    "Order room service, for example 'Can I get some food delivered to my room?' or 'I'd like to order breakfast'.",
    async (ctx) => {
      console.log(`User requested room service. Context: ${JSON.stringify(ctx.slots)}`);
      return "Of course. What would you like to order, and what's your room number?";
    }
  )
  .action(
    "check_in_status",
    "Inquire about check-in status, e.g., 'What's my check-in status?' or 'Is my room ready?'.",
    async (ctx) => {
      console.log(`User checking in. Context: ${JSON.stringify(ctx.slots)}`);
      return "Please provide your reservation number or last name to check your status.";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. Can you please rephrase your request?");

// Simulate user interactions
async function runConversation() {
  console.log("User: Can I book a table?");
  let reply = await conciergeAgent.interact("Can I book a table?");
  console.log("Agent: " + reply);

  console.log("User: I want to order some food.");
  reply = await conciergeAgent.interact("I want to order some food.");
  console.log("Agent: " + reply);

  console.log("User: Is my room ready?");
  reply = await conciergeAgent.interact("Is my room ready?");
  console.log("Agent: " + reply);

  console.log("User: Tell me a joke."); // Unhandled intent
  reply = await conciergeAgent.interact("Tell me a joke.");
  console.log("Agent: " + reply);
}

runConversation();

In this example, the system prompt sets the agent's persona. Each action defines a specific capability, complete with a descriptive phrase that helps the JEV model understand user intent and an asynchronous function to execute the action. The fallback handles anything the agent doesn't understand. This clear, declarative structure ensures predictable behavior.

The Unbeatable ROI: Cost Savings and Hyper-Responsiveness

The architectural choice of JEV and VoiceGraph over a continuous LLM loop for intent resolution translates directly into significant business advantages:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

Let's quantify the cost savings. Consider an enterprise handling 50,000 customer calls per month, with each call averaging 5 conversational turns. This amounts to 250,000 conversational turns monthly.

Traditional LLM-based Agent: At an average cost of $0.04 per turn for LLM inference, the monthly cost for intent resolution alone would be 250,000 turns * $0.04/turn = $10,000.
Felona Voice Agent: With JEV-powered in-memory intent matching, the cost per turn for intent resolution is effectively $0.00. This completely eliminates the token-based LLM billing for the core decision-making process.

This represents a 100% saving on the most frequent and costly part of conversational AI – intent resolution. While you might still use specialized LLMs for complex generative responses in certain action contexts, the foundational routing and decision-making are free and instant. This dramatically cuts infrastructure bills, especially at scale, leading to a 90-95% reduction in overall operating costs for voice agent interactions.

Beyond cost, the sub-10ms latency is transformative. It allows for truly fluid, human-like conversations, where users can barge in seamlessly without frustrating delays. This responsiveness is critical for customer satisfaction and operational efficiency.

Pluggable Audio Pipelines and Local Development

Felona Voice isn't just about the core logic; it's a complete framework. It offers pluggable audio pipelines supporting WebSockets, WebRTC, and integrations with services like Deepgram, Whisper, ElevenLabs, and Cartesia. This flexibility means you can choose the best-of-breed ASR and TTS providers for your needs.

Crucially, for local development and testing, Felona Voice requires zero external API keys for deterministic routing. This significantly speeds up the development cycle, allowing engineers to iterate rapidly without incurring cloud costs or being dependent on external services until deployment.

Conclusion: Embrace Deterministic Voice AI

The future of enterprise voice AI is deterministic, hyper-responsive, and cost-efficient. Felona Voice, with its innovative JEV and VoiceGraph architecture, provides the TypeScript toolkit needed to build agents that meet these stringent demands.

Move beyond the limitations of slow, expensive, and unpredictable LLM-only solutions. Empower your applications with voice agents that are fast, reliable, and truly enterprise-grade.

Ready to Build the Next Generation of Voice Agents?

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice

📦 Install via npm: npm install felona-voice

📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.

Mastering Predictable Voice AI: Building Enterprise Agents with Felona Voice & TypeScript

Felona Voice — Thu, 08 Oct 2026 19:09:26 +0000

In the rapidly evolving landscape of artificial intelligence, voice agents have emerged as a critical interface for customer interactions, support, and automation. Yet, the promise of seamless, intelligent voice experiences often collides with the harsh realities of latency, cost, and unpredictability when relying solely on large language models (LLMs) for core conversational logic.

Traditional LLM-driven voice agents, while powerful for open-ended conversations, frequently struggle with the demands of structured, intent-driven interactions. Imagine a banking bot needing to confirm a transaction, or a healthcare assistant scheduling an appointment. In these scenarios, milliseconds matter, accuracy is paramount, and hallucinations are unacceptable. This is precisely where Felona Voice (https://github.com/mohitjoer/felona_voice) steps in, offering a revolutionary, open-source framework for building ultra-low-latency, deterministic voice agents in TypeScript.

The Unpredictable Nature of LLM-Centric Voice AI

Before diving into Felona Voice's innovation, let's understand the pain points of current voice AI architectures that lean heavily on LLMs for every conversational turn:

Crippling Latency: Each turn requires audio transcription, sending a prompt to an LLM API, waiting for token generation (Time To First Token + subsequent tokens), and then text-to-speech synthesis. This entire loop often takes 800ms to 1800ms or more. For natural human conversation, anything above 300ms feels unnatural and disruptive, leading to awkward pauses and frustrated users.
Soaring Costs: LLM APIs are typically billed per token. For high-volume applications, a few cents per turn quickly escalates into tens of thousands or even hundreds of thousands of dollars monthly, just for intent resolution and response generation.
Hallucinations and Unpredictability: LLMs are probabilistic by nature. While fantastic for creative text generation, this inherent probabilistic behavior means they can sometimes 'hallucinate' or deviate from expected responses, leading to compliance risks and inconsistent user experiences, especially in regulated industries like banking or healthcare.
Complex Developer Experience: Building robust, error-proof conversational flows with LLMs often devolves into intricate prompt engineering, guardrails, and retries, making the development process cumbersome and the outcomes hard to guarantee.

Felona Voice: A Paradigm Shift with JEV and VoiceGraph

Felona Voice rethinks the core architecture of voice agents, moving beyond the limitations of auto-regressive LLM token generation for structured interactions. Its core innovation lies in the combination of Joint Embedding Vectors (JEV) and stateful conversational transition graphs, dubbed VoiceGraph.

Instead of sending an entire prompt to an LLM for every decision, Felona Voice leverages pre-computed Joint Embedding Vectors. When a user speaks, their utterance is transcribed and immediately converted into an embedding. This embedding is then compared against a set of action-specific JEVs in sub-10ms (~5ms), directly within memory. This similarity matching instantly determines the user's intent and triggers the appropriate action defined in your VoiceGraph.

Key Advantages of this approach:

Ultra-Low Latency: Intent resolution happens in milliseconds, eliminating the multi-second lag of LLM inference. This enables instant barge-in and fluid, natural conversations.
Zero Hallucinations: Decisions are based on deterministic similarity matching against predefined JEVs and graph transitions, not probabilistic token generation. This ensures 0% hallucination risk, vital for compliance and reliability.
Zero Token Cost for Intent Resolution: Since intent matching is an in-memory operation, it incurs no per-token API costs, dramatically reducing operational expenses.
Robust State Management: VoiceGraph provides a clear, auditable flow for your agent's behavior, making it easy to design, debug, and maintain complex conversational logic.

Developer Empowerment with a Fluent TypeScript API

Felona Voice is built for developers, by developers. Its fluent TypeScript builder API makes crafting sophisticated voice agents intuitive and powerful. You define your agent's personality, available actions, and fallback behaviors with clear, concise code.

import { createAgent } from "felona-voice";

// Define a simple voice agent for a concierge service
const conciergeAgent = createAgent("ConciergeBot")
  .system("You are an intelligent voice concierge for a luxury hotel. Your goal is to assist guests with bookings and information.")
  .action("book_table", "Book a restaurant reservation for a guest", async (ctx) => {
    // In a real application, this would interact with a booking system
    console.log(`Booking request received for: ${ctx.utterance}`);
    return "Certainly, I can help with that. For what time and how many people?";
  })
  .action("check_in", "Assist with guest check-in process", async (ctx) => {
    console.log(`Check-in request received for: ${ctx.utterance}`);
    return "Welcome! Do you have a reservation name or number?";
  })
  .action("ask_weather", "Provide local weather information", async (ctx) => {
    // Simulate fetching weather data
    console.log(`Weather inquiry: ${ctx.utterance}`);
    return "The current weather in our location is sunny with a temperature of 25 degrees Celsius.";
  })
  .fallback("I'm sorry, I didn't quite catch that. Could you please rephrase or tell me how I can assist you today?");

// Simulate an interaction
async function runInteraction() {
  console.log("User: Can I book a table for dinner?");
  let reply = await conciergeAgent.interact("Can I book a table for dinner?");
  console.log(`Agent: ${reply}`); // Output: Certainly, I can help with that. For what time and how many people?

  console.log("User: What's the weather like?");
  reply = await conciergeAgent.interact("What's the weather like?");
  console.log(`Agent: ${reply}`); // Output: The current weather in our location is sunny with a temperature of 25 degrees Celsius.

  console.log("User: Tell me a joke.");
  reply = await conciergeAgent.interact("Tell me a joke.");
  console.log(`Agent: ${reply}`); // Output: I'm sorry, I didn't quite catch that. Could you please rephrase or tell me how I can assist you today?
}

runInteraction();

This example demonstrates how easily you can define intents (book_table, check_in, ask_weather) and their corresponding actions. The fallback mechanism ensures graceful handling of out-of-scope requests. Furthermore, Felona Voice offers pluggable audio pipelines for seamless integration with services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia, giving you full control over your audio stack.

Crucially, for local development and testing, Felona Voice allows for deterministic routing without needing any external API keys, making the development loop incredibly fast and efficient.

Quantifying the Impact: Cost & Latency Breakdown

Let's put the benefits into perspective with a direct comparison:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

The Mathematical Impact on Your Infrastructure Bill

Consider an enterprise handling 500,000 customer calls per month, with each call averaging 5 conversational turns. This translates to 2.5 million conversational turns monthly.

Traditional LLM Approach: At an average cost of $0.04 per LLM inference turn, your monthly bill for just the LLM reasoning would be $0.04 * 2,500,000 = $100,000.
Felona Voice Approach: By leveraging in-memory JEV matching for intent resolution, Felona Voice incurs zero token cost for these critical decision points. The primary costs shift to transcription and text-to-speech (TTS), which are necessary for any voice agent. By removing the per-turn LLM inference cost, Felona Voice effectively slashes the variable operational expenditure for intent resolution by 95% or more, transforming a six-figure monthly bill into virtually nothing for the core logic.

This isn't just an optimization; it's a fundamental shift that enables massive scalability without proportional increases in operational costs for your core conversational intelligence.

Why Predictability Matters for Enterprise Voice AI

For enterprise applications, predictability isn't a luxury; it's a necessity. In sectors like finance, healthcare, or critical customer support, every interaction must be reliable, compliant, and consistent. The deterministic nature of Felona Voice's JEV and VoiceGraph architecture provides:

Compliance & Auditing: Clear, auditable conversational paths ensure adherence to regulatory requirements.
Consistent CX: Every customer receives the same high-quality, reliable experience, building trust and satisfaction.
Reduced Operational Overhead: Less time spent debugging LLM prompts or dealing with unexpected responses means your team can focus on innovation, not mitigation.

Get Started with Felona Voice Today!

Felona Voice empowers you to build the next generation of voice agents: fast, reliable, cost-effective, and entirely predictable. Move beyond the chaos of LLM-centric voice AI and embrace a framework designed for enterprise-grade performance and developer satisfaction.

Ready to build your ultra-low-latency voice agent?

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.

Mastering Stateful Voice AI: Dynamic Graphs with Felona Voice's JEV

Felona Voice — Tue, 06 Oct 2026 14:54:04 +0000

The promise of intelligent voice agents has long captivated developers and businesses alike. Imagine a customer support bot that understands nuance, a concierge that books services instantly, or an assistant that guides users through complex processes without frustrating delays. Yet, the reality often falls short: sluggish responses, unpredictable behavior, and escalating cloud bills plague traditional voice AI solutions.

Today, we're diving deep into Felona Voice, an open-source TypeScript framework that redefines what's possible in real-time conversational AI. At its core, Felona Voice introduces a groundbreaking approach: VoiceGraph, a stateful conversational transition graph powered by Joint Embedding Vectors (JEV). This powerful combination delivers sub-10ms intent resolution, zero hallucinations, and unprecedented cost savings, making truly dynamic and deterministic voice AI a reality.

The Dilemma: LLMs vs. Static Graphs in Voice AI

Traditional voice AI architectures often find themselves at a crossroads, forced to choose between two imperfect paradigms:

Pure Large Language Models (LLMs) in a Loop: These offer incredible flexibility and natural language understanding. However, they come with significant drawbacks for real-time voice applications:
- High Latency: Each conversational turn requires a new API call, leading to 500ms-1200ms+ per response. This breaks natural human conversation flow, leading to awkward silences and user frustration.
- High Cost: Every token generated by an LLM incurs a cost. At scale (thousands to millions of calls), these costs skyrocket, turning a promising solution into a budget nightmare.
- Hallucinations: LLMs are probabilistic text generators. While powerful, this means they can generate factually incorrect or off-script responses, which is unacceptable for enterprise applications requiring precision and compliance (e.g., banking, healthcare).
Hardcoded, Static Conversational Graphs (IVRs): These systems are deterministic and cost-effective. They define rigid paths and expected user inputs. While reliable, they are:
- Inflexible: Any deviation from the script, natural language variations, or barge-in attempts can break the flow, leading to a frustrating experience.
- Difficult to Scale: Managing complex, branching logic in static graphs quickly becomes a maintenance nightmare.

Felona Voice elegantly solves this dilemma by introducing VoiceGraph and leveraging Joint Embedding Vectors (JEV), offering the best of both worlds: the flexibility of natural language understanding within a deterministic, high-performance, and cost-efficient structure.

VoiceGraph + JEV: The Secret Sauce for Dynamic, Stateful Voice AI

Imagine a conversational agent that can understand the intent behind a user's words with lightning speed, then use that understanding to navigate a predefined, yet flexible, conversational flow. That's the power of Felona Voice.

VoiceGraph provides the structured backbone. It's a stateful conversational transition graph where each node represents a specific conversational state or action, and edges define valid transitions. This ensures deterministic behavior and prevents hallucinations, as the agent always operates within a known, controlled domain.

Joint Embedding Vectors (JEV) are the game-changer for dynamic traversal within this graph. Instead of sending user input to an LLM for interpretation, Felona Voice transforms both user utterances and predefined action descriptions into JEVs. When a user speaks, their utterance's JEV is compared against the JEVs of all valid actions/intents within the current VoiceGraph state using ultra-fast similarity matching. This entire process happens in sub-10ms (~5ms).

How it works:

Semantic Encoding: Each action within your VoiceGraph (e.g., book_table, check_status) is semantically encoded into a JEV.
User Utterance: When a user speaks, their speech is transcribed, and the text is also encoded into a JEV.
Ultra-Fast Matching: Felona Voice performs an in-memory cosine similarity comparison between the user's utterance JEV and the JEVs of all available actions/intents at that specific point in the conversation.
Deterministic Transition: Based on the highest similarity score, the agent deterministically triggers the corresponding action and transitions to the next state in the VoiceGraph.

This synergy means your agent is both dynamic (interpreting natural language via JEV) and deterministic (operating within the defined VoiceGraph structure). It can handle barge-in, context switching, and complex multi-turn conversations with unparalleled speed and accuracy, without ever generating an off-script response.

Developer Experience: Fluent & Powerful

Felona Voice is designed for developers, offering a fluent builder API in TypeScript that makes agent creation intuitive and powerful.

import { createAgent } from "felona-voice";

// Define your agent
const agent = createAgent("Concierge")
  .system("You are an intelligent voice concierge for a luxury hotel.")
  .action("book_table", "Book a restaurant reservation", async (ctx) => {
    // In a real app, you'd integrate with a booking system
    console.log(`Booking table for: ${ctx.utterance}`);
    return "Certainly, I've noted your request to book a table. What time and for how many people?";
  })
  .action("check_in", "Check into your room", async (ctx) => {
    console.log(`Checking in: ${ctx.utterance}`);
    return "Welcome! Do you have a reservation number or a last name?";
  })
  .action("request_valet", "Request valet service for your car", async (ctx) => {
    console.log(`Valet requested: ${ctx.utterance}`);
    return "Valet service has been dispatched to your location. Please wait outside.";
  })
  .fallback("I'm sorry, I didn't understand that. How can I assist you with hotel services?");

// Interact with the agent
(async () => {
  let reply = await agent.interact("Can I book a table?");
  console.log(`Agent: ${reply}`); // Agent: Certainly, I've noted your request to book a table. What time and for how many people?

  reply = await agent.interact("I'd like to check in.");
  console.log(`Agent: ${reply}`); // Agent: Welcome! Do you have a reservation number or a last name?

  reply = await agent.interact("I need my car from the valet.");
  console.log(`Agent: ${reply}`); // Agent: Valet service has been dispatched to your location. Please wait outside.

  reply = await agent.interact("Tell me a joke.");
  console.log(`Agent: ${reply}`); // Agent: I'm sorry, I didn't understand that. How can I assist you with hotel services?
})();

Felona Voice also boasts pluggable audio pipelines, supporting WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia, ensuring flexibility for any deployment scenario. Crucially, for local development and deterministic routing, zero external API keys are needed, allowing for rapid prototyping and testing.

The Cost & Latency Revolution: Why JEV Trumps LLMs

This is where Felona Voice delivers truly transformative value for businesses. The architectural shift from auto-regressive LLM token generation to in-memory JEV similarity matching fundamentally alters the cost and performance profile of voice agents at scale.

Let's break down the numbers:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

The Mathematical Advantage:

Consider an enterprise handling 100,000 conversational turns per month. With a traditional LLM-based agent, even at a conservative $0.03 per turn, your monthly inference bill would be $3,000.

Traditional LLM Cost: 100,000 turns * $0.03/turn = $3,000 / month

With Felona Voice, the JEV similarity matching occurs entirely in-memory or on a local vector database. Once the embeddings are generated (a one-time or infrequent cost), the subsequent matching operations incur zero per-turn inference cost. The only costs are for the underlying compute (which is minimal for ~5ms operations) and any external ASR/TTS services, which you'd pay for regardless. For the intent resolution itself, the cost is effectively $0.00 per turn.

Felona Voice JEV Cost: 100,000 turns * $0.00/turn = $0.00 / month (for intent resolution)

This translates to a 90-95% reduction in infrastructure bills specifically for the core intelligence of your voice agent when scaling to thousands or even millions of calls. The savings are massive and directly impact your bottom line, transforming voice AI from a cost center into a powerful, efficient tool.

Real-World Impact: Enterprise-Grade Control

For industries like banking, healthcare, and high-volume customer support, the benefits of Felona Voice are immense:

Compliance & Accuracy: Zero hallucination risk ensures that agents always adhere to predefined scripts and never provide incorrect information, crucial for regulated environments.
Superior User Experience: Sub-10ms latency eliminates awkward pauses, allowing for fluid, natural conversations, even with complex requests or barge-ins.
Scalability & Cost-Efficiency: Drastically reduced per-turn costs mean you can scale your voice operations without fear of ballooning cloud bills.
Dynamic, Yet Controlled: VoiceGraph provides the necessary structure and control, while JEV allows for the dynamic interpretation of natural language, striking the perfect balance for complex enterprise workflows.

Felona Voice empowers you to build sophisticated voice agents that are not only intelligent and responsive but also predictable, compliant, and incredibly cost-effective. It's time to move beyond the limitations of legacy systems and embrace the future of conversational AI.

Get Started with Felona Voice Today!

Ready to build ultra-low-latency, deterministic, and cost-efficient voice agents? Felona Voice is your answer.

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice

📦 Install via npm: npm install felona-voice

📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

Join the open-source movement and revolutionize your voice AI applications with Felona Voice!

This article was originally published on felona-voice.mohitjoe.tech.

Beyond Generative AI: Why Streaming LLMs Fail Structured Voice, and JEV Wins

Felona Voice — Thu, 01 Oct 2026 18:42:13 +0000

The promise of conversational AI has always been grand: natural, fluid interactions that feel human. Yet, for all the advancements in large language models (LLMs), a glaring gap persists in real-time, structured voice AI. We've been trying to fit a square peg in a round hole, using auto-regressive, token-generating LLMs for tasks that demand precision, speed, and determinism. It's time to acknowledge the fundamental mismatch and embrace a new paradigm: Felona Voice, powered by Joint Embedding Vectors (JEV) and stateful conversational transition graphs (VoiceGraph).

The LLM Paradigm: A Costly Latency Trap for Structured Voice

Traditional voice agents, often built around a loop of ASR -> LLM -> TTS, face inherent limitations when applied to structured conversational flows like booking a table, checking an account balance, or routing a customer service call. Here's why the streaming LLM approach fundamentally struggles:

Crippling Latency: LLMs are auto-regressive, meaning they generate responses token by token. This process involves:
- Time To First Token (TTFT): The initial delay before any output begins (often 200-500ms).
- Token Generation Time: Subsequent tokens are generated sequentially. For a typical conversational turn, this translates to an 850ms to 1,800ms+ delay for the LLM to process input and formulate a response (even if it's just a decision for the next action). This latency breaks natural human conversation, making barge-in difficult and creating awkward silences.
Astronomical Costs at Scale: Every interaction with an LLM, no matter how simple the intent resolution, consumes tokens. At scale, this becomes a significant infrastructure cost. Imagine a contact center handling tens of thousands or hundreds of thousands of calls a month, each with multiple conversational turns. Every turn means paying for input and output tokens, even for simple 'yes/no' or intent confirmation. These costs quickly escalate into tens of thousands of dollars monthly for intent resolution alone, let alone full generative responses.
Hallucinations and Lack of Determinism: LLMs are probabilistic by nature. While fantastic for creative generation, this probabilistic approach leads to 'hallucinations' – generating plausible but incorrect or off-script responses. For enterprise applications in banking, healthcare, or critical customer support, non-deterministic behavior and the risk of generating inaccurate information are simply unacceptable. Compliance and user trust demand predictable outcomes.
Heavy Network Dependency: Relying on external cloud LLM APIs means constant network round-trips for every decision. This adds to latency and introduces points of failure, making local testing and development cumbersome without API keys.

The Felona Voice Revolution: Precision with Joint Embedding Vectors (JEV)

Felona Voice (https://github.com/mohitjoer/felona_voice) introduces a paradigm shift by recognizing that structured voice AI doesn't need generative reasoning for intent resolution. Instead, it leverages Joint Embedding Vectors (JEV) and VoiceGraph to deliver unparalleled performance:

Joint Embedding Vectors (JEV): Instead of generating text, Felona Voice embeds both the user's spoken input and the predefined actions/intents of your agent into a shared vector space. When a user speaks, their utterance is embedded, and a lightning-fast similarity search determines the closest matching action. This completely bypasses the slow, token-by-token generation process of LLMs for intent resolution.
VoiceGraph: Stateful Conversational Transition Graphs: Felona Voice uses a VoiceGraph – a deterministic, stateful graph that defines the conversational flow. JEV similarity matching then guides the agent through this graph, ensuring predictable and precise transitions. This eliminates hallucinations entirely because the agent's actions are explicitly defined and matched, not probabilistically generated.

The Result: Unprecedented Speed and Efficiency

By leveraging JEV for intent resolution, Felona Voice achieves:

Ultra-low Latency: Intent decisions are made in sub-10ms (~5ms). This is an order of magnitude faster than LLMs, enabling instant barge-in and truly natural, real-time conversational turns.
Zero Cost for Intent Resolution: JEV matching is an in-memory operation. There are no tokens to generate, no external API calls for intent. This means $0.00 per conversational turn for intent resolution.
Zero Hallucinations: With a deterministic VoiceGraph guided by precise JEV matching, your agent will always follow defined paths, ensuring compliance and reliability.
Local-First Development: Test and develop your voice agents locally without needing any external API keys for deterministic routing.

Felona Voice in Action: TypeScript Simplicity

Felona Voice is built for developers, offering a fluent TypeScript API that makes building sophisticated voice agents intuitive. Let's see how simple it is to define an agent with precise actions:

import { createAgent } from "felona-voice";

interface MyAgentContext {
  userName?: string;
  tableSize?: number;
}

const agent = createAgent<MyAgentContext>("RestaurantConcierge")
  .system("You are an intelligent voice concierge for 'The Golden Spoon' restaurant. Your primary goal is to book tables and answer FAQs.")
  .action(
    "book_table",
    "Book a restaurant reservation. I want to reserve a table. Can I get a booking?",
    async (ctx, input) => {
      // In a real app, you'd extract parameters from 'input' or prompt for them
      if (!ctx.tableSize) {
        return "Certainly! For how many people would you like to book a table?";
      }
      // Simulate booking logic
      console.log(`Booking a table for ${ctx.tableSize} people.`);
      return `Okay, I've booked a table for ${ctx.tableSize} people. Is there anything else?`;
    }
  )
  .action(
    "check_hours",
    "What are your operating hours? When are you open?",
    async () => "The Golden Spoon is open Monday to Saturday, from 5 PM to 11 PM."
  )
  .action(
    "greet_user",
    "Hello. Hi there. Good morning. Good afternoon. Good evening.",
    async (ctx) => {
      if (ctx.userName) {
        return `Welcome back, ${ctx.userName}! How can I assist you today?`;
      }
      return "Hello! Welcome to The Golden Spoon. How may I help you?";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. How can I assist you today?");

// Example interaction
async function runConversation() {
  let currentContext: MyAgentContext = {};

  let reply = await agent.interact("Hi", currentContext);
  console.log(`Agent: ${reply.response}`); // Agent: Hello! Welcome to The Golden Spoon. How may I help you?

  currentContext = { ...currentContext, ...reply.context };

  reply = await agent.interact("I'd like to book a table", currentContext);
  console.log(`Agent: ${reply.response}`); // Agent: Certainly! For how many people would you like to book a table?

  currentContext = { ...currentContext, ...reply.context };

  reply = await agent.interact("For four people", currentContext);
  currentContext.tableSize = 4; // Manually update context for this example
  reply = await agent.interact("", currentContext); // Re-interact to trigger action with updated context
  console.log(`Agent: ${reply.response}`); // Agent: Okay, I've booked a table for 4 people. Is there anything else?

  currentContext = { ...currentContext, ...reply.context };

  reply = await agent.interact("What are your hours?", currentContext);
  console.log(`Agent: ${reply.response}`); // Agent: The Golden Spoon is open Monday to Saturday, from 5 PM to 11 PM.
}

runConversation();

Felona Voice also offers pluggable audio pipelines for WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia, ensuring flexibility for your production environment.

The Stark Reality: Traditional Voice Agent (LLM Loop) vs. Felona Voice (JEV + VoiceGraph)

Let's put the architectural differences into perspective:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching
Developer Experience	Complex prompt engineering, slow iteration	Fluent builder API, fast local testing

The Mathematical Truth: 90-95% Cost Reduction

Let's quantify the cost savings. Consider an enterprise-level voice agent handling 50,000 calls per month, with an average of 5 conversational turns per call for intent resolution. This amounts to 250,000 intent decisions per month.

Traditional LLM Approach: At an average cost of $0.04 per turn (a conservative estimate for a mix of input/output tokens), your monthly bill for intent resolution alone would be:
250,000 turns * $0.04/turn = $10,000 per month
Felona Voice Approach: With JEV-powered intent resolution, the cost per turn is $0.00. This means your monthly bill for intent resolution is:
250,000 turns * $0.00/turn = $0 per month

This represents a 100% saving on the intent resolution component of your voice agent's operation. When you factor in the additional costs of LLM inference for generative responses (which Felona Voice can still integrate for, but only when truly needed), the overall savings on infrastructure bills can easily exceed 90-95% for systems that primarily rely on structured interactions.

This isn't just an optimization; it's a fundamental shift that makes high-volume, real-time voice AI economically viable and technically superior for structured use cases.

The Future of Structured Voice AI is Here

The era of forcing generative LLMs into every voice AI problem, regardless of suitability, is coming to an end. For structured, real-time, and mission-critical voice interactions, the architectural shortcomings of streaming LLMs are simply too great. Felona Voice offers a purpose-built, highly performant, and cost-effective alternative by leveraging Joint Embedding Vectors and VoiceGraph.

It's time to build voice agents that truly feel real-time, are reliably deterministic, and won't break the bank. Embrace the future of voice AI with Felona Voice.

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice

📦 Install via npm: npm install felona-voice

📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.

Beyond Generative LLMs: Architecting Sub-10ms Voice AI with Felona Voice

Felona Voice — Mon, 28 Sep 2026 20:03:21 +0000

The promise of conversational AI has always been a natural, instantaneous interaction. Yet, for many developers building voice agents, the reality has often been a frustrating battle against latency, unpredictable responses, and escalating cloud costs. We've largely relied on large language models (LLMs) to power these interactions, leveraging their incredible generative capabilities. But what if the very architecture of generative LLMs is fundamentally misaligned with the requirements of real-time, structured voice interactions?

Enter Felona Voice, an open-source, ultra-low-latency voice agent framework for TypeScript that dares to challenge this paradigm. It redefines how we build voice AI by moving beyond the auto-regressive token generation of LLMs, introducing a groundbreaking architecture powered by Joint Embedding Vectors (JEV) and stateful conversational transition graphs (VoiceGraph).

The LLM Bottleneck: Why Generative Models Falter in Voice AI

Traditional voice agents often follow a simple, yet costly and slow, loop:

Speech-to-Text (ASR): User speaks, audio is converted to text.
LLM Inference: The text prompt is sent to an LLM, which generates a response.
Text-to-Speech (TTS): The LLM's text response is converted back to audio and played to the user.

While LLMs excel at open-ended conversations and creative text generation, this architecture introduces several critical challenges for structured voice agents:

Crippling Latency: The LLM inference step is a major bottleneck. There's the "Time To First Token" (TTFT) and then the subsequent token generation. For a typical conversational turn, this can easily add 500ms to 1200ms or more per turn to the interaction. This often leads to awkward pauses, broken conversational flow, and prevents natural barge-in capabilities.
Exploding Costs: Every interaction with an LLM incurs token costs. For applications scaling to thousands or hundreds of thousands of calls per month, these micro-transactions quickly accumulate into significant cloud bills. Even small context windows add up.
Hallucinations and Undeterministic Behavior: LLMs are probabilistic by nature. While powerful, this means they can sometimes generate irrelevant, incorrect, or "hallucinated" responses. For enterprise applications in banking, healthcare, or critical customer support, this lack of deterministic control is a non-starter.
Network Dependency: Constant API calls to remote LLMs introduce network overhead and potential points of failure.

These issues highlight an architectural mismatch: we're using a powerful, general-purpose text generator for a task that often requires precise intent recognition and deterministic action execution.

Felona Voice's Paradigm Shift: JEV and VoiceGraph

Felona Voice tackles these challenges head-on by fundamentally re-thinking the core of voice agent intelligence. Instead of generating a response from scratch, Felona Voice focuses on matching user intent to predefined actions using Joint Embedding Vectors (JEV).

Here's how it works:

Intent Embedding: When you define an action in Felona Voice, you provide a description. This description (and potentially other examples) is converted into a numerical vector (an embedding). User utterances are also converted into embeddings.
JEV Similarity Matching: Instead of sending a prompt to an LLM, Felona Voice takes the user's utterance embedding and compares it against the embeddings of all available actions/intents in your VoiceGraph. This similarity matching is an incredibly fast, in-memory operation.
VoiceGraph for Stateful Transitions: The VoiceGraph provides the crucial context. It's a stateful conversational transition graph that guides the agent. Based on the current state and the highest JEV similarity match, Felona Voice deterministically decides the next action.

This architecture delivers mind-blowing performance:

Sub-10ms Intent Resolution: Because it's an in-memory vector similarity lookup, Felona Voice resolves user intent in ~5ms. This is orders of magnitude faster than any LLM-based approach, enabling instant barge-in and truly fluid conversations.
Zero Hallucinations: The decision-making is deterministic. If an intent matches, the defined action is executed. If not, the fallback is triggered. There's no probabilistic text generation to introduce errors.
Massive Cost Reduction: Since the core intent resolution doesn't require constant LLM API calls, the operational cost for this crucial step drops to virtually zero.

Developer Experience: Fluent and Powerful

Felona Voice is designed with developers in mind, offering a fluent builder API in TypeScript:

import { createAgent } from "felona-voice";

// Define your voice agent
const agent = createAgent("Concierge")
  .system("You are an intelligent voice concierge. Your goal is to assist users with hotel-related requests.")
  .action(
    "book_table",
    "Book a restaurant reservation; I want to reserve a table; Can I get a booking?",
    async (ctx) => {
      console.log("User wants to book a table.");
      // In a real app, you'd integrate with a booking system
      return "Certainly, I can help you book a table. What time and for how many people?";
    }
  )
  .action(
    "request_room_service",
    "Order room service; I need food delivered to my room; Can I get something to eat?",
    async (ctx) => {
      console.log("User wants room service.");
      return "Of course, what would you like to order from room service?";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. How can I assist you with your hotel stay?");

// Simulate user interaction
async function runInteraction() {
  console.log("Agent: " + (await agent.interact("Hello, I'd like to reserve a table for dinner.")));
  console.log("Agent: " + (await agent.interact("I need some food delivered to my room.")));
  console.log("Agent: " + (await agent.interact("Tell me a joke."))); // This should hit fallback
}

runInteraction();

This API allows you to define actions, their descriptions (which form the basis for JEV matching), and their associated logic. Felona Voice also offers:

Pluggable Audio Pipelines: Easily integrate with WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia for end-to-end audio handling.
Zero External API Keys for Local Testing: For deterministic routing and local development, you don't need any external LLM API keys, streamlining your development workflow.

The Numbers Don't Lie: Cost and Latency Breakdown

This architectural shift translates directly into tangible benefits for your bottom line and user experience:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

The Math Behind 90-95% Cost Reduction

Let's consider an enterprise-scale voice agent handling 100,000 conversational turns per month. If each LLM inference for intent resolution costs, conservatively, $0.04:

Traditional LLM Cost: 100,000 turns * $0.04/turn = $4,000 per month (just for LLM inference, excluding ASR/TTS).

With Felona Voice, the core intent resolution using JEV is an in-memory operation, incurring $0.00 in API fees. While you still pay for ASR and TTS services (which Felona Voice integrates with), the most expensive and slowest part of the LLM loop—the generative inference—is eliminated for structured intent handling.

This means that for the intent resolution component, you achieve a 100% cost saving. When factoring in the overall cost of ASR, TTS, and intent resolution, Felona Voice can easily reduce your monthly infrastructure bills by 90-95% compared to a heavily LLM-dependent architecture, especially for high-volume, structured interactions.

Real-World Impact: Where Felona Voice Shines

This architecture is a game-changer for applications requiring:

High-Volume Customer Support: Instantly route calls, answer FAQs, and complete transactions without frustrating delays.
Interactive IVRs: Replace rigid, number-based IVRs with natural language interfaces that are fast and intuitive.
Transactional Voice Agents: Securely guide users through purchases, bookings, or data entry with deterministic outcomes.
Voice Control for Devices: Offer snappy, reliable voice commands for smart homes, industrial interfaces, or automotive systems.

Conclusion: The Future of Voice AI is Fast, Deterministic, and Open Source

Felona Voice represents a significant leap forward in voice AI. By recognizing the architectural limitations of generative LLMs for structured, real-time interactions and instead leveraging the power of Joint Embedding Vectors and deterministic VoiceGraphs, it delivers unparalleled speed, cost efficiency, and reliability.

It's time to build voice agents that truly feel natural and responsive, without breaking the bank or sacrificing control. Join the movement towards a more efficient and effective voice AI future.

Get Started with Felona Voice Today!

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice

📦 Install via npm: npm install felona-voice

📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.

Voice AI Latency & Cost: Felona Voice Cuts LLM Inference by 95% with JEV

Felona Voice — Sun, 27 Sep 2026 15:34:59 +0000

The promise of natural, low-latency voice AI has long been tantalizing, yet often constrained by the realities of cost and performance. Traditional voice agents, relying heavily on large language models (LLMs) for conversational intelligence, frequently fall short of real-time expectations and rack up significant infrastructure bills. But what if there was a better way? An open-source, TypeScript-native framework that delivers sub-10ms intent routing with zero token cost and zero hallucinations?

Enter Felona Voice, a groundbreaking open-source project that’s redefining the economics and performance of voice AI. At its core, Felona Voice leverages Joint Embedding Vectors (JEV) to achieve neural routing that’s not just fast, but virtually free for structured conversational turns. Let's dive deep into why this innovation is a game-changer for your next voice AI project.

The Latency & Cost Conundrum of Traditional LLM-First Voice Agents

Most modern voice agents follow a similar, expensive, and slow pipeline:

Speech-to-Text (STT): User's speech is converted to text (e.g., Deepgram, Whisper).
LLM Inference: The transcribed text is sent to an LLM (e.g., GPT-4o, Claude 3 Opus) to understand intent, generate a response, or decide the next action.
Text-to-Speech (TTS): The LLM's text output is converted back into natural-sounding speech (e.g., ElevenLabs, Cartesia).

While powerful for open-ended dialog, this LLM-centric approach introduces several critical bottlenecks:

High Latency: Each step involves network round-trips and significant processing time. STT, LLM inference (especially for larger models or longer responses), and TTS can easily accumulate to 500ms, 800ms, or even 1200ms+ per conversational turn. This makes for a frustrating, unnatural user experience.
Exorbitant Costs: LLM API calls are priced per token. A single conversational turn, including input and output tokens, can cost anywhere from $0.02 to $0.06 or more, depending on the model and token count. These costs scale linearly and rapidly become prohibitive for high-volume applications.
Hallucination Risk: LLMs are probabilistic text generators. While incredibly capable, they can

This article was originally published on felona-voice.mohitjoe.tech.

Unlock Sub-10ms Voice AI: Real-time Agents with TypeScript & Felona Voice

Felona Voice — Sun, 27 Sep 2026 15:32:24 +0000

The dream of truly real-time, human-like voice agents has long been hampered by one critical bottleneck: latency. Traditional voice AI systems, heavily reliant on large language models (LLMs) for conversational understanding and response generation, introduce significant delays. We're talking 500ms to 1200ms or more per conversational turn, creating an awkward, unnatural experience that frustrates users and limits adoption.

But what if you could eliminate that latency? What if your voice agent could understand intent and react in mere milliseconds, without sacrificing accuracy or breaking the bank? Enter Felona Voice – an innovative, open-source TypeScript framework designed to build ultra-low-latency voice agents that feel genuinely real-time.

The Latency & Cost Problem with Traditional LLM Voice Agents

Let's face it: LLMs are powerful, but they're not built for instant, sub-10ms decision-making in high-volume voice interactions. Here's why traditional approaches struggle:

High Latency: Every conversational turn often involves an API call to a remote LLM. This introduces network latency, processing time on the LLM server, and token generation time. The combined effect easily pushes response times into the hundreds of milliseconds, creating noticeable pauses.
Exorbitant Costs: Each LLM API call costs money, typically per token. At scale, with thousands or millions of interactions, these costs quickly skyrocket, making real-time voice AI a luxury few can afford.
Hallucination Risk: LLMs are probabilistic by nature. While incredibly creative, they can sometimes generate inaccurate or off-topic responses, leading to frustrating user experiences and a lack of control for developers.
Rigid Static Graphs: On the other end of the spectrum, hardcoded decision trees are fast and deterministic but too brittle. They break down the moment a user deviates slightly from the expected script, leading to dead ends and frustrated users.

Felona Voice: The Sub-10ms Revolution Powered by JEVs and VoiceGraph

Felona Voice tackles these challenges head-on with a revolutionary approach: Joint Embedding Vectors (JEV) combined with intelligent, stateful conversational transition graphs, which we call VoiceGraph.

Instead of sending every user utterance to an LLM for interpretation, Felona Voice leverages JEV similarity matching. Here's the magic:

Intent Mapping: Your agent's defined actions and their descriptions are transformed into high-dimensional embedding vectors (JEVs).
Real-time Matching: When a user speaks, their transcribed utterance is also converted into a JEV. Felona Voice then performs ultra-fast similarity matching between the user's utterance JEV and your agent's action JEVs.
Sub-10ms Decisions: This similarity matching operation is incredibly efficient, happening in sub-10ms (typically ~5ms). It's an in-memory, local computation that requires zero external API calls for intent decision-making.
Zero Hallucinations: Because decisions are based on deterministic similarity matching against predefined actions, there's no room for LLM hallucinations. Your agent will always follow the intended conversational flow.
Dynamic VoiceGraph: The VoiceGraph isn't a rigid, static decision tree. It's a stateful graph that uses JEV similarity to dynamically decide the next best action, allowing for natural transitions and graceful handling of off-script speech, far beyond what traditional static graphs can offer.

The Stark Reality: Traditional vs. Felona Voice

Let's put it into perspective with a direct comparison:

Metric	Traditional Voice Agent (LLM Loop)	Felona Voice (JEV + VoiceGraph)
Intent Decision Latency	850ms – 1,800ms	~5ms (Sub-10ms)
Inference Cost / Turn	$0.02 – $0.06+ / turn	$0.00 / turn
Hallucination Risk	High (probabilistic text tokens)	0% (deterministic transition graph)
Network Dependency	Requires constant cloud LLM API	Local/In-memory embedding matching

Cutting Infrastructure Bills by 90-95%

The cost savings with Felona Voice are not just significant; they're transformative. Let's do the math for the intent decision component alone:

Consider an application scaling to 1,000,000 conversational turns per month. If each turn involves an LLM call for intent detection at a conservative average of $0.02 per turn:

Traditional LLM Agent: 1,000,000 turns * $0.02/turn = $20,000 per month for intent decision alone.
Felona Voice Agent: 1,000,000 turns * $0.00/turn = $0 per month for intent decision.

This translates to a 100% reduction in LLM costs specifically for intent detection. While you still need ASR (Speech-to-Text) and TTS (Text-to-Speech) services (which Felona Voice integrates seamlessly), by eliminating the most expensive and slowest part of the conversational loop – the LLM-based intent inference – you can realistically cut your overall voice AI infrastructure bills by 90-95% when scaling to thousands or millions of calls. This is a game-changer for businesses looking to deploy cost-effective, high-performance voice agents.

Developer Experience: Fluent and Powerful in TypeScript

Felona Voice is built with developers in mind, offering an incredibly fluent and intuitive API in TypeScript. You can define complex conversational flows with ease:

import { createAgent } from "felona-voice";

const agent = createAgent("ConciergeBot")
  .system("You are an intelligent voice concierge for a luxury hotel.")
  .action("book_room", "Book a hotel room", async (ctx) => {
    const roomType = ctx.get("roomType");
    if (roomType) {
      return `Booking a ${roomType} for you.`;
    } else {
      return "What type of room are you looking for?";
    }
  })
  .action("check_inquiry", "Check reservation status or details", async (ctx) => {
    const reservationId = ctx.get("reservationId");
    if (reservationId) {
      return `Checking details for reservation ${reservationId}.`;
    } else {
      return "Please provide your reservation ID.";
    }
  })
  .fallback("I'm sorry, I couldn't understand that. Can you please rephrase or ask about booking a room or checking a reservation?");

async function runAgent() {
  console.log("User: I want to book a suite.");
  let reply = await agent.interact("I want to book a suite.");
  console.log("Bot: ", reply); // Expected: "Booking a suite for you."

  console.log("User: What's my reservation status?");
  reply = await agent.interact("What's my reservation status?");
  console.log("Bot: ", reply); // Expected: "Please provide your reservation ID."

  console.log("User: My reservation ID is 12345.");
  reply = await agent.interact("My reservation ID is 12345.");
  console.log("Bot: ", reply); // Expected: "Checking details for reservation 12345."

  console.log("User: Tell me a joke.");
  reply = await agent.interact("Tell me a joke.");
  console.log("Bot: ", reply); // Expected: "I'm sorry, I couldn't understand that. Can you please rephrase or ask about booking a room or checking a reservation?"
}

runAgent();

Key developer benefits include:

Pluggable Audio Pipelines: Felona Voice is designed for flexibility, allowing you to integrate with various audio processing services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia, ensuring you can use your preferred ASR/TTS providers.
Zero External API Keys for Local Testing: Develop and test your agent locally without needing to hit costly external LLM APIs. Intent routing is deterministic and works entirely in-memory.
TypeScript Native: Leverage the power of TypeScript for type safety, better tooling, and improved maintainability.

Getting Started with Felona Voice

Ready to build your own ultra-low-latency voice agent? Getting started is incredibly simple:

First, install the package via npm:

npm install felona-voice

Then, create your first agent:

import { createAgent } from "felona-voice";

const simpleAgent = createAgent("GreetingBot")
  .system("You are a friendly bot that greets users.")
  .action("greet_user", "Say hello to the user", async (ctx) => {
    const name = ctx.get("name") || "there";
    return `Hello ${name}! How can I help you today?`;
  })
  .fallback("I didn't quite catch that. Could you say hello?");

async function testSimpleAgent() {
  console.log("User: Hi, my name is Alice.");
  let reply = await simpleAgent.interact("Hi, my name is Alice.");
  console.log("Bot: ", reply); // Expected: "Hello Alice! How can I help you today?"

  console.log("User: Just saying hi.");
  reply = await simpleAgent.interact("Just saying hi.");
  console.log("Bot: ", reply); // Expected: "Hello there! How can I help you today?"

  console.log("User: What's the weather?");
  reply = await simpleAgent.interact("What's the weather?");
  console.log("Bot: ", reply); // Expected: "I didn't quite catch that. Could you say hello?"
}

testSimpleAgent();

This basic example demonstrates how quickly you can define actions and their corresponding responses, setting up a robust conversational flow without the typical LLM overhead.

The Future of Real-Time Voice AI is Here

Felona Voice isn't just another framework; it's a paradigm shift. By moving intent decision-making from slow, expensive, and probabilistic LLM calls to ultra-fast, deterministic, in-memory JEV similarity matching, Felona Voice unlocks true real-time voice AI.

Imagine customer service bots that respond instantly, gaming NPCs that react without delay, or smart assistants that feel genuinely present. This is the future Felona Voice enables, today. It's about building production-ready voice AI that is not only performant and reliable but also incredibly cost-effective at scale.

Get Involved and Start Building!

Join the revolution in real-time voice AI. Felona Voice is open-source, community-driven, and ready for your contributions.

🌟 Star the repository on GitHub: https://github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: https://felona-voice.mohitjoe.tech/docs

Start building your sub-10ms voice agent today and experience the difference true real-time AI makes!

This article was originally published on felona-voice.mohitjoe.tech.

Deterministic Voice AI: Sub-10ms & Hallucination-Free with Felona Voice

Felona Voice — Sun, 27 Sep 2026 15:28:38 +0000

The promise of natural, real-time conversational AI has long been tempered by the realities of latency and unpredictable responses. Traditional voice bots, often reliant on large language models (LLMs) for decision-making, grapple with response times upwards of 500ms and the notorious issue of 'hallucinations' – generating plausible but incorrect information. Hardcoded conversational flows, while deterministic, are too rigid for the organic nature of human speech, breaking down at the slightest deviation. This is where Felona Voice steps in, offering a revolutionary approach to building ultra-low-latency, hallucination-free voice agents.

Felona Voice, an open-source TypeScript framework, redefines the landscape of voice AI by introducing VoiceGraph, a stateful conversational transition graph powered by Joint Embedding Vectors (JEV). This innovative combination allows for neural, yet deterministic, routing of conversational turns in sub-10ms, effectively eliminating the trade-off between flexibility and reliability.

The Unpredictability of Pure Prompt-Based Voice Bots

Before diving into Felona Voice's solution, let's understand the challenges with existing paradigms:

Pure LLM-Based Routing: While powerful for understanding context and generating nuanced responses, LLMs introduce significant latency. Each conversational turn requires processing, which can take 500ms to 1200ms or more. This delay creates an unnatural, disjointed user experience. Furthermore, LLMs are known to 'hallucinate,' fabricating answers or taking unexpected conversational paths, making them unsuitable for mission-critical applications requiring precision.
Hardcoded Static Graphs: These offer determinism and low latency, as every path is explicitly defined. However, they are inherently inflexible. Any deviation from the script, a common occurrence in natural human conversation, can break the flow, leading to frustrating dead ends or generic fallback responses. They lack the intelligence to interpret intent beyond exact keyword matches.

Both approaches fall short of delivering the seamless, reliable, and real-time voice interactions users expect.

Felona Voice's Innovation: VoiceGraph with Neural Transitions

Felona Voice addresses these limitations head-on with its core innovation: VoiceGraph driven by Joint Embedding Vectors (JEV). Imagine a sophisticated state machine where transitions between states aren't just hardcoded rules, but intelligent, neural decisions made in milliseconds.

Joint Embedding Vectors (JEV): The Brain Behind the Speed

At the heart of Felona Voice's ultra-low latency is the concept of Joint Embedding Vectors. Instead of relying on token-by-token LLM inference for every decision, Felona Voice pre-computes semantic embeddings for all possible actions and user intents defined within your agent. When a user speaks, their utterance is quickly embedded into the same vector space. Felona Voice then performs a lightning-fast similarity match between the user's intent embedding and the embeddings of all available actions.

This similarity matching allows Felona Voice to decide the next action in approximately 5ms. This is not just faster; it's a paradigm shift. Zero token latency means no waiting for LLMs to generate text or process complex prompts for routing decisions.

VoiceGraph: Stateful and Deterministic Conversations

VoiceGraph acts as the architectural blueprint for your conversational flow. It's a state machine where each state represents a point in the conversation, and transitions are governed by the JEV similarity matching. This provides the best of both worlds:

Determinism: By mapping user intent directly to predefined actions via JEV, Felona Voice ensures that the agent always responds predictably. No more hallucinations or unexpected tangents.
Flexibility: Unlike rigid static graphs, VoiceGraph's transitions are "neural." They understand semantic meaning, allowing for natural variations in user speech to trigger the correct action without explicit keyword matching.
Statefulness: The VoiceGraph maintains the conversational state, enabling complex multi-turn interactions while always knowing where it is in the flow.

Let's compare the approaches:

Feature	Pure LLM Routing	Hardcoded Static Graph	Felona Voice (JEV + VoiceGraph)
Decision Latency	High (500ms - 1200ms+)	Low (hardcoded lookups)	Ultra-low (~5ms)
Hallucinations	Frequent, unpredictable	None (if matched)	None (deterministic)
Flexibility	High (but unpredictable)	Low (rigid, breaks easily)	High (neural understanding)
Determinism	Low	High	High
Cost	High (token usage)	Low	Low (zero token latency)
Developer Effort	Prompt engineering	Extensive rule definition	Fluent API, intent-driven

Building Hallucination-Free Agents with Felona Voice's Fluent API

Felona Voice is designed with developer experience at its core. Its fluent builder API in TypeScript makes defining complex conversational agents intuitive and powerful. You can define system instructions, specific actions, and fallback behaviors with ease.

To get started, simply install the package:

npm install felona-voice

Here’s a practical example of building a simple concierge agent that can book a table or provide general assistance:

import { createAgent } from "felona-voice";

const agent = createAgent("ConciergeAgent")
  .system("You are an intelligent voice concierge for a high-end restaurant. Your primary role is to assist with reservations and answer common inquiries.")
  .action(
    "book_table",
    "Book a restaurant reservation for a specific time and number of guests. Examples: 'I want to book a table for two tonight', 'Can I get a reservation for 7 PM on Saturday for four people?'",
    async (ctx) => {
      // In a real application, you'd integrate with a booking system here.
      // ctx.transcript would contain the user's full utterance.
      console.log(`Booking request received: ${ctx.transcript}`);
      return "Certainly, I've noted your request to book a table. Please provide your preferred time and number of guests.";
    }
  )
  .action(
    "check_menu",
    "Inquire about the menu or specific dishes. Examples: 'What's on the menu?', 'Do you have vegetarian options?', 'Tell me about your specials.'",
    async (ctx) => {
      console.log(`Menu inquiry received: ${ctx.transcript}`);
      return "Our menu features a delightful selection of seasonal dishes. We offer vegetarian, vegan, and gluten-free options. Would you like to hear about today's specials?";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. Could you please rephrase your request, or are you looking to book a table or inquire about the menu?");

async function testAgent() {
  console.log("--- Testing Agent Interactions ---");

  // Test case 1: Booking intent
  let reply1 = await agent.interact("I'd like to book a table for two this evening.");
  console.log(`User: I'd like to book a table for two this evening.`);
  console.log(`Agent: ${reply1}`); // Expected: Reply from 'book_table' action

  // Test case 2: Menu inquiry intent
  let reply2 = await agent.interact("What kind of food do you serve?");
  console.log(`User: What kind of food do you serve?`);
  console.log(`Agent: ${reply2}`); // Expected: Reply from 'check_menu' action

  // Test case 3: Unhandled intent, triggers fallback
  let reply3 = await agent.interact("What's the weather like today?");
  console.log(`User: What's the weather like today?`);
  console.log(`Agent: ${reply3}`); // Expected: Fallback message

  // Test case 4: Another booking intent, slightly different phrasing
  let reply4 = await agent.interact("Can I reserve a spot for four on Friday?");
  console.log(`User: Can I reserve a spot for four on Friday?`);
  console.log(`Agent: ${reply4}`); // Expected: Reply from 'book_table' action
}

testAgent();

In this example:

createAgent("ConciergeAgent") initializes your voice agent with a unique name.
.system(...) provides the overarching context and persona for your agent.
.action("book_table", "description", async (ctx) => { ... }) defines a specific, executable action. The second argument is a detailed natural language description of what this action handles, which is crucial for JEV similarity matching. This description helps Felona Voice understand a wide range of user phrasing that maps to book_table.
.fallback(...) specifies the response when no defined action matches the user's intent, ensuring a graceful degradation rather than a confused silence.

Felona Voice's deterministic routing means that as long as the user's intent semantically aligns with an .action's description, the correct function will be invoked. This eliminates the unpredictability and hallucinations common in pure LLM-driven systems while retaining the flexibility to understand natural language.

Furthermore, Felona Voice supports pluggable audio pipelines for various services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia. For local testing and deterministic routing, you don't even need external API keys, making development and iteration incredibly fast.

The Impact of Sub-10ms Decision Making

The ability to make conversational decisions in sub-10ms fundamentally changes the user experience. It enables:

Truly Duplex Conversations: Users don't have to wait for the bot to finish processing before speaking again, mirroring human-to-human interaction.
Highly Responsive Interfaces: Applications can react instantly to user commands, making voice control feel natural and intuitive.
New Interaction Paradigms: Real-time gaming, high-stakes customer service, and assistive technologies can leverage this speed for experiences previously impossible.

Felona Voice isn't just about speed; it's about reliability. By combining the semantic understanding of neural networks (via JEV) with the predictability of state machines (VoiceGraph), it offers a robust solution for building the next generation of voice AI applications.

Join the Felona Voice Community!

We believe Felona Voice represents a significant leap forward in voice AI development, offering a powerful, open-source solution to long-standing challenges. We invite you to explore its capabilities and become a part of our growing community.

🌟 Star the repository on GitHub: https://github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: https://felona-voice.mohitjoe.tech/docs

Start building your ultra-low-latency, hallucination-free voice agents today with Felona Voice!

This article was originally published on felona-voice.mohitjoe.tech.

Why 500ms Latency Kills Voice AI: Inside Felona Voice's Sub-10ms Neural Routing Engine

Felona Voice — Sun, 27 Sep 2026 15:25:30 +0000

Imagine talking to a voice assistant, and after every sentence, there's a noticeable, awkward pause. That frustrating 500ms to 1200ms delay isn't just an inconvenience; it's a fundamental flaw that kills the natural flow of conversation, making voice AI feel clunky and unintelligent. This is the inherent challenge of relying solely on large language models (LLMs) for conversational turn-taking.

At the heart of this problem is the token generation loop. Each time a user speaks, the audio is transcribed, sent to an LLM, the LLM processes the prompt, generates a response (token by token), and then that response is converted back into audio. This entire cycle, while impressive for its intelligence, is a sequential bottleneck that introduces unacceptable latency for real-time voice interactions.

The LLM Latency Trap: Why Traditional Voice AI Stumbles

Traditional voice AI applications often follow a pattern:

Speech-to-Text (STT): User speaks, audio is converted to text.
LLM Inference: The text and conversation history are sent to an LLM.
LLM Response Generation: The LLM deliberates, token-by-token, to formulate a reply or decide on an action. This is the primary source of latency, often ranging from 500ms to well over a second for complex prompts.
Action/Response: Based on the LLM's output, an action is triggered, or a Text-to-Speech (TTS) engine generates an audio reply.

While LLMs excel at understanding nuance and generating creative text, their generative nature is a liability for real-time decision-making in voice. The milliseconds add up, creating a disjointed experience that feels more like a walkie-talkie conversation than a natural human interaction. Furthermore, relying on LLMs for routing can lead to unpredictable behavior and hallucinations, as their responses aren't always deterministic.

Enter Felona Voice: Sub-10ms Neural Routing with Joint Embedding Vectors (JEV)

This is where Felona Voice, an open-source, ultra-low-latency voice agent framework for TypeScript, dramatically changes the game. Felona Voice tackles the latency problem head-on by separating the routing (deciding what to do next) from the generative aspects of AI. Its core innovation lies in using Joint Embedding Vectors (JEV) with stateful conversational transition graphs, which we call VoiceGraph.

Instead of sending every utterance to an LLM for interpretation and decision-making, Felona Voice pre-computes semantic embeddings for all defined actions and fallback phrases. When a user speaks, the input utterance is also converted into an embedding. Felona Voice then performs a lightning-fast similarity matching against its pre-computed JEVs.

This JEV-powered routing allows Felona Voice to decide the next action in an astonishing sub-10ms (typically around 5ms). This is not just faster; it's an order of magnitude faster than traditional LLM-based routing, effectively eliminating token latency and the associated conversational pauses. Because the routing is based on semantic similarity to predefined actions, it also offers zero hallucinations for decision-making.

How JEV Transforms Voice UX:

Feature	Traditional LLM-based Routing	Felona Voice (JEV-powered)
Decision Latency	500ms - 1200ms+	~5ms (Sub-10ms)
Token Latency	High (waiting for token generation)	Zero (no tokens generated for routing)
Hallucinations	Possible (LLM may misinterpret/invent)	Zero (deterministic similarity match)
Determinism	Low (LLM can vary responses)	High (predictable routing)
Resource Usage	High (constant LLM inference)	Low (pre-computed embeddings, fast match)
Conversational Flow	Disjointed, unnatural pauses	Instant, natural, seamless

Developer Experience: Fluent and Powerful

Felona Voice is built with developers in mind, offering a fluent builder API in TypeScript that makes defining complex conversational flows intuitive. You define your agent's persona, its actions, and its fallback behaviors, and Felona Voice handles the ultra-fast routing.

Let's look at a simple example:

import { createAgent } from "felona-voice";

// 1. Define your agent with a system prompt and actions
const conciergeAgent = createAgent("Concierge")
  .system("You are an intelligent voice concierge for a luxury hotel. Your goal is to assist guests with bookings and information.")
  .action(
    "book_table",
    "Book a restaurant reservation for a guest. Keywords: restaurant, table, reservation, dine.",
    async (ctx) => {
      // In a real app, you'd integrate with a booking system
      console.log(`Booking request received: ${ctx.input}`);
      return "Certainly, I can book a table for you. What time and how many people?";
    }
  )
  .action(
    "check_in",
    "Assist a guest with checking into the hotel. Keywords: check in, arrival, room key.",
    async (ctx) => {
      console.log(`Check-in request received: ${ctx.input}`);
      return "Welcome! Do you have a reservation number or a name?";
    }
  )
  .fallback("I'm sorry, I didn't quite catch that. Could you please rephrase or ask about booking a table or checking in?");

// 2. Interact with the agent
(async () => {
  console.log("User: Can I book a table for two tonight?");
  let reply = await conciergeAgent.interact("Can I book a table for two tonight?");
  console.log(`Agent: ${reply}`); // Agent: Certainly, I can book a table for you. What time and how many people?

  console.log("User: I'd like to check into my room.");
  reply = await conciergeAgent.interact("I'd like to check into my room.");
  console.log(`Agent: ${reply}`); // Agent: Welcome! Do you have a reservation number or a name?

  console.log("User: What's the weather like?");
  reply = await conciergeAgent.interact("What's the weather like?");
  console.log(`Agent: ${reply}`); // Agent: I'm sorry, I didn't quite catch that...
})();

Notice how each action is defined with a descriptive string that Felona Voice uses to generate the JEV. This string, combined with the system prompt, allows the agent to semantically understand the user's intent with incredible speed and accuracy.

Pluggable Audio Pipelines & Local Testing

Felona Voice isn't just about routing; it's a complete framework. It offers pluggable audio pipelines, allowing you to integrate seamlessly with various services like WebSockets, WebRTC, Deepgram, Whisper, ElevenLabs, and Cartesia. This flexibility ensures you can build voice agents tailored to your specific needs and infrastructure.

Crucially, for local testing and deterministic routing, zero external API keys are needed. You can rapidly prototype and test your conversational flows without incurring costs or relying on external services for the core routing logic. This makes development faster, more reliable, and more privacy-respecting.

The Future of Voice AI is Instant

The era of clunky, delayed voice interactions is coming to an end. Felona Voice represents a significant leap forward, offering a robust, open-source solution that prioritizes a natural, instant conversational experience. By leveraging Joint Embedding Vectors and VoiceGraph, it bypasses the inherent latency of LLM token generation for routing, delivering sub-10ms decision times that feel truly instantaneous.

For developers building the next generation of voice assistants, customer service bots, or interactive voice experiences, Felona Voice provides the foundational technology to create applications that are not just smart, but also genuinely pleasant and efficient to use.

Ready to build ultra-low-latency voice agents?

🌟 Star the repository on GitHub: github.com/mohitjoer/felona_voice
📦 Install via npm: npm install felona-voice
📖 Explore full documentation: felona-voice.mohitjoe.tech/docs

This article was originally published on felona-voice.mohitjoe.tech.