DEV Community: JITENDRA KUMAR SINGH

Build a Real-Time Sales Coaching AI Agent with OpenAI Realtime API + LangGraph

JITENDRA KUMAR SINGH — Sun, 21 Jun 2026 07:35:03 +0000

Sales reps lose deals in the moments they can't see themselves — talking over the prospect, missing a buying signal, going silent when objections hit. A coach whispering in their ear would fix most of that. Most companies can't afford one per rep. An AI agent listening to the live call can be that coach.

Here's how to build one using the OpenAI Realtime API for low-latency audio understanding and LangGraph to orchestrate the coaching logic.

Why This Stack

OpenAI Realtime API — streams audio in and text/audio out over a WebSocket, with latency low enough to feel live instead of batched.
LangGraph — gives you a stateful graph instead of a single prompt loop, so "detect objection → check playbook → generate coaching tip → suppress duplicate tips" becomes explicit, debuggable nodes instead of one giant system prompt hoping for the best.

A single LLM call can transcribe and react. It can't reliably track call state (are we in discovery? pricing? closing?), avoid repeating the same tip twice, or escalate only when it matters. That's an orchestration problem — which is exactly what LangGraph is for.

Architecture

Mic/Call Audio
     ↓
OpenAI Realtime API (streaming transcription + voice activity detection)
     ↓
LangGraph State Machine
  ├── transcript_node      → accumulates rolling transcript
  ├── stage_classifier_node → discovery / pitch / objection / pricing / closing
  ├── signal_detector_node  → buying signals, objections, talk-time ratio
  ├── coach_node            → generates a short, actionable tip (or stays silent)
  └── dedupe_node           → suppresses repeat tips within a time window
     ↓
Coaching tip → rep's screen (text overlay or quiet TTS in their earpiece)

The key design decision: the coach node should be allowed to say nothing. A coaching agent that fires a tip every 10 seconds is noise the rep will mute in week one.

Step 1: Stream Audio into the Realtime API

import asyncio
import websockets
import json

REALTIME_URL = "wss://api.openai.com/v1/realtime?model=gpt-realtime"

async def stream_call_audio(audio_chunks):
    async with websockets.connect(
        REALTIME_URL,
        extra_headers={"Authorization": f"Bearer {OPENAI_API_KEY}"}
    ) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "modalities": ["text"],
                "input_audio_transcription": {"model": "whisper-1"}
            }
        }))

        async for chunk in audio_chunks:
            await ws.send(json.dumps({
                "type": "input_audio_buffer.append",
                "audio": chunk  # base64-encoded PCM16
            }))

        async for message in ws:
            event = json.loads(message)
            if event["type"] == "conversation.item.input_audio_transcription.completed":
                yield event["transcript"]

This gives you a live stream of transcribed utterances, speaker-segmented if you're feeding in separate rep/prospect audio tracks (recommended — talk-time ratio is one of the highest-signal coaching metrics).

Step 2: Define the LangGraph State

from typing import TypedDict, Literal
from langgraph.graph import StateGraph, END

class CallState(TypedDict):
    transcript: list[str]
    stage: Literal["discovery", "pitch", "objection", "pricing", "closing"]
    last_tip_at: float
    detected_signals: list[str]
    coaching_tip: str | None

def transcript_node(state: CallState) -> CallState:
    # append latest utterance, trim to last ~2 min for context window discipline
    return state

def stage_classifier_node(state: CallState) -> CallState:
    # cheap heuristic + LLM fallback for ambiguous turns
    state["stage"] = classify_stage(state["transcript"])
    return state

def signal_detector_node(state: CallState) -> CallState:
    state["detected_signals"] = detect_signals(state["transcript"], state["stage"])
    return state

def coach_node(state: CallState) -> CallState:
    if not state["detected_signals"]:
        state["coaching_tip"] = None
        return state
    state["coaching_tip"] = generate_tip(state["detected_signals"], state["stage"])
    return state

def dedupe_node(state: CallState) -> CallState:
    import time
    if state["coaching_tip"] and time.time() - state["last_tip_at"] < 45:
        state["coaching_tip"] = None  # too soon since last tip
    else:
        state["last_tip_at"] = time.time()
    return state

Step 3: Wire the Graph

graph = StateGraph(CallState)

graph.add_node("transcript", transcript_node)
graph.add_node("stage", stage_classifier_node)
graph.add_node("signals", signal_detector_node)
graph.add_node("coach", coach_node)
graph.add_node("dedupe", dedupe_node)

graph.set_entry_point("transcript")
graph.add_edge("transcript", "stage")
graph.add_edge("stage", "signals")
graph.add_edge("signals", "coach")
graph.add_edge("coach", "dedupe")
graph.add_edge("dedupe", END)

app = graph.compile()

Run this graph on every new transcribed utterance:

async for transcript_chunk in stream_call_audio(audio_chunks):
    state["transcript"].append(transcript_chunk)
    state = app.invoke(state)
    if state["coaching_tip"]:
        push_to_rep_screen(state["coaching_tip"])

What Makes the Coaching Actually Useful

A few non-obvious lessons from building agents like this:

Silence is a feature. Gate every tip through a dedupe/cooldown node. Reps tune out agents that talk too much.
Stage-aware tips beat generic tips. "Ask a follow-up question" means nothing without knowing you're in discovery vs. pricing.
Talk-time ratio is the cheapest high-value signal. You don't need an LLM to compute it — a running word-count ratio between speaker tracks catches "rep is monologuing" instantly.
Keep the coach node's prompt narrow. One job: turn a detected signal into one short, actionable sentence. Don't let it also try to summarize the call — split that into its own node.
Log every tip + outcome. You'll want to evaluate which tips actually correlate with better close rates, and that requires structured logging from day one, not an afterthought.

Where to Go Next

Add a post-call summary node that runs once the call ends, rolling up every signal and tip into a CRM-ready note.
Add a playbook retrieval node (RAG over your team's actual sales playbook) so tips are grounded in your specific methodology, not generic SaaS sales advice.
Run an eval suite against recorded calls before shipping to live reps — silent failures (no tip when one was clearly needed) are worse than noisy ones.

This pattern — realtime audio in, LangGraph for stateful decision logic, narrow single-purpose nodes — generalizes well beyond sales coaching. Swap the stage classifier and playbook for your domain and you've got the same architecture for support-call QA, interview coaching, or live compliance monitoring.

Have you shipped a realtime voice agent? What's tripped you up most — latency, state management, or getting the agent to know when to shut up?

Forward Deployed Engineer (FDE): The Role That's Quietly Eating the AI Job Market

JITENDRA KUMAR SINGH — Sun, 21 Jun 2026 07:23:59 +0000

For the last decade, the dream job in tech was working on the model. Bigger context windows, better benchmarks, frontier research — that's where the prestige (and the comp) lived.

That's flipped in 2026. The fastest-growing, highest-paid role in AI right now isn't building the model. It's getting the model to actually work inside a real company's broken, legacy, regulation-heavy environment.

That role is called a Forward Deployed Engineer (FDE).

The number that explains everything

A widely cited MIT study looked at 300 enterprise AI projects and found that ~95% produced no measurable business impact. Not because the models were bad — because nobody could get them integrated into the customer's actual SAP instance, SSO setup, and legacy ETL pipeline.

Models got commoditized fast. Deployment didn't. So the bottleneck — and the money — moved.

Job postings for FDE roles grew 729% year-over-year between April 2025 and April 2026. OpenAI spun up an entire company around it ("The Deployment Company," $4B+ raised). Anthropic formed a $1.5B JV with PE firms just to embed engineers inside customer orgs. Palantir — the company that invented the role — has more open FDE roles right now than its next two competitors combined.

What an FDE actually does

It's not consulting. It's not sales engineering. An FDE:

Sits inside (physically or virtually) a customer's environment
Scopes the real problem — which is never what the kickoff call said it was
Writes actual production code: RAG pipelines on messy proprietary data, agentic workflows, integrations with whatever legacy system the customer has
Builds eval suites to catch hallucinations/regressions before they hit users
Fights through enterprise SSO, security review, and data governance to get real prod access
Owns the outcome until the system is actually running, not until the demo looks good

The cleanest mental model: a Solutions Architect draws the blueprint and sells the dream. An FDE is on-site pouring concrete and personally on the hook for whether the building stands.

One data point that confirms this is engineering, not sales: an analysis of ~1,000 FDE job postings found 0% carried a sales quota.

The interview is famous for a reason

Palantir popularized a format almost every company hiring FDEs now uses: you get a massive, ambiguous, real-world problem and 60 minutes on a whiteboard.

Classic version: "A city wants to cut 911 response times. They have call data, traffic data, ambulance GPS. Go."

2026 AI-native version: "A logistics firm wants an agent to auto-reroute delayed shipments. They have SAP data, weather APIs, and 500 warehouse managers. Design the eval suite so the agent doesn't overspend on shipping while holding 99% delivery rate."

There's no "correct answer." Interviewers are watching whether you resist jumping straight to "build an AI to predict X!" before you've actually interrogated the data quality and constraints.

FDE vs. everything that sounds like it

Role	Ships production code?	When they're involved
Sales Engineer	No	Pre-sale
Solutions Architect	Rarely	Pre-sale → early onboarding
Customer Success Engineer	No	Post-sale, ongoing
FDE	Yes	Post-sale → long-term ownership

CSEs guide within what the product already supports. FDEs extend the product — they ship features that don't exist anywhere else yet because the customer's environment demanded it.

What it pays (2026)

Frontier labs (OpenAI, Anthropic): mid-level $300K–$450K total comp, senior $450K–$550K, principal $600K–$1M+. Equity is now 55–70% of comp at the top.
Palantir (FDSE): median ~$215K — lower equity weighting than frontier labs.
Across all postings: median advertised salary ~$174K, equity in ~70% of offers.
India: ₹18–28 LPA entry, ₹30–50 LPA mid, ₹50–80+ LPA senior — concentrated in Bengaluru, Gurgaon, AI startups, and GCCs.

Interesting twist: NYC has overtaken SF as the largest US FDE hub, mostly because regulated industries (finance, insurance, healthcare) hire embedded deployment roles more aggressively than the average SF startup does.

Who's hiring

OpenAI, Anthropic (often titled "Applied AI Engineer"), Palantir, Databricks, Snowflake, Cohere, Scale AI, Google Cloud, Salesforce, Stripe, Ramp, Rippling, Adobe (for Firefly), plus vertical AI startups like Sierra, Harvey, Decagon, Cognition, and ElevenLabs. Even EY, PwC, and McKinsey have entered the space.

59% of hiring companies are Seed–Series A. This isn't just a frontier-lab thing — it's structural across the whole AI ecosystem.

Should you go for it?

Good fit if you want to actually ship things, like being close to customers, and don't mind genuine ambiguity and travel.

Bad fit if you want deep, uninterrupted focus on one codebase, find context-switching across customers draining, or need a predictable environment. You're catching pressure from both the customer and your own product org at the same time — it's a demanding seat.

If you're targeting these roles: ship something into actual production, not just a notebook. Build eval suites, not just demos. And practice the non-technical part — these interviews test communication and trade-off reasoning as hard as they test code.

TL;DR: Models got commoditized. Deployment didn't. The FDE is the role built to close that gap, and right now it's paying better than most ML research seats at the senior level. If you're a strong engineer who also likes being in the room with customers, this is probably the best-leveraged seat in AI hiring today.

What's your take — is FDE a durable role, or a 18-month hiring spike that gets automated away by better agent tooling?

I Built Flash Attention From Scratch — Here's What Nobody Tells You About It

JITENDRA KUMAR SINGH — Sat, 20 Jun 2026 14:16:48 +0000

Everyone uses Flash Attention. Almost nobody has implemented it.

Call F.scaled_dot_product_attention() in PyTorch and you get blazing-fast, memory-efficient attention — for free.

But that convenience hides three ideas that, once you actually implement them, change how you think about every transformer you'll ever work with:

1. The memory wall, not compute, is the real bottleneck

Standard attention materializes the full N×N score matrix in GPU HBM (high-bandwidth memory). For long sequences, that's the actual bottleneck — not FLOPs. Flash Attention's core insight is refusing to materialize that matrix at all.

2. Tiling turns attention into a streaming problem

Instead of computing the full softmax at once, Flash Attention processes the sequence in blocks, keeping each tile in fast on-chip SRAM. The catch: softmax needs the full row to normalize correctly — so you can't just tile naively.

3. Online softmax is the trick that makes tiling possible

This is the part that actually breaks people's brains the first time. You maintain a running max and a running sum across tiles, and rescale previous partial outputs every time you see a new tile with a higher max. It's numerically stable, incremental softmax — and once it clicks, you understand why this algorithm is genuinely elegant, not just "an optimized kernel."

We built a hands-on course that walks through implementing this from raw math up — block-wise QKV processing, online softmax rescaling, and the IO-aware design that made Flash Attention the default in every modern LLM stack.

If you've used attention but never built it, this is the gap worth closing.

👉 Build Flash Attention From Scratch

What part of Flash Attention do you think is most underrated — the tiling, or the backward pass recomputation trick?

Day 1/60: Building My First RAG Agent for Developers

JITENDRA KUMAR SINGH — Fri, 19 Jun 2026 18:43:12 +0000

🚀 Starting my AI course journey on Dev.to!

I'm Jitendra, an AI/ML developer from Delhi building practical
AI courses for developers.

Today: Setting up my first RAG (Retrieval-Augmented Generation)
project with LangGraph.

What I'm learning:
✅ How RAG combines LLMs with real-time data retrieval
✅ Building AI agents that use tools autonomously
✅ Creating production-ready AI systems

Follow for daily updates + course teasers!

👇 What AI topic should I cover first?

RAG deep dive
AI agents with LangGraph
OpenAI Realtime API
Agentic AI patterns

ai #machinelearning #rag #langgraph #generativeai