How we built an AI buyer that pushes back (instead of agreeing with you)

Kulissa — Tue, 27 Oct 2026 18:56:11 +0000

If you've ever asked a chat model to "play a tough customer", you know what happens. It raises an objection, you give a so-so answer, and two turns later it says something like "That's a great point, I think we can move forward!"

That's a pleasant thing in an assistant and useless in a sparring partner. We're building Maya, an AI sales coach where reps practise live voice roleplays with AI buyers that interrupt, object and hesitate. The whole product only works if the buyer doesn't fold. This post covers how we got there: the voice pipeline, the persona harness that keeps the buyer honest, and the separate judge that grades the conversation.

Why LLMs want to agree with you

Instruction-tuned models are trained to be helpful, and the human preference data behind that training tends to reward answers that agree with the user and sound accommodating. This is usually called sycophancy. In a roleplay it shows up in three ways:

Concession drift. Every turn the model re-reads the whole conversation, sees a friendly rep, and moves a little further towards yes.
Forgotten constraints. "Your budget is fixed" was in the system prompt 40 turns ago. Now it's competing with a lot of persuasive text from the rep.
Role leakage. The model starts coaching the rep ("You might want to mention ROI here") because helping is its default mode.

More prompting ("Be skeptical! Do NOT agree easily!") helps a bit. But you're asking one model to hold a stance, track hidden facts and judge argument quality at once, while its context fills up with the other side's arguments.

The core idea: keep the buyer's state outside the prompt

In our buyer-persona harness, the things that decide whether the buyer concedes don't live in the prompt as prose. They live in code:

Persona state: who the buyer is, their current mood, how much they trust the rep, which objections are still open.
Hidden goals: what the buyer actually cares about and doesn't say up front, for example "worried about rollout effort, not price".
Concession rules: explicit conditions that have to be met before the buyer moves, such as "won't discuss pricing until the rep has asked about the current process".

The LLM still does what it's good at: natural, in-character dialogue. But each turn it gets a short, current description of where the buyer is, not the authority to decide where the buyer goes. State changes happen in the harness, based on what the rep actually did.

Here's a simplified version of the loop (illustrative pseudocode, not our production code):

# Simplified illustration of an externally held buyer state.

class BuyerState:
    def __init__(self, persona):
        self.trust = persona.initial_trust          # e.g. "low"
        self.open_objections = list(persona.objections)
        self.hidden_goals = persona.hidden_goals    # never shown to the rep
        self.stage = "discovery"

def take_turn(state, rep_utterance, history):
    # 1. Classify what the rep did, with a narrow, separate call.
    moves = classify_rep_moves(rep_utterance, history)
    # e.g. {"asked_open_question": True, "addressed": ["integration_effort"]}

    # 2. Apply concession rules deterministically.
    for objection in list(state.open_objections):
        if rule_satisfied(objection, moves, state):
            state.open_objections.remove(objection)
            state.trust = raise_one_level(state.trust)

    if not state.open_objections and state.trust == "high":
        state.stage = "ready_to_commit"

    # 3. Only now ask the model to speak, given the current stance.
    stance = render_stance(state)   # short, current, unambiguous
    return generate_buyer_reply(stance, history[-N:], rep_utterance)

A few things fall out of this design:

The model can't drift into yes. If the rules say an objection is still open, the stance says so every turn, however charming the rep is.
Hidden goals stay hidden but consistent. The buyer can hint at them naturally, and the harness knows whether the rep uncovered them.
It's debuggable. When a session feels off, we can look at state transitions instead of guessing what the model "was thinking".

Separating the judgment ("did the rep earn this?") from the performance ("say it like a skeptical CFO would") was the most important architectural choice we made.

Making it a real conversation: the voice pipeline

Sales calls are spoken, so practising them in a text box only gets you part of the way. Maya runs in the browser with a mic, nothing to install, and the pipeline is the classic three stages, all streaming:

streaming STT â†’ LLM â†’ streaming TTS

Each stage starts working before the previous one finishes. Transcription produces partial text while the rep is still talking, the model starts generating as soon as the turn ends, and speech synthesis begins on the first sentence instead of waiting for the full reply.

Latency is a budget, not a number

In conversation, people notice gaps quickly. A pause that's fine in a chat UI feels awkward on a call. So rather than optimising one component, it helps to think of a latency budget for each turn, split across:

detecting that the speaker has finished (endpointing),
final transcription,
time to the model's first token,
time to the first audio chunk,
network and audio buffering in the browser.

Every stage gets a slice, and the harness work from the previous section has to fit in too. That's one reason the rep-move classifier is a narrow, separate step: small focused calls are easier to keep fast than one huge do-everything prompt.

Barge-in: interrupting and being interrupted

Real buyers cut you off. Real reps talk over buyers. So we support barge-in in both directions:

Rep interrupts buyer: when voice activity is detected while the buyer is speaking, we stop playback, cancel the rest of the generation, and record what the buyer actually got to say. That last part matters: the transcript and the state should reflect the half-sentence that was heard, not the full reply that was generated.
Buyer interrupts rep: the persona can decide to jump in, for example when the rep has been pitching for a long time without asking anything. That's a state decision, not something the LLM improvises.

# Simplified barge-in handling
on rep_voice_detected while buyer_speaking:
    stop_audio_playback()
    cancel_llm_and_tts_streams()
    history.append(buyer_reply.truncated_to(audio_played))
    resume_listening()

Skip the truncation and the buyer "remembers" saying things the rep never heard.

Scenarios from real material

A generic "tough buyer" doesn't help a rep selling a specific product. Our scenario generator takes a company's own playbooks, product sheets and pricing and turns them into scenarios: personas, the objections they'd realistically raise, hidden goals and the concession rules that go with them.

The output is structured data that feeds the harness, not a big prompt, so a manager can see and adjust what the buyer cares about.

The judge is not the buyer

The last piece is evaluation. It's tempting to ask the buyer model at the end, "How did the rep do?" We don't, for the same reason you don't let the opposing team referee.

The evaluator is a separate LLM-as-judge, with its own context and instructions:

It scores against the customer's own methodology, not a generic idea of "good selling". If a team uses a particular discovery framework, that's the rubric.
It's evidence-first: every score has to quote the transcript. No quote, no score. That makes feedback concrete ("here's where you skipped the pain question") and makes judge mistakes easy to spot.
It's checked against calibration sets: transcripts with agreed-upon scores that we use to catch the judge drifting when prompts or models change.

Keeping them apart keeps each simpler: the buyer only has to be a convincing, stubborn human, and the judge only has to read carefully and cite.

What we'd tell anyone building something similar

Don't ask the model to be the source of truth for state it's under pressure to change. Keep it in code and give the model a fresh, short view every turn.
Split judgment from performance. Classify, apply rules, then generate.
Treat latency as a budget across stages, and stream everything you can.
Handle interruptions honestly: the history should contain what was heard, not what was generated.
Never let the actor grade itself. Use a separate judge that has to cite evidence, and calibrate it.

Try it (and a small ask)

Maya started as the Cursor Track-winning project at RAISE Summit 2026 in Paris, and we're now a small team in Zurich: Fernando Leyra (CEO), Sergey Novikov (CTO, PhD in AI), Yanik Almer (CPO) and Enes Sarituc (CRO).

If you want to see whether our buyer actually pushes back, try a free roleplay at kulissa.com/maya. It runs in your browser with your mic. And if you think it's worth it, we're launching on StartupBase, where feedback and support help a lot: startupbase.io/products/kulissa.

Questions about the harness, barge-in or judge calibration? Ask in the comments, we're happy to go deeper.,

DEV Community: Kulissa