Building a Self-Hosted Voice AI Interview Platform with Next.js, Supabase, and WebSockets

Qingyuan Yang — Wed, 15 Jul 2026 06:45:15 +0000

Aural started with a simple limitation: static question banks and chatbots do not reproduce the rhythm of a real interview. A useful interview system needs to listen, decide when to probe, preserve context, and produce feedback that is consistent enough to act on.

I built Aural as an open-source platform for conducting interviews over voice, chat, and video. Teams can design an interview, share a link, let the AI ask contextual follow-up questions, and receive a structured report. Candidates can also use the same core workflow for practice.

This post covers the engineering decisions that mattered most when moving from a chat prototype to a self-hostable, real-time product.

What the platform needs to do

A single interview touches several subsystems:

interview configuration and question sequencing
real-time voice, chat, or video interaction
adaptive follow-up generation
coding and whiteboard exercises
transcript storage and cleanup
structured scoring and report generation
organization, project, and access controls

The important design decision was to treat an interview as a stateful workflow, not a series of independent prompts. Every answer changes what the system knows and what it should ask next.

Why voice is harder than chat

Chat gives the application a clean turn boundary: the user presses Send. Voice does not. A real-time interview has to infer when a candidate has finished, handle pauses and filler words, and avoid interrupting a thought that is still developing.

Three problems dominated the implementation.

1. Latency is part of the user experience

Every stage adds delay: audio capture, transport, transcription, follow-up generation, speech synthesis, and playback. Optimizing only the model call is not enough. The entire path has to stream wherever possible, and each hop needs timing data so regressions are visible.

2. Raw transcripts are not reliable application state

Speech recognition output can contain partial phrases, duplicates, false starts, and punctuation changes. Aural keeps the live transcript useful for the interface while normalizing the final answer before it becomes durable interview context. That separation prevents temporary transcription artifacts from polluting later questions and reports.

3. Follow-ups must feel earned

A generic request to elaborate quickly sounds scripted. The follow-up generator receives the interview goal, current question, normalized answer, previous turns, and remaining depth budget. It must either ask a focused follow-up or explicitly move on. A configurable depth limit keeps the conversation useful without letting one answer consume the whole session.

Conceptually, the loop looks like this:

Capture and normalize the candidate response.
Persist the turn and update interview state.
Decide whether the answer needs a follow-up.
Generate the next prompt under the configured depth and time limits.
Stream the prompt back through the active communication channel.
Finalize scoring only after the turn history is stable.

The architecture

Aural uses Next.js and TypeScript for the application, Supabase/PostgreSQL for durable data, tRPC for typed application APIs, and WebSocket relays for real-time voice events. LLM providers are behind adapters so a self-hosted deployment is not tied to one vendor.

The data plane and the AI plane are deliberately separated. Product state such as organizations, interviews, sessions, questions, permissions, and reports lives in PostgreSQL. Model calls receive only the scoped context needed for the current operation. This makes authorization easier to reason about and gives operators a clear place to enforce retention policies.

Long-running operations are also separated from the live turn path. Report generation can be retried without replaying the interview, while the real-time path stays focused on responsiveness.

Structured evaluation instead of one opaque score

An overall number is easy to display but hard to trust. Aural evaluates answers against explicit dimensions and keeps the evidence connected to the transcript. The report can therefore show what worked, what was missing, and what to improve next rather than returning a mysterious score.

The same structure powers candidate practice. Instead of ending with a report alone, the system can turn weaknesses into a revision prompt, a suggested answer outline, or another practice attempt.

Self-hosting and data ownership

Interview data can include resumes, job descriptions, recordings, transcripts, and internal hiring criteria. For many teams, where that data lives is not a minor deployment detail.

The open-source release includes Docker-based self-hosting and documented environment configuration. Operators can choose their database, storage, and supported model provider, then place the deployment inside their own network and retention controls. The hosted service remains useful for teams that do not want to operate the stack, but self-hosting is a first-class path rather than an export feature.

What I learned

The biggest lesson is that a believable AI interview is a systems problem. Prompt quality matters, but so do turn detection, streaming, transcript hygiene, state transitions, authorization, observability, and failure recovery. Improving any one component in isolation has limited value if the full conversation still feels slow or inconsistent.

A second lesson is to keep model output constrained by product state. Follow-up depth, remaining time, question goals, evaluation rubrics, and permitted tools should be explicit inputs. The model should not silently invent the workflow.

Try Aural

Live demo without signup: aural-ai.com
Source code: github.com/1146345502/aural-oss
License: MIT

Feedback is especially welcome on the self-hosting workflow, real-time voice experience, and the privacy boundaries you expect around interview data.

DEV Community: Qingyuan Yang

Open-source AI Interview Platform for chat/voice/video