Aural started with a simple limitation: static question banks and chatbots do not reproduce the rhythm of a real interview. A useful interview system needs to listen, decide when to probe, preserve context, and produce feedback that is consistent enough to act on.
I built Aural as an open-source platform for conducting interviews over voice, chat, and video. Teams can design an interview, share a link, let the AI ask contextual follow-up questions, and receive a structured report. Candidates can also use the same core workflow for practice.
This post covers the engineering decisions that mattered most when moving from a chat prototype to a self-hostable, real-time product.
What the platform needs to do
A single interview touches several subsystems:
- interview configuration and question sequencing
- real-time voice, chat, or video interaction
- adaptive follow-up generation
- coding and whiteboard exercises
- transcript storage and cleanup
- structured scoring and report generation
- organization, project, and access controls
The important design decision was to treat an interview as a stateful workflow, not a series of independent prompts. Every answer changes what the system knows and what it should ask next.
Why voice is harder than chat
Chat gives the application a clean turn boundary: the user presses Send. Voice does not. A real-time interview has to infer when a candidate has finished, handle pauses and filler words, and avoid interrupting a thought that is still developing.
Three problems dominated the implementation.
1. Latency is part of the user experience
Every stage adds delay: audio capture, transport, transcription, follow-up generation, speech synthesis, and playback. Optimizing only the model call is not enough. The entire path has to stream wherever possible, and each hop needs timing data so regressions are visible.
2. Raw transcripts are not reliable application state
Speech recognition output can contain partial phrases, duplicates, false starts, and punctuation changes. Aural keeps the live transcript useful for the interface while normalizing the final answer before it becomes durable interview context. That separation prevents temporary transcription artifacts from polluting later questions and reports.
3. Follow-ups must feel earned
A generic request to elaborate quickly sounds scripted. The follow-up generator receives the interview goal, current question, normalized answer, previous turns, and remaining depth budget. It must either ask a focused follow-up or explicitly move on. A configurable depth limit keeps the conversation useful without letting one answer consume the whole session.
Conceptually, the loop looks like this:
- Capture and normalize the candidate response.
- Persist the turn and update interview state.
- Decide whether the answer needs a follow-up.
- Generate the next prompt under the configured depth and time limits.
- Stream the prompt back through the active communication channel.
- Finalize scoring only after the turn history is stable.
The architecture
Aural uses Next.js and TypeScript for the application, Supabase/PostgreSQL for durable data, tRPC for typed application APIs, and WebSocket relays for real-time voice events. LLM providers are behind adapters so a self-hosted deployment is not tied to one vendor.
The data plane and the AI plane are deliberately separated. Product state such as organizations, interviews, sessions, questions, permissions, and reports lives in PostgreSQL. Model calls receive only the scoped context needed for the current operation. This makes authorization easier to reason about and gives operators a clear place to enforce retention policies.
Long-running operations are also separated from the live turn path. Report generation can be retried without replaying the interview, while the real-time path stays focused on responsiveness.
Structured evaluation instead of one opaque score
An overall number is easy to display but hard to trust. Aural evaluates answers against explicit dimensions and keeps the evidence connected to the transcript. The report can therefore show what worked, what was missing, and what to improve next rather than returning a mysterious score.
The same structure powers candidate practice. Instead of ending with a report alone, the system can turn weaknesses into a revision prompt, a suggested answer outline, or another practice attempt.
Self-hosting and data ownership
Interview data can include resumes, job descriptions, recordings, transcripts, and internal hiring criteria. For many teams, where that data lives is not a minor deployment detail.
The open-source release includes Docker-based self-hosting and documented environment configuration. Operators can choose their database, storage, and supported model provider, then place the deployment inside their own network and retention controls. The hosted service remains useful for teams that do not want to operate the stack, but self-hosting is a first-class path rather than an export feature.
What I learned
The biggest lesson is that a believable AI interview is a systems problem. Prompt quality matters, but so do turn detection, streaming, transcript hygiene, state transitions, authorization, observability, and failure recovery. Improving any one component in isolation has limited value if the full conversation still feels slow or inconsistent.
A second lesson is to keep model output constrained by product state. Follow-up depth, remaining time, question goals, evaluation rubrics, and permitted tools should be explicit inputs. The model should not silently invent the workflow.
Try Aural
- Live demo without signup: aural-ai.com
- Source code: github.com/1146345502/aural-oss
- License: MIT
Feedback is especially welcome on the self-hosting workflow, real-time voice experience, and the privacy boundaries you expect around interview data.
Top comments (1)
Self-hosting changes the shape of a voice AI product. Latency, recording, consent, storage, websocket failure, and transcript quality all become product concerns, not just backend details. The hard part is usually making the conversation feel live while keeping the system auditable.