DEV Community

anusha
anusha

Posted on

I Built an AI That Calls Candidates and Scores Them So Recruiters Don't Have To

The first-round phone screen is the most boring part of hiring.

A recruiter calls a candidate. Asks the same five questions. Takes notes. Decides whether to advance them. Does it again. And again. 200 times for one role.

That's a full week of screening calls before a single second-round interview.

So I built an AI that does that first call instead.

The AI calls the candidate. Conducts a 5-question screen. Hangs up. Produces a scorecard with scores, a recommendation, strengths, concerns, and a summary.

The candidate experience is just a phone call. They answer, talk to a friendly AI interviewer for two minutes, and hang up. No app, no form, no scheduling.

The recruiter's experience is a dashboard. They enter the candidate's name and phone number, click one button, and wait. A scorecard appears within 5 seconds of the call ending.

The code is here:

https://github.com/team-telnyx/telnyx-code-examples/tree/main/ai-hiring-phone-screen-python

The Problem With First-Round Screens

First-round screens are not hard. They are repetitive.

The same five questions. The same evaluation criteria. The same note-taking format. The same pass/hold/advance decision.

A recruiter doing first-round screens at 20 minutes each, for 200 applicants, spends 66 hours on calls before anyone advances. That's a week of work that produces the same structured output every time: name, background, skills, timeline, recommendation.

The first-round screen is also the part of hiring where the human adds the least value. The recruiter isn't evaluating technical depth. They're filtering for communication, basic fit, and interest. That's exactly the kind of structured conversation an AI can run consistently.

How It Works

The architecture has three pieces, and none of them are complicated.

One: trigger the call. The dashboard sends one API call to Telnyx scheduled events. The AI Assistant calls the candidate from a configured phone number.

Two: the call. Telnyx hosts the entire conversation. The AI Assistant (Voice Ultra Katie, powered by gpt-4o) greets the candidate, asks the five questions one at a time, listens, responds, and ends the call. No webhooks. No ngrok. No local call handling. Telnyx does the speech-to-text, the inference, and the text-to-speech in one managed product.

Three: the scorecard. The dashboard polls the Telnyx conversations API every few seconds. When it detects a finished screen conversation, it runs a second inference pass on the transcript to extract a structured scorecard: overall score, communication, technical depth, culture fit, recommendation (advance/hold/pass), key strengths, concerns, and a summary.

The scorecard appears in the dashboard within 5 seconds of the candidate hanging up.

Why AI Assistant Instead of Call Control

The original example in the Telnyx repo uses Call Control webhooks. That means you build the state machine yourself: answer the call, speak a greeting, gather speech, call inference, speak the response, gather again, repeat until done, hang up. You also need ngrok to expose your local server for webhooks.

I rebuilt it with the AI Assistant product instead. Telnyx hosts the entire call loop. You trigger the call, poll for the transcript, and process the result. Three API calls instead of a webhook state machine.

That tradeoff makes sense for this use case. The screening conversation is structured and predictable. You don't need fine-grained control over the call flow. You need the AI to ask five questions, listen, and hang up. That's what AI Assistants are built for.

If you needed custom call flow logic — DTMF menus, transfers, hold detection — you'd use Call Control. For a structured conversation with a known beginning and end, AI Assistants are simpler.

The Scorecard

The scorecard is the output. It's what the recruiter reads instead of taking the call themselves.

Each scorecard has:

  • Overall score (1-10)
  • Communication, technical depth, culture fit (each 1-10)
  • Recommendation — advance, hold, or pass
  • Key strengths — green chips ("strong communicator," "led multi-month infrastructure project")
  • Concerns — red chips ("salary range above band")
  • Summary — one paragraph the AI writes from the transcript

The scoring is a single inference call. The transcript goes in, the JSON comes out. Is it a validated rubric? No. It's an LLM's assessment of a conversation. But it's consistent — every candidate gets scored the same way, on the same criteria, from the same transcript.

For a real hiring pipeline, you'd want to calibrate the scoring prompt against historical data. For a demo, it works.

Where This Actually Works

This pattern works for high-volume, surface-level screening. Hourly roles, retail, seasonal hiring, call centers. Any role where the first-round screen is a filter, not a deep assessment.

It does not work for senior roles. The five questions are intentionally surface-level. They filter for communication and basic fit, not for engineering ability. A staff engineer would ace them in 30 seconds and you'd learn nothing.

It also doesn't replace human judgment. The recruiter reads the scorecard and decides whether to agree with the recommendation. The AI does the call. The human makes the decision.

Try It

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/ai-hiring-phone-screen-python
cp .env.example .env
pip install -r requirements.txt
python app.py
Enter fullscreen mode Exit fullscreen mode

You need a Telnyx account, a phone number, and an API key. The example includes a bootstrap script that provisions everything automatically.

The Bigger Pattern

The interesting thing about this build isn't the hiring use case. It's the shape of the workflow.

An outbound call. A structured conversation. A structured output.

Replace "hiring screen" with "patient intake," "lead qualification," "customer feedback survey," "policy renewal check," or "appointment confirmation." The architecture is the same. The phone is the highest-conversion channel we already have — people answer calls. AI makes it scalable.

The first-round screen is just the most obvious place to start.

Top comments (0)