47 Take-Homes Reviewed for $18: The Hiring Step Nobody

47 Take-Homes Reviewed for $18: The Hiring Step Nobody Automates

Your ATS vendor brags about sourcing, screening, and scheduling. None of those are where you lose candidates. You lose them in the 11-day silence after they submit the take-home, while your hiring manager stares at a folder of 47 zips and PDFs and picks the one on top.

I built an agent for a client's internship cohort that reviews take-homes for 39 cents each, ranks them into Notion, and pings the hiring manager once a day. Offer-to-accept dropped from 19 days to 8. Here's the exact build.

Why take-home review is the real bottleneck

Every mainstream recruiting tool — Greenhouse, Ashby, monday, Workable — automates the cheap steps and leaves the expensive one alone. Sourcing is cheap. Keyword-screening resumes is cheap. Sending a Calendly link is cheap. Reviewing a take-home requires judgment, so the vendors skip it, and it becomes an 11-day dead zone in the middle of your pipeline where the money leaks out.

The client's numbers, no rounding:

Stage	Before	After
Applicants per cohort	203	203
Take-home submissions	47	47
Days to open the folder	11	<1
Days to first reply	14	2
Manager time per candidate	22 min	4 min
Offer-to-accept cycle	19 days	8 days
Review cost (Claude)	—	$18.33 total

Three of their top five picks from the previous cohort had already accepted elsewhere by the time the manager sat down to review. That's not a sourcing problem. That's a silence problem, and silence is the one thing automation is actually good at killing.

The stack: n8n, Claude, Notion, Slack

Nothing exotic. The trigger is a Gmail label. An earlier step in the pipeline labels candidate submission emails as take-home-submitted. n8n polls that label every 5 minutes and fires the workflow.

The first node classifies the submission format, because candidates don't cooperate:

• ~60% send a GitHub repo link
• ~30% send a PDF or Google Doc
• ~10% send a Loom video

Each format needs a different extractor before you burn tokens on Claude:

// n8n Function node - route by submission type
const body = $input.first().json.text;
const githubMatch = body.match(/github\.com\/[\w-]+\/[\w-]+/);
const loomMatch = body.match(/loom\.com\/share\/[\w]+/);
const pdfAttachment = $input.first().json.attachments?.find(a => a.mimeType === 'application/pdf');

if (githubMatch) return [{ json: { type: 'github', url: `https://${githubMatch[0]}` }}];
if (loomMatch)   return [{ json: { type: 'loom',   url: `https://${loomMatch[0]}` }}];
if (pdfAttachment) return [{ json: { type: 'pdf', file: pdfAttachment }}];
return [{ json: { type: 'unknown', raw: body }}];

For GitHub, I call the API and pull three things: the README, the three largest source files, and the last 20 commits. Not the whole repo — you'll blow the context window and pay for boilerplate. For PDFs, I convert to markdown first (I use pdftotext piped through a cleanup script) because raw PDF parsing burns tokens and destroys structure. Loom links go through Whisper for a transcript before Claude ever sees them.

That preprocessing is the difference between 39 cents per submission and $2+.

The scoring rubric (this is where hand-waving fails)

Every AI hiring demo I've watched shows a vague prompt like "evaluate this candidate submission." That gives you vague scores. The rubric has to be sharp, and it has to force a written justification, or the model will just regress to 3-out-of-5 on everything.

Six criteria, scored 1-5, each with a mandatory one-sentence justification:

1. Problem understanding — did they solve the actual problem or a nearby one?
2. Technical execution — does it work, and is the approach reasonable?
3. Code quality — naming, structure, obvious code smells
4. Communication clarity — README, comments, PR description
5. Edge case handling — what happens on bad input, empty state, failure?
6. Scope discipline — did they overbuild or underbuild?

Max score: 30. Here's the actual system prompt (trimmed):

You are a skeptical senior engineer reviewing an internship take-home.
You have seen hundreds of these. You are NOT here to be encouraging.

Score the submission on six criteria, 1-5 each. For each score, write
one sentence justifying it with a specific reference to the code or doc.

Then write a "concern" field: exactly two sentences on the single biggest
issue a hiring manager should look at before an interview. If there is no
real concern, say so - do not invent one.

Finally, flag one of:
  - CLEAN
  - MINOR_FLAG (small issues, worth noting)
  - MAJOR_FLAG (undisclosed AI use, plagiarism, non-working code)

Return JSON only. No preamble.

The concern field is the whole reason this is usable. A hiring manager doesn't want a score — they want to know what to look at. Two sentences of "the auth handler swallows all exceptions silently and there are no tests around the token refresh path" is worth more than any number.

The output schema:

{
  "scores": {
    "problem_understanding": 4,
    "technical_execution": 3,
    "code_quality": 4,
    "communication_clarity": 5,
    "edge_case_handling": 2,
    "scope_discipline": 4
  },
  "total": 22,
  "concern": "Error handling in the main request handler is missing entirely and there is no test coverage for the retry logic. Both are red flags for a role that will touch production ingestion.",
  "flag": "MINOR_FLAG"
}

Notion card + the one Slack message that matters

Step three writes a Notion card. Title is <Candidate Name> — <Score>/30. Body has the six sub-scores, the concern paragraph, the flag, and a direct link back to the submission. Traffic-light status property:

• Green — score >24 AND flag is CLEAN
• Yellow — score 18-24 OR MINOR_FLAG
• Red — score <18 OR MAJOR_FLAG

The Notion database has one saved view sorted by score descending, filtered to the current cohort. The hiring manager opens it once and sees a ranked list with greens at the top.

Step four is the daily Slack ping. Once at 9am, if there are new reviewed submissions in the last 24 hours, the workflow posts a single DM:

Take-home review, Wed Aug 12
3 green candidates ready to schedule
2 yellows to skim
1 red to skip
→ Open ranked view

That's it. No per-submission notifications. No email digest. One message, one link, actionable. Notification fatigue is what kills these systems in month two, and I'd rather ship one signal a day than ten.

Live run: 40 seconds, 34 cents

To show it working end-to-end I submitted a fake take-home to the client's test inbox — a small Node project on GitHub. The Gmail trigger fires within 5 minutes. n8n picks it up, hits the GitHub API, pulls the README and the three main source files. Claude runs the rubric prompt. Total wall-clock time: about 40 seconds. Token cost from the log: $0.34.

The Notion card comes back with a score of 22, yellow status, and the concern reads: "The main handler has no try/catch and there is no test coverage on the transform step. Ask them to walk through their error strategy in the interview."

That's a real concern I'd want a human to look at. It's not the model padding to seem thorough — it's pointing at two specific files and two specific gaps. That's the bar.

Across the full cohort:

• 47 submissions reviewed end-to-end
• $18.33 total Claude spend ($0.39 average)
• 14 hours of manager time saved (22 min → 4 min per candidate)
• 19 → 8 day offer-to-accept cycle

Two of the top three hires in that cohort said the fast turnaround was why they took the offer over a competing one. That's the number that pays for everything.

Three things I got wrong on v1

Pass these on so you don't repeat them.

• Auto-rejecting reds was a mistake. One candidate got flagged MAJOR_FLAG for undisclosed AI use — turned out they'd disclosed it in a section my parser skipped over. Reds now go to a human for a 30-second sanity check before any candidate-facing action fires. The cost of one false rejection on a top candidate is way higher than 30 seconds of manager time.
• Ten criteria was too many. My first rubric had ten. Scores blurred together and the justifications got repetitive. Six is the sweet spot — enough coverage to be fair, few enough that each score carries weight.
• Never skip the concern field. I ran an early version that only produced numeric scores. Managers ignored it. The one-paragraph concern is what makes the hiring manager trust the ranking and stop re-reading every submission themselves.

One more, quieter lesson: the model is a first reviewer, not a decision maker. Every candidate-facing action — reject, invite, offer — still goes through a human. The agent's job is to make sure that human is looking at the right five submissions on day 1, not the wrong 47 on day 11.

Why bizflowai.io helps with this

The take-home reviewer is one of about a dozen hiring-stage agents I've shipped for clients through bizflowai.io — panel schedulers, reference-call bots, offer-letter agents, ghost-prediction sentiment monitors. The pattern is always the same: find the step where a human is doing 20 minutes of judgment work on every candidate, extract the mechanical 80% of that judgment into a scored + flagged artifact, and leave the human to validate. It's not about replacing hiring managers. It's about making sure they're never the reason a good candidate walked.

---

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.