DEV Community

Cover image for Open Source Project of the Day (#105): hiring-agent — HackerRank Open-Sourced Their Resume Scoring System. How Does Your Resume Score?
WonderLab
WonderLab

Posted on

Open Source Project of the Day (#105): hiring-agent — HackerRank Open-Sourced Their Resume Scoring System. How Does Your Resume Score?

#ai #opensource #career #developer

Introduction

"HackerRank open-sourced their internal hiring pipeline — which means you can now run your own resume through a recruiter's scoring system."

This is article #105 in the Open Source Project of the Day series. Today's project is hiring-agent — a resume evaluation pipeline open-sourced by HackerRank/InterviewStreet.

The most important point first.

This tool was built for recruiters: process batches of candidate resumes and produce explainable scored evaluations. For developers, it's most valuable in reverse: run your own resume through a recruiter's scoring criteria and see what the system thinks of you.

Your total score matters less than what you learn from it: where the system deducted points, where it awarded them, what it considered a strength, and what it didn't count at all. That information tells you directly how your resume looks to automated screening systems.

What You'll Learn

  • The four scoring dimensions and their weights — why GitHub contributions matter more than you expect
  • The five-stage pipeline: how a PDF becomes a score
  • GitHub Enrichment specifics: which repositories get picked up, which get ignored
  • Developer perspective: how to use this tool to audit your own resume
  • What the -20 to 120 scoring range means
  • Local (Ollama) vs. cloud (Gemini) setup options

Prerequisites

  • Basic Python (enough to run a project)
  • Resume in PDF format
  • GitHub account (optional, but absence of one costs significant points)

Project Background

What Is hiring-agent?

hiring-agent is a resume evaluation pipeline: input a PDF resume, output a structured scoring report. It uses an LLM (either local Ollama or Google Gemini) for text understanding and scoring, with GitHub data pulled as an additional signal.

HackerRank's stated mission is "changing the world to value skills over pedigree." This scoring system reflects that principle — the highest-weighted dimensions are what you've actually built (open source contributions, personal projects, production experience), not where you went to school.

Author / Team

  • Organization: HackerRank / InterviewStreet
  • License: MIT
  • Language: Python (100%)

Project Stats

  • ⭐ GitHub Stars: 2,200+
  • 🍴 Forks: 599+
  • 📄 License: MIT

Scoring Dimensions: Understand the Rules First

This is the most critical section. The system scores across four dimensions, plus bonuses and deductions:

Base score (100 points total)
  ├── open_source        (open source contributions)
  ├── self_projects      (personal projects)
  ├── production         (production experience)
  └── technical_skills   (technical skills)

Bonus points:     up to +20 (exceptional signals)
Deductions:       up to -20 (inconsistencies, weak signals)

Final score range: -20 to 120
Enter fullscreen mode Exit fullscreen mode

Key data point: Based on community discussion, open_source and self_projects together account for roughly 65 of the 100 base points. GitHub contributions and personal projects represent about two-thirds of the total weight.

The implication for developers is direct: if your GitHub profile is sparse or mostly private, this system will score you low — regardless of how many technologies you list on your resume.

What Each Dimension Measures

open_source (open source contributions)

Contributions to external open source projects — not just repositories you created. Merged PRs and committed changes count; stars and forks do not. Contributions to well-known projects (major frameworks, infrastructure tools) score higher than obscure repositories.

self_projects (personal projects)

Projects you built and maintain. The system filters: only repositories exceeding a minimum commit threshold are counted. Hello-World-level repositories don't qualify. Projects maintained over multiple years carry more weight than short bursts of activity.

production (production experience)

Whether your work history includes real products used by actual users. Descriptions that include impact data ("served 50K daily active users," "processed 10M requests/day") score better than descriptions that only describe responsibilities ("developed the authentication module"). Results, not just tasks.

technical_skills (technical skills)

Assessed depth and breadth of technical capabilities. A clean list of technology names (React, Python, Docker...) scores lower than demonstrating those technologies in specific production or project contexts. What you've done with them, not just that you know them.

The Pipeline: Five Stages 

Your PDF resume
    ↓
Stage 1: PDF → Markdown (PyMuPDF)
    ├── Parse all pages
    ├── Preserve heading hierarchy, tables, links
    └── Output LLM-processable text

Stage 2: Section structuring (Jinja templates + LLM)
    ├── Basics (contact information)
    ├── Work (work history)
    ├── Education
    ├── Skills
    ├── Projects
    └── Awards
    Outputs structured data following JSON Resume standard

Stage 3: GitHub signal enrichment
    ├── Fetch GitHub profile
    ├── Retrieve all public repositories
    ├── Filter: keep repositories exceeding commit threshold
    ├── Classify project types
    └── Select top 7 most representative repositories

Stage 4: Scoring (LLM + rules)
    ├── Score each of four dimensions
    ├── Generate evidence explanation per dimension
    ├── Calculate bonuses and deductions
    └── Apply fairness constraint checks

Stage 5: Output
    ├── Human-readable terminal report (scores + evidence)
    └── CSV row append (optional)
Enter fullscreen mode Exit fullscreen mode

From the applicant's perspective on Stage 3: The system extracts your GitHub username from the resume, then fetches data directly from GitHub. This means:

  • Your GitHub URL must be explicitly in the resume (the system may not find it otherwise)
  • Private repositories are invisible and score nothing
  • Forked repositories with no meaningful modifications score low
  • Empty repositories (README only) are essentially ignored

Using It to Audit Your Own Resume

Installation and Setup 

git clone https://github.com/interviewstreet/hiring-agent
cd hiring-agent

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

cp .env.example .env
Enter fullscreen mode Exit fullscreen mode

Option A: Local with Ollama (no API key required)

# Install Ollama first: https://ollama.ai
ollama pull gemma3:12b  # Better quality, needs ~8GB RAM

# .env configuration
LLM_PROVIDER=ollama
DEFAULT_MODEL=gemma3:12b
Enter fullscreen mode Exit fullscreen mode

Option B: Google Gemini (higher quality, requires API key)

# .env configuration
LLM_PROVIDER=gemini
DEFAULT_MODEL=gemini-2.5-pro
GEMINI_API_KEY=your_key_here
Enter fullscreen mode Exit fullscreen mode

Run the evaluation:

python score.py /path/to/your_resume.pdf
Enter fullscreen mode Exit fullscreen mode

Add a GitHub token to avoid API rate limits:

GITHUB_TOKEN=your_github_pat  # add to .env
Enter fullscreen mode Exit fullscreen mode

Reading the Output

The report doesn't just give a total — each dimension comes with evidence explanations:

=== Evaluation Report ===

open_source: 18/30
  Evidence: 3 merged PRs in tensorflow/models;
            contributed to kubernetes/kubernetes documentation.
  Deduction: No core contributor role found in major OSS projects.

self_projects: 22/35
  Evidence: 4 qualifying repositories detected (commit count > threshold):
            - my-web-scraper: 230 commits, maintained 2 years
            - cli-tools: 180 commits
            - ...
  Bonus: 1 project has 150+ stars

production: 12/20
  Evidence: Work history mentions user scale ("50K daily active users"),
            but lacks specific reliability metrics or system scale descriptions.

technical_skills: 15/15
  Evidence: Skills span frontend, backend, and infrastructure;
            each technology is demonstrated in project or work context.

Bonus points: +5
  Evidence: Published technical article at a top conference.

Deductions: 0

Total: 72/100 (with bonus: 77)
Enter fullscreen mode Exit fullscreen mode

The evidence explanations are more valuable than the score. They show:

  1. What the system found: whether your GitHub projects were correctly identified
  2. What the system missed: information not reflected in your resume
  3. Where points were deducted: which specific dimension is holding you back
  4. What would earn bonus points: what exceptional signals look like

Translating Results Into Actions

Common improvement patterns by dimension:

If open_source score is low:

  • Resume has no GitHub URL → add it
  • GitHub profile is mostly private repositories → consider making relevant ones public
  • Only self-created repositories, no external contributions → find active projects to contribute to

If self_projects score is low:

  • Few repositories with low commit counts → projects exist but code isn't on public GitHub, or lack sustained maintenance
  • Unclear project descriptions → add context to README: background, technical decisions, actual outcomes

If production score is low:

  • Work descriptions say "responsible for developing..." instead of "implemented X, resulting in Y" → rewrite as outcome-oriented descriptions
  • No metrics → add specific numbers within compliance limits (user count, request volume, latency improvements)

If technical_skills score is low:

  • Skills list too broad and shallow → trim to technologies with real depth of experience
  • Skills not reflected in projects or work history → ensure all resume sections tell a consistent story

Limitations: Use It as Input, Not as Authority

A few things to keep in mind:

Public GitHub bias: The system can only see public repositories. If you've done significant work in private company codebases, or kept repositories private for compliance reasons, your score will be systematically lower. This is a measurement limitation, not an accurate assessment of your experience.

PDF parsing quality: Complex layouts (multi-column, heavy graphical design) may cause information extraction errors. A clean single-column format gives the most reliable results.

Model quality affects output: Running with Ollama + a small model (gemma3:4b) and running with Gemini Pro will produce different results. If the evaluation seems obviously wrong, try a larger model.

Scoring reflects HackerRank's hiring values: The weight distribution (open source + personal projects > production experience) reflects HackerRank's own preferences. Large tech companies often prioritize production experience more heavily; startups may value project creativity differently. Use this as one data point, not a universal standard.

Quick Start: Cheapest Configuration

If you just want a quick trial, Gemini Flash is very low cost:

# .env
LLM_PROVIDER=gemini
DEFAULT_MODEL=gemini-2.0-flash   # Fast and cheap
GEMINI_API_KEY=your_key           # Free to get from Google AI Studio

DEVELOPMENT_MODE=True             # Enable caching to avoid redundant LLM calls
Enter fullscreen mode Exit fullscreen mode

Then run:

python score.py ~/Desktop/my_resume.pdf
Enter fullscreen mode Exit fullscreen mode

Links and Resources

Conclusion

For developers, hiring-agent's value isn't understanding "what tools recruiters use." It's answering a specific question: by this scoring rubric, how does your resume score right now, and where does it fall short?

The weight distribution reveals something concrete: in this system, publicly visible GitHub project activity accounts for roughly 65% of base points. That means no matter how polished your resume text is, a sparse GitHub profile leads to a low score. That insight directly affects where you should spend your time — continuing to refine resume phrasing, or spending that time getting a meaningful project onto GitHub.

After running the evaluation, read each evidence explanation carefully. That's where the actionable information lives: which of your projects the system recognized as valuable, which it overlooked, and what kinds of descriptions are missing from your work history. Those are findings you can act on immediately.

Explore PrimeSkills — A marketplace for handpicked AI Agents and skills. Each is validated in real enterprise workflows, stripping away hype and keeping only what truly works.

Welcome to my Homepage for more useful insights and interesting products.

Top comments (0)