This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
Here is a problem I lived through as a fresh graduate.
Hiring has changed. Freshers send dozens of applications, recruiters skim for seconds, and rejections come with no feedback. Every fresher I know is stuck on the same questions:
- Which project should I put on my resume for this job?
- Does this project actually match what the job asks for?
- If something is missing, what exactly should I learn, and in what order?
Nobody answers these. Resume advice is vague: "add more projects," "make it stronger." So I built the tool I wished I had.
Redmargin is an honest reviewer for your GitHub projects. You give it a public repo and a target role. It compares your repo with what job postings ask for, shows where it found evidence of each skill and where it didn't, and gives you a ranked fix plan with a day-by-day timetable.
The "How May I Help" Desk
Not everyone has the same time. A student with two free hours a day and a job-seeker with eight need different plans. The How May I Help Desk builds a personalized plan around the hours you can dedicate to a project, so the fixes and timetable fit your real schedule.
Who would pay, and why?
- Job seekers and freshers pay to learn which projects fit which job, what is missing, and how to close the gap in the time they have.
- Companies and recruiters care about how closely a candidate's projects match the required skill set. That match percentage is something Redmargin could produce from the repo itself. Today the product serves candidates, and recruiter-facing evaluation is the direction I want to take it next.
Built to be trusted
- Numbers are deterministic. Skill percentages, repo signals and fix hours are computed in code.
- The LLM only writes the critique, and every claim must cite evidence that can be checked.
- Unsupported claims are dropped and counted, which gives a real "how often did the model invent things" number.
- Missing evidence is never treated as proof of a missing skill. Skills that can't be inferred from source are marked "not assessed".
Demo
Live demo: https://redmargin.onrender.com
Code
KomalDeep355
/
Redmargin
An evidence-first GitHub repository reviewer that turns skill gaps into actionable improvement plans.
Repo Reviewer (working name — rename freely)
An honest reviewer for your GitHub projects. You give it a repo and a target role; it compares the repo with what real job postings ask for, shows where repository evidence was not found, and gives a ranked fix plan with a day-by-day timetable. Missing repository evidence is not proof that a candidate lacks a skill.
Design rule: numbers are deterministic (signals, percentages, hours). The LLM only writes the critique, and every claim must cite evidence we can check. Unsupported claims are dropped and counted, which gives you a measurable "how often did the model invent things" number for your write-up.
Run it (Windows PowerShell)
cd D:\Projects\repo-reviewer
py -m venv .venv
.venv\Scripts\Activate.ps1 # if blocked: Set-ExecutionPolicy -Scope Process Bypass
pip install -r requirements.txt
copy .env.example .env # then edit .env
uvicorn app.main:app --reload
Open http://127.0.0.1:8000. Run tests…
How I Built It
Stack: FastAPI backend, a plain HTML/CSS/JS frontend, pytest tests, and a Render blueprint for deployment.
- Fetch the repo through the GitHub API (file tree plus key files, no cloning), with limits on tree size, file count and concurrency.
-
Detect signals deterministically: tests, CI, Docker, README quality, committed
.envfiles. -
Build the skill matrix. A taxonomy in
skills.jsondefines how each skill appears in job postings and in a repo (dependencies, paths, keywords). The tool compares how many postings ask for a skill with the evidence found in the repo. -
Plan the fixes. Gaps become ranked fixes from an editable
fix_library.json, then a day-by-day timetable, adjusted by the How May I Help Desk to your available hours. - Write the critique with open-source AI, then validate every claim against readable repo files. Claims without evidence are dropped and counted.
- Extras built on the same evidence: repo-grounded interview questions, evidence-checked resume bullets, and a weighted hackathon-score estimate.
How It Works: The Pipeline
Redmargin has two separate parts: a live review pipeline that users run, and an offline machine-learning experiment that is not yet connected to it.
LIVE REVIEW (what users run today)
──────────────────────────────────
Public repo URL + target role + hours available
│
▼
1. FETCH GitHub API, bounded file set, no clone, no execution
▼
2. SIGNALS tests, CI, deploy files, README quality, hygiene issues
▼
3. SKILL MATRIX skills.json taxonomy + repo markers
│ compared with real job postings
▼
4. PLANNER ranked fixes (fix_library.json) → timetable sized to your hours
▼
5. LLM (optional) open-source model writes the explanation
▼
6. VALIDATOR unsupported claims are dropped and counted
▼
Ranked fix plan + day-by-day timetable
OFFLINE EXPERIMENT (not wired into live scoring yet)
────────────────────────────────────────────────────
3 job-posting CSVs → join → dedupe → clean → role labels
→ split by company → TF-IDF + Logistic Regression
→ validation threshold → held-out test metrics
The key idea: numbers come from code, words come from the model, and a validator checks the words against the repo.
The Data Pipeline and Model Experiment
I wanted Redmargin to be more than an LLM that looks at a GitHub repository and guesses whether it is good enough for a job. So I separated the system into a deterministic review pipeline and an offline machine-learning experiment.
Collecting and preparing the data
My local dataset workflow started with three source CSV files containing job-posting information. It joined the job, summary and skill tables on job_link, removed duplicates, discarded records without usable titles or descriptions, normalized the text, and classified postings into target-role categories.
The initial join produced 12,217 postings, including 12,212 with source-provided skill labels. After filtering and deduplication, 1,223 postings remained:
| Role | Postings |
|---|---|
| Data Scientist | 729 |
| NLP/ML Engineer | 270 |
| Software Engineer | 188 |
| Forward Deployed Engineer | 36 |
I split the records by company, not by random row, so postings from the same company never appear in more than one split. That reduces leakage.
| Split | Postings |
|---|---|
| Training | 841 |
| Validation | 183 |
| Test | 199 |
The labels came from the source data and are not independently verified ground truth. I treat them as noisy labels, not definitive statements about what every employer requires.
Data transparency: the repository documents this workflow and these counts. The original dataset's source, collection date and redistribution licence still need confirmation before I can present it as a fully documented, redistributable corpus.
Training the skill-extraction baseline
For the offline experiment I built a multi-label skill-extraction baseline using TF-IDF vectorization and Logistic Regression. TF-IDF turns job-description text into numerical features based on the words and phrases it contains, and Logistic Regression learns to predict skill labels from those features.
I limited the experiment to the 50 most frequent training labels. The vectorizer and classifier were fitted on the training split only. I used the validation split to choose a global probability threshold of 0.20, then evaluated on the held-out test split.
| Metric | Result |
|---|---|
| Validation micro-F1 | 0.5305 |
| Test micro-F1 | 0.5483 |
| Test macro-F1 | 0.3335 |
These are exploratory results, and they have clear limits:
- The test data contained 3,670 skill-label occurrences outside the training vocabulary, so the model does not cover the long tail of skills.
- The metrics cover only the selected 50-label vocabulary. They are not overall skill-extraction performance and not repository-review accuracy.
The fitted vectorizer, classifier, vocabulary, threshold and training metadata are saved locally as a model artifact. This experiment is not yet integrated into Redmargin's live scoring. It is a starting point for future evaluation, not a claim that the app already uses a trained model to make hiring recommendations.
How the live review works
When a user submits a public GitHub repo and a target role, Redmargin fetches a bounded set of files through the GitHub API without cloning or executing the project. Then it:
- Checks repository signals such as tests, CI configuration, deployment files, README quality and hygiene issues.
- Builds a skill-evidence matrix from the skill taxonomy and repository markers.
- Compares skill mentions in available real job postings with the evidence in the repo.
- Ranks fixes and builds a timetable based on the user's available hours.
- Optionally asks an OpenAI-compatible LLM to write an explanation grounded in repository evidence.
- Validates the generated findings against the evidence and discards unsupported claims.
The numeric scoring and fix estimates are deterministic. Role priorities are human-configured, not learned from the training data. Synthetic sample postings are excluded from market percentages, and the app withholds market-based conclusions when there isn't enough real-posting evidence.
What comes next
The next steps are to confirm the dataset's source and licence, review the noisy skill labels, expand coverage across roles, and build an independently human-labelled benchmark of repository signals. Then I'll compare the offline model against the deterministic baseline before it influences any live decision.
The goal is not to replace evidence with a model prediction. It is to use machine learning where it adds value, while keeping every user-facing finding inspectable and honest about uncertainty.
The Open-Source AI Layer
Redmargin supports OpenAI-compatible endpoints, so it works with local inference through Ollama or a hosted open-weight model. A rule-based mock mode is also available, so anyone can explore the review workflow without configuring an API key.
The project is designed to keep deterministic scoring separate from AI-generated explanations. The model never decides the numbers. It only explains them.
Honest Limitations
- The included job postings are synthetic test data, not evidence of current hiring-market demand. They are excluded from market percentages.
- Real market comparisons require a sufficient collection of real job postings.
- Only public GitHub repositories are supported.
- Fix-hour estimates are editable defaults, not guaranteed predictions.
- The current product focuses on helping candidates improve their projects. Recruiter-facing evaluation is a potential future direction.
Why Does Open Innovation Matter?
A tool that judges people's work has to earn trust, and a closed black box can't do that.
- Privacy and cost for the people who need it most. Freshers can't afford subscriptions, and their code and career plans are personal. With open-weight models and local inference, Redmargin can run on a candidate's own machine without that data leaving it.
- Inspectable, not magical. The skill taxonomy is a plain JSON file. Adding a role, skill or fix is a single file edit, so anyone can extend Redmargin to a new job market without asking a vendor.
- Measurable honesty. Because I control the model layer, I can measure how often the critique makes unsupported claims. That trust metric is the heart of the project.
- Economics that work for a side hustle. Per-call API pricing would eat the margin of a service aimed at students. Open models make a low-cost audit service possible.
My First User
My first user was a friend, and I didn't charge her anything.
She is applying for different kinds of jobs and internships. She was making projects for her resume but had no way to see where the skills were missing in them, or which projects matched which role. That is exactly the problem Redmargin was built for.
Why I Built Redmargin
I built Redmargin because I know how difficult it is to finish a project and still wonder whether it is good enough for the role you're applying for.
I wanted to create something more useful than another score generator: a tool that shows its evidence, admits what it cannot determine, and gives you a practical next step.
Redmargin is built around one principle: don't just tell developers what's wrong with their projects. Show them the evidence, then help them fix it.
Top comments (0)