Hardik

Posted on Oct 16

LocalAudit: an AI security auditor that never sees your secrets and never leaves your laptop

#devchallenge #hf26challenge

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

LocalAudit is a command-line security auditor for source code that runs 100% on your own machine. Point it at a repo and a few seconds later you get a clean HTML report (plus JSON and Markdown) listing hardcoded secrets, injection risks, insecure configuration and unpinned dependencies. Each finding comes with the file and line, the code around it, a plain-English explanation and a concrete fix.

The design rule I cared most about: the LLM never detects anything. Deterministic rules find the issues: 24 code-pattern rules for Python/Django/Flask and JavaScript/Node, 10 built-in secret detectors (AWS, Stripe, Slack, GitHub, Google, JWTs, private keys, high-entropy strings...) and dependency checks. A local open-weight model then triages each finding. It explains it, judges whether it's likely a false positive in context, and writes a fix. If the model is unavailable, you still get the full report.

Who would pay for this? People who need a security pass on code they aren't allowed to upload anywhere:

Freelancers and small agencies working on client code under NDA. A fixed-fee "pre-handoff security check" with a professional report as the deliverable is an easy add-on to sell.
Solo founders about to launch who can't justify an enterprise SAST subscription.
Consultants who want a fast first pass before a manual review.

For all of them, the cloud AI code-review tools have the same problem: your client's source code goes to someone else's server. LocalAudit's answer is that it doesn't go anywhere.

Demo

The demo scans the bundled examples/vulnerable_app, a deliberately insecure Django + Node app full of fake credentials. The rules-only scan takes about a second and finds 43 issues (4 critical, 16 high), for a risk score of 100/100. A few things worth watching for:

Secrets are masked everywhere. You'll only ever see AKIA****. Even a private key's body is replaced with **** before any snippet is built, so raw secrets never reach the report, the console or the model's prompt.
The model catches a decoy. The app contains subprocess.check_output("df -h /", shell=True). The rule correctly flags shell=True, but the command is a constant with no user input. The local model marked it as a likely false positive. LocalAudit doesn't delete it: it moves it to a separate "Likely false positives" section for a human to confirm.
It falls back gracefully. Stop Ollama and the tool prints the exact command to fix it (ollama pull qwen2.5-coder:7b), then carries on with the rules-only report.

Code

hardikgaikwad / localaudit

LocalAudit

A command-line security audit tool for source code repositories that runs 100% locally.

Deterministic rules find the issues: built-in secret detection, about 25 code-pattern rules for Python/Django/Flask, JavaScript/Node and generic files, and dependency pinning checks.
A local LLM triages them through Ollama. For each finding it explains the issue in plain language, judges whether it is likely a false positive, and suggests a concrete fix.
The LLM never detects vulnerabilities, and it never deletes a finding. Anything it judges a likely false positive goes into a separate section for you to review.
The report is complete and useful without the LLM (--no-llm, or when Ollama isn't running).

Output: a self-contained report.html (no CDN or external requests, and it prints cleanly to PDF), plus report.json and report.md.

60-second quickstart

# 1. Get the model (one time, ~4.7 GB)
ollama pull qwen2.5-coder:7b
# 2. Install

…

View on GitHub

ollama pull qwen2.5-coder:7b
pip install -e .
localaudit scan examples/vulnerable_app
# open localaudit-report/report.html

Useful flags: --no-llm, --min-severity high, --max-llm-findings 10, --model <any Ollama model>, and --fail-on high (exits with code 1, for CI).

How I Built It

The open-source AI at the core:

Ollama for local inference over its HTTP API.
Qwen2.5-Coder 7B, an open-weight coding model. It's the default, and you can swap it with --model or LOCALAUDIT_MODEL.

How the pipeline works:

Scan. Walk the repo, skipping .git, node_modules, venvs, build output, binaries, files over 1 MB and anything in .localauditignore.
Detect and redact. Run the secret detectors first and redact every detected value from the file text. Only then run the code rules and build snippets, which is how masking is guaranteed everywhere downstream.
Triage. Send each finding, highest severity first, to the local model. The request includes the rule description, the file path and the masked snippet. It runs at temperature 0, and Ollama's format parameter enforces a JSON schema, so the model must return {explanation, risk_in_context, likely_false_positive, confidence, suggested_fix, fixed_code_example}. Invalid JSON gets one retry, then the finding is marked "LLM triage unavailable".
Cache. Results are cached on disk, keyed by model + rule + snippet, so reruns are instant.
Report. Render a single self-contained HTML file with no CDNs or external requests, plus a print stylesheet so "Save as PDF" gives a client-ready document.

The stack is deliberately small: Python 3.10+, requests, jinja2 and pytest, with 110 tests. Those tests include a mocked Ollama covering invalid JSON, timeouts and the server going down mid-run.

I built it with an AI coding agent (Claude Code) doing much of the implementation while I directed the design and reviewed the results. One moment I liked: testing caught a regex bug where a bare password = "..." wasn't detected. Because the masker only masks what the detector finds, that password was showing up unmasked as a context line in a neighbouring finding's snippet. It's fixed now, and there's a regression test for it.

Why Does Open Innovation Matter?

For this product, open weights aren't a nice-to-have. They're the reason it can exist.

Privacy is the product. The customers I described can't send client code to a third-party API. A local open-weight model is the only way to give them AI explanations without breaking that rule. Nothing leaves the laptop.
The economics work. There's no per-token bill, so you can rescan on every commit, or scan a whole agency's worth of repos, for zero marginal cost. A paid API would make a cheap fixed-fee audit service hard to sustain.
No lock-in. Switching models is one flag. When a better open coding model ships, users upgrade by running ollama pull, not by renegotiating a contract.
Trust comes from transparency. The rules are plain regexes in a readable list, the model is inspectable, and the report says clearly which parts are deterministic and which are advisory.

Honest limitations:

Detection is regex-based and line-by-line, so it misses multi-line patterns and logic bugs.
A small model can misjudge a finding, which is why "false positives" are never deleted and still count toward --fail-on.
Speed depends on your hardware. On my machine the 7B model is split between CPU and GPU and takes about a minute per finding. A GPU that fits the whole model is much faster.
It isn't a pentest. It's a fast, private first pass.

My Agent Session

[AGENT_SESSION_PLACEHOLDER]

Changes from the template, and why:

"My First User" is removed. You don't have one yet, and the template says to delete the section in that case. It's where the bonus points are, though. If a friend or client runs it on their repo before the deadline, add the section back with what they said, even a sentence or two.
"Prize Categories" is removed. None of the 17 partner categories apply: LocalAudit uses Qwen, not Gemma, and doesn't use GitHub Copilot or any other parclearly which parts are deterministic and which are advisory.

Honest limitations:

Detection is regex-based and line-by-line, so it misses multi-line patterns and logic bugs.
A small model can misjudge a finding, which is why "false positives" are never deleted and still count toward --fail-on.
Speed depends on your hardware. On my machine the 7B model is split between CPU and GPU and takes about a minute per finding. A GPU that fits the whole model is much faster.
It isn't a pentest. It's a fast, private first pass.

My Agent Session

Building LocalAudit: a local-only AI security auditor (Claude Code session)

You

Build "LocalAudit": a command-line security audit tool for source code repositories that runs 100% locally. Deterministic rules detect issues, and a local LLM served by Ollama triages and explains them. Build it in the CURRENT working directory (do not create a nested project folder). Keep the scope tight: a small, clean, working tool beats a large, fragile one.

Core principle

The LLM NEVER detects vulnerabilities. Rules find candidate findings; the LLM only (a) explains each finding in plain language, (b) judges whether it is likely a false positive given the surrounding code, and (c) suggests a concrete fix. The tool must still produce a complete, useful report with --no-llm or when Ollama is unreachable.

Stack (keep it minimal)

Python 3.10+, standard library plus requests and jinja2 only. Use argparse for the CLI. Use pytest for tests.
Ollama HTTP API at http://localhost:11434 (/api/chat, stream=false). Default model qwen2.5-coder:7b, overridable with --model and the env var LOCALAUDIT_MODEL.
Package layout: localaudit/ (cli.py, scanner.py, rules.py, secrets.py, llm.py, report.py, models.py, templates/report.html.j2), tests/, examples/vulnerable_app/, README.md, requirements.txt, pyproject.toml (installable with a localaudit console script).

Features

Scanner
- Walk the target path; skip .git, node_modules, venv/.venv, pycache, dist, build, binaries, and files over 1 MB. Respect a simple .localauditignore (one glob per line).
- Each Finding has: id, rule_id, title, severity (critical/high/medium/low/info), category, file, line, snippet (the line plus 3 lines of context either side), and the matched text.
Secret detection (secrets.py), built in with no external tools
- Regexes for: AWS access key IDs, private key blocks, GitHub tokens, Slack tokens, Stripe live keys, Google API keys, JWTs, and generic password/secret/api_key/token = "<literal>" assignments.
- A Shannon-entropy check for high-entropy string literals assigned to suspicious variable names.
- Ignore obvious placeholders (changeme, your_key_here, example, xxx, <...>, ${...}, os.environ lookups).
- ALWAYS mask secret values in all output (show the first 4 characters then ****). Real secret values must never appear in the report, the console, or the LLM prompt.
Code-pattern rules (rules.py), a data-driven list of regex rules with language filters, severity, CWE id, and a short description. Include about 15-20 rules such as:
- Python/Django/Flask: DEBUG = True, ALLOWED_HOSTS = ['*'], hardcoded SECRET_KEY, eval/exec, pickle.loads, yaml.load without SafeLoader, subprocess with shell=True, os.system, string-formatted SQL passed to execute()/raw(), verify=False in requests, weak hashes (md5/sha1) for passwords, CORS_ALLOW_ALL_ORIGINS = True, csrf_exempt, mark_safe, Flask app.run(debug=True).
- JS/Node: eval(, child_process.exec with template strings, innerHTML =, dangerouslySetInnerHTML, rejectUnauthorized: false, Math.random used for tokens, wildcard CORS.
- Generic: private IPs/internal URLs hardcoded, TODO/FIXME mentioning security or auth, chmod 777.
Dependency check (optional, graceful)
- Parse requirements.txt and package.json. Flag unpinned Python dependencies (low severity).
- If pip-audit is installed, run it on requirements.txt and import its results as findings. If not installed, print a one-line notice and continue. Never fail because a tool is missing.
LLM triage (llm.py)
- For each finding (highest severity first, capped by --max-llm-findings, default 25), send the rule description, masked snippet, and file path to Ollama.
- Require structured JSON output using Ollama's format parameter with a JSON schema: {explanation, risk_in_context, likely_false_positive (bool), confidence (low/medium/high), suggested_fix, fixed_code_example}. Temperature 0, num_ctx 4096, request timeout 120s, one retry on invalid JSON, then fall back to marking the finding "LLM triage unavailable".
- Cache results on disk in .localaudit_cache/ keyed by a hash of (model + rule_id + snippet) so reruns are fast.
- Check Ollama availability up front (GET /api/tags). If Ollama is down or the model is not pulled, print the exact command to fix it (ollama pull <model>) and continue in --no-llm mode instead of crashing.
- The LLM may downgrade a finding as a likely false positive but must never silently delete it. Show it in a separate "Likely false positives" section.
Reporting (report.py)
- Output a single self-contained HTML report (inline CSS, no external requests, no CDN) with: title and scan metadata (path, date, model used, duration, files scanned), a severity summary with counts, a risk score out of 100, findings grouped by severity (each with file:line, code snippet, explanation, fix, and fix example), a collapsed "Likely false positives" section, and a print stylesheet so Save as PDF from the browser looks clean.
- Also write report.json (machine-readable) and report.md.
- CLI flags: localaudit scan PATH [--model M] [--out DIR] [--no-llm] [--max-llm-findings N] [--min-severity LEVEL] [--format html,json,md] [--fail-on LEVEL]. --fail-on returns exit code 1 if any finding is at or above that severity (for CI use). Show a concise coloured console summary and a progress indicator during LLM triage.
Demo and tests
- Create examples/vulnerable_app/: a small intentionally vulnerable Django-style app and a small Node file, with fake credentials, that triggers most of the rules. Add a prominent comment that it is intentionally insecure.
- pytest tests: each rule fires on a known-bad snippet and does not fire on a safe one; secrets are masked; placeholders are ignored; the scanner finds the expected findings in the example app; the LLM module is tested with a mocked HTTP response (including invalid JSON and Ollama-down cases); --no-llm end-to-end run produces all three report f [... truncated 1197 chars]

Agent

Plan: models → rules/secrets → scanner → report (with --no-llm) → tests → LLM module → README and example app → run tests, run the scan, check whether Ollama is available.

Empty directory; Python 3.14; jinja2 is missing; Ollama is installed but not answering on :11434 right now. Writing the package now.

Fixing one thing: provider-specific keys (e.g. a live Stripe key used as an os.environ.get default) should still be caught; only the generic/entropy checks should skip env lookups.

Now the code-pattern rules.

Adding snippet_start to Finding so reports can number snippet lines, then the scanner (including dependency checks).

Making dependency checks also work on redacted text (a requirements line could carry a tokenized URL):

That package.json line is needlessly convoluted; simplifying it.

Now the reporting module and HTML template.

Now the LLM triage module.

Progress should show before each (possibly slow) request, not after. Adjusting:

Now the CLI.

Install and do a first rules-only scan:

The console script installed to a user Scripts dir that isn't on PATH; I'll call it by full path for now (and mention it at the end).

The heredoc turned \\n into a real newline. Fixing:

42 findings, secrets masked. One miss: "PASSWORD": "Sup3****" in the settings dict wasn't flagged. Debugging:

Bug: \b[A-Za-z_] must consume one char before the keyword, so a bare password/token/secret name never matched. Fixing both regexes:

Sed didn't match; using a Python replace instead.

Now the tests.

The three test findings share a snippet, so the cache answered them (correct behavior — the test was wrong). Giving each a distinct snippet:

110 passing. Now I want to actually look at the HTML report. Checking for a headless browser to screenshot it:

Ollama is up now and qwen2.5-coder:7b is pulled, so I'm running a real LLM scan (default cap: 25 findings):

It's slow (>10 min so far). While it runs, I'll check its progress and write the README.

About 55 s per finding: the 7B model is split 55/45 CPU/GPU on this machine, so 11 of the 25 findings are done so far. Writing the README in the meantime.

The monitor is armed and I'll be notified when the report lands. Meanwhile I'm checking that the python -m localaudit fallback works, since the console script isn't on PATH here:

(exit=0 there is from tail, which ends the pipe; the exit code 1 itself is covered by test_fail_on.) Waiting for the LLM scan to finish.

You

write the README with bash instead

Agent

I'll write the README with a bash heredoc, as you asked. I also see that you stopped the LLM scan and its monitor.

The heredoc didn't parse, probably because of the apostrophes in the body. Checking with a minimal test:

A small heredoc works, so something in the body is breaking it. Writing the README in parts to find the culprit:

LocalAudit is built in the current directory and the README is now written. All 110 tests pass, and the rules-only scan of the demo app produces all three reports with every secret masked. A full scan with the LLM never finished, because you stopped it after 23 of the 25 findings had been triaged.

Rules-only scan of examples/vulnerable_app: 43 findings (4 critical, 16 high, 15 medium, 8 low) and a risk score of 100/100. I viewed the HTML report in headless Chrome and it renders cleanly. I also checked every output (HTML, JSON, Markdown, console) for each fake secret, and none of the real values appear anywhere.

LLM run (qwen2.5-coder:7b): about 55 seconds per finding, because the model is split 55% CPU / 45% GPU on your machine. All 23 results I read from the cache were valid JSON and made sense:
- The SQL injection, pickle, eval and secret findings were explained correctly, with the right fixes (parameterised queries, yaml.safe_load, execFile, environment variables).
- It correctly marked the deliberate decoy, subprocess.check_output("df -h /", shell=True) with no user input, as a likely false positive.
- The run never wrote a report, so I haven't seen the "Likely false positives" section filled with real model output. That path is covered by a mocked test.

How to run it:
pip install -e ".[dev]" python -m localaudit scan examples/vulnerable_app --no-llm python -m localaudit scan examples/vulnerable_app --max-llm-findings 5 # with the LLM pytest
On your machine pip put localaudit.exe in ~\AppData\Roaming\Python\Python314\Scripts, which isn't on PATH. Add that folder to PATH to use the localaudit command directly, or use python -m localaudit as above. A rerun of the LLM scan reuses the 23 cached results, so only the remaining 2 hit the model.

Things I fixed or decided along the way:
- I found and fixed a bug where bare names like password = "..." weren't detected as secrets. Because of it, an unmasked password was showing up as a context line in another finding's snippet. There's now a regression test for this.
- Live provider keys (Stripe, AWS and so on) are still flagged when used as an os.environ.get default. Only the generic checks skip environment lookups.
- --fail-on also counts findings the LLM marked as likely false positives, so the model can't turn a CI gate green on its own.

Not verified:
- pip-audit isn't installed. The "not installed" notice is checked on a real scan, but importing pip-audit results is only tested with a mocked output.
- I didn't look at the PDF output of the print stylesheet (no PDF renderer here). Chrome produced the PDF without errors.

Small deviations from the spec:
- The Django SECRET_KEY check sits in secrets.py rather than rules.py, so its value gets masked like any other secret.
- The dependency checks live in scanner.py.
- I added localaudit/__main__.py so python -m localaudit works.

You

Is this a command line based tool?

Agent

Yes, it's purely a command-line tool. There's no GUI, web server or interactive mode. You run one command, it scans and writes its reports, then exits:

python -m localaudit scan PATH [--model M] [--out DIR] [--no-llm] [--max-llm-findings N]
                               [--min-severity LEVEL] [--format html,json,md] [--fail-on LEVEL]

Terminal output: a coloured summary with counts by severity, the risk score and the top findings, plus a progress line while the LLM triages.
Files: report.html, report.json and report.md in the output folder (./localaudit-report by default).
Exit code: 0 normally, 1 when --fail-on is triggered, 2 for bad arguments. That lets it run in CI scripts.

The HTML report is the only visual part. It's a static file you open in a browser after the scan, not a running app.

Once the Scripts folder is on your PATH (see my last message), you can type localaudit scan ... instead of python -m localaudit scan .... Both run the same entry point.

You

what codebase did you test it with?

Agent

I only tested it on code I wrote myself during this session. I haven't run it on any real-world codebase. Specifically:

examples/vulnerable_app/, the demo app I wrote for this project: about 10 files, made up of a Django-style settings.py, views.py and integrations.py, plus a Node server.js, a React widget.jsx, requirements.txt, package.json and deploy.sh. It's deliberately vulnerable, with fake credentials. All the scan results I reported (the 43 findings and the LLM triage) come from this app.
One-line snippets in the unit tests, a known-bad and a safe example for each rule and secret pattern.
Small temporary folders created by the tests, to check the skipped folders, binary and oversized files, and .localauditignore.

That's a real limitation. I wrote the rules and the demo app together, so the app naturally triggers them. The run tells you the tool works end to end. It doesn't tell you how it performs on code it wasn't designed around: how many false positives it produces, what it misses, how fast it runs on thousands of files, or how it handles unusual encodings and minified files.

To get a meaningful answer, the next step is a rules-only scan of a real project. It's quick and makes no LLM calls. A few options:
- One of your own repositories. This is the most useful, because you can judge which findings are real.
- A well-known open-source Django or Express project. I'd need to clone it, which means a network download.
- LocalAudit's own source. This is the quickest. It should be mostly clean, but the test files are full of deliberately bad snippets, so it's a good false-positive check.

Do you want me to run one of these? If you have a repository in mind, give me the path.

You

I have a project that i need to be audited, How can I test that? The codebase is locally saved

Agent

Point the tool at the project folder. Start with a quick rules-only scan, then add the LLM for the most serious findings.

1. Rules-only scan (seconds, no Ollama needed)

cd <projects>\LocalSec
python -m localaudit scan "D:\path\to\your\project" --no-llm --out "D:\audit\myproject"

Use your project's real path, and put quotes around it if it contains spaces. Writing the reports outside the project keeps them out of your repository. When it finishes, open D:\audit\myproject\report.html in a browser.

2. Reduce the noise if needed

If a lot of findings come from folders that aren't your code (vendored libraries, generated files, test fixtures), create a file called .localauditignore in the project's root folder:

vendor/
static/libs/
*.min.js
tests/fixtures/*

Then run the scan again. .git, node_modules, venv, dist and build are already skipped automatically.

3. Add LLM triage for the worst findings

Make sure Ollama is running, then:

python -m localaudit scan "D:\path\to\your\project" --min-severity high --max-llm-findings 10 --out "D:\audit\myproject"

On your machine the 7B model takes about a minute per finding, so 10 findings is roughly 10 minutes. For quicker but rougher triage, add --model qwen2.5-coder:3b. Results are cached, so running the same scan again is fast.

Things to keep in mind
- Nothing leaves your machine. The scan and the model both run locally, and secrets are masked in the reports.
- The one exception is pip-audit. If you install it, it looks up your dependency names and versions online. It isn't installed right now, so nothing goes out.
- This is the tool's first run on real code. Expect some false positives, and some real issues it can't catch, such as multi-line patterns, logic flaws and broken access control.
- It's a first pass, not a full audit. Treat the findings as a checklist to review, not as a verdict. If the project is going to production or handles sensitive data, it still needs a proper review.

If you tell me the project's path, I can run the rules-only scan, go through the findings with you, and tune the ignore file or rules wherever they're noisy.

You

PS ~> python -m localaudit scan "<projects>\getpyqjec" --min-severity high --max-findings 10 --out "<projects>"
usage: localaudit [-h] [--version] {scan} ...
localaudit: error: unrecognized arguments: --max-findings 10
PS ~> localaudit scan "<projects>\getpyqjec" --min-severity high --max-findings 10 --out "<projects>"
localaudit : The term 'localaudit' is not recognized
as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or
if a path was included, verify that the path is
correct and try again.
At line:1 char:1
+ localaudit scan "<projects>\getpyqjec"
--min-severity high ...
+ ~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (localau
dit:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

PS ~>

Agent

There were two separate problems.

1. The option name was wrong (my fault). I mixed up the name in my message at one point. The correct option is --max-llm-findings, not --max-findings. The python -m localaudit form did work: it started and only rejected that one option.

2. localaudit isn't recognized because its install folder isn't on your PATH. Use python -m localaudit for now.

Run this:

python -m localaudit scan "<projects>\getpyqjec" --min-severity high --max-llm-findings 10 --out "<projects>\getpyqjec-audit"

I also changed --out to a new subfolder. With --out "<projects>", the three report files would go straight into your Project folder alongside your other projects. When it finishes, open <projects>\getpyqjec-audit\report.html. Make sure Ollama is running first, and expect up to about 10 minutes for the LLM step.

If you want plain localaudit ... to work, run this once in PowerShell, then open a new terminal:

$p = [Environment]::GetEnvironmentVariable("Path", "User")
[Environment]::SetEnvironmentVariable("Path", "$p;~\AppData\Roaming\Python\Python314\Scripts", "User")

If you'd rather I run the scan here and go through the findings with you, just say so.

You

What is the command to run the application on examples you created, because i would not like to record my project findings and post it on dev

Agent

Run these from the LocalSec folder, since the demo app path is relative to it:

cd <projects>\LocalSec

# Rules only (finishes in about a second)
python -m localaudit scan examples\vulnerable_app --no-llm --out demo-report

# With LLM triage
python -m localaudit scan examples\vulnerable_app --max-llm-findings 10 --out demo-report

Then open demo-report\report.html in a browser.

For a recording:
- Everything in the demo is safe to show. The credentials are fake and masked in the output.
- The LLM run will look instant. 23 results from the earlier run are cached, so --max-llm-findings 10 answers straight from the cache.
- To show the model working live, first run Remove-Item -Recurse .localaudit_cache. Each finding then takes about a minute on your machine, so a small cap like --max-llm-findings 3 keeps the video short.
- Good spots to point out in the report: the subprocess ... "df -h /" finding is the decoy the model marked as a likely false positive, and the private key block shows its body replaced with ****.
- The report header shows your local path (<projects>\LocalSec\...). If that matters to you, crop it or run from a neutral folder.

I can also draft the DEV post text for you to review.

You

give me steps to connect this to a fresh github repo and push everything with commit statement

Agent

Here are the steps. Run everything in PowerShell from the project folder.

1. Create an empty repository on GitHub

Go to github.com/new and name it (for example localaudit). Leave all of the "Add README / .gitignore / license" boxes unticked, because the project already has those files and an extra initial commit on GitHub would make the first push fail.

2. Commit and push

cd <projects>\LocalSec

# Keep demo report output out of the repo (build artefacts and the LLM cache are already ignored)
Add-Content .gitignore "demo-report/"

git init
git add .
git status          # check the list: no demo-report, .localaudit_cache, __pycache__ or *.egg-info
git commit -m "Initial release of LocalAudit: local-only security audit CLI" -m "Deterministic rules detect secrets, insecure code patterns and unpinned dependencies; a local Ollama model only explains findings, flags likely false positives and suggests fixes. Outputs self-contained HTML, JSON and Markdown reports. Includes an intentionally vulnerable demo app and a pytest suite."
git branch -M main
git remote add origin https://github.com/YOUR-USERNAME/localaudit.git
git push -u origin main

Replace YOUR-USERNAME with your GitHub username. If git commit complains that it doesn't know who you are, run these once and commit again:

git config --global user.name "Your Name"
git config --global user.email "[REDACTED]"

3. GitHub may block the push at first

The demo app and tests contain fake keys in real formats, such as ghp_..., sk_live_..., xoxb-... and AIza.... GitHub's secret-scanning push protection may reject the push with a message like "Push cannot contain secrets". That is expected, because the demo is designed to trip exactly these detectors. To get past it:
- The error lists each detected value with a link. Open each link and choose "It's used in tests" to allow it.
- Run git push -u origin main again.

All of these values are made up. Still, read the list in the error before allowing anything, to confirm each flagged file is under examples/vulnerable_app/ or tests/.

If you have the GitHub CLI (gh) installed and logged in, you can skip creating the repo on the website. After the git commit line, run gh repo create localaudit --public --source . --push.

I can also run the git steps here for you. You would still create the repo and approve the push yourself.

You

git push -u origin main
Enumerating objects: 38, done.
Counting objects: 100% (38/38), done.
Delta compression using up to 20 threads
Compressing objects: 100% (35/35), done.
Writing objects: 100% (38/38), 40.87 KiB | 5.11 MiB/s, done.
Total 38 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
remote: error: GH013: Repository rule violations found for refs/heads/main.
remote:
remote: - GITHUB PUSH PROTECTION
remote: —————————————————————————————————————————
remote: Resolve the following violations before pushing again
remote:
remote: - Push cannot contain secrets
remote:
remote:
remote: (?) Learn how to resolve a blocked push
remote: https://docs.github.com/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-from-the-command-line#resolving-a-blocked-push
remote:
remote:
remote: —— Slack API Token ———————————————————————————————————
remote: locations:
remote: - commit: [REDACTED]
remote: path: examples/vulnerable_app/shop/integrations.py:6
remote:
remote: (?) To push, remove secret from commit(s) or follow this URL to allow the secret
remote: <push-protection-bypass-link>
remote:
remote:
remote: —— Stripe API Key ————————————————————————————————————
remote: locations:
remote: - commit: [REDACTED]
remote: path: examples/vulnerable_app/shop/integrations.py:7
remote: - commit: [REDACTED]
remote: path: tests/test_scanner.py:11
remote: - commit: [REDACTED]
remote: path: tests/test_secrets.py:11
remote: - commit: [REDACTED]
remote: path: tests/test_secrets.py:11
remote: - commit: [REDACTED]
remote: path: tests/test_secrets.py:59
remote:
remote: (?) To push, remove secret from commit(s) or follow this URL to allow the secret
remote: <push-protection-bypass-link>
remote:
remote:
remote: —— Slack API Token ———————————————————————————————————
remote: locations:
remote: - commit: [REDACTED]
remote: path: tests/test_secrets.py:10
remote: - commit: [REDACTED]
remote: path: tests/test_secrets.py:10
remote:
remote: (?) To push, remove secret from commit(s) or follow this URL to allow the secret
remote: <push-protection-bypass-link>
remote:
remote:
remote: —— Amazon AWS Access Key ID ——————————————————————————
remote: locations:
remote: - commit: [REDACTED]
remote: path: examples/vulnerable_app/shop/settings.py:18
remote: - commit: [REDACTED]
remote: path: tests/test_scanner.py:10
remote:
remote: (?) To push, remove secret from commit(s) or follow this URL to allow the secret
remote: <push-protection-bypass-link>
remote:
remote:
remote: —— Amazon AWS Secret Access Key ——————————————————————
remote: locations:
remote: - commit: [REDACTED]
remote: path: examples/vulnerable_app/shop/settings.py:19
remote: - commit: [REDACTED]
remote: path: tests/test_scanner.py:10
remote:
remote: (?) To push, remove secret from commit(s) or follow this URL to allow the secret
remote: <push-protection-bypass-link>
remote:
remote:
remote:
To https://github.com/hardikgaikwad/localaudit.git
! [remote rejected] main -> main (push declined due to repository rule violations)
error: failed to push some refs to 'https://github.com/hardikgaikwad/localaudit.git'

Agent

That's the expected block. GitHub flagged five fake values, and every location is either in examples/vulnerable_app/ or in tests/. They are the made-up keys I wrote so the scanner has something to find, not real credentials.

To push:

Open each of the 5 links from the error while signed in as hardikgaikwad:
- Slack token (integrations.py): <push-protection-bypass-link>
- Stripe key: <push-protection-bypass-link>
- Slack token (tests): <push-protection-bypass-link>
- AWS access key ID: <push-protection-bypass-link>
- AWS secret access key: <push-protection-bypass-link>
On each page, choose "It's used in tests" and click Allow secret.
Push again within a couple of hours, while the allowances are still valid. You don't need to change the commit: powershell git push -u origin main

What to expect afterwards:
- If the repo is public, GitHub may still open secret-scanning alerts for these values under the repo's Security tab. Close them as "Used in tests".
- GitHub may also pass the AWS, Slack and Stripe values to those providers to check. They're fake, so nothing happens.
- The fake GitHub token, Google key and JWT weren't flagged. GitHub's checks for those formats are stricter, and my made-up values don't pass them. That's fine.

If you'd rather not bypass push protection: I can change the tests to build the fake keys at runtime (for example "sk_" + "live_" + ...), so the literal strings never appear in them. The demo app has to keep literal keys in its files, though, or the scanner has nothing to find. That would cut the flags down to the demo files, but you'd still need to allow those, so the five-click route above is simpler.

You

Make sure the README mentions that there are fake API keys in the example application

Agent

I couldn't make this change. Auto mode's safety check returned no verdict for both README edits, the same thing that happened with the first README write. I haven't routed around it this time. You approved using Bash for that earlier write, but I won't assume that approval carries over to this edit.

You have two options:
- Tell me to use Bash again and I'll apply both changes that way.
- Paste them in yourself. Here is the exact text.

1. In the quickstart, directly after the line You can try it without Ollama: `localaudit scan examples/vulnerable_app --no-llm`. add:

> **Heads-up: the demo app contains fake API keys.** `examples/vulnerable_app/` and the tests include made-up credentials in real formats (AWS, Stripe, Slack, GitHub, Google, a JWT, a private key block) so the secret rules have something to find. None of them are real or valid. See [Demo app](#demo-app).

2. Replace everything under ## Demo app (the last section) with:

`examples/vulnerable_app/` is **intentionally insecure**. It exists only to show the rules firing. Don't deploy or copy it.

**It contains fake API keys and credentials.** To exercise the secret detector, the demo app and the tests (`tests/test_secrets.py`, `tests/test_scanner.py`) hold made-up values in the exact formats of real ones:

| Kind | Where |
|---|---|
| AWS access key ID and secret access key | `shop/settings.py` |
| Database password, Django `SECRET_KEY` | `shop/settings.py` |
| GitHub token, Slack bot token, Stripe live key, Google API key, JWT, signing salt | `shop/integrations.py` |
| RSA private key block | `shop/integrations.py` |

These values are **not real, have never been valid, and grant access to nothing**. They were invented for this project. Some contain `FAKE` or `NOTAREALKEY`, but not all of them do, because real formats don't always leave room for it.

Because they match real formats, other secret scanners will flag them too. GitHub's push protection blocks the first push until you allow each one as "used in tests". Repository secret-scanning alerts can be closed as "used in tests". If you fork this project, expect the same.

Never replace them with real credentials, even for testing.

After the README is updated, commit it and push. You still need to allow the five flagged values first, using the links from the push error, because the blocked first commit is part of this push.

git add README.md
git commit -m "Document fake API keys in demo app and tests"
git push -u origin main

You

use bash to update README

Agent

I updated the README using Bash, and it now says clearly that the demo app contains fake API keys:

Quickstart (line 28): a short notice right under the demo command saying the demo app and tests contain made-up credentials in real formats. It links to the Demo app section.
Demo app section: rewritten with a table of every fake credential and the file it's in. It says the values are invented and grant access to nothing, warns that GitHub push protection and other secret scanners will flag them, and says never to replace them with real credentials.

To publish it, first allow the five flagged values using the links from the push error, since the blocked commit is still part of this push. Then run:

git add README.md
git commit -m "Document fake API keys in demo app and tests"
git push -u origin main

You

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

My Agent Session

My First User

Prize Categories

Use this as the template and rewrite this for the devpost, leave placeholders for demo and github embeds. Tell me separately how to add devrelay session in the submission

Agent

I'll look up the challenge details and the session-embed instructions first, so the prize categories and embed syntax are accurate rather than guessed.

Here's the post, ready to paste into DEV. I checked the details against the live challenge page:
- Theme: Side Hustle.
- Deadline: October 18, 11:59 PM PDT, which is two days from now.
- Required tag: #hf26challenge.

---
title: LocalAudit: an AI security auditor that never sees your secrets and never leaves your laptop
published: false
tags: hf26challenge, security, opensource, ai
---

*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle](https://dev.to/challenges/hacktoberfest-week2-2026-10-12)*

## What I Built

**LocalAudit** is a command-line security auditor for source code that runs 100% on your own machine. Point it at a repo and a few seconds later you get a clean HTML report (plus JSON and Markdown) listing hardcoded secrets, injection risks, insecure configuration and unpinned dependencies. Each finding comes with the file and line, the code around it, a plain-English explanation and a concrete fix.

The design rule I cared most about: **the LLM never detects anything.** Deterministic rules find the issues: 24 code-pattern rules for Python/Django/Flask and JavaScript/Node, 10 built-in secret detectors (AWS, Stripe, Slack, GitHub, Google, JWTs, private keys, high-entropy strings...) and dependency checks. A local open-weight model then *triages* each finding. It explains it, judges whether it's likely a false positive in context, and writes a fix. If the model is unavailable, you still get the full report.

**Who would pay for this?** People who need a security pass on code they aren't allowed to upload anywhere:

- **Freelancers and small agencies** working on client code under NDA. A fixed-fee "pre-handoff security check" with a professional report as the deliverable is an easy add-on to sell.
- **Solo founders** about to launch who can't justify an enterprise SAST subscription.
- **Consultants** who want a fast first pass before a manual review.

For all of them, the cloud AI code-review tools have the same problem: *your client's source code goes to someone else's server.* LocalAudit's answer is that it doesn't go anywhere.

## Demo

<!-- DEMO VIDEO: paste your video link below, e.g. {% embed https://www.youtube.com/watch?v=... %} -->
[DEMO_VIDEO_PLACEHOLDER]

The demo scans the bundled `examples/vulnerable_app`, a deliberately insecure Django + Node app full of **fake** credentials. The rules-only scan takes about a second and finds 43 issues (4 critical, 16 high), for a risk score of 100/100. A few things worth watching for:

- **Secrets are masked everywhere.** You'll only ever see `AKIA****`. Even a private key's body is replaced with `****` before any snippet is built, so raw secrets never reach the report, the console or the model's prompt.
- **The model catches a decoy.** The app contains `subprocess.check_output("df -h /", shell=True)`. The rule correctly flags `shell=True`, but the command is a constant with no user input. The local model marked it as a *likely false positive*. LocalAudit doesn't delete it: it moves it to a separate "Likely false positives" section for a human to confirm.
- **It falls back gracefully.** Stop Ollama and the tool prints the exact command to fix it (`ollama pull qwen2.5-coder:7b`), then carries on with the rules-only report.

## Code

<!-- GITHUB EMBED: e.g. {% github hardikgaikwad/localaudit %} -->
[GITHUB_EMBED_PLACEHOLDER]

```bash
ollama pull qwen2.5-coder:7b
pip install -e .
localaudit scan examples/vulnerable_app
# open localaudit-report/report.html
```

Useful flags: `--no-llm`, `--min-severity high`, `--max-llm-findings 10`, `--model <any Ollama model>`, and `--fail-on high` (exits with code 1, for CI).

## How I Built It

**The open-source AI at the core:**

- **[Ollama](https://ollama.com)** for local inference over its HTTP API.
- **Qwen2.5-Coder 7B**, an open-weight coding model. It's the default, and you can swap it with `--model` or `LOCALAUDIT_MODEL`.

**How the pipeline works:**

1. **Scan.** Walk the repo, skipping `.git`, `node_modules`, venvs, build output, binaries, files over 1 MB and anything in `.localauditignore`.
2. **Detect and redact.** Run the secret detectors first and redact every detected value from the file text. Only then run the code rules and build snippets, which is how masking is guaranteed everywhere downstream.
3. **Triage.** Send each finding, highest severity first, to the local model. The request includes the rule description, the file path and the *masked* snippet. It runs at temperature 0, and Ollama's `format` parameter enforces a JSON schema, so the model must return `{explanation, risk_in_context, likely_false_positive, confidence, suggested_fix, fixed_code_example}`. Invalid JSON gets one retry, then the finding is marked "LLM triage unavailable".
4. **Cache.** Results are cached on disk, keyed by model + rule + snippet, so reruns are instant.
5. **Report.** Render a single self-contained HTML file with no CDNs or external requests, plus a print stylesheet so "Save as PDF" gives a client-ready document.

The stack is deliberately small: Python 3.10+, `requests`, `jinja2` and pytest, with 110 tests. Those tests include a mocked Ollama covering invalid JSON, timeouts and the server going down mid-run.

I built it with an AI coding agent (Claude Code) doing much of the implementation while I directed the design and reviewed the results. One moment I liked: testing caught a regex bug where a bare `password = "..."` wasn't detected. Because the masker only masks what the detector finds, that password was showing up unmasked as a *context line* in a neighbouring finding's snippet. It's fixed now, and there's a regression test for it.

## Why Does Open Innovation Matter?

For this product, open weights aren't a nice-to-have. They're the reason it can exist.

- **Privacy is the product.** The customers I described *can't* send client code to a third-party API. A local open-weight model is the only way to give them AI explanation
[... truncated 3409 chars]

You

Sure save the devrelay session

Agent

I'll start by finding this session's transcript on disk.

There are two transcript files, so the session probably continued from one file into the other. Checking how each is structured:

The transcript is built: 26 messages, about 200 KB. Only one project path was replaced, though, which seems too low. Before going further I'll grep the output for anything that still leaks:

The scrub is incomplete: 54 project paths and some home-directory paths are still in there. Debugging the path regex:

DEV Community

LocalAudit: an AI security auditor that never sees your secrets and never leaves your laptop

What I Built

Demo

Code

hardikgaikwad / localaudit

LocalAudit

60-second quickstart

How I Built It

Why Does Open Innovation Matter?

My Agent Session

My Agent Session

Core principle

Stack (keep it minimal)

Features

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

My Agent Session

My First User

Prize Categories

Top comments (0)