DEV Community: NEXADiag Nexa

How I caught a hardcoded API key that a Senior Expert missed (using Multi-LLM Consensus)

NEXADiag Nexa — Tue, 28 Jul 2026 16:05:01 +0000

Last week, I decided to run a "Truth Test" on my latest project, Propulse.

I invited a Senior Infrastructure Expert to perform a manual security audit. He spent over an hour digging through the code. He did an amazing job, finding 27 vulnerabilities, including 4 critical ones related to Authentication and Session Storage.

Then, I ran the same codebase through my consensus engine, NexaVerify.

The result? It took 12 seconds.

The Human vs. Machine Showdown

Metric	Human Expert	NexaVerify (8-AI Consensus)
Time Spent	~60 minutes	12 seconds
Issues Found	27	91 (validated by consensus)
Silent Killers	Missed	1 (Hardcoded Cloudinary API Key)

The "Invisible" Leak

The most striking discovery wasn't in the logic files (.tsx or .ts). It was buried in a documentation file: HANOUTI-GUIDE.html at line 662.

NexaVerify flagged a Cloudinary API Secret exposed in plain text inside a guide. Because humans (and most standard linters) often ignore the /docs or static .html folders during a rush, this leak had been sitting there for weeks.

Why Consensus Wins

Why did the AI find it when the human missed it? It’s not because the AI is "smarter." It's because the AI doesn't get tired and my architecture forces Consensus.

In this run, Gemini, Groq, and Cerebras acted as a jury. When multiple models flag a pattern as a security risk, the signal overrides the noise.

I also identified 10 conflict zones where the models disagreed. This is where I saved the most time: instead of re-reading 12,000 lines, I only focused on the areas where the AI jury was fighting.

Engineering > Vibe Coding

We are moving away from the era of "asking an AI if the code is good." That's probabilistic gambling.

The future is Multi-stage validation:

Deterministic Triage (Static scans)
Multi-LLM Consensus (Semantic reasoning)
Runtime Reality (Sandbox execution)

Conclusion

The machine didn't replace the expert; it prepared the ground for him. By automating the "obvious" and the "hidden" patterns, NexaVerify allowed me to spend my mental energy on architecture, not on hunting for hardcoded strings.

How many secrets are sleeping in your /docs folder right now?

I'm Redha, Software Architect at NEXADiag. Building local-first tools for engineers who value determinism.
Test NexaVerify Beta here

Stress-testing my Multi-LLM engine: 93 chunks, 8 models, and one "Insufficient Balance" error.

NEXADiag Nexa — Sat, 25 Jul 2026 17:30:29 +0000

Last week, I decided to push my consensus engine, NexaVerify, to its absolute limits. I wanted to see if my architecture could handle a massive parallel load while maintaining high reliability.

I chose to test it against a real-world infrastructure: the DEVUP AI Gateway.

The Setup

Files scanned: 92
Logic Chunks: 93 (split using my ast_chunker.py logic)
Jury: 8 AI models running in parallel (including DeepSeek, Gemini, and Groq).
Mode: Full Consensus Audit.

The Results: A War of Latency

The technical feedback was immediate.

The Rate Limit Wall: Giant providers like Groq and Gemini Flash hit the 429 Too Many Requests wall almost instantly due to the burst of parallel chunks.
Sovereign Stability: The DEVUP Gateway (local Algerian infra) handled the rafale of requests with zero latency penalty. It proved that a dedicated circuit breaker at the cluster level is superior to generic API endpoints for mass audits.
The Real-Time Billing Test: The scan ended when I hit a 402 Payment Required error. While frustrating, it was a victory: it proved the real-time DZD billing synchronization works perfectly under heavy load.

The Lesson: Disagreement is the Signal

Why run 8 models? Because when you are dealing with 30k+ lines of code, one AI's "Looks good" is a gamble.

In this test, the engine found 99 issues. The killer signal wasn't the agreement, but the friction. By forcing models to challenge each other (Adversarial Review), I reduced the noise and focused on the logic flaws that actually matter.

What's Next?

I'm currently preparing v1.6.1 which will officially integrate Claude Opus 5 and Kimi K3 via the DEVUP infrastructure.

How do you handle AI hallucinations in your PRs? Do you trust a single model or do you use a validation layer?

I'm Redha, Software Architect at NEXADiag. Building local-first tools for engineers who value determinism.
Test NexaVerify Beta here

Why I stopped asking one AI to review my code (and built a Consensus Engine instead)

NEXADiag Nexa — Tue, 21 Jul 2026 09:45:03 +0000

Why I stopped asking one AI to review my code (and built a Consensus Engine instead)

We’ve all been there: You ask Claude or GPT-4o to review a Pull Request. It says "Looks good! ✨". You ship. Production crashes 10 minutes later.

The problem isn't that the AI is bad; it's that it's probabilistic. In engineering, we need determinism.

As a former Mechanical Designer turned Software Architect, I learned early that "looks right" is a recipe for disaster. One wrong dimension in a blueprint leads to a scrapped part. I applied this same "Short Leash" principle to my code by building a multi-LLM consensus engine.

1. The "Thinkslop" Trap

Single-model code reviews are dangerous because LLMs tend to be "polite." They follow the path of least resistance.

I call it Thinkslop: code that looks fluent and compiles perfectly but is architecturally fragile or contains silent logic failures. When you rely on one model, you are essentially asking an intern to grade their own homework.

2. Why SAST isn't enough anymore

Static Application Security Testing (SAST) tools like Sonar, Snyk, or Semgrep are essential. They are the "police" of code: they check your seatbelt and your tire pressure.

But SAST is blind to intent. It can't tell you if your dynamic import logic will fail in a specific production edge case.

Consensus is the monitor of vital signs. It doesn't just look for "illegal" code; it looks for inconsistency.

3. The Power of Structured Disagreement

In my tool, NexaVerify, I force 8 different models (GPT-4o, Claude, Gemini, Groq, Cerebras, Mistral, etc.) to audit the same codebase in parallel.

The real signal isn't when they all agree. The value is in the disagreement.

If 7 models stay silent but one (like Cerebras or Groq) flags a potential NameError in a sub-module, that's your most valuable data point. It forces the human architect to look exactly where the "AI brain" started to hallucinate.

4. Real World Result: The 30,103 Lines Test

I recently ran a consensus scan on NexaVerify's own source code.

Lines scanned: 30,103
Issues found: 99
The "Killer" bug: A silent NameError in app/config/constants.py.

Because of a dynamic import, my local linter and a single pass with GPT-4o missed it. The consensus engine caught it because three different models couldn't agree on the scope of the global variables.

Conclusion: From "Vibe Coding" to Engineering

We are moving from "Coding with AI" to "Architecting with AI components." If you are building production-grade software, you can't trust a single oracle. You need a jury.

I’m building NexaVerify in public to tackle this trust gap.

What’s your current stack for AI code validation? Do you trust a single model or do you have a manual verification checklist?

I'm Redha, Software Architect at NEXADiag. I build local-first tools for developers who care about reliability.
Check out NexaVerify Beta here

I’m sick of AI “Thinkslop” in my PRs

NEXADiag Nexa — Wed, 15 Jul 2026 10:20:29 +0000

I started my career as a “Dessinateur Projeteur” (Design Drafter) in mechanical engineering. In that world, if your plan is off by 1mm, the part is scrap. There is no “hotfix” for a piece of steel once it’s machined.

When I started building my own software tools using LLMs, I brought that same paranoia with me. And honestly, it’s the only thing saving my codebases right now.

Lately, I’ve been fighting a specific enemy: Thinkslop.

It’s that code that looks “clean,” passes a linter, and even runs. But the logic is hollow. It’s the result of asking a single AI to “just fix this” and hitting copy-paste because you’re in a rush. It’s dangerous because it looks right.
One model is not a code review

I don’t care if you’re using Claude 3.5 Sonnet, GPT-4o, or a custom fine-tune. A single model has a single perspective based on its training. If the model is biased toward a specific pattern, it will confidently hallucinate a bug right into your production.

Most devs think that if the AI says “Looks good!”, they are safe.
The truth? The disagreement is the only signal that matters.
The “Consensus” approach

I got so tired of manually comparing browser tabs or doubting my own code that I started running my logic through 8 different models at once (Claude, GPT, Gemini, Llama, Groq, etc.).

When they all agree? It’s probably safe.
When Claude flags a potential memory leak but GPT doesn’t? That’s exactly where the real bug is hiding.

I built a local-first engine called NexaVerify to handle this orchestration for me. It doesn’t just “vote” on the code; it surfaces the friction. It highlights the spots where 8 different “brains” can’t agree on the architecture.

I ran it on its own 26k line codebase last week. It found a NameError in a constant file that had been sitting there for a month. A single pass from GPT had missed it every time because the “vibe” of the code was correct.
The takeaway

Stop treating AI as an oracle. Treat it as a junior dev who is prone to lying but very fast at it.

As a former “Dessinateur Projeteur,” I know that a structure is only as strong as its verified points. The only way to trust AI code is to force it into a consensus.

How are you guys catching AI hallucinations before they hit prod? I’m curious if anyone else is building multi-model pipelines or if everyone is still just “vibe coding” and hoping for the best.

# I stopped trusting a single AI for code review — here's

NEXADiag Nexa — Mon, 01 Jun 2026 18:45:41 +0000

I stopped trusting a single AI for code review — here's why

We've all been there:

You ask GPT-4o or Claude 3.5 to review your PR.
It says "Looks good!"
You ship.
Production crashes on an edge case the AI hallucinated its way past.

The problem isn't that AI makes mistakes. It's that we trust a single model. Every family has blind spots — GPT overexplains but misses logic errors, Gemini is great at structure but weak on security patterns, Claude is thorough but sometimes invents APIs that don't exist.

One model's "looks good" is just one vote.

Multi-LLM consensus

I built NexaVerify v1.6.0 — a local-first Windows tool that runs your code through 8 AI engines in parallel (Claude, GPT-4o, Gemini, Groq, Cerebras, Mistral, OpenRouter, Ollama).

The core idea:

Agreement is noise. If all models say it's fine, it probably is.
Disagreement is the signal. When Claude flags a security risk but GPT ignores it — that's where you need to look.

Every issue gets a confidence score based on how many providers confirmed it. Disagreements are surfaced, not buried.

Proof by fire: scanning itself

I ran v1.6.0 against its own 26,000-line codebase with 3 free-tier providers (Gemini, Groq, Cerebras).

99 real issues found. Including a potential NameError in constants.py and a missing try-except in main.py that would crash the app before logging initializes. The bug-finder had bugs — and it found them.

The full report is live here.

Under the hood

3-stage JSON repair — syntactic repair → fallback extraction → schema validation. Catches truncated LLM responses instead of silently returning [].
RSA Proof Bundles — every verdict carries a SHA-256 audit trail. Verifiable after the fact.
Local-first, BYOK — your code and keys never touch my server.
Ollama support — full offline consensus for sensitive projects.
Free tier available today — 3 analyses/day, 3 providers (Gemini, Groq, Cerebras), 10 files/scan.

No signup wall. Download, add your API keys, run.

→ Try it free or grab Pro (€19 lifetime): https://nexaverify.netlify.app/

I ship solo. Every feature is driven by what early users actually need. What's your current workflow for verifying AI-generated code? Running a single pass, or already testing multi-model pipelines?

Why JSON.parse() Fails Silently on Truncated LLM Responses (And What I Did About It)

NEXADiag Nexa — Wed, 13 May 2026 11:41:17 +0000

Why JSON.parse() Fails Silently on Truncated LLM Responses (And What I Did About It)

If you've shipped anything that asks an LLM to return JSON, you've already hit this bug. You just may not have noticed.

The LLM returns a response. Your code parses it. Most of the time it works. Sometimes it returns {} and you assume the LLM didn't find anything. The reality is darker: the JSON was truncated mid-object, your parser silently failed, and your downstream code is now operating on an empty dictionary instead of the partial result the LLM actually produced.

I lost six weeks to this bug. Here's what I learned.

The setup

I run code review with multiple LLMs in parallel. Each one returns a JSON array of issues found:


json
[
  {"file": "main.py", "line": 47, "type": "security", "severity": "high", "description": "..."},
  {"file": "main.py", "line": 89, "type": "smell", "severity": "low", "description": "..."}
]

When the LLM hits its max_tokens limit mid-response, the response gets cut off. You receive something like:
json

[
  {"file": "main.py", "line": 47, "type": "security", "severity": "high", "description": "..."},
  {"file": "main.py", "line": 89, "type": "smell", "seve

json.loads() raises JSONDecodeError. Most code catches the exception and returns []. The issues that WERE successfully parsed before the truncation are lost.
The dumb solution that actually works

You don’t need a streaming JSON parser. You need a bracket-counting repair function:
python

def guard_truncation(text: str, provider_id: str, file_path: str) -> str:
    stripped = text.strip()
    if not stripped.startswith("["):
        return text

    try:
        json.loads(stripped)
        return text  # already valid
    except json.JSONDecodeError:
        pass

    # find last complete object
    last_close = stripped.rfind("}")
    if last_close == -1:
        return "[]"

    # rebuild a valid array from the last complete object backward
    repaired = stripped[: last_close + 1] + "\n]"
    try:
        json.loads(repaired)
        return repaired
    except json.JSONDecodeError:
        return "[]"

It’s not elegant. It works. You recover 80-90% of the partial result instead of 0%.
The second bug that this revealed

Here’s where it gets worse.

My downstream code assumed every entry in the parsed list was a dictionary. Most of the time it was. But occasionally an LLM would return a string entry in the middle of the array:
json

[
  {"file": "main.py", "line": 47, ...},
  "I noticed there might be an issue here but I'm not sure",
  {"file": "main.py", "line": 89, ...}
]

My code did entry.get("file") on every entry. When it hit the string, AttributeError: 'str' object has no attribute 'get'. The exception was caught by a try/except too wide to be useful. The entire scan silently produced empty results for that file.

Six weeks. No error log. The only signal was “the report has fewer issues than usual for this codebase”.

The fix:
python

for entry in raw_issues:
    if not isinstance(entry, dict):
        continue
    # safe to call entry.get(...) here

Three lines. That’s it.
The bigger lesson

I don’t think LLM output should ever be trusted to match a schema. Even when you tell it “return valid JSON only”, you’ll get:

    Truncated JSON when you hit token limits
    Strings injected mid-array as informal commentary
    Wrong types in correct keys (line: "approximately 50" instead of line: 50)
    Extra keys not in your schema
    Missing required keys

The temptation is to use Pydantic or a JSON schema validator and reject malformed responses entirely. That’s the worst possible choice — you lose all the partial work the LLM did. The better choice is to repair what you can, type-check defensively at every step, and log what you couldn’t recover so you can iterate.

Three patterns that have saved me from similar bugs:

    Always isinstance(x, dict) before .get() on LLM-derived data. Always.
    Bracket-repair truncated JSON before declaring failure. 80% recovery beats 0%.
    Log what you discarded. If you silently filter bad entries, you’ll never know how often it happens. I now log every malformed entry with the provider name and file path.

Why this matters in 2026

Most teams treat LLM output as “either it works or it doesn’t”. The reality is closer to “it partially works most of the time, and the partial-failure modes are silent”. Production code that runs LLM output needs to be more paranoid than production code that talks to a normal API, because LLMs don’t have HTTP status codes — they have a single channel that mixes intent, format, and content.

I built my entire scanning workflow around the assumption that any single LLM response will be 5-10% broken. That assumption has been a better friend than any prompt engineering trick.

What’s your experience? Anyone else burned by silent truncation, or am I the last one to notice?

Stop copy-pasting AI code: The 6-step validation checklist for devs.

NEXADiag Nexa — Wed, 15 Apr 2026 14:42:58 +0000

It is impossible to be 100% certain that a tool or code generated by an LLM (like ChatGPT, Claude, etc.) is bug-free. LLMs are text predictors: they generate code that looks correct, but they do not "compile" or execute the code internally. Consequently, they can invent functions that do not exist (hallucinations) or make subtle logic errors.

However, you can achieve a very high level of confidence by following a rigorous validation method. Here are the essential steps:

1. Code Review (Never just copy-paste)

Have the code explained: Ask the LLM: "Explain this function to me line by line." If the explanation is logically sound, that is a good sign.
Check the business logic: Does the tool do exactly what you want, or did it simplify the problem to provide a faster answer?
Watch for LLM "habits": LLMs tend to use popular libraries even if they aren't the best fit, or they might ignore error handling (try/catch).

2. Edge Case Testing

This is where LLMs fail most often. A tool might work perfectly with normal data but crash with unusual data. Test for:

Empty inputs: What happens if you provide nothing?
Extreme values: A negative number where it should be positive? A text string of 10,000 characters?
Special characters: Accents, emojis, or HTML tags (<script>).
Wrong format: If the tool expects a date (DD/MM/YYYY), what happens if you type "Monday"?

3. Dependency Validation

LLMs sometimes invent package names or use obsolete functions.

Verify that every import (Python), require (Node.js), or using (C#) corresponds to an actual, existing library.
Check that the library version is compatible with your environment.

4. Use Automated Tools (Don't do everything manually)

Run the LLM's code through real development tools:

Linters: Tools like ESLint (JavaScript), Pylint (Python), or Ruff detect syntax errors and poor practices.
Type Checkers: If using TypeScript or Python with "Type Hints," the compiler will catch many silent errors (e.g., passing a string to a function expecting a number).
Ask the LLM to write unit tests: Ask: "Write unit tests (using Jest, PyTest, etc.) for this code including nominal and edge cases," then execute those tests.

5. Security Check (Crucial)

Never trust an LLM with security.

Check for hardcoded passwords or API keys in the script.
If the tool interacts with a database, ensure there is protection against SQL injections (using parameterized queries).
If the tool takes user input, ensure the data is sanitized before being displayed or processed.

6. Cross-Checking Technique (Pitting LLMs against each other)

If you have doubts about a complex piece of code:

Take the code generated by ChatGPT.
Open Claude or Gemini and ask: "Here is code generated by an AI. Find the bugs, security flaws, or performance issues."

LLMs have different biases. An error that goes unnoticed by one is often caught by another.

A note: this checklist is partly automated in NexaVerify, the multi-LLM consensus scanner I'm building. Step 6 (LLM cross-checking) is its core mechanic. Free tier on Gumroad if you want to try it.