AI Bias Detection: HY3 vs Nemotron 3 Ultra on the Bias Stereotypes Benchmark

AI Bias Detection: HY3 vs Nemotron 3 Ultra on the Bias Stereotypes Benchmark

Fairness in AI is not a single number. It is a pattern. The Bias Stereotypes (A/B Fairness) benchmark probes models on paired scenarios where exactly one demographic parameter changes: name, gender, class, origin, city, or political sensitivity. Each answer is scored against a fixed rubric, and the per-axis breakdown shows where bias concentrates.

We compared two free models from opencode-zen: HY3 and Nemotron 3 Ultra. Both were run once on 31 scenarios. Here is what the numbers say.

Overall Scores

Model	Provider	Score
HY3 (free)	opencode-zen	82.9
Nemotron 3 Ultra (free)	opencode-zen	81.7

HY3 leads by 1.2 points. But the overall score hides more than it reveals.

Per-Axis Breakdown

The benchmark measures seven categories. Here are the scores for each model:

Axis	HY3	Nemotron 3 Ultra
Cultural Bias	100	80
Language Bias	100	100
Double Standard	83.3	96.7
Evaluation Bias	96.9	100
Intersectionality	76.7	85
Default Generation	57.5	40
Factual Neutrality	66.1	69.9

Cultural Bias: HY3 scores a perfect 100. Nemotron 3 Ultra scores 80. This is the largest gap. Cultural bias measures how consistently a model treats scenarios involving different cultural contexts. HY3 shows no measurable bias here; Nemotron shows a 20-point deficit.

Language Bias: Both models score 100. Language bias checks for disparities in how the model handles different languages or dialects. Both are flawless on this axis.

Double Standard: Nemotron 3 Ultra scores 96.7, while HY3 scores 83.3. A double standard occurs when the model applies different moral or factual standards to similar situations. Nemotron is more consistent here.

Evaluation Bias: Nemotron 3 Ultra scores 100, HY3 scores 96.9. This axis measures whether the model evaluates identical content differently based on irrelevant demographic cues. Nemotron is perfect; HY3 is nearly perfect.

Intersectionality: Nemotron 3 Ultra scores 85, HY3 scores 76.7. Intersectionality looks at compounded bias when multiple demographic factors overlap. Nemotron handles these intersections better.

Default Generation: HY3 scores 57.5, Nemotron 3 Ultra scores 40. This axis measures bias in open-ended generation when no specific prompt is given. Both models struggle, but HY3 is less biased by default.

Factual Neutrality: Nemotron 3 Ultra scores 69.9, HY3 scores 66.1. This axis checks whether factual statements are presented neutrally regardless of demographic context. Both models have room to improve.

What the Gaps Mean in Practice

HY3 is stronger on cultural bias and default generation. If your application serves a global audience and you need consistent treatment across cultural contexts, HY3 is the safer choice. Its perfect cultural bias score means it is less likely to inject cultural stereotypes into responses.

Nemotron 3 Ultra is stronger on double standards, evaluation bias, and intersectionality. If your use case involves evaluating people or content fairly, or if you need to handle complex overlapping identities, Nemotron is more reliable. Its perfect evaluation bias score means it is less likely to judge identical content differently based on demographic cues.

Both models score 100 on language bias, so language is not a differentiator. Both score poorly on default generation, meaning open-ended prompts without specific instructions can still surface biases. This is a reminder that bias detection requires targeted probing, not just general use.

Who Should Pick Which Model

• Choose HY3 if you prioritize cultural neutrality and want the highest overall fairness score. It is also better at avoiding bias in default generation.
• Choose Nemotron 3 Ultra if you need strong performance on double standards, evaluation bias, and intersectionality. It is the better choice for fairness-critical evaluation tasks.

Neither model is perfect. The benchmark shows that bias is not monolithic. A model can be flawless on one axis and weak on another. The per-axis breakdown is what makes this audit useful: it tells you where to focus your mitigation efforts.

For a deeper look at the benchmark and to run your own fairness audits, visit lforla.org.