DEV Community: Gulrez Qayyum

Can LLMs Keep Score? I Benchmarked Tennis and Padel Score-Tracking

Gulrez Qayyum — Sat, 10 Oct 2026 18:28:23 +0000

This is a submission for the Kaggle Benchmarking Challenge

What I Benchmarked

I wanted a test where the right answer is unambiguous, so I picked something simple that is surprisingly easy to get wrong: keeping score in a match.

Each case gives a model a list of points in order ("A A B A B ...", each letter is who won that point). The model has to report the state right after the last point: sets, games, points in the current game, and who serves next. I wrote a small scoring engine in Python that computes the true answer, so every reply is simply right or wrong. No judge model is involved.

The benchmark covers tennis (advantage scoring) and padel (golden point). It includes sequences of 30, 60 and 100 points, plus special cases forced into a 6-6 tiebreak, where the serving order is awkward. Every point sequence is asked twice: once with the full rules written in the prompt, and once with only "Sport: padel (golden point scoring...)", so the model has to rely on what it already knows. The final benchmark has 48 cases.

Models Tested

I ran it on five models available in Kaggle Benchmarks: Gemini 3.7 Flash, gpt-oss-20b, Claude Haiku 4.5, GPT-5.4 nano, and Gemma 4 26B A4B (this one errored on Kaggle, so it has no score). I chose a mix of a strong fast model, a small reasoning model, and small models that answer quickly, because I expected the score to depend on how much a model works things out step by step. Everything ran with default settings, once per case.

Findings

Model	Accuracy (48 cases)
Gemini 3.7 Flash	0.96 (46/48)
gpt-oss-20b	0.42 (20/48)
Claude Haiku 4.5	0.00 on the Kaggle leaderboard (1/48 when I reran it myself)
GPT-5.4 nano	0.00
Gemma 4 26B A4B	error

Score vs. total cost for the three models Kaggle had cost data for (gpt-oss-20b and Gemma 4 are not shown). Claude Haiku 4.5 spent about $0.29 and scored about 0, so spending more tokens didn't help it.

The spread is huge. Keeping score in a match is easy for a person, and two models got almost none of 48 right. The rough 95% ranges are about 86-99% for Gemini and 29-56% for gpt-oss-20b, so those two do separate clearly.
Not working it out vs working it out badly. GPT-5.4 nano wrote about 25 tokens per case, just the answer line, and guessed. Some answers were impossible states, such as a match at 2-1 in sets while it was still going. Claude Haiku 4.5 wrote about 1,100 tokens of step-by-step counting per case and still scored about 0. I checked whether this was a formatting problem: every Haiku reply I looked at ended with a correctly formatted answer line that my scorer parsed. The mistakes were in game counts and who serves next. So writing out the counting isn't enough on its own.
Some of the errors were noise. In an earlier 100-case run, all 5 of Gemini's mistakes were in the no-rules condition, which looked like "the model needs the rules." I reran those 5 cases twice and 9 of 10 passed. With only 10 reruns that is consistent with noise but not proof (the plausible range is roughly 60-98%). Haiku shows the same variation: 0.00 on the leaderboard, 1/48 in my own rerun. Single-run results overstate differences between conditions.
I did not find a padel weakness or a rules effect. The 48 cases are really 24 point sequences, each asked twice (rules in the prompt, rules not in the prompt), so the two conditions are paired. The right way to compare them is to count the sequences where the answer changed between the two prompts. I had 5 such cases, which isn't enough to claim an effect. Padel and tennis scores for Gemini were within one error of each other.

What I'd Measure Next

Run each case several times, because one run per case is too noisy to compare conditions.
Turn on extended reasoning for the models that scored 0.00, to see whether the problem is the model or the default setting.
Get per-sport and per-length results for every model, not just Gemini.
Try 200-point sequences, since the best model was close to the ceiling.
Compare with-rules vs without-rules as paired results (count the sequences whose answer flips) instead of two separate averages.

Limits

48 cases (24 point sequences, each asked twice), one run per case, default settings, and only a handful of models. Differences between close results aren't reliable.
Run-to-run variation is real: Haiku scored 0.00 on the leaderboard and 1/48 in a rerun, and some of Gemini's early errors disappeared on rerun.
A score of 0/48 only rules out a true rate above roughly 7%.
I checked the padel rules (golden point, tiebreak serving order, who serves first in the next set) against padel rules sources, but real events sometimes use a super tiebreak in the third set, which this benchmark doesn't model.

My Benchmark

https://www.kaggle.com/benchmarks/gulrezqayyum/tennis-and-padel-state-tracking-benchmark

Board Game Rule Conflict Resolver

Gulrez Qayyum — Sun, 04 Oct 2026 14:53:08 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

Board Game Rule Conflict Resolver is a source-grounded tournament judge for resolving ambiguous board game interactions.

Players enter:

The cards involved
The current game phase
Their rules question

The agent retrieves structured card text, official interaction rulings, errata, and core rule priorities from Sanity before producing a ruling.

The resolver is designed to avoid inventing rules. When Sanity does not contain enough authoritative information, it returns a low-confidence response explaining that there is insufficient official data rather than guessing.

The demo dataset contains fictional board game content, including:

Card mechanics and effect text
Trigger phases and keywords
Core rule priorities
Explicit multi-card interaction rulings

Demo

Try the deployed application:

Open Board Game Rule Conflict Resolver

Recommended demo flow:

Open the resolver.
Ask about Mirror Shield versus Piercing Bolt.
Try the three-card interaction involving Chain Lightning, Sanctuary Zone, and Blood Pact.
Try an unknown card such as Mystic Dragon to see the strict low-confidence fallback.
Open Sanity Studio at /studio and update the official ruling for an interaction.
Submit the same question again and observe how the retrieved Sanity content changes the answer.

Code

GitHub repository

The main implementation is organized around:

Next.js App Router
/api/resolve-conflict agent endpoint
Sanity schemas and seeded content
GROQ-backed local tools
Sanity Context MCP integration
A browser-based resolver UI
An end-to-end agent test suite

How I Used Sanity

Sanity is the structured source of truth for the agent.

I modeled three document types:

`gameCard`

Stores:

Card name
Stable card slug
Card type
Trigger phase
Keywords
Effect text

`gameRule`

Stores:

Rule title
Rule code
Rule category
Priority order
Rule text

`interactionConflict`

Stores:

Conflict title
Linked cards
Conflict description
Official ruling
Governing rule

The agent exposes tools for:

Looking up card documents by name or slug
Finding interaction conflicts linked to the retrieved cards
Retrieving ordered core rules by category

The intended agent workflow is:

Retrieve every named card from Sanity.
Use the returned card identifiers to search for linked conflict and errata documents.
Retrieve relevant rule-priority documents.
Use the official ruling as primary evidence when an explicit conflict document exists.
Return a JSON ruling with a verdict, reasoning, cited documents, and confidence level.

The server can use Sanity Context MCP tools when configured. It also includes local GROQ-backed tools for the same structured content workflow.

The agent is explicitly instructed to rely only on retrieved Sanity context. If no definitive card, conflict, errata, or rule data exists, it returns:

Insufficient official data in Sanity Content Lake to render an absolute judgment.

This makes content editors part of the rules-maintenance workflow: updating a ruling in Sanity changes the evidence used by the resolver without changing application code.

Sanity Project Details

Sanity project ID: kjwkn2a2
Dataset: production

The project contains the structured demo cards, core rules, and interaction conflict documents used by the resolver.

The seeded content is fictional demo content created for this challenge. It is intended to demonstrate the content architecture and agent workflow rather than represent official rules for an existing commercial game.

Technical Highlights

Next.js 16 App Router
React 19
TypeScript
Sanity Studio
GROQ queries
Sanity Context MCP
Groq-compatible model tool calling
Strict JSON ruling responses
Low-confidence fallback for unknown content
Vercel production deployment

GiveBack — Turn Generosity Into Action

Gulrez Qayyum — Sun, 06 Sep 2026 16:01:55 +0000

This is a submission for Weekend Challenge: Generosity Edition

What I Built

GiveBack is an AI-powered platform designed to make generosity easier and more actionable.

The idea is simple: people often want to help others, but don't always know how they can contribute, where their help is needed, or what meaningful action they can take.

GiveBack uses AI to turn a user's intention to help into practical suggestions and actions.

Instead of simply asking:

"How can I donate?"

GiveBack focuses on:

"What can I do today that actually helps someone?"

The goal is to encourage everyday generosity — whether that's donating resources, volunteering time, helping someone in the community, or contributing in another meaningful way.

I wanted to build something that demonstrates that AI doesn't always need to be about productivity or automation. It can also be used to encourage people to help each other.

Demo

Live Application: https://give-back-five.vercel.app/

Code

GitHub: https://github.com/GulrezQayyum/give-back

The project contains the frontend and backend components of GiveBack.

How I Built It

I built GiveBack with a simple full-stack architecture:

Frontend

The frontend was built with React + Vite.

It provides the user interface where people can interact with GiveBack and describe how they would like to contribute.

Backend

The backend was built with Python and FastAPI.

The backend handles application logic and acts as the bridge between the frontend and the AI functionality.

Google AI

The core AI functionality uses Google AI / Gemini.

Gemini is used to understand the user's intent and generate meaningful, context-aware suggestions for ways they can give back.

Rather than returning generic suggestions, the goal is to make the interaction feel like a conversation:

User intention → AI understands the situation → GiveBack suggests meaningful actions

Why AI?

Generosity is highly contextual.

Someone might have:

time but no money
money but limited time
technical skills they want to contribute
resources they don't need anymore
a desire to help their local community

AI provides a natural way to understand these different situations and suggest possibilities based on what a person is actually able and willing to give.

Prize Categories

Best Use of Google AI

GiveBack is submitted for Best Use of Google AI.

I integrated Google Gemini into the application to understand user intentions and generate personalized suggestions for meaningful acts of generosity.

The AI isn't being used simply as a chatbot. It is part of the core experience: helping users move from "I want to help" to "Here's something meaningful I can actually do."

Why I Built This

The idea behind GiveBack came from a simple observation:

We often think generosity requires money.

But generosity can also be time, knowledge, skills, attention, resources, or simply helping someone who needs it.

I wanted to experiment with how AI could help people discover those opportunities.

GiveBack is my attempt to build technology around a more human goal:

Use AI not only to help us do more, but to help us give more.

RAGBench: Stop Guessing, Start Measuring Your RAG Pipeline

Gulrez Qayyum — Wed, 02 Sep 2026 16:43:14 +0000

Hey builders

RAGBench just hit Hugging Face.

It's an open evaluation framework for RAG systems, built for developers, with the goal of making RAG evaluation more practical and reproducible.

Instead of guessing whether your chunking strategy actually works, test it. Compare it. Measure it.

I built RAGBench to make it easier to evaluate different RAG approaches using the same documents, queries, and evaluation metrics.

So far, I've focused on comparing semantic, document-level, and parent-child chunking and measuring how they affect retrieval and answer quality.

The goal is simple:

Stop guessing. Start measuring.

RAGBench is:
• Open source
• Open data
• Reproducible
• Built for community feedback

Try the interactive benchmark:
https://huggingface.co/spaces/Gul55555/ragbench

Contributions and feedback are welcome.

I'm especially interested in:
• What RAG evaluation metrics do you actually use?
• What chunking strategies have worked best for you?
• What problems have you encountered when evaluating RAG?
• What should I add to RAGBench next?

What's broken in your RAG pipeline?

I'd love to hear what you're working on.

🍝 Pasta Shape Guide — An Interactive Guide to the Pasta We Love

Gulrez Qayyum — Sat, 15 Aug 2026 16:47:55 +0000

This is a submission for Frontend Challenge - Comfort Food Edition, Perfect Landing

What I Built

For the Frontend Challenge: Comfort Food Edition, I decided to build around one of my favorite comfort foods: pasta. 🍝

Instead of creating a simple recipe landing page, I wanted to make something people could actually explore.

So I built Pasta Shape Guide, an interactive guide to different pasta shapes, their origins, cooking times, recommended sauces, recipes, and cooking tips.

You can:

🔍 Search pasta by name or sauce
🍝 Browse different pasta shapes
🌍 Learn about their origins
⏱️ Check recommended cooking times
🍅 Discover sauce pairings
📖 Read detailed recipes and cooking tips

Built With

React
Vite
Tailwind CSS
React Router
JavaScript
Vercel
Pexels for the pasta imagery

Demo

pasta-guide-with-react.vercel.app

Journey

This project was especially interesting for me because it was my first React project.

I'm more familiar with Flutter development, so moving into React gave me a chance to learn a different approach to building interfaces.

During the project, I learned about component-based architecture, React state, client-side routing, dynamic URLs, filtering data, responsive design, and deploying a production React application.

One thing I particularly enjoyed was making the application data-driven.

The pasta information is separated from the UI, so adding another pasta doesn't require creating an entirely new component. The same components can render the new information automatically.

What I'm Proud Of

I'm most proud that I turned what could have been a static landing page into an actual little interactive experience.

The user isn't just looking at a pasta image.

They can:

Discover → Search → Explore → Learn → Find a Sauce → Try a Recipe

That was the experience I wanted to create.

I'd love to know: what pasta shape would you add to the guide? 🍝

Building an AI-Powered Network Traffic Analyzer with FastAPI and Scapy (Looking for Contributors)

Gulrez Qayyum — Fri, 17 Jul 2026 12:11:12 +0000

Over the past few weeks I've been building NetGuard, an open-source AI-powered Network Traffic Analyzer and Active Defense Platform.

The goal is to combine traditional intrusion detection with machine learning into a single platform capable of:

Capturing live packets using Scapy
Detecting suspicious traffic using IDS rules
Classifying traffic with a Random Forest model
Automatically blocking malicious IPs
Providing remediation recommendations
Generating incident reports

Most of the system is already working, including packet capture, IDS detection, remediation, dashboard integration, and IP blocking.

The remaining work is mainly around integration.

The most interesting bug is that the FastAPI orchestrator successfully receives HTTP 200 OK from the classifier service, yet ai_result is always returned as null.

I've investigated payload formatting, response parsing, and model loading, but I'm convinced another pair of eyes could spot what I'm missing.

I've documented the architecture, setup instructions, debugging notes, and open issues in the repository.

If you're interested in Python, FastAPI, networking, cybersecurity, or machine learning, I'd love your feedback or contributions.

Repository:
https://github.com/GulrezQayyum/netguard

Open-source is one of the best ways to learn, so even documentation improvements or code reviews are welcome.

Apex F1 – The Ultimate Formula 1 Racing Simulator

Gulrez Qayyum — Mon, 13 Jul 2026 12:51:08 +0000

This is a submission for Weekend Challenge: Passion Edition

Demo

https://apex-f1-159.web.app

Code

https://github.com/GulrezQayyum/apex-f1

Inspiration

Formula 1 has always fascinated me because it's not just about speed—it's a sport driven by strategy, teamwork, weather, tyre management, and split-second decisions. I wanted to capture those elements in a mobile app rather than building a traditional arcade racing game. Apex F1 is my attempt to recreate the excitement of managing a race weekend while combining my passion for Formula 1 with Flutter development.

How I Built It

I built Apex F1 entirely with Flutter, focusing on creating a realistic Formula 1 racing and strategy experience for mobile devices. As a huge Formula 1 fan, I wanted to go beyond a simple racing game and recreate the tactical decisions that make the sport exciting.

The project started with designing the app's architecture and user interface in Flutter. I then developed the race simulation engine, which powers every lap of the race. The simulator considers factors such as tyre degradation, fuel strategy, weather conditions, pit stops, safety cars, and AI driver behavior to produce dynamic race outcomes.

To make each race feel unique, I implemented:

GPS-inspired circuit layouts based on real Formula 1 tracks.
Twenty AI-controlled drivers with different performance characteristics.
Dynamic weather that can change race strategy.
Tyre wear and pit stop strategy affecting lap times.
Championship standings that carry progress across an entire season.
Multiple seasons (2023–2025) with updated teams and drivers.

The UI was designed to be clean and immersive, allowing users to follow race events, monitor strategy, and track championship progress without feeling overwhelmed.

The biggest challenge was balancing realism with performance. Building a simulation that remained responsive on mobile devices required careful optimization and repeated testing. I refined the race logic over many iterations until the races felt both competitive and unpredictable.

Overall, Apex F1 combines my interests in Flutter development, simulation programming, and Formula 1 into a single project that reflects both my technical skills and my passion for the sport.

Prize Categories

Best Use of Google AI

Apex F1 integrates Google Gemini AI to enhance the racing experience beyond traditional simulation. Google AI powers intelligent race insights and assists users with strategy recommendations, making the app more interactive and engaging. By combining AI-driven assistance with a realistic Formula 1 race simulator built in Flutter, Apex F1 delivers a smarter and more immersive experience for F1 fans.

Built by: Gulrez Qayyum — an indie developer passionate about Flutter, AI, and Formula 1.

Built a Network Traffic Classifier with Random Forest (96.8% Accuracy)

Gulrez Qayyum — Sun, 17 May 2026 18:47:18 +0000

I recently completed a cybersecurity + machine learning project where I trained a Random Forest model to classify network traffic into multiple attack categories using the NSL-KDD dataset.

The classifier can detect:

DoS attacks
Probe/reconnaissance traffic
R2L brute-force attempts
U2R privilege escalation
Normal traffic
Stack
Python
Scikit-learn
FastAPI
Pandas / NumPy
Results
96.8% Accuracy
<1ms inference time
Production-ready model packaging

I also wrote a detailed Medium article covering:

Dataset preprocessing
Feature selection
Model training
API integration
Real-world deployment considerations
Challenges working with synthetic vs real traffic data

Would appreciate feedback from the community, especially from people working in:

Cybersecurity
ML Engineering
Intrusion Detection
Backend Systems

Article: https://medium.com/@gulrezqayyum/building-a-production-ready-network-traffic-classifier-with-random-forest-and-nsl-kdd-b9559346e197

GitHub:
https://github.com/GulrezQayyum

LinkedIn:
https://www.linkedin.com/in/gulrez-qayyum-999345322/