I built a tool that tells me if readers actually understood my articles

After publishing 66 technical articles on DevOps and AI infrastructure, I realised I had a blind spot.

I knew how many people visited my articles. I had no idea how many people understood them.

Page views, bounce rates, time on page — none of these tell you whether your explanation of Kubernetes state management actually made sense to the reader.

So I built QuizOps.

What it does

You paste your article URL. GPT-4o reads it and generates 10 multiple-choice questions streamed in real time. You review, edit, and publish. Your readers get a quiz link.

You see who passed, who failed, and which specific questions tripped people up.

The technical bits

The generation uses OpenAI's streaming API with NDJSON output — each question is a complete JSON object on its own line. The frontend reads the ReadableStream line by line and renders questions as they arrive with a fade-in animation. It genuinely looks like the AI is thinking in real time.


// Each line from the stream is a question
const lines = buffer.split('\n');
for (const line of lines) {
if (!line.trim()) continue;
try {
const q = JSON.parse(line);
setQuestions(prev => [...prev, q]);
} catch {}
}

Stack:

• Next.js 14 (App Router)
• Supabase (auth + DB with RLS)
• OpenAI API (gpt-4o-mini, streaming)
• Vercel (hosting + serverless)
• Resend (email notifications)
• Playwright (E2E test suite)
• Sentry (error monitoring)

Content moderation

Before generating questions, the API runs two classification checks:

1. Domain intent check — is this URL likely to contain inappropriate content?
2. Content check — is the fetched article appropriate for educational quiz generation?

Both use a lightweight GPT-4o-mini call before the main generation request.

Community quiz banks

Beyond publisher quizzes, there are community quiz banks (JSON files in the repo) covering: Prompt Engineering, LLM APIs, AI Agents, RAG & Vectors, Kubernetes, Terraform, GitHub Actions, Python for AI Engineers, Cloud Native, and AI Security.

Anyone can add a new bank by dropping a JSON file — no code changes needed.

Try it

quiz.autoshiftops.com — free, no credit card.

Would love feedback from other technical writers on what's missing.

Comments

[Kit Validación Rápida]

Congratulations—this solves a much cleaner problem than generic engagement analytics. I’d make the first screen answer three things before a reader scrolls: whose article gets pasted, what the reader receives, and what signal returns to the author. A simple visual like ‘Article URL → editable quiz → question-level comprehension report’ could remove ambiguity. ‘QuizOps’ is memorable, but pairing it with a literal headline would help first-time visitors understand it immediately. Hope this is useful.

[Sajja Sudhakararao AutoShiftOps]

Absolutely, this is the genuine legitimate comment I received so far, I will incorporate your suggestion in to the site right away. Thanks for taking the time to review the site. Send me your email, would love to provide pro access.

[Sajja Sudhakararao AutoShiftOps]

Just shipped the flow pill you suggested - homepage now shows:

📝 Your article URL → ✨ AI generates quiz → 📊 Comprehension report

Exactly what you described. Also updated the subtext to explicitly answer all three questions before a visitor scrolls.

Your feedback was the most actionable I've received. Pro access is yours - email me at [e-mail]