DEV Community: Yashwanth Gadagani

Stop Trusting a Single Model's Answer

Yashwanth Gadagani — Thu, 01 Oct 2026 14:41:20 +0000

Here's an uncomfortable truth about every AI app shipping today: when your app calls one model and prints what comes back, you've built an opinion dispenser, not an answer engine. The model is confident. The UI is clean. The answer might be wrong. And nothing in the pipeline would ever tell you.

I learned this building Forge, an AI STEM solver. In education, a confident wrong answer isn't a minor glitch — it's a student copying a false derivation into their exam prep. So I stopped treating the model's output as the answer and started treating it as a claim to be checked. That one reframing changed the architecture more than any prompt tweak ever did.

The core argument: verification is a separate job from generation

We obsess over which model generates the best response — Nemotron vs. DeepSeek vs. Llama, temperature settings, system prompts. That's the generation side, and the industry has gotten very good at it. But generation quality has a ceiling that prompting can't break: a model cannot reliably grade its own homework. Ask it "are you sure?" and it will confidently say yes, because confidence is what it was trained to perform.

The fix isn't a better prompt. It's a second, independent opinion. In Forge, every solver result goes through a cross-check: a separate model independently solves the same problem, and the UI reports agree, minor difference, or disagree. Two models converging independently is evidence. One model's confidence is theater.

And for genuinely hard questions, Forge has debate mode: three models answer the same prompt simultaneously, side by side, and then a judge model picks the winner and explains why. Watching three frontier models disagree with each other is the fastest cure for AI sycophancy — you see exactly where the confident consensus breaks down.

"Just use a bigger model"

The most common objection: a bigger, smarter model makes verification unnecessary. It doesn't, for three reasons.

First, errors don't scale away linearly. A stronger model makes fewer mistakes, but its mistakes get harder to spot — they're fluent, well-structured, and wrong in the middle step of a derivation. Verification catches what capability can't prevent.

Second, the failure mode is silent. When a single model is wrong, there's no signal. When two models disagree, the disagreement is the signal. That signal is worth more than a marginal capability bump, because it tells the user when to be careful — which is precisely when it matters.

Third, diversity beats size. A 70B Llama and a DeepSeek model failing differently gives you more information than a single larger model failing confidently. Independent errors cancel; correlated confidence compounds.

The honest tradeoffs

I'll steelman the other side, because verification isn't free:

Latency. Two model calls take longer than one. Forge mitigates this by streaming the primary solution first — the user reads steps while the verifier works in the background. Perceived latency matters more than actual latency.
Cost. You're paying for 2–3x the tokens. On NVIDIA NIM's pricing this is real money at scale. My answer: verify where errors are expensive (education, code, medical-adjacent) and skip it where they're cheap (brainstorming, drafts).
The judge can be wrong too. In debate mode, the judge model is itself a single model with opinions. It's judges all the way down — but each layer converts silent failure into visible disagreement, and visible disagreement is something a human can act on.

What it looks like in practice

A concrete example of what debate mode is for: take a tricky physics question — say, a rotational dynamics problem with a subtle sign convention. Run three models on it and you might see two agreeing on the setup while one confidently flips a sign in step three. A good judge verdict doesn't just say "Model B wins" — it points at where the derivations diverge, which is the exact step a student needs to scrutinize.

That divergence map is the real product. Without debate mode, the student gets one derivation and no reason to doubt step three. With it, they get a highlighted disagreement and an explanation of the contention. The feature isn't "three answers" — it's one answer with its weak points labeled.

The same principle works outside education. Code review agents that run the linter and a second model produce fewer escaped bugs than either alone. Support bots that check a generated reply against the knowledge base before sending hallucinate less. The pattern is always the same: generate, then verify with something that fails differently.

What this means for your AI app

You don't need debate mode. Start smaller:

Never show a high-stakes answer without a confidence signal. A cross-check verdict, a citation, something beyond the model's own assurance.
Make disagreement visible, not hidden. Retry loops that silently re-roll until something looks plausible are worse than showing the conflict — they manufacture false confidence.
Verify with diversity, not repetition. The same model twice is a ritual. A different model — or better, a different method (a calculator for arithmetic, a linter for code) — is a check.

The broader point: we've spent two years making models sound more certain. The next leap in trustworthy AI apps won't come from more capable generators — it'll come from architectures that assume the generator is wrong and check anyway. Build the courtroom, not just the witness.

Forge is open source at github.com/Asdfyash1/GPAI — the cross-check pipeline, debate mode, and the whole verification architecture are in there.

Automating Marketing Reports with n8n and Looker Studio

Yashwanth Gadagani — Thu, 01 Oct 2026 14:34:32 +0000

Here's a ritual every marketing team knows: Monday morning, someone opens Meta Ads Manager, Google Ads, and GA4, exports three CSVs, pastes them into a spreadsheet, fixes the date formats, and rebuilds the same charts they rebuilt last week. It takes an hour, it's error-prone, and nobody enjoys it.

This guide walks through automating that entire pipeline with n8n (self-hosted workflow automation) and Looker Studio (free dashboards). The pattern: n8n pulls spend and performance data from each ad platform on a schedule, normalizes it into one Google Sheet, and Looker Studio visualizes it — refreshed automatically, forever.

A note on honesty up front: this is a build walkthrough based on my real n8n + Looker Studio automation work — I've wired n8n workflows into Google Sheets and visualized the results in Looker Studio. The Meta Ads and GA4 legs below are walked through from the API docs and working n8n patterns, but I haven't run those two legs end-to-end against live ad accounts myself. The Google Ads API leg I have not run all the way through either — Google's developer-token approval is the known hard part there, and I'll flag exactly where that bites.

The architecture

Meta Marketing API ─┐
                    ├─▶ n8n (scheduled) ─▶ normalize ─▶ Google Sheets ─▶ Looker Studio
GA4 Data API ───────┘         ▲
                              │  (Google Ads API — see caveat below)

Why Google Sheets in the middle instead of writing straight to Looker Studio? Because Looker Studio's native ad-platform connectors refresh on their schedule and give you limited control over blending. A Sheet you own means one schema, one timezone, and blended cross-platform metrics (total spend, blended ROAS) computed before visualization. For a weekly marketing report, Sheets is plenty — upgrade to BigQuery when the row counts get silly.

Step 1: Pull Meta Ads data with the HTTP Request node

Meta's Marketing API has an insights edge that returns exactly what a report needs. In n8n, use the HTTP Request node (not the Meta Ads node — the generic HTTP node gives you full control over fields and breakdowns):

Method: GET
URL: https://graph.facebook.com/v21.0/act_<AD_ACCOUNT_ID>/insights
Query parameters:
- fields: campaign_name,spend,impressions,clicks,actions
- date_preset: last_7d
- level: campaign
- access_token: your token (store it in n8n Credentials, never hardcoded)

Request actions and you'll get conversions broken down by type — filter for purchase or lead in the next step. One gotcha: Meta returns actions as an array of {action_type, value} objects, not flat columns. You'll flatten that in the normalize step.

Token setup: create a Meta app, add the Marketing API product, generate a token with ads_read scope. For a scheduled job, you want a system user token (doesn't expire when someone changes their Facebook password) — this is the detail that breaks most first attempts when the report silently stops updating two months in.

Step 2: Pull GA4 data with the Data API

GA4's Data API uses OAuth2 service accounts, which n8n supports natively:

In Google Cloud Console, enable the Google Analytics Data API, create a service account, download the JSON key.
In GA4 Admin → Property Access Management, add the service account email as a Viewer.
In n8n, create a Google Analytics credential (OAuth2 / service account) and use the Google Analytics node with the Run Report operation:
- Dimensions: date, sessionDefaultChannelGroup
- Metrics: sessions, conversions, totalRevenue
- Date range: last 7 days

The failure mode to expect here is the property-access step. If n8n returns a 403, it's almost always the service account missing the Viewer role in GA4, not your n8n config.

Step 3: Google Ads — the hard one (partially tested)

Google Ads data comes from the Google Ads API, and this is where you should budget real time. The API requires a developer token, and Google approves those only for accounts in good standing — the application asks about your use case and intended API usage. Test accounts get an unapproved token with heavy limits.

The honest status: the n8n side is straightforward (HTTP Request node against googleads.googleapis.com, GAQL query like SELECT campaign.name, metrics.cost_micros, metrics.impressions FROM campaign WHERE segments.date DURING LAST_7_DAYS), but I have not completed a live pull against a production account because of the developer-token gate. If you plan to run this in production, start the developer-token application on day one — it's the longest pole in the tent.

Pragmatic fallback: until the API is approved, the Google Ads scheduled email reports (CSV to inbox) plus n8n's IMAP Email trigger can land the same data in your Sheet. Less elegant, works Monday morning.

Step 4: Normalize everything in a Code node

The three APIs return three different shapes, three different date formats, and two different currencies if you're not careful. One n8n Code node (JavaScript) standardizes them into a single schema:

// Normalize one platform's rows into the canonical schema
function normalize(rows, platform) {
  return rows.map(r => ({
    date: r.date,                       // YYYY-MM-DD, coerced per platform
    platform,                            // 'meta' | 'google_ads' | 'ga4'
    campaign: r.campaign_name || r.campaign || '(organic)',
    spend: Number(r.spend || 0),         // micros → units for Google Ads!
    impressions: Number(r.impressions || 0),
    clicks: Number(r.clicks || 0),
    conversions: Number(r.conversions || 0),
  }));
}
return normalize($input.all().map(i => i.json), 'meta');

The Google Ads cost_micros trap deserves its own warning: costs come back in micro-units, so divide by 1,000,000 or your spend column will claim you spent $4 billion.

Also normalize timezones here — pull everything in the ad account's timezone and stamp the Sheet with it, or your Monday report will attribute Sunday night spend to the wrong week.

Step 5: Write to Google Sheets

n8n's Google Sheets node with the Append operation (service-account credential again) writes the normalized rows to a tab like raw_spend. Keep one tab per platform plus one blended tab, or just one tab with the platform column — the latter blends more easily in Looker Studio.

Idempotency matters for scheduled runs: before appending, use IF + Google Sheets: Read to check whether this week's rows already exist (match on date + platform + campaign). Otherwise a retried run double-counts spend, and nobody notices until the invoice meeting.

Step 6: Build the Looker Studio dashboard

Connect Looker Studio to the Sheet (native connector, 15-minute minimum refresh — fine for weekly reporting). The charts that actually get used:

Scorecards: total spend, total conversions, blended ROAS, week-over-week deltas
Time series: spend vs. conversions by day, with platform breakdown
Table: campaign-level spend / ROAS, sortable, with conditional formatting on ROAS < target

Blend the platforms with a calculated field for blended ROAS: SUM(conversions * value) / SUM(spend) — or simpler, a blended cost per conversion if conversion values aren't tracked consistently across platforms (they usually aren't at first).

Step 7: Schedule it and handle failures

n8n's Schedule Trigger (cron: Mondays 6 AM in the account timezone) kicks off the workflow. Then add the unglamorous parts:

Error branch: n8n's Error Trigger workflow → send yourself a Telegram/Slack message with the failed node name. A report pipeline that fails silently is worse than no pipeline.
Stale-data check: a final node that reads the Sheet and alerts if the newest row is older than 8 days — catches an expired Meta token before anyone relying on the report notices.
Credential rotation reminder: Meta system-user tokens and Google service-account keys should be rotated; put a quarterly reminder in the workflow's notes.

What this actually buys you

About an hour a week back, per report — but the real win is trust. When the numbers are pulled by a machine on a schedule instead of copy-pasted by a human on a Monday, the "are these numbers right?" conversation disappears. The dashboard becomes the source of truth instead of the spreadsheet someone edited.

Start with Meta + GA4 (both testable today with free-tier accounts), add Google Ads once the developer token lands, and keep the Sheet schema stable from day one — every downstream chart depends on it. The n8n workflow itself is maybe 10 nodes. The data hygiene around it is the actual project.

Building an AI STEM Solver That Shows Its Work — and Double-Checks It

Yashwanth Gadagani — Thu, 01 Oct 2026 14:34:04 +0000

Ask any AI chatbot a calculus question and you'll get an answer. Maybe even the right one. But a student doesn't need an answer — they need to understand how to get there, and they need to know the answer is actually correct. Those are two different engineering problems, and I built Forge (repo: Asdfyash1/GPAI) to solve both: a full-stack STEM copilot that turns any problem — typed, photographed, or pasted from a URL — into a step-by-step derivation, then has a second model independently verify the result.

The stack: Next.js 16 (App Router, Turbopack), TypeScript (strict), NVIDIA NIM models (Nemotron, DeepSeek Flash, Llama 3.3) via the Vercel AI SDK, deployed on Vercel's Hobby tier. Here's how the solver works, step by step, with the real architecture.

Step 1: Stream the solution, don't dump it

Nobody reads a wall of math. Forge's solver streams the derivation step-by-step over Server-Sent Events, so steps appear as they're generated and the UI can reveal them one at a time (collapsible step reveal — "show me the next step" instead of the whole answer).

The streaming endpoint lives at /api/educate/stream:

// src/app/api/educate/stream/route.ts (simplified)
import { requireAuth } from "@/lib/api-guard";
import { streamSolverResponse } from "@/lib/orchestrator";

export async function POST(request: Request) {
  const guard = await requireAuth(request);
  if (!guard.ok) return guard.response;

  const body = await request.json();
  const handle = await streamSolverResponse({ problem: body.problem });
  return new Response(handle.textStream, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
}

On the client, a small useStream hook consumes the SSE stream and appends steps to state as they arrive. The response parser (src/lib/response-parser.ts) decomposes the LLM's markdown into structured steps using regex — each step becomes a collapsible card in SolverView.tsx. That decomposition matters: if the model returns one blob, you can't do progressive reveal, follow-up chips ("Why does this step work?"), or per-step quizzes later.

Step 2: Verify with a second model (cross-check)

This is the part most AI solvers skip. Forge runs a cross-check: after the primary model produces a solution, a second model independently solves the same problem, and the two are compared. The UI shows one of three verdicts: agree, minor difference, or disagree.

Why it matters: a single model's confident wrong answer is the most dangerous output in education. Two models arriving at the same answer independently is dramatically stronger evidence than one model's confidence score. The cross-check model is configurable via env var:

NVIDIA_SOLVER_MODEL=meta/llama-3.3-70b-instruct
NVIDIA_VERIFIER_MODEL=meta/llama-3.3-70b-instruct

You can point the verifier at a completely different model family (or even a different provider via the ADDITIONAL_OPENAI_COMPATIBLE_* vars) so the check isn't just the same weights agreeing with themselves. When the verdict is "disagree," the student sees that upfront instead of copying a wrong derivation into their homework.

Step 3: Render math like math

A derivation full of x^2 + 2x + 1 = 0 in monospace is unreadable. Forge renders formulas with KaTeX via rehype-katex + remark-math, layered on react-markdown + remark-gfm. The MathMarkdown.tsx component handles the whole pipeline, so model output containing $...$ and $$...$$ becomes properly typeset math inline.

Small detail, big UX difference: students trust a solution that looks like their textbook. Plain-text math looks like a chatbot guessing; typeset math looks like a worked example.

Step 4: Accept photos, not just text

Half of real homework exists as a photo of a notebook or a textbook page. Forge's OCR pipeline:

User uploads a photo of handwritten work or a printed problem.
Client-side compression shrinks it to 1600px at quality 0.85 — this keeps the payload under the 1.7 MB Vercel API gateway limit (MAX_INLINE_IMAGE_BYTES). Do this in the browser, not the server; it saves bandwidth and avoids gateway rejections.
NVIDIA Nemotron Omni 30B reads the image and extracts the text.
The extracted text feeds straight into the solver pipeline from Step 1.

PDFs get the same treatment via unpdf — a pure-JS parser with no native binaries, which matters because Vercel serverless functions don't love native modules.

Step 5: Make it work with zero API keys (demo mode)

Here's a trick more AI apps should steal: Forge runs without any API keys at all. If NVIDIA_API_KEY isn't set, the app falls back to demo-solver.ts — deterministic, hand-written demo output that exercises the entire UI (step reveal, KaTeX, quiz, cross-check display) with no model calls.

Why bother? Three reasons: (1) anyone can clone and run it in 30 seconds, which is gold for a portfolio project; (2) frontend development doesn't burn API credits; (3) the demo doubles as a UI test fixture. The env table is honest about what's needed for what:

Variable	Needed for
`NVIDIA_API_KEY`	All real AI features
`JWT_SECRET`	Auth (self-generated)
`RESEND_API_KEY`	OTP emails (lazy-initialized — build succeeds without it)
`TELEGRAM_BOT_TOKENS`	Cloud storage backend

Step 6: Secure the endpoints by default

Every AI endpoint sits behind a requireAuth() guard that runs four checks in order: payload size (→ 413), origin validation for CSRF (→ 403), JWT cookie verification (→ 401), and per-user rate limiting (→ 429). The API key lives only in Vercel env vars — the frontend calls your own /api/* routes with an HttpOnly SameSite=Lax cookie and never sees a secret.

The rate limits are tuned per endpoint: 30 req/min for most AI routes, 10/min for debate mode (which fires 3 model calls per request). In-memory sliding-window counters per Vercel isolate, pruned periodically.

Honest gotchas

A few things I'd do differently or that bit me:

The OTP store is in-memory. Serverless cold starts wipe pending OTPs. Acceptable for an MVP, not for production — move it to Redis or Postgres before real users.
Cloud storage has a ceiling. The Telegram-bot storage backend hits a ~4096-char pinned-message cap around ~50 users. Fine for a demo; a document-based registry is needed to scale.
Next.js 16 broke things vs. older training data. Read the framework docs in node_modules, not your memory, when something behaves oddly.
One model to avoid: mistralai/mistral-large-3-675b-instruct-2512 isn't in the NVIDIA NIM catalog — every call 404s. The README warns about it because I learned the hard way.

The takeaway

The pattern generalizes beyond homework: stream structured output, verify with an independent model, render it like the domain expects, and degrade gracefully without keys. Most AI wrappers stop at "call the API and print the text." The difference between a demo and a tool is everything around the model call — and that's where the interesting engineering lives.

The repo (Asdfyash1/GPAI) has the full source, and the live app is at gpai-jade.vercel.app — try photographing a math problem and watch the cross-check verdict.