I kept doing the same thing by hand: ask ChatGPT, copy the answer, paste it into Claude, ask "what's wrong with this?". So I turned that loop into an iPhone Shortcut. It's called Poly Duo, it's free and open source.
What it does
You ask once. ChatGPT and Claude each answer on their own, without seeing the other's answer. Then Claude compares the two answers. You get one result that opens with three short blocks:
- Both agree
- Where they differ
- What to check yourself
…and both full answers below. It lands on screen, in the clipboard and in a markdown journal in your iCloud Drive — in about a minute.
How it works
On iOS 18 the ChatGPT and Claude apps publish their own Shortcuts actions: Ask ChatGPT and Ask Claude. Duo chains them inside one Shortcut: two independent answers, then a comparison. That's the whole trick, and it's why:
- There is no server. The only network traffic is the two official apps talking to their own companies.
- There are no API keys. It runs on the accounts you already have. Free tiers are enough. And if you already pay for Plus or Pro, you get the chat models of your two plans working together at no extra cost.
- There is no computer. Everything runs on the phone. Put it on the Action Button, Back Tap or Siri and it becomes a "second opinion" button.
This kind of thing has existed on computers — browser extensions, paid AI hubs, API setups. On a phone, free, with nothing in between, I couldn't find anything like it. So I built it.
The code is open source (Apache-2.0): https://github.com/vadimchernets/poly-duo
Poly app for iPhone — beta on TestFlight: https://testflight.apple.com/join/xDnucCVD
12 modes in one shortcut
Critique is the default (3 messages: ChatGPT answers, Claude answers, Claude compares). The others are one tap away, and each shows its message cost before you pick:
- Debate until agreement — ChatGPT takes a side, Claude objects, up to three rounds until one concedes
- Dispute map — where they agree, where they disagree, what is checkable; no verdict imposed
- Synthesis — both answer blind, one anchors, the other adds only what's missing
- Side by side, Decision, Debate, Advisor, Clarify, Delta, Image, Auto
Trio adds a third model from a third company on any free key (NVIDIA → Gemini → Groq → OpenRouter) that names only what both of the others missed.
Why two models
On 26 October 2026 I tested the current flagships: GPT-6.1 Sol and Claude Opus 5.5 each answered 100 new, never-published questions on their own. Where they agreed (74 of 100), they were right 97% of the time (72 of 74), and on none of the 100 questions did both give the same wrong answer with high confidence. A model's own confidence says little about whether it is right (nine API models, July 2026) — one of the findings of our research accepted at NeurIPS 2026 workshops. What helps is a second opinion given independently. In August 2026, on 1,459 public questions, the flagships of the day were right 98% of the time when they agreed. A mid-tier pair from the same two companies (GPT-5 mini + Claude Haiku 4.5, July 2026) agreed on fewer than one question in ten and was right only 37% of the time when it did.
The 97% is the accuracy of agreed answers in my test, run through the companies' own apps (100 questions, both models answering blind) — not a measured accuracy of any Poly Duo mode. Research: https://polyhelper.ai/research
Under the hood
- One shortcut, modes routed through a dictionary.
- Prompts are plain text files in
locales/; the shortcut is built from them for 11 languages. -
tools/verify.pyunpacks the signed.shortcutand checks it matches the sources. - Keys for Trio live in iCloud Drive, never inside the shortcut, so sharing it never shares a key.
Works on any iPhone with iOS 18 or later (I run it on an iPhone 14 Pro Max). On Android, install the free Poly app from Google Play and open Duo inside it (free key, no credit card; on Android it is not a ChatGPT + Claude pair): https://play.google.com/store/apps/details?id=ai.polyhelper.a1.
Try it
- One-tap install, 11 languages: https://polyhelper.ai/duo/?ref=devto-en
- Curious how well AIs do on real forecasts? Which AI model forecasts the markets best — the daily league of 14 AIs
I'd love to hear which modes you'd add — or where Claude's comparison made a good answer worse.
Vadym Chernets, independent AI researcher (PhD). Author of two papers on checking AI answers (both accepted at NeurIPS 2026 workshops, Sydney, December 2026): https://neurips.cc/virtual/2026/178299 · https://neurips.cc/virtual/2026/loc/sydney/173020
Top comments (0)