I've been thinking about a specific kind of bad output lately — the kind where you ask an LLM a reasonable-sounding question and get back something confidently vague. It happens a lot with competitor research prompts, and for a while I assumed it was just a limitation of the model. It wasn't. It was a limitation of how I was framing the request.
The underlying issue is simple once you name it: a model like Claude doesn't fetch external data on its own inside a standard chat session. No web crawl, no live lookup, nothing. It works entirely on what's inside the context window — whatever text you've pasted in. Ask it to "analyse a competitor" with no supporting text, and it will still generate a fluent, structured-sounding answer. That's the part that's easy to miss — a confident output isn't the same as a grounded one.
Once I started treating this as a data problem instead of a prompting problem, the results changed almost immediately.
Two things mattered most:
Pasting real, specific source material before asking anything — actual homepage text, a pricing table, a handful of reviews, not a description of them
Asking one narrow question instead of a compound one — "which value proposition is clearer" instead of "give me a full competitive breakdown"
Broad, one-shot prompts that try to cover positioning, pricing, SEO, and tone in a single message produce shallow answers across the board. It's the same failure mode you see asking a model to "review this codebase" with no file contents attached — technically it will answer, but there's nothing underneath the answer.
The other thing worth noting: role-based framing does real work here. Asking the model to respond as a confused first-time user reading a page surfaces friction points a straightforward summary request misses entirely. It's a small reframe, but it consistently produces more specific output — the same way asking "what would a new user get stuck on" produces better UX feedback than "review this flow."
I came across this pattern described clearly in a piece from Impact Digital Marketing Institute, which broke down the exact prompt structures — role-based, extraction, and gap prompts — used for competitor research specifically. Worth a read if you're building anything that leans on LLM-assisted research or analysis, even outside marketing.
The bigger lesson generalizes past competitor research, honestly. Any task where you're tempted to ask an LLM to "just handle it" without feeding it real, specific context is a task where you're going to get a confident, structured, mostly useless answer back.
What's the most surprising context-window gap you've run into with LLM-assisted research or analysis?
Reference: https://impactdigitalmarketinginstitute.in/how-to-use-claude-for-competitor-analysis/
Top comments (0)