When a team reports a high share of pull requests touched by AI, the number is real. It still does not tell you whether the tool is on the work that decides how a service behaves.
Code generation, tests, refactors and summaries are easy to review. You can run the tests, read the diff and reject a bad patch in minutes. Architecture decisions, design trade-offs, requirements and incident analysis are harder to verify, and the stakes are higher. "Good" takes argument.
I think many engineering teams put AI on the first pile and leave the second almost untouched. A study published last week gives that hunch some evidence. The subjects are scientists, not software teams, so the engineering mapping below is mine, not Anthropic's.
What 207,049 sessions show
On 5 November 2026, Anthropic published "AI for scientific discovery: How researchers use Claude". The team analysed 207,049 research sessions on consumer plans from 2 August to 30 September 2026. They compared each task's share of AI sessions with its share of a researcher's ordinary time.
- Data analysis: 33.7% of Claude sessions and 48.1% of Claude Code sessions, against 27.6% of time.
- Writing and communication: 32.0% of Claude sessions, against 16.0% of time.
- Data gathering: 11.5% of Claude sessions and 11.9% of Claude Code sessions, against 28.6% of time.
- Ideation and hypothesis generation: 4.2% of Claude sessions and 2.1% of Claude Code sessions, against 7.1% of time.
Writing gets twice the AI share that time would suggest. Gathering and ideation get less.
Use follows what is easy to check
Anthropic also scored how well AI could handle each research task. Tasks in the top 20% on that score got about twice as much Claude use as time would predict, and about four times as much on Claude Code. Tasks in the bottom 20% were barely used. Anthropic's summary: "scientists appear to use AI where it works best."
That capability score has a correlation of 0.78 with how suitable a task is for reinforcement learning. Anthropic describes those tasks as ones where success is easy to verify, many examples can be simulated and feedback can be delivered quickly.
Here is my inference, not Anthropic's claim. Work with a clear pass mark and a short feedback loop is where the tool performs well, so that is where people use it. Work that needs a longer argument gets less AI, even when a team would say it matters more.
On a software team, I would expect the same split. Generated functions, tests, mechanical refactors and summaries have a quick review loop. System design, trade-off write-ups, requirements and incident analysis do not.
A two-axis map you can copy
Score each recurring task on two questions. Can you check the result quickly? Does the task matter to the outcome?
Paste the table below into a doc, replace the example rows with your own work, and assign a review owner for anything that matters. The rows are examples from ordinary engineering work. Anthropic did not test them.
| Task | Easy to check? | Matters to outcome? | AI role | Review owner |
|---|---|---|---|---|
| Generate a function with existing unit tests | Yes | Medium | Draft the code | Pull request reviewer |
| Write or extend unit tests for a known behaviour | Yes | High | Draft cases and assertions | Service owner |
| Mechanical refactor (rename, extract, format) | Yes | Medium | Apply the change | Pull request reviewer |
| Summarise a pull request or stand-up notes | Yes | Low | Draft the summary | Author |
| Architecture decision or design trade-off | No | High | Draft options and the counter-case | Named decision owner |
| Requirements or incident analysis | No | High | Structure the notes, list open questions | Named investigation or product owner |
Start with the work that is easy to check and matters to the outcome, and keep a review step. Use the tool freely on easy, low-stakes work, and do not count that as a win for the service. For hard, high-stakes work, keep a named human owner. Let AI draft and argue the other side, and put your effort into making the task easier to check.
Make the hard work easier to check
You can often move a hard task one step towards checkable before you ask the model for a draft.
- Write the acceptance tests first, then ask AI to implement against them.
- Fill an architecture decision record template (context, options, decision, consequences) before you ask for a recommendation.
- Keep a short set of worked examples of a good requirement, and compare new drafts against them.
- For incident analysis, agree the timeline fields and evidence rules up front, then use AI to organise the notes.
The cost of review still lands on someone. Name that person before you scale the prompt.
The sessions are on consumer plans, so enterprise use and other tools sit outside the frame. The findings are correlations, and the subjects are scientists. Treat the engineering mapping as a hypothesis to test on your own work.
If you want the longer argument, I wrote a fuller essay on Medium: We hand AI the work that's easy to mark.
Sources
- AI for scientific discovery: How researchers use Claude, Hermle, Jones, Althoff and colleagues, Anthropic, 5 November 2026.
- ChatGPT's expanding world of work, OpenAI Economic Research, 26 October 2026.
Of the last five things your team asked AI to do, how many had a test, a diff or a known total you could check in a few minutes?
Top comments (0)