What if the problem with AI writing novels isn't context length, but how we're using memory?

Surya S S — Thu, 08 Oct 2026 16:39:15 +0000

I keep seeing the same argument whenever people talk about AI writing long-form fiction:

AI can't really write a 100,000-word novel because eventually the context becomes too large, details get lost, and the model starts contradicting itself.

And I think we might be looking at this as an LLM problem when it's actually an architecture problem.

Hear me out.

We've already solved something surprisingly similar in software engineering.

If I'm working on a 200,000-line codebase, I don't need to hold the entire codebase in my head every time I want to fix one function.

The project has files, folders, documentation, dependencies, state, tests, Git history, configuration, and architecture decisions.

I pull the information I need, make a change, test it, and move on.

The entire codebase exists.

But I don't need the entire codebase in my active working context at the same time.

So why does AI-assisted writing often work more like this?

"Here, Claude. Here's the entire 100,000-word novel. Now remember all of it forever and write Chapter 47."

😂

Maybe we need to stop treating a novel as one giant conversation.

The novel shouldn't be one giant prompt

Instead of thinking about a novel as one enormous block of text, imagine treating it as a structured project.

The chapters are the actual story.

Then you have separate information about:

Characters
Plot threads
Timeline
Worldbuilding
Relationships
Writing style
Research
Ideas
Continuity

The important distinction is that the model doesn't need to read everything every time.

It needs the right information for the current task.

For example, if I'm writing Chapter 48, Claude probably doesn't need Chapters 1–47 sitting in its context.

It might need:

Chapter 47
Sarah's current character state
John's current character state
The active betrayal plot
The current timeline
Information about the letter introduced in Chapter 12
Relevant worldbuilding
The established writing style

That's a much smaller context.

But the information from Chapter 12 hasn't disappeared.

It's simply stored somewhere else and retrieved when it's relevant.

That's a fundamentally different way of thinking about the problem.

The interesting part is what happens to ideas

This is where I think things get much more interesting.

Imagine I'm halfway through a novel and suddenly think:

"Wait... what if John's brother is actually working for the antagonist?"

That's not canon.

It's just an idea.

So the system should treat it differently from an established fact.

Maybe it starts as a raw idea.

Later, I develop it:

"Actually, that would explain why John disappeared in Chapter 7."

Now it's becoming a serious possibility.

Eventually I decide:

"Yep. This is canon."

At that point, the system should update the relevant character relationships, plot threads, and timeline.

That's basically version control for story ideas.

And I think that's something current AI writing workflows are missing.

Not every piece of information should have the same status

This might be one of the most important parts.

A writing system shouldn't treat these two statements as equivalent:

"Maybe Sarah has a sister."

and:

"Sarah has a sister named Emily."

The first is an idea.

The second is established canon.

There should probably be a distinction between things that are:

Ideas
Developing
Proposed
Canon
Revised
Deprecated

That gives the AI something incredibly important:

a distinction between possibilities and facts.

One of the most frustrating things about working with AI on a long project is when an idea casually mentioned 30 chapters ago suddenly gets treated as established fact.

A human writer understands:

"I was just brainstorming."

The model may not.

A structured project could make that distinction explicit.

Character memory could work differently too

You could have a persistent profile for every major character.

Not just:

"Sarah is a detective."

But something much richer:

What does Sarah want?
What is she afraid of?
What does she believe about John?
What secret is she hiding?
What does she know that the reader doesn't?
What would she never do?
What would make her break that rule?
How has she changed since Chapter 1?

And importantly, those answers shouldn't necessarily be static.

Sarah in Chapter 5 might be very different from Sarah in Chapter 50.

So instead of telling Claude:

"Remember Sarah."

You could give it:

"Here's Sarah's current canonical state as of Chapter 48."

That's much more useful.

The system isn't trying to make the model permanently remember Sarah.

It's maintaining Sarah's current state and giving the model the relevant version when needed.

Then there's the continuity problem

This is where I think AI-assisted fiction could become genuinely interesting.

Before committing a chapter, the system could run something resembling a test suite.

Imagine Claude finishing Chapter 48 and the system checking:

Continuity check

Sarah says she has never been to Paris.

Chapter 14 says Sarah lived in Paris for three years.

Conflict detected.

John knows about the murder.

Current plot state says John shouldn't know this yet.

Knowledge-state conflict detected.

Sarah's age has changed from 29 to 32.

Character-state conflict detected.

The gun mentioned in Chapter 48 was destroyed in Chapter 21.

Object continuity conflict detected.

That's basically a novel compiler.

😂

And I actually think this could be incredibly useful.

Because a bigger context window doesn't automatically solve continuity.

You could give a model 500k tokens and hide one critical fact somewhere in the middle.

The model may still fail to use it correctly.

The problem isn't necessarily:

"Does the AI have access to the information?"

It might instead be:

"Can the AI reliably identify which information matters right now?"

That's a very different problem.

Context, memory, and source of truth are different things

I think these three concepts get mixed together way too often.

Context is the information the model currently sees.

Memory is information about the project that can be retrieved when necessary.

Source of truth is what has actually been established in the novel.

Those aren't the same thing.

The actual chapter can be the source of truth.

A character profile can be a structured representation of that truth.

A retrieved memory can provide relevant information from the project.

And the active context can contain only what the model needs right now.

That separation seems much more scalable than trying to make the context window itself responsible for everything.

What about deleted scenes and alternate versions?

This is another place where the software analogy becomes interesting.

Writers don't just have one version of a story.

They have:

Deleted scenes
Alternative endings
Previous drafts
Abandoned plotlines
Research
Notes
Scrapped dialogue
Worldbuilding that never made it into the final book

Imagine being able to ask Claude:

"Show me all the abandoned plot ideas involving John's brother."

Or:

"Why did we decide that Sarah couldn't know about the letter yet?"

Or:

"Compare the current ending with the ending from Draft 2."

That's much closer to how humans actually work on large creative projects.

And it's also much closer to how we already manage complex software projects.

You don't throw away the old code every time you change something.

You keep history.

You keep state.

You keep a source of truth.

You create branches when necessary.

Why shouldn't long-form AI writing work the same way?

Retrieval should be intentional

There's another distinction I'd make.

A lot of current AI systems essentially ask:

"What information looks relevant to this prompt?"

But for a large creative project, I think the better question is:

"What project state is required to perform this task correctly?"

If I'm asking Claude to write Chapter 48, the system should know that it probably needs the previous chapter, the current character states, active plot threads, relevant continuity information, the current timeline, and the established writing style.

It probably doesn't need a rejected plot idea from Draft 2.

It doesn't need a deleted scene.

It doesn't need an unrelated subplot from a completely different part of the book.

The goal isn't to retrieve more information.

It's to retrieve the right information.

This could go far beyond novels

The same architecture could work for almost any large creative project.

A screenplay.

A game.

A comic universe.

A research project.

A long-running YouTube channel.

A technical documentation project.

Even software development itself.

The underlying problem is the same:

The project is larger than the model's useful working context.

So instead of constantly increasing the context window, perhaps we should get better at managing the information that enters it.

Maybe the future isn't just bigger context

This is what I'm starting to wonder.

Maybe the future isn't simply:

LLM + giant context window

Maybe it's:

LLM + project state + retrieval + structured memory + validation

The model becomes the reasoning engine.

The project system becomes the persistent memory.

And the application decides what information the model actually needs for each operation.

In other words, instead of asking the model to remember the entire book, we build a system that helps the model navigate the book.

Basically...

I don't think the answer to long-form AI writing is necessarily:

"Give the model a 5-million-token context window."

Because at some point the book gets bigger.

Then there are sequels.

Then character notes.

Then research.

Then alternate drafts.

Then deleted scenes.

Then worldbuilding.

Then 400 random ideas I had at 3 AM.

Eventually you're going to have a giant pile of text and tell the AI:

"Remember all this."

That's not really a memory system.

That's a very expensive folder.

I'd rather have something closer to:

Novel → structured project memory → relevant context → Claude → continuity check → updated story state

The novel remains the source of truth.

The memory system keeps track of what matters.

The context is assembled for the task at hand.

Claude does the reasoning and writing.

Then the resulting changes get fed back into the project state.

Git + RAG + structured memory + an LLM + a novel.

And honestly, that is starting to feel less like a workaround and more like the architecture we'd actually want.

I'm curious if anyone is already doing something along these lines with Claude Projects, Claude Code, Obsidian, RAG, custom MCP servers, or something completely different.

Because if someone has already built "Git for novels", I desperately want to know about it.

Has anyone actually noticed a difference in Opus 5.5 lately?

Surya S S — Thu, 08 Oct 2026 16:26:51 +0000

I've been wondering about this after seeing quite a few discussions about Opus 5.5 supposedly being "nerfed" or behaving differently from when it first launched.

I didn't really notice anything myself at first, but after seeing more people talk about it, I started paying closer attention to my own sessions.

I'm still not convinced there's actually been a change.

What I've noticed is that the experience seems to vary quite a bit depending on the task. For some coding and reasoning tasks, Opus 5.5 still feels extremely strong. On other tasks, I've had responses that seem strangely shallow compared with what I would have expected from the same model.

But that could be:

Different prompts or context
Longer conversations
Changes in the underlying routing/infrastructure
The type of task I'm giving it
Or simply expectation bias after reading everyone else's experiences

So I'm curious about people who have been using Opus 5.5 consistently since launch.

Have you actually noticed a difference?

I'm particularly interested in concrete examples rather than just "it feels worse."

For example:

A coding task it handled well before but now struggles with
Reasoning mistakes you're seeing more frequently
Changes in how well it follows detailed instructions
Context being lost more often
Tasks where it has actually improved
Or situations where you tested the same/similar prompt at different points

I'm also interested in hearing from people who don't think anything changed.

Maybe the "Opus got nerfed" discussion is mostly confirmation bias, or maybe there really is something noticeable happening.

What has your experience been?

How do you handle Claude losing the plot during long coding sessions?

Surya S S — Thu, 08 Oct 2026 16:22:21 +0000

I've been running into something with Claude that I can't quite figure out the best workflow for.

I'll start a coding task and the first few hours are honestly impressive. Claude understands the existing code, finds the relevant files, makes the changes, tests them, and generally stays on track.
But once the session gets really long, things sometimes start getting messy.

For example, I'll fix issue A, then move on to issue B. While working on B, Claude changes something related to A. I point it out, it fixes A again, and then something from an earlier part of the conversation gets lost.

The frustrating part is that the individual responses still look reasonable. It's not necessarily an obvious hallucination. It feels more like the overall understanding of the project slowly drifts as the session gets longer.

I've tried compacting and starting a completely new session, but I'm still not sure when I should use each approach.

For people who use Claude seriously on larger codebases:

What's your workflow for this?

Do you keep one long session until the feature is finished?
Do you start a new session for every major subtask?
Do you maintain a CLAUDE.md with important architectural decisions?
Do you ask Claude to create a handoff document before starting a new session?
Do you explicitly summarize the current state before continuing?
Or do you just restart the session once it gets too large?

I'm especially interested in workflows for existing production codebases, rather than greenfield projects.

I'm trying to figure out whether there's a good way to manage the context intentionally, instead of eventually reaching the point where I'm telling Claude:

"Remember, we already decided this 20 messages ago." 😅

What has actually worked for you?

DEV Community: Surya S S