DEV Community: Ben Mccarthy

Why Giving AI Agents More Context Can Make Them Worse

Ben Mccarthy — Mon, 14 Sep 2026 18:16:38 +0000

Most AI agent demos start with the model.

We ended up spending far more time thinking about what gets put around the model.

I run 23 holiday lets and have been building AI agents into the day-to-day operation. Guest messaging sounds like one of the easier problems:

Guest sends a message.
Give the message to an LLM.
Send back the answer.

That works brilliantly right up until the guest asks:

Can I park a second car?

Now the answer depends on the property, the booking, the parking arrangement, whether they're currently checked in, whether anything has changed since the listing was written and potentially something a member of the team said twenty minutes ago.

The model is suddenly the easy bit.

While building Zugrow, one of the lessons that kept coming back was this:

An agent can have a very good model and still make a bad decision because you gave it the wrong state.

So we stopped treating context as a giant blob of text and started treating it like application data.

1. Don't give the model everything you know

My instinct initially was simple.

More context = better answer.

So if a guest messaged about a booking, why not give the agent:

the property description
all amenities
house rules
the entire conversation
booking details
previous guest questions
internal notes
host instructions

It feels sensible.

It also produces a mess.

Important information gets buried amongst things that have nothing to do with the current question.

Instead, the agent should get the smallest useful view of reality.

Something closer to:

type GuestContext = {
  property: {
    name: string;
    checkInTime: string;
    checkOutTime: string;
    parking: ParkingPolicy;
  };

  booking: {
    arrivalDate: string;
    departureDate: string;
    guestCount: number;
    status: BookingStatus;
  };

  conversation: {
    recentMessages: Message[];
  };
};

If somebody asks about parking, the agent doesn't need the Wi-Fi password, the boiler instructions and six months of pricing history.

Give it what it needs to answer the question in front of it.

That sounds obvious.

In an agent system, it is surprisingly easy to forget.

2. Separate facts from instructions

This made a bigger difference than I expected.

Consider:

Parking is available behind the building.
Guests should normally use Bay 14.
Sometimes another space may be available.
Do not guarantee a second space.

There are two completely different things happening here.

The first three sentences describe the world.

The last sentence describes what the agent is allowed to do.

Mixing those together makes the prompt harder to reason about.

We now think of them separately:

const facts = {
  parkingType: "allocated",
  primaryBay: "14",
  additionalSpacePossible: true
};

const policy = {
  mayGuaranteeAdditionalSpace: false
};

The distinction matters because facts can change.

Policy usually changes much less often.

It also means the application can enforce some rules without relying on the model remembering them.

3. Database state beats listing text

Listings are written for humans.

Agents need structured state.

Suppose the listing says:

Parking is available for guests.

Perfectly reasonable marketing copy.

But the agent needs to know:

{
  parkingAvailable: true,
  guaranteedSpaces: 1,
  extraSpacesRequireApproval: true
}

Those two things communicate roughly the same information to a human.

They are very different inputs for software.

The more agents we added, the more I found myself converting vague property information into explicit state.

Instead of:

Early check-in may sometimes be available.

Store:

{
  standardCheckIn: "15:00",
  earlyCheckInAllowed: true,
  earliestPossibleTime: "13:00",
  requiresTeamApproval: true
}

The agent can now reason from something much closer to reality.

And more importantly, our application can stop it making promises it shouldn't make.

4. Freshness matters as much as accuracy

There is another problem.

A fact can be correct and still be wrong.

Yesterday:

wifi.status = "working";

Today the router has died.

The database technically contains a fact.

It is just stale.

So useful agent context needs some idea of freshness:

type ContextValue<T> = {
  value: T;
  updatedAt: Date;
  source: "host" | "system" | "channel" | "agent";
};

That opens up much better behaviour.

The application can say:

if (hoursSince(wifi.updatedAt) > 72) {
  requireVerification();
}

Or the agent can respond cautiously rather than stating something as certain.

This became an important mental model for me:

Agent context is not knowledge. It is a snapshot.

Snapshots age.

5. Recheck state before doing anything

This matters even more once agents can act.

Imagine this sequence:

10:00:00 Guest asks for early check-in
10:00:02 Agent reads availability
10:00:08 Cleaner changes schedule
10:00:11 Agent confirms early check-in

The model made the right decision using the information it had.

The system still made the wrong decision.

That is a normal software concurrency problem wearing an AI hat.

The fix is boring:

const suggestion = await agent.decide(context);

const latestState = await bookings.getCurrent(bookingId);

if (!stillValid(suggestion, latestState)) {
  return requireHumanReview();
}

return execute(suggestion);

We use the model to decide what it would like to do.

The application checks whether it is still allowed to do it.

That second check matters far more than making the prompt another 500 words longer.

6. Give the agent events, not endless history

Conversation history causes the same problem.

It is tempting to keep throwing every previous message into the context window.

But imagine a guest has sent 70 messages during a two-week stay.

Most of that conversation is irrelevant when they ask:

What time is checkout tomorrow?

Instead of treating history as one enormous transcript, you can reduce it into state and recent events.

Something like:

const context = {
  booking: currentBooking,
  property: relevantPropertyFacts,

  recentEvents: [
    {
      type: "guest_message",
      text: "What time is checkout tomorrow?"
    },
    {
      type: "late_checkout_request",
      status: "not_requested"
    }
  ]
};

The model gets much less information.

But the information it does get matters more.

That is usually the trade I want.

7. Log what the agent actually saw

This is the part I would build earlier if I started again.

When an agent gives a strange answer, knowing the output is not enough.

You need to know:

What did it believe was true at the time?

So every decision should have a trace.

interface AgentTrace {
  agent: string;
  contextVersion: string;
  input: unknown;
  decision: unknown;
  model: string;
  timestamp: Date;
}

Then when somebody asks:

Why did the agent tell this guest they had two parking spaces?

you don't have to guess.

You can inspect the exact state supplied to the model.

A surprising number of apparent "AI mistakes" turn out to be ordinary software mistakes upstream.

Wrong property.

Old data.

Missing field.

Incorrect booking state.

The model simply gave a perfectly reasonable answer to the reality we accidentally handed it.

The pattern we ended up with

Our agent flow increasingly looks like this:

Guest message
      ↓
Intent / task
      ↓
Context builder
      ↓
Relevant current state
      ↓
AI decision
      ↓
Deterministic validation
      ↓
State recheck
      ↓
Human approval or action
      ↓
Audit log

The LLM sits in the middle.

It isn't the application.

That distinction seems obvious written down, but a lot of agent prototypes blur it.

They build:

data → enormous prompt → model → action

Then try to improve reliability by making the enormous prompt even larger.

Eventually you are asking a probabilistic model to compensate for missing application architecture.

That doesn't scale particularly well.

The short version

If I were building an agent system from scratch now:

Give the model the minimum context required for the current task.
Store important facts as structured data rather than prose.
Keep facts and agent permissions separate.
Track when important context was last updated.
Recheck state immediately before an agent takes an action.
Prefer recent events and current state over enormous conversation histories.
Record exactly what context the model saw when it made a decision.

The strange thing about building AI agents is that the longer I work on them, the less time I spend thinking about the model.

The model is important.

But most of the reliability comes from fairly ordinary software engineering around it.

And honestly, I think that is good news.

I built Zugrow, an AI-first property management platform, and use the same systems across the holiday lets I operate. I'm particularly interested in how other people are handling context construction, stale state and pre-action validation in agent systems.

We let AI agents run 23 vacation rental properties. Here is what we never let them do.

Ben Mccarthy — Wed, 26 Aug 2026 20:17:16 +0000

I run short lets. Twenty three of them. About eighteen months ago I started replacing the parts of that job that were eating my evenings, mostly answering the same question about parking for the fortieth time, with AI agents.

They work. They also taught me that almost everything I first believed about shipping autonomous agents was wrong.

This is not a post about prompt engineering. It is about the four rules we ended up with once real money was moving through the thing, and why three of them are about what the agent is not allowed to do.

Rule 1: The guardrail goes in code, not in the prompt

This is the one I got wrong first, and it is the one I see everywhere.

Our pricing agent reprices every property every night. It has a floor: never go below what the property costs to turn around. My first version put that in the system prompt. Something like:

Never suggest a nightly price below the floor of £X.

It held. Most of the time. Then one night it suggested £38 on a property with a £52 floor, because the surrounding context made a cheap night look reasonable and the instruction was, to the model, one consideration among many.

A prompt is a request. Code is a guarantee. The fix is dull and total:

// The model proposes. Code disposes.
const proposed = await pricingAgent.suggest(context);

const floor = Math.max(
  settingsFloor(property),        // what the host configured
  hardFloor(property, gapNights), // what the maths says it cannot go below
);

return Math.max(proposed, floor);

That Math.max is the entire safety property. It does not matter what the model returns. It does not matter if someone jailbreaks the prompt, or if we swap models, or if the context window fills with something strange. The floor holds because the floor is arithmetic, not persuasion.

The general form: any constraint you would be embarrassed to have violated must be enforced after the model returns, in code that the model cannot influence. If your only defence is an instruction in a prompt, you do not have a constraint. You have a preference.

Ask yourself, for every rule in your system prompt: what happens if the model ignores this exactly once? If the answer is "we lose money" or "we upset a customer" or "we break the law", it does not belong in the prompt.

Rule 2: Ship every agent switched off, and make the user turn it up

Every agent we run has three positions. Off, Suggest, Auto.

export type AgentMode = "off" | "suggest" | "auto";

In Suggest, the agent does the whole job and then stops. It writes the reply and waits for you to press send. It works out the new price and shows it to you. All the work, none of the authority.

Everything ships in Suggest. Not as a beta phase we later remove, but permanently, as the default. The user promotes an agent to Auto themselves, per agent, when that particular agent has earned it in their eyes.

This felt like cowardice when I built it. It turned out to be the single thing that made the product usable, for two reasons.

The obvious one is trust. Nobody hands over their inbox on day one. Suggest lets someone watch an agent be right forty times before it gets to act alone, and that is a much better argument than anything on a landing page.

The less obvious one is that Suggest mode is the best evaluation harness you will ever build. Every time a user edits a draft before sending it, that is a labelled failure, free, in production, with the correction attached. You do not have to construct an eval set that guesses at what real inputs look like. Real inputs are showing up, and users are marking your homework because it is in their interest to do so.

We found our worst prompt bug that way. The messaging agent was signing off in a way that read as slightly cold to guests. No test would have caught it. Forty users editing the same sentence out of forty drafts caught it in a week.

Rule 3: When it fails, work out which direction is safe

"Fail safe" is meaningless until you decide what safe means for that specific agent, and the answer differs per agent.

We have an agent that watches booking requests approaching expiry. If a request is about to time out with no decision, the platform counts that against your response rate, which affects your ranking. So this agent acts on the clock.

It fails closed. If it is fifteen minutes from expiry and nothing has happened, it declines. Declining is the recoverable outcome: a guest can rebook, and your response rate survives. Silently letting it expire is not recoverable.

But note the second half, which took a near miss to learn: it only ever acts on silence. If you or another agent has already answered that request, it does nothing at all. The dangerous version of this agent is the one that decides it knows better and overrides a human decision made four minutes ago.

So the rule is two-sided. Pick the safe direction for the specific failure, and define precisely the state in which the agent is allowed to act at all. "No human has touched this" is usually the right precondition, and it is easy to forget.

Rule 4: Some things never get automated, at any autonomy level

Even on Auto, three categories of action are unavailable to our agents:

Anything that spends or refuses money. A price floor never gets crossed. A booking never gets declined by an agent acting on its own judgment, except in the narrow expiry case above where the alternative is worse.
Anything irreversible. No cancellations, no charges.
Anything that would misrepresent the human. Agents do not make promises on the host's behalf that the host has not agreed to.

This is not a technical limit. We could ship it. It is a product decision, and I think it is the right one, because the failure mode is asymmetric. An agent that is too cautious costs you a few minutes. An agent that declines the wrong booking or undercuts your floor costs you a night's revenue and a guest, and you find out afterwards.

When you are deciding where your own line sits, the question is not "can the model do this reliably?" It is "if this goes wrong at 3am while nobody is watching, is the damage recoverable?" If it is not, keep a human in it, however good your evals look.

What this costs

I want to be honest about the trade, because posts like this usually are not.

Constraining agents this heavily makes the product less impressive in a demo. "It drafts a reply and you approve it" is a worse sentence than "it runs your whole inbox". We have lost people at that sentence.

It also means we ship slower. Every new agent needs its guardrails written in code, which is more work than adding a paragraph to a prompt.

What we get for that is an agent estate that has not yet done something I had to apologise for. Eighteen months, twenty three properties, thousands of guest messages. For software touching other people's businesses and other people's holidays, I will take that trade every time.

The short version

If a rule matters, enforce it in code after the model returns. A prompt is a request, not a constraint.
Ship in Suggest mode. It buys trust, and it is a free production eval harness.
Decide which direction is safe for each agent, and define the exact state it may act in. Usually: only on silence.
Never automate the irreversible, no matter how good your evals are.

None of this is clever. That is sort of the point. The interesting work in agents right now is not making them more capable, it is working out what they are allowed to touch.

I build Zugrow, which is where these agents live, and I host twenty three short lets, which is where they get tested. Happy to answer anything in the comments about how a specific guardrail is implemented.