You are looking at a page thinking, "Move this down, use the same spacing as that card, but only on mobile." You know exactly what you mean because you're looking at it. Your coding agent is looking at a repository.
Your coding agent cannot read your mind
So you translate what you see into a prompt. Which element, which page, which breakpoint, this instance or every instance. Add a screenshot and the agent can see the pixels, but it still has to connect them to the right element, component, rule, and scope.
That's the guessing game.
For visual work, the useful context is already there when you make the change. You know the target, the property, the old and new values, the viewport width, and whether you changed one element or a shared rule. Throw that context away and you get to describe it all again in English.
This is the idea behind Pixy. Make the change on the running site and Pixy records the visual context for your coding agent. The agent still decides how to implement it because it has the repository and knows whether that 16px lives in Tailwind, a CSS module, a prop, or somewhere in globals.css you'd rather not discuss.
You already made the change. Describing it again just gives your agent another chance to misunderstand you.
Top comments (2)
Coding agents made implementation cheap, but intent is still expensive.
For UI work, we keep feeding agents more context after the fact: prompts, screenshots, annotations, repo instructions. But the highest-quality context exists a few seconds earlier, while the developer is actually looking at the interface and deciding what should change.
I think the next step is less about making agents better at interpreting our descriptions and more about capturing intent before it has to become a description at all.
The gap you're describing — the human sees rendered state, the agent sees source — is real, and it's the same reason I ended up running a persistent browser session on the agent's side rather than screenshots. The problem with pixel context is exactly what you say: it still has to be mapped back to a component, and that mapping is where the errors stack up.
What I haven't seen anyone solve well: structural context at edit time. Not "here is the DOM" but "here is the rule that produced this box, here is every other element that shares it, here is the breakpoint where the cascade flips." Without that, the agent makes the local change and breaks the shared rule somewhere else.
Does Pixy capture the cascade/scope question, or only the target element and its properties? Because one element out of context is only half of what the human actually knows.