<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Nitin</title>
    <description>The latest articles on DEV Community by Nitin (@nitin_59f9bb54f7686214ae6).</description>
    <link>https://dev.to/nitin_59f9bb54f7686214ae6</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4254304%2Fb20b9447-f7c4-4599-96a7-f2f48c0ae2a3.png</url>
      <title>DEV Community: Nitin</title>
      <link>https://dev.to/nitin_59f9bb54f7686214ae6</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/nitin_59f9bb54f7686214ae6"/>
    <language>en</language>
    <item>
      <title>Pilotfish: Redesigning Browser Agents</title>
      <dc:creator>Nitin</dc:creator>
      <pubDate>Thu, 19 Nov 2026 19:58:50 +0000</pubDate>
      <link>https://dev.to/nitin_59f9bb54f7686214ae6/pilotfish-redesigning-browser-agents-2ngc</link>
      <guid>https://dev.to/nitin_59f9bb54f7686214ae6/pilotfish-redesigning-browser-agents-2ngc</guid>
      <description>&lt;p&gt;An agent opens an invoice. The network is quiet. The DOM has stopped changing. The balance says &lt;strong&gt;$8,400&lt;/strong&gt;, under a banner that reads &lt;strong&gt;"Refreshing balances"&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The wrong optimization is to return that number faster.&lt;/p&gt;

&lt;p&gt;The useful one is to cut repeated page text and model round trips &lt;strong&gt;without losing track of what the agent knows&lt;/strong&gt;. The rule that makes the rest work: a diff must start from the page the caller actually received, not merely the one the browser captured.&lt;/p&gt;

&lt;p&gt;I work on Pilotfish. This is a shorter engineering companion to my &lt;a href="https://nitinreddy-a.github.io/pilotfish-article/" rel="noopener noreferrer"&gt;interactive article&lt;/a&gt;, using a fictional receivables portal. The customer, invoice and amounts below are illustrative.&lt;/p&gt;

&lt;h2&gt;
  
  
  Contents
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Keep the row intact&lt;/li&gt;
&lt;li&gt;Diff against the page the model received&lt;/li&gt;
&lt;li&gt;Run the failure and recovery&lt;/li&gt;
&lt;li&gt;Quiet does not mean ready&lt;/li&gt;
&lt;li&gt;Batch the typing, not the judgment&lt;/li&gt;
&lt;li&gt;What to test&lt;/li&gt;
&lt;li&gt;What the published runs show&lt;/li&gt;
&lt;li&gt;Connect this to a browser&lt;/li&gt;
&lt;li&gt;Video and source notes&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Keep the row intact
&lt;/h2&gt;

&lt;p&gt;A page serializer can drop generated CSS classes without dropping the relationship between an invoice and its balance:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Customer | Invoice | Outstanding
Acme Fasteners | [e5]INV-1042 | $8,400.00
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important part is the row, not the character count. If serialization separates the amount from its invoice, the model gets fewer tokens and a worse problem.&lt;/p&gt;

&lt;p&gt;Pilotfish's plain-row path keeps cells in order and the link's action reference. It does &lt;strong&gt;not&lt;/strong&gt; flatten every table indiscriminately: form controls, stateful cells and nested tables can fall back to the normal representation. The &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/packages/perception/src/serialize.ts#L301-L376" rel="noopener noreferrer"&gt;row serializer&lt;/a&gt; spells out those boundaries.&lt;/p&gt;

&lt;p&gt;Viewport-only output needs another contract: say what was left out and how to read more. Offscreen is not the same as irrelevant. A compact view can still miss the control the task needs, so keep a fuller read available.&lt;/p&gt;

&lt;h2&gt;
  
  
  Diff against the page the model received
&lt;/h2&gt;

&lt;p&gt;Typing a customer name changes a field and a few suggestions. Resending the unchanged invoice table is wasteful. A diff helps, provided the model can apply it.&lt;/p&gt;

&lt;p&gt;The base must be a full observation that &lt;strong&gt;reached the caller&lt;/strong&gt;, not the server's latest capture.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx2x2j9xtn2q39sppldof.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx2x2j9xtn2q39sppldof.png" alt="Illustrative protocol sequence: A is withheld, diff A to B is rejected, full B recovers the base, then diff B to C succeeds" width="800" height="608"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Illustrative protocol model. This is not a recording of a Pilotfish run.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Capture A can exist on the server while the caller still has no page at all. If A is withheld, a later diff from A to B is unusable. The recovery is a full B, not a cleverer diff.&lt;/p&gt;

&lt;p&gt;In the implementation described here, a full observation commits as the base on delivery. A withheld reply leaves no base; the next observation must be full again. See the &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/packages/mcp/src/server.ts#L325-L438" rel="noopener noreferrer"&gt;delivery/base rules&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;A diff also needs page order. Two invoices swapping balances must not look like an unchanged bag of lines. The &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/packages/perception/src/observation.ts#L142-L263" rel="noopener noreferrer"&gt;ordered diff&lt;/a&gt; uses surrounding context, counts replaced lines and names removed interactive refs.&lt;/p&gt;

&lt;p&gt;When the document changes, the base gets old, or the diff becomes too large or ambiguous, send a full observation. Resynchronization is part of the design, not an embarrassing fallback.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run the failure and recovery
&lt;/h2&gt;

&lt;p&gt;Save this as &lt;code&gt;delivered-base.mjs&lt;/code&gt; and run &lt;code&gt;node delivered-base.mjs&lt;/code&gt;. It uses only Node's built-in assertions. No browser, account, API key or model call is needed.&lt;/p&gt;

&lt;p&gt;This is a deliberately small &lt;strong&gt;protocol model&lt;/strong&gt;, not Pilotfish's implementation or an integration test. Each balance stands in for a full page. Delivery is synchronous; a real transport must also handle dropped responses, acknowledgements and resets. Its job is to make the broken-base case reproducible.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;assert&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s1"&gt;'node:assert/strict'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// A small protocol model, NOT the Pilotfish implementation.&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;capture&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;balance&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;balance&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;capture&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'A'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;8400&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;capture&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'B'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5150&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;capture&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'C'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;base&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;receive&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;kind&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="s1"&gt;'diff'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;equal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;base&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'unknown diff base'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;base&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;balance&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;balance&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;base&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// Failure: the server captured A, but A never reached the client.&lt;/span&gt;
&lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;throws&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;receive&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'diff'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;balance&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;balance&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="kt"&gt;unknown&lt;/span&gt; &lt;span class="n"&gt;diff&lt;/span&gt; &lt;span class="n"&gt;base&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'FAILURE CAUGHT: diff A -&amp;gt; B has no delivered base'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;delivered&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;replyFor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;withheld&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;withheld&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// no delivery, no committed base&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;reply&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;delivered&lt;/span&gt;
    &lt;span class="o"&gt;?&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'diff'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;delivered&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;balance&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;balance&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'full'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nf"&gt;receive&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// synchronous delivery in this toy model&lt;/span&gt;
  &lt;span class="n"&gt;delivered&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;reply&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;equal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;replyFor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;equal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;replyFor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'full'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;deepEqual&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;base&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'RECOVERED: full B, balance 5150'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;equal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;replyFor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'diff'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;assert&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;deepEqual&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;base&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'NEXT: diff B -&amp;gt; C, balance 5000'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'PASS: withheld capture never became the diff base'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Checked output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;FAILURE CAUGHT: diff A -&amp;gt; B has no delivered base
RECOVERED: full B, balance 5150
NEXT: diff B -&amp;gt; C, balance 5000
PASS: withheld capture never became the diff base
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Try changing the withheld branch to &lt;code&gt;if (withheld) { delivered = page; return null; }&lt;/code&gt;. Recovery now fails with &lt;code&gt;unknown diff base&lt;/code&gt;: the server sends a diff even though the caller still has no base. That is the bug this model isolates.&lt;/p&gt;

&lt;h2&gt;
  
  
  Quiet does not mean ready
&lt;/h2&gt;

&lt;p&gt;Back to the invoice. A partial payment has arrived, but the balance panel has not caught up.&lt;/p&gt;

&lt;p&gt;There are two checks:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Check&lt;/th&gt;
&lt;th&gt;What it tells you&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;The action settled&lt;/td&gt;
&lt;td&gt;The requests and rendering caused by this action have finished, within the wait's limits.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The needed value is ready&lt;/td&gt;
&lt;td&gt;The page no longer presents it as loading or interim.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Whole-page network quiet is a poor shortcut. Analytics and polling can keep a useful page busy. A quiet page can still display a loading balance.&lt;/p&gt;

&lt;p&gt;Pilotfish's &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/packages/browser/src/page.ts#L1195-L1253" rel="noopener noreferrer"&gt;adaptive settle step&lt;/a&gt; follows work started by the action rather than waiting for all traffic. Write requests get particular attention: an unanswered write is not an accepted write.&lt;/p&gt;

&lt;p&gt;A separate &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/packages/agent/src/agent.ts#L265-L272" rel="noopener noreferrer"&gt;loading-indicator option&lt;/a&gt; waits up to six seconds for signals such as &lt;code&gt;aria-busy&lt;/code&gt; or an indeterminate progress bar. If they remain, the reply says so.&lt;/p&gt;

&lt;p&gt;Neither check proves every website is correct. Loading signals can be absent or misleading. The point is to preserve uncertainty instead of quietly converting it into a reported balance.&lt;/p&gt;

&lt;h2&gt;
  
  
  Batch the typing, not the judgment
&lt;/h2&gt;

&lt;p&gt;Now the balance is confirmed at &lt;strong&gt;$5,150&lt;/strong&gt;. The invoice is not disputed, and no recent reminder exists. The workflow allows a reminder &lt;strong&gt;draft&lt;/strong&gt;, pending human review.&lt;/p&gt;

&lt;p&gt;The recipient, subject, message and internal note are known. These fills can share one model decision:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;decide once
  -&amp;gt; fill email   -&amp;gt; check
  -&amp;gt; fill subject -&amp;gt; check
  -&amp;gt; fill message -&amp;gt; check
  -&amp;gt; fill note    -&amp;gt; check
stop before Send
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The browser steps still run in order. Batching removes the decisions and tool calls between them, not the checks.&lt;/p&gt;

&lt;p&gt;What if the email validator is still spinning after step one? Stop. Known text does not make a changing page safe to keep acting on.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/packages/mcp/src/server.ts#L766-L838" rel="noopener noreferrer"&gt;batch executor&lt;/a&gt; also stops for failures, navigation, dialogs, new tabs, refused or unanswered writes, loading, and sign-in/check states. An invalid field is reported, but is not by itself a stop condition in this implementation. That distinction belongs in the contract.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to test
&lt;/h2&gt;

&lt;p&gt;Before celebrating smaller replies, test the awkward cases:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Swap two rows' amounts. Does the diff preserve ownership?&lt;/li&gt;
&lt;li&gt;Withhold a full reply. Does the next action recover a usable base?&lt;/li&gt;
&lt;li&gt;Leave a loading indicator visible after requests finish. Is the value still treated as interim?&lt;/li&gt;
&lt;li&gt;Leave a write unanswered. Is it distinguishable from success?&lt;/li&gt;
&lt;li&gt;Trigger validation during a batch. Do later actions stop where promised?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Then measure the whole task, including recovery reads and failures. A tiny reply followed immediately by another model call may cost more than a useful first observation.&lt;/p&gt;

&lt;p&gt;Fewer tokens are a result worth measuring. They are not the correctness argument.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the published runs show
&lt;/h2&gt;

&lt;p&gt;On 11 November 2026, the project ran the &lt;strong&gt;same WebVoyager task set separately on each tool&lt;/strong&gt;: Pilotfish open MCP 0.1.39 and Playwright MCP 0.0.84. After matched exclusions, &lt;strong&gt;Pilotfish completed 65 correctly out of 71 eligible tasks; Playwright completed 61 correctly out of the same 71 eligible tasks&lt;/strong&gt;*. Counts and medians were recomputed from the &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/benchmarks/webvoyager-100/results/2026-11-11-luna.jsonl#L1" rel="noopener noreferrer"&gt;published result rows&lt;/a&gt;.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Matched tasks only*&lt;/th&gt;
&lt;th&gt;Pilotfish open MCP&lt;/th&gt;
&lt;th&gt;Playwright MCP&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Correct / eligible tasks&lt;/td&gt;
&lt;td&gt;65 / 71&lt;/td&gt;
&lt;td&gt;61 / 71&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Incorrect / eligible tasks&lt;/td&gt;
&lt;td&gt;6 / 71&lt;/td&gt;
&lt;td&gt;10 / 71&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median input tokens&lt;/td&gt;
&lt;td&gt;29,526&lt;/td&gt;
&lt;td&gt;90,342&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median tool calls&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median seconds&lt;/td&gt;
&lt;td&gt;100.0&lt;/td&gt;
&lt;td&gt;115.7&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;* Exclude a task from &lt;strong&gt;both&lt;/strong&gt; columns if either tool was site-blocked (24 task pairs), then exclude the 5 remaining pairs with a pre-turn error in either run. The resulting 71-task denominator contains identical task IDs for both tools, each run separately. All medians use that same set. These results describe eligible attempts, not overall success or site-access coverage; excluded records remain in the linked data.&lt;/p&gt;

&lt;p&gt;This is one project-run, whole-system comparison: GPT-6 Luna drove and judged both arms on one Windows machine in India, with five tasks running concurrently. It is not independent validation or proof of accuracy superiority. The open MCP comparison does not test the managed service. &lt;a href="https://github.com/gAI-ventures/pilotfish-ai/blob/b990f0f08b6aa552600c8a29c12fa42d8c6bdbcd/benchmarks/webvoyager-100/README.md#what-is-different" rel="noopener noreferrer"&gt;Method and reproduction details&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Connect this to a browser
&lt;/h2&gt;

&lt;p&gt;Project links, grouped by what you want to use:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://pilotfish.gai.ventures/" rel="noopener noreferrer"&gt;Pilotfish website&lt;/a&gt;: product overview and documentation.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/gAI-ventures/pilotfish-ai" rel="noopener noreferrer"&gt;Open-source GitHub repository&lt;/a&gt;: source code and the pinned implementation sections above.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.npmjs.com/package/@gaiventures/pilotfish-mcp" rel="noopener noreferrer"&gt;Open MCP on npm: &lt;code&gt;@gaiventures/pilotfish-mcp&lt;/code&gt;&lt;/a&gt;: connect an MCP client to Chrome on your machine. See the &lt;a href="https://pilotfish.gai.ventures/docs/open" rel="noopener noreferrer"&gt;open MCP setup instructions&lt;/a&gt;; this route needs Node.js 22+ and Chrome or Chromium.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.npmjs.com/package/@gaiventures/pilotfish" rel="noopener noreferrer"&gt;TypeScript SDK on npm: &lt;code&gt;@gaiventures/pilotfish&lt;/code&gt;&lt;/a&gt;: use Pilotfish from TypeScript, rather than registering an MCP server.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For Claude Code, the documented registration command is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;claude mcp add pilotfish -- npx -y @gaiventures/pilotfish-mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This registers the open MCP, not the TypeScript SDK or managed relay. The command is from the setup docs; the local test above runs only the protocol model. Start browser experiments on a disposable local page, then test response withholding and page resets.&lt;/p&gt;

&lt;h2&gt;
  
  
  Video and source notes
&lt;/h2&gt;

&lt;p&gt;The &lt;a href="https://www.linkedin.com/feed/update/urn:li:activity:7529119505084669952/" rel="noopener noreferrer"&gt;62-second LinkedIn explainer&lt;/a&gt; gives a visual overview of Pilotfish. It is not a recording of the fictional invoice task. The &lt;a href="https://nitinreddy-a.github.io/pilotfish-article/" rel="noopener noreferrer"&gt;original interactive article&lt;/a&gt; has the longer walkthrough; implementation and benchmark links here pin commit &lt;code&gt;b990f0f&lt;/code&gt;.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>architecture</category>
    </item>
  </channel>
</rss>
