DEV Community

Cover image for cypress tap: Closing the Loop Between Your Agent and Your Suite
Cypress profile image David Ingraham
David Ingraham for Cypress

Posted on

cypress tap: Closing the Loop Between Your Agent and Your Suite

#cypress #automation #ai

Your agent can write the test and run the test. Until now, it couldn't see the page.

Test failures will happen. Flake will happen. Writing new tests against a product deadline will happen. We've all been there and will be there again in the future.

Software development is predictable that way, and completely fluid at the same time, which is what makes every day a slightly different problem than the one before it.

As we've all started folding AI into that work, we've learned it's extremely good at producing a lot of code, fast. And one rule has held up better than any other: the better your inputs and the more context the tool has, the better the output. That's the whole reason we encode conventions as Skills, point agents at tickets through MCP, and hand them our patterns instead of hoping they guess right.

But there's a ceiling, and it shows up the moment the agent has to work against a running app.

When it's debugging a broken test or writing a new one, it can't see the DOM. It sees whatever I paste in. No matter how good the Skills are or how well-structured the repo is, the agent is reasoning about a page it has never actually looked at.

That's the gap, and it sits in the middle of a loop we all run every day:

  1. The agent writes the test.
  2. The suite runs it.
  3. Somebody reads what happened.
  4. The agent revises.

Steps 1, 2, and 4 have been handled for a while now.

Step 3 was me, until now.

A Read Interface, Not a Writer

cypress tap is a set of CLI commands that read and drive an open-mode Cypress session from your terminal.

Setup is two terminals. cypress open in one, cypress tap from the same project root in the other. No config, no server, no MCP.

The subcommands split into three jobs:

  • Find and run: sessions, specs, status, run
  • Read results: reporter, command
  • Interrogate the page: pin, dom, aria, inspect

Everything prints readable output by default and machine-readable JSON with --json.

The thing worth noticing is what isn't in that list.

There's no tap write. No tap fix. No tap generate. It takes the context the Cypress app was already showing you on screen and puts it somewhere a terminal, or an agent working in one, can actually reach.

Breaking a Test Purposefully

I have a fake pet adoption site I use for testing out new automation features and for creating workshops against. This exciting new feature was one I hooked up to that repo.

To demo what tap actually does, I needed something red to point at, so I broke it deliberately: I renamed an app-side data-testid from favorite-button-* to favorite-toggle-* and left the spec untouched.

Deliberately simple, too. The point here isn't the puzzle in the break, it's how the agent utilizes cypress tap to get to the solution and how this new approach can be scaled to more complicated, real-life scenarios.

Here's the whole diagnosis, terminal only.

1. Is there a session I can reach? cypress tap drives an already-open Cypress, so first prove one exists using tap sessions.

$ npx cypress tap sessions
SESSIONS (1)
  PID    PROJECT                                         TYPE  BROWSER
  29426  /Users/davidingraham/repos/pet-adoption-portal  e2e   Chrome
Enter fullscreen mode Exit fullscreen mode

2. What can it run? Using tap specs it identifies the open spec from the runner.

$ npx cypress tap specs
SPECS (1)
  cypress/e2e/favorites.cy.js  9 minutes ago
Enter fullscreen mode Exit fullscreen mode

3. Trigger the failure. Next it runs the open spec so it can capture the current output.

$ npx cypress tap run cypress/e2e/favorites.cy.js
▶ cypress/e2e/favorites.cy.js
Enter fullscreen mode Exit fullscreen mode

4. The failure as text. The bare reporter command returns the suite summary. As you can see, I only ran one test and I knew it was going to fail. I imagine when you are actively working on building multiple tests at once, the reporter command will be a great translator between what is happening in the runner and what the agent knows as changes are made.

$ npx cypress tap reporter
cypress/e2e/favorites.cy.js  (started at 7:21:21 PM)
✓ --  ✖ 1  ○ --  00:04

Favorites Feature
   r3  ✖ adds a pet to favorites  4.2s
Enter fullscreen mode Exit fullscreen mode

5. Expand the failing test. The agent can pass in additional flags to expand on what happened at each step, all the way up until the captured failure. Note the test id, r3, which every command from here on scopes to.

$ npx cypress tap reporter --test-id r3
✖ Favorites Feature > adds a pet to favorites  failed

ROUTES (1)
  METHOD  MATCHER             STUBBED  ALIAS        #
  POST    **/api/favorites/*  no       addFavorite  -

BEFORE EACH · h1
   1  request  ● POST 200 http://localhost:3000/api/auth/login
   2  request  ● GET 200 http://localhost:3000/api/favorites
   3  visit    /pets

TEST BODY · r3
   1  get      [data-testid="pets-grid"]
  e1    (xhr) ● GET 200 http://localhost:3000/api/auth/me
  e2    (xhr) ● GET 200 http://localhost:3000/api/auth/me
  e3    (xhr) ● GET 200 http://localhost:3000/api/pets?species=&age=&size=&gender=&search=&sortBy=dateAdded&sortOrder=desc&page=1
  e4    (xhr) ● GET 200 http://localhost:3000/api/favorites
   2  -assert  expected <div.grid.md:grid-cols-2.gap-6> to be visible
   3  get      [data-testid^="favorite-button-"] ✖

✖ AssertionError
  Timed out retrying after 4000ms: Expected to find element: `[data-testid^="favorite-button-"]`, but never found it.

cypress/e2e/favorites.cy.js:29:8
    27 |
    28 |     cy.intercept('POST', '**/api/favorites/*').as('addFavorite');
  > 29 |     cy.get('[data-testid^="favorite-button-"]').first().as('favoriteButton').click();
       |        ^
    30 |     cy.wait('@addFavorite').its('response.statusCode').should('eq', 201);
    31 |
    32 |     cy.get('@favoriteButton').should('contain', '❤️');
Enter fullscreen mode Exit fullscreen mode

6. Pull the failing command as data. Command 3 is the one that failed, so the agent asks for it directly. This is the Command Log row as structured output rather than something you hover in the UI.

$ npx cypress tap command --test-id r3 --command-id 3
TEST BODY · r3
✖  3  get  [data-testid^="favorite-button-"]  failed

CONSOLE PROPS
  Selector  [data-testid^="favorite-button-"]
  Elements  0
Enter fullscreen mode Exit fullscreen mode

7. Put the at-failure DOM back on screen. pin pushes that command's snapshot into the live browser frame, so everything that follows is querying the page exactly as it looked when the test broke.

$ npx cypress tap pin --test-id r3 --command-id 3
⚲ PINNED - (1/1)
TEST BODY · r3
   3  get  [data-testid^="favorite-button-"] ✖
Enter fullscreen mode Exit fullscreen mode

8. Ask the page what's actually there. Next it can read the real ARIA tree from the DOM, which shows twelve favorite buttons rendered and healthy.

$ npx cypress tap aria
navigation
  link  🐾 PawMatch
  ...
main
  heading  Adopt a Pet
  ...
  image  Scout
  button  🤍
  heading  Scout
  button  View Details →
  [...eleven more pet cards...]
Enter fullscreen mode Exit fullscreen mode

9. Inspect the element itself. Finally, tap inspect returns the actual attributes on that button, which is where the answer is sitting.

$ npx cypress tap inspect --selector '.grid button[data-testid]' --at 0
ATTRIBUTES (2)
  data-testid  favorite-toggle-18
  class        absolute top-3 right-3 w-10 h-10 bg-white/90 ...

ACCESSIBILITY
  role  button
  name  🤍

BOX
  x 189   y 564   width 40   height 40
Enter fullscreen mode Exit fullscreen mode

Two lines carry the whole diagnosis.

Step 6 says Elements 0. Step 9 says favorite-toggle-18.

The button was there the entire time. Visible, clickable, exactly where it should be. What changed was its contract with the test.

That's a diagnosis I'd previously have assembled by clicking the red row in the Command Log, hovering the snapshot, opening DevTools on the runner, and squinting at the Elements panel to compare attribute names. Here it's text I can paste straight into the fix commit.

The fix is a one-line selector update, and tap run plus tap reporter confirm it green, without ever touching the GUI.

$ npx cypress tap reporter
cypress/e2e/favorites.cy.js  (started at 7:22:21 PM)
✓ 1  ✖ --  ○ --  310ms

Favorites Feature
   r3  ✓ adds a pet to favorites  302ms
Enter fullscreen mode Exit fullscreen mode

Scale This Up

The demo above was a single renamed attribute. Sure I could have solved it by eye, but think about the front ends most of us are actually testing. Virtualized tables that only render what's in view. Nested modals and portals. Shadow DOM. Component libraries that regenerate class names on every build. Accessibility trees that don't match the visual order. Conditional rendering three states deep, where the element exists in one branch and not the other.

When one of those fails, the hard question is almost never what the error was. The error is right there in the stack trace. The hard question is what was on the page at the moment it broke, and that has always meant a human clicking into the runner and looking.

That's the context that's been missing. Not smarter models, not better prompts. Just a way to ask the page what it looked like and get an answer back as text. tap aria and tap inspect produce that on demand now, which is incredibly powerful and exciting.

On a real application, it's the difference between an agent that can help with triage and one that can only guess at it. I'm looking forward to seeing how many wasted agent cycles disappear now that it has the product context it always needed.

What It Needs to Run

It's a local development tool. tap reads an open-mode session, and cypress open has to be running on the same machine. Sessions register themselves in a shared cache, which is how tap sessions finds them.

There's no headless story, so you can't drop tap into a CI job or a nightly run. This isn't a knock on tap, it's just not how it is designed. CI has no open-mode session in a pipeline for it to read. If you were hoping to hand failure context to an agent triaging a 2am CI failure, that isn't this. At least not yet.

It also means someone has to keep the runner open. Close the session and the agent goes blind mid-task. That's a non-issue when you're sitting there working, but worth knowing if you're multi-tasking or handing work off.

Here's the flip side of that though: because cypress open is up, you're watching. The browser is right there. The agent kicks off the spec and you see the run happen while the terminal narrates it. That's a very different experience from an agent working headless and reporting back a summary. You get the machine-readable context and your own eyes on the same run, at the same time.

Final Thoughts

Back to the loop.

  1. The agent writes the test.
  2. The suite runs it.
  3. Somebody reads what happened.
  4. The agent revises.

Steps 1, 2, and 4 were never the hard part and those solutions have been around for a while. cypress tap hands step 3 over. The Command Log as text. The failing command as data. The DOM and the accessibility tree, on demand, exactly as they were when things broke.

Failures are still going to happen. Flake is still going to happen. The deadlines are definitely not moving. What changes is how fast you find out why, and how much of that you have to type out yourself in the age of AI.

Point your agent at it on the next red test and see if your triage gets shorter.

I'm pretty excited about this one and can't wait to fully implement it.

As always, happy testing.

Top comments (0)