DEV Community

Fenju Fu
Fenju Fu

Posted on

Model calls first, model-off cache replay second: Spark-X2.5 tests SkillHub with e2e

e2e is the end-to-end testing framework used to drive and verify the real UI and record actions for replay. TDD means test-driven development, commonly introduced as red, green, refactor. Red: write one failing test to define an expected behavior. Green: add only enough implementation to pass it. Refactor: improve the code's structure while keeping tests passing. Work on one behavior at a time, with one test and one minimal implementation before moving on.

In this episode, tdd is the name of a skill in SkillHub. We search for it, open it in the specified namespace, and inspect Files to verify that SKILL.md exists. That makes it the target for checking SkillHub's search, details, and file pages.

Red, green, refactor is a general introduction to TDD. The current source skill specifies one test and one minimal implementation per red → green cycle, and places refactoring in the review stage. The three-step explanation introduces TDD in general; the tdd skill page is the actual UI test target.

For this episode, I connected three real projects: tester-army/e2e, iFlytek’s SkillHub, and a local Spark-X2.5-1.7B model. e2e was number two in the weekly GitHub Trending snapshot captured on October 12, 2026, Beijing time. The task was a small, concrete registry workflow: type tdd, submit Search, open the skill in trend-tdd-en-hd, select Files, and verify that SKILL.md appears.

Stage A: actual tdd search results in the local SkillHub English UI.

Stage A: actual tdd search results in the local SkillHub English UI.

SkillHub v0.2.22 ran as a Windows-native frontend and Spring backend with actual PostgreSQL persistence. Spark ran through llama.cpp using a community Q4_K_M quantization, a 4096-token context, and two CPU threads. The consumer used official e2e 0.19.0 and @e2e-dev/web 0.14.0 packages.

The integration uses the documented StepExecutor extension. The custom executor observes the current page, supplies its real semantic controls and node IDs to Spark, and exposes one typing or clicking action for the requested step. Spark chooses the node. The executor dispatches that selection through e2e’s native action interface. Recording, trace storage, deterministic locators, assertions, and native replay caching remain in e2e.

The first full attempt produced a useful failure. Opening a skill changed the URL before the detail page was ready. The next observation contained footer controls. The model clicked the Product heading, and the click completed. The following deterministic assertion found that the Files tab was still inactive. This preserved a clear difference between successfully dispatching a click and reaching the intended application state.

The correction waits for the Files tab and lets the application finish loading before recording a stable end state. The next complete workflow passed the search, detail route, active Files tab, and SKILL.md checks. An earlier cache also captured Loading... as its end state. When that transient state no longer appeared, native replay required another model call. After recording a stable detail-page state, the model-off check could proceed.

Stage A repair: the active Files tab and actual SKILL.md file list.

Stage A repair: the active Files tab and actual SKILL.md file list.

I then stopped the owned Spark server and verified that loopback port 8098 refused connections. The same suite ran with the native cache in read-only mode. All four actions replayed. Each agent.act reported zero model calls, the run used zero model tokens, and no physical model request receipt was written. Cache-file hashes were unchanged. The test took 23.63 seconds; the runner reported 25.95 seconds for the complete run.

Stage B: detail page during model-off native cache replay; the zero-call result comes from this replay report.

Stage B: detail page during model-off native cache replay; the zero-call result comes from this replay report.

Zero model calls describes only Stage B: cache replay after the model server was stopped. Stage A's model-assisted runs and repairs used model calls; the repair run mixed cached actions with two new model decisions.

The recorded outcome is specific and reproducible: this verified SkillHub search/details/files workflow passed after the model server was stopped. The video shows the real footer-click failure, the corrected Files result, and the four-action native replay with zero model calls.

Sources: e2e, SkillHub, Spark-X2.5, XHToken, weekly snapshot source.

The presenter retains a sailor outfit colored to match SkillHub's UI. This revision removes all hand and paw gestures, using eye contact, blinks, changes around the eyes, and speech mouth shapes instead.

Music: Oechsler, Organ Prelude, Op.8 No.1; TomatoNCheese using Lipiny Organ sampled by Piotr Grabowski; IMSLP #1063550, Public Domain. Edited excerpt starting at source 3.0 seconds, mixed with narration. Recording source.

Screenshots are native e2e captures from our local SkillHub test instance using the official mock identity and English UI. They are cropped content regions. SkillHub and e2e are licensed under Apache-2.0; the displayed tdd skill text comes from the MIT-licensed mattpocock/skills.

Top comments (0)