DEV Community: Meta Luo

When AI Makes Coding Faster, Testing Matters More

Meta Luo — Tue, 22 Sep 2026 07:05:29 +0000

AI-assisted coding has changed the economics of a small product change.

A form, a component, a UI refinement, or a thin API layer can now go from an idea to a working-looking implementation in minutes. That is useful. It also changes where delivery risk accumulates.

The question used to be: Can we build this fast enough?

Increasingly, it is: How do we know that this change—and the flows around it—still work?

AI can accelerate implementation. It cannot automatically understand every business rule, verify every state transition, or guarantee that a change to one page did not break another. As the cost of changing software falls, the cost of confidently validating those changes becomes more visible.

That is why testing matters more in an AI-assisted workflow—not as a slower approval gate, but as the feedback system that makes speed safe.

Fast code can make systems feel more opaque

AI-generated code often looks plausible. Names are reasonable, components render, and comments read well. But production failures rarely come from code that merely looks untidy. They come from behavior that is subtly wrong:

an authorization rule is skipped on one path;
a successful submission does not refresh the state that users see next;
an edge-case input bypasses validation;
a dependent field stops updating after a refactor;
an existing workflow survives locally but fails for a different role or dataset.

Code review still matters. Unit and integration tests still matter. But neither, by itself, proves that a user can complete a critical journey in the actual application.

There is a second, quieter change. When a meaningful portion of an implementation starts as a prompt, completion, or agent-generated patch, engineers may spend less time constructing every detail from first principles. They should still review the change, of course. But the job shifts: less from writing each line, more toward evaluating whether a candidate solution is correct.

That is not a criticism of AI coding. It is simply a reason to strengthen the feedback loops around it.

Faster UI changes expose weak regression habits

Web products feel this first. If UI changes become cheaper, teams ship more of them: component swaps, layout changes, form reorganizations, dropdown behavior changes, and new client-side state. Small changes are individually reasonable; together, they expand the surface area for regression.

Two existing approaches can struggle under that pace.

Manual regression does not scale with change frequency

If development can produce several viable iterations in a day but validation still depends on a person clicking through the same release checklist, testing becomes the bottleneck. The outcome is not that testing is “too slow.” The confirmation method has simply not kept up with the rate of change.

Manual exploration remains valuable for new behavior, ambiguous requirements, usability, and unexpected interactions. It is a poor long-term substitute for repeatedly checking the same stable business path before every release.

UI scripts can become a maintenance tax

Playwright and Selenium are powerful tools. For teams with a maintained test framework, they offer control, integrations, and a great deal of flexibility.

But UI tests are not free once they exist. Locators need discipline. Waits, fixtures, browser state, failures, traces, and page objects all require ownership. When teams accelerate front-end changes without improving their test contracts, the result is familiar: engineers spend more time repairing selectors than checking whether the business flow is correct.

The answer is not to abandon scripted testing. It is to be deliberate about which layer owns which kind of confidence.

Reliable delivery needs two complementary controls

Teams adopting AI coding usually need to improve two points in the loop.

Before implementation: constrain the problem

Before asking an AI tool to generate a significant change, clarify the behavior it must preserve and the behavior it must introduce:

the user and role involved;
state transitions and business rules;
API or data constraints;
error and empty states;
acceptance criteria and regression risks.

This is not ceremony for its own sake. A clear plan narrows the space in which an AI tool can make a confident but incorrect assumption. It also gives reviewers and testers a shared definition of what “done” means.

After implementation: verify the outcome

No plan can replace execution. A test strategy must answer practical questions:

Did the new behavior work for the intended user?
Did the critical path that led here still work?
Did the UI update after the operation completed?
Does the feature behave correctly with realistic data and permissions?

Put simply:

AI helps generate an implementation faster. Testing verifies that the implementation delivers the intended behavior.

One cannot substitute for the other. A model that generated a feature may also help draft tests or analyze failures, but independent, repeatable checks remain essential—especially for the real browser, account state, and data that a user experiences.

Start with a small, repeatable release path

When a team realizes that testing is falling behind, the tempting reaction is to build a comprehensive automation program immediately. That often fails for the same reason a rushed AI-generated feature fails: the scope is too broad, and ownership is unclear.

Start instead with a minimal release loop. For a typical internal web application, it might include:

Login as a representative user (kept as a separate case).
Create or submit one high-value business object.
Verify that the object can be found and that its important state is correct.
Exercise one configuration, approval, or permission-sensitive path.
Run those cases as a short smoke suite before a release.

The first success criterion is not “full coverage.” It is simpler: can the team rerun these paths reliably, and can a failed result show an unfamiliar engineer where the flow first diverged?

That changes the conversation from “we need to automate everything” to “we need five important checks that we can trust.”

UI automation has to survive change, not just record it

Record-and-replay tools, browser recorders, and generated test code all make it easier to capture a first version of a workflow. The harder question is what happens a month later, after several UI iterations.

A useful UI automation asset should answer at least three questions:

Can it run reliably under the same conditions?
Can a small page change be repaired locally rather than forcing a full rewrite?
When it fails, can the team understand the failure from the result rather than reconstructing the run from scratch?

Those questions are more important than whether the first recording took two minutes or twenty.

Good test design helps: stable test IDs, semantically meaningful labels, state-based waits, isolated data, and explicit assertions after important state changes. These are not merely automation details. They are contracts that make the product more observable and easier to evolve.

Where a record-and-replay platform fits

There is a useful middle layer between “every regression path must be hand-tested” and “every UI check must start as a code project.”

CueCast is designed for that layer: teams can record real browser interactions, store them as editable test steps, replay them, and keep results with screenshots for later review. It is a zero-code Web UI automation platform, not a replacement for API tests or an existing engineering-grade Playwright suite.

For the workflows that are stable, business-critical, and repeatedly checked before releases, this model can reduce the cost of getting started and maintaining shared test assets. CueCast records multiple locator clues—including element semantics, text, component context, CSS, and XPath—rather than treating one brittle selector as the entire identity of an element. Playback uses a Chrome DevTools Protocol path for real browser interaction, with a DOM fallback for applicable cases.

The important point is not that a tool makes UI automation magically permanent. No tool can infer a missing business distinction from an ambiguous page. When the product has two indistinguishable “Save” buttons, or an element has only runtime-generated attributes, the durable fix is still a better UI contract—such as a stable test ID, accessible label, or clear component scope.

What a platform can do is make the maintenance loop visible: identify the failed step, review the screenshot and execution details, adjust a local step, replay the case, and then put it back into a scheduled regression plan.

Optional screenshot placeholder — recording-to-result workflow

Use a redacted CueCast case detail or execution-result screen. Show an editable recorded step, a failure screenshot, and a concise error summary rather than a generic product dashboard.

Do not force every quality problem into UI replay

UI regression is important, but it is not the whole test strategy. Keep the boundaries clear:

Use unit tests for isolated logic and fast feedback.
Use API or integration tests for contracts, complex fixtures, and backend state.
Use browser-level regression for user-visible critical flows.
Use exploratory testing to discover risks that predefined checks did not anticipate.
Keep performance, security, accessibility, and data-quality work as dedicated disciplines rather than incidental side effects of UI tests.

Likewise, a team with a healthy Playwright or Selenium suite does not need to replace it in the name of AI adoption. A record-and-replay layer can coexist with code-based tests, particularly where business stakeholders or manual QA need to own and review high-frequency web flows.

The goal is coverage with clear ownership, not one tool to solve every kind of quality risk.

Testing is becoming the speed multiplier

AI coding is likely to keep reducing the time from idea to a running implementation. That does not automatically reduce release risk. If UI and workflow changes arrive faster while validation remains ad hoc, teams will either ship with less confidence or reintroduce manual checking as an expensive, late-stage ritual.

The better response is not a heavier process. It is a tighter loop:

Define the behavior and boundaries before implementation.
Generate and review the change quickly.
Run the most important repeatable checks against the real product.
Treat failures as evidence to investigate, not prompts to retry blindly.
Maintain the tests that protect high-value paths as the product evolves.

When implementation accelerates, testing becomes more—not less—valuable because it is what turns rapid output into reliable delivery.

AI Agents Are Great at Exploratory Testing. Regression Needs Repeatable Assets.

Meta Luo — Thu, 17 Sep 2026 06:31:34 +0000

AI agents have changed the first few minutes of testing a feature.

Give an agent a goal such as “check whether a user can create a project,” and it can open the product, find a route into the flow, fill a form, react to a modal, and inspect the result. When it hits something unexpected, it can look at the page, source code, logs, or network activity and decide what to try next.

That is genuinely useful. It is especially useful while a feature is new, ambiguous, or changing quickly.

But it does not follow that a team should hand all regression testing to an agent. Exploratory testing and regression testing optimize for different things:

Exploration rewards judgment, adaptation, and trying an alternate path.
Regression rewards the same path, the same checks, and evidence that makes one run comparable with the next.

The useful question is not “AI agent or test automation?” It is: which work should remain flexible, and which work is valuable enough to make repeatable?

Disclosure: I work on CueCast, a no-code web regression-testing product. This article reflects the product problem we are building for, but the workflow below does not depend on using CueCast.

Where agents are already excellent

An AI agent is a strong partner when the testing task contains uncertainty.

For example, after a developer finishes a new discount-rule screen, an agent can help answer questions such as:

Can a user reach the new screen from the normal navigation?
What happens with an empty, invalid, or unusually large input?
Does an unexpected error appear in the browser console?
Which conditions make a button enabled or disabled?
Can the reported bug be reproduced from the current branch?

These are not always fully specified in advance. The agent can inspect the page, form a hypothesis, and alter its next action. It can also combine browser work with code and log inspection in a way that is awkward for a conventional UI test.

That makes agents a practical fit for:

developer self-checks;
exploratory testing of a new workflow;
one-off reproduction of a reported bug;
quick smoke checks when the test path is not yet known; and
investigation after a failure.

The output of this work may be a useful conversation, screenshots, a list of observations, or a bug report. That can be enough when the task is temporary.

Why a successful agent run is not yet a regression test

Now consider a different request:

Before every release, verify that an administrator can create a project, find it in the list, and see the correct status.

This request has to work next week, during the next release, and when a different teammate is on call. It needs a more explicit contract.

An agent may still complete that workflow successfully today. However, unless the team deliberately captures the path and its checks, the next run may differ in meaningful ways. It might enter through a shortcut rather than the sidebar, accept a success toast as proof, or inspect the list only sometimes. Adaptive behavior is helpful during exploration; it makes a regression result harder to compare.

Four things are usually missing when an agent run is treated as the entire test asset.

1. A stable path

Regression is not simply “the product looked OK.” It needs a known sequence of actions and preconditions:

sign in as an administrator
  → create a project with unique data
  → save it
  → search for the saved project
  → verify its status is Draft

The path does not have to be rigid forever. It does need to be visible, reviewable, and intentionally updated when product behavior changes.

2. Explicit assertions

Navigation completing is not evidence that the workflow succeeded. A reliable test should say what must be true: the record exists, a status changed, a permission boundary holds, or an expected error is shown.

Agents can suggest those checks, but the checks themselves should become named, inspectable assertions. Otherwise a passing result can quietly mean only that the agent reached a plausible-looking page.

3. Reusable test data and preconditions

Many UI tests fail on their second run because they reuse a name such as Test Customer, depend on an expired session, or assume a prior approval is still pending. A repeatable asset records how it gets the required state and how it avoids collisions—through generated values, variables, controlled fixtures, or a clear setup step.

This is not glamorous test work, but it is what makes a release check trustworthy.

4. Evidence and ownership when a run fails

“The agent could not complete the task” is the start of diagnosis, not the end of it. The next person needs to know:

which case and which step failed;
what the page looked like at that point;
what was expected and what was observed;
whether the likely cause is the product, test data, environment, or a stale test; and
who is responsible for the next action.

An agent can help interpret this evidence. The evidence should not disappear with the conversation that produced it.

A practical handoff: explore, promote, replay

Rather than choosing one approach for every situation, use a handoff between them.

Stage	Primary mode	Deliverable
A new feature or unclear requirement	AI-assisted exploration	Observations, risks, candidate paths, bug reports
A workflow becomes important and repeatable	Test design and capture	Named steps, assertions, test data rules, owner
Every release or relevant change	Deterministic replay	Pass/fail result, step-level evidence, history
A failure or a material UI change	AI-assisted investigation	Likely cause, update proposal, newly discovered risk

The key moment is the middle one: promote a discovered workflow into a team asset once it is important enough to protect repeatedly.

For example, an agent exploring a new project-creation flow may discover that the meaningful success condition is not the “saved” toast. It is that the new record appears in a filtered list with Draft status and is visible only to administrators. That discovery becomes a regression case with three explicit assertions, unique data, and a saved execution history.

The agent has not been replaced. It has done the higher-leverage job: finding uncertainty and helping the team decide what is worth protecting.

What “repeatable” should mean in practice

Repeatability does not mean pretending that a web application never changes. A healthy regression asset has enough structure to make change visible and enough context to repair it locally.

For each high-value workflow, aim to keep:

A business-readable name. “Administrator creates a draft project” is more useful than “case_014.”
Visible steps. A teammate should be able to see the intended interaction rather than reverse-engineer it from a past chat.
Assertions at the business outcome. Check the saved record, state, permission, or calculation—not just a click or URL change.
A data strategy. Define which values are generated, saved for later steps, reset, or provided by a fixture.
Failure evidence. Keep the failing step, screenshot or page state, error, and relevant execution history.
A maintenance owner. Someone should be able to decide whether a change is a defect, a new requirement, or an asset update.

This is also why “record once and forget it” is not a credible promise. User interfaces evolve. The goal is to avoid rewriting an entire workflow for a small, understandable change—and to make the affected step obvious when maintenance is necessary.

Where CueCast fits

CueCast is designed for the repeatable part of this workflow: teams record actions on a real web application and turn them into editable steps, assertions, variables, and execution records. The product is aimed at recurring web business flows—such as sign-in, form submission, approval, configuration, and release smoke tests—where QA, developers, and business testers need to share the same test asset.

It is not intended to replace code-level tests, API tests, or free-form investigation. Complex data setup, deep mocking, and logic-heavy validation may still be best expressed in code. Likewise, an agent may be the best tool for a brand-new path that no one understands yet.

The combination is more useful than either extreme:

AI agent: explore the unknown and investigate changes
        ↓
Team: decide which workflows are release-critical
        ↓
Repeatable test asset: replay known steps and assertions
        ↓
AI agent: help interpret failures and identify the next risk

Start with five workflows, not a grand automation program

If your team is experimenting with AI-assisted testing, do not begin by asking an agent to autonomously cover the whole product. Pick five workflows that are:

run manually before almost every release;
costly when they break;
sufficiently stable to have a known expected outcome; and
understandable by the people who own the business process.

Use agents to probe new behavior around those flows. Then give the recurring checks a durable home with explicit assertions, data rules, and evidence.

That division of labor is simple: let AI spend its flexibility on change and uncertainty. Let repeatable assets protect the work your team already knows must not break.

CueCast is an AI-assisted, no-code web automation tool for recording, replaying, and reviewing recurring regression workflows. Learn more at icuecast.ai.

How to Sync Data from Elasticsearch to Elasticsearch

Meta Luo — Sat, 09 May 2026 05:50:32 +0000

Overview

Elasticsearch is a popular search engine that forms part of the modern data stack alongside relational databases, caching, real-time data warehouses, and message-oriented middleware.

While writing data to Elasticsearch is relatively straightforward, real-time data synchronization can be more challenging.

This article describes how to migrate and sync data from Elasticsearch to Elasticsearch using BladePipe and the Elasticsearch incremental data capture plugin.

Highlights

Elasticsearch Plugin

Elasticsearch does not explicitly provide a method for real-time change data capture. However, its plugin API IndexingOperationListener can track INDEX and DELETE events. The INDEX event includes INSERT or UPDATE operations, while the DELETE event refers to traditional DELETE operations.

Once the mechanism for capturing incremental data is established, the next challenge is how to make this data available in downstream tools.

We use a dedicated index, cc_es_trigger_idx, as a container for incremental data.

This approach has several benefits:

No dependency on third-party components (e.g., message-oriented middleware).
Easy management of Elasticsearch indices.
Consistency with the incremental data capture method of other BladePipe data sources, allowing for code reuse.

The structure of the cc_es_trigger_idx index is as follows, where row_data holds the data after the INDEX operations, and pk stores the document _id.

{
  "mappings": {
    "_doc": {
      "properties": {
        "create_time": {
          "type": "date",
          "format": "yyyy-MM-dd'T'HH:mm:ssSSS"
        },
        "event_type": {
          "type": "text",
          "analyzer": "standard"
        },
        "idx_name": {
          "type": "text",
          "analyzer": "standard"
        },
        "pk": {
          "type": "text",
          "analyzer": "standard"
        },
        "row_data": {
          "type": "text",
          "index": false
        },
        "scn": {
          "type": "long"
        }
      }
    }
  }
}

Trigger Data Scanning

As for the incremental data generated by using the Elasticsearch plugin, simply perform batch scanning in the order of the scn field in the cc_es_trigger_idx index to consume the data.

The coding style for data consumption is consistent with that used for the SAP Hana as a Source.

Open-source Plugin

Elasticsearch strictly identifies third-party packages that plugins depend on. If there are conflicts or version mismatches with Elasticsearch's own dependencies, the plugin cannot be loaded. Therefore, the plugin must be compatible with the exact version of Elasticsearch, including the minor version.

Given the impracticality of releasing numerous pre-compiled packages and to encourage widespread use, we place the open-source plugin on GitHub.

Procedure

Step 1: Install the Plugin on Source Elasticsearch

Follow the instructions in Preparation for Elasticsearch CDC to install the incremental data capture plugin.

Step 2: Install BladePipe

Follow the instructions in Install Worker (Docker) or Install Worker (Binary) to download and install a BladePipe Worker.

Step 3: Add DataSources

Log in to the BladePipe Cloud.
Click DataSource > Add DataSource, and add 2 DataSources.

Step 4: Create a DataJob

Click DataJob > Create DataJob.
Select the source and target DataSources, and click Test Connection to ensure the connection to the source and target DataSources are both successful.
Select Incremental for DataJob Type, together with the Full Data option.

Note
In the Specification settings, make sure that you select a specification of at least 1 GB.

Allocating too little memory may result in Out of Memory (OOM) errors during DataJob execution.

Select the indices to be replicated.
Select the fields to be replicated.

Note
If you need to select specific fields for synchronization, you can first create the index on the target Elasticsearch instance. This allows you to define the schemas and fields that you want to synchronize.

Confirm the DataJob creation.

Note
The DataJob creation process involves several steps. Click Sync Settings > ConsoleJob, find the DataJob creation record, and click Details to view it.

The DataJob creation with a source Elasticsearch instance includes the following steps:

Schema Migration

Initialization of Elasticsearch Triggers and Offsets

Allocation of DataJobs to BladePipe Workers

Creation of DataJob FSM (Finite State Machine)

Completion of DataJob Creation

Wait for the DataJob to automatically run.

Note
Once the DataJob is created and started, BladePipe will automatically run the following DataTasks:

Schema Migration: The index mapping definition in the source Elasticsearch instance will be migrated to the Target. If an index with the same name already exists in the Target, it will be ignored.

Full Data Migration: All existing data in the Source will be fully migrated to the Target.

Incremental Synchronization: Ongoing data changes will be continuously synchronized to the target instance.