DEV Community

Cover image for Building CodeMapRT: Rethinking Regression Testing with Change-Mapping
yuliadrogunova
yuliadrogunova

Posted on

Building CodeMapRT: Rethinking Regression Testing with Change-Mapping

#testautomation #softwaretesting #qa #cicd

How I turned endless full regressions into fast, targeted runs driven by code changes.

Where It Started

On large enterprise projects, regression testing always felt like a bottleneck. Every pull request meant running the entire suite — hundreds of tests, even for a one-line fix. Developers were frustrated. Pipelines got slower. CI bills grew.

At some point, I asked myself:
“Why are we testing everything, when only part of the code has changed?”

Looking at the Problem

Digging deeper, I noticed a few things:

  • Most commits only touched a handful of modules.
  • Yet, the same giant regression pack kept running over and over.
  • Feedback to developers was delayed, which slowed down releases.

The waste was obvious — we were validating a lot of code that never changed.

The Turning Point

The idea clicked: Instead of running all tests, map what changed in the codebase to the tests that actually matter.

That became the core principle behind CodeMapRT — a framework that aligns code diffs with relevant test cases.

How I Approached It

Step 1. Detecting What Changed

I started simple — just grabbing modified files from Git:

# Detect changed files in the last commit
git diff --name-only HEAD~1 | grep ".java" > changed-files.txt
Enter fullscreen mode Exit fullscreen mode

Then I introduced lightweight annotations in tests to specify which modules they cover:

@Test
@CoversModule({"auth", "login"})
public void loginTest() {
    // ...
}
Enter fullscreen mode Exit fullscreen mode

Tests that didn’t match the changed files? Skipped.
Mapping can be configured via annotations or other metadata sources,
including coverage reports.

Step 2. Keeping It Safe

Selective regression always carries a risk of missing hidden dependencies.
To address this, CodeMapRT provides two safety layers:

  1. Fallback full run — can be triggered for major releases or whenever coverage data looks incomplete.
  2. Safety tests — a configurable set of additional checks added on top of the selected tests. This way, even if only part of the suite is executed, critical scenarios remain validated.

Step 3. CI/CD Integration

For real adoption, I built a CLI tool that detects changes, maps them to relevant tests, and outputs a filtered test list.

Example with GitHub Actions:

jobs:
  codemaprt:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run CodeMapRT
        run: |
          ./codemaprt detect
          ./codemaprt select
          ./gradlew test --tests $(cat selected-tests.txt)
Enter fullscreen mode Exit fullscreen mode

CodeMapRT can be integrated with any CI/CD system — GitHub Actions, Jenkins, GitLab CI, TeamCity — making it scalable for enterprise workflows.

Example Use Case

Imagine a mobile banking app:

  • A developer modifies the login module.
  • Instead of re-running the entire regression suite, CodeMapRT analyzes the code changes and selects only the most relevant automated tests for that module.
  • Result: targeted test execution, faster pipelines, quicker releases, and reduced infrastructure load.

What Changed in Practice

With CodeMapRT, regression testing no longer requires executing the full suite for every commit. Instead:

  • Small commits trigger only the tests linked to the changed modules.
  • Medium features run a focused subset instead of hundreds of unrelated tests.
  • Full regression runs remain possible for major releases, but they no longer block day-to-day development.

This approach shortens the feedback loop, optimizes CI/CD resources, and gives developers faster confidence in their changes.

Benefits of CodeMapRT

  • Accelerates release cycles — less time spent running unnecessary tests.
  • Maintains coverage — critical paths are still validated.
  • Reduces expenses — fewer resources consumed in regression runs.
  • Scales for enterprise projects — adaptable to large and complex systems.

The Results: Smarter Regression

After building a demo project with 50 automated tests and integrating CodeMapRT:

  • Without CodeMapRT: all 50 tests ran on every commit.
  • With CodeMapRT: the framework selected 28 tests directly linked to the changed module, plus 5 additional safety tests (as described in Step 2) → 33 tests in total.
  • Result: ~34% fewer test executions, reducing CI load while ensuring that critical scenarios remain validated.

This demo shows a one-third reduction, but the approach scales: in enterprise projects with thousands of tests, CodeMapRT can deliver even greater efficiency, since typically only a small fraction of the suite is relevant to a given commit.

You can find demo results in the GitHub Actions of the CodeMapRT repository

Top comments (0)