How should we determine the impact of an API change?

richocolate — Sat, 19 Sep 2026 05:40:10 +0000

One thing I've been thinking about lately:

As software gets easier to change, does understanding the impact of a change become the harder problem?

A change that looks small in one repository can affect:

another service
an API consumer
an existing workflow
a dependency
a different team
something that isn't obvious from the code you're currently looking at

And knowing what changed isn't necessarily the same as knowing what could be affected.

I've been experimenting with an open-source project called Kaktoos around this idea.

The workflow I'm exploring is:

Understand → Change → Impact → Verify

Before making a change, the system can provide context about existing services, APIs, workflows, dependencies, ownership, and related work.

After the change, it analyzes what actually changed and what could potentially be affected, then connects that to existing verification workflows.

The interesting part for me is the last step.

Instead of generating another set of tests, can we reuse the verification workflows that already represent how the system is expected to behave?

For example:

POST /orders
    ↓
200 OK
    ↓
GET /orders/123
    ↓
404 Not Found

The API returned success, but the expected outcome didn't actually happen.

That made me wonder:

How do you currently determine what might break when you change something?

Is it mostly:

developer knowledge?
code search?
documentation?
dependency graphs?
CI?
integration tests?
architecture diagrams?
something else?

And once you identify the potentially affected parts, how do you decide what actually needs to be verified?

I'm interested in real workflows here, especially the messy ones that aren't captured by tooling.

I'm building Kaktoos in public, but I'm more interested in learning how other teams solve this problem than trying to prescribe a particular solution.

When AI Writes Both the API Integration and the Tests, What Are We Actually Verifying?

richocolate — Fri, 11 Sep 2026 14:40:00 +0000

I've been thinking about a problem with coding agents that I keep coming back to.

An agent can write an API integration and then write tests for that integration. Everything passes, but the tests may just be confirming the same assumptions the agent made while writing the code.

For example, the agent thinks an endpoint returns:

{
  "total": 100
}

It writes the integration expecting total, and then writes a test that expects total.

The test passes.

But if the real API contract says something different, the whole thing can still be wrong.

I'm experimenting with a small open-source project called Kaktoos that puts an independent verification step between the agent and the API:

AI agent → integration → Kaktoos → OpenAPI + real API → result

The idea is that the verification layer shouldn't share the agent's assumptions.

It currently supports multi-step API workflows, OpenAPI response validation, MCP, and GitHub Actions.

I'm still trying to figure out how far this idea should go. One interesting question that came up is whether contract validation is enough, or whether verification should also check the actual outcome of an operation — for example, creating a resource and then reading it back to confirm the state actually changed.

I'm curious how other people building with coding agents are handling this today.

Do you rely mostly on the agent's generated tests, existing integration tests, mocked APIs, live API tests, or some combination?

GitHub: KaktoosLabs/kaktoos

DEV Community: richocolate

How should we determine the impact of an API change?

When AI Writes Both the API Integration and the Tests, What Are We Actually Verifying?