DEV Community: Nico Krijnen

MCP Apps - Finally a UI paradigm that speaks the language of intent

Nico Krijnen — Sat, 21 Mar 2026 00:00:00 +0000

Earlier this month I spoke at the Dutch AI Conference. After my own talk, I attended a talk by Glenn Reyes about MCP-UI. While listening, something profound struck me, something that I think not many others may have realized. So I figured this insight is worth writing down and sharing.

Glenn argued that due to the text-based nature of our interactions with LLMs and agents, we are throwing away decades of UX expertise. MCP-UI has been an attempt to change that, allowing interactive visual components to be provided through MCP (Model Context Protocol), an open standard that gives AI assistants the ability to connect to external tools and take real action, not just generate text. Development of MCP-UI went quite fast and currently, interactive HTML-based interfaces are an official part of the MCP Apps standard.

MCP Apps solve the how do we make agent UIs better problem. But sitting in that audience, a different question grabbed me: how do you design what those agent interactions should be? Because building a beautiful MCP App that solves the wrong problem is still solving the wrong problem.

That question has been answered before. Not by the AI community, but by a different group of practitioners who’ve spent the past decade perfecting collaborative techniques for exactly this: understanding complex behavior, modeling intent, and building shared language between the people who know the domain and the people who write the code. These two worlds have barely met. So let’s see if we can bring these communities together!

Troublesome state

UIs have always been built around state. Lists, forms, search bars, toggles, modals, loading spinners, all of these require state to be managed. Many frameworks have been built in attempts to tame state complexity (Redux, MobX, signals), but the fundamental model stays the same: UI reacts to data, components reflect state. Even with these frameworks, UI code is still hard to reason about, hard to test, and far away from the language of the business.

Language of the business

What do I mean with “language of the business”, and why is that important? Developers (and their coding agents) write the code. To build the right thing, they need to understand the what and why behind what they are building or changing. Otherwise they solve the wrong problem…

The understanding that is needed typically lives with non-technical people: the users of the system or the people who know how things really work. They don’t speak your technical language and they shouldn’t have to. What you need is to build deep understanding by using a shared non-technical language. More on that later…

Intent

What struck me during Glenn’s talk was that MCP-UI and MCP Apps don’t manage state in the usual way that we’ve been doing that for frontend components. Instead, the primitive is the message or tool call, not the component. The UI becomes a surface for expressing goals, not navigating menus. Instead of “fill form, submit, handle error state”, the interaction is “user states intent, agent invokes tool, result surfaces”.

In MCP Apps, every meaningful interaction is a named action with context. Things like: “SearchProducts”, “PlaceOrder”, “SummarizeDocument”. Sounds familiar? That’s because it’s very close to the commands used in EventStorming. Think of an MCP tool call like agent’s way of issuing a command to the outside world. A command is simply a request to do something. It captures what someone wants to happen in a name and with the information needed to do it. Commands model intent directly, they express “I want this to happen” rather than “set this field to that value”. They carry meaning, not mechanics.

A command like “ApproveExpenseReport” says why something needs to happen, not just how the data changes. When modeled correctly, commands speaks the language of the business, not the language of the database.

Designing Intent-based AI interactions

Let’s start with a concrete example. Imagine you’re building an MCP App for a warehouse operations team. Order pickers need to collect items from shelves and get them to the loading dock. A picker says “plan my picks for the next batch” and the agent needs to respond with an optimized walking route through the warehouse. You can’t describe a spatial route in text. This domain needs a visual interface. So how do you design it?

Instead of the usual: “let the dev team dream up how this will work”, let’s do some collaborative modeling and build some deeper understanding.

You get the warehouse team into the room and run an EventStorming session. You start by asking: “What are all the things that happen during a pick run?” Sticky notes go up: PickOrderReceived, RouteOptimized, ItemPicked, CartFull, BatchCompleted. Standard stuff, roughly what you expected.

Then the warehouse supervisor adds one: ObstructionReported. You ask what that means. Turns out aisle blockages happen every day: pallet spills, restocking crews. When a planned route is blocked, pickers currently find out by walking into the obstruction and rerouting on the fly, or by shouting across the floor. This was nowhere in the requirements.

You keep going. ColdChainTimerStarted appears: frozen items have a 12-minute window from cold storage to the insulated shipping container. HazmatSegregationViolationDetected: cleaning chemicals and food products can never share a cart. The process is far richer than anyone expected. Commands pop up: PlanPickRoute, PickItem, ReportObstruction, SplitCart, SubstituteItem. Each maps directly to an MCP tool definition, named in the language of the warehouse team.

Then something interesting happens. When the supervisor explains how obstruction rerouting works, sticky notes aren’t enough. She grabs a marker and draws the floor plan on the whiteboard next to the event timeline. Aisles, shelving sections, cold storage tucked in the back corner, the loading dock. She marks where today’s restocking is blocking aisle 6 and draws the detour path a picker would take. Someone grabs colored markers: orange zones for hazmat storage, a blue snowflake near cold storage, red X’s for blockages.

You zoom in on the tricky parts with Example Mapping. You pick PlanPickRoute and explore the rules:

Rule: cold chain : “Given a batch that includes frozen items, when the route is planned, then cold storage items must be picked last to stay within the 12-minute window.”
Rule: hazmat segregation : “Given a batch with both cleaning chemicals and food products, when the route is planned, then the cart must be split and items picked in separate runs.”
Edge case: “What if the only route to cold storage is blocked?” The supervisor pauses. “Good question. I think… the picker skips the cold items and comes back for them after the obstruction clears? We’ve never actually decided that.”

The sketch on the wall becomes the MCP App: a floor plan with aisles, an optimized route line, obstructions in red, color-coded item categories. The events and commands from the timeline map to tool definitions. The visual the supervisor drew to explain her world is the visual the picker will use every day. The modeling session didn’t just uncover what the interactions should be, it produced the UI too.

Which techniques did you say?

Understanding how things work and building systems that fit into that is something we’ve been doing for a long time already, so it’s no surprise that there are plenty of tools that we can use to do that effectively. I think the following 4 have a an exceptionally good use for designing AI agent interactions, especially when using MCP and MCP Apps.

EventStorming

This collaborative modeling technique helps you understand system behavior. It maps business processes as a storyline of domain events, triggered by commands from actors. This is exactly how an AI-driven UI interaction flows.

With EventStorming, the initial focus is only on events: facts that have happened. That is not by accident. When you think primarily about command and actions, it is easy to fall into the trap of modeling how things work. They model intention: things may still end up different than intended. The big benefit of starting with events is that they force you to focus on desired outcomes, not on how you get there. That gives you the freedom to find better ways to do things, to find a simpler command trigger for an event, still arriving at the same outcome. Both EventStorming and Event Modeling also include sketching UI interactions directly on the wall: wireframes, screens, visual cues that show what the user sees at each step. For MCP Apps, those sketches naturally become the design for custom visual components.

Domain Storytelling

More suitable for interview style, domain storytelling captures how actors work together to achieve goals through concrete work stories. These stories help you understand how the domain actually works. Again, this is a very natural fit for mapping the intent-flows of AI agent and MCP interactions, told in the language of the domain. When you use this technique, you draw an easy to understand visual story of what the person you’re talking to is explaining. The key here is the quick feedback and validation. If you misunderstood something, they will instantly point out the mistake in your drawing or point out edge cases not yet covered.

Event Modeling

Very similar to EventStorming, also modeling the intent (commands), the facts (events) and the views (read models). This feels like an almost perfect conceptual match for MCP tool calls, conversation history and context windows.Event Modeling organizes parts of the flow as vertical slices, each slice a complete flow from intent to state change to updated view. The key being that each slice is an isolated piece of behavior that can be implemented as one unit of work. This makes estimation and planning a breeze. As each slice is isolated, it can also be implemented in parallel with other slices without waiting for the rest of the system to be done. Productivity-wise, this is rather interesting now that we can put many AI agents to work in parallel to each build a piece of a system. Slices also map nicely to how MCP Apps work, each slice essentially one tool interaction: what the user asks for, what happens and what they see as a result.

Example Mapping

The more concrete you can make it, the easier it is to spot edge cases or incorrect assumption. Example Mapping is an excellent tool to quickly explore the rules around a specific command or intent: given this situation, when this happens, then this should be the outcome. Perfect to capture exactly what an MCP tool should do and how it should respond in different scenarios.

Nowadays, you can take the raw output of an example mapping session, give it to your coding assistant and turn it into a test-suite. And with good behavioral tests and the right architecture guidelines, your coding agent will have a breeze creating the implementation for you too.

What I find most interesting is how all of these techniques are not just similar to how MCP tools work, conceptually they map almost one-to-one.

The real superpower

All these are not just modeling tools, they are collaborative discovery tools. They work so well at uncovering complex behavior because they put experts and developers in the same room : working side by side without any intermediaries. This direct contact creates shared understanding before a single line of code is written. It also creates that insight very quickly and much deeper than with written specs. Result: high focus during implementation, with very few questions or points of confusion. And when the inevitable unforeseen edge case emerges, the personal connections that developers have built up with the experts mean that they know exactly who to call.

Contrast that with the traditional approach: BA → Specs → PO → Story → Tasks → Dev/AI → Code. Where every handoff is a translation loss: the telephone-game effect in action. By the time the requirements reach the implementor, they are abstract, ambiguous, full of assumptions and often wrong in ways nobody notices until late. The typical result is expensive rework, long feedback cycles and clarity arriving at the worst possible moment.

The language aspect matters even more for our interaction with AI agents, where the “UI” is largely language. If you get the intent vocabulary wrong, the whole thing feels broken to it’s users. This focus on precise, shared language has its roots in Domain-Driven Design and is now more important than ever. Getting the vocabulary right is no longer just a modeling concern, it’s a usability concern. I guess that deserves a blog of its own.

What this all means for you

AI agents are becoming part of how we build, design and use software. Intent-based interaction through MCP fit naturally to command and event based design tools. Collaborative modeling techniques like EventStorming, Domain Storytelling, Event Modeling, and Example Mapping have been perfected over the past decade for exactly this kind of problem: understanding what a system should do by working directly with the people who know. If you want to build MCP Apps that actually solve the right problems, start by getting the right people in the room.

If you’d like a hand getting started, whether it’s facilitating a good modeling session or designing MCP tool interactions that speak your domain’s language, reach out. I’d love to help.

Traceable Tests

Nico Krijnen — Fri, 27 Jan 2023 00:00:00 +0000

Originally published on Luminis on 27 Jan 2023.

Have you ever spent hours or days trying to figure out why some API test is failing? Whenever something like that happens to me, my immediate thought is: what could have helped me find this problem faster?

Unit tests are much better at this, when they fail, you just run them again locally, set a breakpoint if you have to, and find out what failed. But with tests that run against your API (yes, you need those too!) it can be much harder to figure out exactly what code got executed. There are all sorts of boundaries that make this hard, especially when those tests run as part of your pipeline (yes, you need to do that!). You can’t just set a breakpoint in your pipeline and even if you have logging in your backend, how do you know which log entries resulted from which test scenario?

I’ve been playing with the idea of linking backend logs to tests for some time. Years back, I had a plan to add the name of the test to a custom HTTP header when calling the API from tests. In the backend I could add that value to the MDC context that we attach to log entries. The result would be that for each log entry, you could see which test triggered it.

Déjà vu

Somehow I never got around to implementing that idea. Last year though, I helped out a team working on an API-only service that ran into the very same problem. Their system is comprised of a set of microservices and some of the API calls go through several of these services to handle the request. Next to unit tests, one of their other automated validations is a regression test suite that runs against the API to validate expected behavior.

So far, so good, and that test suite was really valuable. It allowed the team to quickly implement new functionality without spending a lot of time checking that existing functionality kept working.

You know what’s coming now… whenever a test would fail, it was a nightmare to figure out why.

Luckily, I had just finished hooking up all the services to an Elastic Cloud environment, with APM-powered tracing and all the good stuff. The traces gave excellent insight into what happened inside and between the microservices. However, it was still really hard to link a specific test failure to a set of traces.

OpenTelemetry FTW

I still had that idea from before in my head, and when I saw the team struggling with these failing tests, a new idea popped in: what if I linked the traces to tests?

The typical method to pass trace IDs across service calls is by using a trace header. OpenTelemetry provides the traceparent header for that, and the Elastic APM agent being used in the services picks that up out-of-the-box.

My first idea was to return the traceparent header in the API responses and then log them with each test. However, that’s not really how the OpenTelemetry traceparent header is supposed to be used. You’re supposed to include the header in the calling request. So that’s what I tried next.

Show me the code!

First thing I did was locate how the test suite was making the API calls. Turned out this was done with HttpClient, which offers a nice way to inject additional headers:

import static TraceInstrumentationInit.openTelemetry;
import static TraceInstrumentationInit.tracer;

public class HttpClientTraceInstrumentation implements HttpProcessor {
 @Override
 public void process(HttpRequest request, HttpContext context) {
 injectTraceHeaders(request);
 }

 @Override
 public void process(HttpResponse response, HttpContext context) {
 }

 private static void injectTraceHeaders(HttpRequest request) {
 // TODO
 }

 public static HttpClient registerTraceInstrumentation(
 DefaultHttpClient httpClient) {
 HttpClientTraceInstrumentation traceInstrumentation =
 new HttpClientTraceInstrumentation();
 httpClient.addRequestInterceptor(traceInstrumentation);
 httpClient.addResponseInterceptor(traceInstrumentation);
 return httpClient;
 }
}

Ok, so that gives us a way to inject the traceparent header. First I figured I could generate the trace ID myself. However, it turns out the traceparent value is not just a trivial UUID, it has a very specific structure and I wouldn’t advise trying to generate it yourself.

Luckily, OpenTelemetry has excellent SDKs that make it easy to generate and inject these headers. So we need a dependency on the OpenTelemetry API and SDK:

<dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>io.opentelemetry</groupId>
 <artifactId>opentelemetry-bom</artifactId>
 <version>1.18.0</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
</dependencyManagement>

<dependencies>
 <dependency>
 <groupId>io.opentelemetry</groupId>
 <artifactId>opentelemetry-api</artifactId>
 </dependency>
 <dependency>
 <groupId>io.opentelemetry</groupId>
 <artifactId>opentelemetry-sdk</artifactId>
 </dependency>
</dependencies>

Next, we had to initiate a trace for each scenario, and a sub-span for each API call being made.

After looking at the codebase, we saw we could add that to an already present base class used by all tests to report scenario results to TestRail. Personally I’m not a big fan of using base classes for tests as it can be hard to maintain a sensible hierarchy and the use of inheritance creates tight coupling between tests, which we don’t want. In JUnit for example, I would use @ExtendWith to keep such responsibilities separated and easier to re-use and to avoid the use of static instances as seen here. But we didn’t want to overhaul the complete test suite just to introduce this, better not to mix a big refactoring like that. The main goal at this point was to validate that traces could work.

As you see, we also passed the traceId to TestRail, more on that later.

public class Hooks extends BaseSteps {
 private static final TestRailReporter testRailReporter =
 TestRailReporterFactory.get();
 private static final ScenarioTraceInstrumentation instrumentation =
 new ScenarioTraceInstrumentation();

 @Before
 public void setup(Scenario scenario) {
 instrumentation.startTrace(scenario);
 }

 @After
 public void teardown(Scenario scenario) {
 String traceId = instrumentation.endTrace();
 testRailReporter.reportResultToTestRail(scenario, traceId);
 }
}

For every scenario we start a new span, including attributes to link it to the scenario. We also do some rudimentary logging for every scenario, making it easy to spot in the logs which scenario ran at which point and what the trace ID was for that scenario run.

import static TraceInstrumentationInit.tracer;

public class ScenarioTraceInstrumentation {
 private final ThreadLocalSpan spanState = new ThreadLocalSpan();

 public void startTrace(Scenario scenario) {
 Span span = spanState.start(
 tracer.spanBuilder(scenario.getName())
 .setSpanKind(SpanKind.CLIENT)
 );
 span.setAttribute("test.scenario_id", scenario.getId());

 System.out.println("|---------\n"
 + "| SCENARIO: " + scenario.getId() + " - " + scenario.getName() + "\n"
 + "| trace.id: " + span.getSpanContext().getTraceId());
 }

 public String endTrace() {
 String traceId = spanState.traceId();
 spanState.end();
 return traceId;
 }
}

To keep track of span state, I used a ThreadLocal, exposing a simple API to start and end the span that we could use from the ScenarioTraceInstrumentation and HttpClientTraceInstrumentation. As you see I did get lazy a bit here, using a Pair to store both the Span and Scope, using a record for that would make the code easier to read.

class ThreadLocalSpan {
 private final ThreadLocal<Pair<Span, Scope>> state = new ThreadLocal<>();

 Span start(SpanBuilder spanBuilder) {
 Span span = spanBuilder.startSpan();
 Scope scope = span.makeCurrent();
 state.set(Pair.of(span, scope));
 return span;
 }

 String traceId() {
 return state.get().getLeft().getSpanContext().getTraceId();
 }

 void end() {
 Pair<Span, Scope> pair = state.get();
 pair.getRight().close();
 pair.getLeft().end();
 }
}

The OpenTelemetry SDK also needs some initialization. Keeping it simple for this first experiment, and since some of the other code was static already, we just created two package scoped statics to be used from the ScenarioTraceInstrumentation and HttpClientTraceInstrumentation.

class TraceInstrumentationInit {
 static final OpenTelemetry openTelemetry = initOpenTelemetry();
 static final Tracer tracer = openTelemetry.getTracer("regressionTest");

 private static OpenTelemetry initOpenTelemetry() {
 SdkTracerProvider sdkTracerProvider = SdkTracerProvider
 .builder().build();
 OpenTelemetrySdk sdk = OpenTelemetrySdk.builder()
 .setTracerProvider(sdkTracerProvider)
 .setPropagators(ContextPropagators.create(
 W3CTraceContextPropagator.getInstance()
 ))
 .build();
 Runtime.getRuntime().addShutdownHook(
 new Thread(sdkTracerProvider::close)
 );
 return sdk;
 }
}

And finally, we could put all the pieces together in the HttpClientTraceInstrumentation, see below.

We create an additional span for the request [1] and then inject the trace header into the request [2].

The OpenTelemetry SDK code is very generic, allowing you to inject headers into pretty much anything, from HTTP requests to messages with all sorts of client libraries. The extremely generic code does make it a bit hard to read, but essentially you need to pass it a method that accepts two string arguments, the header name and header value. The HttpRequest from HttpClient has exactly such a method: setHeader [3].

Last but not least, we also log every request being made [4], giving a nice overview in the test logs of what API calls are being made for each scenario.

import static TraceInstrumentationInit.openTelemetry;
import static TraceInstrumentationInit.tracer;

public class HttpClientTraceInstrumentation implements HttpProcessor {
 private static final String W3C_TRACEPARENT_HEADER = "Traceparent";

 private static final TextMapPropagator textMapPropagator =
 openTelemetry.getPropagators().getTextMapPropagator();

 private static final TextMapSetter<HttpRequest> setter =
 HttpRequest::setHeader; // [3] http client setHeader method

 private final ThreadLocalSpan spanState = new ThreadLocalSpan();

 @Override
 public void process(HttpRequest request, HttpContext context) {
 Span span = spanState.start(
 tracer.spanBuilder("/")
 .setSpanKind(SpanKind.CLIENT)
 );
 try (Scope ignored = span.makeCurrent()) {
 // [1] add span details
 span.setAttribute(HTTP_METHOD, request.getRequestLine().getMethod());
 span.setAttribute(HTTP_URL, request.getRequestLine().getUri());
 span.setAttribute("component", "http");

 injectTraceHeader(request);
 logRequest(request);
 }
 }

 @Override
 public void process(HttpResponse response, HttpContext context) {
 spanState.end();
 }

 private static void injectTraceHeader(HttpRequest request) {
 // [2] let opentelemetry sdk propagate any required headers
 textMapPropagator.inject(Context.current(), request, setter);
 }

 private static void logRequest(HttpRequest request) {
 System.out.println( // [4]
 "|- " + request.getRequestLine() + "\n" +
 "| ^- " + request.getLastHeader(W3C_TRACEPARENT_HEADER));
 }
}

Everything coming together

Now when tests are run, their logs clearly show each of the scenarios, their trace ID and the API calls being invoked. And for every API call, the complete traceparent header is shown. With these details being propagated to our backend, we can now look up all the calls for a scenario in Elastic, or even search there for the span ID to locate the trace for a specific API call.

|---------
| SCENARIO: T2748705 - Validate if method PUT was added to controller and it's possible to use it
| trace.id: d28a64332015de5d324cb3e0f0380eba
|- PUT http://.../api/...
| ^- traceparent: ...
...
...
|---------
| SCENARIO: ...
| trace.id: ...
|- GET http://.../api/...
| ^- traceparent: ...
...
...

Integrate existing tools

At this client, the team used TestRail to centrally view reports of all test runs. To find the cause for a failing test, wouldn’t it be nice if we could skip sifting through logs? As you saw before, we had the trace ID available when we posted results to TestRail, so we added a link there to take you directly to the corresponding trace in Elastic.

Results!

When you follow that link to Elastic, it shows you all the API calls (transactions) that were part of that test scenario (trace).

From there you can navigate to each API call and look at individual traces. Elastic shows a lot of useful info, like a latency distribution chart that makes it super easy to quickly select all slow calls or all failing calls.

There is also a dedicated errors section, which allows you to quickly locate details for any errors that occurred as part of the scenario traces.

Tip: If you want to take a closer look at what’s in these kind of traces, there is a nice free Elastic demo environment where you can play around with traces in the APM section.

Conclusion

With just a little bit of code, we managed to reduce the amount of time needed to pinpoint test failures dramatically. Which shows again that optimizing for quick work loops and short feedback cycles are crucial for improving software delivery performance.

Move Faster with New Feature Flag Capabilities in AWS

Nico Krijnen — Mon, 13 Dec 2021 00:00:00 +0000

Originally published on Luminis on 13 Dec 2021.

I always get excited about things that allow you to bring new ideas to production faster. And it seems we are in luck! During the yearly AWS re:Invent conference that took place in Las Vegas, AWS launched several services that help you do just that!

Business around us is changing more and more rapidly. Year by year, the State of Devops research confirms that high performing companies excel in adapting to those changes by optimizing their ability to experiment, deliver quickly and gather feedback directly. You need to try many ideas and quickly find out which are successful and which are not. That ability to make many small changes reduces risk and increases the ability to innovate.

The new AWS services around feature flags enable organizations to achieve these goals. I will explain these briefly and show you how you can leverage them.

Moving fast

So what ingredients do you need to go fast? First, you have to optimize your organization and teams so they can come up with diverse ideas. Make sure you enable them to try those out and get feedback quickly without having to go through committees to get approval.

Second, on the engineering side, you need to learn to do everything in small steps. Integrate all your changes with the rest of your team often, at least daily. This way of working is called trunk-based development. The key here is never letting go of high quality and keeping your software in a constant state of being able to deploy it to production. To achieve that, you need a (continuous) delivery pipeline that automates all the steps needed to get from commit to deploy. As excellently explained by Dave Farley, the key responsibility of that pipeline is not to ensure that the software is 100% correct – that is not realistic. Instead, the pipeline needs to do all that it can to figure out if something might be broken or misbehaving. That means you’ll need at least a solid automated test suite that provides you with confidence that on the functional side everything works. Next to that, you can add validations to your pipeline for things like performance, code quality, and any other aspects that are important for your application.

If you do everything in small steps, does that mean you cannot have big innovations? No, of course it doesn’t. Big innovations never happen in one go. They are the combination of smaller increments, even the failed ones that you learned from. Like Edison’s light bulb, which was the result of thousands of failed experiments.

Feedback, as early as possible

A common trick is to separate the deployment of changes from the release of a feature. By using feature flags, you can include the new code you are writing without the feature showing up or being active. With all the code being deployed continuously, the act of “releasing” a feature simply becomes toggling the feature flag.

This way of working helps greatly in catching mistakes or wrong assumptions in your ideas early, while it is still easy to try another approach. It is much better to discover that your idea does not work after a few hours than finding out weeks later, when you’ve already invested a lot of time. You also make it much easier to experiment with new features much earlier in their development, maybe just exposing them to a few interested beta users and receive feedback before you make the feature generally available.

Enter feature flags

I always like keeping things simple. The easiest way to implement feature flags is to just add some booleans and ‘if statements’ in your code and allow these to be toggled between the different environments you have. For example, showing the new feature you are working on in your staging environment, but hiding and deactivating it in production. To be honest, in many cases, a simple mechanism like this is more than sufficient.

Do realize there is a tradeoff when using feature flags. Having these branches in your code means you add complexity and you make your application less predictable. In many cases the benefits greatly outweigh the downside, as long as you keep it under control. Don’t let hundreds of feature flags linger around. Be responsible, be a good craftsman and keep your codebase clean. Once a feature is released, clean up all traces of the flag. That way you minimize complexity and keep your codebase lean and mean and easy to change. After all, you want to keep doing more experiments and keep delivering value in a sustainable way.

Depending on how long your build pipeline takes, having to change booleans in code may mean that it takes a while to toggle a feature in production. This may be ok in some cases, but when it turns out that a feature doesn’t play nice on production yet, you want to toggle it off again quickly. What if you could toggle it in real-time and let someone else handle part of the complexity around feature flags?

Launch 1: AWS AppConfig – Feature Flags

AWS AppConfig has been around since 2019 and allows you to roll out configuration changes to your applications in real-time, without having to deploy code or take your application out of service. It can validate and then roll out configuration changes gradually while monitoring your application and rolling back changes in case of errors, minimizing impact to users.

Recently AWS launched AWS AppConfig Feature Flags, which builds on AppConfig and makes it easier to roll out new configurations of your feature toggles. You can now setup and toggle feature flags directly from the AWS Console. Versions of your feature flags configuration are tracked and as with config changes, you can manage and monitor the gradual deployment to your environments.

As feature flags are just booleans, there is no validation around them. You can also add custom attributes to flags. An example of what I would use these for is creating flags that are only enabled for specific beta tenants. For these attributes you can set up some simple validation rules in the UI. That may not be as powerful as writing a validation lambda as you can do for normal AppConfig validation, but it sure is a lot simpler and probably sufficient. And don’t forget, less handcrafted code means you reduce risk.

Apart from feature flags, you can also use this feature to toggle operational aspects of your application quickly. For example, in case of an incident you can temporarily switch off non-critical functionality or behavior to reduce load and keep the essentials of your service working, allowing your team to focus on resolving the incident.

That all sounds fantastic, but it does mean that releases of features are no longer tracked in your central git history, instead they are changed through the AWS Console. Luckily, the Console does track versions of your configurations and shows you the history of feature flags and configuration deployments that have been done to each of your environments. So, it may not be in one place anymore, but you can still audit when features went live or were rolled back. Just be aware that you are making this tradeoff.

Launch 2: AWS CloudWatch – Evidently

Where AppConfig Feature Flags is geared towards simplifying the technicality of managing feature flags and their deployment, Evidently is all about connecting the data you collect for experiments and A/B tests around feature roll outs that you do with smaller sets of users before making the feature generally available.

Evidently allows you to set up multiple variants of a feature and test these variants on different subsets of users at the same time. Once you have set up and are running such an experiment, Evidently also helps in statistically analyzing which variant is performing better, based on data collected during the experiment, giving you confidence in making the right choices.

In today’s fast paced world, you have to continuously reinvent your business to remain relevant. To do so you have to innovate and Evidently really helps you with tools to apply that mindset of running experiments and basing your decisions on the data that you gather. In today’s fast paced world, you have to continuously reinvent your business to remain relevant. To do so you have to innovate and Evidently really helps you with tools to apply that mindset of running experiments and basing your decisions on the data that you gather.

Launch 3: Amazon CloudWatch RUM (Real User Monitoring)

Now obviously, to make a service like Evidently work well, you need to collect the right data. Data that helps you prove the effectiveness and desired behavior of the features you build. To help with this, AWS launched RUM, another service under the CloudWatch umbrella.

CloudWatch is all about monitoring, but so far that was mostly oriented towards raw infrastructure and backend metrics of apps running inside AWS. RUM enhances that with capabilities to monitor web application performance, aimed at user experience. You inject some JavaScript into your web application frontend which then allows you to anonymously collect page load and layout performance metrics, JavaScript errors and client-side experienced HTTP errors.

Where traditional CloudWatch metrics are somewhat raw, RUM feels more like Google Analytics, giving you out-of-the-box dashboards that provide visibility into your application’s reliability, performance, and end user satisfaction. The service performs analysis on the collected data and breaks it down so you can gain insights about which areas of your application’s user experience can be improved.

On the observability side, CloudWatch RUM integrates with AWS X-Ray, so you can view traces and segments for end user requests. You can even see your clients in CloudWatch ServiceLens service maps.

Pricing

As usual with AWS, pricing for these services are based on usage, which means you can try these out on a small scale and then evaluate if the benefits they bring you are worth the cost. Once you use these services in production, always put up some cost monitoring so you remain aware of what you are spending. Below you will find an overview of the pricing for these services. For details, look at their pricing page.

AWS AppConfig Feature Flags are charged by the number of times you request and receive config changes:

$0.0000002 per configuration request via API Calls
$0.0008 per configuration received

AWS CloudWatch Evidently has a first time free trial that includes 3 million Evidently events and 10 million Evidently analysis units per account. After that you pay:

$5 per 1 million events (user actions & assignment events)
$7.50 per 1 million analysis units

AWS CloudWatch RUM allows you to control costs by configuring it to only analyze a limited percentage of user sessions. There is a first time free trial that includes 1 million RUM events per account. After that you pay:

$1 for every 100,000 events collected (page view, JavaScript error, HTTP error)

Alternatives

Compared to products that specialize in this field like LaunchDarkly or Optimizely, these AWS services may feel somewhat bare bones. This is no coincidence, as Werner Vogels said: AWS services are designed to be building blocks, not frameworks.

AWS may not be as feature rich as some of these others but having it all integrated into the rest of your service stack can be a big benefit. The low barrier to entry means you can start using it today and get value out of it quickly, or switch to something else later if you discover that your needs have grown.

And don’t forget, if you don’t need all the experimentation statistics or real-time flag controls, just keep it simple and add a few booleans in your code.

Conclusion

It is great to see that AWS is serious about giving their users the keys they need for innovation. What I personally find most exciting is that with this combination of services, AWS makes it easier for all of us to adopt that mindset of doing experiment-based and data-driven roll outs.

If you need help to introduce that mindset of experimentation, don’t hesitate to reach out!

Learn more

If you want to learn more about these AWS services, check out the official blogs: