DEV Community: Benoit Averty

Java Streams: The Gatherers API

Benoit Averty — Mon, 31 Mar 2025 07:00:00 +0000

Java 24 introduces a new feature related to streams: the Gatherer interface. This interface (used with the Stream.gather method) allows for more complete and expressive transformation of Java streams than was previously possible with existing methods. Explanations and examples in this article.

Read this article in french on my website

Overview
- What are Streams?
- What exactly was missing?
The Gatherer Interface
- Definition
- Usage and utility methods
Examples
- Compute Bitcoin Price All Time Highs
- Calculating Median Item Prices in Parallel
Conclusion

Overview

What are Streams?

What we commonly refer to as Streams in Java is a set of interfaces, methods, and classes that allow for modeling data transformation pipelines.

At the heart of this set is, of course, the Stream interface, which represents a sequence of data that may be empty or infinite.

This interface includes a number of methods that model successive operations on this data sequence. These operations fall into two types:

Some operations transform the Stream into another Stream. These are intermediate operations, such as map, filter, or limit.
Some operations transform the Stream into a value, a collection, or another type of container. These are terminal operations, such as collect, findFirst, or reduce.

Streams are not another type of collection. The idea is to model a chain of transformations, not a set of values. Streams are lazy: as long as no terminal operation is used, no operation is actually performed, as there is no expected result. This enables modeling infinite sequences without causing your program to run indefinitely.

import java.util.stream.Stream;

void main() {
    // An infinite Stream representing natural numbers.
    var naturalNumbersStream = Stream.iterate(1, i -> i + 1); 

    // Here we model the sequence of operations on our stream: 
    // square the numbers, keep only the even ones, then take the first 100.
    var streamPipeline = naturalNumbersStream 
            .map(i -> i * i)
            .filter(i -> i % 2 == 0)
            .limit(100);

    // At this stage, no computation has taken place yet.
    // We collect the stream into a final value: the average of the numbers.
    int averageOfFirst100EvenSquares = streamPipeline.collect(Collectors.averagingInt(i -> i));

    System.out.println(averageOfFirst100EvenSquares);
}

The other benefit of thinking in terms of operation pipelines rather than collections is access to bonus features such as parallelism (under certain conditions, operations can be performed in parallel on segments of the stream) or other optimizations (the terminal operation count, for instance, can sometimes skip all intermediate operations if it can compute the number of elements directly from the source).

What exactly was missing?

When it comes to terminal operations, the Stream API is relatively complete. We have a number of simple operations for common use cases (findFirst, toArray, forEach, etc.) and a Collector interface that can be implemented to define any type of terminal operation.

/**
 * This class transforms a `Stream<Customer>` into an instance of `Invoice`. It 
 * processes the stream element by element and can use an internal state of type 
 * `Map<UUID, Customer>`. Beyond that, the logic is yours—implement whatever terminal 
 * operation you need.
 */
class MyCollector implements Collector<Customer, Map<UUID, Customer>, Invoice> {
    // ...
}

In contrast, intermediate operations were not as robust. Acting on items one by one is fairly flexible (map, filter, etc.), but there are few stateful operations, which consider multiple items from the input stream to produce one or more output elements. For example, distinct or dropWhile serve specific needs, but there was no way to implement fully custom operations, such as transforming multiple consecutive or non-consecutive elements into a single one (or none at all).

A simple example would be transforming a stream of random numbers into a stream of strictly increasing numbers. filter does not allow consideration of anything but the current item when deciding whether or not to keep it. You would need to convert the stream to a list, apply the operation on the list, then reconvert to a stream, which undermines many of the stream's advantages (laziness, parallelism handling, etc.)

This is exactly the kind of problem that Stream Gatherers solve. Any custom intermediate operation can be defined, just as Collectors allow custom terminal operations.

The `Gatherer` Interface

Definition

Here is a (simplified) definition of this interface:

public interface Gatherer<Input, State, Result> {
    Supplier<State> initializer();

    Integrator<State, Input, Result> integrator();

    BinaryOperator<State> combiner();

    BiConsumer<State, Downstream<? super Result>> finisher();
}

Just like the Collector interface, a gatherer works through four methods that are called at different points during the gathering operation to produce the final result.

The initializer is used at the beginning of the operation to create the gatherer's internal state. This state enables the operation to consume multiple input elements before emitting an output element. This initializer may be called multiple times in case of parallel execution (each thread will have its stream chunk and its own internal state).
The integrator is the core of the gatherer. It receives each input stream element and can emit elements into the output stream (via an interface called Downstream). It also has access to the gatherer's internal state.
The combiner allows the operation to run in parallel. In this case, a state instance will be initialized for each thread, and the combiner will be called to merge these states into one after the operation.
The finisher allows final actions at the end of the operation. It is called once all the elements have been integrated and can optionally emit more elements via the Downstream interface. It also has access to the final combined state.

Usage and utility methods

A gatherer is used with the Stream.gather method.

var resultStream = myList.stream()
        .gather(new MyGatherer());

Defining a gatherer by creating a new class implementing Gatherer can be verbose, especially for simple cases. Therefore, utility methods exist to create a Gatherer instance directly from provided lambdas. For example:

Create a gatherer from the 4 methods:

  Gatherer.of(
        Supplier<State> initializer, 
        Integrator<State, Input, Result> integrator,
        BinaryOperator<State> combiner,
        BiConsumer<State, Downstream<? super Result>> finisher
  );

Create a sequential gatherer (one state, no combiner)

  Gatherer.ofSequential(
        Supplier<State> initializer, 
        Integrator<State, Input, Result> integrator
  );

Create a stateless and parallelizable gatherer

  Gatherer.of(
        Integrator<State, Input, Result> integrator
  );

It's not always easy to understand how the four methods of a gatherer interact. Practicing a few examples is the best way to learn.

Examples

Compute Bitcoin Price All Time Highs

We have a stream of monthly Bitcoin prices in chronological order. We want a stream that emits an item each time the price hits a new all-time high.

Without gatherers, it would have been necessary to convert the stream into a list. For a stream from a large file, for example, that's unfortunate, since it forces us to read the entire file before performing the operation (even if we only want the first 10 highs...).

The gatherer-based solution allows us to handle All-Time Highs in a streaming fashion.

record BitcoinPrice(YearMonth month, double price) {}

/**
 * Our gatherer transforms a `Stream<BitcoinPrice>` into a `Stream<BitcoinPrice>`, which
 * matches the first and third types.
 * The internal state is the highest price seen so far. Since the state must be mutable,
 * we use an array (a common technique with Collectors as well). If you prefer, you may use
 * `AtomicDouble` or `MutableObject` if you have the Apache Commons dependency.
 */
class IncreasingPrices implements Gatherer<BitcoinPrice, Double[], BitcoinPrice> {
    public Supplier<Double[]> initializer() {
        // The initial state is Double.MIN_VALUE so the first stream value is always retained.
        return () -> new Double[] { Double.MIN_VALUE };
    }

    public Integrator<Double[], BitcoinPrice, BitcoinPrice> integrator() {
        return (state, element, downstream) -> {
            // If the current price is higher than the last recorded price, emit the value
            // and update the highest price.
            if(element.price > state[0]) {
                downstream.push(element);
                state[0] = element.price;
            }
            // Returning false from the integrator stops processing.
            // Here, we want to process all elements.
            return true;
        };
    }
}

// our gatherer depends on input data being sorted by date.
Stream<BitcoinPrice> historicalData = getBitcoinSortedHistoricalData(); 

historicalData
        .gather(new IncreasingPrices())
        .forEach(System.out::println);

The full example is available on my GitLab.

Calculating Median Item Prices in Parallel

We have data on the prices of consumer goods in various stores and want to compute the average price for each item. In this case, we can't benefit much from stream laziness (we need all data to compute the correct average), but we can take advantage of parallel processing.

You could already do this with a Collector, but if you want a Stream output, you'd need to recreate a stream from the collector's result. We're going to do the work with a Gatherer instead.

/** Input data **/
record ItemPrice(String item, double priceInEuros, String shop) {}
/** Output data **/
record ItemMeanPrice(String item, double meanPrice) {}

class MeanPriceGatherer implements 
        Gatherer<ItemPrice, Map<String, MeanPriceGatherer.CumulativeSumCount>, ItemMeanPrice> {

    /** Our internal state stores, for each item, the sum and count of prices seen. */
    public Supplier<Map<String, CumulativeSumCount>> initializer() {
      return HashMap::new;
    }

    /**
     * For each item, add the price to the cumulative state and increase the count by 1.
     *
     * Here, we don't emit items during integration because we need all inputs first—
     * that's the finisher's job.
     */
    public Integrator<Map<String, CumulativeSumCount>, ItemPrice, ItemMeanPrice> integrator() {
      return ((state, element, downstream) -> {
        state.merge(
                element.item, 
                new CumulativeSumCount(element.priceInEuros, 1),
                CumulativeSumCount::add
        );
        return true;
      });
    }

    /**
     * To support parallelism, we must be able to combine states.
     * Here, for each item in one state map, we merge it with occurrences in the other.
     */
    public BinaryOperator<Map<String, CumulativeSumCount>> combiner() {
        return (state1, state2) -> {
            state2.forEach((key, value) ->
                    state1.merge(key, value, CumulativeSumCount::add)
            );
            return state1;
        };
    }

  /**
   * The finisher runs after all parallel states have been merged.
   * It iterates over the state, computes the mean, and emits results downstream.
   */
  public BiConsumer<Map<String, CumulativeSumCount>, Downstream<? super ItemMeanPrice>> finisher() {
    return (state, downstream) ->
            state.forEach(
                    (itemName, sumCount) ->
                            downstream.push(new ItemMeanPrice(itemName, sumCount.getMean()))
            );
  }

  record CumulativeSumCount(double sum, int count) {
    public CumulativeSumCount add(CumulativeSumCount cumulativeSumCount) {
      return new CumulativeSumCount(
              sum + cumulativeSumCount.sum, 
              count + cumulativeSumCount.count
      );
    }

    public double getMean() {
      return sum / count;
    }
  }
}

Again, the full demo is available on GitLab

Conclusion

The Gatherer interface is not revolutionary, but it fills a real gap in Java streams. Developers who use streams extensively have likely had to convert a stream to a collection just to perform an operation and convert it back immediately. Gatherers address that need while improving code readability and occasionally even performance.

For more detail, it's always useful to check out the Javadoc, and JEP 485, which introduces this feature.

An advanced Data Table with HTMX

Benoit Averty — Fri, 28 Feb 2025 08:15:00 +0000

A few days ago, I completed and published a demonstration of a DataTable component built with HTMX and AlpineJS. This article describes the approach I implemented and explains some patterns used for the development of this demo.

Read this article in french on my website

The Project
- Technical Stack
- What This Article Is Not
The Demo
- General Structure
- Deleting an Element
- Pagination / Sorting / Filtering
- Total Element Count / Event Synchronization
Experience Feedback
- Web Fundamentals
- Disadvantages

The Project

The goal of this project is to evaluate the relevance of HTMX in various use cases, from content-oriented sites to complex management applications. I’m trying to prove (to myself and others) that HTMX and the hypermedia application approach is not limited to simple applications or basic user experiences.

To do this, I’ve initiated a project aimed at demonstrating various patterns frequently encountered in modern applications, developing them with HTMX and the ecosystem I refer to as "hypermedia application".

You can interact with the demo at this address: https://link.benoitaverty.com/data-table-htmx

Technical Stack

HTMX: The library that makes this approach possible. HTMX allows the development of applications with HTML and HTTP, but by lifting a whole set of limitations, particularly by allowing interaction with the server without necessarily reloading the entire page.
AlpineJS: An enhanced user experience still requires some degree of client-side interactivity. AlpineJS provides this interactivity while remaining within the hypermedia application paradigm. The interactivity is still coded directly in the HTML markup, without writing JS (or almost none). Where HTMX focuses on client/server interactions, AlpineJS allows adding some client-side behavior.
Kotlin / Ktor: Ktor is a Kotlin framework that allows developing HTTP APIs. It’s somewhat equivalent to Express or Fastify in Kotlin. One of the advantages of the Hypermedia approach is that it allows development in a language other than Javascript, and I took this opportunity for these demos.

Together, these three components form the equivalent of the AHA stack (but with Ktor instead of Astro), and the site explains very well the approach used for this project.

It should also be noted that the use of AlpineJS remains moderate: the interactivity provided by Alpine is, in my opinion, less maintainable than with a frontend framework like VueJS or React. Alpine is interesting because it is extremely complementary to HTMX, but I use it sparingly, and only to improve a UX defect that would be too significant with pure HTMX.

What This Article Is Not

This article is not an argument for or against adopting HTMX and the hypermedia approach. I aim to provide material to help make this choice by showing concretely what is possible, and by dismantling certain preconceived ideas.

This article does not show the server-side code used to generate the HTML "enhanced" by HTMX. I will show the final HTML/HTMX code, and you could generate it with any server technology to obtain the same user experience.

The Demo

This demo shows a DataGrid such as can be found in most management applications. It allows interaction with a collection of objects stored in a database or other source. In this case, it’s a list of objects that could be found in an inventory, with quantity in stock, supplier, category, etc.

The list allows you to:

Sort, filter, and access data pages
Delete elements
Create a new element

The result obtained by manipulating these different criteria has a URL that can be shared to find the list in the same state.

All functionalities work if JavaScript is disabled or if one of the scripts (htmx or alpine) fails during downloading or execution.

All features are accessible by keyboard.

General Structure

External resources are limited to HTMX (and a plugin), AlpineJS (and two plugins), and CSS. In total, less than 50kb (minified/compressed).

<html>
  <head>
    <link href="/app.css" rel="stylesheet" type="text/css">
    <link href="/pico.min.css" rel="stylesheet" type="text/css">
  </head>
  <body>
    <!-- The document -->

    <script src="/htmx.js"></script>
    <script src="/alpinemorph.min.js"></script>
    <script src="/alpinefocus.min.js"></script>
    <script src="/alpinejs.min.js"></script>
    <script src="/htmx-ext-alpine-morph.js"></script>
  </body>
</html>

The main element of the page is of course an HTML table with its headers:

<table class="data-table">
    <thead>
    <tr>
        <th>
            <button type="submit" class="link-btn" form="inventory-controller" name="nextSort" value="name" role="link"
                    key="sort-by-name">Name↓↑
            </button>
        </th>
        <th>Description</th>
        <th>
            <button type="submit" class="link-btn" form="inventory-controller" name="nextSort" value="quantity"
                    role="link" key="sort-by-quantity">Qty.↓↑
            </button>
        </th>
        <th>Actions</th>
    </tr>
    </thead>
    <tbody>
    <!-- ... -->
    <tr class="data-row" key="21b8915b-5667-4cb8-bfa3-aef5187eccab">
        <td class="data-cell">
            <div>
                <a href="/data-table/21b8915b-5667-4cb8-bfa3-aef5187eccab"
                    hx-get="/data-table/21b8915b-5667-4cb8-bfa3-aef5187eccab" hx-target=".item-details-placeholder"
                    hx-swap="innerHTML" hx-push-url="true">Wireless Mouse</a>
            </div>
        </td>
        <td class="data-cell">
            <div>A sleek and ergonomic wireless mouse</div>
        </td>
        <td class="data-cell">
            <div>25</div>
        </td>
        <td class="data-cell">
            <div>
                <form action="/data-table/21b8915b-5667-4cb8-bfa3-aef5187eccab/delete" method="post"
                      hx-delete="/data-table/21b8915b-5667-4cb8-bfa3-aef5187eccab" hx-target="closest tr"
                      hx-swap="outerHTML swap:300ms">
                    <button type="submit" class="danger delete-btn" aria-label="Delete">✘</button>
                </form>
            </div>
        </td>
    </tr>
    <!-- Etc. -->
    </tbody>
</table>

This table displays the data in a completely classic way, but already contains a number of HTMX attributes and HTML elements that deserve explanation.

Deleting an Element

The last column of the table contains the following button form:

<form
        method="post" action="/data-table/21b8915b-5667-4cb8-bfa3-aef5187eccab/delete"
        hx-delete="/data-table/21b8915b-5667-4cb8-bfa3-aef5187eccab"
        hx-target="closest tr"
        hx-swap="outerHTML swap:300ms"
>
        <button type="submit" class="danger delete-btn" aria-label="Delete">✘</button>
</form>

The three hx-* attributes are sufficient to code the deletion behavior.

hx-delete: when the form is submitted, a DELETE request is sent to the server at the specified URL. The server handles deleting the item in the database and returns an empty response with status code 200.
hx-target="closest tr": the first tr element in the ancestors of the form element will be replaced at the end of the request
hx-swap: outerHTML specifies that the entire tr element is replaced (by default, it would be only its content), and swap:300ms means that HTMX will apply an htmx-swapping class to the element for 300 milliseconds before replacement.

Since the server response is empty, the table row is simply deleted, which is sufficient for our deletion feature.

Why `swap:300ms`?

Modern UX relies heavily on animations. In addition to offering a more pleasant look & feel, they provide a form of feedback and allow users to better observe the result of their action.

The HTMX mechanism activated by swap:300ms allows adding an animation to our deletion using CSS transitions¹.

.data-cell > div {
    /* ... */
    transition: all 300ms ease;
    height: 4.5rem;
    overflow: hidden;
}

tr.htmx-swapping .data-cell > div {
    height: 0;
    padding-bottom: 0;
    padding-top: 0;
    overflow: hidden;
}

Why use a form and a button of type `submit`?

HTMX allows making requests from any element. So we could have done without the form and used a simple button with the same three hx-* attributes.

Using the form allows the application to function even if JavaScript is disabled or if the htmx.js script fails to execute for some reason. If HTMX is not loaded, the browser will handle making an HTTP request itself thanks to the method and action attributes. In this case, the server will also perform the deletion and redirect the user to the URL where they were. There will be no animation and the entire page will be reloaded, but the deletion works perfectly. This is an example of progressive enhancement made relatively simple to code thanks to the hypermedia application paradigm.

Pagination / Sorting / Filtering

These three functionalities are similar, as they allow controlling which elements are displayed in the table. Additionally, they must work together (changing pages should not reset the filter, for example). Finally, these functionalities will change the URL of the current page to allow the user to bookmark or share any state of the table.

Server-side, it’s relatively simple: a request to the URL /data-table accepts query params that control the displayed elements (sort, page, pageSize, search). Additionally, the server detects if a request was made by HTMX using the HX-Request header and only returns the table (not the entire document) if a request is made by HTMX.

Client-side, we need to use attributes that will trigger a GET request with these parameters. There is a native HTML element that allows storing a state and triggering requests from this state, which is the form. So we will trigger our requests from a form.

<form 
        action="" method="get"
        id="inventory-controller" 
        hx-trigger="submit, input from:[form=inventory-controller] delay:100ms"
        hx-get="/data-table"
        hx-target="#inventory" 
        hx-push-url="true">
</form>

hx-trigger allows more detailed specification of events that trigger a request. By default, forms trigger a request on the submit event. Here, we also trigger the request if an input element belonging to this form changes. This is what allows automatically reacting when a text query is typed or when changing pages. delay:100ms helps avoid sending too many requests when a user quickly types text in the search field.
hx-get="/data-table" specifies the verb and URL of the request made by HTMX
hx-target: When the server receives a request at this URL, it returns the HTML table corresponding to the criteria. HTMX will insert it into the page in place of the element corresponding to the selector #inventory
hx-push-url allows specifying that the page URL should be updated with the request URL. Here, it allows adding the query params corresponding to the form fields to the page URL. This way, the user can refresh their page or share it without losing the state of the table.

At this point, I haven’t yet mentioned the form fields. Form fields (elements input, button, select, etc.) don’t necessarily need to be descendants of the form element. The fields are therefore at the appropriate place in the rest of the document, linked to the form through their form attribute

<!-- Page change -->
<select form="inventory-controller" name="page">
    <option value="1">1</option>
    <option value="2" selected="selected">2</option>
    <option value="3">3</option>
    <option value="4">4</option>
    <option value="5">5</option>
</select>

This is a completely classic select. I’ll skip over the page size change and the filter field, as they are nothing special either. HTMX already triggers a request when one of these fields is modified thanks to the value input from:[form=inventory-controller] delay:100ms of the form's hx-trigger attribute.

Once again, the use of purely HTML forms and fields allows the application to function correctly. The only thing to keep in mind is that the form will not auto-submit on change if HTMX is not present. Therefore, a submission button must be included that the user can click if JavaScript is not available for some reason.

<button
        type="submit"
        form="inventory-controller" 
        x-show="window.htmx == undefined"
>OK</button>

The button is not necessary if HTMX is loaded, so we use AlpineJS to hide it with x-show="window.htmx == undefined".
To see the result, you can disable JavaScript in your browser or block the htmx or alpineJS script. In any case, you will see two submission buttons appear next to the pagination controls and the search field.

Sorting

Sorting is slightly more subtle. The user activates sorting by clicking on the column header. The simplest for this kind of interaction is to use a link:

<a 
        href="?sort=-quantity&amp;search=&amp;page=2&amp;pageSize=10" 

        hx-get="?sort=-name&amp;search=&amp;page=2&amp;pageSize=10"
        hx-target="#inventory" 
        hx-push-url="true">
    Qty.&NonBreakingSpace;↓
</a>

The href attribute allows the link to function without JavaScript, and the hx-* attributes are there for the enhanced UX with HTMX. We include the sort parameter for sorting, and all other parameters to not lose their state.

The problem with the link is that it contains no state. Consequently, if the user changes the page for example, the request sent by the form will not contain the sort parameter and the current sort will be lost.

To remedy this, we include a hidden field in the page that contains the current state of the sort:

<input type="hidden" name="sort" form="inventory-controller" value="quantity">

This way, every time the form that controls the table sends a request, the current sort will be sent as well. And it works without JavaScript.

Total Element Count / Event Synchronization

When an element is deleted or created, the total number of elements under the title is updated. However, this information is not part of the response body for creation and deletion requests.

For this kind of use case, HTMX allows an HTTP response to trigger an event that updates other parts of the page.

Here is the complete HTTP response to an element deletion request:

HTTP/1.1 200 OK
content-length: 0
hx-trigger: x-business:item-deleted

The hx-trigger header informs htmx that an event must be raised by the element that made the request. In this case, an event of type x-business:item-deleted. This is a classic HTML event that will propagate to the body of the document.

Elsewhere in the page, the total number of elements is implemented with this HTML code:

<hgroup 
    hx-trigger="x-business:item-created from:body, x-business:item-deleted from:body"
    hx-get="/data-table/heading"
>
    <h1>Inventory</h1>
    <p>(42 items in stock)</p>
  </hgroup>

The hx-trigger attribute triggers a request when an x-business:item-deleted or x-business:item-created event is detected on the body, and retrieves the new total from the endpoint /data-table/heading.

This is an example of how different parts of the same page can be synchronized without introducing coupling between them thanks to the event system.

Experience Feedback

In this article, I’ve presented some patterns and techniques used to develop applications with HTMX. The most important conclusion I can draw is that web fundamentals are fundamental.

Web Fundamentals

You need to rely heavily on links, forms, URLs, sessions, etc. Since the paradigm is new, a developer who has only known development with React or VueJS may have some difficulty adapting.

Nevertheless, this way of developing applications is infinitely simpler than today’s industry state of the art. There is less tooling, fewer abstraction layers, and the result is more resilient and simpler to evolve.

Disadvantages

There are concessions to make on features. For example, it seems difficult to me to develop a rich text editor solely with AlpineJS. However, these concessions seem minor compared to the gain in simplicity. Also remember that it is very easy to include a piece of React (for example) in the middle of a hypermedia application.

The main disadvantage in my opinion is the lack of an ecosystem around this approach. Although many people have understood the interest of HTMX, this approach remains marginal in today’s web. Consequently, you need to reinvent a bit more of your own tooling and technical foundation. We can hope that this limitation will disappear by itself with time.

That's also why all table cells contain a div, because it’s impossible to reduce the size of a cell below the size of its content. So we reduce the size of the div in the cell to 0... ↩

Your API Doesn’t Always Need to Be a Product

Benoit Averty — Tue, 14 Jan 2025 08:00:00 +0000

HTTP/JSON APIs are ubiquitous in our web applications. Yes, the web involves having a user's browser communicate with the server managing the data, but that doesn't mean we need to treat every API as a standalone product. Sometimes, our APIs might deserve to be a little more discreet in a web application’s architecture.

Read this article in french on my website

A Brief History of the Web
- In the Beginning, Documents
- jQuery, XHR, and the Rejection of Page Navigation
- Single Page Applications: A Browser Within the Browser
APIs: Their Strengths and Limitations
You Don’t Always Need This
- Go Back to Documents (Without Sacrificing Interactivity)
- API as an Implementation Detail
- The Back for Front Pattern
Conclusion

A Brief History of the Web

In the Beginning, Documents

The first web pages looked nothing like what we know today. Take, for example, this webpage from the early 1990s: a few documents without images, without styles, connected by hyperlinks (I’ll let you judge how readable it is compared to modern websites). The web was primarily a network for accessing information in the form of documents organized into a network (web).

The HTTP protocol dates back to this era and is perfectly optimized for transporting hypermedia documents (hence the HT in HTTP), with a bit of interaction based on forms.

jQuery, XHR, and the Rejection of Page Navigation

The usage of the web exploded. Websites became more numerous, richer in content, and notably more interactive. Documents became less static to allow for an increasingly personalized user experience.

Consequently, the web slowed down (pages needed to be generated dynamically, and networks weren’t as fast yet in every household), and some websites began to look more like applications rather than interconnected documents.

This is why developers started to heavily use XHR requests to make websites more interactive¹. Reloading an entire page just to change a small portion of it doesn’t make for a great user experience when loading times are significant.

The browser took a specific turn: rather than increasing the capabilities of the HTML language and the HTTP protocol, JavaScript allowed developers to code specific interactions for each website in the browser itself. Developers could modify parts of a page using HTTP requests directly in JavaScript rather than relying on HTML’s (limited) tools such as links and forms.

Single Page Applications: A Browser Within the Browser

The web development industry dove headlong into this path, resulting in what we know today. The contents of your browser’s viewport are entirely managed by JS code. HTTP requests are sent by JavaScript, and responses are processed by JavaScript. History, caching, routing, and scrolling management are all handled via JavaScript. Essentially, we’ve ended up with a browser within the browser.

To achieve this, JavaScript applications have turned into heavy clients, specific to each application. They retrieve data from APIs, which turned into the HTTP/JSON APIs we’re so familiar with today because they only have to manage data.

APIs: Their Strengths and Limitations

This architecture, with a dedicated frontend and an API solely managing data, has created several opportunities:

Backend and frontend can be managed by different teams.
Multiple frontends can be created for the same API.
A frontend can be developed internally while also serving as a standalone product of your company.

This is the main advantage of a data-oriented API. It enables various features for diverse users. However, you only benefit from these advantages by adhering to all best practices of product management, including:

Backwards compatibility.
Documentation.
Prioritizing and addressing new feature requests from multiple users.
Etc.

You Don’t Always Need This

The problem with the web industry today is that we adopt this architecture far too readily without asking, “Do I need these benefits? Is the cost worth it?”

There are plenty of situations where we could simplify things. If your backend has only one consumer, and that consumer is under your control (such as when a team is building a full-stack product), you can skip backwards compatibility for changes. You can forego user documentation for your API in favor of global documentation for the application. You might even replace JSON with binary formats like Protobuf.

Go Back to Documents (Without Sacrificing Interactivity)

Modern frameworks are over-equipped to interact with JSON data APIs. However, it’s perfectly possible to build an application based on HTML document exchange. Many applications are still built with Ruby on Rails, for instance, leveraging HTML forms, links, and server-side rendering.

The Single-Page Application trend gained popularity partly because it allowed developers to avoid full page reloads during navigation or form submissions. HTMX has been generating buzz recently precisely because it enables bypassing full document reloads. At the same time, it sticks to a document-based approach for browser/server interactions. This eliminates the need to retrofit JSON API constraints. Additionally, it dramatically reduces the amount of JavaScript required on the frontend.

This approach is perfectly suitable for many modern applications where client-side logic is minimal (e.g., management apps with lots of forms, dashboards, etc.)².

API as an Implementation Detail

Even within the JavaScript ecosystem, it’s possible to avoid an HTTP/JSON API (in the sense of a separate API with the constraints mentioned earlier). React Router/Remix loaders, the $server primitive in Qwik, and React Server Components all bring the client and server closer, making the JSON API between the two merely an implementation detail³.

For frameworks that don’t offer this, hiding the HTTP API often involves code generation. Server methods (Controllers) become JavaScript functions to use in your frontend app following an automated code generation process (e.g., based on OpenAPI or Protobuf). Changing a server-side method allows you to use the new version client-side without thinking about "HTTP requests."

In all these cases, there's still an HTTP API between the client’s browser and the server, but you’re not coding it or calling it explicitly. You lose some of the benefits mentioned earlier (like calling this API from another app easily), but you also avoid the constraints of treating an API as a “product.”

The Back for Front Pattern

Sometimes, your application’s backend isn’t under your control. Maybe it’s managed by another team, maybe it’s a third-party product, or maybe it’s exposed to other consumers as a product. In such cases, the constraints of backward compatibility, documentation, etc., exist de facto.

Here, introducing a server component between your frontend and this API can be useful. This component is dedicated to your application (so it can evolve without restrictions) and provides the benefits of a server like shared caching, server-side validation, authentication, algorithmic performance improvements, etc.

This pattern is known as Back for Front and is, in my opinion, still underutilized.

Conclusion

For your next web application, pause before reaching for the REST/JSON API weapon.

Ask the right questions: Who depends on my backend? How many applications? Who controls them?
Consider dropping the client/server architecture entirely in favor of documents, perhaps with HTMX².
Treat your API as an implementation detail, just a way to traverse the network (the approach React, Next.js, Remix, and Qwik take, but one you can adopt too).
If your API is beyond your control or has multiple clients, introduce a Back for Front so your application can benefit from a server under your control.

This approach does, however, have one prerequisite: stop separating frontend and backend developers. If they’re building the same product, they need to work together.

XHR started appearing in browsers as early as 1999. ↩
For further details, see my talk on this subject at BreizhCamp 2024 (in french). ↩
See more in Ryan Florence’s "Mind the Gap" talk. ↩

Writing Good Automated Tests with AI: A Case Study

Benoit Averty — Sun, 03 Nov 2024 23:00:00 +0000

The use of AI is spreading among developers. For better or worse, more and more AI-generated code will make its way into production... and into the tests for this production code. In this article, I present an experiment I conducted on writing automated tests using AI, along with the conclusions I drew from it.

Read this article in french on my website

Genesis
Experimentation
- The Class to Test
- Poor Tests
- Why Is It So Bad?
- A Slightly More Advanced Attempt
- End of the Experiment
Results
- Tips for Using AI Effectively
- My Opinion
Appendices
- The Complete ContentSelectionPipeline Class
- The Tests Generated with the Poor Prompt

Genesis

A while ago, I came across this post on LinkedIn.

The post is in french, here's a traduction:

🦾 Developers, generate your tests using AI

⁉️ But why generate tests?

⚠️ Because some sellers of AI proxies, or even fake AI, make you believe that testing is a tedious activity.

🤝 It can be tempting for a manager, who doesn't know much about it, to be swayed. Especially if their teams don't do testing and perpetuate this idea of its difficulty. The promise then becomes enticing!

⛔️ Spoiler: the generated tests are just a copy of the code: faulty code = green test (when it compiles). However, writing tests requires a deep understanding of the context and product requirements.

🔄 What if we approached things differently: tests as a valuable tool.

👉 Learn to do TDD (Test-Driven Development), rather than coding haphazardly and wanting to generate tests afterward. Use tests as a guide, and you’ll get living documentation, non-regression, and more.

Although I agree with Christophe that tests should not be seen as a tedious or secondary activity, his comments on the quality of AI-generated tests surprised me. My intuition was quite the opposite: I felt that writing tests was one of the more promising use cases for AI, similar to writing a SQL query, a pure function, or a slightly complex TypeScript type. So I decided to try it out myself, beyond intuition, to see what the reality was.

My goal in this experiment was to answer the following questions:

Is it possible to write good automated tests with AI?
What methods help achieve this?
Is it interesting, worthwhile in terms of time and effort?

Experimentation

The Class to Test

One problem with experiments of this kind is that it's hard to find a realistic scenario. Successfully coding tests with AI wouldn't mean much on a single class and without a real project. So I chose to pull out a project from a client of mine and generate tests for one of the core domain classes.

The code for this class is presented in the appendix, and here is a version without the method bodies and private members.

/**
 * Represents the successive steps taken to select the best content from a `bundle` returned by the indexing engine.
 *
 * The content selection pipeline as currently implemented only supports text content.
 */
class ContentSelectionPipeline(
    private val parameters: ContentSelectionParameters,
    private val blacklist: Blacklist,
    private val queryingEditor: Editor,
    private val relatedEditors: List<Editor>,
    private val fetchContentMetadata: suspend (List<DocumentIndexMetadataWithScore>) -> List<ContentMetadata>,
    private val clock: Clock
) {
    // Private attributes omitted. But the class is indeed stateful.

    suspend fun initializeForDocuments(relatedDocuments: RelatedDocumentsBundle) {
        // ...
    }

    fun execute() {
        // ...
    }
    fun isSuccess() : Boolean {
        // ...
    }
    fun getFinalContents(): List<ContentMetadata> {
        // ...
    }

    // Private methods omitted. The real file is 130 lines long.
}

This class comes from a project aimed at sharing Content (Articles...) between multiple Editors (Websites with press articles, videos...). The role of this class is to select three pieces of content from a set of documents (RelatedDocumentsBundle) returned by an IndexationEngine (think Elasticsearch). To fulfill this role, the class must fetch content metadata (ContentMetadata: URL, word count...) corresponding to the documents, and then apply a number of configurable business rules.

The typical use case can be described as follows:

Instantiate the pipeline with several parameters (the group of editors involved, business parameters, an exclusion list...)
Initialize the pipeline with a set of documents from the indexing engine (initializeForDocuments method)
Execute the pipeline (execute method, which applies all business rules to select the content)
Check the operation status (isSuccess) and retrieve the selected content (getFinalContents)

Steps 2 to 4 can be executed multiple times for a single instance, but of course, they cannot be executed out of order.

Poor Tests

The first step of my experiment was to get a sense of what happens when you put in no effort. I opened my chat agent (Jetbrains AI) and asked it to write tests:

Here is a class for which I want to generate tests:
class ContentSelectionPipeline {
 [...]
}
Write the tests for this class.

Again, the full code is in the appendix, but here’s the beginning of the test class and the list of test cases generated:

package fr.sipaof.flink.domain.content.selection

class IaContentSelectionPipelineTests {

    private lateinit var parameters: ContentSelectionParameters
    private lateinit var blacklist: Blacklist
    private lateinit var queryingEditor: Editor
    private lateinit var relatedEditors: List<Editor>
    private lateinit var fetchContentMetadata: suspend (List<DocumentIndexMetadataWithScore>) -> List<ContentMetadata>
    private lateinit var clock: Clock
    private lateinit var contentSelectionPipeline: ContentSelectionPipeline

    @BeforeEach
    fun setUp() {
        parameters = mockk<ContentSelectionParameters>()
        blacklist = mockk()
        queryingEditor = mockk()
        relatedEditors = listOf()
        fetchContentMetadata = mockk()
        clock = mockk()

        contentSelectionPipeline = ContentSelectionPipeline(
            parameters,
            blacklist,
            queryingEditor,
            relatedEditors,
            fetchContentMetadata,
            clock
        )
    }

    @Test
    fun `initializeForDocuments should initialize contents and initialBundle`() = runBlocking {
        // ...
    }

    @Test
    fun `initializeForDocuments should throw IllegalStateException for unsupported content`(): Unit = runBlocking {
        // ...
    }

    @Test
    fun `execute should populate steps and selectedDocuments`() = runBlocking {
        // ...
    }

    @Test
    fun `isSuccess should return true when at least 2 documents are selected`() = runBlocking {
        // ...
    }

    @Test
    fun `isSuccess should return false when less than 2 documents are selected`() = runBlocking {
        // ...
    }

    @Test
    fun `getFinalContents should return selected content metadata`() = runBlocking {
        // ...
    }

    @Test
    fun `getExecutionSummary should return pipeline execution summary`() = runBlocking {
        // ...
    }

}

Unsurprisingly, it’s a disaster. These tests are a textbook example of what not to do.

First, there is a list of attributes in the class that correspond to all the data provided to the class at instantiation. Most of these attributes are initialized by mocks (mockk) before each test, even though most are Kotlin data classes that would be much simpler to instantiate directly (especially since we prefer testing their behavior, as they are used by the ContentSelectionPipeline class we are testing). There’s even a mock for Clock, even though Jetbrains AI surely knows Clock.fixed. In short, too many mocks.

Secondly, if you look at the names of the generated test cases, you’ll find a lot of vocabulary that I didn’t mention when I presented the class to be tested. And rightly so, because it corresponds to private classes, functions, and attributes of ContentSelectionPipeline. Indeed, the tests do not compile without making several of these attributes public, which is another bad practice. In short, the tests are coupled to the implementation of the class.

Lastly, let’s look at one of these test cases in full:

    @Test
    fun `initializeForDocuments should initialize contents and initialBundle`() = runBlocking {
        val relatedDocumentsBundle = mockk<RelatedDocumentsBundle>()
        val textContentMetadata = mockk<TextContentMetadata>()
        val documentIndexMetadataWithScore = mockk<DocumentIndexMetadataWithScore>()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)

        // --
        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)
        // --

        assertNotNull(contentSelectionPipeline.contents)
        assertNotNull(contentSelectionPipeline.initialBundle)
        assertNull(contentSelectionPipeline.steps)
        assertNull(contentSelectionPipeline.selectedDocuments)
    }

The beginning and end of the test illustrate what I mentioned earlier:

We mock classes that should be included in the system under test.
We make assertions on private attributes.

But the middle part (which I marked with // --) shows another problem: calling only the initializeForDocuments method does not correspond to any real use case of the class. The test is doing something that doesn’t make sense, there is nothing to observe, forcing us to make assertions on the private state of the class.

Why Is It So Bad?

Should we conclude that AI is bad at writing tests? No. It’s just that we cannot expect an AI to succeed at anything with the little information I provided in the prompt.

The AI had access only to the ContentSelectionPipeline class, but I did not give it the code for the supporting classes, which are essential to the service provided by ContentSelectionPipeline.
The AI had to guess the boundaries of the system under test (I did not specify them), and it couldn't make relevant guesses because, again, I did not provide the code for the related classes. These classes could be entirely independent, justifying a mock rather than their use in the test.
The AI had no information on how the class is used in the rest of the application, so it couldn't tell which scenarios made sense and which didn’t (although in this regard, a more powerful model like OpenAI o1 could do much better, even with no additional information).

It's easy to conclude that an AI cannot perform certain tasks based on examples like the ones I just showed. But it is important to remember that an LLM is a computational tool, not magic. Like any tool, it requires a minimum amount of effort to be used properly, and most importantly, one must learn to use it. Typing a one-line prompt required no effort and is the first thing any developer who has not tried to improve their LLM usage would do.

To do better, we need to improve at least two aspects:

Provide more information to the AI.
Improve our prompt.

A Slightly More Advanced Attempt

First Prompt

We are going to write unit tests for the class #file:ContentSelectionPipeline.kt and its associated classes (#symbol:ContentSelectionParameters, #file:Blacklist.kt, #file:Editor.kt, #symbol:DocumentIndexMetadataWithScore, #file:ContentMetadata.kt, #file:RelatedDocumentsBundle.kt, #file:TextContentMetadata.kt, #symbol:ContentAndDocumentMetadataWithScore).

Private methods must never appear in the tests. Mocks should be avoided unless I explicitly mention them.

First, we will write a utility method that will help us create input data for the tests. The method will take a list of editors as a parameter and the path to a CSV file with the following columns: editorSlug;contentId;title;url;numberOfWords;publicationDate;indexationId;returnedScore;blacklisted. Each line corresponds to a document present in the Bundle and the associated ContentMetadata.

What to note from this prompt:

I included the complete code of all the classes belonging to the tested system as I envisioned it (that’s what is done with the special syntax #file:xyz.kt available in Jetbrains AI, but if you’re using something else, copying and pasting the file will do just fine). This means that I had thought about the system under test in advance, the AI did not do it for me.
I was explicit about the desired testing practices: no mocks, no private attributes in the tests.
I broke down the task. Although I explained initially that the goal was to write a test, I started by writing a utility method to generate input data, as I knew it would be laborious to instantiate (many attributes and parameters). My knowledge of the system under test allowed me to identify this first step; the AI did not think for me.

In response to this prompt, the AI provided a function matching this signature:

data class TestDocument(
    val editor: Editor,
    val contentMetadata: TextContentMetadata,
    val documentIndexMetadata: DocumentIndexMetadataWithScore
)

fun createTestDocumentsFromCsv(editors: List<Editor>, csvFilePath: String): List<TestDocument> {
    // ...
}

Adjustment

It didn’t suit me because the result of this method was not directly usable as input to my system under test. So I clarified:

The method should return the input data as a RelatedDocumentsBundle and a function corresponding to the fetchContentMetadata parameter of the content selection pipeline.

This time, I decided to keep the result. It wasn’t perfect, but a quick println() test at least convinced me that it worked, and there would always be time to refactor later.

data class TestInput(
    val relatedDocumentsBundle: RelatedDocumentsBundle,
    val fetchContentMetadata: suspend (List<DocumentIndexMetadataWithScore>) -> List<TextContentMetadata>
)

fun createTestInputFromCsv(editors: List<Editor>, csvFilePath: String): TestInput {
    val csvFile = File(csvFilePath)
    val csvLines = csvFile.readLines()

    // Assumes the first line is the header
    val header = csvLines.first().split(";")
    require(header == listOf("editorSlug", "contentId", "title", "url", "numberOfWords", "publicationDate", "indexationId", "returnedScore", "blacklisted"))

    val editorMap = editors.associateBy { it.slug }
    val dateFormatter = DateTimeFormatter.ISO_ZONED_DATE_TIME

    val documentIndexMetadataMap: MutableMap<Collection, MutableList<DocumentIndexMetadataWithScore>> = mutableMapOf()
    val contentMetadataMap = mutableMapOf<InternalContentId, TextContentMetadata>()

    for (line in csvLines.drop(1)) {
        val columns = line.split(";")
        val editorSlug = columns[0]
        val contentId = columns[1]
        val title = columns[2]
        val url = columns[3]
        val numberOfWords = columns[4].toInt()
        val publicationDate = ZonedDateTime.parse(columns[5], dateFormatter)
        val indexationId = UUID.fromString(columns[6])
        val returnedScore = columns[7].toDouble()
        val blacklisted = columns[8].toBoolean()

        val editor = editorMap[editorSlug] ?: throw IllegalArgumentException("Editor not found: $editorSlug")

        val contentMetadata = TextContentMetadata(
            id = OpaqueId(contentId),
            editor = editor,
            title = title,
            url = Url(url),
            publicationDate = publicationDate,
            numberOfWords = numberOfWords,
            hashedText = null // or compute based on the text if available
        )

        val documentMetadata = DocumentIndexMetadataWithScore(
            documentIndexMetadata = DocumentIndexMetadata(indexationId),
            score = returnedScore
        )

        documentIndexMetadataMap.computeIfAbsent(editor.collection) { mutableListOf() }.add(documentMetadata)
        contentMetadataMap[contentMetadata.internalId] = contentMetadata()
    }

    val relatedDocumentsBundle = RelatedDocumentsBundle(documentIndexMetadataMap)
    val fetchContentMetadata: suspend (List<DocumentIndexMetadataWithScore>) -> List<TextContentMetadata> = { documentMetadataList ->
        documentMetadataList.mapNotNull { contentMetadataMap[it.documentIndexMetadata.internalId] }
    }

    return TestInput(relatedDocumentsBundle, fetchContentMetadata)
}

Generating Data for the First Test Case

With this function ready to generate a lot of test data, it was time to write a test case.

Now let’s write the unit test for the happy path of the content selection pipeline. The scenario is as follows:

There are three editors: fake-editor-1, other-editor, and awesome-editor. They each have a freshnessCoefficient of 0.97, 0.97, and 0.90, respectively.

The input parameters for the pipeline are the same as in the production code (in #file:application.yml).

Each editor has three documents in the input document set.

Please write the CSV file in the format defined earlier that corresponds to this scenario.

Again, instead of directly asking for the test, I asked for a CSV with several lines of data. And the AI responded with a CSV matching the specifications I gave in the prompt. It correctly generated scores between the threshold and ceiling specified in application.yml. It even set the blacklisted column to false throughout, explaining that it was for simplicity. Pretty relevant for the happy path I asked for.

Generated CSV. Don’t feel the need to understand every detail...

editorSlug;contentId;title;url;numberOfWords;publicationDate;indexationId;returnedScore;blacklisted
fake-editor-1;content-1-1;Title 1-1;http://example.com/1-1;500;2023-01-01T10:00:00Z;00000000-0000-0000-0000-000000000001;0.75;false
fake-editor-1;content-1-2;Title 1-2;http://example.com/1-2;1000;2023-01-02T10:00:00Z;00000000-0000-0000-0000-000000000002;0.82;false
fake-editor-1;content-1-3;Title 1-3;http://example.com/1-3;750;2023-01-03T10:00:00Z;00000000-0000-0000-0000-000000000003;0.90;false
other-editor;content-2-1;Title 2-1;http://example.com/2-1;600;2023-02-01T11:00:00Z;00000000-0000-0000-0000-000000000004;0.70;false
other-editor;content-2-2;Title 2-2;http://example.com/2-2;1100;2023-02-02T11:00:00Z;00000000-0000-0000-0000-000000000005;0.78;false
other-editor;content-2-3;Title 2-3;http://example.com/2-3;700;2023-02-03T11:00:00Z;00000000-0000-0000-0000-000000000006;0.85;false
awesome-editor;content-3-1;Title 3-1;http://example.com/3-1;550;2023-03-01T12:00:00Z;00000000-0000-0000-0000-000000000007;0.65;false
awesome-editor;content-3-2;Title 3-2;http://example.com/3-2;1200;2023-03-02T12:00:00Z;00000000-0000-0000-0000-000000000008;0.80;false
awesome-editor;content-3-3;Title 3-3;http://example.com/3-3;800;2023-03-03T12:00:00Z;00000000-0000-0000-0000-000000000009;0.88;false

Writing the Test Code

Finally:

Now, let’s write the unit test that uses this file. The test should use the CSV and the utility method to create the input data, use Clock.fixed, create the parameters according to the real values, then instantiate the pipeline. Then, call the methods in the correct order. Finally, verify that the pipeline is successful and that the correct content has been selected.

Once again, I didn’t hesitate to explain exactly what I expected.

The result:

class ContentSelectionPipelineTest {

    private lateinit var csvFilePath: String

    @BeforeEach
    fun setup() {
        // Create a temporary CSV file with the required data
        val csvContent = """
            editorSlug;contentId;title;url;numberOfWords;publicationDate;indexationId;returnedScore;blacklisted
            fake-editor-1;content-1-1;Title 1-1;http://example.com/1-1;500;2023-01-01T10:00:00Z;00000000-0000-0000-0000-000000000001;0.75;false
            fake-editor-1;content-1-2;Title 1-2;http://example.com/1-2;1000;2023-01-02T10:00:00Z;00000000-0000-0000-0000-000000000002;0.82;false
            fake-editor-1;content-1-3;Title 1-3;http://example.com/1-3;750;2023-01-03T10:00:00Z;00000000-0000-0000-0000-000000000003;0.90;false
            other-editor;content-2-1;Title 2-1;http://example.com/2-1;600;2023-02-01T11:00:00Z;00000000-0000-0000-0000-000000000004;0.70;false
            other-editor;content-2-2;Title 2-2;http://example.com/2-2;1100;2023-02-02T11:00:00Z;00000000-0000-0000-0000-000000000005;0.78;false
            other-editor;content-2-3;Title 2-3;http://example.com/2-3;700;2023-02-03T11:00:00Z;00000000-0000-0000-0000-000000000006;0.85;false
            awesome-editor;content-3-1;Title 3-1;http://example.com/3-1;550;2023-03-01T12:00:00Z;00000000-0000-0000-0000-000000000007;0.65;false
            awesome-editor;content-3-2;Title 3-2;http://example.com/3-2;1200;2023-03-02T12:00:00Z;00000000-0000-0000-0000-000000000008;0.80;false
            awesome-editor;content-3-3;Title 3-3;http://example.com/3-3;800;2023-03-03T12:00:00Z;00000000-0000-0000-0000-000000000009;0.88;false
        """.trimIndent()

        val tempFile = createTempFile(suffix = ".csv")
        Files.writeString(tempFile, csvContent)
        csvFilePath = tempFile.toAbsolutePath().toString()
    }

    @Test
    fun `test content selection pipeline happy path`() = runBlocking {
        // Define editors
        val editorsList = listOf(
            Editor(UUID.randomUUID(), "fake-editor-1", 0.97, UUID.randomUUID()),
            Editor(UUID.randomUUID(), "other-editor", 0.97, UUID.randomUUID()),
            Editor(UUID.randomUUID(), "awesome-editor", 0.90, UUID.randomUUID())
        )

        // Create test input using the factory method
        val testInput = createTestInputFromCsv(editorsList, csvFilePath)

        // Define Content Selection Parameters according to application.yml
        val parameters = ContentSelectionParameters(
            scoreThreshold = 0.48,
            scoreCeiling = 0.95,
            freshnessScoreThreshold = 0.3
        )

        // Fixed Clock
        val fixedClock = Clock.fixed(Instant.parse("2023-04-01T00:00:00Z"), ZoneOffset.UTC)

        // Blacklist implementation (no items blacklisted for simplicity)
        val blacklist = object : Blacklist {
            override val allItems = emptyList<BlacklistItem>()
            override fun contains(content: InternalContentId): Boolean = false
        }

        // Instantiate the pipeline
        val pipeline = ContentSelectionPipeline(
            parameters = parameters,
            blacklist = blacklist,
            queryingEditor = editorsList.first(),
            relatedEditors = editorsList.drop(1),
            fetchContentMetadata = testInput.fetchContentMetadata,
            clock = fixedClock
        )

        // Initialize the pipeline
        pipeline.initializeForDocuments(testInput.relatedDocumentsBundle)

        // Execute the pipeline
        pipeline.execute()

        // Check if the pipeline succeeded
        assertTrue(pipeline.isSuccess())

        // Get the final contents
        val finalContents = pipeline.getFinalContents()

        // Assert the correct contents have been selected
        val expectedContents = listOf(
            "Title 1-1", "Title 1-2", "Title 1-3", 
            "Title 2-1", "Title 2-2", "Title 2-3", 
            "Title 3-1", "Title 3-2", "Title 3-3"
        )
        val finalContentTitles = finalContents.map { it.title }

        assertEquals(expectedContents.sorted(), finalContentTitles.sorted())
    }
}

I find this result quite satisfactory (apart from the CSV inline in the class, but that’s trivial to extract). I still had a few adjustments to make to the test scenario itself: change one or two values in the CSV, change the Clock date, and change the assertion, as the AI didn’t understand that in the happy path, only two or three pieces of content could be selected (the only real disappointment).

However, adjusting the scenario this way is clearly the most complex task. Determining, without executing it, what the result of the pipeline will be based on the input data is precisely what gives the test all its value. It seems normal to me to leave this job to the developer once the AI has quickly generated all this code.

And the final result of the experiment, of course, is that this test:

Compiles;
Passes;
Fails when we introduce bugs into the class.

For those who are interested, the complete log of my conversation with Jetbrains AI is available at this link.

End of the Experiment

At this point, I stopped my experiment. The first reason is that this class is already well-tested in reality, so the exercise is somewhat pointless. But mostly, I had formed a much clearer opinion on the original question that interested me.

To continue the experiment, I would need to:

Refactor the CSV parsing method, which is a bit difficult to read, in my opinion.
Create other test scenarios, especially cases that exclude content based on different business rules.¹

Results

Tips for Using AI Effectively

This experiment was conducted on automated tests, but I hypothesize that the lessons I learned apply to other tasks as well. It would need verification, but my other experiments (although less rigorous) seem to confirm this.

AI Works When the Generated Code is Localized

AI works well for generating code that is localized to one or two places.

Generating a test class and a utility method (as I did for this article)
Generating SQL queries
Generating complex TypeScript types
...

Needing context from many files is not a problem thanks to IDE features that quickly pass the code from several files to the LLM.

On the other hand, modifying a large number of files was more challenging and too laborious to save time.

You Must Give the AI a Lot of Context

Providing a single class to generate tests is not enough. You also need to provide the code of the classes used as parameters and attributes to get a correct result.

Think of it like the input data of a problem, like you would expect for solving a math problem. Similarly, if you want to generate an algorithmic function, the LLM must know the code of any classes used as parameters or return types. If you design a web controller, the LLM needs to know what framework you're using, etc.

Be Explicit About Expectations

By default, the LLM wrote tests with many mocks and assertions on the private state of the class. But this is not an inherent limitation of the model. It simply needed me to ask not to do so.

Similarly, if you request a somewhat complex SQL query, do not hesitate to specify the desired output columns, tables to use, indexes that you want the query to prioritize, etc.

My Opinion

I am convinced that it is possible to write excellent tests with the help of an AI. You need to have realistic expectations: the AI will not relieve you from thinking about the system under test, the test cases, or the technical architecture of your testing framework. These are questions you need to ask yourself anyway if you write your tests manually.

I spent about two hours writing this first test. What took the most time was starting over several times to perfect my approach and offer you a summary of a not-too-long discussion session with the AI. It is clear that writing the following test cases would take less time, and in the end, I think this approach would have saved time in writing the tests.

Will I adopt AI for all my future automated tests? I don't know. The environmental cost is high, as is the financial cost, given that AI is the new golden goose for companies worldwide. Despite the rapid progress in these technologies, time savings are not yet miraculous (although they are real in some cases).

On the other hand, it seems important to continue experimenting. In my opinion, using LLMs to produce code is a useful skill for a developer in a context where the rush towards AI shows no sign of stopping.

Appendices

The Complete `ContentSelectionPipeline` Class

/**
 * Represents the successive steps taken to select the best content from a bundle returned by the indexing engine.
 *
 * The ContentSelectionPipeline as currently implemented only supports Text Content.
 */
class ContentSelectionPipeline(
    private val parameters: ContentSelectionParameters,
    private val blacklist: Blacklist,
    private val queryingEditor: Editor,
    private val relatedEditors: List,
    private val fetchContentMetadata: suspend (List) -&gt; List,
    private val clock: Clock
) {
    private var initialBundle: RelatedDocumentsBundleWithContentMetadata? = null
    private var contents: MutableSet? = null
    private var steps: MutableList? = null
    private var selectedDocuments: MutableList? = null

    suspend fun initializeForDocuments(relatedDocuments: RelatedDocumentsBundle) {
        // Keep only TextContentMetadata (the only one that exists in the DB for now)
        val fetchedContentMetadata = fetchContentMetadata(relatedDocuments.allDocuments())
        if(fetchedContentMetadata.any { it !is TextContentMetadata }) {
            throw IllegalStateException("The content selection pipeline was initialized with unsupported content." +
                    " Pipeline can only handle text content.")
        }

        contents = HashSet(
            fetchedContentMetadata.filterIsInstance()
        )
        initialBundle = associateContentToBundle(relatedDocuments, contents!!)
        steps = null
        selectedDocuments = null
    }

    fun execute() {
        val execBundle = initialBundle ?: throw IllegalStateException("Executing non-initialized pipeline")
        steps = mutableListOf()

        steps!!.add(PipelineStep(BLACKLIST, execBundle.applyBlacklist(blacklist)))
        steps!!.add(PipelineStep(SCORE, steps!!.last().bundle.filter(::hasScoreBetweenThresholdAndCeiling)))
        steps!!.add(PipelineStep(FRESHNESS, steps!!.last().bundle.applyFreshnessCoeff(relatedEditors, Instant.now(clock))))
        steps!!.add(PipelineStep(EXCLUDE_FRESHNESS, steps!!.last().bundle.filter(::hasScoreAboveFreshnessThreshold)))

        selectContents()
    }
    fun isSuccess() : Boolean {
        return selectedDocuments?.let {
            it.size &gt;= 2
        } ?: false
    }
    fun getFinalContents(): List {
        val selectedDocumentsCopy = selectedDocuments ?: throw IllegalStateException("Cannot get content on a pipeline that hasn't been executed")

        return selectedDocumentsCopy.map(ContentAndDocumentMetadataWithScore::contentMetadata)
    }

    private fun hasScoreBetweenThresholdAndCeiling(doc: ContentAndDocumentMetadataWithScore): Boolean =
        doc.score &gt; parameters.scoreThreshold &amp;&amp; doc.score &lt; parameters.scoreCeiling

    private fun hasScoreAboveFreshnessThreshold(doc: ContentAndDocumentMetadataWithScore): Boolean =
        doc.score &gt; parameters.freshnessScoreThreshold

    private fun selectContents() {
        selectedDocuments = mutableListOf()
        val finalBundle = steps!!.last().bundle

        // Select the first item : priority to the querying editor
        selectedDocuments!!.add(
            finalBundle.longestContentFromCollection(queryingEditor.collection)
                ?: finalBundle.longestContent()
                ?: return
        )

        // select the second and third item : priority to another editor
        selectedDocuments!!.add(
            finalBundle.longestContent(
                excludeCollections = getCollectionsFromMetadata(selectedDocuments!!),
                excludeDocuments = selectedDocuments!!
            )
                ?: finalBundle.longestContent(excludeDocuments = selectedDocuments!!)
                ?: return
        )
        selectedDocuments!!.add(
            finalBundle.longestContent(
                excludeCollections = getCollectionsFromMetadata(selectedDocuments!!),
                excludeDocuments = selectedDocuments!!
            )
                ?: finalBundle.longestContent(excludeDocuments = selectedDocuments!!)
                ?: return
        )
    }

    fun getExecutionSummary(): PipelineExecutionSummary {
        if(selectedDocuments == null) throw IllegalStateException("Cannot get summary of a pipeline that hasn't been executed")

        return PipelineExecutionSummary(
            initialBundle!!,
            steps!!,
            getFinalContents(),
        )
    }

    private fun getCollectionsFromMetadata(content: List): List {
        val editorsTids = content.map { it.documentIndexMetadata.id.editorTid }
        return relatedEditors.filter { editorsTids.contains(it.tid) }.map(Editor::collection)
    }
}

class PipelineStep(
val name: PipelineStepName,
val bundle: RelatedDocumentsBundleWithContentMetadata,
)

enum class PipelineStepName {
BLACKLIST, SCORE, FRESHNESS, EXCLUDE_FRESHNESS
}

The Tests Generated with the Poor Prompt

package fr.sipaof.flink.domain.content.selection

class IaContentSelectionPipelineTests {

    private lateinit var parameters: ContentSelectionParameters
    private lateinit var blacklist: Blacklist
    private lateinit var queryingEditor: Editor
    private lateinit var relatedEditors: List
    private lateinit var fetchContentMetadata: suspend (List) -&gt; List
    private lateinit var clock: Clock
    private lateinit var contentSelectionPipeline: ContentSelectionPipeline

    @BeforeEach
    fun setUp() {
        parameters = mockk()
        blacklist = mockk()
        queryingEditor = mockk()
        relatedEditors = listOf()
        fetchContentMetadata = mockk()
        clock = mockk()

        contentSelectionPipeline = ContentSelectionPipeline(
            parameters,
            blacklist,
            queryingEditor,
            relatedEditors,
            fetchContentMetadata,
            clock
        )
    }

    @Test
    fun `initializeForDocuments should initialize contents and initialBundle`() = runBlocking {
        val relatedDocumentsBundle = mockk()
        val textContentMetadata = mockk()
        val documentIndexMetadataWithScore = mockk()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)

        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)

        assertNotNull(contentSelectionPipeline.contents)
        assertNotNull(contentSelectionPipeline.initialBundle)
        assertNull(contentSelectionPipeline.steps)
        assertNull(contentSelectionPipeline.selectedDocuments)
    }

    @Test
    fun `initializeForDocuments should throw IllegalStateException for unsupported content`(): Unit = runBlocking {
        val relatedDocumentsBundle = mockk()
        val contentMetadata = mockk() // Not TextContentMetadata

        every { relatedDocumentsBundle.allDocuments() } returns listOf(mockk())
        coEvery { fetchContentMetadata(any()) } returns listOf(contentMetadata)

        assertThrows {
            runBlocking {
                contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)
            }
        }
    }

    @Test
    fun `execute should populate steps and selectedDocuments`() = runBlocking {
        val relatedDocumentsBundle = mockk()
        val textContentMetadata = mockk()
        val documentIndexMetadataWithScore = mockk()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)

        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)

//        every { blacklist.filter(any()) } returns mockk()
        every { clock.instant() } returns Instant.now()
//        every { relatedDocumentsBundle.longestContentFromCollection(any()) } returns mockk()
//        every { relatedDocumentsBundle.longestContent(excludeCollections = any(), excludeDocuments = any()) } returns mockk()
        every { queryingEditor.collection } returns mockk()

        contentSelectionPipeline.execute()

        assertNotNull(contentSelectionPipeline.steps)
        assertNotNull(contentSelectionPipeline.selectedDocuments)
    }

    @Test
    fun `isSuccess should return true when at least 2 documents are selected`() = runBlocking {
        val relatedDocumentsBundle = mockk()
        val textContentMetadata = mockk()
        val documentIndexMetadataWithScore = mockk()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)

        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)
        contentSelectionPipeline.execute()

        // Assume the pipeline selects 2 documents
        contentSelectionPipeline.selectedDocuments?.add(mockk())
        contentSelectionPipeline.selectedDocuments?.add(mockk())

        assertTrue(contentSelectionPipeline.isSuccess())
    }

    @Test
    fun `isSuccess should return false when less than 2 documents are selected`() = runBlocking {
        val relatedDocumentsBundle = mockk()
        val textContentMetadata = mockk()
        val documentIndexMetadataWithScore = mockk()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)

        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)
        contentSelectionPipeline.execute()

        // Assume the pipeline selects 1 document
        contentSelectionPipeline.selectedDocuments?.add(mockk())

        assertFalse(contentSelectionPipeline.isSuccess())
    }

    @Test
    fun `getFinalContents should return selected content metadata`() = runBlocking {
        val relatedDocumentsBundle = mockk()
        val textContentMetadata = mockk()
        val documentIndexMetadataWithScore = mockk()
        val contentAndDocumentMetadataWithScore = mockk()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)
        every { contentAndDocumentMetadataWithScore.contentMetadata } returns textContentMetadata

        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)
        contentSelectionPipeline.execute()

        // Assume the pipeline selects 1 document
        contentSelectionPipeline.selectedDocuments?.add(contentAndDocumentMetadataWithScore)

        val finalContents = contentSelectionPipeline.getFinalContents()
        assertEquals(listOf(textContentMetadata), finalContents)
    }

    @Test
    fun `getExecutionSummary should return pipeline execution summary`() = runBlocking {
        val relatedDocumentsBundle = mockk()
        val textContentMetadata = mockk()
        val documentIndexMetadataWithScore = mockk()
        val contentAndDocumentMetadataWithScore = mockk()
        val pipelineStep = mockk()

        every { relatedDocumentsBundle.allDocuments() } returns listOf(documentIndexMetadataWithScore)
        coEvery { fetchContentMetadata(any()) } returns listOf(textContentMetadata)
        every { contentAndDocumentMetadataWithScore.contentMetadata } returns textContentMetadata
        every { pipelineStep.bundle } returns mockk()

        contentSelectionPipeline.initializeForDocuments(relatedDocumentsBundle)
        contentSelectionPipeline.execute()

        // Assume the pipeline selects 1 document and steps are populated
        contentSelectionPipeline.selectedDocuments?.add(contentAndDocumentMetadataWithScore)
        contentSelectionPipeline.steps?.add(pipelineStep)

        val summary = contentSelectionPipeline.getExecutionSummary()
        assertNotNull(summary)
    }

}

I must say that the AI can probably help with this task as well, although my quick tests were not very convincing: https://chatgpt.com/share/67083a01-e200-8000-bfa2-37b94113fa4c ↩

Implementing Mixins (or Traits) in Kotlin Using Delegation

Benoit Averty — Fri, 18 Oct 2024 11:05:52 +0000

(Read this article in french on my website)

In object-oriented programming, a Mixin is a way to add one or more predefined and autonomous functionalities to a class. Some languages provide this capability directly, while others require more effort and compromise to code Mixins. In this article, I explain an implementation of Mixins in Kotlin using delegation.

Objective
- Definition of the "Mixins" Pattern
- Characteristics and Constraints
Implementation
- Naive Approach by Composition
- Use of Inheritance
- Delegation to Contain the Mixin’s State
- Final Implementation
Limitations
Examples
- Auditable
- Observable
- Entity / Identity
Conclusion

Objective

Definition of the "Mixins" Pattern

The mixin pattern is not as precisely defined as other design patterns such as Singleton or Proxy. Depending on the context, there may be slight differences in what the term means.

This pattern can also be close to the "Traits" present in other languages (e.g., Rust), but similarly, the term "Trait" does not necessarily mean the same thing depending on the language used¹.

That said, here is a definition from Wikipedia:

In object-oriented programming, a mixin (or mix-in) is a class that contains methods used by other classes without needing to be the parent class of those other classes. The way these other classes access the mixin’s methods depends on the language. Mixins are sometimes described as being "included" rather than "inherited."

You can also find definitions in various articles on the subject of mixin-based programming (², ³, ⁴). These definitions also bring this notion of class extension without the parent-child relationship (or is-a) provided by classical inheritance. They further link with multiple inheritance, which is not possible in Kotlin (nor in Java) but is presented as one of the interests of using mixins.

Characteristics and Constraints

An implementation of the pattern that closely matches these definitions must meet the following constraints:

We can add several mixins to a class. We’re in an object-oriented context, and if this constraint weren’t met, the pattern would have little interest compared to other design possibilities like inheritance.
Mixin functionality can be used from outside the class. Similarly, if we disregard this constraint, the pattern will bring nothing we couldn’t achieve with simple composition.
Adding a mixin to a class does not force us to add attributes and methods in the class definition. Without this constraint, the mixin could no longer be seen as a "black box" functionality. We could not only rely on the mixin’s interface contract to add it to a class but would have to understand its functioning (e.g., via documentation). I want to be able to use a mixin as I use a class or a function.
A mixin can have a state. Some mixins may need to store data for their functionality.
Mixins can be used as types. For example, I can have a function that takes any object as a parameter as long as it uses a given mixin.

Implementation

Naive Approach by Composition

The most trivial way to add functionalities to a class is to use another class as an attribute. The mixin’s functionalities are then accessible by calling methods of this attribute.

class MyClass {
    private val mixin = Counter()

    fun myFunction() {
        mixin.increment()

        // ...
    }
}

This method doesn’t provide any information to Kotlin’s type system. For example, it’s impossible to have a list of objects using Counter. Taking an object of type Counter as a parameter has no interest as this type represents only the mixin and thus an object probably useless to the rest of the application.

Another problem with this implementation is that the mixin’s functionalities aren’t accessible from outside the class without modifying this class or making the mixin public.

Use of Inheritance

For mixins to also define a type usable in the application, we will need to inherit from an abstract class or implement an interface.

Using an abstract class to define a mixin is out of the question, as it would not allow us to use multiple mixins on a single class (it is impossible to inherit from multiple classes in Kotlin).

A mixin will thus be created with an interface.

interface Counter {
    var count: Int
    fun increment() {
        println("Mixin does its job")
    }
    fun get(): Int = count
}

class MyClass: Counter {
    override var count: Int = 0 // We are forced to add the mixin's state to the class using it

    fun hello() {
        println("Class does something")
    }
}

This approach is more satisfactory than the previous one for several reasons:

The class using the mixin doesn’t need to implement the mixin’s behavior thanks to default methods
A class can use multiple mixins as Kotlin allows a class to implement multiple interfaces.
Each mixin creates a type that can be used to manipulate objects based on the mixins included by their class.

However, there remains a significant limitation to this implementation: mixins can’t contain state. Indeed, while interfaces in Kotlin can define properties, they can’t initialize them directly. Every class using the mixin must thus define all properties necessary for the mixin’s operation. This doesn’t respect the constraint that we don’t want the use of a mixin to force us to add properties or methods to the class using it.

We will thus need to find a solution for mixins to have state while keeping the interface as the only way to have both a type and the ability to use multiple mixins.

Delegation to Contain the Mixin’s State

This solution is slightly more complex to define a mixin; however, it has no impact on the class using it. The trick is to associate each mixin with an object to contain the state the mixin might need. We will use this object by associating it with Kotlin’s delegation feature to create this object for each use of the mixin.

Here’s the basic solution that nevertheless meets all the constraints:

interface Counter {
    fun increment()
    fun get(): Int
}

class CounterHolder: Counter {
    var count: Int = 0
    override fun increment() {
        count++
    }
    override fun get(): Int = count
}

class MyClass: Counter by CounterHolder() {
    fun hello() {
        increment()
        // The rest of the method...
    }
}

Final Implementation

We can further improve the implementation: the CounterHolder class is an implementation detail, and it would be interesting not to need to know its name.

To achieve this, we’ll use a companion object on the mixin interface and the "Factory Method" pattern to create the object containing the mixin’s state. We’ll also use a bit of Kotlin black magic so we don’t need to know this method’s name:

interface Counter {
    // The functions and properties defined here constitute the mixin's "interface contract." This can be used by the class using the mixin or from outside of it.
    fun increment()
    fun get(): Int

    companion object {
        private class MixinStateHolder : Counter {
            // The mixin's state can be defined here, and it is private if not also defined in the interface
            var count: Int = 0

            override fun increment() {
                count++
            }
            override fun get(): Int = count
        }

        // Using the invoke operator in a companion object makes it appear as if the interface had a constructor. Normally I discourage this kind of black magic, but here it seems one of the rare justified cases. If you don't like it, rename this function using a standard name common to all mixins like `init` or `create`.
        operator fun invoke(): Counter {
            return MixinStateHolder()
        } 
    }
}

class MyClass: Counter by Counter() {
    fun myFunction() {
        this.increment()
        // The rest of the method...
    }
}

Limitations

This implementation of mixins is not perfect (none could be perfect without being supported at the language level, in my opinion). In particular, it presents the following drawbacks:

All mixin methods must be public. Some mixins contain methods intended to be used by the class using the mixin and others that make more sense if called from outside. Since the mixin defines its methods on an interface, it is impossible to force the compiler to verify these constraints. We must then rely on documentation or static code analysis tools.
Mixin methods don’t have access to the instance of the class using the mixin. At the time of the delegation declaration, the instance is not initialized, and we can’t pass it to the mixin.

    class MyClass: MyMixin by MyMixin(this) {} // Compilation error: `this` is not defined in this context

If you use this inside the mixin, you refer to the Holder class instance.

Examples

To improve understanding of the pattern I propose in this article, here are some realistic examples of mixins.

`Auditable`

This mixin allows a class to "record" actions performed on an instance of that class. The mixin provides another method to retrieve the latest events.

import java.time.Instant

data class TimestampedEvent(
    val timestamp: Instant,
    val event: String
)

interface Auditable {
    fun auditEvent(event: String)
    fun getLatestEvents(n: Int): List<TimestampedEvent>

    companion object {
        private class Holder : Auditable {
            private val events = mutableListOf<TimestampedEvent>()
            override fun auditEvent(event: String) {
                events.add(TimestampedEvent(Instant.now(), event))
            }
            override fun getLatestEvents(n: Int): List<TimestampedEvent> {
                return events.sortedByDescending(TimestampedEvent::timestamp).takeLast(n)
            }
        }

        operator fun invoke(): Auditable = Holder()
    }
}

class BankAccount: Auditable by Auditable() {
    private var balance = 0
    fun deposit(amount: Int) {
        auditEvent("deposit $amount")
        balance += amount
    }

    fun withdraw(amount: Int) {
        auditEvent("withdraw $amount")
        balance -= amount 
    }

    fun getBalance() = balance
}

fun main() {
    val myAccount = BankAccount()

    // This function will call deposit and withdraw many times but we don't know exactly when and how
    giveToComplexSystem(myAccount)

    // We can query the balance of the account
    myAccount.getBalance()

    // Thanks to the mixin, we can also know the operations that have been performed on the account.
    myAccount.getLatestEvents(10)
}

`Observable`

The design pattern Observable can be easily implemented using a mixin. This way, observable classes no longer need to define the subscription and notification logic, nor maintain the list of observers themselves.

interface Observable<T> {
    fun subscribe(observer: (T) -> Unit)
    fun notifyObservers(event: T)

    companion object {
        private class Holder<T> : Observable<T> {
            private val observers = mutableListOf<(T) -> Unit>()
            override fun subscribe(observer: (T) -> Unit) {
                observers.add(observer)
            }

            override fun notifyObservers(event: T) {
                observers.forEach { it(event) }
            }
        }

        operator fun <T> invoke(): Observable<T> = Holder()
    }
}

sealed interface CatalogEvent
class PriceUpdated(val product: String, val price: Int): CatalogEvent

class Catalog(): Observable<CatalogEvent> by Observable() {
    val products = mutableMapOf<String, Int>()

    fun updatePrice(product: String, price: Int) {
        products[product] = price
        notifyObservers(PriceUpdated(product, price))
    }
}

fun main() {
    val catalog = Catalog()

    catalog.subscribe { println(it) }

    catalog.updatePrice("lamp", 10)
}

There is a disadvantage in this specific case, however: the notifyObservers method is accessible from outside the Catalog class, even though we would probably prefer to keep it private. But all mixin methods must be public to be used from the class using the mixin (since we aren’t using inheritance but composition, even if the syntax simplified by Kotlin makes it look like inheritance).

`Entity` / `Identity`

If your project manages persistent business data and/or you practice, at least in part, DDD (Domain Driven Design), your application probably contains entities. An entity is a class with an identity, often implemented as a numeric ID or a UUID. This characteristic fits well with the use of a mixin, and here is an example.

interface Entity {
    val id: UUID

    // Overriding equals and hashCode in a mixin may not always be a good idea, but it seems interesting for the example.
    override fun equals(other: Any?): Boolean
    override fun hashCode(): Int
}

class IdentityHolder(
    override val id: UUID
): Entity {
    // Two entities are equal if their ids are equal.
    override fun equals(other: Any?): Boolean {
        if(other is Entity) {
            return this.id == other.id
        }

        return false
    }

    override fun hashCode(): Int {
        return id.hashCode()
    }
}

class Customer(
    id: UUID,
    val firstName: String,
    val lastName: String,
) : Entity by IdentityHolder(id)

val id = UUID.randomUUID()
val c1 = Customer(id, "John", "Smith")
val c2 = Customer(id, "John", "Doe")

c1 == c2 // true

This example is a bit different: we see that nothing prevents us from naming the Holder class differently, and nothing prevents us from passing parameters during instantiation.

Conclusion

The mixin technique allows enriching classes by adding often transverse and reusable behaviors without having to modify these classes to accommodate these functionalities. Despite some limitations, mixins help facilitate code reuse and isolate certain functionalities common to several classes in the application.

Mixins are an interesting tool in the Kotlin developer’s toolkit, and I encourage you to explore this method in your own code, while being aware of the constraints and alternatives.

Fun fact: Kotlin has a trait keyword, but it is deprecated and has been replaced by interface (see https://blog.jetbrains.com/kotlin/2015/05/kotlin-m12-is-out/#traits-are-now-interfaces) ↩
Mixin Based Inheritance ↩
Classes and Mixins ↩
Object-Oriented Programming with Flavors ↩

DEV Community: Benoit Averty

Java Streams: The Gatherers API

Overview

What are Streams?

What exactly was missing?

The Gatherer Interface

Definition

Usage and utility methods

Examples

Compute Bitcoin Price All Time Highs

Calculating Median Item Prices in Parallel

Conclusion

An advanced Data Table with HTMX

Contents

The Project

Technical Stack

What This Article Is Not

The Demo

General Structure

Deleting an Element

Why swap:300ms?

Why use a form and a button of type submit?

Pagination / Sorting / Filtering

Sorting

Total Element Count / Event Synchronization

Experience Feedback

Web Fundamentals

Disadvantages

Your API Doesn’t Always Need to Be a Product

A Brief History of the Web

In the Beginning, Documents

jQuery, XHR, and the Rejection of Page Navigation

Single Page Applications: A Browser Within the Browser

APIs: Their Strengths and Limitations

You Don’t Always Need This

Go Back to Documents (Without Sacrificing Interactivity)

API as an Implementation Detail

The Back for Front Pattern

Conclusion

Writing Good Automated Tests with AI: A Case Study

Contents

Genesis

Experimentation

The Class to Test

Poor Tests

Why Is It So Bad?

A Slightly More Advanced Attempt

First Prompt

Adjustment

Generating Data for the First Test Case

Writing the Test Code

End of the Experiment

Results

Tips for Using AI Effectively

AI Works When the Generated Code is Localized

You Must Give the AI a Lot of Context

Be Explicit About Expectations

My Opinion

Appendices

The Complete ContentSelectionPipeline Class

The Tests Generated with the Poor Prompt

Implementing Mixins (or Traits) in Kotlin Using Delegation

Objective

Definition of the "Mixins" Pattern

Characteristics and Constraints

Implementation

Naive Approach by Composition

Use of Inheritance

Delegation to Contain the Mixin’s State

Final Implementation

Limitations

Examples

Auditable

Observable

Entity / Identity

Conclusion

The `Gatherer` Interface

Why `swap:300ms`?

Why use a form and a button of type `submit`?

The Complete `ContentSelectionPipeline` Class

`Auditable`

`Observable`

`Entity` / `Identity`