DEV Community: Stefano Fago

Clean Design, Strong Client: The way of the Elasticsearch's Java SDK

Stefano Fago — Sun, 21 Sep 2025 19:08:06 +0000

Java has a vast ecosystem of APIs, not all of which are effective or easy to learn. Developing a good API is not trivial: misdesigning key elements, defining simple abstractions, and threading models are among the themes that must be addressed. The official Elasticsearch Java SDK is a project with a design effort that has been made to address these elements.

Recently, this project surprised me, and I tried to examine the design ideas that make it interesting and effective, while also having some inevitable trade-offs to be aware of.

Code generation and the single source of truth

The Elastic Java SDK is not totally handwritten: it is generated from a canonical API specification, developed in Typescript, but there are also handcrafted artifacts. The client generation pipeline produces the Java model classes, builders, serializers, and the top-level namespace methods from this specification. This generated approach explains the consistency of naming, shape, and coverage across hundreds of endpoints and multiple language clients. The handcrafted (by Elastic engineers) parts are:

Transport integration with the Low Level REST Client (LLRC)
Core infrastructure: authentication, TLS, retries, test harnesses, Continuous Integration, JSON mapper setup.
API ergonomics: builder patterns, nullability conventions, naming.
ADR-driven decisions: Handcrafted parts are backed by Architecture Decision Records, documenting why certain design choices were made.

The Builder Pattern Done (Mostly) Right

The Elasticsearch SDK leans heavily on the Builder Pattern: it's expressed in a base ObjectBuilder<T> interface with a single build() method. You're building search queries, index mappings, bulk operations, etc. Without builders, it'd be chaos.

The SDK provides builder-lambda overloads (e.g., () -> ObjectBuilder<T>) so nested objects can be constructed inline with type-safe closures; these overloads are visible in the public Javadocs as function-typed builder setters. Internally, the generated builders inherit from ObjectBuilderBase, which enforces single-use via a _checkSingleUse() when a call to the build() method is needed.

Once you call build(), you're done. Reusing the builder is considered unsafe because internal structures, especially collections, may be shared between the builder and the built object. Mutating one could silently corrupt the other. So they close the gate after construction with a clean contract: configure it once, build, and forget. Every request and response class follows this pattern.

SearchRequest request = SearchRequest.of(s -> s
    .index("products")
    .query(q -> q
        .match(m -> m
            .field("name")
            .query("laptop")
        )
    )
);

At first glance, this resembles a lambda-heavy mess. But what it's doing is chaining a set of typed, nested DSL constructs. The of(...) is a static shortcut that instantiates the builder, applies a configuration function, and finalizes the build. You're writing declarative Java code that directly mirrors the Elasticsearch query DSL.

DSL-Style Lambdas That Actually Work

These lambdas provide an effective and IDE-friendly implementation, and make it easier to author deeply nested DSL structures while remaining strongly typed. Java isn't a functional language: different idioms are supported, but its nature is that of an object-oriented language. The SDK still manages to create an idiomatic syntax using lambdas, which leads to an easy declaration of the developer's intentions.

client.search(s -> s
    .index("products")
    .query(q -> q
        .bool(b -> b
            .must(m -> m
                .match(t -> t
                    .field("description")
                    .query("wireless")
                )
            )
        )
    )
);

Each lambda represents a nested configuration step, with type-safe closures. You're not passing in strings or magic maps. You're composing a statically typed tree. That's the win. It avoids a common anti-pattern: chaining .withX(), .setY() methods on dozens of mutable objects, with nulls lurking everywhere. Here, each level is scoped, focused, and immutable.

Tagged Unions and Type-Safe Variants

Let's talk polymorphism. Elasticsearch queries aren't flat structuresthey're variant types. A Query can be a MatchQuery, BoolQuery, RangeQuery, etc. The SDK models this with a Tagged Union pattern.

A tagged union is a data structure that can hold values of different data types, but only one at a time. It's similar to a regular union, but includes a tag (or discriminator) that shows which data type is currently stored. This tag allows type-safe access to the stored value, preventing accidental data misuse.

The client implements a generalized TaggedUnion pattern for many of these variant domains (queries, aggregations, analyzers, etc.). A TaggedUnion exposes the current kind and the strongly typed value; the builders expose explicit methods for each variant, making discovery and correctness easier. This pattern trades a small amount of indirection for compiler-enforced exhaustiveness for better discoverability in the IDE.

Each such union implements:

public interface TaggedUnion<Tag extends Enum<?>, BaseType> {
    Tag _kind();
    BaseType _get();
}

You inspect the _kind() to figure out what it is, then call _get() and cast it safely:

Query query = Query.of(q -> q
    .match(m -> m.field("title").query("elasticsearch"))
);

if (query._kind() == Query.Kind.Match) {
    MatchQuery match = (MatchQuery) query._get();
    // Work with match safely
}

This design solution allows reasoning with the union construct even without the syntactic support of the most recent versions of Java, while maintaining backward compatibility with those who already used Elastic products in previous installations. Starting with Java 16+, we have a taste of structural pattern matching that could be used in the future for an evolution of the SDK; for example, a first step could be:

switch (query._kind()) {
    case Match -> {
        MatchQuery match = (MatchQuery) query._get();
        // use match
    }
    case Term -> {
        TermQuery term = (TermQuery) query._get();
        // use term
    }
}

Modular by Design: The Namespace Client Pattern

Elasticsearch's API is wide. It has search, index management, mapping, ingest pipelines, security, cluster health, and more endpoints. Cramming all of this into one class would be a nightmare.

Instead, the SDK scopes things by domain:

ElasticsearchClient client = new ElasticsearchClient(transport);

client.indices().create(c -> c.index("catalog"));

client.search(s -> s
    .index("products")
    .query(q -> q.match(m -> m.field("name").query("laptop")))
);

Each sub-DSL node (indices(), search(), etc.) exposes only the relevant operations for its context. This is an effective support for both IDEs and our brains. It maps directly to the REST API structure (/_search, /_indices, etc.), making it easier to reason what goes where. It also makes the SDK highly maintainable. Adding new API groups becomes simple: no giant, god-interfaces to refactor.

Targeting a full-featured Java network SDK that must support:

per-connection configuration (auth, headers, timeouts)
multiple concurrent clients
good testability and DI
an ergonomic developer experience

The namespaced instance client pattern is a rational trade-off. You face slightly more ceremony and many small classes, but supporting modularity under the hood while giving a single discoverable root object to users.

Immutability and Transport Abstraction

Request objects don't change. Builders build once. You pass data, not behavior: the SDK is inherently thread-safe and predictable.

Transport concerns are separated from typed models: by default, the Java client delegates protocol handling to a RestClientTransport, which itself uses a low-level HTTP client (for example, the Apache HTTP client) to manage connections, pooling, retries, and node discovery. This separation enables the Java client to focus on typed request/response modeling and (de)serialization while the transport handles operational concerns.
The transport is pluggable, allowing the client's configuration to adapt to different HTTP stacks when needed.

ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
ElasticsearchClient client = new ElasticsearchClient(transport);

This separation of concerns makes the SDK easier to test, extend, and debug.

Could It Be Better?

The presence of many immutable elements impacts memory usage. Therefore, it becomes necessary to understand how to avoid building too many objects that push the developer to consider advanced constructs of the product, such as bundles, data sharing, use of temporal entities, and/or paging the data themselves.

A measurable trade-off of immutability and single-use builders is short-lived object allocation. In most applications, this overhead is negligible; in very hot loops or high-throughput pipelines, you should benchmark and, if necessary, build reusable immutable fragments or tune bulk/batching strategies. Also consider the overhead of serialization: the client provides hooks to use different JsonpMapper implementations (for example, Jackson-based mappers) if you need custom parsing or to send pre-serialized payloads.

Today, Java offers more constructs: records, structural pattern matching with sealed classes, and deconstruction of instanceof; this would have also meant continuous refactoring of the SDK by chasing Java features over time.

This SDK strikes a balance between being long-lasting and developer-friendly without altering the knowledge of previous Elastic products.

Conclusion

What can you learn from this SDK? Here is a summary table of the main concepts that could be useful for your next project. If you're building a client libraryor even just a public-facing APIyou could do worse than borrowing a few ideas from this one. No magic. Just good design... but beware: The Context is the King!

Pattern	What It Solves
Builder (Single-Use)	Safer immutable construction, avoids shared mutable state
Fluent Interface (Lambdas)	Declarative, nested, type-safe request definitions
Tagged Union Pattern	Models Elasticsearch variant types safely and explicitly
Namespace Client Pattern	Logical API grouping aligns with the REST structure
Transport Abstraction	Swappable HTTP and serialization layers
Functional Thinking	Fewer side effects, fewer temporary variables, more declarative code

Clean your Memory: From Finalize to Cleaner

Stefano Fago — Sun, 30 Mar 2025 16:27:12 +0000

Garbage collection in Java manages memory but does not clean up non-memory resources like sockets or file handles. Resource leaks may occur without proper management, leading to performance degradation or crashes. Java’s Cleaner API, introduced in Java 9, provides a modern and efficient mechanism for resource cleanup when objects are no longer reachable. It addresses the shortcomings of the deprecated finalize() method, offering a predictable and efficient way to manage non-memory resources: so let's take a short trip on clean memory from finalize to Cleaner API.

Why is the finalize() method deprecated/removed?

The finalize() method was initially introduced in Java to provide a way for objects to perform cleanup actions before they are garbage collected. It belongs to java.lang.Object and can be overridden by subclasses to release resources like file handles or network sockets. Why was this approach problematic?

Unpredictable Execution: The finalize() method is invoked at an undetermined time when GC decides to collect the object.
Performance Overhead: Objects with a finalize() method take longer to collect, as they must go through an extra GC cycle.
Memory Leaks: If an object is unintentionally retained (e.g., due to an exception in finalize()), it may never be garbage collected.
Finalization Queue Mechanism: finalize() execution happens in a separate thread(the Finalizer Thread), which can lead to thread contention and delays.

How does Cleaner relate to Java Reference classes?

In Java, there are four types of references differentiated by the way by which they are garbage collected:

Strong References: objects having an active, strong reference are not eligible for garbage collection. The object is garbage collected only when the variable, which was strongly referenced, points to null.
Weak References: the objects referenced by a weak reference do not prevent their referents from being made finalizable, finalized, and reclaimed.
Soft References: the objects which are being referenced by soft reference, even if the object is free for garbage collection, it’s not garbage collected until JVM needs memory badly
Phantom References: the objects referenced by phantom references are eligible for garbage collection, but before removing them, the JVM puts them in the "reference queue".

The logic that guides how they are collected is always related to the concept of reachability, as described in the related Javadoc. Java Reference classes are not easy to use, and the resulting code is sometimes complicated. We must also consider that in the case of a Phantom Reference, once the Referent is registered, the reference always returns null, making it seem useless, but it is not!

Here’s an example of PhantomReference:

import java.lang.ref.PhantomReference;
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.util.HashMap;
import java.util.Map;

public class UsingPhantomRef {

    static class Resource {
        public void cleaning() {
            System.out.println("cleaning");
        }
    }

    record ResourceHolder(Resource resource) {}

    private static final Map lookup = new HashMap();

    private static final ReferenceQueue queue = new ReferenceQueue();

    public static void main(String[] args) {

        var holder = new ResourceHolder(new Resource());
        lookup.put(new PhantomReference(holder, queue), holder.resource());

        holder = null;
        System.gc();

        Reference element = null;
        while ((element = queue.poll()) == null) {
            System.out.println("wating for GC");
        }
        System.out.println("GCollected!");
        lookup.remove(element).cleaning();
    }
}

Cleaner shares similarities with Java’s reference classes, but it is more efficient for managing external resources. While PhantomReference enables cleanup logic, Cleaner adds an abstraction layer, making it easier to implement and manage. Internally, Cleaner uses a phantom reference behind the scenes to detect when an object becomes unreachable.

Here’s an example of Cleaner:

import java.lang.ref.Cleaner;

public class BasicCleanerExample {

    private static final Cleaner cleaner = Cleaner.create();

    static class CleaningAction implements Runnable {
        @Override
        public void run() {
            System.out.println("Resource cleaned up!");
        }
    }

    static class ManagedObject {
        private final Cleaner.Cleanable cleanable;

        ManagedObject() {
            cleanable = cleaner.register(this, new CleaningAction());
        }
    }

    public static void main(String[] args) {
        new ManagedObject();
        System.gc();
        pause();
    }
}

Behind the scenes of Cleaner

The implementation behind Cleaner is not trivial. It leverages a combination of the Java PhantomReference and a background daemon thread. Let’s explore the inner workings:

Registration with Cleaner: an object registered with a Cleaner is associated with a Cleanable task. The object is essentially "watched" by a background daemon thread.
Phantom Reference Management: Under the hood, Cleaner uses a PhantomReference to track the object’s reachability.
Daemon Thread for Cleanup: The Cleaner framework uses a dedicated daemon thread (normally named Common-Cleaner) that monitors registered objects. Once an object becomes unreachable, the cleanup task for that object is queued for execution in this background thread.

The following snippet uses a Cleaner to manage files.

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.lang.ref.Cleaner;

public class FileCleanerExample {

    private static final Cleaner cleaner = Cleaner.create();

    static class FileResource implements Runnable {

        private final File file;

        FileResource(File file) {
            this.file = file;
            System.out.printf("resource create with temporary file: %s%n", file.getAbsolutePath());
        }

        @Override
        public void run() {
            if (file.exists()) {
                System.out.printf("temporary file deleted: %s%n",file.delete());
            }
        }
    }

    static class ManagedFile {
        private Cleaner.Cleanable cleanable;

        ManagedFile(String path) throws IOException {
            var file = new File(path);
            cleanable = cleaner.register(this, new FileResource(file));
            new FileWriter(file).write("Temporary data");
        }
    }

    public static void main(String[] args) throws IOException {
        var managed = new ManagedFile("temp.txt");
        managed = null;
        System.gc();
        pause();
    }
    // ...
}

Cleaner vs. try-with-resources

The example with a file might suggest a more idiomatic solution using the try-with-resources construct. It may be a real solution but is less so if we consider particular resources such as images or memory buffers directly mapped to memory. As a last note, we also need to consider that Autocloseable can be used with Cleanable, invoking the clean() method from the close() one, having the same effect as a GC operation. Here’s a dedicated example.

import java.lang.ref.Cleaner;

public class CleanerWithCloseExample {

    private static final Cleaner cleaner = Cleaner.create();

    static class CleaningAction implements Runnable {
        @Override
        public void run() {
            System.out.println("Resource cleaned up!");
        }
    }

    static class ManagedObject implements AutoCloseable{
        private final Cleaner.Cleanable cleanable;

        ManagedObject() {
            cleanable = cleaner.register(this, new CleaningAction());
        }

        @Override
        public void close(){
            System.out.println("close invoked!");
            cleanable.clean();
        }
    }

    public static void main(String[] args) {
        var object = new ManagedObject();
        try(object){
            System.out.printf("using: %s%n",object);
        }
    }
}

The following table allows us to summarize the tools presented and choose the one that best suits our purposes.

Feature	try-with-resources	Cleaner
Resource Type	Autocloseable	It also works with no Autocloaseable object
Cleanup Timing	Immediate	Deferred, asynchronous
Use Case	When precise cleanup timing is required	When working with external resources or objects without explicit close methods
Overhead	Minimal	Higher due to the background thread

Avoid overusing Cleaner

Using Cleaner is more straightforward than using references. Still, we must exercise caution when using this kind of resource: only use Cleaner when the runtime can’t release resources via try-with-resources or explicit close() calls. We also need to consider that we create new elements, and sometimes it’s better to use one Cleaner for multiple resources or to listen to the cleaning action. More aspects need our attention, as for cleaning action, following some rules:

Avoid using lambda because it risks capturing the object reference by referring to fields of the object being cleaned, preventing the object from becoming phantom reachable.
Cleaning actions can be invoked concurrently with other cleaning actions. They should be swift to execute and not block: it may delay processing other cleaning actions registered to the same cleaner.
We can also consider executing a cleaning action and delegating it to another thread pool, but this idea can involve more complexity and not solve concurrency issues.

Conclusion

Java’s Cleaner API provides a modern, efficient approach to resource management, addressing the limitations of finalize(). By leveraging Cleaner, developers can ensure that non-AutoCloseable resources, such as native memory and caches, are properly cleaned up when no longer needed. However, you should always prefer try-with-resources when possible for deterministic resource management.

By understanding when and how to use Cleaner, developers can write more reliable, high-performance, and memory-efficient Java applications. Mastering these best practices will be essential for building robust, scalable software as Java evolves.

Further Resources

The FizzBuzz: a branchless version with Java

Stefano Fago — Sat, 22 Mar 2025 19:38:27 +0000

Let dissect a Java program that implements the FizzBuzz problem with a branchless style, using a mix of bitwise operations, lambdas, and an optimized loop. Let's break it down step by step:

Overview of FizzBuzz Rules

Print numbers from 1 to 100.
If a number is divisible by 3, print "Fizz".
If a number is divisible by 5, print "Buzz".
If a number is divisible by both 3 and 5, print "FizzBuzz".
Otherwise, print the number itself.

Code Breakdown

1. Defining an Interface for Lambda Expressions

interface Provider {
    String value(int i);
}

An interface Provider is needed since generic array are not possible, so Function or other functional interface are non allowed.
It has a single method value(int i), which will be implemented using lambda expressions.

2. Creating an Array of `Provider` Lambdas

Provider [] provider = {i->String.valueOf(i),i->"",i->""};

This array holds three lambda expressions:
- i -> String.valueOf(i): Returns the number as a string.
- i -> "": Returns an empty string.
- i -> "": Another empty string (used later for Fizz/Buzz handling).

3. Defining Text Segments

String [] textSegment = {"", "Fizz","FizzBuzz","Buzz"};

This array maps to different text outputs:
- textSegment[0] = "" (for numbers that aren't Fizz or Buzz)
- textSegment[1] = "Fizz" (for numbers divisible by 3)
- textSegment[2] = "FizzBuzz" (for numbers divisible by both 3 and 5)
- textSegment[3] = "Buzz" (for numbers divisible by 5)

4. Using StringBuilder for Efficient String Concatenation

StringBuilder sb = new StringBuilder(3000);
String newline = System.lineSeparator();

StringBuffer is used for efficient string concatenation.
newline stores the platform-specific line separator.

5. Main Loop with Bitwise Operations

for (int i = 1, a = 0, b = 0, providerIndex = 0, segmentIndex = 0; 
     i <= 100; 
     i++,
     a = ((528 >> i % 15 - 1) & 1) << 2,
     b = ((-2128340926 >> ((i % 15) << 1)) & 3) << 2,
     providerIndex = (b - a) >> 2,
     segmentIndex = (a + b) >> 2)
{
    sb.append(textSegment[segmentIndex])
      .append(provider[providerIndex].value(i))
      .append(newline);
}

This loop iterates from i = 1 to 100 and performs the following calculations:

(1) Calculating `a` - 5 Multiples

a = ((528 >> i % 15 - 1) & 1) << 2

528 (binary: 100001000010000) is a bitmask that identifies numbers divisible by 5.
(i % 15 - 1) is used to shift the bitmask.
& 1 extracts whether the bit is set.
<< 2 makes sure that if it's Fizz, a is 4, otherwise a is 0.

(2) Calculating `b` - 3 and 15 Multiples

b = ((-2128340926 >> ((i % 15) << 1)) & 3) << 2

-2128340926 (binary: 100000010000001000000100000010) is a bitmask that identifies numbers divisible by 3 or 15.
((i % 15) << 1) is used to shift the bitmask.
& 3 ensures only relevant bits are taken.
<< 2 scales b properly.

(3) Determining `providerIndex`

providerIndex = (b - a) >> 2

providerIndex determines whether to print the number (provider[0]) or an empty string (provider[1] or provider[2]) since provides the values: 0,1,2.

(4) Determining `segmentIndex`

segmentIndex = (a + b) >> 2

segmentIndex selects the correct FizzBuzz text from textSegment since provides the values: 0,1,2,3.

6. Appending the Result

sb.append(textSegment[segmentIndex])
  .append(provider[providerIndex].value(i))
  .append(newline);

The correct Fizz/Buzz text (textSegment[segmentIndex]) is added.
The number (or an empty string) is determined by provider[providerIndex].
A newline is appended.

7. Printing the Final Output

System.out.println(sb);

The entire FizzBuzz output is printed at once.

Key Takeaways

Uses bitwise operations for efficient FizzBuzz checking.
Avoids multiple if-else conditions.
Optimizes string concatenation with StringBuilder.
Lambda functions simplify number/string selection.

There are more solutions about FizzBuzz, and maybe the code can be optimized further, but this code is funny and didactic enough to implement FizzBuzz in Java!

Experimenting around FP in PHP

Stefano Fago — Mon, 18 Dec 2023 13:23:01 +0000

Warn: in this Post, it's tried to use as much typing as possible also if sometimes this means to take some distance from official documentation

The evolution of the PHP language has had an effective contribution from some powerful specifications:

These are now enriched by new RFC:

Note: These RFCs make possible a lot of different constructs that go further than what will be exposed in this Post: remember that the "White Rabbit hole" is deep!

Callable, Anonymous Function, Closure, and FP!

Callables are generally used as the main type hint to define callbacks: a summary of how they can be defined/used can be seen in the official documentation. Introducing the __invoke function, even at the interface or anonymous class level, makes it possible to manage an object as a function transforming it into a Callable.


class Butterfly {
  public function __invoke():void {echo "flying...";}
}
$butterfly = new Butterfly();
$butterfly(); // flying...

Closure is the class used to represent anonymous functions: this kind of construct, also in the form of arrow function, is really useful not only as a callback, in place of a callable, but also more usage since the ability to capture a different scope or to simulate FP constructs such as currying like in the following code:


class Person{
  private function __construct(
                     public readonly string $name, 
                     public readonly string $familyName,
                     public readonly int $age){}

    public static function create():\Closure
    {
     return 
          static fn(string $name):\Closure
          => static fn(string $familyName):\Closure
            => static fn(int $age):Person
             => new Person($name,$familyName,$age);
    }
}

$person = Person::create()('Frank')('Jamison')(50);

The introduction of First-Class Callable Syntax makes it possible to create Closure from any kind of function, defining a sort of function reference usable in places different from the definition (what follows reuse the example seen above)


$personBuilder = Person::create(...);
$frank = $personBuilder()('Frank')('Jamison')(45);
$john= $personBuilder()('John')('Smith')(40);

Arrays, arrays functions, and the anonymous functions

The use of anonymous functions is particularly useful when we refer to native functions on arrays: the potential is notable and makes the use of loops superfluous as in the following code.

$stock = [
    ['name' => 'Phone', 'qty' => 0],
    ['name' => 'Notebook', 'qty' => 1],
    ['name' => 'SDD Drive', 'qty' => 0],
    ['name' => 'HDD Drive', 'qty' => 3]
];

$inStock = array_filter($stock, static fn($item) => $item['qty'] > 0);

Native functions on arrays bring out the functional world when it comes down to it to the fold operation (the array_reduce function) that allows us to be a point on which to build other fundamental functions such as map and filter:


/**
 *
 *@template T
 *@template R
 *@param array<T> $data
 *@param callable(T):R $mapper
 *@return array<R>
 *
 */
function map(array $data, callable $mapper):array
{
 return array_reduce($data, 
/** @param T $element */ 
function(array $accumulator, $element) use ($mapper)
             {
              $accumulator[] = $mapper($element);
              return $accumulator;
            }, []);
}

/**
 *
 *@template T
 *@param array<T> $data
 *@param callable(T):bool $predicate
 *@return array<T>
 *
 */
function filter(array $data, callable $predicate):array
{
 return array_reduce($data, 
/** @param T $element */ 
function(array $accumulator, $element)use($predicate){                    
      if($predicate($element)){   
        $accumulator[]=$element;
      } 
 return $accumulator;},[]);
}

The improved performance on arrays, starting from PHP 7.X, and the handling of anonymous functions are convenient from both a performance and memory consumption point of view although the practice of only using arrays sometimes doesn't make you consider the multiple benefits of alternative data structures.

Anonymous functions and closures are easy to distinguish given the use clause that allows you to capture elements from outside the given anonymous function. The value of the captured element is defined when the closure is declared and cannot be changed.


class Product{ ... }    

final class PriceRange {

   private function __construct(private float $minimumValue,
                                private float $maximumValue){}

    public static function of(
                     float $minimumValue, 
                     float $maximumValue): PriceRange{
     return new PriceRange($minimumValue,$maximumValue);
    }

    public function priceFilter():\Closure{
        return fn(Product $value)=> 
               $value->price() >= $this->minimumValue 
                             &&
               $value->price() < $this->maximumValue;
    }
}

/** @var array<Product> $prodcuts */
$products = [ ... ];

$range = PriceRange::of(5.0,10.0);

array_filter($products, $range->priceFilter());

Closures are still handy in design where you want to isolate the visibility of data, always in the light of the FP paradigm, as in the following example.


/** @return \Closure():int */
function rndNumber():\Closure{
    return fn()=> rand(0,getrandmax()-1);
}

/** @return \Closure():int */
function rndConstant():\Closure{
    $constant = rand(0,getrandmax()-1);
    return fn()=> $constant; 
}

$rnd1 = rndNumber();
$c1 = rndConstant();
$c2 = rndConstant();

echo $rnd1(); // 505035335
echo $rnd1(); // 1353685165
echo $c1();   // 2030702172
echo $c1();   // 2030702172

These possibilities are also useful in approaching internal domain-specific languages (DSL) or adopting the higher-order function (in FP slang): the Loan Pattern and the Execute Around Pattern are examples of this concept.

//
//Example of Execute-Around...
// ...also used in DSL definition for nested-dsl
//
 MailSender::Send(
        fn(Mail $config):Mail=>
            $config
                    ->From("john.black@kmail.com")
                    ->To("jack.white@jmail.com")
                    ->Subject("Test message")
                    ->Body("Hello World!")
        );

//
// Example of Load Pattern
//
class ToyBox{

   ...

    /**
     * 
     * @param \Closure(ToyBox):ToyBox $boxFillerPolicy
     * @param \Closure(ToyBox):ToyBox $playActions
     * @return \Closure():void
     */
    public static final function play(\Closure $boxFillerPolicy, \Closure $playActions):\Closure
    {
        return function()use($boxFillerPolicy,$playActions):void{   
                  $box = new ToyBox();
                  $box->Open();
                  try{
                      $box = $boxFillerPolicy($box);
                      $box = $playActions($box);
                  }finally{
                      $box->Close()
                          ->CleanUpToys();
                  }
        };
    }

    ...

    public function nextToy():ToyBox{ ... }
    public function addToy(string $toyName):ToyBox{ ... }    
}

function Usage():void{
        $refill = fn(ToyBox $box):ToyBox
           =>$box
             ->addToy("lego")
             ->addToy("mechano")
             ->addToy("laser"); };

        $play = fn(ToyBox $box):ToyBox{
          => $box
            ->nextToy()
            ->nextToy()
            ->nextToy(); };

        $playSession = ToyBox::play($refill, $play);       
        $playSession();

The Scope element in Closures

Closures can refer to different scopes than those in which they were created thanks to the bind functions. The possibilities of closures become considerable and they could replace the use of Reflection since they can access the private state of objects: this, however, has limitations in the presence of static analysis infrastructures such as PHPStan and PSALM which highlight abuses of closures as in the following example.


class SimpleClass { 
    private int $privateData = 2; 
}

$simpleClosure = function():int {return $this->privateData;};
$instance = new SimpleClass();
$resultClosure = \Closure::bind(
                       $simpleClosure, 
                       $instance, 
                       SimpleClass::class);

echo $resultClosure==false?-1:$resultClosure();

...
//
//PSALM Output
//  
// ERROR: InvalidScope - Invalid reference to $this in a 
//          non-class context
// INFO: MixedInferredReturnType - Could not verify the  return 
// type 'int' for...
//

It's however possible to find ways to overcome these problems and exploit appropriately the potential of the closures from the outside of a class. A possible trick is to define a bridge operation (a protected function, also static if needed) to use when the Closure is created, as follows:


class SimpleClass {
    private int $privateData = 2;
    protected final function data():int{ 
          return $this->privateData;
    }    
}

$instance = new SimpleClass();

// @phpstan-ignore return.unusedType
$accessTo = fn (string $bridgeOperation): ?\Closure
   =>
    \Closure::bind(
        function(SimpleClass $instance) use ($bridgeOperation): int{
            /** @var callable():int $operation */ 
            $operation = [$instance,$bridgeOperation];
            return $operation();
        },
        NULL,
        SimpleClass::class);

/** @var null|\Closure(SimpleClass):int $resultClosure **/
$resultClosure = $accessTo("privateData");

echo ($resultClosure===null?-1:$resultClosure($instance)); // 2

This approach seems more verbose but we can make it reusable and it passes static analysis given also more strong typing. It's also an example of how we can approach the FP concept of Optics.

Conclusion

The importance of the exposed specifications is not only in having introduced types for the different elements but it's also in the new design possibilities. They have also brought PHP even closer to natively approaching Functional Programming: so it's important to study and experiment also with static analysis enabled! It is worth noticing that with PHP you can approach FP both in a more function-oriented way or in a more object-oriented way using generics and advanced techniques thanks also to PHPStan or PSALM.

To Go Furter

More elements need to be considered, here not exposed as for the design consequences of read-only constructs or the Generators, but also if not adopted, it's useful to consider these concepts to enrich our ability to design Clean Software!

Other resources allow the adoption of functional idioms/constructs and to study the concepts and approaches presented in this post: so... Happy Learn!

Designing with PHP 8.1 Enumerations

Stefano Fago — Sun, 03 Dec 2023 17:08:36 +0000

PHP 8.1: The Rise of Enumeration

PHP 8.1 introduces the syntax for the Enumeration. I was curious about the argument and before PHP 8.1 there were more experimental implementations but no direct support. Now Enumeration is here and is a native construct that makes code much more expressive and gives a new way to model software design.

The Enumeration in PHP now has an implicit family where the basic UnitEnum with a minimal contract:

interface UnitEnum
{
  /* Functions */
  public static cases(): array
}

The BackedEnum extends the UnitEnum making it possible to relate primitive values (int or string) to enumeration instances.

interface BackedEnum extends UnitEnum
{ 
   /* Functions */
   public static from(int|string $value): static
   public static tryFrom(int|string $value): ?static

   /* Inherited Functions */
   public static UnitEnum::cases(): array
}

Note:
These elements are interfaces but not in DEV Space since are used by the PHP Engine, introduced for type checking.

I've always seen Enumeration as Domain Symbols ( a limited set of symbols) but the PHP RFC is enough to define some use cases that can be useful to express more design.

PHP Enumeration: some Anti-Pattern!

Before continuing, I want to present some bad ideas in using Enumerations but remain at a high level of abstraction.

The Confusing Constant

It's possible to declare some constants inside an enumeration but the declared constant is not a case of the enumeration. It can be useful to have some edge values inside the enumeration but we need, however, remember to avoid confusing the Devs. Consider also that constants can hide callables!

enum Size
{
    case Small;
    case Medium;
    case Large;


    public const Huge = self::Large;
    public const Max_For_Large_Size = 100;
    public const Cases = ['cases',Size::class];
}

...

var_export(Size::Huge); // alias for Size::Large
var_export(Size::Max_For_Large_Size); // 100
var_export(Size::Cases()); // array (0 => Size::Small,1 =>Size::Medium,2 => Size::Large,)

The Empty Enumeration

It's possible to write an enumeration without any case ( also backed )

enum TheEmptyEnum: int {}

So we have TheEmptyEnum::cases as an empty array while, in our example, no instances are available, and TheEmptyEnum::tryFrom returns null!

The Commodity Enumeration

Since Enumeration can host function and since the Empty Enumeration, it's possible to write an enumeration that plays the role of a Commodity Class.

enum Calculator
{
    static function sum(int $a,int $b):int
    {
        return $a+$b;
    }
}


Calculator::sum(1,2); // 3

The Singleton Anti-Pattern(?)

It is possible to use Enumeration to define the concept of the Singleton.

enum Runtime
{
    case INSTANCE;
    function elapsed():float
    {
        ...
    }   
}
...
Runtime::INSTANCE->elapsed();

Enumerations, Template & Constant Expression

Nowadays, enumeration can't be defined as generics by using docblock template and also if a case can be used as constant, it can't be some element coming from the enumeration function.

Interesting Design Aspect of Enumerations

We can have the following interesting usage of enumeration remembering always the concept that they are enumerable elements.

The Interface Friend

An enumeration can implement an interface; it's interesting to notice that, more than the obligation of the contract, we obtain protection from the fact that what we consider enumerable today, tomorrow will be not but we don't need to change the signature of the functions.

The Trait Friend

An enumeration is a more constrained environment and using traits can be a way to enhance some enumeration abilities, coming from the new PHP 8.X features, as we can see in these projects:

The Enumeration State Machine

A state machine has an enumerable set of states so it's possible to use the new RFC and a UnitEnum to define a functioning State Machine


enum PeachesPipe
{
    case WASH;
    case PEEL;
    case CUT;
    case PUT_IN_JAR;
    case READY;

    function process():PeachesPipe
    {
      return [$this,$this->name]();
    }


    private function WASH():PeachesPipe
    {
        echo $this->name.PHP_EOL;
        return PeachesPipe::PEEL;
    }

    private function PEEL():PeachesPipe
    {
        echo $this->name.PHP_EOL;
        return PeachesPipe::CUT;
    }

    private function CUT():PeachesPipe
    {
        echo $this->name.PHP_EOL;
        return PeachesPipe::PUT_IN_JAR;
    }

    private function PUT_IN_JAR():PeachesPipe
    {
        echo $this->name.PHP_EOL;
        return PeachesPipe::READY;
    }
}
...
$pipeline = PeachesPipe::WASH;

while($pipeline !== PeachesPipe::READY)
{
    $pipeline = $pipeline->process();
}
// WASH
// PEEL
// CUT
// PUT_IN_JAR

The Enumeration Strategy Pattern

We can use an enumerable set of algorithms related to a specific Domain, defining related strategies inside an enumeration.


enum Currency
{
    case EUR;
    case USD;
}


final class Item implements Stringable
{
  function __construct(public readonly string $id, 
               public readonly int $quantity, 
               public readonly int $price, 
               public readonly Currency $currency){}

   function __toString():string
   {
       return '[ id:'.$this->id.
              ', quantity: '.$this->quantity.
              ', price:'.$this->price.' '.$this->currency->name.' ]';
   }
}


final class Basket implements Stringable
{
    /** @param list<Item> $items **/
   function __construct(private array $items){}

    /** @return list<Item> **/
   function items():array
   {
      return $this->items; 
   }

   function __toString():string
   {
       return '(Basket: '.var_export($this->items,true).')';
   }
}


enum DiscountType: int
{
    case BRONZE=5;
    case SILVER=10;
    case GOLD=15;

    static function apply(Basket $basket, DiscountType $discount):Basket
    {
        $factor = $discount->value/100;
        $items = array_map(
                   fn(Item $item)=>
                         new Item($item->id,
                                  $item->quantity,
                                  ($item->price - (int)($item->price*$factor)),
                                  $item->currency),
                  $basket->items()
                );
       return new Basket($items);
    }

}


$items = array(
       new Item('az123',2,100,Currency::EUR),
       new Item('azBCV',5,200,Currency::EUR),
       new Item('azKJI',1,150,Currency::EUR),
    );

$basket = new Basket($items);
$basket = DiscountType::apply($basket,DiscountType::BRONZE);

The Enumeration Factory Pattern

An enumerable set of elements can be created from a function hidden in a specific backed enumeration


final class Topping implements Stringable
{
    function __construct(public readonly string $name){}

    function __toString():string
    {
        return 'Topping('.$this->name.')';
    }
}


final class Cake implements Stringable
{
    function __construct(public readonly CakeType $type, 
                         public readonly Topping $top, 
                         public readonly int $weight){}

    function __toString():string
    {
        return 'Cake('.$this->type->name.','.$this->top.', '.$this->weight.')';
    }
}

interface CakeFactory
{
    function create():Cake;
}

enum CakeType: int implements CakeFactory
{
    case CHOCOLATE=2000;
    case VANILLA=2200;
    case FROSTED=2500;

    function create():Cake
    {
        return match($this)
        {
         CakeType::CHOCOLATE=>new Cake($this,new Topping('Glazed Choco'),$this->value),
         CakeType::VANILLA=>new Cake($this,new Topping('Glazed White'),$this->value),
         CakeType::FROSTED=>new Cake($this,new Topping('Cherries'),$this->value),
        };
    }
}

...
echo CakeType::VANILLA->create(); // Cake(VANILLA,Topping(Glazed White), 2200)

It's also possible to introduce some laziness using closure instead of simple functions and playing with the concept of private const (callable) and/or supplier closure.

The Processor Set Pattern

An enumerable set of processors can be expressed thank to an enumeration also using a touch of FP.


enum Operation: int
{
   case SUM=1;
   case SUB=2;
   case MUL=3;
   case DIV=4;

   /** @return callable(Operation):callable(int,int):int **/
   static function processorsMap():callable
   {
      $processors = array(
        Operation::SUM->value =>fn(int $a, int $b):int=>$a+$b,
        Operation::SUB->value =>fn(int $a, int $b):int=>$a-$b,
        Operation::MUL->value =>fn(int $a, int $b):int=>$a*$b,
        Operation::DIV->value =>fn(int $a, int $b):int=>(int)($a/$b),
      );

      /** @psalm-suppress PossiblyUndefinedIntArrayOffset */ 
     return fn(Operation $opKey)=>$processors[$opKey->value];
   }
}
...
$processors = Operation::processorsMap();
echo $processors(Operation::SUM)(1,3); // 4

The Conditional Processor Pattern

We can have an enumerable set of flags conditioning a specific processing operation and we can use an enumeration for that.


enum EventType: int
{
  case CLICKED=1;
  case FOCUSED=2;
  case EXECUTED=4;
  case BLURRED=8;

  static function eventsCombining(EventType ... $type): int
  {
     $values = array_map(fn(EventType $eventType)=>$eventType->value,$type);
     return (int) array_reduce($values, 
                               /** 
                                * @param int|null $carry
                                * @param int $val
                                * @return int */ 
fn($carry,$val)=> $carry===null?$val: ($carry | $val)); 
  }
}
...
echo 'events set: '.EventType::eventsCombining(
                        EventType::CLICKED,  
            EventType::FOCUSED,
                    EventType::EXECUTED,
                    EventType::BLURRED)
                     .PHP_EOL; // events set: 15

An interesting and more complex example comes from this project:

https://github.com/framjet/php-enum-bitmask

The Enumeration Random Generator

It's also possible to extract randomly from enumeration instances as follows:


 /**
  * @template T of \UnitEnum
  * @param class-string<T> $enumType
  * @return callable():T
  **/
function randomEnumGenerator(string $enumType):callable
{
 /** @var array<T> $cases **/ $cases = [$enumType,"cases"]();
 $len = count($cases)-1;
 return fn()=> $cases[rand(0,$len)];
};

enum Direction
{
    case North;
    case South;
    case East;
    case West;
}
...
$gen = randomEnumGenerator(Direction::class);
var_export( $gen()); // Direction::West

Note: Here are used Template e Template Class to force typing

To Go Further

More are not happy with this kind of design but I hope You can find some useful use cases. We need always to use some discipline, and remember the real meaning of an enumeration!

Lists: do you know the nature of yours? The strange story of a data container in Java

Stefano Fago — Thu, 23 Nov 2023 16:50:58 +0000

Foreword

Lists are among the most used data structures in various programs but, in Java, the choices are limited even if the evolution of the language has led to new elements. To date, the main type of list used is the ArrayList. Other types of lists would also be present (such as SkipLists) but they have been used for different purposes and, given their nature, to effectively build other types of data structures. Still others such as elements deriving from the Dynamic Arrays are hidden in the internals of the JDK!

How many lists we can find in Java?

The concept of List is part of the Java Collection Framework (JCF) and starts from the List interface that is a direct extension of two concepts: Collection and Iterable (so a provider of external Iterator but also an entry point to stream).

We can briefly summarize JCF with the following image:

We have public classes that express the main list data structure available in Java:

Vector (legacy data structure)
Stack (legacy data structure and direct subclass of Vector)
ArrayList
LinkedList
CopyOnWriteArrayList

The most used list in Java is ArrayList which makes it possible to express the concept of the list based on an array implementation: this realization is more effective than the LinkedList as stated in different tests ( caused also by CPU and Memory mechanics)
An ArrayList makes it possible to have random access to the element of the list since it's also an implementation of interface RandomAccess; this implies that having a reference to a List, we can test if we can use the get([index]) method or not. ( e.g. ArrayList vs LinkedList)

private static void ifNullThrows(Object data,int idx)
{
  if (data == null) {
      throw new IllegalArgumentException(
         "Collection contains null values "
         +(idx==-1?"":" at idx: "+idx));
  }  
}

public static final void checkNullElements(
                           Collection<?> collection)
{
    if (collection instanceof RandomAccess && 
        collection instanceof List) {
        final List<?> list = (List<?>)collection;
        for (int i = 0,
             size = list.size(); 
             i < size; 
             ifNullThrows(list.get(i),i),
             i++);
        return;  
    } 
    for (final Object element : collection) {
        ifNullThrows(element,-1);
    }
}

In how many ways You can build a List?

At the moment I'm writing, if we talk about the List interface and we don't consider public classes, we can obtain an instance using different invocations as follows:

Arrays.asList
Collections.unmodifiableList
Collections.nCopies
Collections.emptyList / Collections.EMPTY_LIST
Stream.toList
Collectors.toUnmodifiableList
List.of

Once You have a reference List to manage how can You say what characteristics are owned?
Or more simply how can You say if the List can mutate or not? And How? And...
When designing software, what kind of list You'll return, and how will be created?

Why is it a Problem?

The first problem is at the level of Type System, given that a situation more correct would allow us to distinguish through the Collection Type which abstraction we are operating with, species if definable as mutable or immutable.
The JCF was born at a time when great care was taken to offer immediate operational data structures, and with attention to performance, but with less attention to constructs or uses that are now seen as common.
These concepts have been taken up by other infrastructures from which we certainly cannot fail to mention: Eclipse Collection, Guava Collections, and VAVR.

A second aspect arises from the fact that immutable data structures generally support a fluent API; this style of API allows for better development in the presence of functional programming constructs, as in the following example:

static List<String> flattenListsWithMilestone(
                                 List<List<String>> data,
                                 String milestone)
{
  return data.stream()
             .map(list->
              { 
               list.add(milestone);
               return list;
              })
         .flatMap(List::stream)
         .collect(toList());     
}

Even if this code is a bit contrived, it demonstrates how, without fluent API, as for Scala, we are forced to use lambda with a body, a style which is generally not a good practice and which does not allow us to make the code more readable thanks, also, to the possible use of a reference method. ( Oh Yes, You can however use a private commodity method as a surrogate!)

Another problem is that the differentiation of data structures allows the implementation of specific design and optimization strategies for the given structure. This aspect is evident if we consider the new lists that derive from the List.of factory methods. The JCF allows the creation of any new data structure, in our case a list, but always of a mutable nature rather than a general vision.

Mutable, Immutable, Not Modifiable List

It's easy to find out that mutable lists are ArrayList, LinkedList, and with specific concurrency support, Vector, Stack, and CopyOnWriteArrayList but what can say of the other kind of List<T>?
As a preamble about Immutability, we need to accept that in Java we have unmodifiable lists more than properly immutable ones, and than things can be difficult to change. So let's explore the ways of creating List with the different invocations!

Arrays.asList returns a fixed-size list backed by the specified array. The returned list is serializable and implements RandomAccess. Changes to the returned list write through to the array. It seems unmodifiable but, also if You can't use add method, You can use List.set changing element at the given index. The inner class used is called ArrayList but is not the public one as the following signature:

private static class ArrayList<E> extends AbstractList<E> implements RandomAccess, java.io.Serializable

We also need to notice that the generic signature of asList create problems/ambiguities with arrays of primitive types, like for int[] that can't be passed to Arrays.asList to obtain a List<Integer>. In the future, the support of primitive in generics will change more things but today a safer way to bypass the problem is:

int[] numbers = new int[]{1, 2, 3, 4, 5};
   List<Integer> numberList = 
        Arrays.stream(numbers)
              .boxed()
              .toList();

but can be also useful to use Collectors.toCollection([Supplier]) since base implementation of Stream.toList:

default List<T> toList(){ 
        return (List<T>) Collections.unmodifiableList(
                         new ArrayList<>(
                         Arrays.asList(this.toArray()))); }

Collections.emptyList and Collections.nCopies are commodities methods that return specialized unmodifiable lists. All modification methods will throw Exceptions. Collections.nCopies is a lazy list where only the original element is used and repeated only in the case of toArray methods (logically the copy of provided elements in construction is a reference copy)
Collections.unmodifiableList is similar to the other methods, returning a specialized unmodifiable list; a decoration approach is used by design and all modification methods will throw Exceptions.

public static <T> List<T> unmodifiableList(
                               List<? extends T> list)
{ 
    return (list instanceof RandomAccess ?
             new UnmodifiableRandomAccessList<>(list) :
             new UnmodifiableList<>(list));
}

Stream.toList create unmodifiable list, also If Javadoc ask to not make any assumption on the nature of the list, and remember that a better control on the result is in using Collectors.toCollection([supplier])

List.of represent a set of static method that uses package class ImmutableCollections.java: this class hosts different inner implementation returning fixed-size list backed by a specified array. All modification methods will throw Exceptions. Some aspects need to be considered or reiterated:

wrapping class as Arrays.asList, Collections.unmodifiableList can be modified by altering the original collection or array used in construction
It is possible to obtain an empty list in many ways while actually obtaining different types of lists by:

Arrays.asList()
Collections.emptyList()
Collections.EMPTY_LIST
Collections.nCopies(0, [type])
List.of()

On this last topic, the singletons Collections.emptyList() and Collections.EMPTY_LIST are conceptually aliases that allow a performance gain in returning an empty list but it should be considered whether it is always correct to use these elements rather than returning a new empty list immutable or not, or create, in a specific namespace, another singleton based on one of the above invocations.

...but what if you wanted to recognize the nature of a list?

How can I recognize if a list is mutable or not modifiable? Let's say that from a pure design point of view it shouldn't interest us because by playing on typing it should be possible to act differently but this is not the situation. As a pure exercise let's try to see some solutions.

A first approach can be very simple and take advantage of the basic knowledge we have:

public static final <T> boolean isMutable(List<T> subject)
{
      return(subject instanceof ArrayList  || 
         subject instanceof LinkedList || 
         subject instanceof Vector     || 
         subject instanceof CopyOnWriteArrayList);
}

A second solution, a little more elaborate, could be to memorize the reference classes of the various possible lists and take advantage of the instance-of operator:

static HashSet<Class<? extends List>> set = null;

static
{
   set = new HashSet<>();   
   set.add(List.of("", "").getClass());
   set.add(List.of("", "","").getClass());
   set.add(Collections.emptyList().getClass());
   set.add(Collections.nCopies(0, "").getClass());
   set.add(Collections.unmodifiableList(
                           new ArrayList<>()).getClass());
   set.add(Arrays.asList().getClass());
}

public static <T> boolean isMutable(List<T> subject)
{
  return !set
    .stream()
    .anyMatch(clazz -> subject.getClass()
                              .isAssignableFrom(clazz)); 
}

Note: ...what will happen when having Structural Pattern Matching? We can derive some solutions?

A third solution takes advantage of the fact that the main unmodifiable classes are actually inner classes unlike the mutable ones which are top classes; using this knowledge we can write the following code:

public static <T> boolean isMutable(List<T> subject)
{
  return subject==null?false:
         subject.getClass().getEnclosingClass()==null;
}

An interesting proposition, to make things compatible with the actual List interface, comes from Java Guru Heinz Kabutz: adding default method characteristics as for Spliterator, it would be possible to understand the nature of the given List implementation.

What are my dreams about Lists?

The Java Collection Framework has a high level of extensibility but its design is based on hierarchically defined relationships with interfaces, which define roles of data structures, and abstract classes that act as extension hot spots. In the near future, it will be further modified to meet the needs of well-established use cases.

My personal view is to have a different packaging of data structures to allow for better isolation of them since not all of them are used in all applications, even with the current JCF design.

In relation to the lists it would be nice to have:

to complete the set of array-based dynamic lists, so as to improve performance in the light of mechanical sympathy or optimization of memory use.

It would be useful to have self-organizing lists to approximate use cases such as local caches and allow some sorting of data, based on customizable logic. This in conjunction with use with other data structures would be a way to expand the JCF power and modeling possibilities without resorting to many infrastructures outside the JDK.

Immutable lists should probably appear in a dedicated package and it should be really nice to have persistent data structures to have: the advantages of immutability, a set of fluent APIs, the optimizations in the use of the memory, normally present for this type of data structures.

Conclusions

There are no perfect solutions but perfectible solutions are possible. When choosing a data structure it is necessary to take a moment to reflect on the choice made. In the case of lists, most part of the use cases are covered by what is offered by the JDK.

Logically in the ecosystem of open source projects, there are good frameworks that make it possible to further meet the needs of the day by day (both OO and FP).

What can we learn from what we examined?

To date, we cannot tell if a list is idiomatically mutable or not.
To date, in the design of an API we need to understand when it is possible to reduce the abstraction level or widen it using Iterable or list implementation types, to avoid creating confusion with the only concept of List (however in many cases unavoidable)
Always use Javadoc to be able to describe the type of lists actually returned or that you want to manage in the various functional elements
We can certainly say that Arrays.asList as well as the lists deriving from the Collections class must be used in testing contexts and only as commodities elements and, therefore, their use-cases seem to see them hidden in internal portions of higher-level implementations or realizations.
It is possible to use more structured or effective frameworks, for performance or definition of types, present in the open source world, taking care not to leak anything of the specific framework on the boundaries of a system or in any case on the interfacing elements of the realizations: this to avoid to find our-self with problems of dependencies and inconvenient or "sticky" maintenance
Currently, the lists deriving from List.of must be the ones to be used in case of unmodifiable needs (and check for the absence of nulls), to make fallback on the composition of lists in situations of aggregate fragments of lists from different points of the system.
Consider that the various types of lists are serializable: this possibility must further reflect on what we are offering to a third party. This must bring to reason when, whether or not to actually use an aggregate type instead of a generic data container.

Problem Details for HTTP APIs - RFC 7807 is dead, long live RFC 9457

Stefano Fago — Thu, 23 Nov 2023 15:27:34 +0000

The positive experience of RFC 7807, whose journey began in 2016, is concluded (deprecation) but also confirmed with a new official proposition: the RFC 9457.
The changes made are small but it is a suitable opportunity to analyze the evolution of this topic.

Context

The RFC 7807, and now the RFC 9457 which perfects it, was born after a period in which the Internet Companies have made it clear that the integration of the business, the eventual transformation of the company services into a platform and a part of the organizational flexibility can pass from using the API.

The lack of references and the need to be time-to-market has meant that these same entities pollute the panorama of approaches and implementations in a set of non-interoperable solutions directing to some effects:

invalidating, in part, the effectiveness of the API
creating the need to define alternative tools
defining the need for standards to have a more effective and evolving product, consultancy, and research market over time

Problem Details for HTTP APIs

The motivation leading to these RFCs is the impossibility of expressing errors, both semantically and in terms of contents, using only HTTP Status Codes.
This premise does not mean that:

HTTP status codes are bad
HTTP status codes are not useful
HTTP status codes need to be replaced

Why are we talking about HTTP API?
HTTP API means the possibility of communications machine to machine, implemented upon the HTTP protocol thanks to modular elements defined by specific RFCs. So we are not talking about REST but about a more open superset in favor of large-scale developments.

This RFC therefore offers the benefit of allowing information to be conveyed for both humans and machines: this paves the way for the creation of better related tools.
It is not, however, the goal of the RFC to provide an application debugging tool.
All this is summarized in the specification document of which important sentences are as follows:

...This specifcation's aim is to define common error formats for applications that need one so that they aren't required to define their own or, worse, tempted to redefine the semantics of existing HTTP status codes. Even if an application chooses not to use it to convey errors, reviewing its design can help guide the design decisions faced when conveying errors in an existing format...

...Problem details are not a debugging tool for the underlying implementation; rather, they are a way to expose greater detail about the HTTP interface itself...

...truly generic problems -- i.e., conditions that might apply to any resource on the Web -- are usually better expressed as plain status codes...

...an application might have a more appropriate way to carry an error in a format that it already defines. Problem details are intended to avoid the necessity of establishing new "fault" or "error" document formats, not to replace existing domain-specific formats...

Communication aspects

In the context of HTTP, the first element to consider is the proper representation of the type of information transported:
this also means having a reference to the REST approach and Hypermedia.
The specification adopts two specific media types as representation indicators, also considering the most used standards at the moment, for data transport, namely JSON and XML;
resulting in two registered media types:

application/problem+json
application/problem+xml The RFC does not offer particular recommendations for managing other types of media, even if it proposes the concept of "encapsulating information" in other representations, where possible. This is, for example, the case of HTML5 which has more possibilities for hosting metadata not directly related to rendering web pages (scripts, as for the example that follows, or microformats or microdata). Here's an example:

<script type="application/problem+json">
{ 
 "type": "https://example.com/probs/out-of-credit",
 "title": "You do not have enough credit.",
 "detail": "Your current balance is 30, but that costs 50.",
 "instance": "/account/12345/msgs/abc",
 "balance": 30,
 "accounts": ["/accounts/12345",
 "/account/67890"]
}
</script>

The RFC confirms the use of the proactive content negotiation approach.
The communication includes the main HTTP headers valid for content negotiation such as Content-Type, Accept but also Accept-Language, aiming to provide non-binding hints on the language for the texts present in the HTTP responses.

What happened to the header problem? A curiosity

One draft of the new specification, initially called RFC 7807bis, introduced the Problem HTTP Header: this would allow a Problem to be shown according to the specification bypassing a response body.

HTTP/1.1 200 OK
Content-Type: application/json
Problem: type="https://example.net/problems/almost-out", title="you're almost out of credit", credit_left=20

This idea has not been approved, even because it could have introduced protocol inconsistency, e.g., what concerning the media type? What concerning the representation: is it an error or an answer?
An alternative to the Problem Header can be the embedding approach or the consideration that the specification does not force migration if a previous solution is already present or the possibility of creating customized problem types.

The response body

The specification requires structured content to convey information about errors;
it's worth noting that the non-understanding information, expressed in the body, must not block the activity of a generic HTTP software receiving data, permitting the latter to skip the information unknown.
Different attributes are offered:

We can then have specification-compliant dialogs like in the following example:

POST /purchase HTTP/1.1
Host: store.example.com
Content-Type: application/json
Accept: application/json, application/problem+json
{
 "items": 123456,
 "quantity": 2
}

HTTP/1.1 403 Forbidden
Content-Type: application/problem+json
Content-Language: en
{
 "type": "https://example.com/probs/out-of-credit",
 "title": "You do not have enough credit.",
 "detail": "Your current balance is 30, but that costs 50.",
 "instance": "/account/12345/msgs/abc",
 "balance": 30,
 "accounts": ["/accounts/12345",
 "/account/67890"]
}

An interesting aspect of the specification is the definition of a default type in case you simply want to report the status code information but in the format provided by the RFC:

{
 "type": "about:blank ",
 "status": [ number related to the specific HTTP status code ],
 "title": [ string message related to the specific HTTP status code ],
}

Note that the "type" attribute, if not specified, acquires the value about:blank with all that follows.
The response body of this RFC may have extensions discussed later.

The type attribute

It is a mandatory attribute, expressed as a URI that identifies the type of problem.
The value of this attribute can be:

about:blank
URI Locator
URI Tag

If the attribute value is a URI Locator, then it is strongly recommended that dereferencing it results in human-readable documentation for the type of problem.
Clients should not dereference the locator automatically but only as needed to provide documentation to developers.
If the attribute value is a Tag URI, e.g., tag:example@example.org,2021-09-17:OutOfLuck, it is not dereferenceable and uniquely represents the different types of problems.
Using a URI to have a unique ID is not recommended especially from the point of view of integration and use of tools for which providing further information at the Developer Experience level is a fundamental objective.
The locator must use absolute rather than relative URIs to avoid confusion and malfunctions

The status attribute

It is an optional attribute that has an informative nature.
When created it must report the same status code defined in the HTTP response and its purpose is to allow Consumers to be aware of the original status code possibly modified by an HTTP intermediary.
Being a duplicate in the HTTP response code value, this attribute can be a point of attack if used without awareness.

The title attribute

It is a mandatory attribute that has an informative nature for all those users who do not know or cannot know the semantics of the problem expressed by the URI of the type attribute.
The value of this attribute is strongly recommended to remain constant between one occurrence and another of the problem, except for possible localization needs.

The detail attribute

It is an optional attribute containing a human-readable explanation of the specific occurrence of the problem.
It’s highly recommended that the value of this attribute does not contain technical information for debugging purposes which instead can be introduced in extension attributes: this attribute should therefore not be subject to particular manipulations.
This is also a point to remember for security purposes to reduce the amount of information in transit that aids API attacks.

The instance attribute

It is an optional attribute containing a URI that identifies the specific occurrence of the problem.
The value of this attribute can be:

URI Locator
URI Tag

When the URI is a dereferenceable locator, the problem details object can be retrieved from it.
It may also return information about the problem occurrence in other formats through the use of proactive content negotiation.
When the URI is a Tag (non-dereferenceable), it serves as a unique identifier for the problem occurrence which may be meaningful to the server but is opaque to the client.
As for the type attribute, absolute rather than relative addresses are to be preferred.

Extension members

It is possible to customize the body of a response with other attributes that offer more details about the problem that has occurred.
The specification does not contain particular rules to limit these extensions, except for adherence to the media type and relative naming of reference, but it highlights how these can be entirely avoided by a client:
unrecognized extensions must be ignored and therefore not block management of the answer.

Handling multiple problems

What does multiple problems mean? Who are they referring to? Can they happen?

The answers come from analyzing some coordinates:

To what (which is the subject) the errors found refer
How to represent a set of errors
Which errors are priority or not

RFC 9457 clarifies and responds with simple and immediately adoptable concepts:

A response of type Problem Details refers to a specific type, declared in the same, for which there may be multiple problems that must refer to the same resource (type) of the issued request. The presence of JSON among the main formats allows support for JSON Pointers to highlight the parts of the Json document that are affected by errors.
A set of errors defined, in the reference media type, thanks to the concept of body extensions (Extension Members): those additional attributes of the response body, are possible in the respect of the specification itself

POST /details HTTP/1.1
Host: account.example.com
Accept: application/json
{
 "age": 42.3,
 "profile": { "color": "yellow" }
}

HTTP/1.1 422 Unprocessable Content
Content-Type: application/problem+json
Content-Language: en
{
 "type": "https://example.net/validation-error",
 "title": "Your request is invalid.",
 "errors": [
   { "detail": "must be a positive integer", "pointer": "#/age" },
   { "detail": "must be 'green', 'red' or 'blue'", "pointer": "#/profile/color" }
  ]
}

The creation of a set of errors that do not concern the same type becomes strongly discouraged; this implies that of the possible errors, relating to different types, only the one considered a priority (relevant or urgent) is reported consistently with the above. It should also be remembered that if a format already exists for specific application needs, it is not required to convert to the RFC format.

The proposed vision, in the presence of multiple problems, may seem limiting but it is a balancing point between protocol cleanliness, representation effectiveness, and tool support.

In the deprecated RFC 7807, the use of the HTTP 207 status code was proposed, which allows you to report multiple statuses and errors in the same body. This solution, technically convenient, can lead to ambiguity and also reopens the issue of multi-request and multi-response which are not concepts natively provided for in the HTTP protocol.

The importance of the type

The specification highlights the idea of type related to a specific problem thus allowing two essential evolutions:

The ability to define custom types
The possibility of recording types of Problems in favor of their reuse where appropriate

The concept of type is significant and requires careful planning and the need to follow rules.
Such perspective leads to a balanced use of the specification and the possibility of improving and enriching the automation, and therefore supporting with products, of API design.

Defining custom problems involves documenting:

A specific URI that is relative to the type, this URI is preferable to be a locator and therefore dereferenceable in the documentation that indicates how to approach the problem
A title that briefly and appropriately describes the problem
The indication of the HTTP status code with which the type of problem is supposed to be related.

A custom type can have extensions (which must follow naming rules) and can use the Retry-After response header (if appropriate). The reference to Web Linking is interesting as a possibility to be used in extensions to refer to other resources and address the problem in question.

The registration of a new problem, in the registry provided for this specification at the address https://iana.org/assignments/http-problem-types, involves the use of a reference template, a possibly usable prefix URI, and the submission of a request to a group of experts for the validation of the proposition.

Conclusion

This new specification defines better support to obtain the order, conceptual cleanliness, and appeal in the significant market of APIs. RFC 9457 is conceptually simple but offers the elements to operate on almost all the use cases that can be experienced. It’s one of more components, the HTTP API RFCs, that should not be seen in isolation but moved in the continuous trade-off of own development/integration needs and efforts to be open and flexible in a market full of initiatives and possible new tools.

to go further

A Retrospective on Error Management: Where Do We Go From Here?

Stefano Fago — Tue, 21 Nov 2023 09:18:42 +0000

Error management is a fact of life in software development as it is often inevitable and generated by different causes that also include incorrect or incomplete understanding of the requirements or even lack of knowledge of some tools or elements used during development.

Let’s go on a small trip into the evolutions and different concepts related to error management, analyzing why it is difficult and why we are going in new directions after the moment of Exceptions in programming languages.

Error management philosophy in the ages

In languages such as C and C++, error handling was primarily based on return codes: specific values indicate the success or failure (sentinel value). This approach required the caller to check the return code, which could be ignored explicitly, and the control flow became convoluted due to the numerous checks scattered throughout the code base.

The notion of exception was introduced to address these challenges. The idea was to transfer the responsibility of error handling from the caller to a designated exception handler. When an exceptional situation arises, the runtime system throws an exception, which must be caught and handled by appropriate code. The introduction of already allocated exception handlers, like in Java, made it possible to catch/process Exceptions even if the system runs out of memory.

In recently defined programming languages (i.e.,, Rust, Go, Zig), we return to the dichotomy: exception vs. error. The distinction between recoverable and non-recoverable errors is considered.
Unrecoverable ones lead to unmanageable software crashes. Those defined as recoverable return to being values, not primitive types, managed in the standard functionality code. There are no exceptions to throw, the
compiler forces handling, and no longer specific scopes to place management code. These values are supported by dedicated constructs and, in some cases, strongly inspired by functional programming.

The transition to the Cloud has also altered the vision of error management. From the concept of robustness, we have increasingly migrated to considerations linked to the resilience of a system.
Furthermore, the usefulness of stack traces has been partly questioned: more distributed code, as well as the emphasis on Observability, makes simpler and more timely information necessary compared to the
habit of examining deep stack traces.

Why Exceptions aren’t enough?

When we think about using exceptions, there’s the habit of considering different problems to address:

If generated by the runtime, we must catch them to avoid the collapse
of the system
Some exceptions become part of the signature of a method or function,
also defining an increase in the coupling level to consider
By its nature, the handling of an exception breaks the flow of
execution. Without the right design, the specialized handler increases
cognitive overload or lowers the readability of the code base.

We can open a parenthesis relating to the widespread adoption of Runtime exceptions compared to other types, especially in general-purpose frameworks. It may be a misstep justified by the typical design of the frameworks themselves: operating with decoration or proxying patterns seems a natural choice for the use of Runtime exceptions also if it pushes for a precise knowledge of the framework, increasing the cognitive load.

Note:
The need to reduce the exceptions thrown directly by the runtime is addressed today in new languages in the continuous effort to limit low-level error conditions linked to memory usage, concurrency, or I/O. We have safer languages but also the promise of evolution regarding languages such as C++

But is it all the fault of the Exceptions?

What is our role, as software creators, about Errors?

An intriguing sentence comes from Michael Feathers reporting that Errors are just conditions we refuse to take seriously. So, what is the percentage value dedicated to the study error conditions and/or error management?

To understand what is occurring, we should consider that error management is conditioned by macroscopic aspects, that we often don’t think about holistically, such as:

The analysis of requirements and their level of volatility
How we design for software modularity (decoupling and cohesion)
Performance and the impact of error management on it
The development experience, and therefore the readability of the code
base, but also the awareness of the context in which it is being
produced

Another classification we often don’t notice is the one that sees error management locally or at the edges of a system. Software design continuously alternates a complexity seen locally (a class, for example) or that is seen at the system level or between different systems.

Exceptions are for exceptional cases

When should exceptions be used? The classical answer is: In the presence of exceptional cases! But what does it mean? The sentence is cryptic from a practical point of view, and when it is our duty, it becomes rather complicated.

If we consider Modularity, we should have exceptions strictly linked to the dysfunctions that classes can present within a single module. At the same time, if we assume the collaboration between multiple modules, the errors we wanted to define must be part of something shared between the various modules. It’s necessary to study which are those cases where the stability of an ecosystem is disturbed by invalid states and express them accordingly.

If we consider Performance, we should strive to have a limited number of exceptions thrown: creating and processing Exceptions can be expensive, mainly if they occur too frequently. Defining all error situations only on Exceptions does not consider the performance aspects, as well as increasing the cognitive load factor, and it is for this reason that the advice not to use the exception for the control flow is invoked.

Another hint comes from Bertrand Meyer, who reports to **use exceptions when you cannot know whether a call/invocation will succeed or fail. This vision justifies the adoption of a design-by-contract where we consider preconditions, invariants, and post-conditions and where exceptions occur due to their violation. For example, software that delivers a protocol as the target is related to the *design-by-contract*; the idea of a *formal contract between parts, as explained by Joe Armstrong, can be an influential adoption in defining error conditions in distributed systems.

It is worth noting that adopting formal methods to model error conditions is increasing for large-scale system concurrency, and the adoption of TLA+.

A criterion that we should use, but which we often forget, is to understand the usefulness and objectives of the system: why are we creating this software?
For example, data entry validation can be considered different from applying business rules: in the first subject, we can avoid exceptions, and in the second subject, we have specific exceptions related to the entities of that domain, and we need to reason about when to stop error management.

Start to migrate: what are good practices for Exceptions?

What we have discussed may be valid in general as well as for exceptions, but we can deduce some general suggestions from the book: A Philosophy of Software Design by John Ousterhout. Summarizing what is proposed by the text, we can distinguish the following cases:

Masking exceptions

encapsulating the management of exceptions where they occur so that there is no propagation and so that possible corrective action takes place promptly

Aggregate the exceptions
in contrast to what was said before, it’s suggested to place the actions that can define similar exceptions together to have blocks of instructions to manage in individual handlers

Not using exceptions
avoiding the use of exceptions means replacing them with return values and with explicit management of the returned values

Eliminate exceptional cases
avoid having conditional logic for special cases, i.e., for those values at the edges of a domain or introduced to make up for a sudden requirement. These can lead to unexpected errors and increase the code base's cognitive load.

Coordinates to migrate from Exceptions

The last two points start from what we see as a move away from exceptions.

In this context, Michael Feathers indicates the need to extend our domain to avoid the use of exceptions: this means reworking the problem to bypass or limit the presence of errors as much as possible thanks to hints like Tell don't Ask. As an example, the author reformulates the suggestion from a data point of view, revealing that "asking for data" can fail while providing it, when we have it, cannot!

Source.withData(ID, data->data.someOperation(...))

We can see that error handling is hidden and has a limited scope due to how we operate on the data.

It becomes essential again to have a way to express the error condition without taking control away from the Developer. For this purpose, it is
reasonable to rely on the Type System and support for Generics.

Avoiding the use of null both as a return value and as a sentinel value, patterns such as the NullObject, containers such as Optionals, use
of Default Values or functions, and encapsulating communications in an event approach are starting points that need to be revised.

Constructs from Functional Programming have been embraced in
programming languages (Optional, Either, Try Monad, Railway Oriented
Programming, Structural Pattern Matching) where the importance of expressions is highlighted compared to statements.

We can see more of the concepts exposed in the Rust language.

The Result type is not only a Union Type but supports a series of methods that allow error management to be expressed as a Fluent API and to transform the Result outcome in an Optional or using mapping function.
An interesting aspect is that there is a compiler enforcement on this type that needs to be managed to avoid compilation warnings.


io::stdin().read_line()
           .ok() // We have the new line
           .expect("Failed to read a line!"); // no new line so we crash with a message

However, to avoid having too much code, the Rust syntax allows you to take advantage of specialized operators to synthesize error expressions, It is also possible to have structured pattern-matching to manage unions or closures to specialize or nest the processing.


let greeting_file_result = File::open("hello.txt");

let greeting_file = match greeting_file_result{
    Ok(file) => file,
    Err(error) => panic!("Problem opening the file: {:?}", error),
};

Compiler support in the case of Rust is also an essential ingredient for error propagation mechanisms and for defining code optimized for example, the presence or absence of stack traces.

No Exception at all?

In everyday life, there may be situations where interrupting the flow of an execution can be helpful.
It is interesting what is suggested in the Rust documentation, talking about the use of the panic macro (which defines a software crash):

it is acceptable in defining code examples, prototype code, and tests
it is acceptable when your code can lead to a "bad state". From a design-by-contract perspective, an invalid state occurs when some assumptions, guarantees, contracts, or invariants have been broken: invalid, contradictory, or missing values are passed to the code.
it is acceptable for security reasons due to the inappropriate state of the program.

Conclusion

Error management requires attention and care equal to that used to define business logic. It is a complex task, but many reference points and different conceptual tools exist.
Leaving aside trends and partisan discussions, it is better to consider the evolution of the design of error management because they are supported or because they can be adopted by one's language, paying attention to the limits of one's domain and rationalizing the effort in favor of the actual usefulness of the own software.

DEV Community: Stefano Fago

Clean Design, Strong Client: The way of the Elasticsearch's Java SDK

Code generation and the single source of truth

The Builder Pattern Done (Mostly) Right

DSL-Style Lambdas That Actually Work

Tagged Unions and Type-Safe Variants

Modular by Design: The Namespace Client Pattern

Immutability and Transport Abstraction

Could It Be Better?

Conclusion

Clean your Memory: From Finalize to Cleaner

Why is the finalize() method deprecated/removed?

How does Cleaner relate to Java Reference classes?

Behind the scenes of Cleaner

Cleaner vs. try-with-resources

Avoid overusing Cleaner

Conclusion

Further Resources

The FizzBuzz: a branchless version with Java

Overview of FizzBuzz Rules

Code Breakdown

1. Defining an Interface for Lambda Expressions

2. Creating an Array of Provider Lambdas

3. Defining Text Segments

4. Using StringBuilder for Efficient String Concatenation

5. Main Loop with Bitwise Operations

(1) Calculating a - 5 Multiples

(2) Calculating b - 3 and 15 Multiples

(3) Determining providerIndex

(4) Determining segmentIndex

6. Appending the Result

7. Printing the Final Output

Key Takeaways

Experimenting around FP in PHP

Callable, Anonymous Function, Closure, and FP!

Arrays, arrays functions, and the anonymous functions

The Scope element in Closures

Conclusion

To Go Furter

Designing with PHP 8.1 Enumerations

PHP 8.1: The Rise of Enumeration

PHP Enumeration: some Anti-Pattern!

The Confusing Constant

The Empty Enumeration

The Commodity Enumeration

The Singleton Anti-Pattern(?)

Enumerations, Template & Constant Expression

Interesting Design Aspect of Enumerations

The Interface Friend

The Trait Friend

The Enumeration State Machine

The Enumeration Strategy Pattern

The Enumeration Factory Pattern

The Processor Set Pattern

The Conditional Processor Pattern

The Enumeration Random Generator

To Go Further

Lists: do you know the nature of yours? The strange story of a data container in Java

Foreword

How many lists we can find in Java?

In how many ways You can build a List?

Why is it a Problem?

Mutable, Immutable, Not Modifiable List

...but what if you wanted to recognize the nature of a list?

What are my dreams about Lists?

Conclusions

What can we learn from what we examined?

Problem Details for HTTP APIs - RFC 7807 is dead, long live RFC 9457

Context

Problem Details for HTTP APIs

Communication aspects

What happened to the header problem? A curiosity

The response body

The type attribute

The status attribute

The title attribute

The detail attribute

The instance attribute

Extension members

Handling multiple problems

2. Creating an Array of `Provider` Lambdas

(1) Calculating `a` - 5 Multiples

(2) Calculating `b` - 3 and 15 Multiples

(3) Determining `providerIndex`

(4) Determining `segmentIndex`