DEV Community: Zone 2 technologies Ltd.

How to improve third-party libraries with Kotlin extensions

Spas Poptchev — Tue, 06 Dec 2022 21:03:36 +0000

Do you ever find yourself using a library that lacks the API you require? I've lost track of how many utility functions and adapters I've had to write that, in retrospect, should have been part of the third-party library in the first place.

This guide will not save you from having to write a utility function, but it will show you how to easily add functionality to virtually any interface offered by a third-party library. The best approach to accomplish this in Kotlin is to use the language's extension features. Extensions, as opposed to other prevalent design patterns such as decorators and adapters, allow you to add new functionality to a class or interface in a simplified manner. You don't have to use inheritance or delegation to add new functions and properties to classes. Instead, you can add them directly to the classes themselves.

Extensions can either be declared as functions or properties.

class Host(val name: String)

// extension function
fun Host.resolveIP() = "Resolving IP for $name"

// extension property
val Host.port
    get() = "3030 is the port for $name"

Once defined, the extensions can be used just like any other function or property of the class or interface to which they belong. Public member functions and properties (e.g. name) can be used as part of the extension.

fun main() {
    val host = Host("example.com")

    println(host.name) // -> example.com

    println(host.resolveIP()) // -> Resolving IP for example.com

    println(host.port) // -> 3030 is the port for example.com
}

The extensions are not particularly useful in this scenario because the described functionality can be incorporated into the Host class. On the other hand, they flourish in test frameworks like Kotest and enable the rapid development of useful add-ons like custom matchers. Extending third-party libraries with utility functions is another prevalent use case. In the next sections, we'll zero in on this specific aspect.

Improving Mime4J

To give you a practical example, we will improve the Mime4J library.
Mime4J can read plain RFC822 and MIME email messages. One of its major drawbacks is that it lacks predefined methods that return the HTML and plain text parts of a message. The intrinsic complexity of knowing MIME and all of its non-standard implementations exacerbates this disadvantage.

While looking for a good solution to extract the HTML and text parts, I came across the following code hidden in the Apache James mail server: It has a 160-line class called MessageContentExtractor to extract the content as well as a 50-line inner static class called MessageContent, which is used to hold the data.

When you break down the MessageContentExtractor, you will see the following main conditions that identify the HTML and
plain text parts:

The body part has to be of type TextBody.
The mime-type has to be text/plain or null for the text part and text/html for the HTML part.
Furthermore, the processed part cannot be an attachment. This is determined by evaluating whether the Content-Disposition is null. Accepting a part with an inline Content-Disposition when it lacks a CID is an exception to this rule.

Now that we know what to look for, we can evaluate how we want to extend the Mime4J Message interface. From the consumer's point of view, adding two new properties, html and text looks like the best solution. Both properties return the content of their respective parts as a String.

val Message.text: String?
    get() = TODO()

val Message.html: String?
    get() = TODO()

Implementing the search conditions

In the beginning, we will concentrate on point number one: find all body parts with the TextBody type. To actually do this, we need some background information on how the Message interface is structured.

┌───────────────────┐     ┌───────────────────┐
│      Entity       │     │        Body       │
└───────┬───────────┘     └──────────┬────────┘
        │                            │
        │   ┌───────────────────┐    │
        └───►      Message      ◄────┘
            └───────────────────┘

Message inherits from both the Entity and Body interfaces. The Entity interface offers the methods required to access the body (content) of a message, or the body parts in case of a multipart message. Additionally, the Body interface indicates that the message itself can be part of a message body. Based on the previous points, we can conclude that the Message interface represents a recursive data type. A message can be part of a message, which can likewise be part of another message, and so on.

As a consequence we must attach a function to the Entity interface in order to find the body part we are looking for. We'll call it findTextBody. This function will return any message body that is a text, even if it is in a text format (for example, CSS) that we are not interested in.

fun Entity.findTextBody(): TextBody? {
    // Check if Entity contains a TextBody. Return it if it does
    if (body is TextBody) {
        return body as TextBody
    }

    // Check if the message has multiple parts
    if (!isMultipart) {
        return null
    }

    // Call the findTextBody function recursively on all
    // body parts and return the first one that is not null,
    // or return null when no part is found
    return (body as Multipart).bodyParts
        .firstNotNullOfOrNull { it.findTextBody() }
}

After we've gone through the most significant section of our conditions, we can look for parts in plain text or HTML format (our second condition). Our second condition said that the message body's MIME type must be text/plain or null for the plain text part and text/html for the HTML part. To do this, all we have to do is add an extra condition when our function finds a TextBody.

fun Entity.findTextBody(validMimeTypes: Set<String?>): TextBody? {
    // Added mimeType condition
    if (body is TextBody && mimeType in validMimeTypes) {
        return body as TextBody
    }

    if (!isMultipart) {
        return null
    }

    return (body as Multipart).bodyParts
        .firstNotNullOfOrNull { it.findTextBody(validMimeTypes) }
}

Now we can add our final condition:

Furthermore, the processed part cannot be an attachment. This is determined by evaluating whether the Content-Disposition is null. An exception to this rule is accepting a part with an inline Content-Disposition when it lacks a CID.

As you can see, the condition contains two parts, which we can implement by adding two extension properties so that our code stays clean and readable.

private val Entity.isNotAttachment
    get() = dispositionType == null

private val Entity.isInlinedWithoutCid
    get() = dispositionType == "inline" && header.getField(FieldName.CONTENT_ID) == null

fun Entity.findTextBody(validMimeTypes: Set<String?>): TextBody? {
    // Added extension properties to the condition
    if (body is TextBody && mimeType in validMimeTypes && (isNotAttachment || isInlinedWithoutCid)) {
        return body as TextBody
    }

    if (!isMultipart) {
        return null
    }

    return (body as Multipart).bodyParts
        .firstNotNullOfOrNull { it.findTextBody(validMimeTypes) }
}

The private modifier is added to both functions because they should only be accessible from the Kotlin file where the findTextBody function is declared.

Extracting the content

Let's add our new function to the extension properties we set up at the start.

val Message.text: String?
    get() {
        val textBody = findTextBody(setOf("text/plain", null)) ?: return null

        return TODO()
    }

val Message.html: String?
    get() {
        val textBody = findTextBody(setOf("text/html")) ?: return null

        return TODO()
    }

All that we have left now is to convert the TextBody into a String. We achieve this by writing the TextBody into a ByteArrayOutputStream and converting it to a String based on the TextBody charset. If the charset name is invalid, we will use the default charset (ASCII).

private val TextBody.content: String
    get() {
        val byteArrayOutputStream = ByteArrayOutputStream()

        writeTo(byteArrayOutputStream)

        return String(byteArrayOutputStream.toByteArray(), contentCharset)
    }

private val TextBody.contentCharset
    get() = try {
        Charset.forName(mimeCharset)
    } catch (e: IllegalCharsetNameException) {
        Charsets.DEFAULT_CHARSET
    } catch (e: IllegalArgumentException) {
        Charsets.DEFAULT_CHARSET
    } catch (e: UnsupportedCharsetException) {
        Charsets.DEFAULT_CHARSET
    }

The final result

To finish our extension properties, we can access the content extension property of the TextBody.

val Message.text: String?
    get() {
        val textBody = findTextBody(setOf("text/plain", null)) ?: return null

        return textBody.content
    }

val Message.html: String?
    get() {
        val textBody = findTextBody(setOf("text/html")) ?: return null

        return textBody.content
    }

The implementation of our two new extension properties is now complete. We can use them as part of the Message interface.

fun main() {
    val textPart = BodyPartBuilder.create()
        .setBody("plain text content", "plain", Charsets.UTF_8)
    val htmlPart = BodyPartBuilder.create()
        .setBody("<html><body>content</body></html>", "html", Charsets.UTF_8)

    val multipart: Multipart = MultipartBuilder.create("alternative")
        .addBodyPart(textPart)
        .addBodyPart(htmlPart)
        .build()

    val message = Message.Builder.of()
        .setBody(multipart)
        .build()

    println(message.text) // -> plain text content
    println(message.html) // -> <html><body>content</body></html>
}

Conclusion

Extensions in Kotlin are a powerful technique for implementing utility functions for third-party libraries. In this particular situation, they assisted us in reducing a 210-line Java class to approximately 50 lines of readable and maintainable Kotlin code*. If you want to see the entire code, check out this Github gist.

Here are a few general recommendations when you are getting started with Kotlin extensions:

When you can put the properties or functions directly into the class or interface, don't use extensions.
Expose the bare minimum set of functions or properties as extensions. Everything else should be kept private.
Don't forget that extension functions are resolved statically. So, mocking them in a test is only possible when you use libraries like PowerMock or MockK.
Another common pitfall is that extension functions are invoked based on expressions and not types. A good example can be found here.

Thank you for following me on this journey. Do you have any questions? Was this article helpful? Let me know in the comments below.

*) The Java code also includes logic for loading plain text and HTML depending on the multipart type (alternative, related, etc.). As a result, the comparison on the number of lines may not be fair in this case, but is still a pretty good indicator how Kotlin improves our development experience.

Writing Keycloak extensions: Key concepts and anti-patterns

Spas Poptchev — Fri, 11 Nov 2022 12:05:58 +0000

Keycloak comes preconfigured with a number of industry-standard providers, themes, and connectors. These built-in features are fairly robust, but they may not always be sufficient. To implement custom features, Keycloak allows you to use a number of service provider interfaces (SPI). By extending them, you can add a new social identity provider or make a unique form action to tell the difference between human and automated access.

Key concepts

There are two fundamental components that make up a custom Keycloak service provider: a ProviderFactory and the actual Provider itself.

ProviderFactory

As implied by its name, the ProviderFactory is the component that is in charge of actually creating the Provider. The factory will be discovered once the Keycloak server is launched, and only one instance of it will exist for the duration of the server's operation. Its interface, limited to the essential bits, is as follows:

public interface ProviderFactory<T extends Provider> {

    void init(Config.Scope config);

    void postInit(KeycloakSessionFactory factory);

    void close();

    T create(KeycloakSession session);

    // not mentioned: getId, order, getConfigMetadata

}

Now, let's examine each method in greater depth.

void init(Config.Scope config)

During the initialization of the factory, init will be called once and only once. The configuration will be obtained from the keycloak_server.json file. You can use this method to initialize and create objects that will be utilized in your implementation of the provider. To illustrate, HTTP clients and database clients are two examples of objects that should almost never be created more than once.

void postInit(KeycloakSessionFactory factory)

After all provider factories have been initialized, the function postInit will be invoked. You can now access any additional ProviderFactories and SPIs that you require. Further more, if your business logic calls for it, you might also initialize recurring tasks.

void close()

The close method is called when the server shuts down. Resources, such as HTTP clients, created in the init or postInit methods should be released here.

T create(KeycloakSession session)

This is the method that creates the actual provider instance. Keycloak is calling this method on each request so the provider object itself should be light-weight.

Provider

Each time a request is made, Keycloak invokes the create function on the ProviderFactory to generate a new provider object. In the provider, you'll implement the business logic of the functionality you want to add to your system. The only truly essential part of the interface is the close method, which frees up any resources that were allocated for the request but are no longer in use. Other methods will depend on the actual provider interface you want to implement.

Provider discorvery

To tell Keycloak where to find the ProviderFactory, you have to set up a file in META-INF/services named after the specific service provider implementation and the fully qualified name of your service ProviderFactory as the file content.

Anti-Patterns

Having covered the fundamentals of what a custom provider should look like, we can now examine bad practices that may arise while developing a Keycloak extension.

AP#1 - Implementing ProviderFactory and Provider in the same class

Take a look at the following example, in which a ProviderFactory and a Provider are implemented in the same class.

class AntiPatternProvider implements ProviderFactory<AntiPatternProvider>, Provider {

    private HTTPClient httpClient;

    @Override
    public AntiPatternProvider create(KeycloakSession session) {
        httpClient.post("https://example.com");

        return this;
    }

    @Override
    public void init() {
        httpClient = new HTTPClient();
    }

    @Override
    public void close() {
        httpClient.close();
    }

    // other implementation specific functions...
}

Yes, it would appear odd, but in theory, this is a legitimate provider. Can you identify any potential issues?

The server will start up without any problems, but as soon as the provider receives a second request, an error message stating that the HTTP client has already been closed will appear. Keycloak has no means of knowing that we only want to close the HTTP client when the server is shutting down. Instead, each request will result in the client closing. This is because we are implementing the close method of both the ProviderFactory and the Provider, which, as we have seen, have different responsibilities.

We can avoid the issue by implementing the Provider and the ProviderFactory in separate classes.

class BestPracticeProvider implements Provider {

    private final HTTPClient httpClient;

    public BestPracticeProvider(HTTPClient httpClient) {
        this.httpClient = httpClient;
    }

    @Override
    public void close() {
        // This method will be called on each request. 
        // Do not close the HTTP client here!
    }
}

class BestPracticeProviderFactory implements ProviderFactory<BestPracticeProvider> {

    private HTTPClient httpClient;

    @Override
    public BestPracticeProvider create(KeycloakSession session) {
        return new BestPracticeProvider(httpClient);
    }

    @Override
    public void init() {
        httpClient = new HTTPClient();
    }

    @Override
    public void close() {
        // The server is shutting down. 
        // We can safely close the HTTP client.
        httpClient.close();
    }

    // other implementation specific functions...
}

AP#2 - Creating heavy-weight objects during requests

In the example below, we will look at another anti-pattern that may arise while creating a provider.

class AntiPatternProvider implements Provider {

    private final HTTPClient httpClient;

    public AntiPatternProvider() {
        httpClient = new HTTPClient();
    }

    @Override
    public void close() {
        httpClient.close();
    }
}

class AntiPatternProviderFactory implements ProviderFactory<AntiPatternProvider> {

    @Override
    public AntiPatternProvider create(KeycloakSession session) {
        return new AntiPatternProvider();
    }

    // other implementation specific functions...
}

In this provider implementation, the HTTP client is created and closed during each request. The issue is that the client is a heavy object. Creating it for every request is expensive. Similarly to how we fixed AP#1, we need to have the ProviderFactory handle both the creation and closing of the HTTP client.

Conclusion

Keycloak offers a lot of possibilities to extend its built-in functionality. Anyone working on an extension should familiarize themselves with the Keycloak SPI first. In my opinion, Keycloak's documentation and examples aren't great, especially for developers who are just getting started.

So, here are a few general recommendations when you are starting out:

Treat each provider as a self-contained application.
Keep an eye out for resources that have to be released once the server is shut down or the request is finished. Otherwise, you may unintentionally create memory leaks.
Avoid having multiple providers in the same library whenever possible. Instead, try to split them up per domain. This will make it easier to test, maintain, and deploy them.
Keep your providers up to date with the latest Keycloak version.
Do not package external libraries with shadowJar (only applies to the Quarkus distribution). Instead, add the libraries to the providers directory.

Thank you for reading

Do you have any questions? Was this article helpful? Let me know in the comments below.