DEV Community: PiotrekSt

Let's take a quick look at Project Loom.

PiotrekSt — Sun, 22 Nov 2020 16:31:32 +0000

This article gives a quick look into Project Loom - one of the current buzzwords in the Java world.

What is Project Loom about?

Project Loom started in late 2017. The main goal of the project is to reduce the complexity of creating and maintaining the high-throughput concurrent applications. It introduces the concept of a lightweight concurrency model based on virtual threads. What is the virtual thread? Virtual thread instead of being managed by the operating system as the standard one is scheduled by a Java virtual machine. It results in that such threads can be efficiently scheduled allowing synchronous code to be executed as well as asynchronous code in terms of performance. The implementation is based on an idea such as continuation and operations related to them defined as parking and unparking.

If you would like to get more information about that idea, I highly recommend getting familiar with project wiki site.

Demo time

Note: The article is based on JDK from Project Loom Early Access Build - Build 16-loom+7-285 (2020/11/4)

Assume we got some time-consuming tasks, which we want to run in the background of our application.

private static Runnable timeConsumingTask(int id) {
    return () -> {
        out.println(format("[%s][%s] Starting time consuming task [id=%s]", now(), Thread.currentThread(), id));
        try {
            Thread.sleep(Duration.ofSeconds(5));
        } catch (InterruptedException e) {
            out.println(format("[%s][%s] Oops interruption occurred [id=%s]!", now(), Thread.currentThread(), id));
        }
        out.println(format("[%s][%s] Ended time consuming task [id=%s]", now(), Thread.currentThread(), id));
    };
}

As you can see the task definition is ridiculously simple: log start, sleep for 5 seconds, log end of the task.

Let's start with the "classic" daemon thread. We will use for that the ExecutorService which is known since JDK 1.5.

private static ExecutorService standardSingleExecutorService() {
    var factory = Thread.builder().name("standard-thread").daemon(true).factory();
    return Executors.newSingleThreadExecutor(factory);
}

First of all, we create a thread factory with setting daemon option to true, and then an instance of a single thread executor which will use that factory. Next, our main goal is to execute four "time-consuming" tasks using the created executor.

public static void main(String[] args) throws InterruptedException {
    var ex = standardSingleExecutorService();
    long startTime = System.nanoTime();
    IntStream.range(0, 4)
            .forEach(id -> ex.execute(timeConsumingTask(id)));
    ex.shutdown();
    ex.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
    long stopTime = System.nanoTime();
    out.println(format("[%s][%s] Processing took = %s ms", now(), Thread.currentThread(), TimeUnit.MILLISECONDS.convert(stopTime - startTime, TimeUnit.NANOSECONDS)));
}

The output looks like:

[2020-11-22T17:15:54.303382][Thread[standard-thread,5,main]] Starting time consuming task [id=0]
[2020-11-22T17:15:59.323550][Thread[standard-thread,5,main]] Ended time consuming task [id=0]
[2020-11-22T17:15:59.324083][Thread[standard-thread,5,main]] Starting time consuming task [id=1]
[2020-11-22T17:16:04.324808][Thread[standard-thread,5,main]] Ended time consuming task [id=1]
[2020-11-22T17:16:04.325580][Thread[standard-thread,5,main]] Starting time consuming task [id=2]
[2020-11-22T17:16:09.330546][Thread[standard-thread,5,main]] Ended time consuming task [id=2]
[2020-11-22T17:16:09.331408][Thread[standard-thread,5,main]] Starting time consuming task [id=3]
[2020-11-22T17:16:14.335599][Thread[standard-thread,5,main]] Ended time consuming task [id=3]
[2020-11-22T17:16:14.336839][Thread[main,5,main]] Processing took = 20064 ms

Everything works as you probably expected. Our four tasks were executed one by one on the single thread pool. It results in the total time needed for execution to be about 20 seconds (4 tasks each with 5 seconds Thread.sleep()) Yeah! The math still works, 4 times 5 equals 20.

Now let's change our executor service to use the virtual threads. This change is quite simple. To be sure that our virtual thread is scheduled by a single carrier thread we can assign a specific one by specifying it in virtual() method. Carrier thread is a name for a scheduler worker thread that is responsible for executing a virtual thread. See the example below:

private static ExecutorService virtualThreadExecutorService() {
    var factory = Thread.builder().name("carrier").daemon(true).factory();
    var executor = Executors.newSingleThreadExecutor(factory);
    var virtualThreadFactory = Thread.builder().name("virtual-thread", 0).virtual(executor).factory();
    return Executors.newThreadExecutor(virtualThreadFactory);
}

Now it is time to change the code of the main method to use the ExecutorService based on virtual threads:

var executorService = virtualThreadExecutorService();

and the output should be similiar to:

[2020-11-22T17:16:49.114727][VirtualThread[virtual-thread0,carrier,main]] Starting time consuming task [id=0]
[2020-11-22T17:16:49.161798][VirtualThread[virtual-thread1,carrier,main]] Starting time consuming task [id=1]
[2020-11-22T17:16:49.162620][VirtualThread[virtual-thread2,carrier,main]] Starting time consuming task [id=2]
[2020-11-22T17:16:49.163415][VirtualThread[virtual-thread3,carrier,main]] Starting time consuming task [id=3]
[2020-11-22T17:16:54.163348][VirtualThread[virtual-thread0,carrier,main]] Ended time consuming task [id=0]
[2020-11-22T17:16:54.164141][VirtualThread[virtual-thread1,carrier,main]] Ended time consuming task [id=1]
[2020-11-22T17:16:54.165216][VirtualThread[virtual-thread2,carrier,main]] Ended time consuming task [id=2]
[2020-11-22T17:16:54.166347][VirtualThread[virtual-thread3,carrier,main]] Ended time consuming task [id=3]
[2020-11-22T17:16:54.166858][Thread[main,5,main]] Processing took = 5106 ms

As you can see now the execution of the same four tasks takes about 5 seconds. All tasks run in parallel without blocking each other, using the given carrier thread. We achieved it just by changing the type of thread from standard one a virtual. Magic!

Ok, cool... but how?

To understand how it is possible we need to dive into Thread.sleep() method.

public static void sleep(Duration duration) throws InterruptedException {
    long nanos = duration.toNanos();
    if (nanos < 0)
        return;

    Thread thread = currentThread();
    if (thread.isVirtual()) {
        if (ThreadSleepEvent.isTurnedOn()) {
            ThreadSleepEvent event = new ThreadSleepEvent();
            try {
                event.time = nanos;
                event.begin();
                ((VirtualThread) thread).sleepNanos(nanos);
            } finally {
                event.commit();
            }
        } else {
            ((VirtualThread) thread).sleepNanos(nanos);
        }
    } else {
        // convert to milliseconds, ceiling rounding mode
        long millis = MILLISECONDS.convert(nanos, NANOSECONDS);
        if (nanos > NANOSECONDS.convert(millis, MILLISECONDS)) {
            millis += 1L;
        }
        sleep(millis);
    }
}

As you can see there is a conditional statement where the implementation of sleep behaves differently when is performed on a virtual thread. The sleepNanos(long nanos) method from VirtualThread class gives us a clue.

void sleepNanos(long nanos) throws InterruptedException {
...
                    while (remainingNanos > 0) {
                        parkNanos(remainingNanos);
                        if (getAndClearInterrupt()) {
                            throw new InterruptedException();
                        }
                        remainingNanos = nanos - (System.nanoTime() - startNanos);
                    }
...
    }

There is a line that says that our virtual thread will be parked (parkNanos(remainingNanos)). Parking a virtual thread means yielding its continuation. Virtual thread parked for some time? Let's use that time for other virtual threads!

Finally, after going deeper and deeper into the implementation of park operation we reached the part of the code which is responsible for scheduling the unparking of the given virtual thread. Unparking the virtual thread results in that its continuation is being resubmitted to the scheduler. In our case means that after 5 seconds of sleep our virtual thread can be continued and able to print the ending log.

/**
 * Schedules this thread to be unparked after the given delay.
 */
@ChangesCurrentThread
private Future<?> scheduleUnpark(long nanos) {
    //assert Thread.currentThread() == this;
    Thread carrier = this.carrierThread;
    // need to switch to carrier thread to avoid nested parking
    carrier.setCurrentThread(carrier);
    try {
        return UNPARKER.schedule(this::unpark, nanos, NANOSECONDS);
    } finally {
        carrier.setCurrentThread(this);
    }
}

The UNPARKER is the default ScheduledExecutorService which is created for virtual threads purposes.

private static final ScheduledExecutorService UNPARKER = createDelayedTaskScheduler();

This example was based on analyzing the Thread.sleep() method, but also other blocking methods from different libraries were optimized for usage by virtual threads. The list of "virtual threads friendly" methods can be found here.

Summary

In my opinion Project Loom and the benefits it provides can be a game-changer in Java world. Providing the lightweight concurrency built-in standard libraries and performance of asynchronous code in synchronous implementations can be the fast and easy way to increase the efficiency of existing systems. I'm curious how the project will change the approach to concurrency in Java and its impact on popular libraries and frameworks.

Code samples can be found on here.

Follow me on Twitter.

Move your mTLS configuration from the codebase to a service mesh!

PiotrekSt — Fri, 13 Nov 2020 15:20:31 +0000

Have you ever wondered if you can move the implementation related to communication over mTLS from the code level to a higher layer of abstraction? In this article, I will try to show you one approach on how to make your codebase lighter.

Note: The code samples will be based on Java, Spring Boot and Istio

The case!

You've got service with business logic and some external system for which communication has to be established over mTLS connection.

In the picture above, there is an external service (the "red" one) managed by a third party to which we should perform requests using mTLS. On the other hand, service "blue" is managed by your team where you are responsible for providing the implementation of a secured connection to the external system.

First approach

The first solution that comes to our mind is to handle such a connection directly from the code. In this case, you create a dedicated HTTP client and supply it with all required data - such as keystore (with client certificate and private key), truststore (with server certificates), or a protocol version.

Sample solution of RestTemplate for handling mTLS connection can look like:

@Bean("externalServiceRestTemplate")
RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
    return restTemplateBuilder
            .requestFactory(this::requestFactory)
            .build();
    }

private ClientHttpRequestFactory requestFactory() {
    try {
        final KeyStore keyStore = KeyStore.getInstance("PKCS12");
        keyStore.load(keyStoreResource.getInputStream(), keyStorePassword.toCharArray());
        KeyStore trustStore = KeyStore.getInstance("JKS");
            trustStore.load(trustStoreResource.getInputStream(), trustStorePassword.toCharArray());

        final SSLConnectionSocketFactory socketFactory =
                new SSLConnectionSocketFactory(
                        new SSLContextBuilder()
                                .loadKeyMaterial(keyStore, keyStorePassword.toCharArray())
                                    .loadTrustMaterial(trustStore, new TrustSelfSignedStrategy())
                                .setProtocol("TLSv1.2")
                                .build());

        final HttpClient httpClient = HttpClients.custom()
                .setSSLSocketFactory(socketFactory)
                .build();
        final ClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClient);
        return requestFactory;
    } catch (Exception e) {
        e.printStackTrace();
        throw new IllegalStateException("Could not create client request factory");
    }
}

As you can see there is a lot of data such as files and passphrases which is needed to be passed directly to your source code (e.g. via K8S secrets, FlexVolumes, AWS Secret Manager, etc.).

Then calling the destination service is based on wiring a correct instance of RestTemplate and performing the HTTPS request.

@Autowired
@Qualifier("externalServiceRestTemplate")
private RestTemplate restTemplate;

@GetMapping("/client-endpoint")
public ResponseEntity<String> clientEndpoint() {
    LOGGER.info("Received request. Calling server via mTLS");
    String response = restTemplate.getForObject("https://external.system.com/secured-endpoint", String.class);
    LOGGER.info("Received response from external server {response={}}", response);
    return ResponseEntity.ok(response);
}

Service mesh to the rescue!

In this article I will not dive into the concept of what Service Mesh is. If you are not familiar with that term, I recommend you this article by William Morgan

The example below is based on Istio, which is one of the leading open source service mesh implementations. If you are interested in Istio jump to their documentation and get familiar with sample BookInfo application

The idea behind this solution is to move the responsibility for establishing a connection via mTLS to the layer outside our application. Let's let Istio do it!

The figure below shows a simplified explanation of how it will work. We've got our service, in front of it there is an istio-proxy container. It will be responsible for handling a mTLS communication in our case.

Our service will call the external system as a normal HTTP system and the proxy will upgrade the connection to HTTPS and handle stuff related to mTLS.

The configuration is as simple as it can.

Follow these steps:

Create a secret in K8S cluster, which contains required data. The structure of secret should look like:

apiVersion: v1
  kind: Secret
  type: Opaque
metadata:
  name: external-system-mtls-data
  namespace: $YOUR_NAMESPACE
data:
  cert: $CLIENT_PEM_CERTIFICATE_IN_BASE64
  key: $CLIENT_PEM_KEY_IN_BASE64
  cacert: $TRUSTED_CERTIFICATE_PEM_IN_BASE64

and then apply it:

Add resources specific for Istio:

ServiceEntry - describes the API which is external to the mesh
VirtualService - routes the traffic to external system on port 80 (http) to port 443 (http to https)
DestinationRule - defines a policy to apply mTLS when traffic is routed to the external system and specifies the directory of certificates to use

#ServiceEntry--
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-system-service-entry
  namespace: project
spec:
  hosts:
    - external.system.com
  location: MESH_EXTERNAL
  ports:
    - number: 80
      name: http-port
      protocol: HTTP
    - number: 443
      name: https-port
      protocol: HTTPS
  resolution: DNS
#DestinationRule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: external-system-dest-rule
  namespace: project
spec:
  host: external.system.com
  trafficPolicy:
    portLevelSettings:
      - port:
          number: 443
        tls:
          mode: MUTUAL
          clientCertificate: /etc/mtlsfiles/cert
          privateKey: /etc/mtlsfiles/key
          caCertificates: /etc/mtlsfiles/cacert
          sni: external.system.com
#VirtualService
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: external-system-virtual-service
  namespace: project
spec:
  hosts:
    - external.system.com
  http:
    - match:
        - port: 80
      route:
        - destination:
            host: external.system.com
            port:
              number: 443

Adjust your deployment to mount files with certificates. For these purposes, the annotations sidecar.istio.io/userVolume and sidecar.istio.io/userVolumeMount are used.
Below you can find a sample deployment with applied changes.

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: devel
  name: sample-app
  labels:
    app: sample-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "true"
        sidecar.istio.io/userVolume: '[{"name":"external-system-mtls-data","secret":{"secretName":"external-system-mtls-data"}}]'
        sidecar.istio.io/userVolumeMount: '[{"mountPath":"/etc/mtlsfiles","name":"external-system-mtls-data","readonly":true}]'
      labels:
        app: sample-app
    spec:
      containers:
      - name: sample-app
        image: your-application-image:latest
        resources:
          requests:
            memory: 300Mi
            cpu: 500m
          limits:
            memory: 450Mi
            cpu: 750m

Notice that in sidecar.istio.io/userVolume the name of secret created in Step 1 has to be used. In sidecar.istio.io/userVolumeMount path for volume mount must be consistent with one provided in definition of DestinationRule.

Verify the configuration.

First of all you can verify that your certificates are correctly mounted. To do that, just go inside your pod and verify files under the configured directory.

Next, it is time to verify the handling of mTLS connection. Now execute the HTTP request to the external system inside an application container.

Voilà! HTTP call is proxied via sidecar and mTLS connection established successfully on top of that.

Adjust your application code. In my example application code will be simplified to:

 @Autowired
 private RestTemplate restTemplate;

 @GetMapping("/client-endpoint")
 public ResponseEntity<String> clientEndpoint() {
     LOGGER.info("Received request. Calling server via mTLS");
     String response = restTemplate.getForObject("http://external.system.com/secured-endpoint", String.class);
     LOGGER.info("Received response from external server {response={}}", response);
     return ResponseEntity.ok(response);
 }

Without any special configuration of RestTemplate - just simple HTTP call to destination service.

Summary

It was quite simple to achieve moving the implementations from the application code to the service mesh. Thanks to that, it is transparent for our application to know how the connection is secured. The application's codebase is easier to understand and independent from external system security requirements.

Istio provides also an easy way to implement other patterns typical for microservices such as retry policies, circuit breaking, or API versioning.

If you have any thoughts or questions feel free to comment. The code samples can be found on my GitHub. Follow me on Twitter.