DEV Community: valerialistratova

WebSockets with ActiveJ

valerialistratova — Fri, 27 Nov 2020 11:34:45 +0000

In this tutorial we'll create a basic client-server application with a WebSocket connection using ActiveJ framework.

You need to import the following dependencies:

<dependencies>
    <dependency>
        <groupId>io.activej</groupId>
        <artifactId>activej-launchers-http</artifactId>
        <version>3.0</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.2.3</version>
    </dependency>
</dependencies>

The Logback dependency is optional.

In this tutorial we'll explicitly use the following ActiveJ technologies:

ActiveJ HTTP - high-performance asynchronous client and server implementations.
ActiveInject - lightweight and powerful dependency injection library.
ActiveJ Launcher - highly generalized main method implementation for managing application lifecycle.
ActiveJ Eventloop - provides efficient management of asynchronous operations without multithreading overhead.
ActiveJ Promise - high-performance and handy Java Future alternative.
ActiveJ Service Graph - designed to be used in combination with ActiveInject and ActiveJ Launcher as a means to start/stop application services according to their dependency graph.

WebSocket Pong Server

Let’s create a “Pong” WebSocket server. For this purpose we need to provide a RoutingServlet and use mapWebSocket method to map a Consumer of WebSocket as a servlet on /path. Our server will simply accept messages, print them out, and stream back a “Pong” message.

public final class WebSocketPongServerExample extends HttpServerLauncher {

    @Provides
    AsyncServlet servlet() {
        return RoutingServlet.create()
                .mapWebSocket("/", webSocket -> webSocket.readMessage()
                        .whenResult(message -> System.out.println("Received:" + message.getText()))
                        .then(() -> webSocket.writeMessage(Message.text("Pong")))
                        .whenComplete(webSocket::close));
    }

    public static void main(String[] args) throws Exception {
        WebSocketPongServerExample launcher = new WebSocketPongServerExample();
        launcher.launch(args);
    }
}

WebSocket Ping Client

Now let’s create a client that will send a “Ping” message to server via WebSocket connection.

public final class WebSocketPingClientExample extends Launcher {
    @Inject
    AsyncHttpClient httpClient;

    @Inject
    Eventloop eventloop;

    @Provides
    Eventloop eventloop() {
        return Eventloop.create();
    }

    @Provides
    AsyncHttpClient client(Eventloop eventloop) {
        return AsyncHttpClient.create(eventloop);
    }

    @Override
    protected Module getModule() {
        return ServiceGraphModule.create();
    }

    @Override
    protected void run() throws ExecutionException, InterruptedException {
        String url = args.length != 0 ? args[0] : "ws://127.0.0.1:8080/";
        System.out.println("\nWeb Socket request: " + url);
        CompletableFuture<?> future = eventloop.submit(() -> {
            System.out.println("Sending: Ping");
            return httpClient.webSocketRequest(HttpRequest.get(url))
                    .then(webSocket -> webSocket.writeMessage(Message.text("Ping"))
                            .then(webSocket::readMessage)
                            .whenResult(message -> System.out.println("Received: " + message.getText()))
                            .whenComplete(webSocket::close));
        });
        future.get();
    }

    public static void main(String[] args) throws Exception {
        WebSocketPingClientExample example = new WebSocketPingClientExample();
        example.launch(args);
    }
}

First, we provide the required dependencies: AsyncHttpClient and an Eventloop for it.
Next, we need to override Launcher's getModule and run methods. getModule supplies application with basic business logic module, while run represents the main Launcher method.
In run method we create a supplier and override its get method via lambda. Here we call AsyncHttpClient.webSocketRequest that sends a request and returns a Promise of a WebSocket. Then we create a Function that sends a “Ping” message and receives a response from server.

And that's it, your WebSocket application is ready to be used. First, launch the server implementation, and then client.
You can find the source code of the examples on GitHub.

Java RPC Application with ActiveRPC library

valerialistratova — Thu, 26 Nov 2020 14:43:22 +0000

What is ActiveRPC?

ActiveRPC is a lightweight and convenient Java library for developing high-load distributed applications and Memcached-like solutions. It introduces a completely alternative approach to microservices implementation and overcomes the overheads of utilizing HTTP protocol with JSON or XML encoding.

ActiveRPC is powered by one of the world's fastest JVM serializer ActiveSerializer, runs on TCP, and has a custom high-performance binary streaming protocol. It features highly optimized flexible server and client implementations along with predefined cloud strategies. With these technologies you can conveniently create even complex scalable solutions like high-performance microservices or Memcached-like servers.

Basic example

public class RpcExample extends Launcher {
    private static final int SERVICE_PORT = 34765;

    @Inject
    private RpcClient client;

    @Inject
    private RpcServer server;

    @Inject
    private Eventloop eventloop;

    @Provides
    Eventloop eventloop() {
        return Eventloop.create();
    }

    @Provides
    RpcServer rpcServer(Eventloop eventloop) {
        return RpcServer.create(eventloop)
                .withMessageTypes(String.class)
                .withHandler(String.class,
                        request -> Promise.of("Hello " + request))
                .withListenPort(SERVICE_PORT);
    }

    @Provides
    RpcClient rpcClient(Eventloop eventloop) {
        return RpcClient.create(eventloop)
                .withMessageTypes(String.class)
                .withStrategy(server(new InetSocketAddress(SERVICE_PORT)));
    }

    @ProvidesIntoSet
    Initializer<ServiceGraphModuleSettings> configureServiceGraph() {
        // add logical dependency so that service graph starts client only after it started the server
        return settings -> settings.addDependency(Key.of(RpcClient.class), Key.of(RpcServer.class));
    }

    @Override
    protected Module getModule() {
        return ServiceGraphModule.create();
    }

    @Override
    protected void run() throws ExecutionException, InterruptedException {
        CompletableFuture<Object> future = eventloop.submit(() ->
                client.sendRequest("World", 1000)
        );
        System.out.printf("%nRPC result: %s %n%n", future.get());
    }

    public static void main(String[] args) throws Exception {
        RpcExample example = new RpcExample();
        example.launch(args);
    }
}

RpcExample class extends ActiveJ Launcher to help us manage logging and the whole application lifecycle.

Next, we use dependency injection library ActiveInject to provide RpcServer and RpcClient with relevant configurations and required dependencies. RpcClient sends requests with a String message to the specified server according to the provided RPC strategy (in this case, getting a single RPC-service). For RpcServer we define the type of messages to proceed, a corresponding RpcRequestHandler and a listener port.

Since we extend Launcher, we will also override 2 methods: getModule to provide ServiceGraphModule and run to describe the main logic of the example.

Finally, we define the main method, which will launch our example.

You can find example sources on GitHub.

Combining strategies

In the previous example we've used one of the simplest strategies which simply works with a single provided port. Yet, as it was mentioned above, ActiveRPC includes pre-defined strategies, which you can simply use and combine.

For example, let's combine Round Robin and First Available strategies. First, we create 4 connections without putting connection3 into the pool. Then we start sending 20 requests. As a result, all the requests will be equally distributed between connection1 (as it is always first available) and connection4 (as connection3 isn’t available for the pool):

pool.put(ADDRESS_1, connection1);
pool.put(ADDRESS_2, connection2);
// we don't put connection3
pool.put(ADDRESS_4, connection4);
int iterations = 20;
RpcStrategy strategy = roundRobin(
        firstAvailable(servers(ADDRESS_1, ADDRESS_2)),
        firstAvailable(servers(ADDRESS_3, ADDRESS_4)));

RpcSender sender = strategy.createSender(pool);
for (int i = 0; i < iterations; i++) {
    sender.sendRequest(new Object(), 50, assertNoCalls());
}

Let's check out one more example, Type Dispatch Strategy.
This strategy simply distributes requests among shards in accordance to the type of the request. In the example all String requests are sent on the first shard which has First Valid Result strategy for the servers. Requests with all other types are sent to the second shard with First Available Strategy. As a result, connection1 and connection2 will process 35 requests, connection3 - 25 requests, while connection4 and connection5 - 0 requests as connection3 is always First Available:

pool.put(ADDRESS_1, connection1);
pool.put(ADDRESS_2, connection2);
pool.put(ADDRESS_3, connection3);
pool.put(ADDRESS_4, connection4);
pool.put(ADDRESS_5, connection5);
int timeout = 50;
int iterationsPerDataStub = 25;
int iterationsPerDataStubWithKey = 35;
RpcSender sender;
RpcStrategy strategy = typeDispatching()
        .on(String.class,
                firstValidResult(servers(ADDRESS_1, ADDRESS_2)))
        .onDefault(
                firstAvailable(servers(ADDRESS_3, ADDRESS_4, ADDRESS_5)));

sender = strategy.createSender(pool);
for (int i = 0; i < iterationsPerDataStub; i++) {
    sender.sendRequest(new Object(), timeout, assertNoCalls());
}
for (int i = 0; i < iterationsPerDataStubWithKey; i++) {
    sender.sendRequest("request", timeout, assertNoCalls());
}

In the next tutorial we'll create a Memcached-like solution that can handle up to 10 millions requests per second on a single core using ActiveRPC.

MVC Web Application with ActiveJ

valerialistratova — Thu, 19 Nov 2020 19:41:41 +0000

Intro

In this tutorial we will create an asynchronous servlet that adds contacts to a list, parses requests and processes form validation with the help of ActiveJ framework. You can find the source code of this tutorial on GitHub.

Setting up the project

We'll need the following dependencies:

<dependencies>
  <dependency>
    <groupId>io.activej</groupId>
    <artifactId>activej-launchers-http</artifactId>
    <version>3.0</version>
  </dependency>
  <dependency>
    <groupId>com.github.spullara.mustache.java</groupId>
    <artifactId>compiler</artifactId>
    <version>0.9.4</version>
  </dependency>
  <dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.2.3</version>
  </dependency>
</dependencies>

We'll use the following ActiveJ technologies:

ActiveInject - lightweight and powerful dependency injection library with great performance and no third-party dependencies.
ActiveJ HTTP - high-performance asynchronous HTTP clients and servers.
ActiveJ Launcher - takes care of full application lifecycle, service management, and logging. Perfectly compatible with ActiveInject.

Time to code!

This tutorial represents the MVC pattern:

To model a Contact representation, we will create a plain Java class with fields (name, age, address), constructor, and accessors to the fields:

class Contact {
    private final String name;
    private final Integer age;
    private final Address address;

    public Contact(String name, Integer age, Address address) {
        this.name = name;
        this.age = age;
        this.address = address;
    }

    public String getName() {
        return name;
    }

    public int getAge() {
        return age;
    }

    public Address getAddress() {
        return address;
    }

    @Override
    public String toString() {
        return "Contact{name='" + name + '\'' + ", age=" + age + ", address=" + address + '}';
    }
}

The Address class is pretty simple:

class Address {
    private final String title;

    public Address(String title) {
        this.title = title;
    }

    public String getTitle() {
        return title;
    }

    @Override
    public String toString() {
        return "Address{title='" + title + '\'' + '}';
    }
}

To simplify the tutorial, we will use an ArrayList to store the Contact objects. ContactDAO interface and its implementation are used for this purpose:

interface ContactDAO {
    List<Contact> list();

    void add(Contact user);
}

class ContactDAOImpl implements ContactDAO {
    private final List<Contact> userList = new ArrayList<>();

    @Override
    public List<Contact> list() {
        return userList;
    }

    @Override
    public void add(Contact user) {
        userList.add(user);
    }
}

To build a view we will use a single HTML file, compiled with the help of the Mustache template engine:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Decoder example</title>
  <style>
    table {
      font-family: arial, sans-serif;
      border-collapse: collapse;
      width: 100%;
    }

    td, th {
      border: 1px solid #dddddd;
      text-align: left;
      padding: 8px;
    }

    tr:nth-child(even) {
      background-color: #dddddd;
    }

    div {
      text-align: center;
    }

    span {
      color: red;
    }

  </style>
</head>
<body>
<div>
  <form action="/add" method="post">
    <table>
      <tr>
        <th>Name: <input type="text" name="name"> <span>{{errors.name}}</span></th>
        <th>Age: <input type="text" name="age"> <span>{{errors.age}}</span></th>
        <th>Address: <input type="text" name="title"> <span>{{errors.contact-address-title}}</span>
        </th>
      </tr>
    </table>
    <input type="submit" value="Submit">
  </form>
</div>
<br>
<table>
  <tr>
    <th>Name</th>
    <th>Age</th>
    <th>Address</th>
  </tr>
  {{#contacts}}
  <tr>
    <th>{{name}}</th>
    <th>{{age}}</th>
    <th>{{address.title}}</th>
  </tr>
  {{/contacts}}
</table>
</body>
</html>

An AsyncServlet will be used as a controller. We will also add RoutingServlet for routing requests to the needed endpoints. We'll get to this in a moment.

Let’s create HttpDecoderExample class which extends HttpServerLauncher. By extending HttpServerLauncher we will take care of the server’s lifecycle and service management. Next, we provide two custom parsers based on Decoder- ADDRESS_DECODER and CONTACT_DECODER - which will be used for validation. Decoder class provides you with tools for parsing requests.

public final class HttpDecoderExample extends HttpServerLauncher {
    private static final String SEPARATOR = "-";

    private static final Decoder<Address> ADDRESS_DECODER = Decoder.of(Address::new,
            ofPost("title", "")
                    .validate(param -> !param.isEmpty(), "Title cannot be empty")
    );

    private static final Decoder<Contact> CONTACT_DECODER = Decoder.of(Contact::new,
            ofPost("name")
                    .validate(name -> !name.isEmpty(), "Name cannot be empty"),
            ofPost("age")
                    .map(Integer::valueOf, "Cannot parse age")
                    .validate(age -> age >= 18, "Age must not be less than 18"),
            ADDRESS_DECODER.withId("contact-address")
    );

Also, we need to create applyTemplate(Mustache mustache, Map<String, Object> scopes) method to fill the provided Mustache template with the given data:

private static ByteBuf applyTemplate(Mustache mustache, Map<String, Object> scopes) {
    ByteBufWriter writer = new ByteBufWriter();
    mustache.execute(writer, scopes);
    return writer.getBuf();
}

Next, let’s provide a ContactDAOImpl factory method:

@Provides
ContactDAO dao() {
    return new ContactDAOImpl();
}

Now we have everything needed to create the controller AsyncServlet to handle requests:

@Provides
AsyncServlet mainServlet(ContactDAO contactDAO) {
    Mustache contactListView = new DefaultMustacheFactory().compile("static/contactList.html");
    return RoutingServlet.create()
            .map("/", request ->
                    HttpResponse.ok200()
                            .withBody(applyTemplate(contactListView, map("contacts", contactDAO.list()))))
            .map(POST, "/add", AsyncServletDecorator.loadBody()
                    .serve(request -> {
                        Either<Contact, DecodeErrors> decodedUser = CONTACT_DECODER.decode(request);

                        if (decodedUser.isLeft()) {
                            contactDAO.add(decodedUser.getLeft());
                        }
                        Map<String, Object> scopes = map("contacts", contactDAO.list());
                        if (decodedUser.isRight()) {
                            scopes.put("errors", decodedUser.getRight().toMap(SEPARATOR));
                        }
                        return HttpResponse.ok200()
                                .withBody(applyTemplate(contactListView, scopes));
                    }));
}

Here we provide an AsyncServlet, which receives HttpRequests from clients, creates HttpResponses depending on the route path and sends it.

Inside the RoutingServlet two route paths are defined. The first one matches requests to the root route / - it simply displays a contact list. The second one, /add - is an HTTP POST method that adds or declines adding new users. We will process this request parsing with the help of the aforementioned Decoder by using decode method:

Either<Contact, DecodeErrors> decodedUser = CONTACT_DECODER.decode(request);

Either represents a value of two possible data types which are either Left(Contact) or Right(DecodeErrors). In order to determine whether a parse was successful or not, we check it’s value by using the isLeft() and isRight() methods.
Finally, write down the main method which will launch our application:

public static void main(String[] args) throws Exception {
    Launcher launcher = new HttpDecoderExample();
    launcher.launch(args);
}

Time to test!

You've just created and launched an MVC web application with asynchronous and high-performance server! Now you can test the app by visiting localhost:8080.

Cluster file storage with ActiveFS library

valerialistratova — Tue, 10 Nov 2020 17:33:56 +0000

Purpose

In this tutorial we will create a cluster storage with three servers that communicate via TCP protocol. There also will
be a cluster client. Each server will have an HTTP GUI server for simplicity of use and debugging. The cluster storage
will support file uploads with automatic repartitioning according to the provided rule and replication count.

What ActiveJ technologies will be used?

ActiveFS - a lightweight abstraction for managing distributed file storage.
ActiveInject - fast and powerful dependency injection library.
ActiveJ HTTP - high-performance HTTP clients, servers and servlets.
ActiveJ Launchers - a tool for managing application lifecycle.
ActiveJ Eventloop - efficient management of asynchronous operations.
ActiveJ Config - a useful extension for properties files.

What is ActiveFS?

ActiveFS is one of the most important technologies in this tutorial. It provides a lightweight abstraction on top of common file operations like upload, download, append, list, copy, move, delete, and others. It allows operations with local, remote or distributed file storage. ActiveFS is a stand-alone technology of ActiveJ Java platform, it can be used independently of the platform.

Import the components to your project

Add all the required Maven dependencies to your project:

<dependencies>
   <dependency>
       <groupId>io.activej</groupId>
       <artifactId>activej-launchers-fs</artifactId>
       <version>3.0-rc2</version>
   </dependency>
   <dependency>
       <groupId>com.github.spullara.mustache.java</groupId>
       <artifactId>compiler</artifactId>
       <version>0.9.4</version>
   </dependency>
   <dependency>
       <groupId>ch.qos.logback</groupId>
       <artifactId>logback-classic</artifactId>
       <version>1.2.3</version>
   </dependency>
</dependencies>

activej-launchers-fs module will import all the required ActiveJ technologies that were mentioned before.

Set up Server Launcher

The first thing we need to do is to create a launcher class ClusterServerLauncher for our servers. We’ll need
the following instances:

ActiveFsServer,
local ActiveFS,
AsyncHttpServer for GUI that will simplify working with your cluster storage,
utils for repartitioning management like task schedulers, ClusterRepartitionController, and FsPartitions for tracking alive partitions and their statuses.

The partitions will communicate via TCP protocol, while GUI server will use HTTP protocol.

public final class ClusterServerLauncher extends Launcher {
    public static final Path DEFAULT_PATH = Paths.get(System.getProperty("java.io.tmpdir"), "fs-storage");

    @Provides
    public Eventloop eventloop() {
        return Eventloop.create();
    }

    @Provides
    Executor executor() {
        return Executors.newCachedThreadPool();
    }

    @Provides
    ActiveFs localActivefs(Eventloop eventloop, Executor executor, Config config) {
        return LocalActiveFs.create(eventloop, executor, config.get(ofPath(), "activefs.path", DEFAULT_PATH));
    }

    @Eager
    @Provides
    ActiveFsServer activeFsServer(Eventloop eventloop, ActiveFs activeFs, Config config) {
        return ActiveFsServer.create(eventloop, activeFs)
                .withInitializer(ofActiveFsServer(config.getChild("activefs")));
    }

    @Provides
    AsyncServlet guiServlet(ActiveFs fs, ClusterRepartitionController controller) {
        return ActiveFsGuiServlet.create(fs, "Cluster server [" + controller.getLocalPartitionId() + ']');
    }

    @Provides
    @Eager
    AsyncHttpServer guiServer(Eventloop eventloop, AsyncServlet servlet, Config config) {
        return AsyncHttpServer.create(eventloop, servlet)
                .withInitializer(ofHttpServer(config.getChild("activefs.http.gui")));
    }

    @Provides
    FsPartitions fsPartitions(Config config, Eventloop eventloop, ActiveFs fs) {
        Map<Object, ActiveFs> partitions = new LinkedHashMap<>();
        partitions.put(config.get("activefs.repartition.localPartitionId"), fs);

        return FsPartitions.create(eventloop, partitions)
                .withInitializer(ofFsPartitions(config.getChild("activefs.cluster")));
    }


    @Provides
    ClusterRepartitionController repartitionController(Config config, ActiveFsServer localServer, FsPartitions partitions) {
        String localPartitionId = first(partitions.getAllPartitions());
        assert localPartitionId != null;

        return ClusterRepartitionController.create(localPartitionId, partitions)
                .withInitializer(ofClusterRepartitionController(config.getChild("activefs.repartition")));
    }

    @Provides
    @Eager
    @Named("repartition")
    EventloopTaskScheduler repartitionScheduler(Config config, ClusterRepartitionController controller) {
        return EventloopTaskScheduler.create(controller.getEventloop(), controller::repartition)
                .withInterval(Duration.ofSeconds(1));
    }

    @Provides
    @Eager
    @Named("clusterDeadCheck")
    EventloopTaskScheduler deadCheckScheduler(Config config, FsPartitions partitions) {
        return EventloopTaskScheduler.create(partitions.getEventloop(), partitions::checkDeadPartitions)
                .withInterval(Duration.ofSeconds(1));
    }

    @Provides
    Config config() {
        return Config.create()
                .with("activefs.listenAddresses", "*:9000")
                .with("activefs.http.gui.listenAddresses", "*:8080")
                .overrideWith(Config.ofClassPathProperties("activefs-server.properties", true))
                .overrideWith(Config.ofSystemProperties("config"));
    }

    @Override
    protected final Module getModule() {
        return combine(
                ServiceGraphModule.create(),
                ConfigModule.create()
                        .withEffectiveConfigLogger());
    }

    @Override
    protected void run() throws Exception {
        awaitShutdown();
    }

    public static void main(String[] args) throws Exception {
        new BasicClusterServerLauncher().launch(args);
    }
}

Client Launcher

Now we can move on to creating a client launcher ClusterTcpClientLauncher. We need to provide a task scheduler to
detect dead partitions, AsyncHttpServer for GUI, remote ActiveFS, and FsPartitions for managing partitions.
We also need an instance of ClusterActiveFs class, an ActiveFs implementation that operates on other partitions as a cluster and contains some redundancy and
fail-tolerance capabilities.

public class ClusterTcpClientLauncher extends Launcher {
   public static final String PROPERTIES_FILE = "activefs-client.properties";

   public static final String DEFAULT_DEAD_CHECK_INTERVAL = "1 seconds";
   public static final String DEFAULT_GUI_SERVER_LISTEN_ADDRESS = "*:8080";

   @Provides
   Eventloop eventloop() {
       return Eventloop.create();
   }

   @Provides
   @Eager
   @Named("clusterDeadCheck")
   EventloopTaskScheduler deadCheckScheduler(Config config, FsPartitions partitions) {
       return EventloopTaskScheduler.create(partitions.getEventloop(), partitions::checkDeadPartitions)
               .withInitializer(ofEventloopTaskScheduler(config.getChild("activefs.repartition.deadCheck")));
   }

   @Provides
   @Eager
   AsyncHttpServer guiServer(Eventloop eventloop, AsyncServlet servlet, Config config) {
       return AsyncHttpServer.create(eventloop, servlet)
               .withInitializer(ofHttpServer(config.getChild("activefs.http.gui")));
   }

   @Provides
   AsyncServlet guiServlet(ActiveFs activeFs) {
       return ActiveFsGuiServlet.create(activeFs, "Cluster FS Client");
   }

   @Provides
   ActiveFs remoteActiveFs(Eventloop eventloop, FsPartitions partitions, Config config) {
       return ClusterActiveFs.create(partitions)
               .withInitializer(ofClusterActiveFs(config.getChild("activefs.cluster")));
   }

   @Provides
   FsPartitions fsPartitions(Eventloop eventloop, Config config) {
       return FsPartitions.create(eventloop)
               .withInitializer(ofFsPartitions(config.getChild("activefs.cluster")));
   }

   @Provides
   Config config() {
       return createConfig()
               .overrideWith(Config.ofClassPathProperties(PROPERTIES_FILE, true))
               .overrideWith(Config.ofSystemProperties("config"));
   }

   protected Config createConfig(){
       return Config.create()
               .with("activefs.http.gui.listenAddresses", DEFAULT_GUI_SERVER_LISTEN_ADDRESS)
               .with("activefs.repartition.deadCheck.schedule.type", "interval")
               .with("activefs.repartition.deadCheck.schedule.value", DEFAULT_DEAD_CHECK_INTERVAL);
   }

   @Override
   protected final Module getModule() {
       return combine(
               ServiceGraphModule.create(),
               JmxModule.create(),
               ConfigModule.create()
                       .withEffectiveConfigLogger());
   }

   @Override
   protected void run() throws Exception {
       awaitShutdown();
   }

   public static void main(String[] args) throws Exception {
       new ClusterTcpClientLauncher().launch(args);
   }
}

Here’s the architecture of our distributed P2P storage:

You can create as many partitions as you wish, cluster client is optional as you can create an alternative client
implementation that supports ActiveFS protocol.

Testing out the storage

Let’s launch three partitions on different ports. For this purpose you need to create three Run/Debug configurations for ClusterTcpServerLauncher and provide the following VM options:

-Dconfig.activefs.listenAddresses=*:9001
-Dconfig.activefs.http.gui.listenAddresses=*:8081
-Dconfig.activefs.cluster.partitions=127.0.0.1:9001,127.0.0.1:9002,127.0.0.1:9003
-Dconfig.activefs.repartition.replicationCount=2
-Dconfig.activefs.path=/tmp/server1
-Dconfig.activefs.repartition.localPartitionId=127.0.0.1:9001

-Dconfig.activefs.listenAddresses=*:9002
-Dconfig.activefs.http.gui.listenAddresses=*:8082
-Dconfig.activefs.cluster.partitions=127.0.0.1:9001,127.0.0.1:9002,127.0.0.1:9003
-Dconfig.activefs.repartition.replicationCount=2
-Dconfig.activefs.path=/tmp/server2
-Dconfig.activefs.repartition.localPartitionId=127.0.0.1:9002

-Dconfig.activefs.listenAddresses=*:9003
-Dconfig.activefs.http.gui.listenAddresses=*:8083
-Dconfig.activefs.cluster.partitions=127.0.0.1:9001,127.0.0.1:9002,127.0.0.1:9003
-Dconfig.activefs.repartition.replicationCount=2
-Dconfig.activefs.path=/tmp/server3
-Dconfig.activefs.repartition.localPartitionId=127.0.0.1:9003

Next, provide the following VM options for ClusterTcpClientLauncher:

-Dconfig.activefs.http.gui.listenAddresses=*:8080
-Dconfig.activefs.cluster.partitions=127.0.0.1:9001,127.0.0.1:9002,127.0.0.1:9003

The storage will run with the following configurations:

Server1: TCP-connection port 9001, HTTP GUI port 8081
Server2: TCP-connection port 9002, HTTP GUI port 8082
Server3: TCP-connection port 9003, HTTP GUI port 8083
Client: HTTP GUI port 8080

Launch all the created configurations and go to 127.0.0.1:8080 in your browser to work with the storage. Switch between ports (8081, 8082, 8083) to check your partitions’ GUI.
You can try to upload files, create directories, kill some partitions to check repartitioning.

ActiveSerializer. Ultra-fast Java Serialization Library

valerialistratova — Thu, 05 Nov 2020 18:24:31 +0000

ActiveSerializer is a modern bytecode generator of extremely fast and space-efficient serializers for data transferring. It's core features are ultimate performance and streamlined programming model. ActiveSerializer doesn't create redundant layers of data transfer objects and works straight with Java classes with the help of annotations.

ActiveSerializer is a really powerful tool:

Works directly with Java classes via annotations. No additional layers of intermediate DTO classes.
Implemented using runtime bytecode generation to be compatible with dynamically created classes.
Stable binary format with backward binary compatibility.
Support of the scheme evolution: changeable versions, added or removed fields, etc.
Can be easily extended - you can write your own plugins for specific classes.
Includes special hints for even more efficient code: String formats, nullable, varlen, etc.
Provides little endian format for JVM intrinsics.
Support of unsafe mode for the best performance.
Cyclic references of any complexity.
UTF-8, UTF-16 and ISO8859-1 encoding.
Compatible even with complex collections, generics and nullable values.

Performance

According to the ActiveSerializer website, it claims to be the fastest JVM-based serializer among the existing according to the results of a popular JVM Serializer benchmark tool.

Such performance of ActiveSerializer is a result of well-designed architecture and wide use of ActiveCodegen library for dynamic runtime code generation. Let's test ActiveSerializer out!

Simple Object Serialization

Let's create a serializable POJO using @Serialize and @Deserialze annotations:

public static class Person {
    //use @Deserialize annotation with property name in constructor
    public Person(@Deserialize("age") int age, @Deserialize("name") String name) {
        this.age = age;
        this.name = name;
    }

    // Use @Serialize annotation with order number on properties or their getters.
    // The name of the property must be the same as declared in @Deserialize annotation.
    // order number provides compatibility.
    @Serialize(order = 0)
    public int age;

    @Serialize(order = 1)
    public final String name;

    private String surname;

    @Serialize(order = 2)
    public String getSurname() {
        return surname;
    }

    public void setSurname(String surname) {
        this.surname = surname;
    }
}

This is enough to make a class serializable. Now let's create a Person instance and a BinarySerializer (represents a serializer that will encode and decode <Person> values to byte arrays). So we will also need a byte array to store the result of the serialization.

Person john = new Person(34, "John");
john.setSurname("Smith");
byte[] buffer = new byte[200];
BinarySerializer<Person> serializer = SerializerBuilder.create(getSystemClassLoader())
    .build(Person.class);

In order to encode and decode our instance, simply use encode and decode methods:

serializer.encode(buffer, 0, john);
Person johnCopy = serializer.decode(buffer, 0);

Let's move on to something a little bit more complicated.

Generics and Interfaces Serialization

ActiveSerializer allows to work with interfaces and generics with a simple DSL. First, let's create a Skill class:

public static class Skill<K, V> {
    private K key;
    private V value;

    public Skill(@Deserialize("key") K key, @Deserialize("value") V value) {
        this.key = key;
        this.value = value;
    }

    @Serialize(order = 0)
    public K getKey() {
        return key;
    }

    @Serialize(order = 1)
    public V getValue() {
        return value;
    }
}

Next, we'll create a Person interface. It will have a method that returns a list of skills:

public interface Person&lt;K, V> {
    @Serialize(order = 0)
    List<Skill<K, V>> getSkills();
}

Now we need to create a class that will implement Person interface. Let it be a Developer:

public static class Developer implements Person<Integer, String> {
    private List<Skill<Integer, String>> list;

    @Serialize(order = 0)
    @Override
    public List<Skill<Integer, String>> getSkills() {
        return list;
    }

    public void setSkills(List<Skill<Integer, String>> list) {
        this.list = list;
    }
}

Now create an instance of Developer, BinarySerializer<Developer>, and a byte array:

Developer developer = new Developer();
developer.setSkills(Arrays.asList(
    new Skill<>(1, "Java"),
    new Skill<>(2, "ActiveSerializer")));

byte[] buffer = new byte[200];
BinarySerializer<Developer> serializer = SerializerBuilder.create(getSystemClassLoader())
        .build(Developer.class);

Now you can serialize and deserialize Developer instance with encode and decode methods:

serializer.encode(buffer, 0, developer);
Developer developer2 = serializer.decode(buffer, 0);

Have any questions? You're welcome in the comment section!

Processing AST with ActiveSpecializer

valerialistratova — Thu, 05 Nov 2020 17:42:41 +0000

Intro

ActiveSpecializer is a lightweight library that automagically optimizes your code for JVM and makes it significantly faster. Unlike traditional compiler optimization techniques, it relies on an alternative concept of using runtime information of classes instances, rewriting the bytecode in runtime. It transforms all class fields into static class fields, and de-virtualizes all virtual methods calls, replaces them with static method calls.

It is a really powerful tool, particularly when it comes to tree-like structures. Let's take a look at a use case.
We will use ActiveSpecializer for transforming ASTs that we'll receive after equation parsing. In the end of the article we'll make some benchmarks to see how ActiveSpecializer coped with the problem. Let's get started!

Parsing Equation to AST

This tutorial is based on the Parsec calculator tutorial. Yet, it has an important distinction. In the original tutorial Parsec parses expressions to double values:

Parser<Double> parser = new OperatorTable<Double>()
    .infixl(op("+", (l, r) -> l + r), 10)
    .infixl(op("-", (l, r) -> l - r), 10)
    .infixl(Parsers.or(term("*"), WHITESPACE_MUL).retn((l, r) -> l * r), 20)
    .infixl(op("/", (l, r) -> l / r), 20)
    .prefix(op("-", v -> -v), 30)
    .build(unit);

Instead, we will parse the expressions to an AST:

private static final Parser<CalculatorExpression> EXPRESSION = new OperatorTable<CalculatorExpression>()
        .infixl(DELIMITERS.token("+").retn(Sum::new), 10)
        .infixl(DELIMITERS.token("-").retn(Sub::new), 10)
        .infixl(DELIMITERS.token("*").retn(Mul::new), 20)
        .infixl(DELIMITERS.token("/").retn(Div::new), 20)
        .infixl(DELIMITERS.token("%").retn(Mod::new), 20)
        .prefix(DELIMITERS.token("-").retn(Neg::new), 30)
        .infixr(DELIMITERS.token("^").retn(Pow::new), 40)
        .build(ATOM);

For example, 2 - 4 * 6 will be parsed like this:

We will parse to an AST the following equation: ((2 + 2 * 2) * -x) + 5 + 1024 / (100 + 58) * 50 * 37 - 100 + 2 * x ^ 2 % 3.

ActiveSpecializer will transforms the received AST to a set of static final classes with baked-in values of the provided equation. During runtime JIT significantly optimizes and inlines these classes. As a result, we will receive an optimized reusable expression instance.

All you need to do to use ActiveSpecializer is:

public static final Parser<CalculatorExpression> PARSER = EXPRESSION.from(LEXER, IGNORED);

private static final Specializer SPECIALIZER = Specializer.create(Thread.currentThread().getContextClassLoader());

public static void main(String[] args) {
    double x = -1;

    CalculatorExpression expression = PARSER.parse("((2 + 2 * 2) * -x) + 5 + 1024 / (100 + 58) * 50 * 37 - 100 + 2 * x ^ 2 % 3");
    CalculatorExpression specialized = SPECIALIZER.specialize(expression);
    System.out.println(specialized.evaluate(x));
}

Benchmarks

It’s time for some benchmarks. We will process the aforementioned equation in three different ways and compare the performance:

String source = "((2 + 2 * 2) * -x) + 5 + 1024 / (100 + 58) * 50 * 37 - 100 + 2 * x ^ 2 % 3";

manual = x -> ((2.0 + 2.0 * 2.0) * -x) + 5.0 + 1024.0 / (100.0 + 58.0) * 50.0 * 37.0 - 100.0 + 2.0 * (Math.pow(x, 2.0)) % 3.0;
ast = SpecializerCalculatorExample.PARSER.parse(source);
specialized = SPECIALIZER.specialize(ast);

manually enter the equation
parse the equation to an AST and evaluate it without specialization
parse the equation to an AST and evaluate it with specialization

We used JMH in AvarageTime mode as a benchmark tool. All the results are represented as nanoseconds per operation.

Benchmark                        Mode  Cnt    Score   Error  Units
CalculatorBenchmark.ast          avgt   10  828.924 ± 8.369  ns/op
CalculatorBenchmark.manual       avgt   10  115.985 ± 1.009  ns/op
CalculatorBenchmark.specialized  avgt   10  117.635 ± 1.500  ns/op

As you can see, specialized AST was processed just as fast as a a manually typed equation, while non-specialized AST was processed 8 times slower. ActiveSpecializer proved its substantial efficiency!

ActiveInject. Fast and Lightweight Dependency Injection Library

valerialistratova — Mon, 19 Oct 2020 14:59:19 +0000

What is ActiveInject?

ActiveInject is a lightning-fast and powerful dependency injection library. It has a lot of tools and features to offer: support for nested scopes, singletons and transient bindings, modules, multi-threaded and single-threaded injectors.

At the same time it’s thoroughly optimized with all the dependencies graph preprocessing performed at startup time. According to the benchmarks, in some scenarios ActiveInject is 5.5 times faster than Guice and hundreds of times faster than Spring DI. You can check the benchmark sources here.

ActiveInject is an independent technology of ActiveJ platform. It has no third-party dependencies on its own and can be used as a stand-alone DI library.

Getting started

Let’s try the library out and bake some virtual cookies using ActiveInject. A cookie requires the following ingredients: Flour, Sugar and Butter. These ingredients form a Pastry which can be baked into a Cookie. Assume each of these entities has a POJO. Let’s start with a basic example:

public void provideAnnotation() {
  Module cookbook = new AbstractModule() {
     @Provides
     Sugar sugar() { return new Sugar("WhiteSugar", 10.f); }

     @Provides
     Butter butter() { return new Butter("PerfectButter", 20.0f); }

     @Provides
     Flour flour() { return new Flour("GoodFlour", 100.0f); }

     @Provides
     Pastry pastry(Sugar sugar, Butter butter, Flour flour) {
        return new Pastry(sugar, butter, flour);
     }

     @Provides
     Cookie cookie(Pastry pastry) {
        return new Cookie(pastry);
     }
  };

  Injector injector = Injector.of(cookbook);
  injector.getInstance(Cookie.class).getPastry().getButter().getName());
}

Here we’ve created an AbstractModule named cookbook that contains all the required bindings, or “recipes”, for the ingredients. Call Injector.getInstance method to get an instance of the Cookie.

How does it work from the inside? Injector provides all the required dependencies for the component recursively traversing the dependencies graph in a postorder way. So it first created Sugar, Butter and Flour, the next was Pastry, and finally a Cookie.

Named bindings

But what if you need some special cookie recipes for different people? For example, if you need a sugar-free Cookie in addition to a regular one. In this case, you can use @Named annotation:

public void namedAnnotationSnippet() {
  Module cookbook = new AbstractModule() {
     @Provides
     @Named("zerosugar")
     Sugar sugar() { return new Sugar("SugarFree", 0.f); }

     @Provides
     @Named("normal")
     Sugar sugar2() { return new Sugar("WhiteSugar", 10.f); }

     @Provides
     Butter butter() { return new Butter("PerfectButter", 20.f); }

     @Provides
     Flour flour() { return new Flour("GoodFlour", 100.f); }

     @Provides
     @Named("normal")
     Pastry pastry1(@Named("normal") Sugar sugar, Butter butter, Flour flour) {
        return new Pastry(sugar, butter, flour);
     }

     @Provides
     @Named("zerosugar")
     Pastry pastry2(@Named("zerosugar") Sugar sugar, Butter butter, Flour flour) {
        return new Pastry(sugar, butter, flour);
     }

     @Provides
     @Named("normal")
     Cookie cookie1(@Named("normal") Pastry pastry) {
        return new Cookie(pastry);
     }

     @Provides
     @Named("zerosugar")
     Cookie cookie2(@Named("zerosugar") Pastry pastry) { return new Cookie(pastry); }
  };
}

The usage of the @Named annotation is pretty self-illustrative. After annotating bindings you can call either injector.getInstance(Key.ofName(Cookie.class, "normal")) or injector.getInstance(Key.ofName(Cookie.class, "zerosugar")) and get different instances of the Cookie class.

Non-singleton instances

One of the important ActiveInject features is that all the created instances are singleton by default. If you call injector.getInstance(Cookie.class) several times, each time you’ll get the very same instance from cache. You can easily check it with the following code:

AbstractModule cookbook = new AbstractModule() {
    @Provides
    Integer giveMe() {
        return random.nextInt(1000);
    }
};
Injector injector = Injector.of(cookbook);
Integer firstInt = injector.getInstance(Integer.class);
Integer secondInt = injector.getInstance(Integer.class);
System.out.println("First : " + firstInt + ", second  : " + secondInt);

This is a useful optimization, but what if we need non-singleton cookies? In this case, you can use scopes.

Scope creates “local singletons” which live as long as the scope itself. ActiveInject scopes are a bit different from other DI libraries. The internal structure of the Injector is a prefix tree and the prefix is a scope. If you create an Injector that is set to a particular scope, it means that Injector enters this scope. This can be done several times, so that there are multiple injectors in a single scope.

Let’s create a scope for our cookies. The identifiers of the tree are annotations. So you can simply create your custom scope annotations:

@ScopeAnnotation(threadsafe = false)
@Target({ElementType.METHOD})
@Retention(RUNTIME)
public @interface OrderScope {
}

Next, instantiate your scope:

public static final Scope ORDER_SCOPE = Scope.of(OrderScope.class);

Add @OrderScope annotation to the instances that are needed as non-singletons (Cookie, Pastry, Sugar, Flour, Butter) in the following way:

Module cookbook = new AbstractModule() {
  @Provides
 @OrderScope
  Sugar sugar() { return new Sugar("WhiteSugar", 10.f); }

…

  @Provides
  Kitchen kitchen() {return new Kitchen();}
};

Now each time we need a new instance of Сookie, our Injector will create a subinjector to enter the order scope and create a new instance of Cookie, while the Kitchen will always stay singleton in the root scope.

Using ActiveInject with ActiveSpecializer

One more interesting ActiveInject feature is that it is fully compatible with another ActiveJ library named ActiveSpecializer.

ActiveSpecializer is a unique technology that relies on a ground-breaking concept of using runtime information about instances of the classes. ActiveSpecializer rewrites your code directly in runtime, using runtime information that is contained in the instances of your classes. To be more precise, it transforms all class fields into static class fields, de-virtualizes all the virtual methods calls and replaces them with static method calls. ActiveSpecializer shows its best with AST-like structures, so it is very efficient with ActiveInject. According to the benchmarks, ActiveSpecializer can speed up your code to up to 7 times in some use cases.

To apply ActiveSpecializer to your Injector simply call Injector.useSpecializer method before Injector instantiation.

Summary

ActiveInject goes far beyond these basic examples and is capable of fine tuning to match all your needs. You can find more examples on the official ActiveInject website.

Since ActiveInject is a part of ActiveJ, it is perfectly compatible with all the platform’s components: HTTP servers and servlets, RPC implementation, bytecode manipulation tools, abstractions for efficient management of distributed file storage and others.