DEV Community: Darshit Patel

Using Java's serialVersionUID

Darshit Patel — Sun, 24 Aug 2025 11:47:22 +0000

"What's a serialVersionUID used for?"

"What if you do not define it?"

These seem to be pretty popular Java interview questions. The second one builds on the top of the first. But, what's more popular than these questions are the blog posts explaining what the right answer is.

Here's the gist:
serialVersionUID is used to track the version of an object. Each JVM, when serializing an object, by uses this field to version it. When deserializing this object, the JVM then references the version again, to check if the objects are of the same class, and if the version does not match, throws up the InvalidClassException.

In what case is there a version mismatch? When you do not define the serialVersionUID, and the JVM compiler implementation differs. For example: Oracle JDK vs Amazon Corretto JDK. The autogenerated serialVersionUID may have different values.

The intention of using in the same serialVersionUID is that when a class changes, the version should be kept intact to allow for backward compatibility.

Fair, this is correct. But what these blogs fail to mention, and the interviewers fail to question is, why would you use Java serialization in the current age?

If the intention is backward compatibility amongst different JVMs, there are better encodings available - JSON, Protobuf, etc.

Over the network transfer and serialization in most modern apps has been taken over by these encodings. It's human-readable, inherently backward compatible (for adding/removing fields, not modifying fields), for plain serialization and deserialization at the very least.

The most underrated part - these encodings can be used to test APIs. Imagine trying to create a Java Serialized object to test a simple API with Postman.

Most APIs now use JSON, and if you indeed want to version it, you can, without breaking the backward compatibility! No going over the hoops to learn the why's and when's to use serialVersionUID.

Want to read more on Java Serialization? Look up these links:

Mastering ExecutorService Shutdown: Tracking ThreadPool Termination

Darshit Patel — Sat, 04 Jan 2025 08:19:53 +0000

Let's say you want to execute some tasks. Since executing it through a single thread might take you quite some time to get the result, you decide to use the ever dependable ExecutorService to process it through multiple threads.

Here's a sample:

public static void main(String[] args) {
    ExecutorService executorService = Executors.newFixedThreadPool(5);
    for (int i = 0; i < 5; i++) {
        int temp = i;
        executorService.submit(() -> {
            task(temp);
        });
    }
    executorService.shutdown();
    System.out.println("ExecutorService is shutdown");
}

private static void task(int temp) {
    try {
        TimeUnit.SECONDS.sleep(1L);
        System.out.println("Task " + temp + " completed");
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
}

Of course, as usual, no example of Threads is ever complete without using "sleep" as an archetype of task execution.

It outputs,

ExecutorService is shutdown
Task 1 completed
Task 2 completed
Task 0 completed
Task 4 completed
Task 3 completed

Now imagine there's an endless queue of tasks, the number of which you don't know about. Maybe they are determined by the number of entries in a database which get added dynamically.

For example, a bank, wherein which it has to process a number of transactions throughout the day. The transaction end time will be 5 PM, beyond which it will not accept any additional tasks.

However, you do know that the number of tasks will be finite, and will end at some point of time.

How do you know the point in time when all the tasks have completed?

If you notice the above code snippet, the ExecutorService.shutdown() enables the main thread to exit immediately, but the background threads still process the accepted tasks to completion. Is there a way when you can get notified about the completion?

A couple of solutions come to mind:

Use a CountDownLatch to count the tasks - but since you don't know the number of tasks, it's impractical to use it.
Use ExecutorService.awaitTermination(). However, the time here is still undeterministic. You can use a very liberal ExecutorService.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS) or something similar. But that is again a blocking call.

Is there a better way to solve this?

Java does provide a better and relatively unknown way to get around this. The "trick" here is to know that Executors.newFixedThreadPool is essentially a ThreadPoolExecutor with predefined values. Let's check the implementation of Executors.newFixedThreadPool.

public static ExecutorService newFixedThreadPool(int nThreads) {
    return new ThreadPoolExecutor(nThreads, nThreads,
                                  0L, TimeUnit.MILLISECONDS,
                                  new LinkedBlockingQueue<Runnable>());
}

I really recommend to read the doc for ThreadPoolExecutor here. ExecutorService is a convenient wrapper over ThreadPoolExecutor.

... programmers are urged to use the more convenient Executors factory methods Executors.newCachedThreadPool() (unbounded thread pool, with automatic thread reclamation), Executors.newFixedThreadPool(int) (fixed size thread pool) and Executors.newSingleThreadExecutor() (single background thread), that preconfigure settings for the most common usage scenarios.

The section that will help us resolve out problem is:

Hook methods

This class provides protected overridable beforeExecute(Thread, Runnable) and afterExecute(Runnable, Throwable) methods that are called before and after execution of each task. These can be used to manipulate the execution environment; for example, reinitializing ThreadLocals, gathering statistics, or adding log entries. Additionally, method terminated() can be overridden to perform any special processing that needs to be done once the Executor has fully terminated.

We can use the terminated method to notify us of the same! But how do we use it?

public static void main(String[] args) {
    ExecutorService executorService = new ThreadPoolExecutor(5, 5,
            0L, TimeUnit.MILLISECONDS,
            new LinkedBlockingQueue<>()) {
        @Override
        protected void terminated() {
            super.terminated();
            System.out.println("ExecutorService is terminated");
        }
    };
    for (int i = 0; i < 5; i++) {
        int temp = i;
        executorService.submit(() -> {
            task(temp);
        });
    }
    executorService.shutdown();
    System.out.println("ExecutorService is shutdown");
}

private static void task(int temp) {
    try {
        TimeUnit.SECONDS.sleep(1L);
        System.out.println("Task " + temp + " completed");
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
}

If you do not prefer Anonymous classes (like me), you can always extend ThreadPoolExecutor yourself to create a custom one.

public static void main(String[] args) {
    ExecutorService executorService = getThreadPoolExecutor();
    for (int i = 0; i < 5; i++) {
        int temp = i;
        executorService.submit(() -> {
            task(temp);
        });
    }
    executorService.shutdown();
    System.out.println("ExecutorService is shutdown");
}

private static ThreadPoolExecutor getThreadPoolExecutor() {
    return new CustomThreadPoolExecutor(5, 5,
            0L, TimeUnit.MILLISECONDS,
            new LinkedBlockingQueue<>());
}

static class CustomThreadPoolExecutor extends ThreadPoolExecutor {
    public CustomThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
    }

    @Override
    protected void terminated() {
        super.terminated();
        System.out.println("ExecutorService is terminated");
    }
}

private static void task(int temp) {
    try {
        TimeUnit.SECONDS.sleep(1L);
        System.out.println("Task " + temp + " completed");
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
}

Here's the output to verify if it works as per our expectations.

ExecutorService is shutdown
Task 0 completed
Task 3 completed
Task 2 completed
Task 4 completed
Task 1 completed
ExecutorService is terminated

What are some other relatively unknown snippets you use? Let me know in the comments!

SSH Tunnelling with Java

Darshit Patel — Sat, 21 Sep 2024 15:59:25 +0000

You heave a sigh of relief, as the QA has approved a long-awaited feature for deployment on Prod. However, as a part of the process, it is first deployed on the UAT env, where there are test accounts that can be used to certify the feature works outside of the local developer and QA testing systems.

But... running the test suite results in a lot of failures!

Reply with the always relevant "But it works on my machine!"

What do you do? You're sure that the code works. Defintely sure. Maybe the problem is with the UAT environment then? But what could be the problem with the environment? Maybe the test accounts which are newly created, are configured incorrectly? Probably yes, you think. You have access to the logs for the environment, but the scenario which it is failing for, has limited to no logs to identify the problem. You internally curse your ancestral developers.

Another option? Remote debugging! While this is a good idea, it is seldom practical for environments where the apps are under constant use. If your code is in the "hot path", good luck figuring out what requests are yours. Also it may slow down the app significantly.

Essentially, what you want is to debug the application, as if it was deployed on your local machine, but the database of the UAT server. But since UAT server is not directly accessible to your local application, you're out of luck.

Or are you? Fret not, because an SSH Tunnel is here to your rescue.

SSH Tunnel

What is an SSH Tunnel? And how do I use it?

Short answer:

SSH Tunneling will allow your application to behave as if it is deployed on a remote system, through which the UAT database is accessible.

Long answer:

The Linux ssh command provides a functionality to "port forward". Admittedly, the term "port forward" is pretty non-descriptive. Hence, I'll let StackOverflow provide to you a detailed, and a far easier explanation of SSH Tunneling than what I can here. You can read it here: https://unix.stackexchange.com/a/115906. I recommend you to read the answer because it has diagrams that are far easy to understand than the text based answers.

However, I'll still copy the relevant sections here.

ssh -L 123:farawayhost:456 remotehost
local: -L Specifies that the given port on the local (client) host is to be forwarded to the given host and port on the remote side.
ssh -L sourcePort:forwardToHost:onPort connectToHost means: connect with ssh to connectToHost, and forward all connection attempts to the local sourcePort to port onPort on the machine called forwardToHost, which can be reached from the connectToHost machine.

Example

For our use case, our sample command will be:

ssh -L <Local Port>:<UAT Database IP>:<UAT Database Port> <JumpHost IP>

Note, here the JumpHost is a system which can connect to <UAT Database IP> with the port <UAT Database Port>.

Once we have this figured out, everything else is a cakewalk!

You can simply run the command, and you'll be able to access the UAT Database on your localhost:<Local Port>. Almost feels like magic!

If you need to connect to multiple databases, you will need to run this command multiple times. Or you can write a bash script which can dynamically read a config file to open multiple SSH Tunnels.

I, being a Java programmer, would rather deal with statically compiled Java code, than worry about a dynamically typed language like bash. So, I utilized the jsch library to whip up an extensible project, which creates and maintains multiple SSH Tunnels. You can check it out here: https://github.com/darshitpp/java-ssh-tunnel

Structure

java-ssh-tunnel
└── src
    └── main
        ├── java
        │   └── dev
        │       └── darshit
        │           └── java_ssh_tunnel
        │               ├── Main.java
        │               ├── MultiTunneler.java
        │               ├── Tunneler.java
        │               └── ssh
        │                   ├── TunnelDetails.java
        │                   └── UserDetails.java
        └── resources

Usage

Download the project
Load up in your IDE
Run mvn clean install
Change Main.java with required details like SSH username, SSH password, JumpHost sshHost
(Optional) Maybe load up localPort, remoteHost, and remotePort details from a file
Run Main.java

If all things go well, you'll see the following output on your stdout

Starting tunneling...
<remoteHost>:<remotePort> is available on localhost:<localPort>
Press Enter to terminate the tunnels...

Caveats

While the above is very convenient, DO NOT USE IT TO CONNECT TO PROD. Yes, that had to be written in bold with emphasis. Shall I print the message to stderr too? Comment below.

Handling limited sudo access through Ansible

Darshit Patel — Sun, 31 Mar 2024 06:49:42 +0000

Running specific commands with sudo using Ansible

It seems a trivial use case, but it's not. Consider a situation when you're not the user allowed a full sudo access on the remote machine. However, you are only allowed the sudo access on certain commands, which, of course, requires a password. For example, starting or stopping a service.

Ansible allows privilege escalation, but that will not solve your problem because:

Elevated access is changed to the user root by default
Elevated access requires a password, i.e. password for the root user

You obviously don't have access to the root user.

There are a couple of other directives you can use, which the ansible doc makes it hard to understand even though it tries to be simple in language. You can try to use the ansible_become_user to change to another user while escalating the privilege besides root. However, if you are already the user who has the sudo command access, this will not help.

If you have access to change (or get changed) the /etc/sudoers file, you can change the sudo command to be run through NOPASSWD.

If you are here, it is likely that you do not have the permissions to change the sudoers file either.

Resolution

I saw a couple of workarounds mentioned across Github¹, StackOverflow², and Reddit³⁴, but they were outdated, and didn't work for me.

One of the solution⁵ suggests to use sudo with -S and echo the password from the terminal. This will, however, probably leak your password on the shell, and we don't want this, do we?

However, is there a way Ansible can input the password to sudo? Yes! Perhaps we can use something like the -y flag that is often used to skip prompts of yes/no on the terminal. Thankfully, Ansible is built over Python, and most Python modules are available within Ansible. "How does that help?", you ask. Python has a very helpful module called pexpect. Similarly, Ansible has expect.

- name: Stop service
  ansible.builtin.expect:
    command: "sudo systemctl stop service"
    responses:
      (?i)password: "{{ pass }}"
  no_log: true

This solves our problem! We can load this password through Ansible vault, and pass it in without being logged anywhere! We also do not have to rely on having access or permissions through the /etc/sudoers file, saving a lot of headache of trying to get approvals.

Dynamic Enums

Darshit Patel — Sat, 03 Jul 2021 15:16:53 +0000

But can one even make Enums dynamic?

Enums, by definition, are static. They are used to "predefine" constants. I'll let the official Oracle Java Tutorial on Enums to help explain the definition.

An enum type is a special data type that enables for a variable to be a set of predefined constants. The variable must be equal to one of the values that have been predefined for it. Common examples include compass directions (values of NORTH, SOUTH, EAST, and WEST) and the days of the week.

public enum Day {
    SUNDAY, MONDAY, TUESDAY, WEDNESDAY,
    THURSDAY, FRIDAY, SATURDAY 
}

The values of an enum are known to the compiler at compile time, are not supposed to change. Imagine if someone tried to introduce a new day of the week. This is pretty unlikely, and thus, for all intents and purposes, the days of the week remain the same.

However, you are now working on an application which has a legacy codebase. This legacy code has an enum named Colour (the British spelling, because I'm not in the US). The actual enum might be something different and complicated, but the example of Colour works for the purposes of this article.

Colour has three colours defined inside it -- RED, GREEN, and BLUE.

public enum Colour {

    RED(255, 0, 0),
    GREEN(0, 255, 0),
    BLUE(0, 0, 255);

    int r, g, b;

    Colour(int r, int g, int b) {
        this.r = r;
        this.g = g;
        this.b = b;
    }
    // getters and toString()
}

The Colour enum is being used across lots of applications as a dependency. First, you wonder why someone needed to put the Colours as an Enum datatype. But unable to do anything about it, you just accept it and work with whatever is in your fate.

The problem with the above enum is, every time some app needs a new colour, you need to add the new colour to your codebase. After adding a new colour, you now need to test, and then re-deploy all your applications. This has to be done every time a new colour is requested. You see the problem -- the colours are supposed to be dynamic, but someone took a decision an eternity ago, and now you are the one in soup.

You now want to change enums to make them dynamic, so that any new colour added, is picked up from a database, with minimal code changes across your stack. How do you proceed?

Enums are also classes

Yes! Enums are also classes. We know that enums are static, and classes are not. Technically, you could replace Enum with a class. If you replace an Enum with a Class, you would not even need to change imports in other classes. No part of your code base would even realize it! This will be the basis of our solution -- turn Enum into a Class.

But wait! It's not so straightforward. Enums can also be referred directly, like Colour.RED. We do not want to meddle with existing usages of Colour.RED in our code.

Step 1: Change `enum` to `class`

When you change the enum to class, your IDE should instantly throw up an error. This is because Enums are compile time constructs which are being referred in other parts of you application. But not to worry!

Step 2: Create constants of existing Colours

To keep the usages of Colour.RED same, we would need to create constants for all the colours defined in the enum (now class).

public final class Colour {

    public static final Colour RED = new Colour(255, 0, 0);
    public static final Colour GREEN = new Colour(0, 255, 0);
    public static final Colour BLUE = new Colour(0, 0, 255);

    private final int r, g, b;

    private Colour(int r, int g, int b) {
        this.r = r;
        this.g = g;
        this.b = b;
    }
    // getters and toString()
}

We also do not want the objects of Colour to be created outside the class. If we do not do this, we are essentially allowing anyone to create Colour objects, and defeats the purpose of mimicing the enum. Thus, we would mark the constructor as private.

Congrats! The IDE should stop raising an error now (inside the Colour class atleast). The above refactoring makes sure that you can still refer to Colour.RED as earlier. Also, now that we made the variables of r, g, and b as private final as we do not want them to change after object creation.

Step 3: Implement other Enum methods like `values()` and `valueOf()`

We know that methods like values() and valueOf() are used quite often with enums. To ensure that these usages do not break, we would need to "mimic" these methods. How can we do that?

Note the return types of values() and valueOf().

valueOf(String) returns in instance of Colour defined by the name provided as a parameter.
values() returns an array of Colour, i.e. Colour[]

Let us start with the valueOf() method, and it will lead us to the solution of values()

As we know, the valueOf() accepts a String as an argument, and returns an instance of Colour. What can we use to preserve the mapping between a String and Colour? A Map!! ~~We can use a HashMap or ConcurrentHashMap for this.~~

Update 17-07-2021: DO NOT USE HashMap or ConcurrentHashMap, instead use LinkedHashMap, for reasons that will be explained later in the article.

Note that we currently have no way of knowing what colour an instance is. Of course, we have a constant declared as

public static final Colour RED = new Colour(255, 0, 0);

but even if we load it into a map, how would we know if the object is for Colour.RED or Colour.GREEN? Is there a way we can get the "text"/"name" of the variable as a String? Not directly, no.

Enter: Java Reflection

It allows an executing Java program to examine or "introspect" upon itself, and manipulate internal properties of the program. For example, it's possible for a Java class to obtain the names of all its members and display them.

We can use this! Let us first define a Map of type Map<String, Colour> as a HashMap or ConcurrentHashMap. We will then use Java Reflection to load up the values inside the Map through a static block.

public final class Colour {

    public static final Colour RED = new Colour(255, 0, 0);
    public static final Colour GREEN = new Colour(0, 255, 0);
    public static final Colour BLUE = new Colour(0, 0, 255);

    private static final Map<String, Colour> map = new LinkedHashMap<>();

    static {
        loadClassData();
    }

    private static void loadClassData() {
        Arrays.stream(Colour.class.getDeclaredFields())
                .filter(declaredField -> declaredField.getType() == Colour.class)
                .forEach(Colour::putInMap);
    }


    private static void putInMap(Field declaredField) {
        try {
            map.putIfAbsent(declaredField.getName(), (Colour) declaredField.get(null));
        } catch (IllegalAccessException e) {
            System.err.println("Could not initialize Colour Map value: " + declaredField.getName() + " " + e);
        }
    }


    private final int r, g, b;

    private Colour(int r, int g, int b) {
        this.r = r;
        this.g = g;
        this.b = b;
    }


    // getters and toString()
}

The above codeblock uses Java Streams to load up the map. Let us go through what it does.

The Colour.class.getDeclaredFields() fetches all the fields declared inside the Colour class.
For each of the field returned, we only want the fields of type Colour. (The previous statement would also return Map<String, Colour>)
For each of the fields of Colour, we would call the putInMap() method.
The putInMap() takes a parameter of type Field, and loads the data in the map. The variable name is obtained by declaredField.getName(), and the actual object is returned by declaredField.get(null)

If you do not understand the above Stream based code, or you're just working with Java 7 and lower, you can use the following:

private static void loadClassData() {
    for (Field declaredField : Colour.class.getDeclaredFields()) {
        if (declaredField.getType() == Colour.class) {
            putInMap(declaredField);
        }
    }
}

We have the data in the map! We can now just implement the valueOf() method as follows:

public static Colour valueOf(String name) {
    Colour colour = map.get(name);
    if (colour == null) {
        throw new IllegalArgumentException("No Colour by the name " + name + " found");
    }
    return colour;
}

Note that in Enums, the valueOf() returns an IllegalArgumentException if no value is found within the enum. Similarly, we will ensure that our implementation also returns the same exception.

The usage of Map enables us to easily implement the values() method. We can implement it by using the map.values() method.

public static Colour[] values() {
    return map.values().toArray(Colour[]::new).clone();
}

~~Update 14-07-2021: The map does not have ordered values, so we need to sort it! The tests have been updated in the later sections.~~

Update 17-07-2021: Since the values() method produces an array in the order in which the Enum values are defined, we need to preserve the order in the Map as well. This is why we need to use a LinkedHashMap, rather than another Map implementation.

Every time the values() is called, it returns the clone of the array values in the map.

Step 4: Load data from the Database

However, the purpose of having a dynamic Enum is still not achieved. We want the values of the colour to be loaded up from a database. The problem now seems trivial. Similar to the way we loaded up the class data inside the map, we must also fetch the data from the database and load it. This needs to take place in the static block as well.

However, in this case we cannot use Java Reflection to get the variable name, simply because there isn't any static variable we can refer to.

Thus, we must add another field into the class named colourName.

public final class Colour {

    public static final Colour RED = new Colour("RED", 255, 0, 0);
    public static final Colour GREEN = new Colour("GREEN", 0, 255, 0);
    public static final Colour BLUE = new Colour("BLUE", 0, 0, 255);

    private static final Map<String, Colour> map = new LinkedHashMap<>();

    // other implemented methods


    // new field
    private final String colourName;
    private final int r, g, b;

    private Colour(String colourName, int r, int g, int b) {
        this.colourName = colourName;
        this.r = r;
        this.g = g;
        this.b = b;
    }


    // getters and toString()
}

Thus, when we load up the data from the database, we would know what key to use for the Map.

Our static block would now have

static {
    loadClassData();
    loadDataFromDb();
}

private static void loadDataFromDb() {
    List<ColourDB.ColourData> colourData = new ColourDB().getColours();
    for (ColourDB.ColourData colourDatum : colourData) {
        map.putIfAbsent(colourDatum.getColourName(), new Colour(colourDatum));
    }
}

Our ColourDB contains the static class ColourData, which is exactly the same as the Colour POJO. As we cannot create Colour objects, we need another type to put the data in, and get the data from.

The ColourDB is like the following:

public class ColourDB {

    public List<ColourData> getColours() {
        // data from DB
    }

    static class ColourData {
        String colourName;
        int r;
        int g;
        int b;

        public ColourData(String colourName, int r, int g, int b) {
            this.colourName = colourName;
            this.r = r;
            this.g = g;
            this.b = b;
        }
        // getters
    }
}

We can now add another private constructor within Colour that accepts ColourData.

private Colour(ColourDB.ColourData colourDatum) {
    this.colourName = colourDatum.getColourName();
    this.r = colourDatum.getR();
    this.g = colourDatum.getG();
    this.b = colourDatum.getB();
}

Notice that using this method allows you to make Colours dynamic, but prevents you from creating static objects like Colour.RED. If we add the data for the colour "BLACK" in the DB, we cannot refer to it as Colour.BLACK after this change. We would need to refer to it as Colour.valueOf("BLACK"), and get the value. This is a trade off required to make it dynamic. However, this allows us to ensure that the existing code is not impacted.

If you have a implemented a getter method for colourName, refactor it from getColourName() to name().

Similarly, if you use ordinal() method of the enum, ensure that you introduce the ordinal field in the Colour class as well. You would need to store the ordinal field in the DB too.

You can also implement the equals() method and change it to compare using == as is done in Enums. One would also need to implement the Comparable interface with the compareTo() method.

We will also implement the Serializable interface to enable us to serialize objects. This will ensure we conform to the Enum functionality. Also, Enums cannot be cloned. Hence, we will also implement the clone method and throw CloneNotSupportedException.

The updated code would look like the following:

public final class Colour implements Comparable<Colour>, Serializable {

    public static final Colour RED = new Colour("RED", 255, 0, 0, 0);
    public static final Colour GREEN = new Colour("GREEN", 0, 255, 0, 1);
    public static final Colour BLUE = new Colour("BLUE", 0, 0, 255, 2);

    private static final Map<String, Colour> map = new LinkedHashMap<>();

    static {
        loadClassData();
        loadDataFromDb();
    }

    private static void loadClassData() {
        Arrays.stream(Colour.class.getDeclaredFields())
                .filter(declaredField -> declaredField.getType() == Colour.class)
                .forEach(Colour::putInMap);
    }

    private static void loadDataFromDb() {
        List<ColourDB.ColourData> colourData = new ColourDB().getColours();
        for (ColourDB.ColourData colourDatum : colourData) {
            map.putIfAbsent(colourDatum.getColourName(), new Colour(colourDatum));
        }
    }

    public static Colour[] values() {
        return map.values().toArray(Colour[]::new).clone();
    }

    public static Colour valueOf(String name) {
        Colour colour = map.get(name);
        if (colour == null) {
            throw new IllegalArgumentException("No Colour by the name " + name + " found");
        }
        return colour;
    }

    private final String colourName;
    private final int r, g, b;
    private final int ordinal;

    private Colour(String colourName, int r, int g, int b, int ordinal) {
        this.colourName = colourName;
        this.r = r;
        this.g = g;
        this.b = b;
        this.ordinal = ordinal;
    }

    private Colour(ColourDB.ColourData colourData) {
        this.colourName = colourData.getColourName();
        this.r = colourData.getR();
        this.g = colourData.getG();
        this.b = colourData.getB();
        this.ordinal = colourData.getOrdinal();
    }

    private static void putInMap(Field declaredField) {
        try {
            map.putIfAbsent(declaredField.getName(), (Colour) declaredField.get(null));
        } catch (IllegalAccessException e) {
            System.err.println("Could not initialize Colour Map value: " + declaredField.getName() + " " + e);
        }
    }

    public String name() {
        return colourName;
    }

    public int ordinal() {
        return ordinal;
    }

    // getters
    // ..
    // ..

    @Override
    public boolean equals(Object o) {
        return this == o;
    }

    @Override
    public int hashCode() {
        return Objects.hash(colourName, r, g, b, ordinal);
    }

    @Override
    public final int compareTo(Colour o) {
        Colour self = this;
        return self.ordinal - o.ordinal;
    }

    @Override
    protected Object clone() throws CloneNotSupportedException {
        throw new CloneNotSupportedException();
    }

    @Override
    public String toString() {
        return "Colour{" +
                "colourName='" + colourName + '\'' +
                ", r=" + r +
                ", g=" + g +
                ", b=" + b +
                ", ordinal=" + ordinal +
                '}';
    }
}

and ColourDB as

public class ColourDB {

    public List<ColourData> getColours() {
        // data from DB
    }

    static class ColourData {
        String colourName;
        int r;
        int g;
        int b;
        int ordinal;

        public ColourData(String colourName, int r, int g, int b, int ordinal) {
            this.colourName = colourName;
            this.r = r;
            this.g = g;
            this.b = b;
            this.ordinal = ordinal;
        }
        // getters
    }
}

Step 5: Test

Ordinarily, this would work. Unless, of course, you actually want to write unit tests for the class.

However, how can we test it? Notice that testing it is almost impossible, unless you have an actual DB. This is because of the following piece of code.

private static void loadDataFromDb() {
    List<ColourDB.ColourData> colourData = new ColourDB().getColours();
    for (ColourDB.ColourData colourDatum : colourData) {
        map.putIfAbsent(colourDatum.getColourName(), new Colour(colourDatum));
    }
}

Here, the dependency for the Database new ColourDB() is hard-coded inside the class. Testing would require an actual Database connection. We would want to Mock it during testing. If we were using a DI framework like Spring, we could have injected it. However, using pure Java code would require some more refactoring.

First, we need to extract ColourDB into an interface and include the actual implementation as ColourDbImpl.

public interface ColourDB {

    List<ColourData> getColours();

    class ColourData {
        // existing
    }
}

public class ColourDBImpl implements ColourDB {

    @Override
    public List<ColourData> getColours() {
        // get from DB
    }
}

We will now create a class named DB:

public class DB {
    private static ColourDB COLOUR_DB;

    public DB(ColourDB colourDB) {
        COLOUR_DB = colourDB;
    }

    public static ColourDB getColourDb() {
        if (COLOUR_DB == null) {
            COLOUR_DB = new ColourDBImpl();
        }
        return COLOUR_DB;
    }

    public static void setColourDb(ColourDB colourDb) {
        COLOUR_DB = colourDb;
    }
}

We can now replace ColourDB in loadDataFromDb with DB.getColourDb(). The code now looks like

private static void loadDataFromDb() {
    List<ColourDB.ColourData> colourData = DB.getColourDb().getColours();
    for (ColourDB.ColourData colourDatum : colourData) {
        map.putIfAbsent(colourDatum.getColourName(), new Colour(colourDatum));
    }
}

Now that we have refactored it, we can Mock it successfully.

You can see the test class ColourTest as follows

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ColourTest {

    public static final String BLACK = "BLACK";
    public static final String WHITE = "WHITE";
    public static final String YELLOW = "YELLOW";
    public static final String RED = "RED";

    @Mock
    ColourDBImpl colourDB;

    @InjectMocks
    DB db;

    @BeforeAll
    void setUp() {
        MockitoAnnotations.openMocks(this);
        ColourDB.ColourData black = new ColourDB.ColourData(BLACK, 255, 255, 255, 3);
        ColourDB.ColourData white = new ColourDB.ColourData(WHITE, 0, 0, 0, 4);
        ColourDB.ColourData yellow = new ColourDB.ColourData(YELLOW, 255, 255, 0, 5);
        Mockito.when(colourDB.getColours()).thenReturn(List.of(black, white, yellow));
    }

    @Test
    void test_values() {
        Colour[] values = Colour.values();

        assertEquals(Colour.RED.name(), values[0].name());
        assertEquals(Colour.valueOf(YELLOW).name(), values[values.length - 1].name());
        assertTrue(Arrays.stream(values).anyMatch(colour -> colour.name().equals(Colour.RED.name())));
        assertTrue(Arrays.stream(values).anyMatch(colour -> colour.name().equals(Colour.valueOf(WHITE).name())));
        assertEquals(6, values.length);
    }

    @Test
    void test_if_instances_are_same() {
        assertSame(Colour.RED, Colour.valueOf(RED));
        assertSame(Colour.valueOf(RED), Colour.valueOf(RED));
        assertEquals(Colour.valueOf(RED), Colour.valueOf(RED));

        assertSame(Colour.valueOf(WHITE), Colour.valueOf(WHITE));
        assertEquals(Colour.valueOf(WHITE), Colour.valueOf(WHITE));
    }

    @Test
    void test_ordinal() {
        assertEquals(0, Colour.RED.ordinal()); // static
        assertEquals(5, Colour.valueOf(YELLOW).ordinal()); // dynamic
    }

    @Test
    void test_invalid_colour() {
        String magenta = "MAGENTA";
        IllegalArgumentException exception = assertThrows(IllegalArgumentException.class, () -> Colour.valueOf(magenta));
        assertEquals("No Colour by the name " + magenta + " found", exception.getMessage());
    }

    @Test
    void test_compareTo() {
        int red = Colour.RED.compareTo(Colour.valueOf("WHITE"));
        assertTrue(red < 0);

        int yellow = Colour.valueOf("YELLOW").compareTo(Colour.RED);
        assertTrue(yellow > 0);
    }

    @Test
    void test_name() {
        assertEquals("RED", Colour.RED.name());
        assertEquals("YELLOW", Colour.valueOf("YELLOW").name());
    }
}

You will need to use @BeforeAll with @TestInstance(TestInstance.Lifecycle.PER_CLASS) as the static block is only executed once.
Else Mockito will throw an UnnecessaryStubbingException because the Mock would not execute every test.

Step 6: Serialization!

We did not consider what would happen if we serialize the Colour class. Enum constants are serialized differently than ordinary serializable or externalizable objects. Only the name of the field is serialized, and deserialization uses valueOf() method to get the Enum constant back.

Enums are effectively singletons. However, the singleton property of our class can be broken during deserialization. Thus, we need to ensure that we preserve the singleton property of Colour as well. We can do that by implementing the readResolve¹ method. This will ensure that we only receive an instance of the same class as the one that we have already created. We already store the colour name. So when a new object is created, it will still return the already existing objects that we expect.

public final class Colour implements Comparable<Colour>, Serializable {
    // Existing code
    //
    //

    private Object readResolve() {
        return Colour.valueOf(colourName);
    }
}

We can add a test and verify if this will work.

@Test
void test_serialization_deserialization() throws IOException, ClassNotFoundException {
    serialize(Colour.valueOf(BLACK));
    Colour black = deserialize();

    assertNotNull(black);
    assertEquals(Colour.valueOf(BLACK), black);
    assertSame(Colour.valueOf(BLACK), black);

    serialize(Colour.RED);
    Colour red = deserialize();

    assertNotNull(red);
    assertEquals(Colour.valueOf(RED), red);
    assertSame(Colour.valueOf(RED), red);

    assertEquals(Colour.RED, red);
    assertSame(Colour.RED, red);
}

void serialize(Colour colour) throws IOException {
    try (FileOutputStream fos = new FileOutputStream("data.obj");
         ObjectOutputStream oos = new ObjectOutputStream(fos)) {
        oos.writeObject(colour);
    }
}

Colour deserialize() throws IOException, ClassNotFoundException {
    try (FileInputStream fis = new FileInputStream("data.obj");
         ObjectInputStream ois = new ObjectInputStream(fis)) {
        return (Colour) ois.readObject();
    }
}

It does! :)

Limitations

As you know, this has a limitation that you cannot statically infer Enum constants, except the ones defined inside Colour. The values in DB must be referred through Colour.valueOf(). Moreover, you would need to change any switch-case statements to use the value of the the Constant, instead of static enum types supported by switch-case. An example:

switch (Colour.RED.name()) {
    case "RED" :
        System.out.println("RED");  
        break;
    default:
}

I discovered that this is also known as a type-safe enum, which was used before Enum as types were introduced in Java 5. Granted, this a step lower than Enums, but... you know.

The above is a hack though. In an ideal world, you would never have to try and implement these hacks. However, if you find yourself in such a spot, you know what to do. Please, try to refactor and remove this code though.

You can find the code for the project on my Github repository: dynamic-enums.

Done! We have now created a "Dynamic Enum"! 😎

You can now rest in peace, and wish/hope that the next developer touching this piece of code does not try to contact you.

References and Reading material:

Java Serialization Docs ↩

The Java Constants Interface Anti-Pattern

Darshit Patel — Sun, 08 Nov 2020 11:16:33 +0000

How do you define and use constants in Java?

Most advice on Internet has the following opinions:

Declare public static final for constants in a class
Do not use Interfaces for constants

The most common way to define a constant is in a class and using public static final. One can then use the constant in another class using ClassName.CONSTANT_NAME. Constants are usually defined in upper cases as a rule, atleast in Java.

So if I were to define a constant for the value of Pi(π), it would be something like:

public final class Constants {
    public static final double PI = 3.14;
}

This can then be used as Constants.PI whenever we want to reference the value of Pi.

Another way one can define constants is by the use of interfaces.

public interface Constants {
    double PI = 3.14;
}

However, this is not recommended by most sources on the internet. Why? Because it is an anti-pattern.

But is it really an Anti-pattern?

Let's examine the difference by using both the methods.

Creating a Constants class:

package constants;

public final class MathConstantsClass {
    public static final double PI = 3.14;
}

Creating an interface:

package constants;

public interface MathConstantsInterface {
    double PI = 3.14;
}

Let us define another interface which will help us test both the above methods.

package operations;

public interface CircleArea {
    double calculate(double radius);
}

The above interface would help us define a contract to calculate the area of a circle. As we know, the area of a circle is dependent only on its radius, and thus is reflected in the above interface.

The following class provides the implementation of calculating the area of a circle.

import constants.MathConstantsClass;
import operations.CircleArea;

public class MathConstantsClassImplementation implements CircleArea {
    public double calculate(double radius) {
        return MathConstantsClass.PI * radius * radius;
    }
}

To test the the above code, let us write a Test class using JUnit.

import operations.CircleArea;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

class MathConstantsClassImplementationTest {

    @Test
    void calculate() {
        CircleArea area = new MathConstantsClassImplementation();

        double circleArea = area.calculate(1.0);
        assertEquals(3.14, circleArea);
    }
}

If you run the above piece of test code, the test would pass.

For testing how we can use the constants with Interface, let's write another class called MathConstantsInterfaceImplementation.

import constants.MathConstantsInterface;
import operations.CircleArea;

public class MathConstantsInterfaceImplementation implements MathConstantsInterface, CircleArea {
    public double calculate(double radius) {
        return PI * radius * radius;
    }
}

Similarly a test for the above class is as follows:

import operations.CircleArea;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

class MathConstantsInterfaceImplementationTest {

    @Test
    void calculate() {
        CircleArea area = new MathConstantsInterfaceImplementation();

        double circleArea = area.calculate(1.0);
        assertEquals(3.14, circleArea);
    }
}

The above Test would pass. However, the argument against the implementation is that it is not a good practice as there could be field shadowing, and that will override the original value of the constant within the class.

It can be better understood with the following example:

import constants.MathConstantsInterface;
import operations.CircleArea;

public class MathConstantsWithInterfaceImplementationAndConstantShadowing implements MathConstantsInterface, CircleArea {
    private static final double PI = 200;

    public double calculate(double radius) {
        return PI * radius * radius;
    }
}

If, by chance, someone overrode the value of PI inside the class, it would lead to an incorrect output. It can be easily verified by the following test.

import operations.CircleArea;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

class MathConstantsWithInterfaceImplementationAndConstantShadowingTest {

    @Test
    void calculate() {
        CircleArea area = new MathConstantsWithInterfaceImplementationAndConstantShadowing();

        double circleArea = area.calculate(1.0);
        assertEquals(3.14, circleArea);
    }
}

The above test fails. The answer returned by calculate() is 200.0 instead of the expected 3.14. Another argument is, using the interface would pollute the namespace and also lead to the value propagated across the subclasses.

The above arguments are valid, and hold true.

However, what no one mentions is that you can still directly use the constants from the interface without implementing the interface. Just like the first example where we use MathConstantsClass.PI, we can also use MathConstantsInterface.PI without affecting the namespace and inheritance and shadowing issues.

This can also be easily verified:

import constants.MathConstantsInterface;
import operations.CircleArea;

public class MathConstantsInterfaceWithoutImplementation implements CircleArea {
    public double calculate(double radius) {
        return MathConstantsInterface.PI * radius * radius;
    }
}

Test class:

import operations.CircleArea;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

class MathConstantsInterfaceWithoutImplementationTest {

    @Test
    void calculate() {
        CircleArea area = new MathConstantsInterfaceWithoutImplementation();

        double circleArea = area.calculate(1.0);
        assertEquals(3.14, circleArea);
    }
}

It would make no difference to our way of implementation. Even the number of imports remain same. Moreover, you do not need additional boilerplate of public static final as members in an interface are public static final by default.

public static final double PI = 3.14;

double PI = 3.14;

What would you prefer? Cleaner code, anyone?

I have seen most constants almost grouped together if they are used throughout the application. You could also suggest that interface should only be used for contracts, and in most cases they are. However, keeping an interface for solely storing constants doesn't seem to be wrong to me either!

Unless, of course, some developer tries to implement a class which solely contains constants -- which would beget the question -- WHY?

You can check out the code at my Github: https://github.com/darshitpp/JavaConstants

References:

Constants in Java: Patterns and Anti-Patterns

How to change Java Version/JAVA_HOME for Ant?

Darshit Patel — Mon, 15 Jun 2020 05:17:31 +0000

Want to change Java Version/JAVA_HOME for Ant builds?

Open ~/.antrc file by running vim ~/.antrc
Add JAVACMD=<NEW_JAVA_HOME>/bin/java and save

The Ant wrapper script for Unix will source (read and evaluate) the file ~/.antrc before it does anything.
On Windows, the Ant wrapper batch-file invokes %HOME%\antrc_pre.bat at the start and %HOME%\antrc_post.bat at the end.
You can use these files, for example, to set/unset environment variables that should only be visible during the execution of Ant.

The wrapper scripts use the following environment variables (if set):

JAVACMD—full path of the Java executable. Use this to invoke a different JVM than JAVA_HOME/bin/java(.exe).

ANT_OPTS—command-line arguments that should be passed to the JVM. For example, you can define system properties or set the maximum Java heap size here.

ANT_ARGS—Ant command-line arguments. For example, set ANT_ARGS to point to a different logger, include a listener, and to include the -find flag. Note: If you include -find in ANT_ARGS, you should include the name of the build file to find, even if the file is called build.xml.

Source: https://ant.apache.org/manual/running.html

Check out my website at: https://darshit.dev

How To Implement Two Way SSL With Nginx

Darshit Patel — Sat, 13 Jun 2020 13:25:19 +0000

A couple of weeks ago, I was tasked with figuring out a way to enable two way SSL. I am a programmer, and have had only a limited experience with networking concepts like SSL/TLS in my short career. So I turned up blank on how I could make it possible. Moreover, the terminology you'd find online is not uniform. From a programming point of view, the term "Two way SSL" led me to limited results, and I soon realized that other communities have different terminology. For example the "Two way SSL" is also known as "Mutual TLS" or "mTLS" or "Client Certificate Authentication" in Cloud/DevOps communitites. This makes finding the right resources online more difficult.

Our services have SSL enabled, but only the usual one -- similar to the one you'd find while visiting this website, but this was a unique thing for me as I didn't even know two way SSL existed. I figured out how it works. However, there were more problems with trying to understand how to implement this. Most guides that I found on the internet were very incomplete, and straight away skipped many parts for someone who'd be new to this. This is my attempt at explaining what I have learned, and documenting the same for future reference. It will not explain how two way SSL works, but how to make it work. If you need a quick refresher however, I would direct you to this easy to understand article by Cloudflare explaining where and how mTLS is used.

This guide is only for Unix like systems like macOS or Linux. Though I tried this on Windows, but could not make it work.

Prerequisites

Nginx
sudo privileges on your system

Installing and Configuring Nginx

Install

Install¹ nginx using apt command on your Linux system.

sudo apt update
sudo apt install nginx

This will install all of Nginx on the path /opt/nginx. All the configuration files we will be editing for two-way SSL would be found within this directory.

Configure Nginx to start

Before starting up Nginx for use, we need to enable some ports on the firewall that our Nginx can listen incoming connections from.

sudo ufw app list

Output:

Available applications:
  Nginx Full
  Nginx HTTP
  Nginx HTTPS
  OpenSSH

The utility ufw can be used to manage our firewall. Though a knowledge of ufw is not necessary for us, more information can be found here.

You can see in the above output that the utility responds us with four profiles. When we installed Nginx, it registered itself with the ufw utility with three profiles.

Nginx HTTP allows Nginx to listen through port 80, for normal HTTP traffic.
Nginx HTTPS allows Nginx to listen through port 443, for HTTPS traffic.
Nginx Full is a combination of the above both, enabling port 80 and 443 both.

We will enable Nginx Full as we have to use our server for SSL, but using normal HTTP connections is not an uncommon use-case either. To enable it, run:

sudo ufw allow 'Nginx Full'

You should see the activated profile if you run the below command

sudo ufw status

Output:

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
Nginx Full                 ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)             
Nginx Full (v6)            ALLOW       Anywhere (v6)

The Nginx service should already be up and running now. You can check by executing

systemctl status nginx

Output:

● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-04-20 16:08:19 UTC; 3 days ago
     Docs: man:nginx(8)
 Main PID: 2369 (nginx)
    Tasks: 2 (limit: 1153)
   CGroup: /system.slice/nginx.service
           ├─2369 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           └─2380 nginx: worker process

Creating Certificates

Terminology

Certificate Authority (CA): This is an Organization which provides you a Certificate (the .crt file). In reality, anyone (including yourself) can be the CA which issues certificates
Certificate Signing Request (CSR .csr): An "input form" with details (like Name, Organization, Address, etc.) filled by the requester of the certificate which is submitted to the CA
Private Key (.key): The private key file used by the CA in conjunction with CSR to sign and generate the certificates
Certificate (.crt): The certificate generated by the CA
Self Signed Certificate: Consider yourself as a Certificate Authority (CA) and generate the .crt files.

Now that we know the terminology, it would be easier to proceed with generating the certificates. The following part of the tutorial requires root/superuser privileges, so switch to the super user using the sudo su command.

Create a directory called certs under the root of the file-system which will hold all the certificates and related files.

# Create directory
mkdir /certs 
# Change directory to /certs
cd /certs
# Verify the present working directory
pwd

Output:

/certs

In this tutorial, we would be designating ourselves as a Certificate Authority, and then self-sign and generate the certificates.

Generate Certificate Authority (CA) files

We would be first generating a CA key, which would be the basis for all further certificate generation.

openssl genrsa -des3 -out ca.key 4096

Output:

Generating RSA private key, 4096 bit long modulus (2 primes)
.............................................................++++
....................................................................................................................................................................................++++
e is 65537 (0x010001)
Enter pass phrase for ca.key:<passphrase>
Verifying - Enter pass phrase for ca.key:<passphrase>

Command options:

genrsa: generate RSA private key
-des3: encrypts the output key using des3 encryption
-out: specifying the output key file
4096: size of private key in bits

This generates a file named ca.key in the current directory. You would have to provide a password/passphrase for the key. For the purposes of this tutorial, I'll be using the same passphrase whenever required.

<passphrase> = root

The next step is the generation of ca.crt

openssl req -new -x509 -days 3650 -key ca.key -out ca.crt

Output:

Enter pass phrase for ca.key:<passphrase>

Command options:

req: used for creating certificate requests
-new: generates new certificate request
-x509: outputs a new self-signed certificate instead of a certificate request
-days 3650: validity of certificate (in this case, 10 years/3650 days)
-key: RSA key to be used for generating the certificate
-out: output file for the certificate

Once you enter the passphrase (which is the same as we put in the previous step), it'll ask you to provide more information for the self-signed certificate.

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:CertAuth
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:

Note the Organization Name parameter is named as CertAuth for this example. This is because we are a CA and CAs should ideally be independent to enable other parties to trust other parties and establish a chain of trust.

A PEM file certificate would also be needed. This file is way of encoding the certificates.² To create a PEM file, simply do the following:

cat ca.key > ca.pem
cat ca.crt >> ca.pem

The ca.pem would be something like

Generating Client/User Certificates

At this point, we have three files in the certs directory.

ca.key
ca.crt
ca.pem

We now would be generating client/user certificates.
The command to generate the user.key is similar to the one used to create the ca.key

openssl genrsa -des3 -out user.key 4096

Output:

Generating RSA private key, 4096 bit long modulus (2 primes)
.....................................................................................................................................................++++
.......................++++
e is 65537 (0x010001)
Enter pass phrase for user.key:
Verifying - Enter pass phrase for user.key:

As mentioned earlier, the passphrase to be used for user.key is still the same.

The client/user certificate aren't supposed to be self-signed, and we would need to generate a CSR. It means the signing is to be done by a CA(even though in this case, we own the CA!). The way to do this is

openssl req -new -key user.key -out user.csr

Output:

Enter pass phrase for user.key:<passphrase> 
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:User
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

The Organization Name would be User as this certificate is for a Client/User. DO NOT keep it the same as the one for CA.

The next step would be to create a User Certificate from the User CSR.

openssl x509 -req -days 365 -in user.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out user.crt

Output:

Signature ok
subject=C = AU, ST = Some-State, O = User
Getting CA Private Key
Enter pass phrase for ca.key:<passphrase>

Command options:

x509: standard format for public key certificates
-CA: the CA certificate
-CAkey: the key to generate the certificate from
-set_serial 01: the serial version of the certificate to be generated. The user certificate can be regenerated by incrementing the serial number from the same user.csr after 365 days when it expires.

You can see that it displays data from the CSR, and then uses the CA key to generate user.crt.

To enable us to use the certificate from a web browser, we would need to create the certificate in the PFX file format.

openssl pkcs12 -export -out user.pfx -inkey user.key -in user.crt -certfile ca.crt

Output:

Enter pass phrase for user.key:<passphrase>
Enter Export Password:<passphrase>
Verifying - Enter Export Password:<passphrase>

You can verify if the generated certificate can be decrypted using the CA certificate by the following command

openssl verify -verbose -CAfile ca.crt user.crt

Output:

user.crt: OK

The files that we currently have in the directory are

ca.key
ca.crt
ca.pem
user.key
user.csr
user.crt
user.pfx

Generating Server Certificates

This is similar to generating the User certificates, with the following commands

Generate the Server key. For server certificates, the standard naming convention seems to be <website-domain-name>.<key>

openssl genrsa -out nginx.mssl.com.key 4096

Output:

Generating RSA private key, 4096 bit long modulus (2 primes)
.....................................++++
.........................++++
e is 65537 (0x010001)

Use the above generated key to generate a CSR.

openssl req -new -key nginx.mssl.com.key -out nginx.mssl.com.csr

Output:

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Server
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:nginx.mssl.com
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Make sure to put the Organization Name as Server and Common Name (which is the website name) as nginx.mssl.com

We can now use the CSR along with the CA files to generate the CRT for our server.

openssl x509 -req -days 365 -sha256 -in nginx.mssl.com.csr -CA ca.crt -CAkey ca.key -set_serial 1 -out nginx.mssl.com.crt

Output:

Signature ok
subject=C = AU, ST = Some-State, O = Server, CN = nginx.mssl.com
Getting CA Private Key
Enter pass phrase for ca.key:<passphrase>

All the certificates are now ready for our use!

ca.key
ca.crt
ca.pem
user.key
user.csr
user.crt
user.pfx
nginx.mssl.com.key
nginx.mssl.com.csr
nginx.mssl.com.crt

Setting up our "Website" with Nginx

Of course, we aren't setting up a real website. But we do want to make our SSL authentication work on the domain nginx.mssl.com. We can do this by a little "hack" by changing our /etc/hosts file.

Fire up a new shell, and use

sudo vim /etc/hosts

Add the following line to the file and save.

127.0.0.1 nginx.mssl.com

This will enable your local machine to resolve the domain nginx.mssl.com to your locahost.

The website will not work at the moment -- you'd have to specify the sources/web pages it needs to serve when requested.

Create a file index.html on the directory path /usr/share/nginx/mssl with the contents:

<html>
  <body>
    <h1>Welcome to nginx!-mutual ssl test</h1>
  </body>
</html>

If the TLS handshake is successful, the web server should serve the file created above.

Configuring the Nginx server to enable two way SSL

All our certificates are at root in /certs directory. For simplicity, we would be copying the certificates to /etc/nginx/certs directory.

cp -r /certs /etc/nginx/certs

Also ensure that the certificates can be used by nginx service by providing them the access

chmod 777 -R certs/

We would not be fiddling with the default nginx configuration which is present in the nginx.conf file. Instead, we would be creating a new file called proxy.conf using in the /etc/nginx/sites-available directory.

Create the file proxy.conf using the command

cd /etc/nginx/sites-available
touch proxy.conf

Enter the contents as the following (Notice the comments have more explanation):

server {
    # Listen on port 443 for HTTPS connections
    listen  443;

    # Turn SSL on
    ssl on;

    # Name of the server/website
    server_name nginx.mssl.com;

    # See https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_server_name
    proxy_ssl_server_name on;

    # This is the server SSL certificate
    ssl_certificate      /etc/nginx/certs/nginx.mssl.com.crt;

    # This is the server certificate key
    ssl_certificate_key /etc/nginx/certs/nginx.mssl.com.key;

    # Important: 
    # This is the CA cert against which the client/user will be validated
    # In our case since the Server and the Client certificate is 
    # generated from the same CA, we use the ca.crt 
    # But in actual production, the Client certificate might be 
    # created from a different CA
    ssl_client_certificate /etc/nginx/certs/ca.crt;

    # Enables mutual TLS/two way SSL to verify the client
    ssl_verify_client on;

    # Number of intermediate certificates to verify. Good explanation of 
    # certificate chaining can be found at
    # https://cheapsslsecurity.com/p/what-is-ssl-certificate-chain/
    ssl_verify_depth 2;

    # Any error during the connection can be found on the following path
    error_log /var/log/nginx/error.log debug;

    ssl_prefer_server_ciphers on;
    ssl_protocols TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK';

    keepalive_timeout 10;
    ssl_session_timeout 5m;

    # Matches the "root" of the website
    # If TLS handshake is successful, the request is routed to this block
    location / {
        # path from which the website is served from
        root /usr/share/nginx/mssl;
        # index file name
        index index.html index.htm;
    }
}

Nginx has a good Beginner's Guide to Nginx.

To enable Nginx to use the above configuration, we also have to link the same in the /etc/nginx/sites-enabled directory.

To accomplish this, use

ln -s /etc/nginx/sites-available/proxy.conf /etc/nginx/sites-enabled/proxy.conf

This creates a symbolic link into /etc/nginx/sites-enabled/proxy.conf from /etc/nginx/sites-available/proxy.conf. Consider it as you having many server configurations in the sites-available, but only what you chose to "enable" in the sites-enabled will be active.

We now just have to restart the Nginx for the configuration to be active!

systemctl restart nginx

I'll be using Postman to test our changes. However, you'll need to configure Postman to send the Client Certificates with the request³.

To configure the Certificates, navigate to Settings -> Certificates Tab.

There will be a section to add the CA Certificate named CA Certificates, and this certificate should be a PEM file. Select the ca.pem from /etc/nginx/certs. A mistake would be to select the file from the root directory /certs but this will not work as Postman wouldn't be able to access the file.

There would be another section below for Client Certificates. Click on Add Certificate, and put the details as the following:

Host: nginx.mssl.com
CRT File: /etc/nginx/certs/user.crt
KEY File: /etc/nginx/certs/user.key
PFX File: /etc/nginx/certs/user.pfx
Passphrase: <passphrase>

Now try to fire a GET request to the domain https://nginx.mssl.com. If everything was configured successfully, you'll get the following response:

<html>
  <body>
    <h1>Welcome to nginx!-mutual ssl test</h1>
  </body>
</html>

If there's an error in the configuration, you'll get the following error response:

<html>
  <head>
    <title>400 No required SSL certificate was sent</title>
  </head>

  <body bgcolor="white">
    <center>
      <h1>400 Bad Request</h1>
    </center>
    <center>No required SSL certificate was sent</center>
    <hr>
    <center>nginx/1.14.0 (Ubuntu)</center>
  </body>
</html>

Hope this post was of some help!!

Reference:

This article is good for the demo part.
The post Client Certificate Auth With Nginx was instrumental in explaining the ssl_client_certificate directive and how to use it.
This post is as close to perfection as it gets regarding the steps for generating Certificates, but I couldn't manage to make it work fully with Nginx. (Especially because it skips the demo part + it could be a little more descriptive with generation of the certificates)
This article is good, but still unclear for someone starting out.
I used this article to actually learn about generating the certificates especially because it provides easy commands and decent explanation.

The main resource for installing Nginx is this tutorial by DigitalOcean. All the relevant steps have been described in my article. ↩
More info here ↩
Client Certificates: Postman Learning Center ↩

How to achieve a 50% reduction in Redis memory usage

Darshit Patel — Sun, 10 May 2020 15:55:01 +0000

Yes, you read that right.

To give you some context, some time ago, our (my org's) Redis usage was un-tracked -- meaning we didn't know why our Redis memory was being occupied as much as it was. Our 2.5GB of Redis ElastiCache was almost close to being full, and if it somehow reached its limit, our system would start to fail. Though there were fallbacks in place, Redis could turn out to be a bottle-neck.

In this post, I would try to explain how we reduced the storage occupied by the data by more than 50%. This would also kind of be a step by step guide from the basics, so if you're just interested in how Redis is being used, just skip the and go to the Optimization section.

Basic Setup

I would be using the latest version of Spring Boot from https://start.spring.io. Firstly, select our two of our main dependencies - Spring Boot Web and Spring Data Reactive Redis.

You would find these in the pom.xml file when you download the starter project.

The Spring Boot Web is for building basic web applications with Spring Boot, whereas Spring Data Reactive Redis would be used for connecting and using Redis inside the application. At its core, the Redis dependency by default uses the Lettuce Redis client, and is supported by the latest versions of Spring Boot.

Note that I'm going to skip the installation of Redis, as there are other guides available for every operating system. You do need the Redis Server to be started for our application to work successfully.

After downloading the basic application, you'll need to extract and open it in your favourite IDE (my favourite one is IntelliJ IDEA).

In my case the project name is redis-util, and you'll find my "base packages" to be named com.darshitpp.redis.redisutil. This base package would have a class called RedisUtilApplication, which in my case has the following configuration.

@SpringBootApplication
@ComponentScan(basePackages = {"com.darshitpp.redis.redisutil"})
public class RedisUtilApplication {

    public static void main(String[] args) {
        SpringApplication.run(RedisUtilApplication.class, args);
    }

}

I have manually added the @ComponentScan annotation to specify a top-level package name under where Spring should look for defined Beans/Configurations.

To connect to Redis, I create a configuration class called LettuceRedisConfiguration, under a new package named configuration(note that this should be under the basePackages path defined above.

You could define the configuration in the RedisUtilApplication class itself, but I want this to be as "production-ready" as possible. Thus, it's a good practice to separate out your different parts of application.

My configuration class is

@Configuration
public class LettuceRedisConfiguration {

    @Bean
    public LettuceConnectionFactory redisConnectionFactory() {
        return new LettuceConnectionFactory(new RedisStandaloneConfiguration("localhost", 6379));
    }

}

It is a very simple class, which has the configuration of which URL to connect to for Redis. In my case, it is localhost, but in most production apps, it would be an external Redis server. Port 6379 is the default port on which the Redis server starts. This Bean would return us a "factory" of Redis connections. Think of this as something which would allow you to connect to Redis when required.

At this point, my package structure looks like:

->src
  ->main
    ->java
      ->com.darshitpp.redis.redisutil
        ->configuration

Now that we know how to connect to a Redis server, we need to figure out what data we need to store in Redis. In our case, we would be storing User data. This is the "domain model" of our application (domain model could be translated to a table in a Database, but we don't have a table in our scenario). This User is stored in a package called domain.

The User would have three fields, namely, firstName, lastName, and birthday.

Before storing the objects in Redis, it is a good idea to identify how you will store the data so that it's efficient to fetch it back. What that means is Redis being a simple Key-Value store, you would need to identify the Key you would be storing the Value with. In our case, I am choosing firstName as the key. The data would be stored in a hash, so the hashKey that we select would be the lastName and the value mapped to the hashKey is the User object.

This is because Hashes in Redis have the following structure:

key1 --- hashKey1 === value1
     --- hashKey2 === value2
     --- hashKey3 === value3
key2 --- hashKey4 === value4
     --- hashKey5 === value5
.
.
.

You could also imagine it as a tree with the top level nodes being the Keys, the immediate next level to be hashKeys, and the leaf nodes to be the values. To access value2, you would need to have key1 and hashKey2.

Our example is a bit incorrect, as a User could have same key=firstName and hashKey=lastName as another user, and Redis will overwrite value. However, for brevity, we will assume there are unique Users using our application.

We would now be creating a controller class called NormalController which would act as an entry point for our API. We have named it NormalController for reasons that will be clear further in this article.

@RestController
@RequestMapping("/normal")
public class NormalController {

    private final NormalService normalService;

    @Autowired
    public NormalController(NormalService normalService) {
        this.normalService = normalService;
    }

    @GetMapping("/get")
    public User get(@RequestParam("firstName") String firstName, @RequestParam("lastName") String lastName) {
        return normalService.get(firstName, lastName);
    }

    @PostMapping("/insert")
    public void insert(@RequestBody User user) {
        normalService.put(user);
    }

    @PostMapping("/delete")
    public void delete(@RequestParam("firstName") String firstName) {
        normalService.delete(firstName);
    }
}

NormalController also has a service named NormalService which is Autowired.
The class should be defined in a new packaged named controller after which the package structure would look like

->src
  ->main
    ->java
      ->com.darshitpp.redis.redisutil
        ->configuration
        ->domain
        ->controller

Our basic operations would be simple CRUD like operations which NormalService implements using a custom Operations interface.

public interface Operations {
    User get(String firstName, String lastName);
    void put(User user);
    void delete(String firstName);
}

To use Lettuce in our application, we need to do a couple of more things though. Just like to access JDBC, there's a provision for a JdbcTemplate, you must similarly use a RedisTemplate to operate on Redis. We must also define in what format will Redis store the data inside it. By default, it stores data as a String. However, know that you'll be storing User in Redis, and in order to facilitate the storage and fetch from Redis, you would need a way through which Redis will be able to identify and convert it back to the appropriate type of data you want.

Think of this as talking with someone who doesn't know the same language as you do. If you want to communicate with someone who only speaks Spanish, you would need to find a translator who would convert English into Spanish for you. This process of conversion and recovery is known as Serialization and Deserialization.

English to Spanish = Serialization
Spanish to English = Deserialization

Thus, we need a translator or a Serializer in our case too. We would be using Jackson for this process. Jackson is a nifty library which Spring Boot supports out-of-the-box to handle Json.

We would need to create a Serializer which implements RedisSerializer for our purposes. In our case, I have created a class JsonRedisSerializer inside a new package called serializer.

class JsonRedisSerializer<T> implements RedisSerializer<T> {
    public static final Charset DEFAULT_CHARSET;
    private final JavaType javaType;
    private ObjectMapper objectMapper = new ObjectMapper()
            .registerModules(new Jdk8Module(), new JavaTimeModule(), new ParameterNamesModule(JsonCreator.Mode.PROPERTIES))
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, true)
            .configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
            .setSerializationInclusion(JsonInclude.Include.NON_NULL);

    public JsonRedisSerializer(Class<T> type) {
        this.javaType = JavaTypeHandler.getJavaType(type);
    }

    public T deserialize(@Nullable byte[] bytes) throws SerializationException {
        if (bytes == null || bytes.length == 0) {
            return null;
        } else {
            try {
                return this.objectMapper.readValue(bytes, 0, bytes.length, this.javaType);
            } catch (Exception ex) {
                throw new SerializationException("Could not read JSON: " + ex.getMessage(), ex);
            }
        }
    }

    public byte[] serialize(@Nullable Object value) throws SerializationException {
        if (value == null) {
            return new byte[0];
        } else {
            try {
                return this.objectMapper.writeValueAsBytes(value);
            } catch (Exception ex) {
                throw new SerializationException("Could not write JSON: " + ex.getMessage(), ex);
            }
        }
    }

    static {
        DEFAULT_CHARSET = StandardCharsets.UTF_8;
    }

}

As you can see, it has two methods called serialize and deserialize. Each of these methods use the Jackson's ObjectMapper for conversion.

There is also a class named JavaTypeHandler which helps you get the Type of the object you're trying to serialize.

final class JavaTypeHandler {

    static <T> JavaType getJavaType(Class<T> clazz) {
        return TypeFactory.defaultInstance().constructType(clazz);
    }

}

Consequently, we would also need a class which returns us a RedisTemplate which utilizes this serializer. I would name this class RedisSerializationBuilder.

public final class RedisSerializationBuilder {

    public static <T> RedisTemplate<String, T> getNormalRedisTemplate(final LettuceConnectionFactory factory, final Class<T> clazz) {
        JsonRedisSerializer<T> jsonRedisSerializer = new JsonRedisSerializer<>(clazz);

        RedisTemplate<String, T> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(factory);
        redisTemplate.setDefaultSerializer(RedisSerializer.json());
        redisTemplate.setKeySerializer(RedisSerializer.string());
        redisTemplate.setValueSerializer(RedisSerializer.string());
        redisTemplate.setHashKeySerializer(RedisSerializer.string());
        redisTemplate.setHashValueSerializer(jsonRedisSerializer);
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }

}

Notice that the above method will return you a template specific to a particular domain model(in our case, the User) using Generics. It also specifies what connection factory is to be used, what should be the default key/value/hashKey/hashValue serializers.

Consequently, the NormalService looks like

@Service
public class NormalService implements Operations{

    private final RedisTemplate<String, User> redisTemplate;
    private final HashOperations<String, String, User> hashOperations;

    public NormalService(LettuceConnectionFactory redisConnectionFactory) {
        this.redisTemplate = RedisSerializationBuilder.getNormalRedisTemplate(redisConnectionFactory, User.class);
        this.hashOperations = this.redisTemplate.opsForHash();
    }

    @Override
    public User get(String firstName, String lastName) {
        return hashOperations.get(firstName, lastName);
    }

    @Override
    public void put(User user) {
        hashOperations.put(user.getFirstName(), user.getLastName(), user);
    }

    @Override
    public void delete(String firstName) {
        hashOperations.delete(firstName);
    }
}

I then inserted a User, using the POST method, and URL: localhost:8080/normalService/insert
Request Body:

{
  "firstName": "Priscilla",
  "lastName": "Haymes",
  "birthday": "2020-04-12T11:15:00Z"
}

If I then run this application for 100 Users, I find the following stats for the memory usage in Redis (I used the memory stats command using the redis-cli)

21) "keys.count"
22) (integer) 100
23) "keys.bytes-per-key"
24) (integer) 1044
25) "dataset.bytes"
26) (integer) 32840

Using the hgetall command for a key gives me

127.0.0.1:6379>hgetall "Priscilla"
1) "Haymes"
2) "{\"firstName\":\"Priscilla\",\"lastName\":\"Haymes\",\"birthday\":1586690100000}"

Notice that 2) gives us the actual type of data stored in Redis -> Json!

Our basic structure for further optimizations is in place! Yay!

Optimization

MessagePack is here to the rescue! As I said, you'd need a "transalation" mechanism. What if the translator is an expert, and converts your English into Spanish in as few words as possible? MessagePack is the same!

You would need to add two more dependencies in your pom.xml file.

    <dependency>
        <groupId>org.msgpack</groupId>
        <artifactId>msgpack-core</artifactId>
        <version>0.8.20</version>
    </dependency>
    <dependency>
        <groupId>org.msgpack</groupId>
        <artifactId>jackson-dataformat-msgpack</artifactId>
        <version>0.8.20</version>
    </dependency>

We create a controller called MsgPackController and a service called MsgPackService almost similar to NormalController and NormalService. We would create a MsgPackSerializer to serialize using MessagePack.

class MsgPackRedisSerializer<T> implements RedisSerializer<T> {
    public static final Charset DEFAULT_CHARSET;
    private final JavaType javaType;
    private ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory())
            .registerModules(new Jdk8Module(), new JavaTimeModule())
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, true)
            .configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
            .setSerializationInclusion(JsonInclude.Include.NON_NULL);

    public MsgPackRedisSerializer(Class<T> type) {
        this.javaType = JavaTypeHandler.getJavaType(type);
    }

    public T deserialize(@Nullable byte[] bytes) throws SerializationException {
        if (bytes == null || bytes.length == 0) {
            return null;
        } else {
            try {
                return this.objectMapper.readValue(bytes, 0, bytes.length, this.javaType);
            } catch (Exception ex) {
                throw new SerializationException("Could not read MsgPack JSON: " + ex.getMessage(), ex);
            }
        }
    }

    public byte[] serialize(@Nullable Object value) throws SerializationException {
        if (value == null) {
            return new byte[0];
        } else {
            try {
                return this.objectMapper.writeValueAsBytes(value);
            } catch (Exception ex) {
                throw new SerializationException("Could not write MsgPack JSON: " + ex.getMessage(), ex);
            }
        }
    }

    static {
        DEFAULT_CHARSET = StandardCharsets.UTF_8;
    }

}

The only major noticeable change is an instance of MessagePackFactory being passed into the ObjectMapper. This would act as a bridge between binary and String formats of data between Redis and our Spring Boot application.

Testing our changes (after clearing the previously utilized storage from redis gives us the following:

127.0.0.1:6379> hgetall "Priscilla"
1) "Haymes"
2) "\x83\xa9firstName\xa9Priscilla\xa8lastName\xa6Haymes\xa8birthday\xcf\x00\x00\x01qn\x19\x8b "

127.0.0.1:6379> memory stats
.
.
.
21) "keys.count"
22) (integer) 100
23) "keys.bytes-per-key"
24) (integer) 876
25) "dataset.bytes"
26) (integer) 15976

Compare the dataset.bytes from the current memory to the previously recorded one. 15976 bytes vs 32840 bytes, nearly 50% reduction already!

But wait, we can reduce it further. How, you ask. Compression! What if we compress the data and then store it? In our case it would work! This time, Snappy to the rescue!

Your first question after this would be: compression and decompression takes time. Wouldn't it be detrimental on production? Snappy has the answer to this too.

It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression.

Using Snappy is also as simple as adding the dependency in pom.xml, and a couple of lines of code changes. Just add Snappy.compress while serialization and Snappy.decompress while deserialization.

    <dependency>
        <groupId>org.xerial.snappy</groupId>
        <artifactId>snappy-java</artifactId>
        <version>1.1.7.3</version>
    </dependency>

Testing it again with the same inputs returns the following

127.0.0.1:6379> hgetall "Priscilla"
1) "Haymes"
2) "7\\\x83\xa9firstName\xa9Priscilla\xa8la\t\x13`\xa6Haymes\xa8birthday\xcf\x00\x00\x01qn\x19\x8b "

127.0.0.1:6379> memory stats
.
.
.
21) "keys.count"
22) (integer) 100
23) "keys.bytes-per-key"
24) (integer) 873
25) "dataset.bytes"
26) (integer) 15720

You can see that the size of the data set is smaller, 15720 bytes vs 15976 bytes, a marginal difference, but with larger amounts of data, this difference increases.

In my case, cleaning and restructuring the data, and utilizing the above techniques, we brought down the memory usage from 2GB to less than 500MB.

The full code can be found on my Github for redis-util.

Special mention to Rahul Chopda (@_RahulChopda) for his guidance! You have been a best mentor anyone could ask for!

Using IntelliJ IDEA Live Templates

Darshit Patel — Sun, 01 Sep 2019 08:32:10 +0000

IntelliJ IDEA is an awesome IDE, and a lesser known and used feature is Live Templates.

Live Templates enable you to use code snippets with just a few keystrokes. A lot of great ones are provided out-of-the-box by IntelliJ. You can view them using the shortcut press Double Shift and then typing Live Templates. The shortcut works regardless of the OS you're currently using (and I am too lazy to specify OS specific menus).

Some of the examples of Live Templates are:

Typing psvm replaces it with

public static void main(String[] args){

}

Typing psfs magically turns it into

public static final String

I was recently refactoring a lot of Classes and I had to replace a lot of legacy logging initialization statements to using slf4j logging library like the following:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class LoggerTest {
    public static final Logger logger = LoggerFactory.getLogger(LoggerTest.class);
}

I had more than 30 different classes to refactor as the above, and I certainly didn't want to painstakingly write everything by hand again (confirms that I'm lazy).

Fortunately, IntelliJ Live Templates came to my rescue! I fired up the Live Templates menu using the shortcut mentioned above, and clicked on the + button at the top right.

I then clicked on Live Template button. The UI now points to the bottom which asks you to put an abbreviation.

Let's input the abbreviation as psfl which stands for public static final Logger, which can be also put in the description.

Write the following code in the Template text box:

public static final Logger logger = LoggerFactory.getLogger();

But hang on, the IDE gives us a warning to define a context where it would be used at. We want the template to be only used in Java, so we click on the Define button, and select Java.

You may now notice the IDE now applies syntax highlighting on the template.

Wait, we are still not there yet. I certainly don't want to manually write every class name inside the getLogger function! At this point, I was not sure how I could achieve that. Cue in a bit of googling, stackoverflow again came to the rescue.

I found the following answer: https://stackoverflow.com/a/8552882/4840501

public static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger($CLASS_NAME$);
$END$

So I copy-pasted the code in my template screen(what did you expect? :P)

You'd then need to define what $CLASS_NAME$ means. To do that, click on the Edit Variables button and select className() in the Expression box.

The $END$ variable means where you want your cursor at, after the template is applied.

Click on Apply and Ok.

We're done!

Fire up your classes and refactor with 10x speed!

Relevant link: https://www.jetbrains.com/help/idea/creating-and-editing-live-templates.html

DEV Community: Darshit Patel

Using Java's serialVersionUID

"What's a serialVersionUID used for?"

"What if you do not define it?"

Mastering ExecutorService Shutdown: Tracking ThreadPool Termination

Hook methods

SSH Tunnelling with Java

SSH Tunnel

Example

Structure

Usage

Caveats

Handling limited sudo access through Ansible

Running specific commands with sudo using Ansible

Resolution

Dynamic Enums

But can one even make Enums dynamic?

Enums are also classes

Step 1: Change enum to class

Step 2: Create constants of existing Colours

Step 3: Implement other Enum methods like values() and valueOf()

Step 4: Load data from the Database

Step 5: Test

Step 6: Serialization!

Limitations

The Java Constants Interface Anti-Pattern

But is it really an Anti-pattern?

How to change Java Version/JAVA_HOME for Ant?

Want to change Java Version/JAVA_HOME for Ant builds?

How To Implement Two Way SSL With Nginx

Prerequisites

Installing and Configuring Nginx

Install

Configure Nginx to start

Creating Certificates

Terminology

Generate Certificate Authority (CA) files

Generating Client/User Certificates

Generating Server Certificates

Setting up our "Website" with Nginx

Configuring the Nginx server to enable two way SSL

How to achieve a 50% reduction in Redis memory usage

Basic Setup

Optimization

Using IntelliJ IDEA Live Templates

Step 1: Change `enum` to `class`

Step 3: Implement other Enum methods like `values()` and `valueOf()`