DEV Community: harithay

Better way to store Enum values in MongoDB

harithay — Sat, 26 Sep 2020 21:41:27 +0000

If you want to save Java Enum value to DB as Enum name, then the Mongo driver supports that. For example, if you have an enum

public enum ProcessType {
    CONVERT_ONE_TO_ONE,
    CONVERT_ONE_TO_MANY;
}

and it is registered with mongo codec provider as

import org.bson.codecs.pojo.ClassModel;
import org.bson.codecs.pojo.PojoCodecProvider;
import org.bson.codecs.pojo.PojoCodecProvider.Builder;
import com.ps2pdf.models.enums.ProcessType; // Local
...
Builder builder = <your mongo builder>
ClassModel<ProcessType> classModel = ClassModel.builder(ProcessType.class).build();
builder.register(classModel);

then, whenever you save an instance of a class with a property type ProcessType to DB, the resulting Mongo document will have string value CONVERT_ONE_TO_ONE or CONVERT_ONE_TO_MANY for that property.

if that is all you need, then the following is not for you. In that case, you can follow Mongo POJO tutorial to guide you.

Following is a way to store the value associated with a Java Enum in the MongoDB. Why would someone want to do that?

Java (also TypeScript) has a convention of using upper case name in Enums, which is probably inherited from the use of uppercase names for constants.
I prefer to assign lowercase values to Object properties (as many people do)
Prevent tying property name to its value. I prefer to keep the variable names short and the value assigned to it could be anything.

Above are a few reasons for saving Enum values instead of names to MongoDB.

Another pain point for me was comparing decoded Enum values in front-end. Following is the front-end TypeScript Enum for above Java Enum.

export enum WebsocketProcessType {
    CONVERT_ONE_TO_ONE = 'convert-one-to-one',
    CONVERT_ONE_TO_MANY = 'convert-one-to-many',
}

If we were to use the default Enum decoder provided by Mongo Java driver, then our values have to be the same as names on Java Enum, which is too coupled and strict for us to write better readable code.

With instruction bellow and the use of Class Transformer to decode data sent from backend, you will be able to seamlessly map Java classes to TypeScript(js) classes.

Implementation

Steps:

Create and register a codec provider with Mongo Code Registry which Mongo uses to determine which Enum decoder to use a Java Enum value
Create and register Enum decoder for ProcessType
Create and register Enum with DB

I will make some classes as Generic since this can be used to decode all any Enum.

Create a codec provider

I will not provide imports as you should have Mongo Java Driver and with modern IDEs, you can auto-import all imports.

public class EnumCodecProvider implements CodecProvider {
    @Override
    public <T> Codec<T> get(Class<T> clazz, CodecRegistry registry) {
        if (clazz == ProcessType.class) {
            return (Codec<T>) new ProcessTypeCodec();
        } 
        return null; // Don't throw here, this tells Mongo this provider doesn't provide a decoder for the requested clazz
    }
}

This is pretty simple. Mongo decoder, call get method of provider to get a decoder for a Class that it doesn't know how to decode. When it calls ....get(ProcessType.class, MongoRegisty) we will return our ProcessTypeCodec, which knows how to decode a ProcessType Enum.

CodecRegistry pojoCodecRegistry = 
                fromRegistries(MongoClient.getDefaultCodecRegistry(),
                    CodecRegistries.fromRegistries(
                        CodecRegistries.fromProviders(new EnumCodecProvider())
                    ), 
                );
MongoClientOptions options = MongoClientOptions.builder().codecRegistry(pojoCodecRegistry).build();
// Register above option with the MongoClient

Above registers the EnumCodeProvider with the mongo registry.

Create Enum Codec to encode/decode our Enum

I made an abstract decoder to put all common code that required to decode our Enum to avoid code duplication

abstract class AbstractCodec<T extends Enum<T>> implements Codec<T> {
    public AbstractCodec() {
    }

    @Override
    final public void encode(final BsonWriter writer, final T value, final EncoderContext encoderContext) {
        String val = ((Enum) value).toString();
        writer.writeString(val);
    }

    @Override
    final public T decode(final BsonReader reader, final DecoderContext decoderContext) {
        try {
            String value = reader.readString();
            Method method = getEncoderClass().getDeclaredMethod("fromValue", String.class);
            T enumName = (T) method.invoke(null, value);
            return enumName;
        }catch(Exception e) {
            try {
                String value = reader.readString();
                Method method = getEncoderClass().getDeclaredMethod("getDefaultValue");
                T storageType = (T) method.invoke(null, value);
                return storageType;
            } catch (Exception e1) {
                e1.printStackTrace();
            }
            e.printStackTrace();
        }
        return null;
    }

    public abstract Class<T> getEncoderClass();
}

Note that we call toString on the encode method above. This toString method must be implemented on ProcessType Enum class to provide the value of the Enum name.

On decode method, we call fromValue and getDefaultValue on our ProcessType Enum to get Enum name associated with a particular value stored on DB. Yes, you have to use Java reflection to execute method on a object of a class type T. If you don't like to use reflection, you can push the decode class to the ProcessTypeCodec and directly call the static method (see Enum Implementation below).

To sum up, when the decoder gets a request with a string value, i.e. "convert-one-to-one", we get the class name associated with this codec and calls a static method fromValue to get the Enum name that corresponds to the string value.

Following is the ProcessTypeCodec.

public class ProcessTypeCodec extends AbstractCodec<ProcessType> {

    public ProcessTypeCodec() {
        super();
    }

    @Override
    public Class<ProcessType> getEncoderClass() {
        return ProcessType.class;
    }
}

This just let Mongo know the class which this Codec can encode/decode.

Implement and register ProcessType enum

public enum ProcessType {
    CONVERT_ONE_TO_ONE("convert-one-to-one"),
    CONVERT_ONE_TO_MANY("convert-one-to-many");

    private String value;
    private static final Map<String, ProcessType> ENUM_MAP;
    static {
        Map<String, ProcessType> map = new HashMap<String, ProcessType>();
        for (ProcessType instance : ProcessType.values()) {
            map.put(instance.value(), instance);
        }
        ENUM_MAP = Collections.unmodifiableMap(map);
    }
    ProcessType(String type) {
        this.value = type;
    }

    public String value() {
        return this.value;
    }

    public static ProcessType fromValue(String value) {
        return ENUM_MAP.get(value);
    }

    /**
     * Used by the Mongo codec
     * 
     * @return
     */
    public static ProcessType getDefaultValue() {
        return CONVERT_ONE_TO_ONE;
    }

    /**
     * Required to properly convert Java Enum name to value.
     * Value is used by front-end and usually uses <br>
     * 1. lowercase <br>
     * 2. dashes instead of underscores <br> <br>
     */
    @Override
    public String toString() {
        return this.value;
    }
}

ENUM_MAP is just to speed up the process. It allows us the decoder to convert a string to Enum name in O(1) time complexity. Default is your preference, I used here an Enum name but this is usually null.

See above for registering classes with the Mongo class registry.

Our XConvert Video Compressor takes in some augments as Enum to populate the command argument needed for FFMPEG to compress or convert video files. For example, we have a output extension Enum on front-end

export enum OutputExtension {
    MP4 = '.mp4',
    WEBM = '.webm'
}

and on Back-end

public enum OutputExtension {
    MP4(".mp4"),
    WEBM(".webm")
    // ... rest of the code similar to above ProcessType Enum
}

when we store command line argument generated from TypeScript to DB on a document, it stores the actual value that we want i.e. .mp4 extenstion on DB. On the back-end, our decoder maps that value to related Java Enum properly. When we want to use this to build the FFMPEG command we can actually use the Enum value directly.
i.e.

class Request { // Sample class that deals with request document stored in DB
    OutputExtension outoutExtenstion;
}

List<String> cmd = List.of("ffmpeg", ..., "-o", Request.outoutExtenstion);
// This generates % ffmpeg ... -o .mp4

Hope this helps you to write more readable code. If you find any errors in this document, please let me know to rectify them.

Introduction to Java ProcessHandle

harithay — Wed, 25 Mar 2020 03:40:41 +0000

This is a quick introduction to application of Java ProcessHandle through a use case.

Background:

I developed a website call xconvert.com. It uses a range of free software to process user requests. To put it simply, it is an orchestrator for bunch of free opensource software such as FFMPEG. While there are libraries that allows me to use most of these software directly from my Java application (Mostly though JNI), they are always bit outdated or unstable. I, being a brave programmer, choose to use Java ProcessBuilder and Process to execute programs such as FFMPEG with arguments. Managing of processes spawn like that is bit chaotic. Until the release of Java 9, I had a code that mainly uses Runtime.getRuntime().exec("ps aux...") to find and kill processes that hangs or exceed time limit. I had quite a bit of code, an ugly code, written to manage processes. I got really exited when I came across Java 9 ProcessHandle interface. It exposes many information about processes running on a machine (specifically when used in Linux systems) which I could use to do a major cleanup.

About ProcessHandle:

Java ProcessHandle provides a interface to native processes. Interface allows reading information about processes, such as checking whether a process is alive or invoking simple operations on those process such as killing it.

Use case:

I will walk you though following use case which I used to explain some of the useful part of the ProcessHandle interface.

Get all running processes on the system
Filter ffmpeg processes that meets a certain criteria
1. Check if they are running
2. Check if they are running for more than 4 hours
3. Check if input file fed to ffmpeg is deleted
Kill all hanged ffmpeg processes

Step 1: Finding all processes

Stream<ProcessHandle> allProcesses = ProcessHandle.allProcesses();

Step 2: Filter processes

Stream<ProcessHandle> toDeleteProcessHandles = allProcesses.filter(processHandle -> {
    // Check if process is alive
    if(processHandle.isAlive()){
        Info info = processHandle.info();
        String executableWithPath = info.command().orElse(null);
        // Check if this is an ffmpeg instant
        if(executableWithPath != null && executableWithPath.endsWith("ffmpeg")){
            // Check if they are running for more than 4 hours
            if(Duration.between(Instant.now(), info.startInstant().get()).toHours() > 4){

                // Check if input file fed on to ffmpeg is deleted
                String[] arguments = info.arguments().orElse(null);
                String inputFile = Arrays.stream(arguments).filter(arg -> arg.startsWith("/home/username/content/input")).findFirst().orElse(null);
                if(inputFile != null && !Files.exists(Paths.get(inputFile))){
                    return true;
                }
            }
        }
    }
    return false;
});

Step 3: Kill hanged processes

toDeleteProcessHandles.forEach(processHandle -> processHandle.destroy());

This interface combined with Stream API allow writing concise code. You may have noticed that I used orElse(null) in some places. Which is from Java Optional class introduced in Java 1.8 with Stream API. You can further clean up this code by using methods such is isPresent() on Optional class instead of clustering with someVariable != null && doSomethingWithSomeVariable inside if expressions checks.

Another useful static method on this class is

ProcessHandle handle = ProcessHandle.of(long pid);

This is useful when you start a process using ProcessBuilder or have a Java Process object. Java Process object (Java 9+) exposes PID of a process though someProcess.pid(). Which you can use with ProcessHandle.of() to keep track of a process.

You could find more about ProcessHandle and ProcessHandle.Info by reading their documentation.

This very code is in action every time someone convert a video or compresses a jpeg on XConvert.

Thank you for reading. You are welcome to ask questions or give a feedback.