Building an OSGi-like Plugin System in Java

If you are not familiar with what OSGi is, you can find additional details here and here.

The complete project can be found on github here: https://github.com/Larisho/osgi

The Spec

By the end of this tutorial, we will have an MVP that should:

• accept a path to a plugin (jar file)
• creates an instance of the plugin
• runs the plugin

Requirements

To follow along, you will only need the following:

• Java 8+
• Maven 3+

Phase 1: Boilerplate!

Let's get right into it. Open up a terminal and follow along.

mkdir ~/osgi
cd ~/osgi

mvn archetype:generate -DgroupId=com.gabdavid.osgi                       \
                       -DartifactId=engine                               \
                       -DarchetypeArtifactId=maven-archetype-quickstart  \
                       -DinteractiveMode=false
mvn archetype:generate -DgroupId=com.gabdavid.osgi                       \
                       -DartifactId=sample-plugin                        \
                       -DarchetypeArtifactId=maven-archetype-quickstart  \
                       -DinteractiveMode=false

Make sure that your pom.xml files look like the following:
~/osgi/engine/pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.gabdavid.osgi</groupId>
  <artifactId>engine</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>engine</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.2.0</version>

        <configuration>
          <archive>
            <manifest>
              <mainClass>com.gabdavid.osgi.engine.Engine</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

~/osgi/sample-plugin/pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.gabdavid.osgi</groupId>
  <artifactId>sample-plugin</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>sample-plugin</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.2.0</version>

        <configuration>
          <archive>
            <manifest>
              <mainClass>com.gabdavid.osgi.sampleplugin.App</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

NOTE: I've moved and renamed where the main source file is in each project. Feel free to not do that, just make sure that the mainClass tags reflect where the actual main classes are.

Now we're ready to roll! Next up, a little theory.

Phase 2: The Theory

Interfacing with the Plugin

The first thing we need to do is figure out a way for the engine to interface with the plugin.

One option would be to rely on a class in the Java standard library. A benefit of this option is that we do not need a third project that contains classes that both the engine and the plugins can depend on. A downside is that we have no way of specifying "plugin system specific" methods in a type-safe way. For the sake of simplicity of our MVP, this is the option that we're going to go with.

The class from the standard library we will be using is the Runnable class. We will be able to make use of its run() method to run the plugin itself.

Reading a jar file

The second issue we need to address is how to read the jar file. For those not "in the know", I'll let you in to a little secret: jar files are just zip files organized in a certain way! Which means that you could unzip a jar and read the class files somehow (we'll get to that in a second) and create instances of the classes defined. Luckily, we don't need to go through all of that work because Java's standard library comes with a JarInputStream that will iterate through all of the "entries" (files) in the jar (zip) file and allow you to access them.

Finding the right class file to load

The next issue is finding the right file to read. For the sake of simplicity, we will require that the plugin include the entry class's canonical name (<package>.<classname>) in the MANIFEST.MF. This simplifies things because the JarInputStream has a getManifest method which will give us a Manifest object containing all of the properties defined.

Loading the class

The final issue is actually reading the class file and running it! We will be using a UrlClassLoader to dynamically load the classes. We will be using its loadClass method to load our class from byte code. The loadClass method will return a Class object which we will use to create instances of the class using the reflection API.

Class Loading 101

If you already know how class loading works in Java, you can skip this section. Huge caveat: this is a quick explanation for the sake of context and is not a 100% complete explanation!

Java source code is compiled into byte code (the content of a class file) which is then run by the JVM. All references to objects outside of the class file are resolved at runtime, as opposed to a language like C which resolves all include macros at compilation time. The way that these classes are resolved at runtime is with a ClassLoader. If a ClassLoader cannot find a particular class it will throw a ClassNotFoundException. The class loader is able to take byte code and turn it into an actual Class object.

Phase 3: Writing the Engine

Here is our MVP for the engine:
~/osgi/engine/src/main/java/com/gabdavid/osgi/engine/Engine.java

package com.gabdavid.osgi.engine;

import java.io.File;
import java.io.FileInputStream;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.jar.Attributes;
import java.util.jar.JarInputStream;
import java.util.jar.Manifest;

public class Engine {

  public static void main(String[] args) throws Exception {

    // Have to check that a path has been passed in
    if (args.length != 1) {
      System.out.println("Path must be passed in");
      return;
    }

    String path = args[0];
    File jarFile = new File(path);

    String className;  
    try (JarInputStream jarStream = new JarInputStream(new FileInputStream(jarFile))) {
      Manifest manifest = jarStream.getManifest();

      // Get Main-Class attribute from MANIFEST.MF
      className = manifest.getMainAttributes().getValue(Attributes.Name.MAIN_CLASS);
    }

    System.out.println("Class name found in manifest is: " + className);

    // Here, we initialize the class loader with the path to the jar file.
    //
    // We use the jar protocol here. The !/ is used to separate between the URL to the actual
    // jar resource and the path within the jar. For example, I could download a whole jar by
    // calling jar:http://something.com/myjar!/ or I could download only 1 class file by calling
    // jar:http://something.com/myjar!/com/gabdavid/myjar/File.class
    // See https://docs.oracle.com/javase/7/docs/api/java/net/JarURLConnection.html for more details
    URLClassLoader classLoader = new URLClassLoader(
        new URL[]{new URL("jar:" + jarFile.toURI().toURL() + "!/")}
    );

    Class<?> pluginClass = classLoader.loadClass(className); // Load the class

    System.out.println("Package of plugin class: " + pluginClass.getPackage());
    System.out.println("Interface of plugin class is: " + pluginClass.getInterfaces()[0].getName());
    System.out.println("Creating a new instance now....!");

    // Create an instance of the loaded class and cast it from an Object
    // to a Runnable. We assume that this will succeed because that is the
    // contract we've agreed to (see theory section).
    Runnable plugin = (Runnable) pluginClass.newInstance();
    System.out.println("Running instance now....!");
    plugin.run();

    System.out.println("Exiting successfully!");
  }
}

Phase 4: Writing the Plugin

Here is our super simple plugin:
~/osgi/sample-plugin/com/gabdavid/osgi/sampleplugin/App.java

package com.gabdavid.osgi.sampleplugin;

public class App implements Runnable {

  // This is here for the sake of completeness, really. Wouldn't want to tell
  // Maven that this class has a main method when it doesn't actually have one!
  public static void main(String[] args) {
    new App().run();
  }

  public void run() {
    System.out.println("Hello from the Sample Plugin!");
  }
}

Phase 5: Let's Run the Thing Already!

Let's run it!

cd ~/osgi/engine
mvn package

cd ~/osgi/sample-plugin
mvn package

java -jar ~/osgi/engine/target/engine-1.0-SNAPSHOT.jar ~/osgi/sample-plugin/target/sample-plugin-1.0-SNAPSHOT.jar

# Output:
# Class name found in manifest is: com.gabdavid.osgi.sampleplugin.App
# Package of plugin class: package com.gabdavid.osgi.sampleplugin
# Interface of plugin class is: java.lang.Runnable
# Creating a new instance now....!
# Hello from the Sample Plugin!
# Exiting successfully!

There you have it! You've now dynamically loaded a different jar file and run their code! From here, the possibilities are endless.

Conclusion

Drawbacks of our current solution

Using Runnable

As we discussed in the theory section, using the Runnable interface as our "adapter", if you will, limits us in the number of operations that we can call in a type-safe way. We may want to have lifecycle hooks or something as well as the main run method. I keep saying "type-safe" because we could always just cast the plugin to an Object and use reflection to call whatever methods that we want but that leaves a lot of room for error and could lead to (fictional) plugin-writers becoming frustrated.

Doesn't Really Do Anything

This really doesn't do anything! A future post will be to expand on this project so that it could actually do something.

Future Plans?

I plan on writing a part 2 to this post where I expand on the engine. If you have any suggestions of features to be added, please comment!

Here's a short list of things that I'm thinking of improving on:

• Turn the engine into a repl of sorts where we can load and run different plugins
• Have the ability to reload a plugin
• Create a library where we can define an actual contract specific to our engine. That way, we can specify lifecycle hooks and get a little more creative with our plugins.