Modules, and Java, and Windows, Oh My!

I get that desktop applications are out of style these days. But they are still a good way to deliver powerful features and high performance graphics.

However, delivering modular Java applications on Windows is a challenge. It almost seems like Java and Windows have conspired against each other. Java requires that all module jars are listed in one command line parameter. That works fine when Unix shells have a >100K character command line. Windows batch files have a relatively short limit (8191) on the lengths of their command lines. This makes it impossible to launch large-ish modular Java applications on Windows using the standard scripting model of every parameter on the JVM’s launch command line.

The work around is to specify an argument file (e.g. @app.args) when launching Java. Building that argument file takes a trick or two. You can jump ahead and see the full solution for DepanFX at commit 19435687 Customize command script for Windows ….

Problem Statement

The DepanFX application provides visual tools for analyzing large programs. It is written in Java, and its architecture adheres to the structures of the Java Platform Module System (JPMS). It's structure is typical of medium to large modular Java applications. It depends on a large number of external libraries,
and many of these components must be listed on the module path when
the Java JVM is started.

When any modular Java application is launched, the Java JVM requires that the --module-path argument is followed by a list of all of the module jars.
It’s easy to have dozens of modules, so that module path list argument very quickly gets to be very long.

20 characters per jar file name * 80 module jars = 1600 characters

Not too bad, but the absolute installation path can easily add 80 characters per module entry.

80 path characters per module jar * 80 module jars = 6400 characters
Total characters per module file = 8000 characters.

All of the sudden, the command line is up against the Windows command line length limit. Many of the remaining 191 characters get taken up by the Java launch command, required command flags, and the application's classname, and required punctuation. Conventional launch scripts crash when this happens.

The Workaround

As Microsoft so blithely advises:

To work around the limitation …. Modify programs that require long command lines so that they use a file that contains the parameter information, and then include the name of the file in the command line.

Despite the inherent complexity of this suggestion, Java has implemented an argument file mechanism for long parameters. All that is required is to write the arguments into a temporary file and replace them on the command line with an @app.args on the command line argument.

Revising the Launch Script

One of the first challenges is that the Windows launch script is a generated artifact. Like many applications, DepanFX’s overall compile, assemble, and publish processes are coordinated by Gradle.

The Gradle application plugin handles construction of the launch scripts for both Unix-like and Windows operating systems. The Unix scripts assume a POSIX-ish platform is available to /bin/sh scripts. The default Windows script follows a similar functional pattern. These start scripts:

• Establish the APP_HOME variable as the base for the installation.
• Confirm or discover the path for the Java command.
• Run Java with all the parameters required to execute the application.

For both Unix and Windows, running Java includes setting a number of environment variables that are used in the launch command. If all the options are stuffed into a temporary file, none of those environment variables will be necessary.

Both of the launch scripts are configured using replaceable templates. The Windows specific template can be replaced by adding the following snippet to the application’s build.gradle file:

tasks.named("startScripts") {
  windowsStartScriptGenerator.template =
     resources.text.fromFile('gradle\\customWindowsStartScript.txt')
}

With this in place, Gradle will build the application’s Window launch script using the customized start template instead of the build-in template.

The text of the built-in template is available via github as windowsStartScript.txt in the plugins-application subproject. It’s a great place to start, with Windows specific commands to configure APP_HOME and to establish a viable JAVA_EXE. These can be left as is.

The sections that configure Java related environment variables need to be converted to write these values to a temporary file. The environment variables are only used in the Java launch command, and that is being converted to use the temporary file.

Instead of one line setting an environment variable to a potentially very long template supplied value:

set MODULE_PATH=$modulePath

The new template file writes a separate line to the launch script for each element of the module path. Each batch file line will append the full path name of an installed module to the argument file.

>>"%ARG_SPEC%" echo --module-path
<% for (moduleEntry in modulePath.split(';')) {
  %>echo "${moduleEntry}" >>"%ARG_SPEC%"
<%
}%>

The full set of changes to capture all the Java launch parameter as an argument file are visible at depanWindowsStartScript.txt lines 76 to 96.

With all the Gradle specified options moved from the command line into an arguments file, it should be straightforward to launch the Java application with the following Windows command line

"%JAVA_EXE%" @"%ARG_FILE%" %*

This will start the Java JVM ("%JAVA_EXE%") using the arguments in the argument file (@"%ARG_FILE%") and any command line arguments provided by the user (%*).

Unfortunately, this doesn’t work. The arguments written by Windows are in the wrong format. Most importantly, path names for the modules are saved as Windows filenames, with single back-slashes ("\"). Additionally, multi-line parameters don’t have the correct end of line terminators.

Converting the Argument File

On Unix platforms with a normal shell (/bin/sh) and $PATH, these kinds of corrections can be handled with common text editing tools like sed. None of these tools for manipulating text are available in a vanilla Windows installation.

The one extended resource that is available is the Java JVM, available at "%JAVA_EXE%". With the addition of a very small Java application, it’s straightforward to rewrite the Java launch specification file written by the launch script into a launch argument file that does comply with Java’s rules for backslash encoding and other argument rules.

Simply run this before running the intended application, and all the arguments are provided to the Java JVM in a manner that works for Windows.

"%JAVA_EXE%" -cp "%APP_HOME%\\lib\\DepanFxLauncher.jar" com.pnambic.depanfx.launcher.DepanFxArgFile "%ARG_SPEC%" "%ARG_FILE%"

The main class DepanFxArgFile runs to just under 200 lines of text, including whitespace and comments. The whole jar file is under 7KB. It reads a launch spec file from the supplied filename, and writes a Java argument file to the second filename. The transformations are simple, but include:

• backslash escaping
• separating multi-argument lines into individual lines
• concatenation of --module-path and --classPath

Adding the launcher project to the application project as a runtime dependency ensures that Gradle’s application builder places the DepanFxLauncher.jar in the expected %APP_HOME%\lib directory.

All Together Now

The entire set of changes for DepanFX is encapsulated in commit 19435687 Customize command script for Windows ….

This commit modifies the Gradle generated Windows launch script with these changes:

• Add the Gradle launcher project, with the DepanFxLauncher.jar artifact.
• Use the launcher project as a runtime dependency for the application.
• Set the Gradle template for the Windows script to be customized for the project.
• Revise the Windows start script template to create a launch specification from the template values.
• Add the transform of the launch specification into a Java argument file.
• Adapt the application launch to use the generated argument file.

With these changes in place, it appears that Gradle’s application plugin can continue its happy role as the packager of choice for DepanFx.

Perhaps these notes will show a path forward for your modular Java desktop applications.