debugging the micronaut package and code smells in nix

I am the maintainer for NixOS's micronaut package. This package installs the micronaut cli which can be used for creating projects and classes. The package is very simple. Here is the original.

{ stdenv, fetchzip, jdk, makeWrapper, installShellFiles }:

stdenv.mkDerivation rec {
  pname = "micronaut";
  version = "1.3.2";

  src = fetchzip {
    url = "https://github.com/micronaut-projects/micronaut-core/releases/download/v${version}/${pname}-${version}.zip";
    sha256 = "0jwvbymwaz4whw08n9scz6vk57sx7l3qddh4m5dlv2cxishwf7n3";
  };

  nativeBuildInputs = [ makeWrapper installShellFiles ];

  installPhase = ''
    runHook preInstall
    rm bin/mn.bat
    cp -r . $out
    wrapProgram $out/bin/mn \
      --prefix JAVA_HOME : ${jdk} 
    installShellCompletion --bash --name mn.bash bin/mn_completion
    runHook postInstall
  '';

  meta = ...;
}

This package copies the content into $out, wraps the mn command, and installs shell completion.

There are two problems with this script. First, --prefix is used for JAVA_HOME instead of --set. This causes two paths to be in the variable which can cause problems. Second, my user's PATH contains jdk14 but the script should use the default jdk.

{ stdenv, fetchzip, jdk, makeWrapper, installShellFiles }:

stdenv.mkDerivation rec {
  pname = "micronaut";
  version = "1.3.2";

  src = fetchzip {
    url = "https://github.com/micronaut-projects/micronaut-core/releases/download/v${version}/${pname}-${version}.zip";
    sha256 = "0jwvbymwaz4whw08n9scz6vk57sx7l3qddh4m5dlv2cxishwf7n3";
  };

  nativeBuildInputs = [ makeWrapper installShellFiles ];

  installPhase = ''
    runHook preInstall
    rm bin/mn.bat
    cp -r . $out
    wrapProgram $out/bin/mn \
      --prefix PATH : ${jdk}/bin 
    installShellCompletion --bash --name mn.bash bin/mn_completion
    runHook postInstall
  '';

  meta = ...;
}

This still introduces the user's entire path into the script breaking purity of the install. One principle of NixOS is that the environment is pure. Every input for the application is declared and the ouput is deterministic. Even though at this point the package works on my computer it is incorrect. Since PATH can contain anything there could be missing input in the package. The package may work when run locally but it may not always work with every user's PATH variable. Using --prefix for PATH should be considered a code smell in nix. --set should be used instead.

{ stdenv, fetchzip, jdk, makeWrapper, installShellFiles }:

stdenv.mkDerivation rec {
  pname = "micronaut";
  version = "1.3.2";

  src = fetchzip {
    url = "https://github.com/micronaut-projects/micronaut-core/releases/download/v${version}/${pname}-${version}.zip";
    sha256 = "0jwvbymwaz4whw08n9scz6vk57sx7l3qddh4m5dlv2cxishwf7n3";
  };

  nativeBuildInputs = [ makeWrapper installShellFiles ];

  installPhase = ''
    runHook preInstall
    rm bin/mn.bat
    cp -r . $out
    wrapProgram $out/bin/mn \
      --set PATH ${jdk}/bin 
    installShellCompletion --bash --name mn.bash bin/mn_completion
    runHook postInstall
  '';

  meta = ...;
}

This results in an error because there is in fact a missing dependency for micronaut that was not accounted for.

Error: Could not find or load main class io.micronaut.cli.MicronautCli

This is an interesting problem. At this point I almost gave up. The script mn builds a variable CLASSPATH which is used by java to run the application. If the class cannot be found this means the classpath is incorrect. Something I learned about bash is that it can be debugged using set -x. This can be patched into the mn script using nix.

  patchPhase = ''
    sed -i '2iset -x' bin/mn
  '';

Running mn again reveals some missing dependencies that result in classpath not being setup correctly.

...
++ dirname /nix/store/mhdwrfxd9dj0hipzygbdy4h70dhkq3yh-micronaut-1.3.4/bin/.mn-wrapped
/nix/store/mhdwrfxd9dj0hipzygbdy4h70dhkq3yh-micronaut-1.3.4/bin/.mn-wrapped: line 24: dirname: command not found
...
++ basename /nix/store/mhdwrfxd9dj0hipzygbdy4h70dhkq3yh-micronaut-1.3.4/bin/.mn-wrapped
/nix/store/mhdwrfxd9dj0hipzygbdy4h70dhkq3yh-micronaut-1.3.4/bin/.mn-wrapped: line 29: basename: command not found
...
/nix/store/mhdwrfxd9dj0hipzygbdy4h70dhkq3yh-micronaut-1.3.4/bin/.mn-wrapped: line 53: uname: command not found
+ CLASSPATH=//cli-1.3.4.jar
...

The missing dependencies are all located in the coreutils package. Its path can be added.

--set PATH ${coreutils}/bin:${jdk}/bin

Now when mn is run java appears to run and start the application but there is an exception.

Exception: java.lang.StackOverflowError thrown from the UncaughtExceptionHandler in thread "main"

The problem here is not obvious at all. It requires debugging the jvm. To do this from a nix expression options need to be added to the start the application with the debugger enabled. The mn script passes these options by setting MN_OPTS.

  installPhase = ''
    runHook preInstall
    rm bin/mn.bat
    cp -r . $out
    wrapProgram $out/bin/mn \
      --set MN_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000 \
      --set JAVA_HOME ${jdk} \
      --set PATH ${coreutils}/bin:${jdk}/bin
    installShellCompletion --bash --name mn.bash bin/mn_completion
    runHook postInstall
  '';

To debug and set breakpoints I needed to checkout micronaut-core and open it in intellij. After a few hours I was able to narrow the problem down to the jline dependency. This code calls sh which is not on PATH.

The final result:

{ stdenv, coreutils, fetchzip, jdk, makeWrapper, installShellFiles }:

stdenv.mkDerivation rec {
  pname = "micronaut";
  version = "1.3.4";

  src = fetchzip {
    url = "https://github.com/micronaut-projects/micronaut-core/releases/download/v${version}/${pname}-${version}.zip";
    sha256 = "0mddr6jw7bl8k4iqfq3sfpxq8fffm2spi9xwdr4cskkw4qdgrrpz";
  };

  nativeBuildInputs = [ makeWrapper installShellFiles ];

  installPhase = ''
    runHook preInstall
    rm bin/mn.bat
    cp -r . $out
    wrapProgram $out/bin/mn \
      --set JAVA_HOME ${jdk} \
      --set PATH /bin:${coreutils}/bin:${jdk}/bin
    installShellCompletion --bash --name mn.bash bin/mn_completion
    runHook postInstall
  '';

  meta = with stdenv.lib; {
    description = "Modern, JVM-based, full-stack framework for building microservice applications";
    longDescription = ''
      Micronaut is a modern, JVM-based, full stack microservices framework
      designed for building modular, easily testable microservice applications.
      Reflection-based IoC frameworks load and cache reflection data for 
      every single field, method, and constructor in your code, whereas with 
      Micronaut, your application startup time and memory consumption are 
      not bound to the size of your codebase.
    '';
    homepage = "https://micronaut.io/";
    license = licenses.asl20;
    platforms = platforms.all;
    maintainers = with maintainers; [ moaxcp ];
  };
}

To recap the only changes made were to set JAVA_HOME instead of prefix it and to set PATH to all inputs into the application runtime. This problem was very easy to cause and difficult to debug. It is possible that it is in other packages in nix. Especially packages I wrote.