DEV Community

Dwayne Crooks
Dwayne Crooks

Posted on

Announcing dwayne/elm2nix

#elm #nix #haskell

Last year I worked a lot with Nix at my consulting job and I started to love it even more. I finally ascended its steep learning curve and began to enjoy using it for my development environments, CI setups, and VM configurations.

I wasn't doing much Elm or Haskell in my day-to-day so I was longing for a practical and interesting project to work on that might be beneficial to either the Elm or Haskell community, while at the same time, would extend my skills within functional programming.

When I researched cachix/elm2nix it seemed to fit perfectly with my goals because it combined all three technologies that I love, Elm, Haskell, and Nix, to produce a useful tool for Elm developers. On reading the code and seeing how Nix and Haskell were being used I immediately saw opportunities for growing my skills while making improvements to the tool.

Fast forward by about four months and I'm happy I took the plunge. I learned so much about each technology that I didn't know before and I was also able to find little ways to positively improve upon the project.

It is with great delight that I announce dwayne/elm2nix: A rewrite of cachix/elm2nix with a few changes and improvements.

What is dwayne/elm2nix?

Among other things, it is primarily a tool that helps you compile your Elm web application within a Nix build environment.

How to use dwayne/elm2nix?

In the folder containing your Elm web application's elm.json you use

elm2nix lock
Enter fullscreen mode Exit fullscreen mode

to generate an elm.lock lock file.

The lock file is used by a custom Nix build helper, called buildElmApplication, that builds your Elm web application.

myApp = buildElmApplication {
    name = "my-app";
    src = ./.;
    elmLock = ./elm.lock;
}
Enter fullscreen mode Exit fullscreen mode

If you have nix installed on your machine you can build the example Elm web application as follows:

nix build github:dwayne/elm2nix?dir=example
less result/elm.js
Enter fullscreen mode Exit fullscreen mode

And, that's basically it. For more details please read the usage section of the README.

What's changed or improved?

Several things actually. Let's go through them one by one.

You can input one or more elm.json files to create the lock file

The lock file is a JSON file that has support for multiple versions of the same package.

[
    ...
    {
        "author": "elm",
        "package": "html",
        "version": "1.0.1",
        "sha256": "1m045b6cixyqygll4cjf9d9s3k1lj839kxc3gbid0fpqc4g10i6b"
    },
    {
        "author": "elm",
        "package": "json",
        "version": "1.1.3",
        "sha256": "1hx986yqw1v2bpkrh6brszl8n8awwg1s8zi7v5qg0p1rqwvjlicz"
    },
    {
        "author": "elm",
        "package": "json",
        "version": "1.1.4",
        "sha256": "0niq7id7r78ckvgap4psq0xdlnhcv1dz2j81lrnhl4fsgn0awvs8"
    },
    {
        "author": "elm",
        "package": "parser",
        "version": "1.1.0",
        "sha256": "06xx29rmagc5r45qfpvrd393lz83ylngidfp08432f1qc8y6r3lh"
    }
    ...
]
Enter fullscreen mode Exit fullscreen mode

It acts as the source of truth for your project's Elm dependencies and it contains all the information Nix needs in order to prepare your ELM_HOME with everything the Elm compiler requires to compile your project offline.

Why would you want to pass multiple elm.json files to elm2nix lock?

One use case I came across that wasn't handled well by cachix/elm2nix was being able to run elm-review within a Nix build. Your review folder contains an elm.json file that contains depedencies used by elm-review but not by your Elm web application. However, to prepare the cache, to allow elm-review to run offline, those dependencies need to be considered.

As an example, the example project's elm.lock lock file was generated using

elm2nix lock elm.json review/elm.json
Enter fullscreen mode Exit fullscreen mode

registry.dat is automatically generated for you

buildElmApplication generates the registry.dat file for you on the fly. That's one less file you have to consider managing.

You can display any registry.dat file as JSON

The registry.dat file is a binary file that encodes all the Elm packages in your cache. The registry subcommand of elm2nix has a view subcommand that allows you to display any valid registry.dat file as JSON.

elm2nix registry view
Enter fullscreen mode Exit fullscreen mode

You can certainly use a tool like xxd to get a sense of its contents but I found that having a human-readable view was a huge benefit for debugging purposes.

In fact, if you're curious and want to understand the binary format used by the registry.dat file you can read the binarySerializationSpec.

The buildElmApplication build helper allows you to build an Elm web application in a variety of ways by setting options

All the typical tasks you'd want to perform during a build of your Elm web application are possible by changing a few options.

Turn on the time-travelling debugger 

debuggedMyApp = myApp.override {
    enableDebugger = true;
    output = "debugged.js";
}
Enter fullscreen mode Exit fullscreen mode

Fail the build if your Elm source code is improperly formatted 

formattingCheckedMyApp = myApp.override {
    doElmFormat = true;
    elmFormatSourceFiles = [ "review/src" "src" "tests" ];
    output = "formatting-checked.js";
}
Enter fullscreen mode Exit fullscreen mode

It runs elm-format --validate on the Elm source code it finds in the given directories.

Fail the build if your Elm tests fail 

testedMyApp = formattingCheckedMyApp.override {
    doElmTest = true;
    output = "tested.js";
}
Enter fullscreen mode Exit fullscreen mode

It runs your Elm tests.

BTW, since I overrode formattingCheckedMyApp, the testedMyApp derivation will also fail for improper formatting reasons. So, you can override buildElmApplication-based derivations in the standard Nix ways and it will work as expected.

Fail the build if elm-review fails 

reviewedMyApp = testedMyApp.override {
    doElmReview = true;
    output = "reviewed.js";
}
Enter fullscreen mode Exit fullscreen mode

It runs elm-review.

Thank you to Jeroen Engels for all his help debugging the various issues I had getting elm-review working in a Nix build environment.

Turn on optimizations 

optimizedMyApp = reviewedMyApp.override {
    enableOptimizations = true;
    output = "optimized.js";
}
Enter fullscreen mode Exit fullscreen mode

There are three optimization levels. By default, it uses the first level which compiles your Elm code with elm make --optimize.

optimized2MyApp = optimizedMyApp.override {
    optimizeLevel = 2;
    output = "optimized2.js";
}
Enter fullscreen mode Exit fullscreen mode

Levels 2 and 3 both use elm-optimize-level-2. The only difference is that level 3 provides the --optimize-speed flag to elm-optimize-level-2.

Combine multiple Elm web applications into one JavaScript file 

combinedMyApp = optimizedMyApp.override {
    entry = [ "src/App1.elm" "src/App2.elm" ];
    output = "combined.js";
}
Enter fullscreen mode Exit fullscreen mode

The Elm compiler provides support for combining multiple web applications into one JavaScript file and buildElmApplication supports that use case as well through the entry option.

However, it must be noted that since elm-optimize-level-2 doesn't support that use case, you will not be able to combine multiple Elm web applications when level 2 or 3 optimization is turned on. In fact, this situation is correctly detected by buildElmApplication and explicitly disallowed.

Enable minification with UglifyJS or Terser 

minifiedMyApp = optimized2MyApp.override {
    doMinification = true;
    useTerser = true;
    output = "minified.js";
}
Enter fullscreen mode Exit fullscreen mode

By default it uses UglifyJS as recommended by What I've learned about minifying Elm code.

Minification is done in a separate minification phase, called minificationPhase, that's easily customizable. What this means is that you can configure it to do anything that lydell suggested in his Elm Discourse post.

In fact, everything I've shown you up to this point and will show you afterwards happens in independent phases so it's all easily customizable once you understand how phases work in Nix.

Enable compression with gzip and brotli 

compressedMyApp = minifiedMyApp.override {
    doCompression = true;
    output = "compressed.js";
}
Enter fullscreen mode Exit fullscreen mode

It compresses the JavaScript file using both gzip and brotli. By default, it passes the -9 flag to gzip and the -Z flag to brotli. You can configure both by passing the relevant flags to the gzipFlags and brotliFlags options as you see fit.

Show a report about the changes in your file size due to minification and compression 

reportedMyApp = compressedMyApp.override {
    doReporting = true;
    output = "reported.js";
}
Enter fullscreen mode Exit fullscreen mode

When you turn on reporting and build the derivation using nix build -L you will see a report detailing the impact that minification and compression had on the output size of the JavaScript that's produced. If you forget to build with -L then you can always view the logs generated by a build of your derivation using nix log.

The report is similar to the one used in the Asset Size section of the Elm Guide.

Enable content hashing for cache busting purposes 

hashedMyApp = reportedMyApp.override {
    doContentHashing = true;
    output = "hashed.js";
}
Enter fullscreen mode Exit fullscreen mode

Content hashing or fingerprinting takes your hashed.js, finds its sha256sum, and rewrites the name of the file to be something like hashed.abcd1234.js. This is useful for cache busting purposes.

By default, it uses a hash length of 8 and it removes the files with no hash in their names. Use the hashLength option to change the hash length and set keepFilesWithNoHashInFilenames to true so as not to remove the original files.

In addition, a manifest.json is generated to help you map the original filenames to their hashed equivalents when referring to them in your HTML or other resources.

All at once

Up to this point, I presented the possibilities by overriding previous derivations. However, you can definitely just configure it all at once.

As an example, the hashedMyApp derivation that we ended with is equivalent to the following finalMyApp derivation which configures everything in one go:

finalMyApp = buildElmApplication {
    name = "my-app";
    src = ./.;
    elmLock = ./elm.lock;

    doElmFormat = true;
    elmFormatSourceFiles = [ "review/src" "src" "tests" ];

    doElmTest = true;
    doElmReview = true;

    enableOptimizations = true;
    optimizeLevel = 2;

    doMinification = true;
    useTerser = true;

    doCompression = true;
    doReporting = true;
    doContentHashing = true;

    output = "hashed.js";
}
Enter fullscreen mode Exit fullscreen mode

In my opinion, the best way to understand the extent of what's possible is to read the Nix code for buildElmApplication.

Advanced: The derivations and scripts that make buildElmApplication work are readily available for you to use for your own purposes

The elm2nix flake gives you access to several derivations and scripts which you can access as follows:

inherit (elm2nix.lib.elm2nix pkgs)
    buildElmApplication
    generateRegistryDat
    prepareElmHomeScript
    dotElmLinks
    symbolicLinksToPackagesScript
    fetchElmPackage
    ;
Enter fullscreen mode Exit fullscreen mode

You've already seen buildElmApplication. Let's talk about prepareElmHomeScript and dotElmLinks.

prepareElmHomeScript 

elmLock = ./elm.lock;

registryDat = generateRegistryDat {
    inherit elmLock;
};

examplePrepareElmHomeScript = prepareElmHomeScript {
    inherit elmLock registryDat;
};
Enter fullscreen mode Exit fullscreen mode

It returns a string of Bash commands that will be used to configure ELM_HOME. If you have nix installed on your machine you can view the script from the example Elm web application as follows:

nix repl
> :lf github:dwayne/elm2nix?dir=example
> :p outputs.scripts.x86_64-linux.examplePrepareElmHomeScript
echo "Prepare .elm and set ELM_HOME=.elm"
cp -LR "/nix/store/wr561wqfhpr98xvji2q1g72y5ylgmq3p-dot-elm-links" .elm
chmod -R +w .elm
export ELM_HOME="$PWD/.elm"
Enter fullscreen mode Exit fullscreen mode

N.B. The above example assumes an x86_64-linux system but there is also support for aarch64-darwin, aarch64-linux, and x86_64-darwin.

dotElmLinks 

exampleDotElmLinks = dotElmLinks {
    inherit elmLock registryDat;
};
Enter fullscreen mode Exit fullscreen mode

The derivation creates an Elm cache containing precisely the dependencies in your elm.lock lock file. If you have nix installed on your machine you can view the output of the exampleDotElmLinks derivation from the example Elm web application as follows:

nix build github:dwayne/elm2nix?dir=example#exampleDotElmLinks -L
tree result
Enter fullscreen mode Exit fullscreen mode

Recently, there's been a lot of interesting discussions in the Elm community about issues compiling Elm web applications on external machines, for example in CI or on Digital Ocean VMs. If all you need is an Elm cache of your application's dependencies, then dotElmLinks can be used to prepare it for you. Either way, if you're using elm2nix then those issues won't be issues for you at all.

Conclusion

Nix may have a steep learning curve but I'm convinced it's an immensely valuable tool to have in your toolbox. dwayne/elm2nix brings the power of Nix to Elm and doesn't compromise on the versatility of Nix. The possibilities are endless and this is just the beginning. I hope we can all get to see the full power of Nix applied to the Elm ecosystem.

Finally, thank you to Domen Kožar for sharing his ideas, back in 2016, that's almost 10 years ago, on how elm2nix could work in the first place (elm2nix 0.1). Over the years a lot of developers have contributed to molding cachix/elm2nix to what it is today and it is their work that I've been able to build upon and improve, so I'm immensely greatful to all the contributors of that project as well.

Subscribe to my newsletter

If you're interested in improving your skills with Elm then I invite you to subscribe to my newsletter, Elm with Dwayne. To learn more about it, please read this announcement.

Top comments (0)