Reloaded workflow with nbb & expressjs

nbb (whatever the 'n' represents 😄 ) is a fascinating project. It brings the power of babashka to nodejs. I spent this week playing with it and want to share what I've found. It's pretty cool!

tl;dr

You can view the code for this blog post here. Supply this as a dependency using clj -Spath and nbb --claspath.

Hello, world

A strong learning experience for this project consists of a "hello world" web server, command-line argument parsing, and state management to simulate a database.

Along the way, I should learn something about dependency management and tooling.

Tooling

I almost can't believe it, but nbb can start a nrepl server. It's a bit fussy (not all clojure-mode commands operate correctly in Emacs, for example), but it works.

To launch a nrepl server, run nbb nrepl-server.

Then, in Spacemacs open a .cljs file. Then SPC m i (sesman-start) and connect to localhost: with cider-connect-clj. This operation will connect you to the nrepl-server with a sweet 2-dimensional buffer.

There are a handful of things that don't currently work (like cider-switch-to-repl-buffer)¹, but you can switch to it with SPC b b (list-buffers).

So far, nbb's nrepl-server has blown me away with its polish at this early stage of development.

Parsing command-line arguments with yargs.

I began with yargs, and while it functioned, yargs was not ideal.

• yargs complects arguments with commands/options.

The following code illustrates how you cannot describe commands and options without first providing user arguments.

(-> argv # argv should be unecessary
    yargs
    (.command ...)
    (.options ...)

• yargs kills the process after handling --help

This behavior is not ideal because it makes testing difficult at the repl. I should be able to craft help instructions without starting a new process.

Luckily, borkdude packaged tools.cli with v0.3.0 of nbb. Of course, if you need to use subcommands, yargs might still be a better option, but I'm going with tools.cli for now.

Parsing command-line arguments with tools.cli.

tools.cli works the same as in Clojure. Feel free to skip this section if you are already familiar with tools.cli.

The application's entry-point is a 'main' function to which command-line arguments are passed as varargs. nbb also stuffs arguments into a seq called *command-line-args*.

First, create a hello_world.cljs file, then paste the following code.

(ns hello-world
  (:require [clojure.tools.cli :as cli]))

(def default-port 3000)

(def cli-options
  [["-p" "--port PORT" "Port number"
    :default default-port
    :parse-fn js/Number
    :validate [#(< 1024 % 0x10000) "Must be a number between 1024 and 65536"]]
   ["-h" "--help"]])

(defn handle-args [args] (println args))

(defn main
  [& args]
  (handle-args
   (cli/parse-opts
    args cli-options)))

Try this at the repl to see how tools.cli works.

hello-world> (main)
{:options {:port 3000}, :arguments [], :summary   -p, --port PORT  3000  Port number
  -h, --help, :errors nil}

hello-world> (main "--port" "9093")
{:options {:port 9093}, :arguments [], :summary   -p, --port PORT  3000  Port number
  -h, --help, :errors nil}

hello-world> (main "--help")
{:options {:port 3000, :help true}, :arguments [], :summary   -p, --port PORT  3000  Port number
  -h, --help, :errors nil}

hello-world> (main "--port" "foobar")
{:options {:port 3000}, :arguments [], :summary   -p, --port PORT  3000  Port number
  -h, --help, :errors [Failed to validate "--port foobar": Must be a number between 1024 and 65536]}

cli/parse-opts generates a map containing all the components we need for handling command-line arguments.²

• :options: The parameters the application will use
• :summary: A formatted string we can print for help docs
• :errors: Any validation errors. You can see our custom error message here.

Let's change the definition of handle-args to do something useful.

(defn start-app [{:keys [port]}]
  (println "starting server on port" port))

(defn print-help [summary]
  (println "hello world server")
  (println summary))

(defn print-errors
  [{:keys [errors summary]}]
  (doseq [e errors]
    (println e))
  (print-help summary))

(defn handle-args
  [{:keys [options summary errors] :as args}]
  (cond
    (seq errors) (print-errors args)
    (:help options) (print-help summary)
    :else (start-app options)))

Feel free to run the same things from the repl again. You should see formatted text no matter what you pass in.

Running from the terminal

This next task admittedly gave me some trouble, but three discoveries helped immensely.³

1. A --main <ns>/<fn> parameter can be supplied to nbb the command line.
2. You shouldn't pass the script as an argument. Instead, ensure it's in the classpath with --classpath <dir1:dir2:...>.
3. nbb automatically includes the current directory in the classpath.

#2 is particularly notable because you could add all your scripts to a central directory, include that directory by default in your shell init, and run your scripts without specifying their name or filesystem location.

Feel free to do that, but the rest of this article will assume you're executing from the directory where you saved hello_world.cljs.

$ nbb --main hello-world/main --help
hello world server
  -p, --port PORT  3000  Port number
  -h, --help

$ nbb --main hello-world/main
starting server on port 3000

$ nbb --main hello-world/main --port 9093
starting server on port 9093

$ nbb --main hello-world/main --port foobar
Failed to validate "--port foobar": Must be a number between 1024 and 65536

expressjs

The installation process for expressjs is mundane if you are familiar with nodejs. First, run npm install express to get expressjs. Then, alter the namespace form to make it available to our project.

(ns hello-world
  (:require [clojure.tools.cli :as cli]
            ["express$default" :as express]))

You can start a server with the following code, but don't do that just yet. We need to take a brief detour.⁴

(.listen
  (doto (express)
        (.get "/" (fn [_ res] 
                    (.send "hello, world"))))
  default-port)

The reloaded workflow

If you are not familiar with the Clojure ecosystem, there is an idea made trendy by Stuart Sierra called "the reloaded workflow." Most large Clojure applications use it, and there are many libraries from which to select.

The basic idea is that it provides a way to quickly stop and start stateful resources without halting the main process. It's a necessity for a killer repl experience.⁵

After reviewing the options, I settled on weavejester/integrant because it's small - only one dependency and two source files in total.

Integrant isn't suitable for nbb in its current state, so I eliminated a couple of features, and now it works fine. View the GitHub project @crinklywrappr/integrant.

The shortlist of cut features:

• EDN configuration
• spec validation

It's npm for node dependencies and clj for Clojure dependencies.

$ classpath="$(clj -A:nbb -Spath -Sdeps '{:aliases {:nbb {:replace-deps {com.github.crinklywrappr/integrant {:git/tag "v1.0.3" :git/sha "8462388"}}}}}')"

$ nbb --classpath $classpath nrepl-server

Using Integrant with expressjs

First, let's define our handler.

(defn hello-world [count]
  (fn [_ res]
    (swap! count inc)
    (.send res (str "Hello, World! (count: " @count ")"))))

We will use count to simulate a database. We will count how many requests users have made to the server and restart the count at 0 whenever we start the server.⁶

The best place to start with Integrant is with a config map.

(ns hello-world
  (:require [integrant.core :as ig]
            ["express$default" :as express]
            [clojure.tools.cli :as cli]))

(def config
  {:express/server {:port default-port :app (ig/ref :express/app)}
   :express/app {:handler hello-world :count (ig/ref ::count)}
   ::count {:start 0}})

This config map is as simple as it looks. Each key-value pair refers to the configuration of a future stateful component. You specify dependencies with the (ig/ref <qualified-key>) function.

Next, we tell Integrant how to start everything up. This process is accomplished semi-declaratively with the ig/init-key multimethod. The first parameter is the key corresponding to the component, and the second parameter is a map of that component's config, replaced with all initialized dependencies.

(defmethod ig/init-key :express/app [_ {:keys [handler count]}]
  (doto (express)
    (.get "/" (handler count))))

(defmethod ig/init-key :express/server [_ {:keys [port app]}]
  (.listen app port))

(defmethod ig/init-key ::count [_ {:keys [start]}]
  (atom start))

Only the server needs to be closed. We can specify how to do that with the ig/halt-key! multimethod. Again, we are only interested in the second parameter, which is the server object. This function should be idempotent.

(defmethod ig/halt-key! :express/server [_ server]
  (when (and (some? server) (.-listening server))
    (.close server)))

Feel free to test this at the repl.

hello-world> (def system (ig/init config))

; now visit localhost:3000/ and refresh a few times

hello-world> (ig/halt! system)

If you found this section confusing, Let me encourage you to inspect system or peruse the 'canonical' Integrant README. Doing so will be very enlightening if you feel that I have glossed over some details.

Putting it all together

We will define a couple of start/stop functions to simplify the process of bringing the system up and down.

(def system (atom nil))

(defn start
  "system is an atom"
  ([] (start config))
  ([config] (start config system))
  ([config system] (reset! system (ig/init config))))

(defn stop
  "system is an atom"
  ([] (stop system))
  ([system]
   (when (map? @system)
     (swap! system ig/halt!))))

Finally, re-define start-app to call start with the (possibly) user-modified config.

(defn start-app [{:keys [port]}]
  (-> config
      (assoc-in [:express/server :port] port)
      start))

Congratulations! You now have a script suitable for command-line consumption and repl development.

hello-world> (start) ; or eg (start-app {:port 9093})
hello-world> (stop)

$ nbb --classpath $classpath --main hello-world/main --port 9093

Going one step further

You may notice that ctrl+c is required to stop the server from the command line. That's fine, but what if expressjs doesn't clean up after itself properly?

Maybe it does already: I'm no expert. But what if you switch to a different server that doesn't? It might be good to hook our stop function up to SIGINT.

(defn exit
  [& _]
  (stop)
  (.exit js/process 0))

(.on js/process "SIGINT" exit)

Happy hacking!

Closing thoughts about nbb

During this process, the only 'bug' I encountered was that I couldn't specify the request handler using partial, e.g. (partial hello-world count). To make it work, I returned a closure from hello-world. I'm not sure if this is an nbb problem or an expressjs problem.

I love nbb. Maybe even more than bb 😉. The biggest issue is the ergonomics around specifying Clojure dependencies and that it can't currently read jars. But I am hopeful both of those aspects will improve.

I don't think that will stop me from using it.

---

1. Emacs thinks it's a Clojure repl, but it's connected to an nbb server - we've confused it a bit. ↩

2. arguments isn't essential for us right now, but if you run (main "foobar"), you can see it in action. ↩

3. I later discovered the new clj build tool also does this. ↩

4. Most expressjs "hello, world" tutorials would stop here. ↩

5. In my experience, "Clojure" will automatically restart altered components on eval (and their dependents). I'm not sure which tool provides this feature (Cider, nrepl, something else...), and I didn't tempt fate to determine if that works with this approach. 😁 ↩

6. Using an actual database like SQLite would be a good learning step to do next. ↩

Comments

[Rob Culliton]

Cool post. Very helpful to see a bird's eye view of an integrant system and an express app!