nbb and lambda function URL: turn a boring task into a piece of cake.

Hi 👋, Eugene is here.
A few weeks ago I ran into the talk given by Michiel Borkent about nbb - a tool for ad-hoc CLJS scripting on Node.js.
I truly liked his presentation, so I went away inspired and ready to transition new knowledge from theory to practice. So if you are curious to know what I’ve ended up with, let’s jump in.

In my team at Hyde-Housing we promote the culture of experimentation. It creates a great opportunity for trying out new things in non-critical project areas with further internal demos and discussions about whether we should adopt innovations wider or call it a day.
For such experiments, I often try to choose either something with a clear outcome or something that I've implemented many times. One of our internal services uses a Node.js lambda that generates presigned URLs for file uploads. Quite boring standard thing. Sounds like a perfect candidate for rewriting, doesn't it?

The project uses AWS SDK for JavaScript v3 that has a new modular architecture. So the package.json includes two libraries @aws-sdk/client-s3 and @aws-sdk/s3-presigned-post to import an S3 client and createPresignedPost method that generates a presigned url. Also we need to add nbb as a dependency. The full package.json file looks like:

{
   "dependencies": {
       "@aws-sdk/client-s3": "^3.67.0",
       "@aws-sdk/s3-presigned-post": "^3.67.0",
       "nbb": "^0.3.4"
   }
}

nbb allows you to develop in the same REPL-driven manner which is a usual workflow for clojure programmers. That’s fantastic, isn’t it?
Though the lambda code is relatively straightforward, I still would like to give some explanations. The link to the full project can be found at the end of the post.

(ns handler
 (:require ["@aws-sdk/client-s3" :refer [S3Client]]
           ["@aws-sdk/s3-presigned-post" :refer [createPresignedPost]]
           [clojure.string :as s]
           [goog.string.format]
           [applied-science.js-interop :as j]
           [promesa.core :as p]))

(def s3 (S3Client. #js{:region "eu-west-1"}))

(def bucket-name-template "%s-docs-upload-bucket")

(defn handler [event _ctx]
     (p/let [{:keys [env folderName fileName]} (-> event
                                                   (j/get :pathParameters)
                                                   (j/lookup))
             key (str (s/replace folderName #"_" "/") "/" fileName)
             bucket-name (goog.string/format bucket-name-template env)
             response (createPresignedPost s3 (clj->js {:Bucket     bucket-name
                                                        :Key        key
                                                        :Expires    300
                                                        :Fields     {:key key}
                                                        :Conditions [["eq", "$key", key]
                                                                     ["content-length-range", 0, 10485760]
                                                                     ["starts-with", "$Content-Type", "text/"]]}))]
            (clj->js {:statusCode 200
                      :headers    {"Access-Control-Allow-Origin"  "*"
                                   "Access-Control-Allow-Headers" "Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent,Access-Control-Allow-Origin",
                                   "Access-Control-Allow-Methods" "OPTIONS,GET"}
                      :body       (js/JSON.stringify response nil 2)})))

#js {:handler handler}

• careful reader might ask why some of the the dependencies in the :require have double quotes while others do not? This convention came from the shadow-clj and adopted by nbb to denote an npm library;
• the p/let - a neat way of chaining promises: each expression will be treated as a promise expression and will be executed sequentially;
• the lambda gets the environment, folder and filename parameters from the incoming API Gateway event. Those are needed to understand which bucket and folder the upload is going to happen to. j/lookup method from the js-interop library is a quick and handy way that allows you to use the clojure destructuring;
• according to the business logic S3 folders may be nested. The folderName parameter uses underscore to reflect the nesting nature, f.e. "folder1_folder2_folder3" should be converted to "folder1/folder2/folder3";
• when all required parameters are prepared the signing bit comes into play: the createPresignedPost method requires the s3 client instance and the options object;
• clj->js does exactly what it says on the tin - recursively transforms cljs values to javascript;
• the returned response consists of url and fields that we need to pass on to the caller. They will use it for the actual upload;
• again using clj->js to assemble a lambda function response that consists of a status, headers (for the sake of simplicity "Access-Control-Allow-Origin" is configured to "allow all", however it's recommended to specify the exact origin) and body;

There's one file that needs to be added - index.mjs - an ES6 module that’s required for Node.js applications.

Please note that promesa and js-interop are built-in libraries so you can use them straightaway!
The whole project and deployment instructions can be found at this github repo.

Recently AWS announced Lambda Function URLs support. This feature allows configuring an https endpoint to the AWS lambda. It is a huge improvement for simple use cases where you don’t need the advanced API Gateway functionality.
In order to benefit from it we need slightly tweak our stack declaration and the handler because the incoming event structure has changed to something like:

{
 "version": "2.0",
 "routeKey": "$default",
 "rawPath": "/dev/folder1/report1.csv",
 "rawQueryString": "",
 "headers": {
   "x-amzn-trace-id": "Root=1-4a40e828-b893-47b2-937c-19623c4f88e4",
   "x-forwarded-proto": "https",
   "host": "kafts7qpofkzbbvxbxxzavlv6i0aelqr.lambda-url.eu-west-1.on.aws",
   "x-forwarded-port": "443",
   "x-forwarded-for": "aeaf:c432:4e41:50ee:060d:fc8d:5d42:df65",
   "accept": "*/*",
   "user-agent": "curl/7.64.1"
 },
 "requestContext": {
   "routeKey": "$default",
   "stage": "$default",
   "time": "11/Apr/2022:07:54:29 +0000",
   "domainPrefix": "kafts7qpofkzbbvxbxxzavlv6i0aelqr",
   "requestId": "cd14fcd8-ff00-4fd9-a13a-3bc772f038ea",
   "domainName": "kafts7qpofkzbbvxbxxzavlv6i0aelqr.lambda-url.eu-west-1.on.aws",
   "http": {
     "method": "GET",
     "path": "/dev/folder1/report1.csv",
     "protocol": "HTTP/1.1",
     "sourceIp": "aeaf:c432:4e41:50ee:060d:fc8d:5d42:df65",
     "userAgent": "curl/7.64.1"
   },
   "accountId": "anonymous",
   "apiId": "kafts7qpofkzbbvxbxxzavlv6i0aelqr",
   "timeEpoch": 1649663669499
 },
 "isBase64Encoded": false
}

This is how we obtain environment, folder and filename now:

(defn handler [event _ctx]
     (p/let [[env folderName fileName] (-> event
                                           (j/get-in [:requestContext :http :path])
                                           (s/split #"/")
                                           (rest))
...

The rest of the handler's implementation remained the same.

Thank you for reading 🙏, I hope you found it useful.
Please let me know if you have any questions, suggestions and feedback.