DEV Community: Yoshiya Hinosawa

You can omit `run` from `deno run` command

Yoshiya Hinosawa — Wed, 18 Dec 2024 07:34:34 +0000

Since Deno version 1.46, you can omit run of deno run command.

You can execute hello world program like the below:

// hello.js
console.log("hello world");

$ deno hello.js
hello

Happy hacking!

Write AWS Lambda function in Deno

Yoshiya Hinosawa — Fri, 10 May 2019 02:59:58 +0000

Update(2019-12-01) @hayd Andy Hayden created deno-lambda which is a very good starting point for writing Lambda function in Deno. The repository is well maintained. I highly recommend you try that tool as well.

This guide describes how to write AWS Lambda function in deno at the time of this writing (deno v0.4.0). Many things could be improved later and you'll be able to skip some of these steps in future.

AWS Lambda supports Custom Runtimes. You can write your own runtime in any language and use it in AWS Lambda. In this guide, I'll show you how to write a custom runtime in deno and deploy it to AWS.

Prerequisites

This guide describes the 2 ways (Part 1 and Part 2) to create a lambda function in deno. In both cases, you need the followings:

AWS IAM user which can create a lambda function
AWS Role for lambda function (You need a role which has "AWSLambdaBasicExecutionRole" policy)
- In this article, I suppose it has the name arn:aws:iam::123456789012:role/lambda-role. Please replace it with your own one on your side.
AWS CLI installed

See the Prerequisites section of AWS Lambda Custom Runtime tutorial for more details.

Part 1. Do everything on your own

Build Custom Deno

The current official deno binary doesn't run on the operating system of Lambda because of the glibc compatibility issue. You need to build your own deno for it. What you need to do is to build deno in this image which is the exact image Lambda uses. (In addition, you need to set use_sysroot = false flag on .gn file. I don't understand this flag, but anyway it works. See the comment in the above issue if you're interested in details.)

If you want to avoid building deno on your own, please download the binary from here which I built based on a recent version of deno with the above settings. I confirmed this works in the Lambda environment.

Write Custom Runtime

You need to write a custom runtime in deno. A custom runtime is a program which is responsible for setting up the Lambda handler, fetching events from Lambda runtime API, invoking the handler, sending back the response to Lambda runtime API, etc. The entrypoint of a custom runtime have to be named bootstrap. The example of such program is like the below (This is Deno program wrapped by Bash script.)

#!/bin/sh
set -euo pipefail

SCRIPT_DIR=$(cd $(dirname $0); pwd)
HANDLER_NAME=$(echo "$_HANDLER" | cut -d. -f2)
HANDLER_FILE=$(echo "$_HANDLER" | cut -d. -f1)

echo "
import { $HANDLER_NAME } from '$LAMBDA_TASK_ROOT/$HANDLER_FILE.ts';
const API_ROOT =
  'http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/';
(async () => {
  while (true) {
    const next = await fetch(API_ROOT + 'next');
    const reqId = next.headers.get('Lambda-Runtime-Aws-Request-Id');
    const res = await $HANDLER_NAME(await next.json());
    await (await fetch(
      API_ROOT + reqId + '/response',
      {
        method: 'POST',
        body: JSON.stringify(res)
      }
    )).blob();
  }
})();
" > /tmp/runtime.ts
DENO_DIR=/tmp/deno_dir $SCRIPT_DIR/deno run --allow-net --allow-read /tmp/runtime.ts

import { $HANDLER_NAME } from '$LAMBDA_TASK_ROOT/$HANDLER_FILE.ts';

In this line, you import lambda function from the task directory. $LAMBDA_TASK_ROOT is given by Lambda environment. $HANDLER_NAME and $HANDLER_FILE are the first and second part of handler property of your lambda which you'll set through AWS CLI. If you set the handler property function.handler, for example, then the above line becomes import { handler } from '$LAMBDA_TASK_ROOT/function.ts'. So your lambda function need to be named function.ts and it needs to export handler as the handler in that case.

(async () => {
  while (true) {
    ...
  }
})();

This block creates the loop of event handling of Lambda. A single event is processed on each iteration of the loop.

    const next = await fetch(API_ROOT + 'next');
    const reqId = next.headers.get('Lambda-Runtime-Aws-Request-Id');

These 2 lines fetches the event from Lambda runtime API and stores the request id.

    const res = await $HANDLER_NAME(await next.json());

This line invokes the lambda handler with the given event payload and stores the result.

    await (await fetch(
      API_ROOT + reqId + '/response',
      {
        method: 'POST',
        body: JSON.stringify(res)
      }
    )).blob();

This line sends back the result to Lambda runtime API.

DENO_DIR=/tmp/deno_dir $SCRIPT_DIR/deno run --allow-net --allow-read /tmp/runtime.ts

This line starts the runtime script with net and read permissions. If you want to more permissions, you can add here the options you want. DENO_DIR=/tmp/deno_dir part is very important. Because Lambda environment doesn't allow you to write to the file system except /tmp, you need to set DENO_DIR somewhere under /tmp.

Write Lambda function

Now you need to write your lambda function in deno. The example looks like the below:

export async function handler(event) {
  return {
    statusCode: 200,
    body: JSON.stringify({
      version: Deno.version,
      build: Deno.build
    })
  };
}

This lambda function returns a simple object which contains status code 200 and deno's version information as body.

Deploy

Now you have 3 files deno, bootstrap (bash script), and function.ts (deno script). These are all files you need to run your Lambda function. You need to zip them:

$ zip function.zip deno bootstrap function.ts

Then you can deploy it like the below:

$ aws lambda create-function --function-name deno-func --zip-file fileb://function.zip --handler function.handler --runtime provided --role arn:aws:iam::123456789012:role/lambda-role

(Note: Replace arn:aws:iam::123456789012:role/lambda-role to your own role's arn.)

--runtime provided option means this lambda uses a custom runtime.

Test

You can invoke the above lambda like the below:

$ aws lambda invoke --function-name deno-func response.json
{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}
$ cat response.json
{"statusCode":200,"body":"{\"version\":{\"deno\":\"0.4.0\",...}}}

Part 2. Use the shared layer

AWS Supports the Lambda Layer. A lambda layer is a ZIP archive that contains libraries, a custom runtime, or other dependencies. I published the above deno binary and bootstrap script as a public layer. You can reuse it as a custom deno runtime.

In this case, what you need to do is just to write a lambda function in deno and deploy it to AWS.

Create Deno Lambda Function using Public Deno Runtime

An example function.ts looks like the below (The same as the above):

export async function handler(event) {
  return {
    statusCode: 200,
    body: JSON.stringify({
      version: Deno.version,
      build: Deno.build
    })
  };
}

Then zip it and deploy it:

$ zip function-only.zip function.ts
$ aws lambda create-function --function-name deno-func-only --layers arn:aws:lambda:ap-northeast-1:439362156346:layer:deno-runtime:13 --zip-file fileb://function-only.zip --handler function.handler --runtime provided --role arn:aws:iam::123456789012:role/lambda-role

(Note: Replace arn:aws:iam::123456789012:role/lambda-role to your own role's arn.)

Where the arn arn:aws:lambda:ap-northeast-1:439362156346:layer:deno-runtime:13 is a public lambda layer which implements deno runtime. The --layers arn:aws:lambda:ap-northeast-1:439362156346:layer:deno-runtime:13 option specifies this lambda function uses it as the shared layer.

Test it

You should be able to invoke the above lambda function like the below:

$ aws lambda invoke --function-name deno-func-only response.json
{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}
$ cat response.json
{"statusCode":200,"body":"{\"version\":{\"deno\":\"0.4.0\",...}}}

That's it. Thank you for reading.

References

All examples are available in this repository.

Markdown-based task runner - 'saku'

Yoshiya Hinosawa — Sun, 13 May 2018 07:10:23 +0000

tl;dr

It's painful to use make command as a task runner. It's especially intolerable when writing .PHONY directives for evey task.
I found some simple altenatives (like go-task, robo), but I didn't like them.
I came up with the idea that markdown seems a better syntax for task definition.
I made a markdown-based task runner, named saku!

Motivation

Because I've been using JavaScript both at work and hobby always, I use npm run-script as my task runner. run-script has some flaws such that it's unable to be documented well in a decent way, or that it's a little hard to read because it's written in JSON file. But in my opinion, it's sufficient tool for defining the necessary tasks with the support of the tools like npm-run-all, cross-env etc.

One day I started using terraform for our team's infrastructure management and I was setting up the git repository for it.

I was searching the web for how to set up the terraform project and many people seem using Makefile for organizing the commands in it.

I thought it was a good idea to use task runner to invoke terraform because terraform is too long to type, but I also felt uncomfortable about using Makefile in 2018. It's depressing to write enigmatic .PHONY directive again and again. In the first place make isn't a tool for running tasks, but a tool for compiling source code of the language like C, C++ etc. It is a hack to use make as a task runner and I thought there must be some other tools which are more appropriate for just running tasks in 2018.

go-task

I found several tools. Among them, go-task seemed the closest to what I was looking for. go-task is a simple tool. It uses the yaml format for task configuration.

However I felt something was wrong about it. That was the key names in the yaml DSL. In go-task, you write commands and descripions as values of keys cmds and desc.

tasks:
  build:
    cmds:
      - go build -v -i main.go

  assets:
    cmds:
      - minify -o public/style.css src/css

To my eyes, these key names look a little redundant and seem noisy. In Makefile, targets are expressed as their names and colons and the commands are expressed by tab indents.

build:
    go build -v -i main.go

assets:
    minify -o public/style.css src/css

That was more elegant. Because I'm looking for more elegant replacement of make, it's so unfortunate if the notations get less elegant.

Think syntax

I considered whether it's possible or not to build more simpler version of configuration in yaml syntax, but I gave up because it seemed impossible because of the characteristics of yaml. In yaml, if you need to define something, you always need key: value pair for anything and I felt it's impossible to get a simpler version than go-task's design.

Then I thought about several languages which I was familiar with and finally I found markdown is very much suitable for my needs because it allows bare strings in many places in different contexts.

Markdown-based task runner

I couldn't find markdown-based task runner and therefore I created one. I named it saku!

I used remark as markdown parser and used minimist as CLI option parser. I reused several toolchains for building CLI base structure which I had prepared for writing some other tools like kocha or bulbo. I stole some techniques from yarn to show emojis only in TTY environment.

I modeled the domain of saku according to what DDD book teaches me. I like this style of progarmming these days quite a lot. Modeling makes the software development quite straightforward and easy for reasoning and testing. In the course of writing suku, I created the only one model class Task and the entire implementation seemed very organized ☺️.

🐕 Dogfooding

I started dogfooding it at 12th commit by writing down the tasks in saku.md which are necessary to build saku itself.

# test
> Runs unit tests

    npx kocha --require src/__tests__/helper src{/,/**/}__tests__/*.js

# lint
> Runs lint checks

    npx standard

# fix
> Runs auto fixer of lint tool

    npx standard --fix

# cov
> Makes coverage reports

    npx nyc --reporter=text-summary --reporter=lcov saku test

# codecov
> Posts reports to codecov.io

    saku cov
    npx codecov

As I expected, I felt it was easy to write, and unexpectedly I found that the rendering of the saku.md in github UI was so beautiful).

Wrap up

It's so fun to create things from scratch! 😄
Though markdown doesn't have a natural logical structure like yaml, personally I think it has a great potential as DSL because it has various places where bare strings can be written without quotes.

Update: 2018-05-12

I've completely rewritten saku in go language. Please check this version as well 😉