DEV Community: Alexandre

I published a book - Deno Web Development 🦕

Alexandre — Thu, 01 Apr 2021 11:42:01 +0000

As some of you might know, I've been working on something for the last six months -- a book!

Presenting Deno Web Development, a book about Deno, a secure runtime for JavaScript and TypeScript, and how to use it for web development.

The book focuses on how you can write, test, maintain, and deploy JavaScript and TypeScript web applications using Deno.

The approach

It is a guided journey discovering this new runtime, understanding the reasons for its creation, what problems it solves, and how to use it to build a real-world application.

The book starts with "hello world" and finishes with an API deployed in a cloud environment. In the meantime, it covers topics such as dependency and configuration management, connecting with the database, structuring an application, testing, among many others.

What to expect

By the end of "Deno Web Development", the reader will have all the knowledge to decide whether to use Deno in their next project.

As the book progresses, the reader will get to know the ups and downsides of the technology, not only because they're listed there, but also because they'll experience them as they build an application from scratch.

The audience

I tried to make the book approachable as an introduction to the technology but audacious enough to include many engineering best practices.

"Deno Web Development" targets people with JavaScript knowledge that want to know more about Deno, understanding how to use it to solve common industry challenges.

Special thanks

Publishing the book wouldn't be possible without the opportunity offered to me by Packt

The same applies to my friends: Felipe, Gonçalo, Bruno, Nuno, János, João, and Miguel. Writing this book wouldn't be possible without your help. Thank you for always being available to discuss a variety of topics and providing meaningful feedback. You're the best! 🙏

And of course, my family and friends for how they encouraged me to follow this "adventure" of joining two of my passions: technology and writing, and writing a book.

Where can I buy it?

The book is now available globally, on Amazon in Kindle and paperback versions.

I'd be happy to hear any feedback you have about it, so feel free to reach out to me if you have any questions about it.

If you're curious, I'm thinking of writing a little more on how was the experience of writing the book in the following weeks, adding to what I already said on a previous interview.

Thank you

Second adventure in deno land

Alexandre — Tue, 13 Oct 2020 00:01:00 +0000

On my last post, I wrote about my first adventure in deno.land.

It was a fun one, the excitement of trying some new technology was always there. deno left me thinking about new possibilities and asking myself what am I going to build with it.

Last time around I have built a small twitter bot in order to scratch the surface of the standard library and to get to know deno a little better in deeper context than a simple "hello world".

It was very well-received, way more than what I was expecting. It ended up being, at the time, my most read and reacted post ever. I guess I got quite lucky on the "deno hype train".

Back to what brought me here today. After exploring the standard library, module imports, simple permission system and dependency management, today I'll explore the following topics:

Lock files
Official VSCode extension
Documentation generation
Permission system
Running deno code in the browser
Testing

If you want to follow the code, here you have it.

Lock files

To complete the information from the last post about dependencies, I'll write about lock files. They're a standard practice in many languages in pretty much every production app. They're used to describe an exact tree of dependencies, in order to make installations more repeatable, avoiding issues that may arise out of version misalignement.

In deno you can generate a lock file for the used dependencies by running

deno cache --lock=lock.json --lock-write ./src/deps

This command will cache (that local installs the used dependencies) based on a lock.json file. The --lock-write flag updates or creates the lock.json file. The last parameter is the file that uses the dependencies.

To install the dependencies while integrity checking every installed resource, one can run:

$ deno cache -r --lock=lock.json deps.ts

The generated file is nothing more than a json object of dependencies and its checksum

Official VSCode extension

The official vscode extension has been launched! However, it is the exact same that I have mentioned on my previous post. It simply got moved to the official repo, as the changelog states

Moved from https://github.com/justjavac/vscode-deno to https://github.com/denoland/vscode_deno in order to have an "official" Deno plugin.

It works very well, autocompletes and files imports are fine, as expected. There's a small problem though, when you cmd + click on external dependencies, it does not detect the language, so the file appears without any highlighting.

I'm sure it will be fixed soon but it's also a good oportunity for contribution that I might take, after I get the time to understand the code from vscode and the plugin itself.

Documentation

Another of the advantages presented by Ryan in his talk was that deno included a documentation generator on its toolchain. It doesn't have (yet) a section on the website, but we'll explore it a bit here.

Browse modules documentation

Even though deno has no install step, the cache lets you develop in an airplane (as you did with node_modules), as it loads the modules the first time and then uses the cached one.

While working on an airplane, what if you want to look at third party code docs? You can use your editor of choice, yes, but there's an alterantive.

deno provides a cool way to see the third party code documentation without having to browse the code.

$ deno doc https://deno.land/std/http/server.ts

This outputs the methods exposed by the standard library's http server.

function listenAndServe(addr: string | HTTPOptions, handler: (req: ServerRequest) => void): Promise<void>
  Start an HTTP server with given options and request handler

function listenAndServeTLS(options: HTTPSOptions, handler: (req: ServerRequest) => void): Promise<void>
  Start an HTTPS server with given options and request handler

function serve(addr: string | HTTPOptions): Server
  Create a HTTP server

function serveTLS(options: HTTPSOptions): Server
  Create an HTTPS server with given options

class Server implements AsyncIterable

class ServerRequest

interface Response
  Interface of HTTP server response. If body is a Reader, response would be chunked. If body is a string, it would be UTF-8 encoded by default.

type HTTPOptions
  Options for creating an HTTP server.

type HTTPSOptions
  Options for creating an HTTPS server.

Very neat, right? A very nice way of having an overview of the modules exported symbols.

To see the documentation for a specific symbol, one can also run.

$ deno doc https://deno.land/std/http/server.ts listenAndServe

Which outputs

function listenAndServe(addr: string | HTTPOptions, handler: (req: ServerRequest) => void): Promise<void>
    Start an HTTP server with given options and request handler
        const body = "Hello World\n";     const options = { port: 8000 };     listenAndServe(options, (req) => {       req.respond({ body });     });
    @param options Server configuration @param handler Request handler

You can use the documentation command to have an overview of your code's exported modules, it works the same way.

$ deno doc twitter/client.ts

Output:

const search
  Searches for the recent tweets of the provided username that have more than 5 likes

interface Tweet
  Fields in a tweet

interface TweetResponse
  The response from Twitter API

The --json flag is also supported (however, not for symbols), and allows generating the documentation in the json format, enabling programmatic uses.

One great example of the documentation generation uses is deno runtime API.

It uses Deno to generate modules with the --json command and provides a really nice layout around it. We just needed to add typedoc to our modules, like what we did here.

Fine grained permissions

As we talked on my previous post, one thing that deno got very well were permissions. They're easy to use and secure by default. Previously, I've explained that in order for a script to be able to access the network, for instance, you'd have to explicitly use --allow-net flag when running it.

That is true, however, I was alerted by my friend Felipe Schmitt that in order for it to be stricter, we can use:

deno run --allow-net=api.twitter.com,0.0.0.0 index.ts

This will, as you probably guessed, allow network calls to api.twitter.com and from 0.0.0.0 but disallow all the other calls. Instead of allowing complete access to network, we're allowing just part of it, whitelisting and blocking everything else by default.

This is now very well explained on the Permissions page, which is one of the documentation improvements that were added after the v.1.0.0 launch.

Running code in the browser

Another very interesting feature of deno, is the bundle command.

It allows us to bundle your code into a single .js file. That file can be run as any other deno program, with deno run.

What I find interesting is that the generated code, when it doesn't use the Deno namespace, is that it can run on the browser.

The possibilities for this are limitless. For instance, what if I wanted my API to generate an HTTP client for frontends to interact with it?

We can write that client in deno, reusing API code (and types). Here's the code to get the popular tweets.

import { TweetResponse } from "../twitter/client.ts"

export function popular(handle: string): Promise<TweetResponse> {
  return fetch(`http://localhost:8080/popular/${handle}`)
    .then(res => res.json())
    .catch(console.error)
}

Code

This code lives on the API codebase and it is written in deno. It uses the same types from the twitter client the API uses.

Having the API client living on the API codebase means that whoever updates the API can also update the client, abstracting the backend and API changes from the frontend code. This is not a deno feature but something that it enables via this bundling feature.

Then, we can run the bundle command and put the generated file in a folder.

$ deno bundle client/index.ts public/client.js

It will generate the client.js file that can run on the browser.

For demonstration purposes we can create a public/index.html file with the following code.

<script type="module">
  // Imports the generated client
  import * as client from "./client.js"

  async function fetchTwitter(event) {
    // Uses the methods on it
    const result = await client.popular(event.target.value)

    /*
      Omitted for brevity
    */
  }
</script>

Code

This code uses client that was initially written in deno, and is now a js file.

Now, this public folder can be served by any webserver. Since we're talking about deno, we can take advantage of it and serve the files with standard library's file server.

# inside the public folder
$ deno run --allow-net --allow-read https://deno.land/std/http/file_server.ts

We can now visit http://0.0.0.0:4507/ and use our very archaic frontend to query for popular tweets. With this, our code can use the client that was originally written in deno on the frontend, to interact with the API.

Testing

Following deno's goal of shipping the essentials in a single binary, testing is obviously included. The documentation is, again, quite good on this. With the help of the doc command, it gets very straight-forward to write tests.

$ deno doc https://deno.land/std/testing/asserts.ts

This prints the documentation for the assert module from deno standard library

function assert(expr: unknown, msg): <UNIMPLEMENTED>
  Make an assertion, if not `true`, then throw.

function assertEquals(actual: unknown, expected: unknown, msg?: string): void
  Make an assertion that `actual` and `expected` are equal, deeply. If not deeply equal, then throw.

function assertMatch(actual: string, expected: RegExp, msg?: string): void
  Make an assertion that `actual` match RegExp `expected`. If not then thrown

/* Cut for brevity */

function fail(msg?: string): void
  Forcefully throws a failed assertion

Let's write a simple test to our "isomorphic" client we created a few lines above.

Deno.test will be used to declare the test body, assertEquals will be called to make the assertion. In order to mock the server responses, we'll spin up a server using standard-library http_server.

We expect that, the popular method from the client calls /popular/:twitterHandle. Let's create a test for that.

import * as ApiClient from "./index.ts"
import { assertEquals } from "../deps.ts"
import { runServer } from "../util.ts"

Deno.test("calls the correct url", async () => {
  runServer(async req => {
    assertEquals(req.url, "/popular/ampsantos0")
    await req.respond({ body: JSON.stringify({ statuses: [] }) })
  })

  await ApiClient.popular("ampsantos0")
})

Code

Our runServer util is a very simple function that just spawns a web server that closes after the first connection.

import { serve, ServerRequest } from "./deps.ts"

export const runServer = async (
  handler: (req: ServerRequest) => Promise<any>
) => {
  const server = serve(":8080")

  for await (const req of server) {
    await handler(req)
    server.close()
  }
}

Code

We can now run the test. Remember that everytime we run a deno program we need to pass the permission flags.

It will run as test every file matching the following regex {*_,*.,}test.{js,ts,jsx,tsx}:.

TLDR: It runs every file that has test in its name and ends with one of the mentioned file extensions.

$ deno test --allow-net

If the test is failing, we're presented with a nice diff.

test calls the correct url ... error: Uncaught AssertionError: Values are not equal:

    [Diff] Actual / Expected


-   "/popular/ampsantos0"
+   "/popular/handle-that-doesnt-work"

  throw new AssertionError(message);
        ^
    at assertEquals (https://deno.land/std/testing/asserts.ts:170:9)
    at file:///Users/alexandre/dev/personal/deno/testing-deno/client/index.test.ts:7:5
    at runServer (file:///Users/alexandre/dev/personal/deno/testing-deno/util.ts:9:11)

In order to run just part of the tests, the --filter flag can be used.

$ deno test --allow-net --filter="correct url"

And a path can also be sent as the last argument

$ deno test --allow-net ./client

You can follow all the listed changes related to tests on this commit.

And with this, we have pretty much everything we need in order to write compreensive tests. Again, the essentials are there, we can (and probably will) create a couple more test utilities composing on top of the standard library, but for simple tests as the one we just wrote, this is enough.

Conclusion

On our second adventure, we went a little further than just presenting the language.

We explored some other parts of the runtime. From testing, generating documentation, lock files, stricter permissions, to generating javascript code and running it on the client. All of them are features we considered useful when writing production code. We let benchmarks out for the next adventure 😉.

We can't forget that we did all of this with the toolset that is included in deno, no libraries were used.

This doesn't mean that developers will not use and write libraries. It means though that the standard library is very well written and easy to use. This opens the door for developers to write powerful, and meaningful abstractions on top of it.

Together with the standard library, the toolchain offered with the main binary proves itself very complete, aligned the goal of Only ship a single executable, mentioned on deno docs. I'd say all the essentials are there and you don't have to worry about tooling.

This adventure took a little longer that expected. Thanks for reading it.

I hope it was as fun reading as it was for me exploring it. I'd love to hear what you have to say about it and answer all the questions you might have.

Adventures in deno land

Alexandre — Mon, 12 Oct 2020 23:57:56 +0000

Earlier this year deno was released.

As I was very excited ever since I first heard about it on Ryan Dahl's talk at jsconf, I had to give it a try.

This talk is one of my personal favorites, it is a lesson on humility.
Having Ryan looking at what he built 10 years ago with a criticizing tone is interesting. Even when node is used by millions of people, its creator still feels bad about some decisions made at the time.

Getting back to what brought me here... After hearing of the launch of v1.0 I took some hours to learn more about it. The documentation is very well written and structured, which by following what they call the manual one could have a very good understanding of how to start using it.

Building something

After reading the documentation, it looked great, in theory. But my default way to learn is normally to build something with it. It normally helps me identify pains I'd have in the real world if I had to build a real application with it.

The decision was to build an API that connects to twitter and returns 15 tweets from a user with more than 5 likes, I called it popular tweets. This small server should then run on a Kubernetes environment.

If you wanna follow the code, here you have it

At first, I was kinda lost and didn't know any APIs. I've decided to go explore the standard library. I got very impressed by how approachable was the code, took some time to read it, and learned a ton.

It got this idea on the back of my mind, which might lead to a future article, similar to what Paul Irish did 10 years ago on 10 things I learned from the jquery source but for deno source, might actually do it!

After getting to know the basics, installing the VSCode plugin and deno, we were ready to start my adventure.

To be honest, it wasn't a real adventure, everything looked so familiar that I almost forgot I was using a different runtime.

Getting to code

By using the standard library's http server it was very easy to build a server and get it up running handling requests.

import { serve } from "./deps.ts";

const s = serve({ port: 8080 });

for await (const req of s) {
  req.respond({
    status: 200,
    body: "Hello world",
  });
}

Step 2 was to connect it to twitter API. Having fetch already included on deno made it very easy and familiar.

fetch(
  "https://api.twitter.com/1.1/search/tweets.json&q=(from: ampsantos0 min_faves: 5)",
  { headers: new Headers([["content-type", "application/json"]]) }
)

Deno opted for mimicking existing Web APIs where they existed, rather than inventing new proprietary ones. For APIs that are not web standard, the Deno namespace is used. This looks like a smart choice to me, improving discoverability and reusing knowledge developers already have of the existing APIs.

Running it

Running the code was a breeze. One of deno's selling points is security and I couldn't agree more, it improved over node. You notice it the first time you try to run a program:

$ deno run ./index.ts

Once we are, in this case, using network to both expose our endpoint (:8080) and access Twitter's API without our explicit consent, here's what you get:

error: Uncaught PermissionDenied: network access to "0.0.0.0:8080", run again with the --allow-net flag
    at unwrapResponse ($deno$/ops/dispatch_json.ts:43:11)
    at Object.sendSync ($deno$/ops/dispatch_json.ts:72:10)
    at Object.listen ($deno$/ops/net.ts:51:10)
    at listen ($deno$/net.ts:152:22)

This is a very reasonable and comprehensive error, again, good job on this!

A good approach for this is by enabling whitelist permissions by using the flag --allow-net which deno does it in a very simple and pragmatic way:

$ deno run --allow-net=0.0.0.0:8080,api.twitter.com index.ts

When running the code, the --inspect flag enables developers to use Chrome Dev Tools the same way they did in node, the debugging experience is as good as developers are used to.

Module resolution

When Ryan first talked about deno, and the mistakes made in node's design, one of the big things he mentioned that node's way of importing modules was too complicated and had lots of edge cases.

Example:

const path = require("path")

The dependency we're importing, path might come from node standard library. At the same time, it can come from node-modules, or you could have installed a dependency named path, right? Ok, now you found the dependency, do you know what is the file you are requiring? Is it index.js? What if package.json has a different main file defined?

Lots of unknowns...

What about local imports? When you do:

const add1 = require("./utils/math")

Is math a file? Or a folder with an index.js inside of it? What is the file extension? Is it .js, .ts?

You get the point... Node imports are hard.

Deno follows a golang like approach, of having absolute urls. If it sounds strange to you, bare with me. Let's look at the advantages:

It solves local imports by adding the extension to it.

import { add1 } from "./utils/math.ts"

You know just from reading it that math.ts is a file.

It solves third party imports by having an absolute URL

import { serve } from "https://deno.land/std/http/server.ts"

No more magic module resolution.

This absolute module resolution enabled some fun stuff like what R. Alex Anderson did, running code from a set of gists.

https://twitter.com/ralex1993/status/1261039838100221952

Note: VSCode plugin functions well with the third party imports, you can cmd+click on dependency and you're directed to the code, as usual.

Keeping track of dependencies

Let's talk about managing dependencies. As deno simplified the module imports, it allowed it to automatically cache dependencies.

When you first try to run it, it downloads the dependencies, caches them, and then runs with the cached version.

To force the caching of a module without running it, you can run $ deno cache [module url].

You are probably thinking it is strange and error-prone to URLs all around the code? That's right. You can manage it however you want, as all modules have absolute URLs now, it's just code at the end of the day.

Deno recommends having a deps.ts file, you can call it whatever you want but since it is in the documentation, I see this start becoming a standard. On that file, you can import all the dependencies from the URLs and export the methods used.

// deps.ts
export { serve } from "https://deno.land/std/http/server.ts"
export { parseDate } from "https://deno.land/std/datetime/mod.ts"

// index.ts
import { serve } from "./deps.ts"

Having one single deps.ts file allows you to do some caching (as you did in package.json) on docker builds.

COPY deps.ts .
RUN deno cache deps.ts

By doing this, the RUN command will only run if the deps.ts file changed. With this, and as the installation step is now automatic, running it on docker became simpler.

There is one thing that has to be taken care of with deno, we have to send the flags for the permissions.

CMD ["run", "--allow-net", "index.ts"]

Deno binaries

Deno provides an install command. But, as I said earlier, it does not install dependencies on the project, as that is done automatically.

Its usage is similar to the npm install --global flag, citing the explanation on the official website about install:

This command creates a thin, executable shell script which invokes deno using the specified CLI flags and main module. It is placed in the installation root's bin directory.

When you install a global binary, you have to specify what permissions it will need to run, again, secure by default.

$ deno install --allow-net --allow-read https://deno.land/std/http/file_server.ts

And you can then run $ file_server

Conclusion

Coming from the JS/TS world I'd say deno got lots of things right. It has the familiarity of JS and TS with small twists, to the better side. Having the standard library written in TS is a big plus, at it isn't always straightforward to set it up in node.

The standard library is great, it looks both readable and well thought. Quoting the deno_std main repo:

deno_std is a loose port of Go's standard library. When in doubt, simply port Go's source code, documentation, and tests.

This is funny and interesting at the same time, deno used the effort the golang community put in its standard lib to drive its own, the result looks great.

The permission system is great and intuitive. Module resolution is now simpler and removes pretty much all the magic we got used to in node lands.
All the async APIs return Promises now. It means using await and .then everywhere, not incurring into callback hell and not needing tools like promisify and such.

Adding to all of this, deno also got inspiration from golang by shipping a lot of the essential tools in the main binary. Discussions about bundler, formatter, and test runner will no longer be a thing, and even if they are, there's an official way now. I haven't tried the test suite and the documentation generator yet, I might write about it later.

Gotta say the overall experience of building a (very small) application with it was very good and intuitive. Can't wait to build something more real with it!

I'm excited to see how this will evolve and thus I wrote another article, my second adventure in deno.land where I go a little deeper on the runtime.

Using mocked APIs to increase developer productivity

Alexandre — Wed, 22 May 2019 15:57:49 +0000

When we’re developing a frontend, we just might be lucky enough to have a full-fledged API, although in the real world, for the most time, we’ll get ourselves into a position where we have to develop without an API is available with all the functionality we require in the future.

There are a few tools that enable us to minimize this dependency, I remember ember has the great ember-cli-mirage. It provides the functionality you need, deeply integrated with ember and makes you develop by default with a completely mocked API, which I think is a great principle to start with.
It is also great to be able to run the application without depending on external servers (imagine on a plane, on a trip without internet, or just on a computer with low memory).
You do not want your UI developers to run 3 servers just to develop a button.

Usecase

The last days we were faced with a similar issue. We're developing a screen, we had the API signature, but the API was not ready yet.

One alternative would be to just have the payload somewhere hardcoded, while we were waiting for the finished API.
Sure, we can develop your button with that but then it is not possible to implement the part of the application that does the requests. We also wanted, in future, to spend as little effort as possible to change from the mocked data to a real API.

That solution did not look right, especially when we still remember ember-cli-mirage. Unfortunately, we can't use ember-cli-mirage with other libraries, even though we really miss the functionality it provides.

The solution we ended up with was a combination of json-server with route-parser and using axios request interceptors.

In our http-client we had calls to axios, so, the first thing we did was to add this line.

// httpClient.js
import mockApiRequestInterceptor from './mockApiRequestInterceptor';

axios.interceptors.request.use(mockApiRequestInterceptor);
// Please keep in mind that if you use axios instance, 
// you have to do it on the instance

And we've created the request interceptor

// mockApiRequestInterceptor.js
import UrlPattern from 'route-parser';
import { API_BASE_URL } from 'appConfig';

const MOCK_API_BASE_URL = 'http://localhost:3001/api';
const MOCK_URLS = [
  `${API_BASE_URL}/:user/posts/:id`
];

export default (config) => {
    if (!IS_DEVELOPMENT) {
        return config;
    }

    const matchedUrl = MOCK_URLS.find(mockUrl => {
        const pattern = new UrlPattern(mockUrl);
        return pattern.match(config.url);
    });

    if (matchedUrl) {
        config.url = config.url.replace(API_BASE_URL, MOCK_API_BASE_URL);
    }

    return config;
};

The mocked URLs would be added to MOCK_URLS, and if there is a mocked version of them they will be redirected.

So, this code would redirect requests made from a URL that is https://mycrazyserver.com/api/asantos00/posts/123 to http://localhost:3001/api/asantos00/posts/123, and part of the problem was solved.

After this, we took advantage of another great package json-server (TLDR: Serves JSON files through REST routes).

We've created an npm script that runs this json-server with a few configurations (I will explain them later).

// package.json
{
    "scripts": {
        "mock-api:serve": "json-server --watch mock-api/db.json --port 3001 --routes mock-api/routes.json"
    }
}

And created a db.json file

// mock-api/db.json
{
    "posts": [
        {
            "id": 1,
            "title": "Mocking an API with axios",
            "author": "asantos00",
            "createdAt": 1557937282,
            "body": "Lorem ipsum dolor sit amet, consectetur."
        },
        {
            "id": 2,
            "title": "Whatever post",
            "author": "asantos00",
            "createdAt": 758851200,
            "body": "Lorem ipsum dolor sit amet, consectetur."
        }
    ]
}

It automatically creates rest endpoints for every entity on the db.json file and watches the db file for changes.

As we said before, json-server creates REST endpoints for entities, this means that after we have the entity post on the db.json, the following endpoints are created:

GET /posts - Returns the list of posts
GET /posts/:id - Returns post with the sent id
POST /posts - Creates a post
PUT /posts/:id - Replaces the post with the sent id

The file to create custom routes is routes.json.

// mock-api/routes.json
{
  "/api/*": "/$1",
  "/:user/posts/:id": "/posts?user=:user"
}

The routes file only says that whatever request comes to /:user/posts/:id, we redirect to the automatically created endpoint that is /posts. And yeah,json-server also supports filtering, we use the query param user to do it here.

It's done! Now, our app will continue making the requests to the endpoints that are implemented. But we can keep developing this page with the mocked API.
As soon as we want to use the real API, it is just a matter of removing the url from MOCK_URLS (which can also be extracted to a separate file) in mockApiRequestInterceptor file.

Conclusion

In the end, this solution really improves the process, helping us not being concerned about backend endpoints being done or not, while implementing all the cases in a closer to real environment.
Another nice thing about this is that it is also library agnostic as it only depends on the http-client so you can use it from react to backbone.js or even to react-native.

None of this would be possible without the amazing open source packages I've used, it was just a matter of getting the pieces together for the explained use case.

What solutions do you use to solve this problem? Is it something you stumble upon frequently?