DEV Community: Miguel Crespo

Canceling requests in Javascript

Miguel Crespo — Wed, 20 Dec 2023 07:00:05 +0000

If you're a developer like me, you probably had to write some custom logic to handle stale requests that you triggered in the past, but whose response you no longer need. But did you know that it's possible to cancel HTTP requests from your code even after triggering them?

There's an API for this specific case and I recently had the opportunity to use it, and let me tell you that it's more useful than what I initially thought. Here are the reasons why you should use it:

Why you should cancel stale HTTP requests?

Cancelling HTTP requests when no longer needed has several advantages and can reduce the complexity of your code by a lot. Some of the benefits you get when you cancel a request instead of just ignoring them are:

The resource associated to them get released as well
It prevents unnecessary data transfer and the subsequent parsing of the response, which can be a heavy task depending on the size of the response
Prevents unexpected but common issues such as accidentally overwriting newer information when a stale request gets resolved
Simplifies the code as you no longer need to have custom logic to ignore stale requests
It opens the possibility to free resources in the backend as well if you listen to when clients cancel their requests

The API

All the knowledge I have about Javascript tells me that the JS way to cancel a fetch request would be something like:

const promise = fetch('https://dummyjson.com/products').then(res => res.json()).then(console.log);

// On user activity...
promise.abort() // Or .cancel()

But I was wrong, and yet, the real cancelation API is easy and simple. It basically consists of one object called AbortController that has one method .abort(). Although it's weird looking for a JS developer, the API is quite straightforward, and it's also built generic enough to be easily adopted by other APIs.

Canceling a fetch request

Canceling a fetch request is a simple as creating an AbortController instance and passing its .signalproperty as a property to the fetch function:

const controller = new AbortController();

// URL with big file so it takes time to download, taken from mdm web docs
fetch('https://mdn.github.io/dom-examples/abort-api/sintel.mp4', { signal: controller.signal })
  .then(res => {
    console.log("Download completed")
   })
   .catch((err) => {
     console.error(`Download error: ${err.message}`)
   })

const onCancel = () => {
  controller.abort()  
}

Things to note:

Reusing an AbortController for a new fetch request will cancel the request immediately if the signal was aborted in the past
Aborting a signal after a request has been resolved won't trigger any error

Cancelling multiple fetch requests

You can use one signal to cancel multiple fetch requests:

const controller = new AbortController();

fetch('https://dummyjson.com/products', { signal: controller.signal })
fetch('https://dummyjson.com/products', { signal: controller.signal })
fetch('https://dummyjson.com/products', { signal: controller.signal })
fetch('https://dummyjson.com/products', { signal: controller.signal })

setTimeout(() => {
  controller.abort()
}, 400);

Handling user cancelation errors

One thing to notice is that canceling requests will throw an AbortError so you need to handle them in your catch method. In most of the cases, you just want to ignore them, since they are not really errors.

fetch('https://mdn.github.io/dom-examples/abort-api/sintel.mp4', { signal: controller.signal })
  .then(res => console.log("Download completed"))
   .catch((err) => {
     if (err.name !== 'AbortError') {
       console.error(`Download error: ${err.message}`)
     }
   })

Knowing this will prevent you from getting spammed by your logging system --If you have one--.

What about other libraries?

Axios

Since v0.22.0, Axios supports canceling requests out of the box with a similar API. The only thing to watch out for is that Axios returns a different error for canceled requests: CanceledError.

const controller = new AbortController();

axios.get('https://dummyjson.com/products', { signal: controller.signal })
     .catch((e) => {
        if (e.name !== 'CanceledError') {
          console.error(e);
        }
      });

More details here.

React query

Another popular library that supports request cancelation out of the box is React Query.

The following is an example taken from the official documentation.

const query = useQuery({
  queryKey: ['todos'],
  // Get the signal from the queryFn function
  queryFn: async ({ signal }) => {
    const todosResponse = await fetch('/todos', {
      // Pass the signal to one fetch
      signal,
    })
    const todos = await todosResponse.json()

    const todoDetails = todos.map(async ({ details }) => {
      const response = await fetch(details, {
        // Or pass it to several
        signal,
      })
      return response.json()
    })

    return Promise.all(todoDetails)
  },
})

Food for thought

As said before, the entire cancelation API is quite generic, and it's designed to be implemented by other APIs and be adapted to other usages. The implementations are not only limited to HTTP requests, but could also be used to cancel heavy Javascript logic. For example, the following code could be a costly function that is run somewhere along your entire logic and could use a signal to prevent using resources when not necessary:

const  veryHeavyOperation = (signal) => {
  Array.from({ length: 10 }).forEach((_, index) => {
    if(signal.aborted) return; // If the signal is cancel, don't do anything 🦥
      setTimeout(() => {
        // HEAVY OPERATIONS
        console.log("Heavy Operations: ", index)
       }, 1000);
     });
};

Conclusion

I think this is an underrated API that deserves more credit. It's easy to use, It's supported in all the major browsers, and It could improve a lot the experience of the people using our products, as well as our experience as developers.

How to write a neovim plugin in lua

Miguel Crespo — Wed, 06 Dec 2023 07:00:06 +0000

I've been using Neovim as my main editor for more than 3 years now, but I sporadically use Emacs from time to time, specially to write and keep notes. In fact, this blog is written using Emacs and one of its plugins called Org Mode.

But recently, I tried to use Emacs as my main editor and although it didn't quite go as I planned, in the process I came to learn and love many things about Emacs, like elisp and the scratch buffer.

In this post, I will explain how to write a basic plugin that can create a scratch buffer like the one in emacs. While at the same time showing the basic architecture of a neovim plugin and some other things I learned in the process.

What are we building?

We're building a small implementation of the scratch buffer but for neovim. In a scratch buffer, you can quickly try new pieces of code and later decide if you want to save it or just discard it when closing neovim.

The original implementation supports evaluating code directly in the buffer (REPL), but we won’t be implementing this functionality as it’s quite complex; therefore this plugin will be small.

The anatomy of a Neovim plugin

Let's start by seeing how the architecture of a common neovim plugin looks like, usually a neovim plugin is structured in the following way:

.
├── LICENSE
├── plugin
│  └── plugin-file.lua
├── lua
│  └── main-file.lua
└── README.md

The plugin and lua folder are special cases and have the following meanings:

plugin folder

All files in this folder will get executed as soon as Neovim starts, this is useful if you want to set keymaps or autocommands regardless of the user requiring the plugin or not

lua folder

The lua folder is in most cases where your plugin's code lives, this code will only be executed when the user explicitly requires your plugin, for our example require('scratch-buffer')

For this plugin, we don't want to execute any code if the user has not required the plugin, and therefore all the code will live inside the lua folder.

Let’s create the main file for our plugin, the naming is important and will usually be the same as the plugin, there are two ways to do it:

Having a single lua file named after the plugin, e.g: scratch-buffer.lua
Having a folder named after the plugin with an init.lua inside of it, e.g lua/scratch-buffer/init.lua\.

Let's go with the first option and create a new file named scratch-buffer.lua\ inside the lua/\ folder and put the following code on it:

.
├── LICENSE
├── lua
│  └── scratch-buffer.lua
└── README.md

print("Hello from our plugin")

Our plan

Let’s stop for a moment to think about what we need to do exactly for this plugin… We need to:

Create a command that gets executed when neovim starts…
Create an empty scratch buffer and assign a filetype to it
Set the buffer in the initial window

How to load our extension

We can have the code of our extension wherever we want in our filesystem, but we need to tell Neovim where our plugin's code is, so it can load the files correctly. Since I use lazy.nvim this is the way to load a plugin from a local folder:

{
  dir = "~/Documents/projects/scratch-buffer", -- Your path
  name = "scratch-buffer",
  config = function ()
    require('scratch-buffer')
  end
}

Now if you restart your neovim you will see the message Hello from our plugin in the command line

Execute code when neovim starts

Autocommands are a way to tell neovim to execute commands after a specific event, these events can be, for example:

When Neovim starts
When a file is written/loaded
When leaving or entering a buffer/window

One of the best ways to explore the Neovim documentation is by using the official documentation, if you have telescope.nvim installed, you can just run :Telescope help_tags and search for autocommands

After reading the documentation, we know that to execute a command when neovim loads, we can use the event VimEnter, this is what the documentation says:

After doing all the startup stuff, including loading vimrc files, executing the "-c cmd" arguments, creating all windows and loading the buffers in them.

When creating an autocommand we also need to define an augroup for our extension, which is just a way to group all our commands. It's a good idea to group them based on the name of the extension, so if the user needs to review an autocommand it is clear which extension is creating it.

local augroup = vim.api.nvim_create_augroup("ScratchBuffer", { clear = true })

local function main()
  print("Hello from our plugin")
end

local function setup()
  vim.api.nvim_create_autocmd("VimEnter",
    { group = augroup, desc = "Set a fennel scratch buffer on load", once = true, callback = main })
end

return { setup = setup }

In the autogroup we set the option clear = true so if our plugin gets called multiple times it just clears the previous autocommands instead of creating multiple autocommands.

We're also exporting a function called setup, there's nothing special about this name, but it's the most common convention used across neovim plugins to initialize a plugin.

Let’s update the require of our plugin to call the setup function:

require('scratch-buffer').setup()

Now, when you exit and enter neovim, you will see the string Hello from our plugin in the command line.

Create a buffer using neovim APIs

Now we need to create a buffer that will function as our scratch-buffer. As we did before, the first thing to do is to search the documentation, in there we find that we need to use the nvim_create_buf({listed}, {scratch}) function.

This function, as the documentation explains, receives two parameters:

listed

Whether the buffer is shown when you run :buffers

scratch

Very convinient property

Creates a "throwaway" |scratch-buffer| for temporary work (always 'nomodified'). Also sets 'nomodeline' on the buffer.

Let's create a new function that creates the buffer and returns it. Additionally, let's set the filetype to lua and the name of the buffer to *scratch*

local function create_buffer()
  local buf = vim.api.nvim_create_buf(true, true)
  vim.api.nvim_buf_set_name(buf, "*scratch*")
  vim.api.nvim_set_option_value("filetype", "lua", { buf = buf })
  return buf
end

If you restart neovim and list the buffers, you will notice the buffer is created and listed correctly

Set the scratch buffer as the initial buffer

We have by now created a buffer when the user enters Neovim, but we still need to set it up as the initial buffer, to accomplish that, we need to use the nvim_win_set_buf, this is what the documentation says:

Sets the current buffer in a window, without side effects

Attributes: ~ not allowed when |textlock| is active

Parameters: ~ • window Window handle, or 0 for current window • buffer Buffer handle

Since we’re touching this, let's also add some welcome text to the scratch buffer and put the cursor in the last line for more convenience

local buf = create_buffer()

vim.api.nvim_buf_set_lines(buf, 0, -1, true, { "-- Welcome to Neovim!", "" })

vim.api.nvim_win_set_buf(0, buf)

vim.api.nvim_win_set_cursor(0, { vim.api.nvim_buf_line_count(buf), 0 })

How can we publish the extension to the world?

You’re probably aware that the preferred way to distribute neovim is to use git repositories, most of the people use Github for this, and it’s not different with this plugin.

To distribute it, we can simply create a new repository on Github and set the visibility it to public. Now users can install it like any other package:

{
  "miguelcrespo/scratch-buffer.nvim",
  config = function ()
    require('scratch-buffer').setup()
  end
}

Conclusion

In this blog post, I wanted to share what I learned while writing my first small plugin called scratch-buffer. The original implementation of this plugin on GitHub is written in a language called Fennel, which belongs to the lisp family and compiles to lua, but the functionality is basically the same. I have written more about Fennel in a previous blog post.

After reading this blog post, you hopefully understand much better now how neovim plugins work and how to write one. You also know how to search the neovim documentation to find the functions you need. I must admit this step took me a long time since I’m very spoiled as a Javascript developer to easily find blog post about things I need, but you get used to it.

Without more to say, happy coding!

All the code…

Just in case you want to see all the code together

local function create_buffer()
  local buf = vim.api.nvim_create_buf(true, true)
  vim.api.nvim_buf_set_name(buf, "*scratch*")
  vim.api.nvim_set_option_value("filetype", "lua", { buf = buf })
  return buf
end

local function main()
  print("Hello from our plugin")
  local buf = create_buffer()

  vim.api.nvim_buf_set_lines(buf, 0, -1, true, { "-- Welcome to Neovim!", "" })

  vim.api.nvim_win_set_buf(0, buf)

  vim.api.nvim_win_set_cursor(0, { vim.api.nvim_buf_line_count(buf), 0 })
end

local function setup()
  local augroup = vim.api.nvim_create_augroup("ScratchBuffer", { clear = true })

  vim.api.nvim_create_autocmd("VimEnter",
    { group = augroup, desc = "Set a fennel scratch buffer on load", once = true, callback = main })
end

return { setup = setup }

Configuring Neovim with Fennel

Miguel Crespo — Fri, 10 Nov 2023 14:25:35 +0000

I like changing the configuration of my Neovim quite a lot, and recently I have reached a plateau where my configuration does everything I want, most people would be happy once they reach this point but I was not because I like moving stuff so I started looking for things that looked cool and I found many people talking about Fennel…

What is Fennel?

Fennel is a Lisp language that compiles to lua, it's easy to learn and since learning lisp has been on my todo list for quite some time after I tried Emacs, I saw this as a good opportunity to do it.

Why Fennel?

If you like lisp languages or want to learn one, fennel is a good way to start since it's quite a simple language that compiles to another simple language (lua) while being fully compatible with it and having zero overhead once compiled.

It has many syntax improvements over lua, but the most remarkable features of Fennel are:

No global variables by default

One of the things I didn't like much about lua is how easy is to accidentally declare global variables by accident, when you look for many lua tutorials you see people using global variables without they knowing because it's simply the default behavior:

-- This is a global variable
name = 'User 1'

-- Non-global variable
local name = 'User 1'

In most cases, you want to create a local variable, and fennel defaults to it every time you create a variable:

; Declare a local variable
(local name "User 1") ; Compiles to local name = "User 1"
(var language "fennel") ; Compiles to local language = "fennel"

Immutable variables by default

This behavior is becoming quite popular in other modern languages like Rust, which means that once you assign a value to a variable the value cannot change unless you explicitly allow it.

The advantage of this behavior is that it makes it easier to know how a variable is being used in your code just by looking at how it was defined

Examples in lua:

-- There's no way to know if the value can change in the code by just looking at it
local language = "lua"

-- You might or might not see this somewhere in your code
language = "fennel"

Fennel:

; Once you see this, you know this is a constant and cannot change
(local language "fennel")

(set language "lua")
; Last line will generate the error:
; Compile error: expected var language
;
; (set language "lua")
; * Try declaring language using var instead of let/local.
; * Try introducing a new local instead of changing the value of language.

Of course, you can still declare mutable variables with var:

(var language "fennel")

(set language "lua") ; It works...

It doesn't declare variables with typos

In lua is very easy to redeclare new variables by mistake when you have a typo and when you do this you will probably only catch the error at runtime

local my_variable = 1

myvariable = 2 -- There's a typo in this line and this will just create a new global variable called myvariable

In fennel this cannot happen since the compiler will warn you at compile time:

(var my_variable 1)

(set myvariable 2)
; Compile error: expected local myvariable
;
; (set myvariable 2)
; * Try looking for a typo.
; * Try looking for a local which is used out of its scope.

Tables have a more familiar syntax

Another small change is that fennel uses a more traditional way to represent arrays and tables

; Table or dictionary
(local person {:name "User 1"}) ; => local person = {name = "User 1"}

; Array
(local numbers [1 2 3]) ; => local numbers = {1, 2, 3}

Destructuring

Fennel supports destructuring like many other modern languages

(local person {:name "User 1" :age 12})

; Destructuring table
(local {:name person_name} person)

(print person_name) ; => User 1

(local my-numbers [1 2 3 4 5])

; Destructuring a list
; & c means the rest goes to this variable
(local [a b & c] my-numbers)

(print a) ; => 1
(print b) ; => 2
(print c) ; => [3 4 5]

Macros

Fennel is a lisp language which means it supports macros, if you ever read anything about lisp you see people talking about them as the holy grail

And indeed, macros are a powerful way to metaprogram and make languages easier to read. Macros are kinda normal functions but they get executed at compile time and output code.

With macros, you can extend the fennel language and create domain-specific function, for example, for Neovim, you could create a macro to set a vim option, like:

(set! tabstop 2)
; Or to set keymaps
(map! [:n] "-" (fn [] (print "Opened")) "Open parent directory")

This whole functionality requires an entire blog post, but you can read the extensive fennel macros documentation

Picking a transpiler

Now that I convinced you to use Fennel, let's see how we can start configuring Neovim with it.

First, you need to find a way to compile the fennel code into lua code, there are many options for this:

I picked tangerine since it's quite simple to setup and has a nice way to preview the compiled lua code which is very useful while you're learning fennel.

It also comes with an optional package called hibiscus with useful macros like set! and map!

Configuring Tangerine and hibiscus with lazy

To start using tangerine.nvim we first need to install it, and since we need to load tangerine before anything else to output our lua files.

This is how I did it:

Move my existing init.lua to another file inside the /lua folder, e.g: /lua/main.lua
Replace the code in init.lua with:

The code below was partly taken from the official documentation and will:

Download automatically tangerine and hibiscus (optional)
Configure tangerine to generate the lua code when we save a fennel file or start neovim

It will store the compiled lua files in .local/share/nvim/tangerine for MacOS (this changes depending on the operating system)

local function bootstrap(url, ref)
  local name = url:gsub(".*/", "")
  local path = vim.fn.stdpath [[data]] .. "/lazy/" .. name

  if vim.fn.isdirectory(path) == 0 then
    print(name .. ": installing in data dir...")

    vim.fn.system { "git", "clone", url, path }
    if ref then
      vim.fn.system { "git", "-C", path, "checkout", ref }
    end

    vim.cmd [[redraw]]
    print(name .. ": finished installing")
  end
  vim.opt.runtimepath:prepend(path)
end

bootstrap("https://github.com/udayvir-singh/tangerine.nvim")

-- Optional and only needed if you also want the macros
bootstrap("https://github.com/udayvir-singh/hibiscus.nvim")

require 'tangerine'.setup {
  target = vim.fn.stdpath [[data]] .. "/tangerine",

  -- compile files in &rtp
  rtpdirs = {
    "ftplugin",
  },

  compiler = {
    -- disable popup showing compiled files
    verbose = false,

    -- compile every time changes are made to fennel files or on entering vim
    hooks = { "onsave", "oninit" }
  },
}

Create a init.fnl file that will be loaded automatically by tangerine when neovim starts
Optional: Move all your files in the /lua folder to a new /fnl folder By moving all your lua files to the /fnl folder you can require lua code from fennel and vice versa

We also need to put tangerine.nvim in the list of Lazy plugins so it gets updated as a normal plugin

In the end, you will end up with a folder structure like

.
├── fnl
│  ├── theme.fnl
│  ├── main.lua
├── init.fnl // The real init
├── init.lua // Load tangerine

Writing the configuration

This is a part of my real init.fnl

(require :options) ; Normal neovim options
(require :theme) ; Load the theme config
(require :plugins) ; /fnl/plugins.lua This is still a lua file and will load all the plugins with Lazy

And here is a part of my options.fnl file

(import-macros {: set! : set+} :hibiscus.vim)

; ...

;; Indentation
(set! expandtab)
(set! shiftwidth 2)
(set! tabstop 2)

;; Line numbers
(set! number)
(set! relativenumber)
(set! numberwidth 3)
(set! numberwidth 3)

;; Whitespace
(set! list)
(set! listchars {:trail "·" :tab "→ " :nbsp "·"})

;; Insert-mode completion
(set+ :shortmess :c)

; ...

An example of setting mappings:

(import-macros {: map!} :hibiscus.vim)

(local oil (require :oil))

(oil.setup {:default_file_explorer true
            :keymaps {:q :actions.close}
            :view_options {:show_hidden true}})

(map! [:n] "-" oil.open "Open parent directory")

Extra: Improving fennel DX in Neovim

Install a code formatter

We will use fnlfmt to format our fnl code

Install it with your favorite package manager

Using brew

brew install fnlfmt

Install a LSP for fennel

If you're using language servers for completion and diagnostic then you will be glad to know there's one for fennel called fennel_language_server

This is my configuration:

require 'lspconfig.configs'.fennel_language_server = {
  default_config = {
    cmd = { 'fennel-language-server' },
    filetypes = { 'fennel' },
    single_file_support = true,
    -- source code resides in directory `fnl/`
    root_dir = lspconfig.util.root_pattern("fnl"),
    settings = {
      fennel = {
        workspace = {
          -- If you are using hotpot.nvim or aniseed,
          -- make the server aware of neovim runtime files.
          library = vim.api.nvim_list_runtime_paths(),
          checkThirdParty = false, -- THIS IS THE IMPORTANT LINE TO ADD
        },
        diagnostics = {
          globals = { 'vim' },
        },
      },
    },
  },
}

lspconfig.fennel_language_server.setup {
  on_attach = function(client, bufnr)
    -- Support formatting with fnlfmt
    vim.keymap.set('n', '<leader>cf', function()
      vim.lsp.buf.format({ async = true })
    end, { noremap = true, silent = true, buffer = bufnr, desc = "Format code" })
  end,
}

Install conjure plugin

Conjure is an amazing plugin that lets you execute pieces of code inside Neovim, this is very useful when working with fennel but it also supports lua, python and many other languages.

To install it using Lazy.nvim:

{
  "Olical/conjure",
  -- [Optional] cmp-conjure for cmp
  dependencies = {
    {
      "PaterJason/cmp-conjure",
    },
  },
  config = function()
    require("conjure.main").main()
    require("conjure.mapping")["on-filetype"]()
  end,
  init = function()
    -- Set configuration options here
    vim.g["conjure#debug"] = true
  end,
},

After you install it, conjure sets some default mapping to the file types it supports:

<localleader>er Evaluate the root form under the cursor
<localleade>ee Evaluate the current form under the cursor
<localleader>eb Evaluate the current buffer contents

Read the Conjure documentation for more details

Conclusion

Configuring Neovim using Fennel has been a really enjoyable experience, partly made possible by using Conjure.

I liked writing fennel in Neovim so much that I even wrote my first neovim plugin using it scratch-buffer.nvim.

But not everything has been great so far, here are some of the downsides I noticed so far:

it's a bit annoying to have to transpile all the lua code you find on the internet to fennel, but this page has helped me a lot with it: https://fennel-lang.org/see
Also the vim API signature is not available using the fennel-language-server In lua you can install neodev and you will get some nice lsp suggestions with the vim API, this is sadly not possible in fennel right now
When there's an error it's a bit annoying to debug since the lines that the error points are in the compiled lua code so you need to open it to understand the compiled lua code

References

Fennel official tutorial
https://fennel-lang.org/tutorial
Fennel reference
With useful information about destructuring and more details in general: https://fennel-lang.org/reference
Convert lua code to fennel and viceversa
https://fennel-lang.org/see

WebSockets 101

Miguel Crespo — Wed, 19 Jul 2023 06:00:13 +0000

Intro

Disclaimer: I didn't know much about Websockets 1 week ago, all the experience I had with Websockets was when I developed a chat application back in 2016 using a JS framework that tried to be a Ruby on Rails implementation called SailsJS, so I decided to research about this technology and consumed multiple resources which I will link in this blog post and each section.

Websockets are a way to handle full-duplex communications (or two-way communications), they are very useful to build applications that needs real time data like chat application or a stock dashboard. Websockets arrived as an improvement over previous solutions like:

Polling

The most simple solution consists of making HTTP requests at a set interval

Long Polling

Desperate Times Call for Desperate Measures

This just means that the client will call the HTTP endpoint and the server will not resolve the request immediately but wait until a message needs to be delivered to the client, this approach worked but had a lot of disadvantages, like timeouts, latency, and it sounds very hacky to begin with.

HTTP streaming and Server-sent events (SSE)

These solutions work well but are more suited to one-way communications as in use cases where the server needs to send a notification to its user, e.g.: A new post was created, etc.

Websocket servers can (but don't have to) be run alongside a normal HTTP server since WebSockets use a different protocol ws and wss (for secured connections like HTTPS).

Websockets are a simple protocol from the user's perspective as they only consist of 3 different events:

open
close
message

Connecting (Handshake)

All WebSocket connections will start with a handshake, always initiated by the client, that if successful will upgrade the connection from HTTP to ws/wss protocol.

Notes:

A client can create as many connections as it desires.
Arbitrary headers cannot be set in the browser when setting up a WebSocket connection.

All the behavior below is defined at the browser level by W3's HTML5 WebSocket API specification and at the protocol level via RFC 6455 "The WebSocket Protocol" and is of course all managed by the system (the browser in JavaScript) and you don't need to worry about it, the following is for information purpose only

The handshake process consists of the following steps:

Client sends the following Headers on a standard http(s) request

HTTP GET ws://127.0.0.1:8000/ 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: hXcK3GM6B2r6yj/4L0Vuqw==
Origin: http://localhost:3000
Sec-WebSocket-Version: 13

Sec-WebSocket-Key

This header is a random string, the server will take these bytes and appends the special string 58EAFA5-E914-47DA-95CA-C5AB0DC85B11, hashed it and encode it in base64 and return in the Sec-WebSocket-Accept header.

If the server accepts the connection, it will respond with the following headers:

HTTP GET ws://127.0.0.1:8000/ 101 Switching Protocols
Connection: Upgrade
Sec-WebSocket-Accept: rXzSb8mhB4ljxko8kbyiCohJ4Fc=
Upgrade: websocket

The connection will be considered as successful only when the server:

Replies with status code 101
Includes Connection Header with value Upgrade
Includes Upgrade Header with value websocket

Example in JavaScript

All of that complexity is already handled by the browser, and establishing a WebSocket connection is quite straightforward:

const socket = new WebSocket('wss://example.com/socket');

socket.addEventListener('open', (event) => {
  console.log('WebSocket connection established!');
});

Message exchange

Once a connection has been established, it is kept alive both in the server and the client.

Both the server and the client can send messages at any moment in either text or binary data (Blob or ArrayBuffer objects). Sending messages in binary data is marginally faster but not by a big factor since sending text data requires a UTF-8 conversion but this process is quite fast nowadays.

Example in Javascript

Sending messages is as simple as:

socket.send(message); // e.g: '{a: 1]'

And receiving them:

socket.addEventListener('message', (event) => {
  console.log('Received message:', event.data);
});

Closing

A WebSocket connection can be closed at any moment by either the client or the server.

From the client's perspective

it's very straightforward:

socket.close();

From the server:

It can be a bit trickier since there might be problems if you wait for the clients to disconnect, clients might hang and eventually will cause performance issues. Two ways to address this is by using the following code (that was taken from Stackoverflow):

Hard shutdown

With this approach, we just terminate the connection without waiting for clients to disconnect

// Soft close
socket.close();

process.nextTick(() => {
if ([socket.OPEN, socket.CLOSING].includes(socket.readyState)) {
    // Socket still hangs, hard close
    socket.terminate();
}
});

Soft shutdown

Here, we give the clients some time to disconnect before terminating them

// First sweep, soft close
wss.clients.forEach((socket) => {
  socket.close();
});

setTimeout(() => {
  // Second sweep, hard close
  // for everyone who's left
  wss.clients.forEach((socket) => {
    if ([socket.OPEN, socket.CLOSING].includes(socket.readyState)) {
      socket.terminate();
    }
  });
}, 10000);

Auth and security

Unfortunately, the WebSocket protocol doesn't suggest a way to handle authentication, but many people have come up with many solutions over time.

As said before, the most traditional way to authenticate in the modern web by sending a Header with a token like a JWT is not possible with WebSockets since sending a custom header is not supported in the browser.

This leaves us with solutions like the one presented in this document:

Sending credentials as the first message in the WebSocket connection

This method is fully reliable but moves the authentication to the application layer and can expose you to leak information if you’re not careful enough. Another negative point is that it allows everyone to open WebSocket connections with your server.

Adding credentials in the WebSocket URI as a query parameter

Another method is to send tokens as a query parameter when opening the WebSocket connection, something like: wss://localhost:3000/ws?token=myToken The downside of this is that this information might end up in the logs of your system, one way to mitigate this would be to use single-use tokens, but the industry tends to consider this risk as unacceptable.

Setting a cookie on the domain of the WebSocket URI

This solution is also reliable as long as your WebSocket server is running in the same domain as your http server if this is not the case then it won't be possible since the Same-Origin Policy doesn't allow setting a cookie on a different origin.

But there are two ways to overcome this problem:

1. Move the WebSocket server to a subdomain of the main `http` server, e.g: `websocket.example.com`

2. Use an `iframe` running on the same WebSocket domain to set the cookie

If you go with this approach you need to consider that as today [2023-07-18 Tue] Google Chrome won’t work with this approach unless you set the SameSite and Secure properties as well, e.g.
document.cookie = 'my_token=token; SameSite=None; Secure;'

Cross-Site WebSocket Hijacking

One thing to be aware of cookie-based authentication solution is that Websocket connections are not restrained by the same-origin policy, and thus it opens a vector attack called Cross-Site WebSocket Hijacking.

This means that if you set a cookie for the websocket domain this cookie will be sent when connecting to the server no matter from which website the WebSocket connection is established, this opens a security problem since malign websites can take advantage that the user already has a cookie set for the domain to also subscribe to messages with the WebSocket server.

One way to mitigate this problem is to always validate the Origin of the client before establishing the connection.

You can much more information in this article

Resources

Binary vs text messages
https://stackoverflow.com/questions/7730260/binary-vs-string-transfer-over-a-stream
Shutting down WebSocket connections in the server
https://stackoverflow.com/questions/41074052/how-to-terminate-a-websocket-connection/49791634#49791634
Authentication in WebSockets
https://websockets.readthedocs.io/en/stable/topics/authentication.html
Cross-Site WebSocket Hijacking
https://christian-schneider.net/CrossSiteWebSocketHijacking.html
Websocket framed protocol
https://sookocheff.com/post/networking/how-do-websockets-work/

How to automate software installation and configuration on macOS using Homebrew and Stow

Miguel Crespo — Thu, 22 Jun 2023 19:16:10 +0000

I'm a person that likes to customize most of my applications in a way that they work as I want them to work, my normal workflow includes several tools that my muscle memory expects to always be there.

I for example, spent many hours customizing my favorite editor Neovim, I know every shortcut defined on it because I created all of them, this is true for many tools I use every day, so you can imagine how time-consuming it would be to do this over and over every time I get a new computer, or I need to configure it from zero.

That's why I came up with a system to help me to:

Install all the software I need, like normal shell applications, mac applications and also fonts
Apply my saved preferred configuration for each of these applications
Configure all macOS preferences automatically

In this post, I will talk about how I manage each one of the above points

Installing software automatically

I install almost 80 applications automatically every time I set up a new computer, Mac applications like Firefox, Shottr, Anki and tools like git, node, etc.

Installing all these tools one by one would be very tiresome, fortunately on Mac we have Homebrew (The Missing Package Manager for macOS), with this tool we can install each application one by one like:

brew install git
# Or
brew install --cask firefox

But we can also define a file with all the software we want on our computer as a sort of package.json for node or a Gemfile for Ruby, this file is commonly named Brewfile (no file extension).

Let's see an example of a Brewfile

tap 'homebrew/cask'
tap 'homebrew/cask-fonts'

brew 'neovim' # Improved Vim version

cask 'firefox' # The best browser

cask 'font-fira-code'

The first two lines are to install additional formulas, formulas are just the way to tell homebrew how to install software, in this case the lines are to:
Install additional Mac applications aka casks
Install additional fonts

In this file you can put all the software you want, and then you just need to run the following command to install everything.

brew bundle

PRO TIP: add the --v to get more details of the progress of the installation 😎.

To know the name of the formulas you need to use the Homebrew page and to know how to install specific versions or pass flags to the command check this document.

Apply saved configuration for each application

Most of the tools store their configuration in a file and most of the names of these files start with a . that's why you see this technique of saving configuration files called dotfiles, there are plenty of examples on GitHub of dotfiles repositories for each specific software, that you can see and take as an inspiration to configure the tools you want.

The main idea here is to configure everything the way you want and then save these files in say a git repository, so when you're configuring a new computer you just need to download this repository and copy the files to the right location.

There's already a tool that we can use to make this process easier, it's called Stow and it's a UNIX tool that helps us to manage the configuration files, instead of just copying the files, Stow will create Symbolic links (More details here), this means in a nutshell, a file that points to another file, the advantage of this approach is that we can have all our configuration files in the same folder and repo and under git and then have a symbolic link file in the places where the final applications are expecting them to find them.

To install Stow just run on your terminal:

brew install stow

Or add it to your Brewfile

brew 'stow'

Stow usage example

Let's create a folder called dotfiles anywhere in our system and let's configure git as an example, git stores all its configuration in a file called .gitconfig in the user's root folder.

Adding a new dotfile to our repository

In order to add a new dotfile to our repository, we need to first create a new folder for this new configuration file, the name can be anything you want, in our example let's call it gitconfig/ then there are two ways to add a new dotfile to our new folder:

Move the existing .gitconfig file if you already configured git before mv ~/.gitconfig dotfiles-folder/gitconfig/.gitconfig
Manually create the file in the gitconfig folder in dotfiles if you don't have any configuration yet Put some configuration in this folder, for example:

[user]
  name = Miguel
  email = pepe@perez.com

No mater which way you choose, the folder structure of the gitconfig/ folder should match exactly the folder structure from the root folder.

Some examples:

if you want to have your dotfile in ~/.gitconfig you will create the following folder structure /folder-name/.gitconfig
Likewise, ~/.config/nvim/init.lua > =folder-name/.config/nvim/init.lua

Repeat this process for all the configuration files you want to add to your new repository.

Applying the dotfile configuration

And now to create the symbolic links or the next time you're configuring a new computer, you just need to run:

stow -nvSt ~ folder-name

If you are satisfied with the output, then remove the -n flag and that's it.

Configuring macOS automatically

I think this was the most tricky part of the whole automatization process because there's no much documentation about this, and it's mostly a trial and error process.

Forget about having to open the Mac system preferences one by one, the idea of this step is to save all the macOS configuration you like, so next time you're setting a new mac from zero you don't have to go to the system preferences, you only need to run a command.

This is for example some of my configuration, it configures the size of the Dock and sets tap to click, dark mode, etc.

# System Preferences > Dock > Minimize windows into application icon
defaults write com.apple.dock minimize-to-application -bool true

# System Preferences > Dock > Magnification:
defaults write com.apple.dock magnification -bool false

# System Preferences > Dock > Size:
defaults write com.apple.dock tilesize -int 36

# System Preferences > Trackpad > Tap to click
defaults write com.apple.driver.AppleBluetoothMultitouch.trackpad Clicking -bool true
defaults write com.apple.AppleMultitouchTrackpad Clicking -bool true

defaults write NSGlobalDomain AppleInterfaceStyle Dark  # Use dark menu bar and dock.

How to know the name of the variables

This is the tricky part… You can either:

Read many dotfile repositories on GitHub until you find the setting that you want.
Find it yourself by opening System Settings and run in the Terminal app

defaults read > before.txt

This will save all the current configuration in the before.txt file

Make the changes to the configuration settings that you want to detect
Use the same defaults read command to display the current user defaults.

defaults read > after.txt

Do a diff to get the name of the key

diff before.txt after.txt

Extra: Installing software from the App Store

Homebrew by itself cannot install software from the App Store but we can use a tool called mas

brew install mas

Or add it to your Brewfile

brew 'mas' # Mac App Store manager

mas 'Pages', id: 409_201_541

You need to get the id of the applications manually by running

mas search Pages # Look for Apple Pages for example

Putting all of this on a single file

To complete the automation, you can put all the commands in a bash file, I have a much more complex setup.sh file, but this is the basic idea:

#!/usr/bin/env bash

echo "Setting up this mac"

echo "Installing brew packages and casks"
brew bundle -v

echo "Moving dotfiles"
stow -vSt ~ neovim/

echo "Etc..."

Next time you're setting a new computer you will only need to run bash ./setup.sh

Questions?

Let me know!

Debugging Javascript applications with Neovim

Miguel Crespo — Thu, 22 Jun 2023 19:13:47 +0000

Introduction

If you're a software developer you will have to debug code at one point or the other and the truth is that debugging in Neovim for the first time can be a daunting task.

The current status of debugging within Neovim requires lots of configuration compared to VS Code

The best way to debug code in my experience is using the Debug Adapter Protocol (DAP) with a tool called nvim-dap, if you want to know more about the details of how this works check the official information or my blog post.

In this blog post, we will see how to configure this tool to:

Debug node applications (Servers, files)
Debug web applications using Chrome
How to keep the session state across multiple debugging
Supporting .vscode/launch.json files

I will assume you already know how to:

Install packages in Neovim
Some basic understanding of Lua

What is possible?

Let's see what is currently possible with Neovim

Debugging Javascript applications with Neovim video

My experience so far

In order to get to this level I sadly had to expend a lot of time because it was not a plug and play experience, the journey looked a bit like this:

Found out that debugging was possible using nvim-dap, previously the only thing I was using VS Code for was to debug applications
Read the nvim-dap documentation and understand nothing because most of the documentation assumes knowledge about DAP
Read the DAP documentation and begin to understand a bit :D
Look for adapters to debug Javascript, typescript and node applications in general
Find nvim-dap-vscode-js and read the documentation and understand nothing cause most of the documentation is leveraging the official https://github.com/microsoft/vscode-js-debug documentation which only speaks about VS Code
Look on the internet until finally get something to work
Be happy (But tired and sad that I needed to spent so much time in this)

The next steps in this post will show you how to configure your Neovim to do the same

Install nvim-dap

We will need to install nvim-dap, this tool is basically a DAP client in Neovim. In order to install it, you only need to add the following line in your favorite package manager

'mfussenegger/nvim-dap'

Then we will load the plugin in our Neovim configuration and set some keymaps to better use DAP

local dap = require("dap")

-- Set keymaps to control the debugger
vim.keymap.set('n', '<F5>', require 'dap'.continue)
vim.keymap.set('n', '<F10>', require 'dap'.step_over)
vim.keymap.set('n', '<F11>', require 'dap'.step_into)
vim.keymap.set('n', '<F12>', require 'dap'.step_out)
vim.keymap.set('n', '<leader>b', require 'dap'.toggle_breakpoint)
vim.keymap.set('n', '<leader>B', function()
  require 'dap'.set_breakpoint(vim.fn.input('Breakpoint condition: '))
end)

In the previous step, we set up some keymaps to interact with nvim-dap like:

F5 to start a new debugging session
F10, F11, F12 to move though the code once you're in a debugging session
leader b to toggle breakpoints
leader B to set conditional breakpoints

Install and configure nvim-dap-vscode-js

Next we need to install nvim-dap-vscode-js this is the adapter that allows Neovim (through the nvim-dap) to communicate with JavaScript and interestingly, it's the same adapter used in VS Code.

Again, install the package using your preferred plugin manager

mxsdev/nvim-dap-vscode-js

Then copy this into your configuration file, this lines of code was taken from the Official Github repository

require("dap-vscode-js").setup({
  -- node_path = "node", -- Path of node executable. Defaults to $NODE_PATH, and then "node"
  debugger_path = "(runtimedir)/site/pack/packer/opt/vscode-js-debug", -- Path to vscode-js-debug installation.
  -- debugger_cmd = { "extension" }, -- Command to use to launch the debug server. Takes precedence over `node_path` and `debugger_path`.
  adapters = { 'chrome', 'pwa-node', 'pwa-chrome', 'pwa-msedge', 'node-terminal', 'pwa-extensionHost', 'node', 'chrome' }, -- which adapters to register in nvim-dap
  -- log_file_path = "(stdpath cache)/dap_vscode_js.log" -- Path for file logging
  -- log_file_level = false -- Logging level for output to file. Set to false to disable file logging.
  -- log_console_level = vim.log.levels.ERROR -- Logging level for output to console. Set to false to disable console output.
})

Some of the problems you might have with the previous configuration:

debugger_path

If you're using anything else than packer to install plugins you need to modify the debugger_path because the path might change depending on your plugin manager, for lazy, you need to change the debugger_path to be vim.fn.stdpath('data') .. "/lazy/vscode-js-debug"

adapters

I needed to add chrome and node to the list of default adapters because I wanted to be able to use the configuration I have in VSCode in my Neovim, and VS Code refers to this adapter as just node and chrome instead of node-pwa and chrome-pwa

Configuring the DAP Adapter

In this step we will add some default configuration for our adapters, we will tell the adapters in what kind of languages we want to active it.

This configuration is based on the launch configuration from VS Code

local js_based_languages = { "typescript", "javascript", "typescriptreact" }

for _, language in ipairs(js_based_languages) do
  require("dap").configurations[language] = {
    {
      type = "pwa-node",
      request = "launch",
      name = "Launch file",
      program = "${file}",
      cwd = "${workspaceFolder}",
    },
    {
      type = "pwa-node",
      request = "attach",
      name = "Attach",
      processId = require 'dap.utils'.pick_process,
      cwd = "${workspaceFolder}",
    },
    {
      type = "pwa-chrome",
      request = "launch",
      name = "Start Chrome with \"localhost\"",
      url = "http://localhost:3000",
      webRoot = "${workspaceFolder}",
      userDataDir = "${workspaceFolder}/.vscode/vscode-chrome-debug-userdatadir"
    }
  }
end

The first entry in the lua table will help you debug single Node.js files
The second will help you debug node processes like express applications
The last entry will help you debug web applications
The line userDataDir will let you save your Chrome profile in a file

Now with this configuration you will be able to start debugging Javascript applications, but let's add another plugin to make the interface easier to use

Install and configure dapui

To make the UI better, let's install dapui

With the following code, you will set up dapui and make the UI automatically open once a debugging session has been created, and also close the UI once the debugging sessions have finished

use { "rcarriga/nvim-dap-ui", requires = {"mfussenegger/nvim-dap"} }

require("dapui").setup()

local dap, dapui = require("dap"), require("dapui")

dap.listeners.after.event_initialized["dapui_config"] = function()
  dapui.open({})
end
dap.listeners.before.event_terminated["dapui_config"] = function()
  dapui.close({})
end
dap.listeners.before.event_exited["dapui_config"] = function()
  dapui.close({})
end

vim.keymap.set('n', '<leader>ui', require 'dapui'.toggle)

Debugging your first code

Debugging single Javascript files

Open the file in your Neovim and press F5 and select Launch File, the dapui UI should appear and let you debug the code similarly to VS Code

If you copied the previous configurations, you will be able to go to a

Debugging server applications like `express`

Start the node process in any other tab in your terminal with the --inspect flag, e.g:

node --inspect ./app.js

Now press F5 and select Attach

Debugging a browser application

Open a JS file from the project and select Start Chrome with Localhost, the application must be running on port 3000 but you can change this in the previous configuration file

Sharing launch configuration with VS Code fans

This step is helpful if you're planning to debug a project in which there's already a launch.json file generated by VS Code users, for this we will need json5 because most of the launch.json that VS Code generates has comments inside and these are not valid JSON files

If you have a Mac

You would need to have installed rust

brew install rustup-init

Then, you need to add the following line to your cargo config, usually it's located in .cargo/config.toml .cargo/config.toml

[target.x86_64-apple-darwin]
rustflags = [
    "-C", "link-arg=-undefined",
    "-C", "link-arg=dynamic_lookup",
]

[target.aarch64-apple-darwin]
rustflags = [
    "-C", "link-arg=-undefined",
    "-C", "link-arg=dynamic_lookup",
]

Then install json5 plugin

Packer

use {
    'Joakker/lua-json5',
    run = './install.sh'
}

Lazy

{
    'Joakker/lua-json5',
    build = './install.sh'
}

You will also need to add this line to your lua config, to be able to load the json5 plugin successfully

-- https://github.com/neovim/neovim/issues/21749#issuecomment-1378720864
-- Fix loading of json5
table.insert(vim._so_trails, "/?.dylib")

require('dap.ext.vscode').load_launchjs(nil,
  { ['pwa-node'] = js_based_languages,
    ['node'] = js_based_languages,
    ['chrome'] = js_based_languages,
    ['pwa-chrome'] = js_based_languages }
)

From now on, you should be able to debug applications in Neovim!

Find some bugs!

My notes of how to compile a typescript library

Miguel Crespo — Wed, 30 Nov 2022 09:00:00 +0000

Introduction

Recently I started developing an internal typescript library for my current company and while I was looking for the best way to transpile and distribute the code for all the clients that will soon implement it, I found these two options:

tsc
rollup

Here is my experience with both options…

Code we will use

To prove the point of this post, we will use this dummy project where

The folder no-export-please

In this folder, we have code that is not supposed to be exported outside the library

The folder not-used

Here we have a function that is not used anywhere, so ideally it should not be included in the final transpiled code

https://stackblitz.com/edit/typescript-vys5ns?file=src/index.ts

The tsconfig file

{
  "compilerOptions": {
    "target": "esnext",
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "declaration": true,
    "outDir": "dist",
    "moduleResolution": "node",
    "allowSyntheticDefaultImports": true,
    "emitDeclarationOnly": false,
    "strict": true,
    "esModuleInterop": true
  },
  "include": [
    "src"
  ],
  "exclude": [
    "node_modules"
  ]
}

Using `tsc`

tsc is the typescript Command Line Interface CLI tool that gets installed in your machine when you install typescript.

The first option I found was to use tsc to transpile the typescript code to javascript files and generate the types

I think this is a good option when you have a very simple typescript file and you only need to transpile it to vanilla javascript, but as the project grows, you start to see problems like:

It exposes implementation details

One of the problems I found while using tsc was that it might lead to exposing implementation details of the library because it keeps the same structure of the src code in the transpiled code and it also exports by default all types and it doesn't matter whether you are exporting them outside of the libraries or not, which means users of the library can import any file even when we don't want them to do it

For example, with our dummy project:

It gets compiled to:

This means client libraries can request the code from the no-export-please folder, which if you remember, was not supposed to be accessible outside the library.

No tree-shaking

It doesn't remove unused code, leading to unnecessary compiling time and disk usage, for example, our unusedFunction in our dummy project still is present in the transpiled code even when it's not used anywhere

No advanced way to tweak the transpilation

There's no way to customize the transpilation for some advanced cases like:

Polyfills/transpiling

To be fair, this also applies to rollup but at least there's a standard API to support it there.

npm dependencies

It doesn't handle well npm dependencies, it doesn't include them in the final bundle and you have to do some tricks to support them, like copying the dependencies to the dist folder, more details can be found in this StackOverflow question

https://stackoverflow.com/questions/41109461/how-include-node-modules-in-output-directory-with-typescript

Only one supported module at a time

The module used in the transpiled javascript code is the one defined in the tsconfig config file, so if your client application is using commonjs and you're only generating es6 modules, you will have problems. You can overcome this issue by executing tsc with different parameters multiple times though, like:

tsc --module esnext
tsc --module commonjs

Rollup

Rollup is a very fast module bundler that supports es modules out of the box and has an incredibly simple plugin ecosystem with a lot of options that let you extend the functionality.

At first, the first experience with rollup can be daunting, we need to install a lot of plugins as npm packages just to start!

But after a while, you will probably start to love the syntax

Installing dependencies

Let's start by installing rollup and all the plugins we need

npm i --save-dev rollup typescript rollup-plugin-peer-deps-external rollup-plugin-dts @rollup/plugin-typescript @rollup/plugin-node-resolve tslib @rollup/plugin-commonjs

Here's a short explanation for each package:

rollup

This is the main binary, you can either install it globally or locally like we're doing here

typescript

Main typescript dependency, you need this anyway if you're using typescript

rollup-plugin-peer-deps

This plugin will delete from the bundled code the peer dependencies

rollup-plugin-dts

This plugin will take all the typescript types in your project and bundle them in a single file while making sure of only exporting the types that you want to be accessible outside

@rollup/plugin-typescript

This plugin is the one in charge of telling rollup how to handle typescript files

@rollup/plugin-commonjs

If you want to import commonjs libraries, you need to add this plugin

@rollup/plugin-node-resolve

Plugin to implement the same module resolve algorithm that node uses, here are more details: https://nodejs.org/api/modules.html#modules_all_together

tslib

Library to avoid code duplication, check more details here: https://www.typescriptlang.org/tsconfig#importHelpers

Rollup config

Then, we need to create a rollup.config.js file that will contain the rollup configuration, it can be a bit long but the syntax is very simple to understand, basically, you're exporting a js array where each entry receives a input, output and plugins

import resolve from '@rollup/plugin-node-resolve';
import typescript from '@rollup/plugin-typescript';
import commonjs from '@rollup/plugin-commonjs';
import dts from 'rollup-plugin-dts';
import peerDepsExternal from 'rollup-plugin-peer-deps-external';

import packageJson from './package.json' assert { type: 'json' };

export default [
  {
    input: './src/index.ts',
    output: [
      {
        file: packageJson.main,
        format: 'cjs',
        sourcemap: true,
      },
      {
        file: packageJson.module,
        format: 'esm',
        sourcemap: true,
      },
    ],
    plugins: [
      peerDepsExternal(),
      commonjs(),
      resolve(),
      typescript({ exclude: ['**/__tests__', '**/*.test.ts'] }),
    ],
  },
  {
    input: './dist/index.d.ts',
    output: [{ file: packageJson.types, format: 'esm' }],
    plugins: [dts()],
  },
];

Updating the tsconfig.json file

Next, we need to update the tsconfig.json file to tell typescript that we only want the type files because all the other things will be handled by rollup

     "outDir": "dist",
     "moduleResolution": "node",
     "allowSyntheticDefaultImports": true,
-    "emitDeclarationOnly": false,
+    "emitDeclarationOnly": true,
     "strict": true,
   },
   "include": [
     "src"

Updating the package.json file

   "name": "typescript-test",
   "version": "1.0.0",
   "description": "",
+  "main": "dist/lib.js",
+  "module": "dist/lib.esm.js",
+  "types": "dist/lib.d.ts",
   "scripts": {
+    "build": "rollup --c"
   },
   "keywords": [],
   "author": "",

main It's the entry point for clients that are looking for commonjs modules
module It's the entry point for clients that are looking for es modules
types It tells typescript where the typescript types are located

Compiling and result

To compile we just need to run npm run build and we will get all the files

Problems

I'm still trying to figure out how to clean all the leftover typescript types that are left after dts bundle everything in one file

How to create a basic implementation of React Hook Form

Miguel Crespo — Wed, 09 Nov 2022 08:42:48 +0000

Introduction

I was introduced to React Hook Forms a couple of weeks ago by a coworker as he was explaining to me how they are using it in their team. My first thought was Who uses a library just for managing forms?, up until now I've just used useState or similar solutions to store the form state and I was fine with the results.

But then I started checking their documentation and found an interesting thing when you changed an input field it doesn't re-render the form…

In comparison this is a normal one:

How are they managing to do so? I wondered… So I started checking their code and this is how it works:

Why does a normal form gets re-rendered when you type a new value

First let's see a normal form, one that uses useState to store the value

function MyForm() {
    const [firstName, setFirstName] = useState("")

    return <form>
        <input name="firstName" value={firstName} onChange={(e) => setFirstName(e.target.value)} />
    </form>
}

Every time you type something you trigger a function that changes the value of firstName and thus trigger a re-render. The thing is that you don't need in most cases to store this value right away because this value is anyway stored in the HTML.

What is the React Hook Form approach?

React exports many hooks to us, like useState and useRef, the latest is usually popular to hold references to DOM elements, but it's more than that, you can think of the useRef hook as a way to have instance variables in a React component, this means that you can have the same value preserved across re-rendering and changing it won't trigger a re-render.

In that order of ideas, in most cases, it makes sense to store the values of a form in a ref and this is basically what React Hook Forms do.

Creating a simple replica of React hook forms

To illustrate better this idea, let's create a basic implementation of React Hook Forms, let's call it React Mini Forms 😛

Creating a useForm hook

The first thing we will need is to create a normal function that will be used as a hook. This function will return the following properties:

register(fieldName: string) This function will be called everything we want to add a new field to the form
handleSubmit() This function can be used to handle onSubmit in forms, it will return the current values of the fields in the form
watch(fieldName: string) In some specific cases, we want to do something when a certain field in a form changes, that's when we want to use this function

We will store the form values in a ref because as we said earlier, every time we change the value this won't retrigger a render, inside the useForm hook, let's put

// useForm.js
import { useRef, useState } from 'react'

export function useForm() {
    const state = useRef()
  const [_, setFormState] = useState({ renderCount: 0 }) // Will explain later

    // This will be the ref storing the current values of the fields
    if(!state.current) {
        state.current= {
            fields: new Map(),
            watching: new Map(),
        }
    }

    return {
        register: function(fieldName) {
        },
        handleSubmit: function() {
        },
        watch: function(fieldName) {
        }
    }
}

Consuming the hook

Now in our React Component, we will use the hook in the following way

import { useForm } from './useForm'

function App() {
    const { register, handleSubmit, watch } = useForm()

    console.log("The current value of the lastName is: ", watch("lastName"))

    const onSubmit = (formData) => {
        console.log("Data: ", formData)
    }

    return <form onSubmit={handleSubmit(onSubmit)}>
        <input {...register("firstName")} />
        <input {...register("lastName")} />

        <button>Submit</button>
    </form>
}

Implementing the register function

The register function as we said earlier will be the one in charge of adding new fields to the form state In this function, we will add the field to the list of fields of the form (If it's not already there) and we will return all the normal attributes of inputs like name, onChange, etc.

register: (name) => {
  if (!state.current.fields.has(name))
    state.current.fields.set(name)

  return {
    name,
    onChange: (event) => {
      state.current.fields.set(
        name,
        event.target.value,
      )
    }
  }
},

Implementing the watch function

This function was the hardest for me to understand, so here's a more basic implementation that works. We have a list of fields that the user is currently "watching". Every time it gets called it adds the field to the watching list if it's not already there and it returns the current value of the field

watch: (name) => {
  if (!state.current.watching.has(name)) {
    state.current.watching.set(name, true)
  }

  return state.current.fields.get(name)
},

Now we need to modify the register function so every time a value changes and the field is on the watch list, we will purposely call a setState to tell React that we need to re-render the component

register: (name) => {
    // ....

  return {
      // ...
    onChange: (event) => {
    if (state.current.watching.has(name)) {
        setFormState(state => ({
          renderCount: state.renderCount,
        }))
      }

    }
  }
},

Implementing the handleSubmit function

This is the easier function, it just returns the current value of all fields

handleSubmit: callback => e => {
  e.preventDefault()
  callback(state.current.fields)
},

The final code

All this code should be enough to have a working solution but remember that this is a basic implementation to demonstrate the idea, thus it doesn't have performance improvements and probably it doesn't handle all edge cases.

One thing to take into account when using this approach is that this could be more performance in a complex component as long as you're not subscribed to all the fields, if you are subscribed to all the fields a new render will happen every time anyway so a simpler solution would be better.

Let me know any comments you might have!

Here's the final code See it on Codesanbox 📦

import { useRef, useState } from "react";

export function useForm() {
  const state = useRef();
  const [_, setFormState] = useState({ renderCount: 0 });

  // This will be the ref storing the current values of the fields
  if (!state.current) {
    state.current = {
      fields: new Map(),
      watching: new Map()
    };
  }

  return {
    register: (name) => {
      if (!state.current.fields.has(name)) state.current.fields.set(name);

      return {
        name,
        onChange: (event) => {
          if (state.current.watching.has(name)) {
            setFormState((state) => ({
              renderCount: state.renderCount
            }));
          }

          state.current.fields.set(name, event.target.value);
        }
      };
    },
    watch: (name) => {
      if (!state.current.watching.has(name)) {
        state.current.watching.set(name, true);
      }

      return state.current.fields.get(name);
    },
    handleSubmit: (callback) => (e) => {
      e.preventDefault();
      callback(state.current.fields);
    }
  };
}

Another way to create modals in react using the HTML dialog element

Miguel Crespo — Tue, 25 Oct 2022 08:00:00 +0000

Introduction

Modals are a frequent UI element in our applications, in fact, I would say all the applications I have coded have some sort of modals on them.

And when it comes to how to code them, we usually tend to use a UI
element (Bootstrap, Material UI, etc.) or develop our own implementation
in plain React, but did you know that there's a standard HTML element
called dialog to display modals?

Why use the standard dialog element

Using the standard dialog element gives us several advantages over
doing our own implementation, advantages like:

It's widely available (92% of browsers support it at the moment)

accessibility out of the box

This element already implements the most basic fundamental of accessibility for HTML dialogs

standard HTML element

Which mean that this element is not going away anytime soon, thus making our application a bit more future-proof

less code

Because we won't need to use the aria-* properties anymore, and also we don't need custom logic to render the modal outside the root component by using a React.createPortal

Using dialog in your application

Adding the dialog element to our component

In our React component, we just need to add a dialog element with the content we want

<dialog>
  <form method="dialog">
    <section>
      <label htmlFor="name">Name: </label>
      <input name="name" placeholder="Type your name" />
    </section>
    <section>
      <label htmlFor="email">Email: </label>
      <input name="email" placeholder="Type your email" />
    </section>
    <menu>
      <button id="cancel">
        Cancel
      </button>
      <button type="submit">Submit</button>
    </menu>
  </form>
</dialog>

method="dialog"

Basically tells our application that when the form gets submitted, the modal needs to be close automatically

Accessing the dialog element on React

To do this, we will use the useRef hook

import React, { useRef } from "react";

function App() {
  const dialogRef = useRef();


  return (
      <dialog ref={dialogRef}>
          {/*...dialog content...*/}
      </dialog>
  )
}

Add a button to show the modal

Let's add a button in our React component that once is called it will call the showModal function to open the modal

import React, { useRef } from "react";

function App() {
  const dialogRef = useRef();

  const showModal = () => {
    dialogRef.current.showModal();
  }

  return (
      <>
        <dialog ref={dialogRef}>
            {/*...dialog content...*/}
        </dialog>

        <button onClick={showModal}>Show Modal</button>
      </>
  )

Handling form submit

Now the only things left is to handle the onSubmit event, so let's create a new function on the React Component to handle and get all the values from the form inside the modal

const onSubmit = (e) => {
  console.log({
    name: e.target.name.value,
    email: e.target.email.value
  })
}

Final code

Let's add a function to close the modal and that will be the final code

import React, { useRef } from "react";

function App() {
  const dialogRef = useRef();

  const openModal = () => {
    dialogRef.current.showModal();
  };

  const onCancel = () => {
    dialogRef.current.close();
  };

  const onSubmit = (e) => {
    console.log({
      name: e.target.name.value,
      email: e.target.email.value,
    });
  };

  return (
    <>
      <dialog ref={dialogRef}>
        <form method="dialog" onSubmit={onSubmit}>
          <section>
            <label htmlFor="name">Name: </label>
            <input name="name" placeholder="Type your name" />
          </section>
          <section>
            <label htmlFor="email">Email: </label>
            <input name="email" placeholder="Type your email" />
          </section>
          <menu>
            <button onClick={onCancel}>Cancel</button>
            <button type="submit">Confirm</button>
          </menu>
        </form>
      </dialog>

      <button onClick={openModal}>Show Modal</button>
    </>
  );
}

export default App;

Styling

To style our dialog we can use normal CSS properties, the only relevant and special thing is that if you want to change the background behind the modal you can use the pseudo-element ::backdrop, example:

dialog::backdrop {
  background: rgba(0, 0, 0, 0.7);
}

Polyfill

If you want to support browsers like Internet Explorer or old versions of Chrome, you will need to add this polyfill to your page:
https://github.com/GoogleChrome/dialog-polyfill

How to debug like a PRO using Neovim 🔥

Miguel Crespo — Wed, 05 Oct 2022 18:36:39 +0000

I've been using Neovim to write code since a couple of years, but I must admit that every time I wanted to debug something I would switch to VS Code or Firefox debugging tools (For Node and Javascript), but recently I decided to research how to set up Neovim to do it as well, I started digging into some documentations and found it so interesting that I ended up writing this article 😅.

Few editors are as extensible and powerful as Neovim, I believe everything you can think of can be implemented on it, including debugging (like a PRO).

In this article, we will learn how to prepare our Neovim to debug any kind of language using DAP (Debug Adapter Protocol) and in the process understand a bit better this technology. We will focus mainly on Golang, but the same knowledge can be applied to any other language with DAP support.

What is DAP

The Debug Adapter Protocol (DAP) is a standardized protocol introduced by Microsoft and used in Visual Studio Code that allows editors and IDEs (Tools) to outsource much of the logic that before had to be implemented in every editor to an intermediary called a Debug Adapter that can be reused across multiple tools, resulting in faster development times for editors' developers and better user experiences for normal developers as they can get similar top code assistance across multiple editors.

Before DAP

Before DAP, every time you developed a new editor, and you wanted to support code debugging inside of it, you had to take care of all the implementation details of features like:

breakpoints
watch expressions
step in, step out
etc

And besides this, you also had to take care of all the UI implementation, all of these of course was a lot of effort for tool's developers that some tools simple could not afford.

After DAP

Now with DAP editors' developers only have to focus on:

Integrating a DAP Client that will handle all the communication with the Debug Adapter
Focus on the UI development to send or show the data from the Debug Adapter

Installing a DAP Client

Currently, Neovim doesn't have a built-in DAP Client, as it does for the Language Server Protocol (LSP). So we need to manually install nvim-dap which is a normal Neovim plugin that will act as the DAP Client.

If you're using Packer, simply add to your plugin list:

use 'mfussenegger/nvim-dap'

Installing a debugger

In order to start debugging, you first need to install a debugger for the language you're using, for Go:

Installing **`delve*`* for `Golang**`

go install github.com/go-delve/delve/cmd/dlv@latest

Configuring the Debug Adapter

The next thing is to tell Neovim how to communicate with delve, you don't really need to install another plugin to do this, but there are multiple plugins that basically do most of the configuration of the adapter for us.

For Go, we can use nvim-dap-go which only require the following lines:

use 'leoluz/nvim-dap-go' -- Install the plugin with Packer
require('dap-go').setup()

What does nvim-dap-go actually do?

You can skip this section unless you want to understand what nvim-dap-go actually does.

This plugin basically tells how Neovim should launch the debugger (in our case delve) by using the API provided by the plugin nvim-dap

First it checks if delve is installed in the system, otherwise it returns an error
Then it starts delve and delve runs as a server
Then it tells nvim-dap that it can connect to delve by attaching to the server that was started in the previous step

A very simplistic version of this extension's code would be:

Tell nvim-dap how to connect to the running delve server

dap.adapters.go = function(callback, config)
  -- Wait for delve to start
    vim.defer_fn(function()
        callback({type = "server", host = "127.0.0.1", port = "port"})
      end,
    100)
end

Configure how the debuggee (our application) should be launched, in this case we're creating a configuration name Debug and telling nvim-dap to start debugging the current opened file

dap.configurations.go = {
    {
      type = "go",
      name = "Debug",
      request = "launch",
      program = "${file}",
    },
...

nvim-dap-go complete code

Defining configuration using `launch.json` file

If you're familiar with how VS Code works, you might know that you can provide the above configuration using a file called launch.json that is located in your project folder, if you prefer this method, this is also possible using nvim-dap!

Simply add the following line to your configuration:

require('dap.ext.vscode').load_launchjs(nil, {})

The first parameter is the path where you store the file, by default is .vscode/launch.json, but you can change it to whatever you want

Making the debugging interface amazing 💅!

Installing `nvim-dap-ui`

This is a nice package that will make the debugging much nicer, it basically puts all the DAP information into Neovim buffers.

To install it:

use { "rcarriga/nvim-dap-ui", requires = {"mfussenegger/nvim-dap"} }

Open automatically when a new debug session is created

local dap, dapui =require("dap"),require("dapui")
dap.listeners.after.event_initialized["dapui_config"]=function()
  dapui.open()
end
dap.listeners.before.event_terminated["dapui_config"]=function()
  dapui.close()
end
dap.listeners.before.event_exited["dapui_config"]=function()
  dapui.close()
end

Make the breakpoints look nicer

In your configuration, put this to change the ugly B text for these icons

vim.fn.sign_define('DapBreakpoint',{ text ='🟥', texthl ='', linehl ='', numhl =''})
vim.fn.sign_define('DapStopped',{ text ='▶️', texthl ='', linehl ='', numhl =''})

Set some keymaps

vim.keymap.set('n', '<F5>', require 'dap'.continue)
vim.keymap.set('n', '<F10>', require 'dap'.step_over)
vim.keymap.set('n', '<F11>', require 'dap'.step_into)
vim.keymap.set('n', '<F12>', require 'dap'.step_out)
vim.keymap.set('n', '<leader>b', require 'dap'.toggle_breakpoint)

What do you think?

What do you think about this article? Leave your comments 💬

How to render pretty code blocks using Contentful Rich Text Editor and Gatsby

Miguel Crespo — Wed, 28 Sep 2022 21:14:38 +0000

In a previous post we learned how to render YouTube videos using the Rich Text Editor from Contentful, in this post we will learn how to render pretty code blocks using the Rich Text Editor from Contentful, I will assume that you already know:

How to connect your Gatsby page to Contentful using the gatsby-source-contentful
How to render the basic content from a rich text editor, here's a tutorial

With that said, let's start!

Adding code snippets to your post content

We will use the built-in code option to insert our snippets to the posts, the problem I faced was that Contentful doesn't allow to specify the language of a snippet like the Github Flavored Markdown, so we cannot do things like this to manage highlighting:



``javascript
console.log("Hello World")
``

Instead, what I came up with was to use the first line of the snippet for metadata, like the language of the snippet, in that way the snippet above would be written like:



---
lang:javascript
console.log("Hello World")

Later in the code we will handle this metadata information, for now let's add it to the content of our post

Adding code highlighting to our blog

I was checking, and the most popular library by far seem to be PrismJS, it's easy to add to our Gatsby website, and it's easy to configure

Installation

To install it, you just need to run this

npm install prismjs babel-preset-gatsby babel-plugin-prismjs

Configuring PrismJS

We installed the babel-babel-preset-gatsby package to be able to tell prismjs which theme to use, and exactly which languages and plugins we want to support in our side and therefore not include unused features, for this we need to create a new file called:
.babelrc



{
  "presets": [
    "babel-preset-gatsby"
  ],
  "plugins": [
    [
      "prismjs",
      {
        "languages": [
          "javascript",
          "css",
          "markup"
        ],
        "plugins": [
          "show-language",
          "line-numbers"
        ],
        "theme": "tomorrow",
        "css": true
      }
    ]
  ]
}

You can see in the snippet above that the languages we will support in the blog are: javascript, css, markup
- To see a list of all the supported languages, check here
The plugins we support are show-language and line-numbers
- Again, if you want to see all the supported plugins, check their doc

Finally, enabling PrismJS in our application

The last step is simply to call Prismjs to start coloring all the <code> tags that it finds, I did it by calling the function inside a useEffect in the React Component that is rendering the blog post



useEffect(() => {
  // call the highlightAll() function to style our code blocks
  Prism.highlightAll();
}, []);

Rendering the right HTML tags

In the section of the code where we render the basic content of the Rich Text Editor, we should add a new option to the renderRichText function, this new option is just a property inside the key renderMark (A code block is a mark in the Contentful language), so it would look like this



renderRichText(post.body, {
  renderMark: {
    [MARKS.CODE]: (text) => {
      return (
        <pre>
          <code>{text}</code>
        </pre>
      );
    },
  },
});

In the code above, every time we find a code block inside the Contentful rich text editor content, we will render the text inside a <pre><code> tags, this is how Prismjs is expecting our code to be

Extracting the language from the code snippet

Now we want to extract the language from the first line of the code snippet, we will look o line to avoid weird behaviors later, and we will use the following regular expression ^lang:(\w+) to test the code snippet.

If the expression doesn't match, we'll render a basic code block
Otherwise, we will render the fancy Prismjs elements

Final code



renderRichText(post.body, {

  renderMark: {

    [MARKS.CODE]: (text) => {

      const regex = /^lang:(\w+)/;

// If the code snippet doesn't have the expected metadata

      if (!regex.test(text)) {

        return <code>{text}</code>;

      }

// Extract the language

      const language = regex.exec(text)[1];

// Remove the first line to avoid including metadata in the rendered version

      return (

        <pre>

          <code className={language-</span><span class="p">${</span><span class="nx">language</span><span class="p">}</span><span class="s2"> line-numbers}>

            {text.split("\n").slice(1).join("\n")}

          </code>

        </pre>

      );

    },

  },

});

Image

Extending Contentful Rich Text editor to render Youtube embed videos

Miguel Crespo — Sat, 24 Sep 2022 22:25:00 +0000

Introduction

I'm currently using Contentful CMS to manage the content of my Gatsby blog, and recently I wanted to display YouTube embed videos on it and found out that it is not directly supported to render videos from sources like YouTube when using the Rich Text Editor, I think this is functionality that should be implemented by default, but thankfully the Rich Text Editor is highly customizable and doing the necessary changes to display the videos in your blog post won't take long to implement.

Add a link to the video using the rich text editor

First, you need to ask the link to the YouTube video using the Rich Text Editor, it doesn't matter which format the URL is as we will parse it later in the code.

Modify the code to render the video

I'm using https://www.gatsbyjs.com/plugins/gatsby-source-contentful/ to connect my Gatsby blog with the Contentful Data, and this plugin already installs the @contentful/rich-text-react-renderer that we need in order to convert the normal content of a Rich Text Editor into HTML (Or React components in my case).

In the part of your code where you call the renderRichText function, you need to add a new value in your options object options:

{renderRichText(post.body, {
    renderNode: {
    [INLINES.HYPERLINK]: node => {
       ...

Then inside this function we have a regex expression to extract the video ID and generate a URL with the right format that we need to render embed videos https://youtube.com/embed/${videoId}

{renderRichText(post.body, {
    renderNode: {
    // If the node is a link
    [INLINES.HYPERLINK]: node => {
        // Only process youtube links
        if (node.data.uri.includes("youtube.com")) {
            // Extract videoId from the URL
            const match = /^.*((youtu.be\/)|(v\/)|(\/u\/\w\/)|(embed\/)|(watch\?))\??v?=?([^#&?]*).*/.exec(
                node.data.uri
            )
            const videoId =
                match && match[7].length === 11 ? match[7] : null
            return (
                videoId && (
                <section className="video-container">
                    <iframe
                        className="video"
                        title={`https://youtube.com/embed/${videoId}`}
                        src={`https://youtube.com/embed/${videoId}`}
                        allow="accelerometer; encrypted-media; gyroscope; picture-in-picture"
                        frameBorder="0"
                        allowFullScreen
                    />
                </section>
                )
            )
        }
    },
    },
})}

Make the embed YouTube video responsive

If you want the video to take the entire width of the container but also keep the ratio, you need to add the following styles to the markup

.video-container {
    position: relative;
    width: 100%;
    height: 0;
    padding-bottom: 56.25%;
}
.video {
    position: absolute;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
}

DEV Community: Miguel Crespo

Canceling requests in Javascript

Why you should cancel stale HTTP requests?

The API

Canceling a fetch request

Cancelling multiple fetch requests

Handling user cancelation errors

What about other libraries?

Axios

React query

Food for thought

Conclusion

How to write a neovim plugin in lua

What are we building?

The anatomy of a Neovim plugin

Our plan

How to load our extension

Execute code when neovim starts

Create a buffer using neovim APIs

Set the scratch buffer as the initial buffer

How can we publish the extension to the world?

Conclusion

All the code…

Configuring Neovim with Fennel

What is Fennel?

Why Fennel?

No global variables by default

Immutable variables by default

It doesn't declare variables with typos

Tables have a more familiar syntax

Destructuring

Macros

Picking a transpiler

Configuring Tangerine and hibiscus with lazy

Writing the configuration

Extra: Improving fennel DX in Neovim

Install a code formatter

Using brew

Install a LSP for fennel

Install conjure plugin

Conclusion

References

WebSockets 101

Intro

Connecting (Handshake)

Example in JavaScript

Message exchange

Example in Javascript

Closing

From the client's perspective

From the server:

Auth and security

Cross-Site WebSocket Hijacking

Resources

How to automate software installation and configuration on macOS using Homebrew and Stow

Installing software automatically

Apply saved configuration for each application

Stow usage example

Configuring macOS automatically

How to know the name of the variables

Extra: Installing software from the App Store

Putting all of this on a single file

Questions?

Debugging Javascript applications with Neovim

Introduction

What is possible?

My experience so far

Install nvim-dap

Install and configure nvim-dap-vscode-js

Some of the problems you might have with the previous configuration:

Configuring the DAP Adapter

Install and configure dapui

Debugging your first code

Debugging single Javascript files

Debugging server applications like express

Debugging a browser application

Sharing launch configuration with VS Code fans

If you have a Mac

My notes of how to compile a typescript library

Introduction

Debugging server applications like `express`

Using `tsc`

Defining configuration using `launch.json` file

Installing `nvim-dap-ui`