DEV Community: AdonisJS

Hot module replacement (HMR) in AdonisJS

AdonisJS — Wed, 24 Apr 2024 07:17:27 +0000

We're thrilled to announce that HMR is now available in AdonisJS! Before anything, let me quickly explain what this enables.

HMR allows us to modify the code of our controllers/middleware and their dependencies (services, repositories, models, etc.) without restarting the entire dev server each time. This results in a faster feedback loop, which in turn improves the developer experience.

I will first explain how to add HMR to your AdonisJS application, and then we'll go into more detail about why we felt the need to add HMR to AdonisJS and how it works under the hood. Grab a coffee - it's going to be a long read!

Adding HMR to your AdonisJS Application

To use HMR in AdonisJS, you must first upgrade @adonisjs/core and @adonisjs/assembler to the latest version and install hot-hook.

npm i -D hot-hook @adonisjs/assembler@latest
npm i @adonisjs/core@latest

Next, update your package.json file with the following configuration:

// title: package.json
{
  "hotHook": {
    "boundaries": [
      "./app/controllers/**/*.ts",
      "./app/middleware/*.ts"
    ]
  }
}

Then run the following command to start your dev server:

node ace serve --hmr

Note that you no longer need the --watch flag. Only --hmr is enough. If you are using npm scripts, you can update your dev script with the following:

// title: package.json
{
  "scripts": {
    "dev": "node ace serve --hmr"
  }
}

Now, try modifying a controller. You will see that the server does not restart, but the modifications will be considered: you'll always have the latest version of your code.

For more information, please consult the official documentation.

Backstory

Having HMR in a backend-first framework is rare. In fact, we didn't need it for all these years and followed the common wisdom of restarting the entire process after a file change.

However, restarting the entire process is slow. It takes a couple hundred milliseconds to restart the server (in a good-case scenario) and might take a second or longer (in worst-case scenarios).

On top of that, we recently made certain changes to the framework, which negatively impacted the server's boot time. These changes include:

Using ES modules. ESM is inherently slower than CommonJS.
Encouraging the use of TSX. TSX files are standard JavaScript modules, so we must restart the process whenever a TSX module changes.
Embedding the Vite dev server in the same process as the AdonisJS HTTP server. This means that every time we restart the AdonisJS process, we also have to reboot the Vite dev server.

These three changes are part of why we felt the need for HMR. Let's discuss each point in more detail.

ESM migration

Bun has written an article recently, where they compare the performance of CommonJS and ES modules.

As you can see, ESM takes 2.5 times longer to load than CommonJS. This benchmark was done on Bun, but the numbers are similar to those on Node.js. ESM takes longer to load because its design is more complex. In the Node.js documentation, you can find the resolution algorithm for these two module systems.

ESM resolution algorithm - https://nodejs.org/api/esm.html#resolution-algorithm
CommonJS resolution algorithm - https://nodejs.org/api/modules.html#all-together

So, migrating to ESM significantly slowed down the boot time of an AdonisJS application. We faced this problem last summer when we noticed that boot time began to explode as we migrated the packages.

This was particularly true on Windows, which has a very slow filesystem compared to Unix systems. On my Windows PC, I could reach up to 2/3 seconds of boot time, while the rest of the team on Mac had a boot time of 300/400ms.

Therefore, to solve part of this issue, which maybe some of you have noticed, we decided to bundle most of the AdonisJS packages with tsup, resulting in fewer files for Node.js to load, thus, a quicker boot time. By bundling most of the packages, we are able to speed up the process by 2 to 3 times.

TSX as a template engine

Now, let's talk about TSX. TSX (TypeScript cousin of JSX) files are standard JavaScript modules imported and executed by the Node.js runtime.

Therefore, modifying a TSX file means restarting the process to pick up new changes. This is where the boot time will start to annoy you. Adding a new class to a div and waiting a couple hundred milliseconds to verify the change is not a great developer experience.

The framework users, particularly Estéban, have raised this problem several times. Here's a discussion thread for your reference.

Vite

Finally, as explained in this article, we've changed our approach with Vite. To summarize, we are launching Vite directly in the main process of AdonisJS rather than as a subprocess.

This means that each time we restart the AdonisJS dev server, we also have to reboot Vite, which has significantly impacted boot time.

These are the main reasons that pushed us to implement HMR in AdonisJS. Keeping aside the specifics of AdonisJS, any large application with many dependencies will inevitably suffer from a significant boot time.

The slow boot time is not a problem in production (since the app boots only once). However, it can slow you down in development.

Existing solutions

Before developing our solution, we looked at what was happening elsewhere.

Hono / NestJS

NestJS proposes Webpack as a solution - So, if you want HMR with NestJS, you must compile your application with Webpack. I don't need to expand much on this because using Webpack to compile a backend application will come with its own set of problems.

Hono proposes Vite - Hono replaces Webpack with Vite, but the problem remains the same. Introducing a transpiler or a bundler to a backend application is not something we recommend.

The philosophy of AdonisJS has always been to keep things simple. "Use the platform and standard tooling" is our mantra. That's why these existing solutions didn't suit us.

Bun / Dynohot

Bun comes with inbuilt support for HMR, and there is Dynohot, a Node.js loader that enables HMR for applications using ES modules.

Both are great tools but rely on code transformations to achieve HMR. The following is an example of a module that is transformed by Dynohot to achieve HMR.

Here is the code written by you 👇

import { importedValue } from "./a-module";
export const exportedValue = "hello world";

console.log(importedValue);

Here is the transformed output 👇

import {
  acquire
} from ["hot:runtime"](hot: runtime);
import _a_module from ["hot:module?specifier=./a-module"](hot: module ? specifier = . / -a - module);

function* execute(_meta, _import) {
  let _$ = yield [
    next => {
      _$ = next
    },
    {
      exportedValue: () => exportedValue
    },
  ];
  yield;
  the exportedValue = "hello world";
  console.log(_$.importedValue());
}
module().load({
    async: false,
    execute
  },
  null,
  false,
  "module", {},
  [{
    controller: _a_module,
    specifier: "./a-module",
    bindings: [{
      type: "import",
      name: "importedValue",
    }],
  }],
);
export default function module() {
  return acquire("file:///main.mjs");
}

More or less, the same approach is followed by Bun as seen here.

Their approach allows them to have better control over HMR and do more things. However, this results in compiled code that radically differs from the original. And in the case of Dynohot, it forces you to install quite heavy dependencies like @babel/traverse and @babel/generator.

Again, to keep things simple, we needed a different approach.

Hot Hook

So, we finally decided to create our solution - Hot Hook. Hot Hook theoretically works with any framework and does not perform any static analysis or code transformations. It registers itself as a Node.js loader hook to intercept imports and perform its magic 😇.

Note
What is a loader hook? Loader hooks are actions that sit between your code and the Node.js ES modules loader implementation. You can use these actions to perform code transformations or rewrite import URLs.

For example, ts-node exposes a loader hook to compile TypeScript code to JavaScript before handing it over to the Node.js for execution.

Okay, now we know that Hot Hook is registered as a loader hook. Here's what happens under the hood.

We start by creating a dependency tree of modules, which looks similar to the following image.

This screenshot is generated using the @hot-hook/dump-viewer package.

We keep a version number for every module intercepted by Hot Hook. When you modify a file, Hot Hook detects this change (using a file watcher) and increments the version number associated with that file.

Then, the next time that same file is imported, Hot Hook will intercept the import and add the version number as a query string to the module file URL.

In a nutshell, the URL of the following import:

import('#controllers/users_controller')

Will be transformed into this.

import('#controllers/users_controller?version=2')

By doing this, we bypass the Node.js cache and import the latest version of the module.

To learn more about the internals, feel free to read the package's README.

Memory leaks

There's a problem with the query param approach to cache busting. There have been many discussions on this subject (nodejs/node#49442, nodejs/help#2806) and it is currently the only way to bust the cache in ESM (or bypass the cache to be technically correct). The problem with this solution is that it causes memory leaks.

Memory leaks sound like a scary term, and in fact, they are. However, in this case, they're not that bad for the following reasons.

The memory leak only happens during development because Hot Hook is not used in production.
These memory leaks are minor. In our internal testing, we noticed the memory consumption increasing by 10-20 MB after multiple days of continuous development.
Also, whenever you install a new package or modify a config file, we perform a full restart of the process. This will reset the memory consumed by the modules' cache.

Limitations

Currently, Hot Hook works only with dynamic imports and not with top-level imports. However, dynamic imports (known as boundaries) can have top-level imports, and they will benefit from HMR.

For example, if you import a controller using the import statement (as shown in the following example), it will not be hot reloaded.

// ❌ Cannot hot-reload this
import UsersController from '#controllers/users_controller'

router.get('/users', [UsersController, 'index'])

However, if we replace the import expression with a dynamic import using a function, then your controller and its imports will be hot reloaded.

- import UsersController from '#controllers/users_controller'
+ // ✅ Can be hot-reloaded
+ const UsersController = () => await import('#controllers/users_controller')

router.get('/users', [UsersController, 'index'])

How AdonisJS architecture is perfect for Hot Hook

As we have just seen, Hot-hook will only work with dynamic imports. Which ultimately is a good thing. Splitting your application with dynamic imports is a very good practice, and luckily, that's what we've been doing in AdonisJS for years.

Using top-level imports everywhere in your backend application will inevitably cause a problem, regardless of the framework you use. The problem is that Node.js will have to load your entire application from the start when you boot it, which can take a lot of time.

Meanwhile, with dynamic imports, all the code hidden behind these imports will be loaded by Node.js only when necessary.

In fact, in AdonisJS, we strongly recommend using dynamic imports for controllers (we even have an eslint rule that auto-fixes this). So, a UsersController (and all its dependencies; imagine a UserService and a UserRepository) will be loaded by Node.js ONLY when we call the /users endpoint. If we never call this endpoint, then the UsersController will never be loaded by Node.js.

The middleware, exception handler, event listeners, and bouncer policies follow the same approach. We lazy import every part of your codebase, so when using Hot Hook, you will not have to change a single line of code in your application.

Conclusion

That's all for this lengthy article!

We are super excited about HMR. It seems like we have got a perfect combination of simplicity and better developer experience with this addition.

We are looking forward to reading your feedback and hope you will enjoy HMR as much as we do. If you encounter any problems, don't hesitate to open an issue or come and discuss it on Discord.

Use TSX for your template engine

AdonisJS — Tue, 26 Mar 2024 14:47:01 +0000

Seven years ago, we decided to build Edge, our templating engine. We created it to ensure longevity, speed, and extensibility.

Edge is one of the best templating engines out there for Node.js. It is not restrictive; any JavaScript expression will work on it; it generates an accurate error stack, helping you debug your templating issue; it provides support for components with props and slots and is easily extendable. In fact, 80% of Edge is built using its public API.

We love Edge and working with it, but it brings a few caveats.

It is a "custom language". We need to have an extension for any IDE to have syntax highlighting. If we want to have type safety, we would need to create a complete LSP (Language Server Protocol) from scratch, which is out of the scope of this project. It also means prettier does not work out of the box with Edge; a custom plugin would be needed.

Those caveats may not be an issue for you, and you love working with Edge, but let me present you another alternative for those who would like better type-safety and IDE support.

JSX

JSX (JavaScript Syntax Extension) is an XML-like syntax extension to ECMAScript without any defined semantics developed by Facebook.

You may have already used it in React, Vue, Solid, or other frontend frameworks. While most of its use cases are inside React, JSX is not tied to it. It has a proper spec for anyone to create and use a parser.

The great part of JSX is its support. All IDE supports JSX; it already has an LSP, a prettier support, and even TypeScript has backed-in support.

It makes it a great candidate for a templating engine.

Examples

For those who have yet to use JSX or TSX (JSX on TypeScript file), let me show you some examples and what it may look like in your code.

First, everything is a component; you define them with a function, and the function returns JSX, which follows an HTML-like syntax.

export function Home() {
  return (
    <div>
      <h1>Hello World</h1>
    </div>
  )
}

Since this code resides in a tsx file, it can execute any JavaScript function like a standard function.

import { cva } from 'class-variance-authority'

interface ButtonProps { /* ... */ }

export function Button(props: ButtonProps) {
  const { color, children, size = 'medium', ...extraProps } = props

  const buttonStyle = cva('button', { /* ... */ }

  return (
    <button class={button({ color, size })} {...extraProps}>
      {children}
    </button>
  )
}

We use cva from the class-variance-authority package in this example.

We can leverage the Async Local Storage of AdonisJS to access the HttpContext anywhere in your template.

We recommend doing props drilling since using ALS will create a performance hit for your application.

export function Header() {
  const { auth } = HttpContext.getOrFail()

  if (auth.user) {
    return <header>Authenticated!</header>
  }

  return <header>Please connect!</header>
}

Setting up TSX

I have tried different packages to use TSX as a templating engine. For this tutorial, we will use @kitajs/html, but feel free to use the one you prefer.

First, you have to install the package. At the time of writing, this package is at version 3.1.1.

We will also install their plugin to enable editor intellisense.

npm install @kitajs/html @kitajs/ts-html-plugin

Once done, we will edit the bin/server.ts file to register Kita.

import 'reflect-metadata'
// insert-start
import '@kitajs/html/register.js'
// insert-end
import { Ignitor, prettyPrintError } from '@adonisjs/core'

// ...

We must also change our tsconfig.json file to add JSX support.

{
  // ...
  "compilerOptions": {
   // ...
   // insert-start
   "jsx": "react",
   "jsxFactory": "Html.createElement",
   "jsxFragmentFactory": "Html.Fragment",
   "plugins": [{ "name": "@kitajs/ts-html-plugin" }]
    // insert-end
  }
}

From now on, you can change the files' extension containing JSX from .ts to .tsx. For example, your route file may become routes.tsx.

Doing so will allow you to use JSX inside those files directly.

import router from '@adonisjs/core/services/router'
import { Home } from 'path/to/your/tsx/file'

router.get('/', () => {
  return <Home />
})

Preventing XSS Injection

When using JSX, you must be careful about XSS injection. You should always escape user input and never trust it.

Always use the safe attribute when rendering uncontrolled HTML.

export function Home(props) {
  const { username } = props

  return (
    <div>
      <h1>Hello World</h1>
      <p safe>{username}</p>
    </div>
  )
}

The @kitajs/ts-html-plugin package provides a script (xss-scan) to scan your code for potential XSS injection.

xss-scan --help

Updating Vite & RC file

You must update your adonisrc.ts and vite.config.ts files to change .edge references to .tsx.

// title: adonisrc.ts
export default defineConfig({
  // ...

  metaFiles: [
    {
      // delete-start
      pattern: 'resources/views/**/*.edge',
      // delete-end
      // insert-start
      pattern: 'resources/views/**/*.tsx',
      // insert-end
      reloadServer: false,
    },
    {
      pattern: 'public/**',
      reloadServer: false,
    },
  ],

  //...
})

// title: vite.config.ts
export default defineConfig({
  plugins: [
    adonisjs({
      /**
       * Entrypoints of your application. Each entrypoint will
       * result in a separate bundle.
       */
      entrypoints: ['resources/css/app.scss', 'resources/js/app.js'],

      /**
       * Paths to watch and reload the browser on file change
       */
      // delete-start
      reload: ['resources/views/**/*.edge'],
      // delete-end
      // insert-start
      reload: ['resources/views/**/*.tsx'],
      // insert-end
    }),
  ],
})

Sending HTML doctype

The <!doctype html> preamble is not added by default when using JSX. You can add it by creating a layout file and using a small "hack".

// title: resources/views/layouts/app.tsx
import { Vite } from '#start/view'
import type { Children } from '@kitajs/html'

interface LayoutProps {
  children: Children
}

export function Layout(props: LayoutProps) {
  const { children } = props

  return (
    <>
      {'<!DOCTYPE html>'}
      <html lang="en">
        <head>
          <meta charset="utf-8" />
          <meta name="viewport" content="width=device-width, initial-scale=1" />

          <title>BoringMoney</title>

          <Vite.Entrypoint entrypoints={['resources/css/app.scss', 'resources/js/app.js']} />
        </head>
        <body>{children}</body>
      </html>
    </>
  )
}

Adding global helpers

In Edge, some helpers are added by AdonisJS to make your life easier. For example, you may use the route helper to generate routes.

<a href="{{ route('posts.show', [post.id]) }}">
  View post
</a>

Again, since TSX files are JS files, you can simply define those functions anywhere in your codebase and then import them.

import router from '@adonisjs/core/services/router'

export function route(...args: Parameters<typeof router.makeUrl>) {
  return router.makeUrl(...args)
}

import { route } from '#start/view'

<a href={route('posts.show', [post.id])}>
  View post
</a>

Examples of some helpers

Here are some helpers you may want to add to your project.

Route Helper

This helper will allow you to generate URLs for your routes.

import router from '@adonisjs/core/services/router'

export function route(...args: Parameters<typeof router.makeUrl>) {
  return router.makeUrl(...args)
}

CSRF Field

This helper will generate a hidden input with the CSRF token.

:::note
We are using ALS in this example, but you can use any other way to access the HttpContext.
:::

import { HttpContext } from '@adonisjs/core/http'

export function csrfField() {
  // Note the usage of ALS here.
  const { request } = HttpContext.getOrFail()

  return Html.createElement('input', { type: 'hidden', value: request.csrfToken, name: '_csrf' })
}

Asset Path

Those helpers will generate the path to your assets. If you are in production, it will also add a hash to the file name.

import vite from '@adonisjs/vite/services/main'

function Image(props: { src: string; alt?: string; class?: string }) {
  const url = vite.assetPath(props.src)

  return Html.createElement('img', { src: url, alt: props.alt, class: props.class })
}

function Entrypoint(props: { entrypoints: string[] }) {
  const assets = vite.generateEntryPointsTags(props.entrypoints)

  const elements = assets.map((asset) => {
    if (asset.tag === 'script') {
      return Html.createElement('script', {
        ...asset.attributes,
      })
    }

    return Html.createElement('link', {
      ...asset.attributes,
    })
  })

  return Html.createElement(Html.Fragment, {}, elements)
}

export const Vite = {
  Entrypoint,
  Image,
}

import { Vite } from '#start/view'

<Vite.Entrypoint entrypoints={['resources/css/app.scss', 'resources/js/app.js']} />

Extending the typings

TSX will not allow you to use any non-standard HTML attributes. For example, if you are using unpoly or htmx, the compiler will complain about the up-* or hx-* attributes.

KitaJS comes with some typings for those attributes (htmx, Hotwire Turbo), but you may want to add your own.

To do so, you need to extend the JSX namespace.

declare global {
  namespace JSX {
    // Adds support for `my-custom-attribute` on any HTML tag.
    interface HtmlTag {
      ['my-custom-attribute']?: string
    }
  }
}

<div my-custom-attribute="hello world" />

Learn more about this in the KitaJS documentation.

Conclusion

I hope this article will help you decide if you want to use TSX as your templating engine. If you have any questions, feel free to ask them on our Discord Server or GitHub Discussion.

Announcing AdonisJS v6

AdonisJS — Wed, 24 Jan 2024 22:00:21 +0000

Alright, sit tight, as this will be a long article. The work for v6 started with the goal of moving to ESM and improving the IoC container to be simple and have fewer responsibilities.

But we have touched almost every part of the framework, smoothing out many rough edges, fixing some long pending issues, and rewriting some packages from scratch.

NOTE

Are you looking to migrate your applications from v5 to v6? Check out the migration-to-v6.adonisjs.com website for a complete list of breaking changes.

Also, we have created a migration CLI that can handle the majority of migration work for you.

Moving to ESM

ESM (ECMAScript Modules) vs CJS (CommonJS) might be a topic of debate among many JavaScript developers. But we are not here to discuss the merits and drawbacks of one or the other.

We went with ESM because it is part of the spec. Yes, CJS might live the entirety of this universe, but the fact that a project using CJS cannot easily import ESM modules is a big enough pain point for us.

Many prolific authors (whom we rely on) already started moving their packages to ESM. As a result, if we keep the AdonisJS source code in CJS, we cannot use the latest versions of their packages, which may also contain several security fixes.

Starting from v6, every new AdonisJS application will use TypeScript and ESM. Yes, you can still install and use packages written in CJS, as ESM allows that.

Stop relying on TypeScript compiler hooks

I am not a fan of hacking into tools, primarily when the code is written for public consumption. However, with AdonisJS v5, we hook into the TypeScript compiler API and rewrite the imports prefixed with the @ioc keyword to IoC container lookup calls.

For example, if you write the following import.

import Route from '@ioc:Adonis/Core/Route'

We will compile it to

const Route = global[Symbol.for('ioc.use')]('Adonis/Core/Route')

There are two problems with the above transformation.

We rely on the official compiler API. As a result, we cannot use other JIT tools like ESBuild or SWC written in other faster languages.
We have to inject a global IoC container variable to resolve the module from the container.

Do not worry if you do not understand the container usage in this example. We have removed all this magic, and imports in v6 are regular JavaScript imports.

If you have been using AdonisJS v5 for a long time, wonder what happened to @ioc prefixed imports. Remember, there were better ways to resolve dependencies from the container. We have found a much simpler way to resolve container dependencies in the form of container services.

Type-safe routes and controllers binding

Earlier, we used magic strings to bind a controller to a route. For example, This is how the route + controller usage looks in v5.

Route.get('posts', 'PostsController.index')

The 'PostsController.index' is a magic string because, for TypeScript, it has no real meaning and cannot detect and report errors.

Starting from v6, we no longer recommend using magic strings. You can directly import controllers and bind them on a route by reference. For example:

import PostsController from '#controllers/posts_controller'

router.get('posts', [PostsController, 'index'])

However, there was one nice thing about magic strings. They allowed us to import the controllers lazily. Since controllers import the rest of the codebase, importing them within the routes file impacts the application's boot time.

In v6, you can lazily import a controller by wrapping it inside a function and using dynamic import.

You can detect and automatically convert controller imports to a lazy import using our ESLint plugin.

const PostsController = () => import('#controllers/posts_controller')

router.get('posts', [PostsController, 'index'])

Type-safe named middleware reference

In AdonisJS v5, you reference the named middleware defined inside the start/kernel.ts file on a route as a string. For example:

Server.middleware.registerNamed({
  auth: () => import('App/Middleware/Auth')
})

Route
  .get('/me', () => {})
  .middleware('auth')

// Passing options
Route
  .get('/me', () => {})
  .middleware('auth:web,api')

Since we are referencing the middleware as a string and passing the options as a string, there is no type-safety.

Starting from v6, the named middleware are defined by reference using the middleware collection exported from the start/kernel.ts file.

export const middleware = router.named({
  auth: () => import('#middleware/auth_middleware')
})

import { middleware } from '#start/kernel'

router
  .get('/', () => {})
  .use(middleware.auth())

// Passing options
router
  .get('/', () => {})
  .use(middleware.auth({
    guards: ['web', 'api']
  }))

Type-safe AdonisRC file

We have moved from a JSON-based RCFile (.adonisrc.json) to a TypeScript-based RCFile (adonisrc.ts).

This change allows us to directly import service providers, commands, and preload files instead of defining their import path as strings.

As a result, TypeScript can detect and report broken imports. Also, you can have better IntelliSense when modifying the values in the RCFile.

Type-safe Event emitter

To make the event emitter type-safe, we define a list of known events as a TypeScript interface, and from thereon, the emitter API only allows dispatching and listening for known events.

interface EventsList {
  'user:registered': User,
  'http:request_completed': {
    duration: [number, number],
    ctx: HttpContext
  }
}

The ability to define an events list as an interface also exists with the older version of AdonisJS.

However, with v6, you can also define events as classes. Class-based events encapsulate the event identifier and the event data within the same class. The class constructor serves as the identifier, and an instance of the class holds the event data. For example:

// Defining event
import type User from '#models/user'
import { BaseEvent } from '@adonisjs/core/events'

export default class UserRegistered extends BaseEvent {
  constructor(public user: User) {} 
}

// Listening for class-based event
import emitter from '@adonisjs/core/services/emitter'
import UserRegistered from '#events/user_registered'

emitter.on(UserRegistered, function (event) {
  console.log(event.user)
})

// Dispatching class-based event
import UserRegistered from '#events/user_registered'

const user = new User()
UserRegistered.dispatch(user)

You can learn about class-based events in the documentation.

Vite integration for bundling frontend assets

Vite has become the de facto standard for building frontend applications. With this release, we ship an official integration for using Vite inside AdonisJS applications.

Also, we no longer recommend using Webpack Encore for new projects. However, we will continue to maintain this package for existing v5 applications.

New scaffolding system and codemods API

The scaffolding system and codemods API are used by the package creators to configure a package or by Ace commands to scaffold a resource.

We have written an in-depth guide on the same that you must read to learn more about it.

New validation library

The current validation module of AdonisJS has served us well, but it desperately needs some improvements. Right now:

It lacks a union data type. There is no way to validate a field as a string or a number.
The API to create custom rules is rough. We have witnessed many individuals struggling to create custom rules.
The state of the package codebase was not great. It made it harder for us to make big changes confidently. A significant rewrite was needed.

Finally, we developed a framework agnostic validation library called VineJS. VineJS will be the official validation system for AdonisJS v6.

VineJS is much faster than the version used in V5, and it's also more comprehensive. It makes it easy to create custom rules and schema types and validate complex schemas.

You can learn more about VineJS in our introduction live stream. https://www.youtube.com/watch?v=YdBt0s8NA4I

What happens to the old validator?

We will maintain the old validator (without bringing any features) you can use while migrating apps to v6. However, we recommend using VineJS for new applications.

New and more Framework agnostic packages

Lately, we have decided to extract/create more framework-agnostic packages (wherever possible) with their dedicated documentation websites.

As a result, we are happy to announce that you can use the following packages with any Node.js framework of your choice.

Japa - A backend focused testing framework for Node.js.
Edge - In the age of complex frontend libraries and frameworks, Edge embraces old-school server-side rendering. Edge is a simple, Modern, and batteries-included template engine for Node.js.
VineJS - VineJS is a data validation library for the Node.js runtime. It is at least 9 times faster than Zod and Yup.
Bento Cache - Bentocache is a robust multi-tier caching library for the Node.js runtime.
Verrou - Verrou is a library for managing locks in a Node.js application.

Improved documentation

There were some lapses in AdonisJS's documentation. For example, topics like IoC Container and Service providers were undocumented. Also, some of the guides should have been more comprehensive.

With AdonisJS v6, we have spent significant time covering all the framework aspects within the documentation. Following are some of the newly added topics.

Documented usage of IoC container and container services.
Dedicated reference section for Edge helpers, available commands, events, and exceptions list.
Extensive documentation for creating and testing commands using Ace.
Dedicated guide for extending the framework.
HTTP introduction guide explaining the flow of an HTTP request.
Dedicated guide for the BodyParser middleware.
Detailed guide for middleware, covering topics like mutating response, exception handling, and testing middleware in isolation.

Better testing experience

We keep testing as one of the top priorities of the framework. Not only do we pre-configure a testing environment for your applications, but we also ensure the core APIs for the framework are testing-friendly.

Starting from v6, we ship:

First-class assertion APIs for testing emitted events.
Ability to test ace commands, write assertions for logger output, and trap prompts.
Fake outgoing emails and write assertions against them.
Support for writing Browser tests using Playwright.
And, a dedicated VSCode extension to run Japa tests without leaving your code editor.

Changes to the folder structure

While 80% of the folder structure of a v6 application remains the same, we move from PascalCase to snake_case for naming files and folders.

Why snake_case? - Last year, I documented the rules and conventions I follow when writing code. I briefly talk about the reasons behind opting for snake_case.

NOTE

The snake_case naming convention is part of the official starter kits. However, you have the freedom to create and use custom starter kits with the naming conventions of your choice.

The rest 20% of folder structure changes include:

Move entry point files to the bin folder. You will no longer see server.ts and test.ts inside the application's root. These files are now inside the bin directory.
Remove the .adonisrc.json file in favor of the adonisrc.ts file. Learn more.
Rename the contracts directory to types.
Move the Controllers directory outside the Http directory. As a result, there is no Http directory in a v6 app. Controllers live directly inside the app directory.
Move the env.ts file from the project root to the start directory.

MJML support and additional mail transports

The @adonisjs/mail package now bundles additional transports for Resend and the Brevo mail services. Additionally, it contributes the @mjml Edge tag to write email content using the MJML markup language.

The contents of the following template will be auto-compiled to HTML on the fly when sending the email. Learn more

@mjml()
  <mjml>
    <mj-body>
      <mj-section>
        <mj-column>
          <mj-text>
            Hello World!
          </mj-text>
        </mj-column>
      </mj-section>
    </mj-body>
  </mjml>
@end

Sunsetting packages

The following packages are no longer used with a brand new AdonisJS v6 application.

Remove @adonisjs/encore in favor of Vite. However, the package is compatible with v6 and can be used until you decide to move to Vite.
Remove @adonisjs/validator in favor of VineJS. However, the package is compatible with v6 and can be used until you decide to move to VineJS.
Remove @adonisjs/sink in favor of the new scaffolding system and code mods API. No longer support v6 applications.
Remove @adonisjs/require-ts in favor of TSNode + SWC. No longer support v6 applications.
Remove @adonisjs/view in favor of directly using Edge.js. No longer support v6 applications.

Other changes

Following is the list of additional notable changes in the v6 release.

Support loading additional dot-env files other than the .env file. Learn more
The @adonisjs/logger package uses the latest version of Pino and supports defining multiple loggers. Learn more
Support for experimental partitioned and priority cookie options.
Remove support for serving static files from the framework core in favor of the new @adonisjs/static package.
Remove support for CORS from the framework core in favor of the new @adonisjs/cors package.

Ready to get started

Head to the official documentation to learn more about AdonisJS and create a new project. Or check out the Let's Learn AdonisJS 6 course from Tom at Adocasts.

Additional Links

AdonisJS v6 - One Step Closer to the Release

AdonisJS — Mon, 09 Oct 2023 18:14:41 +0000

On July 24th, we announced the availability of AdonisJS 6 in its "alpha state" exclusively for our sponsors to explore the framework and provide valuable feedback.

Since that moment, we have been working hard to refine the framework, engage with the alpha testers, and migrate more official packages to the v6 core. Today, we're thrilled to share our progress around what has been accomplished since the last update.

If you weren't aware of the new major release of AdonisJS, you could learn more about the anticipated changes in our previous article titled "What to Expect of AdonisJS 6?". This will give you a solid understanding of the exciting enhancements that await in AdonisJS v6.

NOTE
A common theme with AdonisJS v6 is de-coupling as many pieces from the framework and releasing them as framework-agnostic packages with their documentation. In this update, we continue to work towards that goal.

EdgeJS 6.0

EdgeJS is a simple, modern, battery-included template engine we've used since AdonisJS v4.

With the latest release of Edge, we have reworked some of the internals, making it twice as fast and ESM-only, all while maintaining its framework-agnostic nature.

We've also created a dedicated website, edgejs.dev, where you can find comprehensive documentation and all the information you need to explore this new version. Check out the upgrade guide to ease your transition to EdgeJS version 6.

Japa 3.0

Japa is a home-grown testing framework, again something we have used since AdonisJS v4.

Japa is one of the fastest testing frameworks in the Node.js ecosystem. It does not use transpilers, is significantly smaller than Vitest and Jest, and comes with many features, making testing an enjoyable experience for Node.js libraries and applications.

We introduced a significant update during this period, transitioning to Japa 3. This release incorporates several significant changes, including the shift to ESM-Only, all while retaining its framework-agnostic approach.

For comprehensive information about Japa and its latest version, visit japa.dev. If you've used this package in your Node.js project, check out the migration guide to smoothly transition to the latest version.

Lucid 19.0

Lucid is another home-grown package created and maintained by the AdonisJS core team. It is a SQL query builder and an Active Record ORM built on Knex.

Lucid 19 is a major release in which we migrate the package to ESM-Only and work smoothly with AdonisJS v6 apps. Just so you know, this release contains zero functional breaking changes. So, migrating to Lucid 19 will be a lot simpler.

Aligning with our commitment to creating framework-agnostic packages, you now have the flexibility to integrate Lucid into any of your Node.js projects.

The documentation for Lucid is not public yet because we shifted our focus toward migrating the rest of the packages. However, Lucid will soon have its dedicated documentation website.

Everything else

In addition to our major updates, we've migrated several other essential modules to seamlessly integrate with AdonisJS 6, ensuring they remain compatible without disrupting their APIs.

Ally

Our social authentication provider, Ally, empowers you to authenticate users via OAuth2 with third-party platforms.

I18n

Our Internationalization and Localization provider, I18n, facilitates building applications supporting multiple languages.

Redis

Our Redis provider simplifies working with the ioredis library, offering automatic management of multiple Redis connections and improved pub/sub system support for a smoother developer experience.

Vite

Introducing Vite, our new asset bundler built on Vite, now seamlessly integrated into your AdonisJS 6 projects.

Session

Our Session provider offers flexible management of user sessions, allowing various providers such as cookies, files, Redis, and memory storage methods.

Shield

Our security provider helps you secure your application with CSRF protection, CSP Policy definition, HSTS, X-Frame, and MIME-type sniffer.

What's Next?

As we move closer to the release of AdonisJS 6, we have some exciting plans on the horizon.

Firstly, we're working on migrating a few remaining modules, including Auth, Drive, Bouncer, and the Limiter module, to ensure a smooth migration experience between v5 and v6.

Additionally, we have the following packages in PoC (Proof of Concept). They still need to be completed so that we may release them post v6 official launch.

Transmit (Championed by Romain) - Transmit will be our official package to implement natively SSE (Server-Sent Events) in AdonisJS, allowing you to build real-time applications. Additionally, you will be able to broadcast events across multiple servers or instances using the inbuilt Redis transport layer.
Bentocache (Championed By Julien) - BentoCache will be a framework agnostic cache system for Node.js applications. BentoCache has multi-tier caching, support for multiple storage backends, Cache stamped protection, named caches, and more.

Finally, we are internally working towards a more transparent governance approach and will share the outcome with you once we have a plan.

To sum up. We're incredibly excited about these developments and look forward to sharing more updates with you soon. Together, we're shaping the future of AdonisJS, making it more powerful, accessible, and user-friendly than ever before. Stay tuned for all the incredible things yet to come!

—

Your core team

Use AdonisJS 6 Alpha today

AdonisJS — Mon, 24 Jul 2023 15:31:22 +0000

Note: Before we start. The Alpha release is available for sponsors only. You can learn more about the sponsorship program here.

Recently, we outlined in an article what to expect from our new major version, i.e., AdonisJS v6. We have received a lot of positive feedback on that article, and there is excitement in the community to use v6 as early as possible.

Therefore, we are announcing the Alpha release of AdonisJS v6 today. The Alpha release will provide you access to the following resources.

Updated documentation
Onboarding screencasts
Access to our internal project management dashboard
And the opportunity to join the monthly conference calls

Since we want to work closely with a small group during the Alpha phase, you have to be a sponsor of Harminder Virk (the creator of the framework) before you can access it.

// Detect dark theme var iframe = document.getElementById('tweet-1683375932810182656-377'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1683375932810182656&theme=dark" }

Let us assure you, AdonisJS is NOT becoming a paid framework. We will always live up to the ethos of open source and ensure stable releases are available publicly. However, at the same time, we want to work closely with individuals who deeply care for the framework and have been supporting us in this journey.

Sponsoring us brings you many benefits and helps ensure the framework's longevity.

Your core team

What to Expect of AdonisJS 6?

AdonisJS — Wed, 05 Jul 2023 15:46:11 +0000

If you have been following us on Twitter, you might have seen us talking about AdonisJS v6 - “A new major version of the AdonisJS framework”. We’ve spent four years in the AdonisJS v5 era, and it’s time to move to the AdonisJS v6 era with some fresh ideas.

But don’t worry. Migrating your apps to AdonisJS v6 will be easy. It might take a few hours or a couple of days, depending on the size of your application. We will also provide a command line tool to help move your code to the new version.

AdonisJS is a modular framework with a feature-rich core (i.e., the @adonisjs/core package) and a collection of officially maintained packages. Each package follows its own release cycle and semver versioning. We are significantly updating the framework core in this release (i.e., AdonisJS v6).

The main goal of this update is to work more closely with the platform. We want to use new features of JavaScript and Node.js, ensure your apps have better type safety, and keep improving the framework performance.

Updating minimum requirements

Because we’re using new features from Node.js, your system needs to have at least Node.js version 18 or higher. This version is the current Long-Term Support release.

Note You can easily manage and upgrade your Node.js version with tools like Volta.

Switching to ECMAScript Modules (ESM)

The most significant change in this update is that we’re moving to ECMAScript modules (ESM). Right now, AdonisJS uses and compiles to CommonJS (CJS) format. Moving to ESM lets us use new language features and stay updated with the ecosystem.

What does this mean for your app?

Your apps will be able to use the new platform features that work in ESM, like the Top-Level-Await statement and Node.js subpath imports.

The good part is ESM applications can load and use CommonJS packages. But not the other way around. Therefore moving to ESM is a must.

New asset bundler

Many of you have asked for this, so we’re excited to announce that AdonisJS 6 will include Vite as the default asset bundler.

Note: We will continue to maintain the Webpack Encore as well. Therefore, if you are not ready to move to Vite, you can use Webpack Encore.

Better documentation

There are some lapses in the current documentation of AdonisJS. For example, topics like IoC Container, Service providers, and Package development are undocumented. Also, the docs should be more comprehensive.

With AdonisJS v6, we have spent significant time covering all the framework aspects within the documentation. Here's a sneak-peak of some newly added topics.

The fundamentals section covers the usage of IoC Container, service providers, and package development in-depth.
Ace documentation is rewritten from scratch covering everything you need to know when creating custom ace commands.
The testing section includes documentation for browser testing (via Playwright) and command-line testing.

An important thing to note is that the documentation is a reference guide explaining how the framework works and the available APIs. We imagine the framework users reference documentation when they want complete information about a topic.

The documentation will not teach you how to build an app from scratch. For that, we plan to write step-by-step tutorials that teach you how to build a specific app using the framework.

In the meanwhile, let us introduce you to Adocasts and AdonisJS Mastery if you want tutorial-based learning content. Both of these platforms have been creating content on AdonisJS for years.

Changes to the import module names

As of today (i.e., AdonisJS v5), you will witness many imports prefixed with the @ioc keyword. For example, The router is imported from the @ioc:Adonis/Core/Route module and the Event emitter is imported from the @ioc:Adonis/Core/Event module.

The @ioc keyword is a convention we follow to communicate that this import will be resolved from the IoC Container of AdonisJS. When you compile your code from TypeScript to JavaScript, we use a TypeScript transformer to convert this import into an IoC Container lookup method call.

For instance, if you were importing the Route module like this.

import Route from '@ioc:Adonis/Core/Route

The compiled output will roughly look as follows.

const Route_1 = global[Symbol.for('ioc.use')]("Adonis/Core/Route")

Taking this approach has some downsides.

You must always compile your code using the TypeScript official compiler because the transformer was written for tsc only.
The import syntax is unfamiliar to a JavaScript developer's eye.
Developers had to write module definitions separately from the implementation.
Users cannot CTRL-click the import to see the implementation.

Starting with v6, we will remove @ioc keyword imports with regular ESM imports and still be able to look up dependencies inside the AdonisJS dependency container.

Here's what the router import looks like in both v5 and v6. In v6, if you CTRL+click on the router value, you will see a standard JavaScript module resolving the singleton router instance from the IoC Container and exporting it as a variable.

# In v5
import Route from '@ioc:Adonis/Core/Route'

# In v6
import router from '@adonisjs/core/services/router'

To conclude, in v6, there are no compiler magic converting imports to IoC Container lookup method calls. Everything is straightforward and easy to reason about.

You can learn more about the AdonisJS IoC Container used in v6 from the following links.

IoC Container package README file.
An article covering Why you need an IoC container?
Live stream We did showcase the AdonisJS IoC Container outside of an AdonisJS app.

New Encryption module

AdonisJS encryption module currently encrypts data using a single static algorithm, i.e., aes-256-cbc. We have received a few requests to support additional algorithms in the form of drivers.

AdonisJS v6 will allow you to register custom encryption drivers, and we will bundle drivers for the following encryption algorithms.

aes-256-cbc
aes-256-gcm
chacha20-poly1305

Changes to file system naming conventions

Moving forward, the AdonisJS official starter kits will use the snake_case format for naming files and folders. Currently (in v5), we use lowercase names for root-level directories and PascaleCase for sub-directories.

In the JavaScript ecosystem, there is no general agreement on naming conventions. Some sub-communities use PascalCase for files that export a class and snake_case or dash-case for files that export functions or objects.

We decided to take a simple approach and consistently name all files and folders without relying on what is exported from the file. The snake_case format is inspired from Dart, and Google TypeScript style guide.

To learn more about this decision, check out the article “List of rules and conventions I follow when writing code” by Aman Virk.

Flexible stub system

The scaffolding ace commands under the make namespace are used to create files with the initial boilerplate quickly. For example, You can use the make:controller command to create a controller, the make:model command to create a model, and so on.

Until now, the templates used by these commands were within the package's source code, and there was no way to customize them.

Moving forward (with V6), you can eject the scaffolding templates (stubs) to your application codebase and modify them per your requirements. Next time, when you run the make commands, AdonisJS will use the ejected template.

Here's what the controller template looks like.

{{#var controllerName = generators.controllerName(entity.name)}}
{{#var controllerFileName = generators.controllerFileName(entity.name)}}
---
to: {{ app.httpControllersPath(entity.path, controllerFileName) }}
---
// import { HttpContext } from '@adonisjs/core/http'

export default class {{ controllerName }} {
}

We use the generators object to define variables in the first two lines.
From lines 3-5, we use the YAML front matter to define the destination path of the file we are about to create. This will allow you to create controllers and models in any directory of your choice.
And finally, we define the initial content of the controller.

New Validation Library

The current validation module of AdonisJS has served us well, but it desperately needs some improvements. Right now:

It lacks a union data type. There is no way to validate a field as a string or a number.
The API to create custom rules is rough. We have witnessed many individuals struggling to create custom rules.
The state of the package codebase was not great. It made it harder for us to make big changes confidently. A significant rewrite was needed.

Finally, we developed a framework agnostic validation library called VineJS. VineJS will be the official validation system for AdonisJS v6.

VineJS is much faster than the version used in V5, and it's also more comprehensive. It makes it easy to create custom rules, schema types and validate complex schemas.

You can learn more about VineJS in our introduction live stream. https://www.youtube.com/watch?v=YdBt0s8NA4I

Note
You can continue to use the existing validation module in your v6 projects. There is no need to migrate immediately to VineJS.

In fact, you can use VineJS and the existing validation module together in an AdonisJS application. This will allow you to migrate one validator at a time.

Other Changes

Numerous other changes have been made for the release of AdonisJS 6, both in the core and in some modules.

Following are some highlights:

The VSCode extension has been completely revamped to work with AdonisJS 5 and AdonisJS 6. It introduces new features, such as Inertia support, component-as-tag support in Edge, and auto-fixable issues.
The new core will also use the latest version of pino, the logging package. This allows you to make the most of worker threads to enhance performance.
The new router will allow you to import controllers and bind them directly to a route. The string-based controller reference is still there but not recommended. Importing and binding controllers directly provides better type-safety and refactoring capabilities.
The new event emitter lets you declare events with classes for better type-safety and refactoring capabilities.
The new mail module will include drivers for sendingblue and resend.

All these updates are designed to help you write better, more efficient, and safer code with AdonisJS v6.

The present and the future

We have completed all the changes in the framework core and are ready to migrate official packages to work with AdonisJS v6.

We will not introduce any breaking changes in our official packages. The goal is to keep the breaking changes' surface area as small as possible and help you quickly migrate your apps to v6.

Conclusion

To conclude, the goal of AdonisJS v6 is to remove magic and stay up to date with the language and the platform changes.

We want to make AdonisJS a more robust, efficient, and developer-friendly framework. Trust us; we are so happy with v6 internally and super pumped to release it. Let me share screenshots of an internal conversation we had.

Finally, we thank everyone for using AdonisJS and for sharing their feedback. Having the motivation and energy to work on the framework comes directly from all of you.

Your core team