TypeScript Type Safety with AJV Standalone

Originally published on my blog: https://www.rehanvdm.com/blog/typescript-type-safety-with-ajv-standalone

In this blog we start of by exploring what Type means in TypeScript and how to achieve type safety at runtime. Then a quick comparison of the current tools to achieve this and deep dive into the implementation using the AJV Standalone method.

TL;DR TypeScript does a great job at compile time type safety, but we still need to do runtime checks just like in JavaScript. There are many packages and tools to help with this, we focused on AJV Standalone that outputs JS validation functions at compile time to be used at runtime. Going from TS Types to JSON Schema to JS functions allows us to validate TS Types where the other packages all work with classes and reflection.

The code referenced in this blog can be found in the example project here: https://github.com/rehanvdm/ajv-standalone-type-saftey.

TS in the land of type safety

TypeScript (TS) is a statically-typed safe language that outputs JavaScript(JS) when compiled. The type safety that TS provides at compile time allows for much higher quality projects than using plain JS.

TS will produce JS and then corresponding type definition files (.d.ts) that is used at compile time, many npm packages only expose these two types of files and not the actual TS file. The type definition files informs TS how to use the JS files in the absence of the actual TS file.

Something often forgotten is that TS, similar to JS, does not provide runtime type safety. It can not know that the JSON object being casted to Type A actually has the correct properties and definitions that fit Type A at compile time. This does not fall within the scope of TS goals. Runtime saftey has to be added just like any other JS project.

Ensuring runtime type safety

The following is just a quick rundown of the current tools available to help achieve runtime type safety.

1) Classes with decorators

The most popular implementations is class-validator paired withclass-transformer. It uses decorators on class properties to define the rules that will be evaluated at runtime.

export class Post {
  @Length(10, 20)
  title: string;

  @IsInt()
  @Min(0)
  @Max(10)
  rating: number;
}
let post = new Post();
post.title = 'Hello'; // should not pass
post.rating = 11; // should not pass

validate(post).then(errors => {
  // errors is an array of validation errors
});

The major downside of this approach is that decorators do not work interfaces and types, this forces you to represent every type as a class. You can not construct subtypes from other classes, leading to many duplicate unconnected types.

Composing types from other types using keywords like (Partial, Omit, Pick) is only possible with TS Types, this superpower is lost as soon as you force every type to be a class.

2) All types as classes

Then there are a set of packages that also forces you to represent every type as a class but instead of using decorators to define the rules, the properties of the class are other classes/functions. One of these packages is io-ts.

import * as t from 'io-ts'

const User = t.type({
  userId: t.number,
  name: t.string
})
const userData: Person = person.decode({userId: 1, name: "Piet"});

Composing types from other types is possible, but it gets hairy real quickly. Another library,fp-ts, is needed to do this composition and like io-ts has its own learning curve. Another downside is that we still can’t use the native TS Types and make use of their composability.

In many cases this approach is “technically” better and faster than the previous one that uses class decorators and reflection for validation. I say “technically” because it feels like it ignores the native Type in TypeScript and forces you to use a different type system (opinion).

3) AJV

Another Json Validator (AJV) uses JSON Schema to define the types. JSON Schema is a vocabulary that allows you to annotate and validate JSON documents. Both the data and the JSON Schema is passed to a validator that validates the rules as defined in the schema.

const schema = {
  type: "object",
  properties: {
    foo: {type: "integer"},
    bar: {type: "string"}
  },
  required: ["foo"],
  additionalProperties: false
}

const data = {foo: 1, bar: "abc"}
const valid = ajv.validate(schema, data)
if (!valid) 
  console.log(ajv.errors)

At first glance this does not seem ideal, since you have to define all your TS Types and their corresponding JSON Schemas. Fortunately there are tools that easily transform TS Types into JSON Schema for us, this is what we do in the example project.

One downside is that AJV transforms/compiles JSON Schema to an actual JS function and then caches it for future use. This means that the initial startup time can be significant, especially in short-lived environments like Lambda functions.

4) AJV Standalone

This is where AJV Standalone mode really shines, we save the generated validation function from the compiled output as an actual JS function at compile time. This function is then just imported/required at runtime and used as a standard JS function that validates the input.

Generating the JS Validation function

const fs = require("fs")
const path = require("path")
const Ajv = require("ajv")
const standaloneCode = require("ajv/dist/standalone").default

const schema = {
  $id: "https://example.com/bar.json",
  $schema: "http://json-schema.org/draft-07/schema#",
  type: "object",
  properties: {
    bar: {
      type: "string"
    },
  },
  "required": [
    "bar",
  ]
}

// For ESM, the export name needs to be a valid export name, it can not be `export const https://example.com/bar.json = ...;` so we
// need to provide a mapping between a valid name and the $id field. Below will generate
// `export const Bar = ...;`
// This mapping would not have been needed if the `$id` was just `Bar`
const ajv = new Ajv({schemas: [schema], code: {source: true, esm: true}})
let moduleCode = standaloneCode(ajv, {
  "Bar": "https://example.com/bar.json"
});

// Now you can write the module code to file, notice the file is saved as `.mjs` so that it can be imported as a module.
fs.writeFileSync(path.join(__dirname, "../consume/validate-esm.mjs"), moduleCode)

Consuming the JS Validation function

import {Bar} from './validate-esm.mjs'

const barPass = {
    bar: "something"
}

const barFail = {
    // bar: "something" // <= empty/omitted property that is required
}

let validateBar = Bar
if (!validateBar(barPass))
  console.log("ERRORS 1:", validateBar.errors) //Never reaches this because valid

if (!validateBar(barFail))
  console.log("ERRORS 2:", validateBar.errors) //Errors array gets logged

This means we would end up having multiple JS Types, their corresponding JSON Schema files and then the validation JS function created at compile time. It might seem a bit excessive to achieve runtime type safety, but it is the best option in my opinion. Here is why:

• We can use TS Types and make use of all the TS superpowers like type composition (Partial, Omit, Pick).
• Many other systems and tools can generate/export TS native types from entities like an OpenAPI (aka Swagger) file or a database schema. These types can then be consumed without changing any of their definitions.
• It is faster than most other implementations.

One downside is that an extra compile step is needed to transform the TS Type to JSON Schema to the JS Validation function.

Using AJV Standalone to make a TS project type safe

We will use the AJV Standalone method and multiple other tools to create a TS project that is both compile and runtime safe, as well as still providing a good developer experience (DX).

Project directory structure of example project

Generating the JSON Schema and JS validation function at compile time

Pseudo logic:

• Generate the JSON Schema files from the TS Types and save them as individual JSON Schema files using thetypescript-json-schema package, optionally can be committed.
• Create the AJV validation code in ESM format from the JSON Schema files, save this file.
• Optionally pass the generated code through ES Build to bundle and minify the function and all it’s dependencies into 1 file.
• Create the TS Definition file (typings) from the JS validation file using TSC so that the file can be imported in TS.

These are the basic steps that happen when you npm run build-types, this runs the build_types Gulp task by doing:gulp --color --gulpfile gulpfile.js build_types. Gulp is a build tool that runs JS code and the defined Gulp task build_typesruns the above pseudo logic by calling the CLI where necessary or calling the JS libraries. The same could have been done with a Bash script, but I always favor JS build/automation tools as they are cross-platform.

More notes (extra detail, might want to skip on first read):

• The typescript-json-schema package can only be pointed to a single directory and can not exclude certain paths. It will go through your whole node_modules folder if you point it at the project root. That is why I point it at a specific folder that contains all the project types.
• The generated JSON Schema file will be a single JSON Schema object with each type in the definitions array property. Each of these definitions are then moved into their own JSON Schema objects and saved to file. Only after replacing all the#/definitions/ within the file so that schemas can correctly reference each other (from within AJV) and also have valid $id fields which become the ESM exported name.
• Each of the JSON Schema files are then added to the AJV schemas before compiling the validation function.
• We add the ajv-formats when compiling the JSON schemas into functions because some types have a Date property which is represented in JSON Schema as { "type": "string", "format": "date-time" }. Adding these extra AJV formats, enable automatic identification and parsing of a string in ISO 8601 format into a JS Date type object.

Below is the Pseudo logic as implemented in the Gulp file, this can be skimmed for now as the runtime usage is more important than the generation.

function typeScriptToJsonSchema(srcDir, destDir)
{
  const config = {
    path: srcDir+"/**/*.ts",
    type: "*",
  };

  let schemas = [];
  console.time("* TS TO JSONSCHEMA");
  let schemaRaw = tsj.createGenerator(config).createSchema(config.type);
  console.timeEnd("* TS TO JSONSCHEMA");

  /* Remove all `#/definitions/` so that we can use the Type name as the $id and have matching $refs with the other Types */
  let schema = JSON.parse(JSON.stringify(schemaRaw).replace(/#\/definitions\//gm, ""));

  /* Save each Type jsonschema individually, use the Type name as $id */
  for(let [id, definition] of Object.entries(schema.definitions))
  {
    let singleTypeDefinition = {
      "$id": id,
      "$schema": "http://json-schema.org/draft-07/schema#",
      ...definition,
    };
    schemas.push(singleTypeDefinition);
    fs.writeFileSync(path.resolve(destDir+"/"+id+".json"), JSON.stringify(singleTypeDefinition, null, 2));
  }

  return schemas;
}

function compileAjvStandalone(schemas, validationFile)
{
  console.time("* AJV COMPILE");
  const ajv = new Ajv({schemas: schemas, code: {source: true, esm: true}});
  addFormats(ajv);
  let moduleCode = standaloneCode(ajv);
  console.timeEnd("* AJV COMPILE");
  fs.writeFileSync(validationFile, moduleCode);
}

function esBuildCommonToEsm(validationFile)
{
  console.time("* ES BUILD");
  esbuild.buildSync({
    // minify: true,
    bundle: true,
    target: ["node14"],
    keepNames: true,
    platform: 'node',
    format: "esm",
    entryPoints: [validationFile],
    outfile: validationFile,
    allowOverwrite: true
  });
  console.timeEnd("* ES BUILD");
}

async function generateTypings(validationFile, validationFileFolder)
{
  console.time("* TSC DECLARATIONS");
  await execCommand("tsc","-allowJs --declaration --emitDeclarationOnly \""+validationFile+"\" --outDir \""+validationFileFolder+"\"");
  console.timeEnd("* TSC DECLARATIONS");
}

async function buildTypes()
{
  let paths = {
    types: path.resolve(__dirname + "/types"),
    typesJsonSchema: path.resolve(__dirname + "/types/schemas"),
    validationFile: path.resolve(__dirname + "/types/schemas/validations.js"),
  };

  /* Clear the output dir for the AJV validation code, definition and JSON Schema definitions */
  clearDir(paths.typesJsonSchema);

  /* Create the JSON Schema files from the TS Types and save them as individual JSON Schema files */
  let schemas = typeScriptToJsonSchema(paths.types, paths.typesJsonSchema);

  /* Create the AJV validation code in ESM format from the JSON Schema files */
  compileAjvStandalone(schemas, paths.validationFile);

  /* Bundle the AJV validation code file in ESM format */
  esBuildCommonToEsm(paths.validationFile);

  /* Create TypeScript typings for the generated AJV validation code */
  await generateTypings(paths.validationFile, paths.typesJsonSchema);
}

gulp.task('build_types', async () =>
{
  await buildTypes();
});

The console output of the build_types task:

[20:51:57] Starting 'build_types'...
* TS TO JSONSCHEMA: 2.586s
* AJV COMPILE: 56.419ms
* ES BUILD: 76.501ms
* TSC DECLARATIONS: 2.995s
[20:52:03] Finished 'build_types' after 5.72 s

We also have an additional Gulp task that watches all the types in the type directory and recompiles the JSON Schema and validation function as soon as the types change.

gulp.task('watch_types', async () =>
{
  await buildTypes();
  gulp.watch(['types/*.ts'], async function(cb)
  {
    await buildTypes();
    cb();
  });
});

Consuming the JS validation function for our types at runtime

The Post and NewPost(not used atm) types in /types/Post.ts

export type Post = {
  title: string;
  description?: string;
  rating: number;
  createAt: Date;
}

export type NewPost = Omit<Post, "createAt">;

The Blog type in /types/Blog.ts

import {Post} from "./Post";

export type Blog = {
  site: string;
  about: string;
  email: string;
  // twitter: string; //Test watch command
  posts: Post[];
};

A basic example of using the validation function that was generated in /types/schemas/validations.js by the Gulp task. Here we create a blog with one valid post and then add another invalid post that does not have a createdAt property. This is possible because we cast the object as that type, the TSC does not complain because this is valid at compile time, but it will fail at runtime.

import {Post} from "./types/Post";
import {Blog} from "./types/Blog";
import {DateTime} from "luxon";
import {ValidateFunction} from "ajv";
import * as validations from './types/schemas/validations';

const blog: Blog = {
  site: "rehanvdm.com",
  email: "[e-mail]",
  about: "My blog, the one I never have time to write for but do it anyway.",
  posts: [{
    title: "Valid Post",
    rating: 5,
    createAt: DateTime.now().toJSDate()
  }]
};
let postInValid = {
  title: "Invalid Post! Missing createAt, forcing by casting",
  rating: 1
} as Post;
blog.posts.push(postInValid);

const validateBlog = validations.Blog as ValidateFunction<Blog>;
if(!validateBlog)
  throw new Error("Validate not defined, schema not found");

/* Casting to and from JSON forces the object to be represented in its primitive types.
* The Date object for example will be forced to a ISO 8601 representation which is what we want */
if(!validateBlog(JSON.parse(JSON.stringify(blog))))
{
  console.error(validateBlog.errors);
  console.error(JSON.stringify(validateBlog.errors));
  throw new Error("Blog not valid");
}

The magic happens here:

const validateBlog = validations.Blog as ValidateFunction<Blog>;

Where we cast the function that returns a boolean as a ValidateFunction<Blog>. The function still returns a boolean, but it will add an extra property, an errors array to the function prototype if it evaluated to false.

A more intuitive way of declaring types that are runtime safe can be achieved with a bit of abstraction. As in the following example:

import {Post} from "./types/Post";
import {Blog} from "./types/Blog";
import {DateTime} from "luxon";
import {ValidateFunction, ErrorObject} from "ajv";
import * as validations from './types/schemas/validations';

class TypeError extends Error {
  public ajvErrors: ErrorObject[];
  constructor(ajvErrors: ErrorObject[]) {
    super(JSON.stringify(ajvErrors));
    this.name = "TypeError";
    this.ajvErrors = ajvErrors;
  }
}

function ensureType<T>(
  validationFunc: ((data: any, { instancePath, parentData, parentDataProperty, rootData }?: {
    instancePath?: string;
    parentData: any;
    parentDataProperty: any;
    rootData?: any;
  }) => boolean),
  data: T): T
{
  const validate = validationFunc as ValidateFunction<T>;
  if(!validate)
    throw new Error("Validate not defined, schema not found");

  /* Casting to and from JSON forces the object to be represented in its primitive types.
   * The Date object for example will be forced to a ISO 8601 representation which is what we want */
  const isValid = validate(JSON.parse(JSON.stringify(data)));
  if(!isValid)
    throw new TypeError(validate.errors!);

  return data;
}

const blog: Blog = {
  site: "rehanvdm.com",
  email: "[e-mail]",
  about: "My blog, the one I never have time to write for but do it anyway.",
  posts: [{
    title: "Valid Post",
    rating: 5,
    createAt: DateTime.now().toJSDate()
  }]
};
let postInValid = {
  title: "Invalid Post! Missing createAt, forcing by casting",
  rating: 1
} as Post;
blog.posts.push(postInValid);

try
{
  /* Passes */
  let another: Post = ensureType<Post>(validations.Post, {
    title: "Quick way to ensure type is valid",
    description: "Just initiate differently like this",
    createAt: DateTime.now().toJSDate(),
    rating: 5
  });

  /* Fails, similar to the basic test */
  ensureType<Blog>(validations.Blog, blog);
}
catch (err)
{
  if(err instanceof TypeError)
    console.log("TYPE ERROR WITH STACK:", err.ajvErrors, err.stack);
  else if(err instanceof Error)
    console.log("ERROR:", err)
  else
    throw err;
}

We created a new error TypeError that is used by the ensureType function. This error adds the AJV errors aray to the JS error when it is thrown, we also get a full stack trace so we can easily identify the type that failed at runtime.

The ensureType function takes the validation function as generated by AJV and the unvalidated object as arguments. It uses a bit of generics to return the object when it is valid or throw the TypeError for when it failed. This means your syntax changes from:

  let another: Post = {
    title: "Quick way to ensure type is valid",
    description: "Just initiate differently like this",
    createAt: DateTime.now().toJSDate(),
    rating: 5
  };

To this:

  let another: Post = ensureType<Post>(validations.Post, {
    title: "Quick way to ensure type is valid",
    description: "Just initiate differently like this",
    createAt: DateTime.now().toJSDate(),
    rating: 5
  });

This is a very small change for the benefit of validation and runtime type safety. Another benefit is that you don’t need to change any of your already defined types, you just need to wrap the initialization of those types.

Conclusion (TL;DR)

TypeScript does a great job at compile time type safety, but we still need to do runtime checks just like in JavaScript. There are many packages and tools to help with this, we focused on AJV Standalone that outputs JS validation functions at compile time to be used at runtime. Going from TS Types to JSON Schema to JS functions allows us to validate TS Types where the other packages all work with classes and reflection.

Representing all your types as classes is not the solution, you lose a lot of TS superpowers (type composability) as soon as you do. It is crucial to do runtime checks against your Input and Outputs, the boundaries of the system. Doing internal runtime checks is optional but always welcome.

The code referenced in this blog can be found in the example project here: https://github.com/rehanvdm/ajv-standalone-type-saftey.

PS - The rabbit hole

This blog took quite a while to write(idea + code + blog), the code did not really do what I wanted it to. The AJV standalone functionality only exported the code with CJS (module.exports/require) and I needed it to be ESM (export/import) to correctly generate TS typings and for it to play nice with ES Build and TS. I made my first noteworthy PR to OpenSource to fix these issues:

• Option to generate ESM for standalone validators - https://github.com/ajv-validator/ajv/issues/1523
• Concrete example of using standalone at runtime - https://github.com/ajv-validator/ajv/issues/1831

These two PRs did the trick, the second one is the documentation.

• Add option to generate ESM exports instead of CJS - https://github.com/ajv-validator/ajv/pull/1861
• Concrete example of using standalone at runtime - https://github.com/ajv-validator/ajv/issues/1831

It was fun contributing to the AJV project, the project has great test coverage, guidelines and conventions.