Chris Noring for ITNEXT

Posted on Jul 28, 2019 • Originally published at softchris.github.io

Joi JS Joi — awesome code validation for Node.js and Express

#tutorial #showdev #beginners #javascript

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

Validation of data is an interesting topic, we tend to write code that looks really horrible in the sense that it contains a lot of checks. There are different situations when we need to perform these checks like validating a response from a backend endpoint or maybe verifying what goes into our REST API won’t break our code. We will focus on the latter, how to validate our API.

Consider the below code that we might need to write when we don't have a validation library:

if (!data.parameterX) { 
  throw new Exception('parameterX missing') 
} 
try { 
  let value = parseInt(data.parameterX); 
} catch (err) { 
  throw new Exception('parameterX should be number'); 
} 
if(!/[a-z]/.test(data.parameterY)) { 
  throw new Exception('parameterY should be lower caps text') 
}

I think you get the idea from the above cringe-worthy code. We tend to perform a lot of tests on our parameters to ensure they are the right and/or their values contains the allowed values.

As developers we tend to feel really bad about code like this, so we either start writing a lib for this or we turn to our old friend NPM and hope that some other developer have felt this pain and had too much time on their hands and made a lib that you could use.

There are many libs that will do this for you. I aim to describe a specific one called Joi.

https://github.com/hapijs/joi

Throughout this article we will take the following journey together:

Have a look at Joi’s features
See how we can use Joi in the backend in a Request pipeline
Improve even further by building a middleware for Express in Node.js

Introducing Joi

Installing Joi is quite easy. We just need to type:

npm install joi

After that, we are ready to use it. Let’s have a quick look at how we use it. The first thing we do is import it and then we set up some rules, like so:

const Joi = require('joi'); 
const schema = Joi.object().keys({ 
  name: Joi.string().alphanum().min(3).max(30).required(),
  birthyear: Joi.number().integer().min(1970).max(2013), 
}); 
const dataToValidate = { 
  name 'chris', 
  birthyear: 1971 
} 
const result = Joi.validate(dataToValidate, schema); 
// result.error == null means valid

What we are looking at above is us doing the following:

constructing a schema, our call to Joi.object(),
validating our data, our call to Joi.validate() with dataToValidate and schema as input parameters

Ok, now we understand the basic motions. What else can we do?

Well, Joi supports all sorts of primitives as well as Regex and can be nested to any depth. Let’s list some different constructs it supports:

string, this says it needs to be of type string, and we use it like so Joi.string()
number, Joi.number() and also supporting helper operations such as min() and max(), like so Joi.number().min(1).max(10)
required, we can say whether a property is required with the help of the method required, like so Joi.string().required()
any, this means it could be any type, usually, we tend to use it with the helper allow() that specifies what it can contain, like so, Joi.any().allow('a')
optional, this is strictly speaking not a type but has an interesting effect. If you specify for example prop : Joi.string().optional. If we don't provide prop then everybody's happy. However, if we do provide it and make it an integer the validation will fail
array, we can check whether the property is an array of say strings, then it would look like this Joi.array().items(Joi.string().valid('a', 'b')
regex, it supports pattern matching with RegEx as well like so Joi.string().regex(/^[a-zA-Z0-9]{3,30}$/)

The whole API for Joi is enormous. I suggest to have a look and see if there is a helper function that can solve whatever case you have that I’m not showing above

Joi API

https://github.com/hapijs/joi/blob/v14.3.1/API.md

Nested types

Ok, so we have only shown how to declare a schema so far that is one level deep. We did so by calling the following:

Joi.object().keys({ });

This stated that our data is an object. Then we added some properties to our object like so:

Joi.object().keys({ 
  name: Joi.string().alphanum().min(3).max(30).required(),
  birthyear: Joi.number().integer().min(1970).max(2013) 
});

Now, nested structures are really more of the same. Let’s create an entirely new schema, a schema for a blog post, looking like this:

const blogPostSchema = Joi.object().keys({ 
  title: Joi.string().alphanum().min(3).max(30).required(),
  description: Joi.string(), 
  comments: Joi.array().items(Joi.object.keys({ 
    description: Joi.string(), 
    author: Joi.string().required(), 
    grade: Joi.number().min(1).max(5) 
  })) 
});

Note especially the comments property, that thing looks exactly like the outer call we first make and it is the same. Nesting is as easy as that.

Node.js Express and Joi

Libraries like these are great but wouldn’t it be even better if we could use them in a more seamless way, like in a Request pipeline? Let’s have a look firstly how we would use Joi in an Express app in Node.js:

const Joi = require('joi'); 
app.post('/blog', async (req, res, next) => { 
  const { body } = req; const 
  blogSchema = Joi.object().keys({ 
    title: Joi.string().required 
    description: Joi.string().required(), 
    authorId: Joi.number().required() 
  }); 
  const result = Joi.validate(body, blogShema); 
  const { value, error } = result; 
  const valid = error == null; 
  if (!valid) { 
    res.status(422).json({ 
      message: 'Invalid request', 
      data: body 
    }) 
  } else { 
    const createdPost = await api.createPost(data); 
    res.json({ message: 'Resource created', data: createdPost }) 
  } 
});

The above works. But we have to, for each route:

create a schema
call validate()

It’s, for lack of a better word, lacking in elegance. We want something slick looking.

Building a middleware

Let’s see if we can’t rebuild it a bit to a middleware. Middlewares in Express is simply something we can stick into the request pipeline whenever we need it. In our case, we would want to try and verify our request and early on determine whether it is worth proceeding with it or abort it.

So let’s look at a middleware. It’s just a function right:

const handler = (req, res, next) = { // handle our request } 
const middleware = (req, res, next) => { // to be defined } 
app.post( '/blog', middleware, handler )

It would be neat if we could provide a schema to our middleware so all we had to do in the middleware function was something like this:

(req, res, next) => { 
  const result = Joi.validate(schema, data) 
}

We could create a module with a factory function and module for all our schemas. Let’s have a look at our factory function module first:

const Joi = require('joi'); 
const middleware = (schema, property) => { 
  return (req, res, next) => { 
  const { error } = Joi.validate(req.body, schema); 
  const valid = error == null; 

  if (valid) { 
    next(); 
  } else { 
    const { details } = error; 
    const message = details.map(i => i.message).join(',');

    console.log("error", message); 
   res.status(422).json({ error: message }) } 
  } 
} 
module.exports = middleware;

Let’s thereafter create a module for all our schemas, like so:

// schemas.js 
const Joi = require('joi') 
const schemas = { 
  blogPOST: Joi.object().keys({ 
    title: Joi.string().required 
    description: Joi.string().required() 
  }) 
  // define all the other schemas below 
}; 
module.exports = schemas;

Ok then, let’s head back to our application file:

// app.js 
const express = require('express') 
const cors = require('cors'); 
const app = express() 
const port = 3000 
const schemas = require('./schemas'); 
const middleware = require('./middleware'); 
var bodyParser = require("body-parser"); 

app.use(cors()); 
app.use(bodyParser.json()); 
app.get('/', (req, res) => res.send('Hello World!')) 
app.post('/blog', middleware(schemas.blogPOST) , (req, res) => { 
  console.log('/update'); 
  res.json(req.body); 
}); 
 app.listen(port, () => console.log(`Example app listening on port ${port}!`))

Testing it out

There are many ways to test this out. We could do a fetch() call from a browser console or use cURL and so on. We opt for using a chrome plugin called Advanced REST Client.

Let’s try to make a POST request to /blog. Remember our schema for this route said that title and description were mandatory so let's try to crash it, let's omit title and see what happens:

Aha, we get a 422 status code and the message title is required, so Joi does what it is supposed to. Just for safety sake lets re-add title:

Ok, happy days, it works again.

Support Router and Query parameters

Ok, great we can deal with BODY in POST request what about router parameters and query parameters and what would we like to validate with them:

query parameters, here it makes sense to check that for example parameters like page and pageSize exist and is of type number. Imagine us doing a crazy request and our database contains a few million products, AOUCH :)
router parameters, here it would make sense to first off check that we are getting a number if we should get a number that is ( we could be sending GUIDs for example ) and maybe check that we are not sending something that is obviously wrong like a 0 or something

Adding query parameters support

Ok, we know of query parameters in Express, that they reside under the request.query. So the simplest thing we could do here is to ensure our middleware.js takes another parameter, like so:

const middleware = (schema, property) => { }

and our full code for middleware.js would, therefore, look like this:

const Joi = require('joi'); 
const middleware = (schema, property) => { 
  return (req, res, next) => { 
    const { error } = Joi.validate(req[property], schema); 
    const valid = error == null; 
    if (valid) { next(); } 
    else { 
      const { details } = error; 
      const message = details.map(i => i.message).join(',')
      console.log("error", message); 
      res.status(422).json({ error: message }) 
    } 
  } 
} 
module.exports = middleware;

This would mean we would have to have a look at app.js and change how we invoke our middleware() function. First off our POST request would now have to look like this:

app.post(
  '/blog', 
  middleware(schemas.blogPOST, 'body') , 
  (req, res) => { 
  console.log('/update'); 
  res.json(req.body); 
});

As you can see we add another argument body to our middleware() call.

Let’s now add the request who’s query parameters we are interested in:

app.get(
  '/products', 
  middleware(schemas.blogLIST, 'query'), 
  (req, res) => { console.log('/products'); 
    const { page, pageSize } = req.query; 
    res.json(req.query); 
});

As you can see all we have to do above is add the argument query. Lastly, let's have a look at our schemas.js:

// schemas.js 
const Joi = require('joi'); 
const schemas = { 
  blogPOST: Joi.object().keys({ 
    title: Joi.string().required(), 
    description: Joi.string().required(), 
    year: Joi.number() }), 
  blogLIST: { 
    page: Joi.number().required(), 
    pageSize: Joi.number().required() 
  } 
}; 
module.exports = schemas;

As you can see above we have added the blogLIST entry.

Testing it out

Let’s head back to Advanced REST client and see what happens if we try to navigate to /products without adding the query parameters:

As you can see Joi kicks in and tells us that page is missing.
Let’s ensure page and pageSize is added to our URL and try it again:

Ok, everybody is happy again. :)

Adding router parameters support

Just like with query parameters we just need to point out where we find our parameters, in Express those reside under req.params. Thanks to the works we already did with middleware.js we just need to update our app.js with our new route entry like so:

// app.js 
app.get(
  '/products/:id', 
  middleware(schemas.blogDETAIL, 'params'), 
  (req, res) =>  { 
    console.log("/products/:id"); 
    const { id } = req.params; 
    res.json(req.params); 
  }
)

At this point we need to go into schemas.js and add the blogDetail entry so schemas.js should now look like the following:

// schemas.js

const Joi = require('joi');

const schemas = { 
  blogPOST: Joi.object().keys({ 
    title: Joi.string().required(), 
    description: Joi.string().required(), 
    year: Joi.number() }), 
  blogLIST: { 
    page: Joi.number().required(), 
    pageSize: Joi.number().required() 
  }, 
  blogDETAIL: { 
   id: Joi.number().min(1).required() 
  } 
}; 
module.exports = schemas;

Try it out

The last step is trying it out so let’s first test to navigate to /products/abc. That should throw an error, we are only OK with numbers over 0:

Ok, now for a URL stating /products/0, our other requirement:

Also, that fails, as expected.

Summary

We have introduced the validation library Joi and presented some basic features and how to use it. Lastly, we have looked at how to create a middleware for Express and use Joi in a smart way.

All in all, I hope this has been educational.

Latest comments (32)

Hesham Eldawy • Nov 14 '21

Hello developers, can someone help me with this error plz. thnx in advance stackoverflow.com/q/69960084/15454800

IG:DjArc [Hey-R-C] • Mar 12 '20

Thank you Chris for a great article, I have a question about Unit testing.... The question is how do you Test the next() or how do you test these schemas in general? thank you again.

Chris Noring • Mar 12 '20

hi there. I usually like a mocking approach, cause we are talking middleware right? codewithhugo.com/express-request-r...

IG:DjArc [Hey-R-C] • Mar 12 '20

Hey Chris, thank you very much for that prompt reply. I am ACTUALLY please and surprised with your quickness. So I looked at the example you shared. I have to be honest I am still a novice to this testing life. So I have attached my module and test. Any pointer/guidance would be appreciated.

Here is the module

Here is the test

Thank you

Guille • Feb 1 '20

Thanks Chris! this was really helpful :)

Dmitry Erman • Jan 23 '20 • Edited

Great Article Chris. Covers something I've had in my code for a while. There is one more feature that you're not covering.

Joi returns sanitized values of parameters in a 2nd parameter. For example, you can have Joi validate (truthy, falsy, etc) that 'false', false, 0, and 'N' are all equivalent to false for your filter=N query parameter. It will then return false as a boolean. By default all query parameters come through as strings.

To apply the above to your code you would do something like this to manipulate the request object with the sanitized version of the objects:

const middleware = (schema, property) => { 
  return (req, res, next) => { 
  const { error, values } = Joi.validate(req[property], schema); 
  const valid = error == null; 

  if (valid) { 

    /**
    * Manipulate the existing request object with sanitized versions from Joi. This is an example
    * and I'm sure there are other and more efficient ways.
   **/
    if(value && Object.keys(value).length > 0){
        for(const prop in value){

          //maybe not necessary, but defensive coding all the way.
          if(value.hasOwnProperty(prop)){
            if(req.query && req.query[prop]){
              req.query[prop] = value[prop];
            }

            if(req.body && req.body[prop]) {
              req.body[prop] = value[prop];
            }

            if(req.params && req.params[prop]) {
              req.params[prop] = value[prop];
            }
          }
        }
      }
    next(); 
  } else { 
    const { details } = error; 
    const message = details.map(i => i.message).join(',');

    console.log("error", message); 
   res.status(422).json({ error: message }) } 
  } 
}

You can also append the values object to req.options = values that would allow you to have both. But then your code will need to know which one to grab.

Tuomo Kankaanpää • Dec 4 '19

This was very helpful post, thank you! :)

Chris Noring • Dec 4 '19

Thank you Tuomo :)

David • Sep 23 '19

Thank you!

O'Dane Brissett • Sep 15 '19

This an excellent Article. I was wonder how I was gonna validate route level instead on controller level. Saved me a ton of time.

Chris Noring • Sep 15 '19

Thank you for that. Glad it helped :)

António Avelar • Aug 5 '19 • Edited

I personally use Joi to validate my REST API routes. Each route has a controller, responsible for getting the user request parameters. That controller then passes those params to a service. On that service i have a Joi validation that looks like this:

   async function create(data) {
      return Joi.validate(data, schemas.accountCreation, async (err, value) => {
         if (err) {
            return {
               success: false,
               message: err.message,
               status: 400
            }
         }

         //service logic....        

      }
   }

Each validation is stored in a file that can be reused across all application/microservice. It looks like this:


module.exports = {
    user: {
        accountCreation: Joi.object().keys({
            firstname: Joi.string().min(1).required(),
            lastname: Joi.string().min(1).required(),
            email: Joi.string().regex(EMAIL_REGEX).email().required(),
            password: Joi.string().regex(/^[\x20-\x7E]+$/).min(8).max(72).required()
        }),
        authentication: Joi.object().keys({
            email: Joi.string().regex(EMAIL_REGEX).email().required(),
            password: Joi.string().regex(/^[\x20-\x7E]+$/).min(8).max(72).required()
        })
    },
    collection: {
        create: Joi.object().keys({
            collectionName: Joi.string().regex(ascii).min(1).max(30).required(),
            fields: Joi.array().min(1).required(),
            fieldsToShow: Joi.number().min(1).optional()
        }),
        getCollectionsByOwner: Joi.string().uuid().required(),
        getCollectionsById: Joi.string().uuid().required()
    },
    items: {
        create: Joi.object().keys({
            collectionId: Joi.string().uuid().required(),
            image: Joi.optional(),
            fields: Joi.object({}).min(1).required().unknown()
        }).unknown(),
        getById: Joi.string().uuid().required(),
        deleteItemByUUID: Joi.string().uuid().required()
    }
}

Chris Noring • Aug 5 '19

Thanks for sharing Antonio :)

Oluwaseun Paul • Jul 29 '19

Thanks A lot! great post!

Chris Noring • Jul 29 '19

Thank you :)

Dylan Landry • Jul 28 '19 • Edited

I've got it.
The organization which supports Joi is Hapijs. The repository, hapijs/joi sounds like happy joy.

I remember vaguely a song, something like "Happy happy joy joy; happy happy joy." Google reveals it is a song from the old 1996 television show Ren and Stimpy. Could there be a connection?

The hapi.js logo seems familiar, too. Google reveals this 2016 GitHub issue: New Logo?

"I think it's time. We are no longer using the Ren & Stimpy theme so maybe refresh the logo with something cleaner and more up to date?"

Ladies and gentlemen, we got him.

Chris Noring • Jul 28 '19

Wow.. Thanks for that added context Dylan :)

Carlos Caballero • Jul 28 '19

Thanks!

Joi is a essential element in web development today!

《MohitChauhan /》 • Jul 28 '19

Use of Typescript and html5 validation will be more adequate for such use cases

Jacob Evans • Jul 28 '19

Would TypeScript eliminate the need for Joi (I've used Joi before at work)?

I know Flow is similar to TypeScript so usually, people won't use both.

Yawar Amin • Jul 28 '19

You would get the greatest benefit by using a static typechecker like TypeScript or Flow in combination with a dynamic validator like Joi or Ajv. It works roughly like this:

Define a static type
Define a validator function for that type, using Joi or Ajv as the underlying validation tool
Make the validator function return the input value cast to the static type (if validation succeeded) or the validation error (if validation failed)

For example:

interface Person {
  id: string;
  name: string;
}

const personValidator: Validator<Person> = Validator(
  Joi.object().keys({id: Joi.string(), name: Joi.string()}),
);

...

const result = personValidator.validate(personObj);
if (result instanceof ValidationError) {...}
else {
  // Now we know 100% that result is a valid Person with id and name
}

Justin Gross • Jul 28 '19

I was thinking about Typescript while reading this too. In Typescript you have Typeguards which are basically a validation function (you write it, it's a normal function) which would, like Joi here, validate that the object is what you are expecting. During design time you're relatively safe, as a developer, from using the wrong type of object (because Typescript tslint will yell at you, and tsc will exception) and then during runtime you can validate using the Typeguards.

In my travels of JavaScript I realized I started writing more and more boilerplate and tons of extra code to address the fact that JavaScript is not typed and is functional first and OO second. When I finally tried Typescript I felt relieved that I could finally stop writing so much extra code to make up for the fact that JavaScript wasnt typed and didn't have OO as a first class paradigm. Also the IDE and tooling support is so amazing in Typescript.

They say JavaScript ate the world... Next will be Typescript.

Jacob Evans • Jul 28 '19

I actually plan on learning TypeScript 🤣😆 I just have to get over the setup and config excuse... I know once I do it ill be able too to spin it up faster the next times... Just being the bad kind of lazy, procrastinating.

The superset though is exciting and I look forward to all its tooling and power! VSCode is a prime example of the awesomeness TS can be utilized for.

Chris Noring • Jul 28 '19

hi Jacob. Would you benefit from an article that shows how you set up TypeScript + Jest + TS and shows a CI pipeline?

Jacob Evans • Jul 28 '19

Definitely. Especially if you can tie it into a project that already exists... I use React, Babel, Eslint, Parcel, Yarn if that helps at all.

Justin Gross • Jul 28 '19 • Edited

I've created a GitHub template for Typescript. It doesn't teach how to set up a new project and it is opinionated but if you wanted to play around with Typescript, unit testing, code coverage, dependency injection, auto-doc creation, hot-reload and dts rollups it's a ready to go template with a VS Code workspace to boot! Take it or leave.

github.com/JustinTime4Tea/ts-template

José Muñoz • Jul 28 '19

Great Post! I personally use Yup which is based on Joi but tailored for the frontend, smaller footprint and whatnot, its pretty much the same API and it is wonderful to work with, specially with Formik

slidenerd • Dec 17 '20

does yup work on the backend and frontend, also what is the difference between yup and joi in terms of functionality, is one superior to the other in any way apart from the footprint you mentioned