DEV Community: whiteand

Quartet 9: Allegro | Teaser

whiteand — Fri, 27 Mar 2020 21:33:09 +0000

I am going to release new version of Quartet 9: Allegro data validation library.

I wanted to create library that will have four principles:

simplicity
minimalism
similarity of schema and TypeScript type
performance

Evaluation of the results

For this type

interface Message {
  type: "subscribeAcknowledgement"
  channel: string // with "channel" inside
  data: {
    id: string
    symbol: string
    price: number
    count: number
    accustic: number
    size: number
    mid: number
    last: number
    lastSize: number
    network: number
    open: number
    high: number
    low: number
    volume: number
    interest: number
    change: number
    percents: number
    someNumber: number
    grade: number
    mark: number
    velocity: number
    length: number
    height: number
    width: number
  }
}

We have such validation function:

// validator: (value: any) => value is Message
const validator = v<Message>({
  type: "subscribeAcknowledgement",
  channel: v.and(v.string, v.test(/channel/)),
  data: {
    id: v.string,
    symbol: v.string,
    price: v.number,
    count: v.number,
    accustic: v.number,
    size: v.number,
    mid: v.number,
    last: v.number,
    lastSize: v.number,
    network: v.number,
    open: v.number,
    high: v.number,
    low: v.number,
    volume: v.number,
    interest: v.number,
    change: v.number,
    percents: v.number,
    someNumber: v.number,
    grade: v.number,
    mark: v.number,
    velocity: v.number,
    length: v.number,
    height: v.number,
    width: v.number
  }
})

Lets compare it with one of the fastest library ajv and with one of the most popular library joi.

Ajv Validator:

const ajvValidator = ajv.compile({
  type: "object",
  required: ["type", "channel", "data"],
  properties: {
    type: {
      type: "string",
      enum: ["subscribeAcknowledgement"]
    },
    channel: {
      type: "string",
      pattern: "channel"
    },
    data: {
      type: "object",
      required: [
        "id",
        "symbol",
        "price",
        "count",
        "accustic",
        "size",
        "mid",
        "last",
        "lastSize",
        "network",
        "open",
        "high",
        "low",
        "volume",
        "interest",
        "change",
        "percents",
        "someNumber",
        "grade",
        "mark",
        "velocity",
        "length",
        "height",
        "width"
      ],
      properties: {
        id: { type: "string" },
        symbol: { type: "string" },
        price: { type: "number" },
        count: { type: "number" },
        accustic: { type: "number" },
        size: { type: "number" },
        mid: { type: "number" },
        last: { type: "number" },
        lastSize: { type: "number" },
        network: { type: "number" },
        open: { type: "number" },
        high: { type: "number" },
        low: { type: "number" },
        volume: { type: "number" },
        interest: { type: "number" },
        change: { type: "number" },
        percents: { type: "number" },
        someNumber: { type: "number" },
        grade: { type: "number" },
        mark: { type: "number" },
        velocity: { type: "number" },
        length: { type: "number" },
        height: { type: "number" },
        width: { type: "number" }
      }
    }
  }
});

Joi Validator:

const joiSchema = Joi.object({
  type: Joi.equal("subscribeAcknowledgement").required(),
  channel: Joi.string()
    .pattern(/channel/)
    .required(),
  data: Joi.object({
    id: Joi.string().required(),
    symbol: Joi.string().required(),
    price: Joi.number().required(),
    count: Joi.number().required(),
    accustic: Joi.number().required(),
    size: Joi.number().required(),
    mid: Joi.number().required(),
    last: Joi.number().required(),
    lastSize: Joi.number().required(),
    network: Joi.number().required(),
    open: Joi.number().required(),
    high: Joi.number().required(),
    low: Joi.number().required(),
    volume: Joi.number().required(),
    interest: Joi.number().required(),
    change: Joi.number().required(),
    percents: Joi.number().required(),
    someNumber: Joi.number().required(),
    grade: Joi.number().required(),
    mark: Joi.number().required(),
    velocity: Joi.number().required(),
    length: Joi.number().required(),
    height: Joi.number().required(),
    width: Joi.number().required()
  })
});

And now, benchmark for this case:

joi                x   1,104 ops/sec ±15.76% (80 runs sampled)
Quartet 8          x   1,438 ops/sec ±9.27% (55 runs sampled)
ajv                x  78,991 ops/sec ±2.81% (83 runs sampled)
Quartet 9: Allegro x 149,552 ops/sec ±5.54% (78 runs sampled)

Why is it so fast? Let's look at it

validator.toString()

What we see is an actual result:

function validator(value) {
  if (value == null) return false
  if (value.type !== "subscribeAcknowledgement") return false
  if (!validator.custom(value.channel)) return false
  if (value.data == null) return false
  if (typeof value.data.id !== 'string') return false
  if (typeof value.data.symbol !== 'string') return false
  if (typeof value.data.price !== 'number') return false
  if (typeof value.data.count !== 'number') return false
  if (typeof value.data.accustic !== 'number') return false
  if (typeof value.data.size !== 'number') return false
  if (typeof value.data.mid !== 'number') return false
  if (typeof value.data.last !== 'number') return false
  if (typeof value.data.lastSize !== 'number') return false
  if (typeof value.data.open !== 'number') return false
  if (typeof value.data.high !== 'number') return false
  if (typeof value.data.low !== 'number') return false
  if (typeof value.data.volume !== 'number') return false
  if (typeof value.data.interest !== 'number') return false
  if (typeof value.data.change !== 'number') return false
  if (typeof value.data.percents !== 'number') return false
  if (typeof value.data.someNumber !== 'number') return false
  if (typeof value.data.grade !== 'number') return false
  if (typeof value.data.mark !== 'number') return false
  if (typeof value.data.velocity !== 'number') return false
  if (typeof value.data.length !== 'number') return false
  if (typeof value.data.height !== 'number') return false
  if (typeof value.data.width !== 'number') return false

  return true
}

My Architecture Failures and Solutions

whiteand — Tue, 28 May 2019 20:36:50 +0000

My Architecture Failures and Solutions

Prehistory

I have been working as a front-end developer for one year. On my first project there was an “enemy” backend. This is not a big problem when communication is established.

But in our case it was not the case.

We developed code that relied on the fact that the backend sends us certain data of a certain structure and content. While the backend was considered normal to change the contents of the responses - without warning. As a result, we spent hours trying to find out why a certain part of the site stopped working.

We realized that we needed to check what the backend returns before relying on the data it sent us. We created a task to research the validation of data from the frontend.

This study was entrusted to me.

I made a list of what I want to be in the tool that I would like to use to validate the data.

The most important selection points were the following items:

a declarative description (scheme) of validation, which is transformed into a function-validator, which returns true / false (valid, not valid)
low entry threshold;
similarity of validated data with a description of validation;
ease of integration of custom validations;
ease of integration of custom error messages.

As a result, I found many validation libraries by looking at the TOP-5 (ajv, joi, roi ...). They are all very good. But it seemed to me that for the solution of 5% of complex cases - they doomed 95% of the most frequent cases to be rather wordy and cumbersome.

So I thought: why not develop something that would suit me?
Four months later, the seventh version of my validation library quartet was released.
It was a stable version, fully tested, 11k downloads on npm. We used it on three projects in a campaign for three months.

These three months have played a very useful role. quartet showed all its advantages. Now there is no any problem with the data from the backend. Every time they changed the response - we immediately threw an error. The time spent finding the causes of bugs was reduced dramatically. There are practically no data bugs left.

But there were also disadvantages.

Therefore, I decided to analyze them and release a new version with corrections of all errors that were made during development.
On these architectural errors and their solutions will discuss below.

Architectural rake

"String"-typed schema language

I will give an example of the old version of the scheme for the person object.

const personSchema = {
  name: 'string',
  age: 'number',
  linkedin: ['string', 'null']
}

This scheme validates an object with three properties: name - must be a string, age - must be a number, link to account on LinkedIn - must either be null (if there is no account) or string (if there is an account).

This scheme meets my requirements for readability, similarity with validated data, and I think the entry threshold for learning to write such schemes is not high. Moreover, such a scheme can be easily written from the typescript type definition:

type Person = {
  name: string
  age: number
  linkedin: string | null
}

(As we see, the changes are more cosmetic)

When deciding what should be used for the most frequent validations (for example, those used above). I chose to use strings, like validator names.

But the problem with strings is that they are not available to the compiler or error analyzer. The ‘number’ string for them is not much different from ‘numder’.

Solution

I decided to remove from the quartet 8.0.0 the use of strings as names of validators inside the schema.

The scheme now looks like this:

const personSchema = {
  name: v.string
  age: v.number,
  linkedin: [v.string, null]
}

This change has two big advantages:

compilers or static analyzers of code - will be able to detect that the name of the method is written with an error.
Strings are no more used as an element of the scheme. This means that for them it is possible to allocate a new functional in the library, which will be described below.

TypeScript Support

In general, the first seven versions were developed in pure JavaScript. When switching to a Typescript project, it became necessary to somehow adapt the library for it. Therefore, type declaration files for the library were written.

But this was a minus - when adding functionality, or when changing some elements of the library, it was always easy to forget to update the type declarations.

There were also just minor inconveniences of this kind:

const checkPerson = v(personSchema) // (0)

// ...

const person: any = await axios.get('https://myapi.com/person/42')
if (!checkPerson(person)) { // (1)
  throw new TypeError('Invalid person response')
}
console.log(person.name) // (2)

When we created an object validator on the line (0). We would like to see after checking the real response from the backend on line (1) and handling the error. On line (2) for person to have type Person. But it did not happen. Unfortunately, such a check was not a type guard.

Solution

I made a decision to rewrite the entire quartet library into Typescript so that the compiler would check the library for its type consistency. Along the way, we add to the function that returns the compiled validator - a type parameter that would determine what type of type guard is the validator.

An example looks like this:

const checkPerson = v<Person>(personSchema) // (0)

// ...

const person: any = await axios.get('https://myapi.com/person/42')
if (!checkPerson(person)) {// (1)
  throw new TypeError('Invalid person response')
}
console.log(person.name) // (2)

Now on line (2) the person is of typePerson.

Readability

There were also two cases where the code was poorly read: checking for compliance with a specific set of values(checking enum) and checking the other properties of the object.

a) Enum check
Initially there was an idea, in my opinion a good one. We will demonstrate it by adding the field "sex" to our object.
The old version of the scheme looked like this:

const personSchema = {
  name: 'string',
  age: 'number',
  linkedin: ['null', 'string'],
  sex: v.enum('male', 'female')
}

The option is very readable. But as usual, everything went a little out of plan.
Having the enum announced in the program, for example:

enum Sex {
  Male = 'male',
  Female = 'female'
}

Naturally you want to use it inside the scheme. So that if one of the values changes (for example, ‘male’ -> ‘m’, ‘female’ -> ‘f’), the validation scheme also changes.

Therefore, enum validation is almost always recorded as:

const personSchema = {
  name: 'string',
  age: 'number',
  linkedin: ['null', 'string'],
  sex: v.enum(...Object.values(Sex))
}

That looks rather cumbersome.

b) Validation of rest properties of the object

Suppose we add another characteristic to our object — it may have additional fields, but all of them must be links to social networks — that means they must be either null or be a string.

The old scheme would look like this:


const personSchema = {
  name: 'string',
  age: 'number',
  linkedin: ['null', 'string'],
  sex: v.enum(...Object.values(Sex)),
  ... v.rest(['null', 'string']) // Rest props are string | null
}

Such way of description outlines the remaining properties - from those already listed. Using the spread-operator - rather confuses the person who wants to understand this scheme.

Solution

As described above, strings are no longer part of validation schemes. The validation scheme has only three types of Javascript values. Object - to describe the object validation scheme. Array to describe - several options for validity. Function (generated by the library or custom) - for all other validation options.

This provision made it possible to add functionality that allowed us to increase the readability of the scheme.

In fact, what if we want to compare the value with the string ‘male’. Do we really need to know anything other than the value itself and the ‘male’ string.

Therefore, it was decided to add values of primitive types as an element of the schema. So, where you meet the primitive value in the scheme, this means that this is the valid value that the validator created by this scheme should check. Let me give you an example:

If we need to check the number for equality 42, then we write it like this:

const check42 = v(42)

check42(42) // => true
check42(41) // => false
check42(43) // => false
check42('42 ') // => false

Let's see how this affects the person's scheme (without taking into account additional properties):

const personSchema = {
  name: v.string,
  age: v.number,
  linkedin: [null, v.string], // null is primitive value
  sex: ['male', 'female'] // 'male', 'female' are primitive values
}

Using pre-defined enums, we can rewrite it like this:

const personSchema = {
  name: v.string,
  age: v.number,
  linkedin: [null, v.string],
  sex: Object.values(Sex) // same as ['male', 'female']
}

In this case, the extra ceremoniality was removed in the form of using the enum method and using the spread-operator to insert valid values from the object as parameters into this method.

What is considered a primitive value: numbers, strings, characters, true,false, null and undefined.

That is, if we need to compare the value with them - we simply use these values themselves. And the validation library will create a validator that will strictly compare the value with those specified in the schema.

To validate the residual properties, it was chosen to use a special property for all other fields of the object:

const personSchema = {
  name: v.string,
  age: v.number,
  linkedin: [null, v.string],
  sex: Object.values(Sex),
  [v.rest]: [null, v.string]
}

Thus the scheme looks more readable. And more similar to definitions from Typescript.

Validator's binding to the function that created it.

In older versions, error explanations were not part of the validator. They are folded into an array inside the v function.

Previously, in order to get explanations of validation errors, it was necessary to have a validator with you (to perform a check) and a function v(to get explanations of invalidity). It all looked like this:

a) We add explanations to the schema.

const checkPerson = v({
  name: v('string', 'wrong name')
  age: v('number', 'wrong age'),
  linkedin: v(['null', 'string'], 'wrong linkedin'),
  sex: v(
    v.enum(...Object.values(Sex)),
    'wrong sex value'
  ),
  ... v.rest(
    v(
      ['null', 'string'],
      'wrong social networks link'
    )
  ) // Rest props are string | null
})

To any element of the schema, you can add an explanation of the error using the second argument of the compiler function v.

b) Clear the array of explanations.

Before validation, it was necessary to clear this global array in which all explanations were recorded during validation.

v.clearContext() // same as v.explanations = []

c) Validate

const isPersonValid = checkPerson(person)

During this check, if an invalidity was discovered, and at the stage of creating the scheme — it was given an explanation, then this explanation is placed in the global array v.explanation.

d) Error handling

if (!isPersonValid) {
  throw new TypeError('Invalid person response:' + v.explanation.join(';'))
} // ex. Throws ‘Invalid person response: wrong name; wrong age ’

As you can see there is a big problem.
Because if we want to use the validator
is not in the place of its creation, we will need to pass in the parameters not only it, but also the function that created it. Because it is in it that there is an array in which the explanations will be folded.

Solution

This problem was solved as follows: explanations have become part of the validation function itself. What can be understood from its type:

type Validator = (value: any, explanations?: any []) => boolean

Now if you need an explanation of the error, you are passing the array into which you want to add the explanations.

Thus, the validator becomes an independent unit. A method has also been added that can transform the validation function into a function that returns null if the value is valid and returns an array of explanations if the value is not valid.

Now the validation with explanations looks like this:

const checkPerson = v<Person>({
  name: v(v.string, 'wrong name'),
  age: v(v.number, 'wrong age'),
  linkedin: v([null, v.string], 'wrong linkedin')
  sex: v(Object.values(Sex), 'wrong sex')
  [v.rest]: v([null, v.string], 'wrong social network')
})

// ...
const explanations = []
if (!checkPerson(person, explanation)) {
  throw new TypeError('Wrong person:' + explanations.join(';'))
}

// OR

const getExplanation = v.explain(checkPerson)
const explanations = getExplanation(person)
if (explanations) {
  throw new TypeError('Wrong person:' + explanations.join(';'))
}

Afterword

I identified three prerequisites for which I had to rewrite everything:

The hope that people are not mistaken when writing strings
Using global variables (in this case, the v.explanation array)
Check on small examples during development - did not show the problems that arise when used in real large cases.

But I am glad that I conducted an analysis of these problems, and the released version is already used in our project. And I hope it will be useful to us no less than the previous one.

Thank you all for reading, I hope my experience will be useful to you.

Old problem, new solution

whiteand — Thu, 22 Nov 2018 19:13:47 +0000

Data Validation Practise for Frontend

If you want to know how to validate forms in Vue - this article, is not for you. You should use some standard Vue plugins such as vuelidate

My advice: validation schema must be placed in the component that will submit the form.

We often create software that depends on data from some third side(ex. API calls, Backend, Parent Component, ...), you need to be ready that data you get can have any shape and content. So we need to validate data, that we take from other places.

Solution Requirements
Solution
Validation of types
- Numbers
- Strings
- Other types
- Alternatives
Custom validation rules
Deep validation
- Deep Validation of Object
- Deep Validation of Array
Fixing of invalid data
Tracking
- Messages
- Errors
Additional Possibilities
Other Solutions
Contacts

Solution Requirements

For almost all solution there are more or less useful solutions. And for our problem we set these goals to be achieved:

Validation of types(number, object, array, string, null, undefined,...)
Custom validation rules;
Deep validation;
Fixing of invalid data:
- set default value;
- omit invalid.
Tracking:
- messages,
- errors;
Clear code
- Readable
- Modifiable

Solution

As one of the solutions that we can use to achieve this goals is quartet library.

These library based on this validation definition:

"To validate" is to prove that some data is acceptable for using.

From the definition we see that validation has only two possible results: "data is acceptable" and "data is not acceptable". In javascript we represent this value in such way:

Result	JS value
Data is acceptable	`true`
Data isn't acceptable	`false`

Let's see how do we use quartet to achieve goals described above.

Validation of Types

For testing types we can use default registered validators and custom functions.

// Import library
import quartet from 'quartet'
const v = quartet()

v - is a function that transforms schema into validation function. It takes two arguments

Validation schema (required)
Custom error (optional)

Validation schema is one of:

validation function (function that returns true or false)

names of registered validation functions

array of schemas of validation alternatives.

object like { key1: schemaForKey1, key2: schemaForKey2, ... }

Custom error is any javascript value(except undefined) or function that returns any javascript value. This value will be treated as explanation of schema validation error. And will stored in v.explanation. Example of usage see in Tracking section.

Validation of Numbers

const isNumber = v('number') // returns typeof value === 'number'
isNumber(0)         // true
isNumber(-1)        // true
isNumber(1)         // true
isNumber(1.2)       // true
isNumber(NaN)       // true
isNumber(Infinity)  // true
isNumber(-Infinity) // true

isNumber('1')             // false
isNumber(new Number(123)) // false

Checking of finite numbers (without NaN, Infinity, -Infinity)

// Lets put all values into array
// and find all values that are finite numbers
const numberLikeValues = [0, -1, 1, 1.2, NaN, Infinity, -Infinity, '1', new Number(123)]

// v('filter') is the same function as: value => Number.isFinite(value))
numberLikeValues.filter(v('finite')) // [0, -1, 1, 1.2]

Checking of integer numbers

// v('safe-integer') is the same function as: value => Number.isSafeInteger(value))
numberLikeValues.filter(v('safe-integer')) // [0, -1, 1]

Also we can check number sign:


// v('positive') is the same function as: x => x > 0
numberLikeValues.filter(v.and('positive', 'finite')) // [1, 1.2]

v.and(schema, schema2, schema3, ...) means that validated value must match schema AND schema2 AND schema3 and so on.

// v('negative') is the same function as: x => x < 0
numberLikeValues.filter(v.and('negative', 'number')) // [-1, -Infinity]

// v('negative') is the same function as: x => x < 0
numberLikeValues.filter(v.and('non-positive', 'finite')) // [0, -1]
numberLikeValues.filter(v.and('non-negative', 'safe-integer')) // [0, 1]

Also there is methods that returns number validation functions:

v.min(minValue);
v.max(maxValue);
v.enum(value, value2, ...) checks if validated value is one of passed values.

Let's use them to test rating value:

// v.min(minValue) for numbers is the same function as: x => x >= minValue
// v.max(minValue) for numbers is the same function as: x => x <= maxValue
const isRating = v.and('safe-integer', v.min(1), v.max(5))

isRating(1) // true
isRating(5) // true

isRating('2') // false
isRating(0) // false
isRating(6) // false

The same, but with using of v.enum

// v.enum(...values) is the same function as: x => values.includes(x)
const isRating2 = v.enum(1,2,3,4,5)

isRating2(1) // true
isRating2(5) // true

isRating2('2') // false
isRating2(0) // false
isRating2(6) // false

Validation of strings

const stringLikeObjects = [
  '',
  '123',
  new String('123'),
  Number('string')
]

// lets find only strings
stringLikeObjects.filter(v('string')) // ['', '123']

Also like for numbers there is additional registered validator for strings: 'not-empty':

stringLikeObjects.filter(v.and('not-empty', 'string')) // ['123']

There is also methods for creating string validation functions:

v.regex(regularExpression: RegExp);
v.min(minLength: number);
v.max(minLength: number).

Let's use them to check password (stupid passwords only)

const v = require('quartet')()

const isValidPassword = v.and(
  'string',                   // typeof x === 'string'
  v.min(8),                   // length >= 8
  v.max(24),                  // length <= 24
  v.regex(/^[a-zA-Z0-9]+$/),  // must contain only letters and digits
  v.regex(/[a-z]/),           // at least one small letter
  v.regex(/[A-Z]/),           // at least one big letter
  v.regex(/[0-9]/)            // at least one digit
)
console.log(isValidPassword('12345678'))         // false
console.log(isValidPassword('12345678Password')) // true

Validation of Other Types

You can use next registered validation functions in your validation schemas to check type.

name	condition
'boolean'	`x => typeof x === 'boolean'`
'null'	`x => x === null`
'undefined'	`x => x === undefined`
'nil'	`x => x === null
'object'	{% raw %}`x => typeof x === 'object'`
'object!'	`x => typeof x === 'object' && x !== null`
'array'	`x => Array.isArray(x)`
'symbol'	`x => typeof x === 'symbol'`
'function'	`x => typeof x === 'function'`

Alternatives

Sometimes there is need to validate data that can be different types.

You can use schema of alternatives to get such behavior:

// It is works exactly as OR operator in JS,
// if some of alternatives - true, it will return true immediately
v(['number', 'string'])(1) // true
v(['number', 'string'])('1') // true

v(['number', 'string'])(null) // false
v(['number', 'string'])(new String(123)) // false

v(['number', 'string', 'object'])(null) // true
v(['number', 'string', 'object'])(new String(123)) // true

Custom validation rules

As it was said before: validation function is one of
valid schemas. If you want to add your own rule - you just need to use your validation function as a schema.

const isPrime = n => {
  if (n < 2) return false
  if (n === 2 || n === 3) return true
  if (n % 2 === 0 || n % 3 === 0) return false
  for (let i = 5, j = 7; i * i <= n; i+=6, j+=6) {
    if (n % i === 0) return false
    if (n % j === 0) return false
  }
  return true
}
const isPrimeAndNotLessThan100 = v.and(
  'safe-integer',
  v.min(100),
  isPrime // validation function
)
isPrimeAndNotLessThan100(512) // false, 512 is NOT a prime number
isPrimeAndNotLessThan100(523) // true, 523 > 100, 523 is a prime number

Deep validation

Deep validation means validation of not primitive data structures.
Data structure can be accepted only if it has right type and all parts of it are valid.

The most popular data structures are object and array.

Deep Validation of Object

For validation of object quartet uses object schema.

Object schema is an object of a such structure

 {
  key1: schema1,
  key2: schema2,
  // ...
  // And if you need to validate other properties of object
  // You can use v.rest(schema) method (it returns an object that must be spreaded into object schema)
 ...v.rest(schemaAppliedToOtherValues)
}

Example:

// `v` treats object as an object
const isWorkerValid = v({
  name: v.and('not-empty', 'string'),
  age: v.and('positive', 'safe-integer)',
  position: v.enum(
    'Frontend Developer',
    'Backend Developer',
    'QA',
    'Project manager',
    'Grandpa'
  ),
  salary: v.and('positive', 'finite'),
  project: v.enum(
    'Shoutout',
    'FMEvents',
    'Jobla.co'
  ),
  // Any field can be object too
  skills: {
    JS: 'boolean',
    HTML: 'boolean',
    CSS: 'boolean',
    ...v.rest('boolean') // other keys must be boolean too
  }
})

Let's validate some object with using of this validation function

const worker = {
  name: 'Max',
  age: 31,
  position: 'Grandpa',
  salary: Math.random() * 3000,
  project: 'Jobla.co',
  skills: {
    JS: true,
    HTML: true,
    CSS: true,
    'C++ advanced': false,
    'GPU programming': false
  }
}
isWorkerValid(worker) // true

There is additional methods for dictionary object validation:

v.dictionaryOf(schema) - checks values of object;
v.keys(schema) - checks keys of object;
v.rest(schema) - if other properties will be present - they will be validated with using of the schema.

Example: Validation of dictionary object


const lowLettersDict = {
  A: 'a',
  B: 'b',
  C: 'c'
}
const isValidLettersDict = v.and(
  v.keys(v.regex(/^[A-Z]$/)),
  v.dictionaryOf(v.regex(/^[a-z]$/))
)
console.log(isValidLettersDict(lowLettersDict))

Let's check if keys correspond values with using of
custom validation function

// second parameter of all validation function is
// {
//   key: string|number,
//   parent: any
// }
// (if the parent is present)
function isValueValid (value, { key }) {
  return /^[A-Z]$/.test(key)        // upperCased key
    && /^[a-z]$/.test(value)        // lowerCased value
    && value === key.toLowerCase()  // correspond each other
}

const isValidLettersDict2 = v.dictionaryOf(isValueValid)

console.log(isValidLettersDict2(lowLettersDict)) // true
console.log(isValidLettersDict2({ A: 'b' })) // false, NOT CORRESPONDS
console.log(isValidLettersDict2({ b: 'b' })) // false, b is not UpperCased
console.log(isValidLettersDict2({ B: 'B' })) // false, B is not LowerCased

Deep Validation of Array

For deep validation of array we can use v.arrayOf(schema) method.

const arr = [1,2,3,4]
const invalidArrOfNumbers = [1,2,'3','4']

const isArrayValid = v.arrayOf('number')

isArrayValid(arr) // true
isArrayValid(invalidArrOfNumbers) // false

Also, we can combine array validation schema with object schemas

const isValidPointArray = v.arrayOf({
  x: 'finite',
  y: 'finite'
})
isValidPointArray([
  { x: 1, y: 2},
  { x: -1, y: 3},
  { x: 0, y: 0},
]) // true

And another way: object with array property:

const student = {
  name: 'Valera',
  grades: ['A', 'B', 'C','A', 'D', 'F']
}
const isStudentValid = v({
  name: 'string',
  grades: v.arrayOf(v.enum('A', 'B', 'C', 'D', 'E', 'F'))
})

isStudentValid(student) // true

Fixing of invalid data:

What if some validation errors we can fix. For example, we can replace invalid data with empty valid data. Also, sometimes we can omit invalid data. Or in rare keys - we should try to transform invalid data to valid.

In quartet there are methods for such task. Main method is

v.fix(invalidValue) => validValue

This method is used for applying all fixes that were collected during the validation. It doesn't change invalidValue but returns new value with applied fixes.

Methods v.default(schema, defaultValue), v.filter(schema) and v.addFix(schema, fixFunction) are decorators of validators. It means that they return new validation function that works exactly as passed schema, but with side effect of collecting of fixes.

Decorator	Fix effect, after calling `v.fix`
`v.default`	Replace value with defaultValue
`v.filter`	Removes value from parent
`v.addFix`	Custom fixFunction mutates parents of the value to fix an error

Example:

Let's create several validation functions with different effects.

const arr = [1,2,3,4,'5','6','7']

// Replaces all not numbers with 0
const isArrayValid = v.arrayOf(
  v.default('number', 0)
)

// Removes all not numbers from parent(array)
const isArrayValidFilter = v.arrayOf(
  v.filter('number')
)

// This function will be called on value in the clone of invalid data
// So this mutations - are safe.
function castToNumber(invalidValue, { key, parent }) {
  parent[key] = Number(invalidValue)
}

// casts all not numbers into numbers
const isArrayValidFix = v.arrayOf(
  v.addFix('number', castToNumber)
)

Let's use them to validate arr:

v.clearContext() // remove all fixes stored in `v`
isArrayValid(arr) // false
const validArr = v.fix(arr)
console.log(validArr) // [1,2,3,4,0,0,0]

v.clearContext() // remove previous fixes
isArrayValidFilter(arr) // false
const validArr2 = v.fix(arr) // [1,2,3,4]

v() // same as v.clearContext()
isArrayValidFix(arr) // false
const validArr3 = v.fix(arr) // [1,2,3,4,5,6,7]

// arr is not mutated
console.log(arr) // [1,2,3,4,'5','6','7']

NOTE: if there is "fix effect" on parent "fix effect" of children will not be applied(if fix effect on parent is not "filter effect").

It means we should use such rule: if there is a "fix effect", it must fix all invalidation of the value it must fix.
Example:

const isObjectValid = v({
  arr: v.default( // will be applied
    v.arrayOf(
      v.filter('number') // will not be applied
    ),
    [] // if there will be any not number - all array will be replaced with []
  )
})
const invalidObj = {
  arr: [1,2,3,'4']
}
v()
isObjectValid(invalidObj)
const validObj = v.fix(invalidObj) // { arr: [] }

Also there is v.hasFixes method: it returns true - if some fixes were collected, and ready to be applied. Returns false otherwise.

Tracking

Sometimes we need not only to check if a value is not valid,
But to get an explanation, and possibly to send this explanation to
the user, or to the logger etc.

In quartet we use explanations for it.

Explanation - any JS value(except undefined) you want that describes invalidation error.

We use the second parameter of v to add the effect of storing explanation, it can be either:

explanation;
a function that returns explanation.

We use them to collect error messages and errors into v.explanation array.

Messages

Sometimes we need only data to show to the user. And string explanation of the error is very useful.

Example:

const isValidPerson = v.and(
  v('object!', 'Person data structure is not an object'),
  {
    name: v.and(
      // required, checks if parent has such property
      v('required', 'name field is absent'), 
      v('string', 'Person name is not a string'),
      v('not-empty', 'Person with empty name, really?')
    ),
    age: v.and(
      v('required', 'age field is absent'),
      v('safe-integer', 'Person age is not an integer number'),
      v(v.min(18), 'Person has is not an appropriate age'),
      v(v.max(140), `It was just a healthy food`)
    )
  }
)

Let's use this schema to validate several persons

v.clearContext() // or v()
isValidPerson(null) // false
console.log(v.explanation) // ['Person data structure is not an object']

v.clearContext()
isValidPerson({}) // false
console.log(v.explanation)
/*
* [
* 'Name field is absent',
* 'age field is absent'
* ]
*/
v() // same as v.clearContext()
isValidPerson({ name: '', age: 969 })
console.log(v.explanation)
/**
* [
*   'Person with empty name, really?',
*   'It was just a healthy food'
* ]
*/

We can calculate explanation based on the invalidValue and it's parents.

Example:

const isValidPerson = v.and(
  v('object!', 'Person data structure is not an object'),
  {
    name: v.and(
      v('required', 'name field is absent'),
      v('string', 'Person name is not a string'),
      v('not-empty', 'Person with empty name, really?')
    ),
    age: v.and(
      v('required', 'age field is absent'),
      v('safe-integer', 'Person age is not an integer number'),
      v(v.min(18), age => `Your age: ${age} is to small`),
      v(v.max(140), age => `Your age: ${age} is to big`)
    )
  }
)

v() // same as v.clearContext()
isValidPerson({ name: '', age: 969 })
console.log(v.explanation)
/**
* [
*   'Person with empty name, really?',
*   'Your age: 969 is to big'
* ]
*/

Errors

The same way we use strings we can use objects as an explanation.

// Util for calculating code errors.
// If you want you can create your own type of errors.
const invalidValueToError = code => invalidValue => ({
  invalidValue,
  code
})

It will be useful to add some error codes.
We can use them to get messages sent to the user and other.

// Error Codes
const CODE = {
  PERSON_IS_NOT_AN_OBJECT: 'PERSON_IS_NOT_AN_OBJECT',
  NAME_ABSENT: 'NAME_ABSENT',
  NAME_IS_NOT_STRING: 'NAME_IS_NOT_STRING',
  NAME_IS_EMPTY: 'NAME_IS_EMPTY',
  AGE_ABSENT: 'AGE_ABSENT',
  AGE_NOT_INTEGER: 'AGE_NOT_INTEGER',
  AGE_TO_SMALL: 'AGE_TO_SMALL',
  AGE_TO_BIG: 'AGE_TO_BIG'
}

Schema with added using of the invalidValueToError function that returns function that calculates error explanation.

const isValidPerson = v.and(
  v('object!', invalidValueToError(CODE.PERSON_IS_NOT_AN_OBJECT)),
  {
    name: v.and(
      v('required',  invalidValueToError(CODE.NAME_ABSENT)),
      v('string',    invalidValueToError(CODE.NAME_IS_NOT_STRING)),
      v('not-empty', invalidValueToError(CODE.NAME_IS_EMPTY))
    ),
    age: v.and(
      v('required',     invalidValueToError(CODE.AGE_ABSENT)),
      v('safe-integer', invalidValueToError(CODE.AGE_NOT_INTEGER)),
      v(v.min(18),      invalidValueToError(CODE.AGE_TO_SMALL)),
      v(v.max(140),     invalidValueToError(CODE.AGE_TO_BIG))
    )
  }
)

Let's check some values and see what is stored in explanation

Not an object

v()
isValidPerson(null)
console.log(v.explanation)
//[
//  {
//   invalidValue: null,
//   code: 'PERSON_IS_NOT_AN_OBJECT'
//  }
//]

required fields explanation

v()
isValidPerson({})
console.log(v.explanation)
//[
//  {
//   invalidValue: undefined,
//   code: 'NAME_ABSENT'
//  },
//  {
//   invalidValue: undefined,
//   code: 'NAME_ABSENT'
//  }
//]

not valid values

v()
isValidPerson({ age: 963, name: '' })
console.log(v.explanation)
//[
//  {
//   invalidValue: '',
//   code: 'NAME_IS_EMPTY'
//  },
//  {
//   invalidValue: 963,
//   code: 'AGE_TO_BIG'
//  }
//]

All Together

Rarely, but it's possible to use explanations and fixes at one time.
For such goals, there is v.fromConfig method. That takes the config of the validation and returns validation function that has all set properties.

Example:

This is still the same

const invalidValueToError = code => invalidValue => ({
  invalidValue,
  code
})

// Error Codes
const CODE = {
  PERSON_IS_NOT_AN_OBJECT: 'PERSON_IS_NOT_AN_OBJECT',
  NAME_ABSENT: 'NAME_ABSENT',
  NAME_IS_NOT_STRING: 'NAME_IS_NOT_STRING',
  NAME_IS_EMPTY: 'NAME_IS_EMPTY',
  AGE_NOT_VALID: 'AGE_NOT_VALID'
}

Add using of v.fromConfig

const isValidPerson = v.and(
  v.fromConfig({
    validator: 'object!',
    // explanation if not object
    explanation: invalidValueToError(CODE.PERSON_IS_NOT_AN_OBJECT), 
    // If not valid store default fix (calculate default value)
    default: () => ({ name: 'unknown' })
  }),
  {
    // if several configs are passed, validations will be combined with `v.and`
    name: v.fromConfig(
      { 
        validator: 'required',
        default: 'a',
        explanation: invalidValueToError(CODE.NAME_ABSENT)
      },
      {
        validator: 'string',
        default: 'b',
        explanation: invalidValueToError(CODE.NAME_IS_NOT_STRING)
      },
      {
        validator: 'not-empty',
        default: 'c',
        explanation: invalidValueToError(CODE.NAME_IS_EMPTY)
      }
    ),
    age: v.fromConfig(
      { 
        validator: 'safe-integer',
        filter: true,
        explanation: invalidValueToError(CODE.AGE_NOT_VALID)
      },
      {
        validator: v.min(18),
        default: 18,
        explanation: invalidValueToError(CODE.AGE_NOT_VALID)
      },
      {
        validator: v.max(140),
        default: 90,
        explanation: invalidValueToError(CODE.AGE_NOT_VALID)
      }
    )
  }
)

null object

v()
const value = null
const test1 = isValidPerson(value)
const explanation = v.explanation
const fixedValue = v.fix(value)

console.log({
  value,        // null
  test1,        // false
  explanation,  // [{ invalidValue: null, code: 'PERSON_IS_NOT_AN_OBJECT' }]
  fixedValue    // { name: 'unknown' }
})

empty object

v()
const value2 = {}
const test2 = isValidPerson({})
const explanation2 = v.explanation
const fixedValue2 = v.fix(value2)

console.log({
  value2,  // {}
  test2,   // false

  // [
  //  { invalidValue: undefined, code: 'NAME_ABSENT' },
  //  { invalidValue: undefined, code: 'AGE_NOT_VALID' }
  // ]
  explanation2, 
  fixedValue2   // { name: 'a' }
})

wrong types

v()
const value3 = { age: '963', name: 1 }
const test3 = isValidPerson(value3)
const explanation3 = v.explanation
const fixedValue3 = v.fix(value3)

console.log({
  value3, // { age: '963', name: 1 }
  test3,  // false

  //[
  //  { invalidValue: 1,     code: 'NAME_IS_NOT_STRING' },
  //  { invalidValue: '963', code: 'AGE_NOT_VALID' }
  //]
  explanation3,
  fixedValue3    // { name: 'b' }
})

right type, wrong values

v()
const value4 = { age: 963, name: '' }
const test4 = isValidPerson(value4)
const explanation4 = v.explanation
const fixedValue4 = v.fix(value4)

console.log({
  value4,       // { age: 963, name: '' }
  test4,        // false

  //[
  // { invalidValue: 1,     code: 'NAME_IS_NOT_STRING' },
  // { invalidValue: '963', code: 'AGE_NOT_VALID' }
  //]
  explanation4,
  fixedValue4   // 
})

Valid data

v()
const value5 = { age: 21, name: 'Maksym' }
const test5 = isValidPerson(value5)
const explanation5 = v.explanation
const fixedValue5 = v.fix(value5)

console.log({
  value4,       // { age: 21, name: 'Maksym' }
  test4,        // true
  explanation4, // []
  fixedValue4   // { age: 21, name: 'Maksym' }
})

Clear code

Clear code is code that you can easily understand and modify

Readable

There are some features that make the code more readable:

object validation schema is the object with the same structure as an object that must be validated
text aliases for validation functions

Modifiable

There are some features that make the code more modifiable:

Easy to read sometimes means easy to modify.
methods names and structure - makes it easier to find the place of change
custom validation functions - allows you to make any kind of validation

Additional Possibilities

There is also several additional possibilities:

Method	Description
`v.example(schema, ...examples)`	If examples are not valid, it will throw Error. It can be used as documentation and testing of the shema. Returns validation function, if examples are valid
`v.validOr(schema, defaultValue)`	Returns function that takes `value` and replace it by `defaultValue` if the `value` is not value
`v.omitInvalidProps(objectSchema)`	Returns function that takes `value`. If value is not an object - returns unchanged. If `value` is object - it tests all props that present in `objectSchema` and removes all props that is invalid
`v.throwError(schema, errorMessage)`	returns function that takes `value`. Returns `value` if it's valid. Throws error otherwise. Can be used in pipe of functions.

Contacts

Author	Andrew Beletskiy
Position	Frontend Developer, Adraba
Email	AndrewBeletskiy@gmail.com
Github	https://github.com/whiteand

DEV Community: whiteand

Quartet 9: Allegro | Teaser

Evaluation of the results

My Architecture Failures and Solutions

My Architecture Failures and Solutions

Prehistory

Architectural rake

"String"-typed schema language

Solution

TypeScript Support

Solution

Readability

Solution

Validator's binding to the function that created it.

Solution

Afterword

Old problem, new solution

Data Validation Practise for Frontend

Contents

Solution Requirements

Solution

Validation of Types

Validation of Numbers

Validation of strings

Validation of Other Types

Alternatives

Custom validation rules

Deep validation

Deep Validation of Object

Deep Validation of Array

Fixing of invalid data:

Tracking

Messages

Errors

All Together

Clear code

Readable

Modifiable

Additional Possibilities

Other Solutions

Contacts