DEV Community: Sergii Stotskyi

CASL. Pursuing Perfection II: New Engine

Sergii Stotskyi — Mon, 09 Nov 2020 22:25:32 +0000

This is the second part in the series of articles where I share my experience building and optimizing CASL v5:

CASL. Pursuing Perfection I: Why?
CASL. Pursuing Perfection II: New Engine
CASL. Pursuing Perfection III: Big O
CASL. Pursuing Perfection IV: Type Safety

First time you've heard about CASL? You may want to read "What is CASL?".

As I said in the previous article, to support SQL databases, CASL needed a new checking engine which can evaluate conditions in runtime and can transform them into any database query language. And this is why UCAST was born!

But, let's get deeper into what UCAST actually is

The harness

So, the task is to translate any language to any other language. Doesn't it sound familiar? Think for a moment, please.

If we ask Wikipedia "What is compiler?", we get:

In computing, a compiler is a computer program that translates computer code written in one programming language (the source language) into another language (the target language).

Aha! The task converts to writing a compiler that can translate MongoDB into JavaScript and SQL. There is a lot of theory around compilers, I knew I could read some of it but it would take a lot of time which I didn't have. That's why I used Wikipedia as a reference :)

So, according to Wikipedia:

A compiler is likely to perform many or all of the following operations: preprocessing, lexical analysis, parsing, semantic analysis (syntax-directed translation), conversion of input programs to an intermediate representation, code optimization and code generation.

Quite a lot right? Hopefully, not all are necessary. The most 3 important operations we need to concentrate at is:

parsing
conversion of input programs to an intermediate representation, usually Abstract Syntax Tree (AST)
code generation or AST interpreter (we do not always need to generate another code)

So, to translate MongoDB query to something else, it needs to be parsed into an intermediate representation (i.e., AST) which later can be consumed by a code generator (or an interpreter) to do some useful work.

And you know what? All these I've implemented in @ucast/* ecosystem.

Abstract Syntax Tree

In spite of the somewhat complex naming, Abstract Syntax Tree is a regular tree data structure of objects that contain information about parsed language.

There are 3 classes in @ucast/core package that is used to represent any boolean condition in AST:

FieldCondition represents a condition based on an object's field and operator (e.g., x === 3 or { x: 3 } in terms of MongoDB)
DocumentCondition represents condition that restricts a document or a row as whole (e.g., $where operator in MongoDB query language and EXISTS in SQL)
CompoundCondition represents a compound boolean operation (e.g., logical "and", "or", etc). This one aggregates other conditions in itself what allows us to represent complex expressions such as (x === 5 && x < 2) || (y > 0 && status === "available")

MongoDB query Parser

As we already know, the responsibility of parser is to transform code into AST. And this is exactly what MongoQueryParser class from @ucast/mongo package does. Basically, the result of its work of is a tree of FieldCondition, DocumentCondition and CompoundCondition objects. The really cool thing which I like about this class is that it's composite and consists of parsing instructions that allows us to:

To implement custom operators and extend our own MongoDB-like query language.
To restrict what operators can be used in our MongoDB-like query language, to prevent usage of complex conditions.
To use only pieces we need and get rid of unused code using JavaScript bundlers (e.g., rollup, webpack).

Let’s see a working example to understand how it works:

import { MongoQueryParser, $eq } from '@ucast/mongo';

const parser = new MongoQueryParser({ $eq });
const ast = parser.parse({
  authorId: 1
});

The parser above can parse only $eq operator, so if you try to use $lt for example, it will throw an error. The produced result is a single object of FieldCondition type with eq operator. $eq is actually a special operator which you need to pass in order to use POJO style query.

To learn more about MongoDB query parser, its optimization logic and customization, please refer to the README file of @ucast/mongo.

Interpreter or Code Generator?

UCAST uses word "interpreter" instead of "code generator" as it more clearly explains its purpose. For example, it may interpret it into JavaScript boolean value or into another language.

There are 2 packages that implements interpreter interface:

@ucast/js converts AST into boolean value
@ucast/sql converts AST into SQL string (also provides integration with major ORM libraries through sub modules)

An interpreter is designed in very similar way to a parser but instead of using parsing instructions, it consists of more granular interpreters (1 per operator). Frankly speaking, an interpreter is just a pure function that is composed of other pure functions:

import { createJsInterpreter, eq, lt, gt } from '@ucast/js';

const interpret = createJsInterpreter({ eq, lt, gt });

Later, we can use this function to interpret AST into boolean value. So, to mimic sift.js functionality all we need to do is to compose MongoDB query parser and JavaScript interpreter:

import { MongoQueryParser, allParsingInstructions } from '@ucast/mongo';
import { createJsInterpreter, allInterpreters } from '@ucast/js';

const parser = new MongoQueryParser(allParsingInstructions);
const interpret = createJsInterpreter(allInterpreters);
const ast = parser.parse({ authorId: 1, status: 'published' });

console.log(interpret(ast, { authorId: 1, status: 'published' })); // true
console.log(interpret(ast, { authorId: 2, status: 'published' })); // false

To reduce boilerplate of building MongoDB query language JavaScript runtime interpreter, I created a separate @ucast/mongo2js package which do this for us. @ucast/mongo2js is a drop-in replacement for sift.js and actually used by casl v5 to evaluate conditions in runtime! Moreover, it speeds up conditions evaluation by ~2x times!

The only difference between @ucast/mongo2js and sift.js is how they interpret equal operation on objects.

import { guard } from '@ucast/mongo2js';
import sift from 'sift';

const test = guard({ author: { id: 1 } });
const sifter = sift({ author: { id: 1 } });

console.log(test({ author: { id: 1 } })) // false
console.log(sifter({ author: { id: 1 } })) // true

By default, UCAST doesn’t check deep equality of objects but this can be changed by creating a custom guard function and custom compare function.

Refer to documentation of @ucast/js and @ucast/mongo2js for more details

Usually, you don’t even need such capability because it can be rephrased using dot notation which is supported by ucast as well:

const test = guard({ 'author.id': 1 });
const sifter = sift({ 'author.id': 1 });

console.log(test({ author: { id: 1 } })) // true
console.log(sifter({ author: { id: 1 } })) // true

All UCAST packages are written in TypeScript, so you additionally get type safety and hints in your IDE.

Conclusion

UCAST ecosystem is not only fast, lightweight but also very powerful! By implementing different parsers and interpreters, we can achieve outstanding results by combining one parser with different interpreters and many parsers with one interpreter.

For example, by implementing json-schema parser, we will be able to reuse existing interpreters and convert the result either to JavaScript boolean value or SQL query or MongoDB query or Cassandra query or REST query or GraphQL query or any query language you can imagine!

How do you feel about that? I’m excited.

Did I deserve a cup of coffee?

In the next article, I'll explain what JavaScript optimization technics allowed me to optimize Ability creation by more than 15x times! Stay tuned!

CASL. Pursuing Perfection I: Why?

Sergii Stotskyi — Wed, 28 Oct 2020 06:27:35 +0000

This is the first part in the series of articles where I plan to share my experience building and optimizing CASL v5:

CASL. Pursuing Perfection I: Why?
CASL. Pursuing Perfection II: New Engine
CASL. Pursuing Perfection III: Big O
CASL. Pursuing Perfection IV: Type Safety

First time you've heard about CASL? You may want to read "What is CASL?".

Why?

The long standing issue regarding SQL integration was created 2 months after the initial CASL's release and was not addressed for years. To understand why and why it was a challenge, we need to go back to the days when CASL was designed.

A bit of history

CASL was heavily inspired by cancan ruby gem. This gem provides 3 ways to define conditions for rules:

hash maps, \ can be used for runtime checks and can be transformed to SQL query
ruby blocks, \ similar to lambdas in other languages, only runtime checks
ActiveRecord::Relation and raw SQL queries

Why CASL did not inherited the name of "cancan" is a different story but if you are curious, just read here.

Values in hash maps are interpreted as "equal" operation, so { author_id: 1 } is transformed to post.author_id == 1 in runtime and to author_id = 1 in SQL.

At that time, I worked with MongoDB and our use-cases were a bit more complex. MongoDB itself allows to store and query a bit more complex data structures than SQL databases (before JSON data type). That's why I decided to use MongoDB query language to define conditions for permissions. But there was another issue: I needed a way to interpret MongoDB in JavaScript.

And thanks to sift.js, library that evaluates MongoDB conditions in runtime, the issue was pretty easy to solve :)

Eventually, sift.js was used to interpret conditions in JavaScript and the same conditions, without additional processing were used to query the database.

As I said, there was no additional preprocessing and it was the main reason why there was no official SQL support.

So, no SQL at all?

Frankly speaking, there is a possibility to use CASL with SQL databases thanks to sequelize which accepts "where" conditions that are pretty similar to MongoDB query language. This works pretty well even today but only for cases when all the data required to check conditions is in a single table.

But as soon as you try to define permissions base on a related table, you are on your own because there is no custom operators support, no AST and no all that stuff which is required for transforming languages from one to another.

Finally. Solution

To add better SQL support, I decided to go the same road and ask Craig to implement it :) The details of our conversion can be found in this PR.

Craig is the author of sift.js library and I'd like to say a "Big Thank You" for his awesome work!

Unfortunately, due to how sift.js was internally implemented there was no easy way to change it to the form that would satisfy CASL's requirements. Also a lack of free time didn't allow us to work together effectively.

That's why I decided to implement my own MongoDB query language interpreter, an interpreter that allows us to use CASL not only with MongoDB but also with SQL, ElasticSearch, Cassandra and actually whatever is required for your business case!

Universal Conditions AST (UCAST)

UCAST is a new conditions checking engine which was specifically implemented for CASL v5. Despite that fact, it can be used on its own and its goal is to interpret any conditions to any language. Some examples:

to transform MongoDB to JavaScript boolean value. In other words, interpret MongoDB conditions in JavaScript runtime on Plain Old JavaScript Objects
transform MongoDB to SQL!
transform json-schema to SQL
transform MongoDB query to json-schema and vice-versa
transform an HTTP request to a MongoDB or SQL query

Hopefully, now it's clear that it provides a way to transform X query to Y query or interpret X query in JavaScript. How do you feel about this? I'm excited!

Free Perks

Additionally to database polyglot ability, UCAST makes CASL v5 to check permissions based on attributes in ~2 times faster than in v4! This was the reason which inspired me to further optimize performance in CASL but this is another story.

If you would like to test this yourself, please use the latest @casl/ability@5.1.0-next.9 pre-release version.

Did I deserve a cup of coffee?

More about ucast, compilers and performance improvements, you will find in the next article. Stay tuned!

CASL 4.0. — What’s inside?

Sergii Stotskyi — Sat, 18 Apr 2020 14:08:42 +0000

First time you’ve heard about CASL? You may want to read “What is CASL?”.

I’m glad to announce that CASL 4.0 was released few days ago and brought several powerful possibilities on our desk:

Type Safety

@casl/* packages were rewritten to TypeScript. This makes your apps safer and developer experience more enjoyable. Let’s see how:

I know I know… I was one of those skeptics who have never liked TypeScript until recent time. It’s really powerful now, especially inference logic!

Starting from 4.0 Ability class accepts 2 generic parameters:

application abilities (actions defined for subjects)
conditions shape

This allows you restrict what actions can be applied on specified subjects. For example, in a blog app which has Article, Comment, and User entities, logged in user can:

read, create, update, delete Articles
read, create, update Comments
read other Users

And we can express this as

import { Ability } from '@casl/ability';

type AppAbilities = [
  'read' | 'update' | 'delete' | 'create',
  'Article' | 'Comment' | 'User'
];

export type AppAbility = Ability<AppAbilities>;

Or even stricter

import { Ability } from '@casl/ability';

type AppAbilities = 
  ['create' | 'read' | 'update' | 'delete', 'Article'] |
  ['create' | 'read' | 'update', 'Comment'] |
  ['read', 'User']
;

export type AppAbility = Ability<AppAbilities>;

This allows TypeScript to check that you don’t make a typo in action or subject name at compile time! Moreover, IDE (e.g., VSCode) will suggest possible options to choose from:

This types hints works even in your favorite frontend libraries, thanks to complementary packages! So, you will get hints for React’s Can component and Vue’s $can function in templates. The nice thing about VSCode is that it uses typescript definition files even for JavaScript files, so you will get better IDE support for JavaScript apps as well. Isn’t that cool?

To get more details about new TypeScript support in CASL, read about it in the docs.

Smaller library size

The library is smaller, ~4.5KB mingzipped (together with sift.js!) and has better support for tree-shaking. This was achieved thanks to terser.js minifier and several breaking changes that you can find in the CHANGELOG.

Starting from 4.0, there are classes that allows to create Ability instance:

PureAbility base class that implements core logic
Ability that extends PureAbility and configures it to use mongoQueryMatcher and fieldPatternMatcher

This was done in order to be able to shake out sift.js dependency in case you don’t use conditions at all or when you implement a custom matcher. As a result AbilityBuilder now accepts Ability class as a first argument. By default, the argument is PureAbility, so if you used AbilityBuilder.extract, you will need to change it from:

const { can, cannot, rules } = AbilityBuilder.extract();  
// rule definitions  
const ability = new Ability(rules);

const { can, cannot, rules } = new AbilityBuilder(Ability);  
// rule definitions  
const ability = new Ability(rules);

Customization

In 4.0, you can customize Ability instance behavior. Do you want to extend MongoDB query with custom operator, use json-schema or arrow functions as conditions? Not a problem anymore, just implement a custom conditionsMatcher.

To get more in depth details, read Customize Ability in the docs.

Better subject type detection support

If you use Plain Old JavaScript Objects as models, you have a simpler option to specify subject type now. No need to provide custom detectSubjectType function (subjectName option was renamed to detectSubjectType), just use subject helper:

import { defineAbility, subject as an } from '@casl/ability';

const object = {
  title: 'My article', 
  description: '...',
};

ability.can('read', an('Article', object));

// or

const article = an.bind(null, 'Article');
ability.can('read', article(object));

To get better understanding of how CASL detect subject type, check Subject Type Detection page.

Much better documentation

Documentation was completely rewritten. First of all, now it’s a Single Page Application written on top of awesome lit-html and lit-element libraries (the final mingzipped size of the app is 47KB for modern browsers! It even somehow works in IE11). It’s available for offline usage, all documentation texts less than 1MB, so don’t worry it won’t take much of your disk :)

Now it’s organized and easily extendable. It also tries to be beginner friendly and have cookbook and examples sections! Every complementary package now have a separate page with documentation for it, so you don’t need to look for README files through repository.

The Future

CASL gets more and more attraction from developers. This requires additional time to write new features, complementary packages, documentation and answering questions in gitter chat. Besides doing my best to do all of this, I need to use my unique talents to fill important roles for the organization, otherwise my wife would expel me a long time ago ;)

I like CASL and like to contribute to Open Source. That’s why I’m seeking for a way to work more on Open Source projects. Thankfully, there are platforms that help to support people like me. And I’m thrilled to announce thatCASL is now on Open Collective!

If you want to get more details about what Open Collective is, and you can contribute, please read Sustaining CASL development

If you have the same vision of the world, and you like CASL, your contributions are very welcome! You are not restricted to financial contributions, as usually you can just share CASL with your colleague from another project, help to answer questions in gitter chat and triage issues, share your examples of CASL integrations, and contribute code and documentation! Finally, you can become a core contributor and support CASL on regular basis.

How do I migrate?

CASL has introduced several breaking changes in all @casl/* packages, so please spend time reading CHANGELOG.md of the packages you use. You can find all changes (including breaking ones) in that file of each package in casl repository.

If you find any issues or something is unclear, please fill in an issue on github.

Where can I start as a beginner?

As of now, CASL has quite good documentation, so start from the Guide. Then move to documentation for the frontend or backend package of your choice and read it. Later you can check existing articles that explains how to integrate CASL in popular frameworks:

Despite the fact, that they are a bit outdated, CASL was not changed much during its lifetime, so they are still relevant.

Thanks for trusting CASL! I hope CASL made permission management in your app as easy as pie