DEV Community: BONNAURE Olivier

Fasty, an open source app builder using OpenResty and ArangoDB

BONNAURE Olivier — Mon, 30 Nov 2020 10:10:44 +0000

Why I used this weird stack ?

Here a brief history about why I did create this tool. I was a Ruby on Rails developer. I was concerned about performances, ressources used. Crystal language was looking promising, but I wanted to start something new using a different langage.

Few years ago I was in a meeting in Hong-Kong about ArangoDB (which I already knew) and an another project called OpenResty. The developer was so excited about it, and it was also a RoR developer... So I was intrigued. How a tool can be better than Rails ?

After some times I started digging about OpenResty and I found the excellent framework Lapis

So here I am : I have chose my tools. OpenResty (using Lapis) and ArangoDB for the backend.

What does Fasty Cover ?

Before creating Fasty I wrote foxxy. A tool to generate automatic admin UI based on ArangoDB. It was useful but I was still using my Ruby CMS.

My daily needs are :

Manage views (Layouts, partials, components, pages)
I18n
Manage my backend APIs

So Fasty cover all my needs and much more.

With Fasty you can :

Manage views
Manage I18n content
Manage backend API
Manage NodeJS scripts (think about socketIO scripts)
Upload assets
Manage datatypes (create CRUDs in minutes)
Render views on server side (like ERB for Ruby but using Lua)
Create SPA (it use RiotJS for web components)
Multi Tenant (with a workflow (eg. DEV => QA => PROD))

Looks good, but why Lua / OpenResty / ArangoDB ?

The main advantage of Lua / OpenResty is that it's very fast and don't consume lot of RAM (Fasty use 1,5Mb). OpenResty is a Nginx + some Lua libraries. It's top solid.

Lua also allow you to write procedural code but it can use coroutines to make them asynchronous. So by example http calls won't block your script. It will use Nginx loop and resume your script when ready.

ArangoDB is a multi-document NoSQL database. It allow you to manage complex data with ease and work with Graph documents. It come with Foxx which allow you to host Rest API in the heart of your database. Fasty allow you to manage them and deploy them automatically. It's very fast and reduce the number of http/tcp calls to your database.

Requirements

There is a docker-compose file, so starting with Fasty is very easy. It can run on a tiny VM. But beware, database will consume some RAM if you use millions of documents.

What's next ?

I'm gonna create some videos about how to start with Fasty and how to manage your content, your data, dynamics views & web components.

ArangoDB : The new Graal ? Part 3

BONNAURE Olivier — Tue, 26 Dec 2017 21:49:54 +0000

After the authentification seen in part2, it's time to play with collections.
Here the main.js file.

'use strict';
// Define db & joi for db calls and validation
const db = require('@arangodb').db;
const joi = require('joi');
// Enable cache
require("@arangodb/aql/cache").properties({ mode: "on" });

// Create router and session storage
const createRouter = require('@arangodb/foxx/router');
const sessionsMiddleware = require('@arangodb/foxx/sessions');
const jwtStorage = require('@arangodb/foxx/sessions/storages/jwt');
const router = createRouter();
const sessions = sessionsMiddleware({
  storage: jwtStorage('something secret'),
  transport: 'header'
});


// Use sessions & router
module.context.use(sessions);
module.context.use(router);

If you want to protect your service to registered users only :

// Use the authentification
module.context.use(function (req, res, next) {
  if(!req.session.uid) res.throw('unauthorized');
  res.setHeader("Access-Control-Expose-Headers", "X-Session-Id");
  next();
});

Ok, let's start writing CRUD stuff. In this sample I'll use the posts collection as sample.

// GET /page/:page
router.get('/page/:page', function (req, res) {
  res.send({ data: db._query(`
    LET count = LENGTH(posts)
    LET data = (FOR doc IN posts SORT doc._key DESC LIMIT @offset,25 RETURN doc)
    RETURN { count: count, data: data }
    `, { "offset": (req.pathParams.page - 1) * 25}).toArray() });
})
.header('X-Session-Id')
.description('Returns all objects using pagination');

// GET /:id
router.get('/:id', function (req, res) {
  res.send(db.posts.document(req.pathParams.id));
})
.header('X-Session-Id')
.description('Returns object within ID');

// POST /
router.post('/', function (req, res) {
  var data = {
    field_1: req.body.field_1,
    field_2: req.body.field_2,
    field_3: req.body.field_3
  }
  res.send(db.posts.save(data));
})
.body(joi.object({
  field_1: joi.string().required(),
  field_2: joi.number().required(),
  field_3: joi.any(),
}), 'data')
.header('X-Session-Id')
.description('Create a new object.');

// PUT /:id
router.put('/:id', function (req, res) {
  var object = db.posts.document(req.pathParams.id)
  var data = {
    field_1: req.body.field_1,
    field_2: req.body.field_2,
    field_3: req.body.field_3
  }
  res.send(db.posts.update(object, data));
})
.body(joi.object({
  field_1: joi.string().required(),
  field_2: joi.number().required(),
  field_3: joi.any(),
}), 'data')
.header('X-Session-Id')
.description('Update an object.');

// DELETE /:id
router.delete('/:id', function (req, res) {
  res.send(db.posts.remove("posts/"+req.pathParams.id));
})
.header('X-Session-Id')
.description('delete an object.');

You have here a minimal but working sample of a complete CRUD using Foxx & ArangoDB.

To save my time, I built a very useful tool called foxxy to build administration apps based on Foxx, UIkit, RiotJS & Brunch. It will allow you to manage file upload, tags and more.

You can now build any kind of Rest API which can be linked to any kind of client application.

ArangoDB : The new Graal ? Part 2

BONNAURE Olivier — Thu, 23 Nov 2017 14:04:13 +0000

In part1 I just show you a very basic foxx service. In this part 2 I'll show you how to create a service to manage authentifications using JWT.

Let's start creating a new service. First we need to define what we will need for this service.

'use strict';
const db = require('@arangodb').db;
const joi = require('joi');
const createRouter = require('@arangodb/foxx/router');
const sessionsMiddleware = require('@arangodb/foxx/sessions');
const jwtStorage = require('@arangodb/foxx/sessions/storages/jwt');
const createAuth = require('@arangodb/foxx/auth');
const auth = createAuth();
const router = createRouter();

Ok so we will need :

db object for accessing database
joi for all validations stuff
sessionsMiddleware to manage sessions (see below)
jwtStorage to be able to use JWT
router which is the router
auth auth tool (create & verify methods)

Now it's time to create the session

const sessions = sessionsMiddleware({
  storage: jwtStorage({ secret: "Secret", ttl: 60 * 60 * 24 * 7 }),
  ttl: 60 * 60 * 24 * 7, // one week in seconds
  transport: 'header'
});

Ok easy, we just define the storage, ttl & transport.

Our service need to be linked to this sessions

module.context.use(sessions);
module.context.use(router);

Our service is linked now to our sessions & router

It's time to create endpoints.
Let's start with the signup one.

router.post('/signup', function (req, res) {
  const user = req.body; // get the form defined in the body section below

  try {
    // Create an authentication hash
    user.authData = auth.create(user.password);

    // Delete plain password data
    delete user.password;
    delete user.password_confirmation;

    // Validate user (for demo purpose)
    user.a = true
    const meta = db.users.save(user);
    Object.assign(user, meta); // assign _key, _id to user
  } catch (e) {
    res.throw('bad request', 'Username already taken', e);
  }
  // Set the session uid
  req.session.uid = user._key;
  res.send({success: true});
})
.body(joi.object({
  "fn": joi.string().required(),
  "ln": joi.string().required(),
  "username": joi.string().required(),
  "password": joi.string().min(8).max(32).required(),
  "password_confirmation": joi.string().required(),
}), 'Credentials')
.description('Creates a new user and logs them in.');

Here the login endpoint

router.post('/login', function (req, res) {
  // This may return a user object or null
  const user = db.users.firstExample({
    username: req.body.username,
    a: true
  });
  const valid = auth.verify(
    user ? user.authData : {},
    req.body.password
  );
  // Log the user in
  if(valid) {
    req.session.uid = user._key;
  }
  // Corrs
  res.setHeader("Access-Control-Expose-Headers", "X-Session-Id");
  res.send({success: valid, uid: req.session});
})
.body(joi.object({
  username: joi.string().required(),
  password: joi.string().required()
}).required(), 'Credentials')
.description('Logs a registered user in.');

And finally, here is the logout endpoint

router.post('/logout', function (req, res) {
  if (req.session.uid) {
    req.session.uid = null;
  }
  res.send({success: true});
})
.description('Logs the current user out.');

You can find the complete code in this gist

ArangoDB will provide you a nice swagger documentation to check that everything is ok.

On next part, I'll show you a complete CRUD for managing your collections.

ArangoDB : The new Graal ? Part 1

BONNAURE Olivier — Thu, 16 Nov 2017 18:40:49 +0000

ArangoDB : The new Graal ? Part 1

I discovered ArangoDB 2 years ago ... I was looking for something new, something fresh. I was bored about deploying new application for any new project. Then I found ArangoDB ... And I still wonder why few people know it !

ArangoDB is a multi-model database (key/value; documents; graph) with transactions and joins.

ArangoDB provide Foxx, a micro service engine embedded in ArangoDB. Last but not least, Foxx services scales with database cluster. It's very powerful and can cover your needs for almost all your projects.

To be honest I use ArangoDB in single instance mode for now.

How does a service looks like ?

'use strict';
const db = require('@arangodb').db;
const createRouter = require('@arangodb/foxx/router');
const router = createRouter();

module.context.use(router);

router.get('/', function(req, res) {
    res.send({ hello: 'world' });
});

Easy, isn't it ? You can create any Rest verb you want (e.g. router.get, router.post, router.put, router.delete, router.patch)

Ok that's cool ... but how to interact with the database, the collections, the graphs ?

ArangoDB provide a language request called AQL which will allow you to run requests in database just like SQL.

Let say we have created a collection called posts

Here an endpoint to load all posts filtered by a specific status and also return joined user.

router.get('/posts/:status', function(req, res) {
    const documents = db._query(`
        FOR post IN posts
            FILTER posts.status == @status
            FOR user IN users
                FILTER posts.user_key == user._key
                RETURN { post: post, user: user }
    `, { status: req.pathParams.status });

})

ArangoDB UI will provide you a nice swagger documentation such as :

The biggest benefit to run Foxx is that it is hosted in the heart of the database and requests are blazing fasts.

On Part 2 I'll show you how to secure your APIs with a Json Web Token

For people working on OSX I wrote foxx-framework to simplify my daily workflow.