How to Build Microservices with Fauna, Python/Flask and Deploy to Vercel.

INTRODUCTION.

Microservices, this term is mostly used as a reference to the microservices architecture which is an architectural style that structures an application as a collection of loosely-coupled services.
Each of these services is responsible for a discrete task and can communicate with other services through simple APIs to solve a larger complex business problem.

One big advantage of using this approach is that the constituent services can be developed, deployed and maintained independent of each other. Therefore you are able to easily and gradually grow your application.

FAUNA.

For microservices, Fauna has a number of great features that are well suited for this architecture e.g. multi-tenancy which enables developers to create child databases inside the main database that are isolated from each other. This way you can keep data for each service or microservice separately and securely from the others.

In addition to the features above, Fauna also provides us with a powerful and fully-managed database. This way we don’t need to worry about database correctness, sharding, provisioning, latency, or scaling our database. This is why it’s many times referred to as the truly serverless database.

By including Fauna and Vercel in our application stack, we are able to have a fully managed serverless stack. We can make changes to the application and simply do a git push. Vercel will take care of computing resource provisioning while Fauna will provide us with a fully managed database.

This is a tremendous boost to developer productivity as we are able to focus only on what makes our app unique without wasting a lot of time doing the provisioning and maintenance tasks ourselves.

THE APPLICATION.

In this tutorial, we will be creating two microservices for a classic e-commerce backend to enable the following in our application.

1. Product Catalog Management.
2. Product Reviews Management.

These microservices will allow us to create a new product entry, edit / update an existing product’s details, delete a product entry, create a product review, edit/update an existing product review, and delete a product review.

We will take advantage of the Flask Web Framework which is a very powerful Python micro framework to expose our microservices as a web API. Compared to other frameworks such as Django, Flask is lightweight and allows you a lot of flexibility while developing web applications.

CREATING YOUR DATABASE ON FAUNA.

To hold all our application’s data, we will first need to create a database. Fortunately, this is just a single command or line of code, as shown below. Don’t forget to create a Fauna account before continuing!

Fauna Shell

Fauna's API has many interfaces/clients, such as drivers in JS, GO, Java and more, a cloud console, local and cloud shells, and even a VS Code extension! For this article, we’ll start with the local Fauna Shell, which is almost 100% interchangeable with the other interfaces.

npm install -g fauna-shell

After installing the Fauna Shell with npm, log in with your Fauna credentials:

$ fauna cloud-login

Email: [e-mail]
Password: **********

Now we are able to create our database.

fauna create-database e-commerce

CREATE COLLECTIONS.

Now that we have our database created, it's time to create our collections.

In Fauna, a database is made up of one or more collections. The data you create is represented as documents and saved in a collection. A collection is like an SQL table.
Or rather, a collection, is a collection of documents.
A fair comparison with a traditional SQL database would be as below.

For our two microservices, we will create two collections in our database. Namely.

1. products collection.
2. reviews collection.

To start an interactive shell for querying our new database, we need to run:

fauna shell e-commerce

We can now operate our database from this shell.

$ fauna shell e-commerce
Starting shell for database my_app
Connected to https://db.fauna.com
Type Ctrl+D or .exit to exit the shell
e-commerce>

To create our products collection, run the following command in the shell to create the collection with the default configuration.

e-commerce> CreateCollection({ name: "products" })

Next, let’s do the same for the reviews collections.

e-commerce> CreateCollection({ name: "reviews" })

INDEXING YOUR DATA.

Fauna highly recommends indexing your data for the purposes of searching, sorting and combining results from multiple collections (typically called “joins”).
In the context of our application, “search” indexes will allow us to get all reviews for a particular product or all products under a specific category. Search results are based on an index’s “terms” attribute, where we specify the array path to a document’s field, which must match a search argument to be included in a set of results.

We will create the following indexes.

1. products_by_category - To allow us to retrieve products by their category
2. reviews_by_product - To allow us to retrieve a review based on its product.

To create the products_by_category index, run the following command in a fauna shell for our database.

CreateIndex({
  name: "products_by_category",
  source: Collection("products"),
  terms: [
    {
      field: ["data", "categories"]
    }
  ]
})

Next, do the same for the reviews_by_product index.

CreateIndex({
  name: "reviews_by_product",
  source: Collection("reviews"),
  terms: [
    {
      field: ["data", "product"]
    }
  ]
})

While using Fauna, it is generally ideal to write your read queries to go through your indexes, especially for performance purposes.

PROJECT SETUP

To quickly bootstrap our application and its structure we will take advantage of this Flask Template for Fauna.
Clone the Github repository containing the template using the snippet below.

git clone https://github.com/brianraila/faunadb-hipflask.git ecommerce-tutorial

Once this is done, cd into the directory and install all required dependencies.

cd e-commerce-tutorial
pip3 install -r requirements.txt

This will install flask and faunadb-python, along with any other required dependencies.

CREATE A FAUNA KEY

In order for our application to send and receive data to Fauna we will need to create a key and provide its secret when performing queries.

For this application, a key with a Server Role is sufficient to create, read and delete data.
Head over to your database’s Fauna Shell and create a key using the following command.

CreateKey({
      name: "flask-app",
      role: "server"
   })

# Example result.
# NOTE: Make sure you copy and store the secret!
# {
#   ref: Ref(Keys(), "280185139463529993"),
#     ts: 1603464278974000,
#     role: 'server',
#     secret: 'fnAD62i0bTBCDRjYjkcttAsY3wVyfsvynwUSqxYG',
#     hashed_secret: ...
# }

This next step is critical. Copy the secret generated and set it on your project environment by running the command below. Note that secrets are only shown once after creating keys; you’ll have to create a new key if you lose the original secret.

export FAUNA_SECRET=the-fauna-secret-you-copied

PROJECT OVERVIEW

The cloned repository has the directory structure shown below.

The only bit we are going to focus on is the app folder. This folder contains the following files and folders.

WRITING THE MICROSERVICES.

Using Fauna’s native API, The Fauna Query Language (FQL), we can satisfy all of our CRUD needs when resolving the various REST paths of our microservices.
Given FQL’s functional programming design, queries ranging from the simplest CRUD to complex transactions involving precise data manipulation and retrieval, are easily achievable. Note that FQL and query executions are transactional, meaning no changes are committed if an error occurs or conditions aren’t met; this behavior is particular useful for critical business logic, eg. avoiding over-ordering of limited inventory.

This template takes advantage of Flask blueprints which allow us to make our applications as modular as possible. This will help us in separating our microservices and their views into individual and independent components of the entire application.

For our two services, we will create two files in the app/services folder.

1. app/services/products.py - for the Product Catalog Microservice.
2. app/services/reviews.py - for Product Reviews Microservice.

Eventually, we will have the following endpoints.

Products Microservice.

As described above, this microservice will be responsible for creating , reading , updating and deleting products for our application.

Add the following code to the products microservices file.
[./app/services/products.py]

from flask import (
   Blueprint, render_template, jsonify, request
   )
from app.db import client

# Fauna Imports
from faunadb import query as q
from faunadb import errors
from faunadb.objects import Ref

bp = Blueprint('products', __name__, url_prefix='/products')

@bp.route("", methods=["GET", "POST"])
def products():
   if request.method == "GET":
       category = request.args.get("category") or None

       if category:
           # Executes the provided FQL expression.
           data = client.query( 
              # Maps over an array or FQL page and calls an expression/lambda on each item.
              # In this case, we are calling q.get on each item returned from the 2nd parameter’s q.paginate.
               q.map_(
                   # q.get takes a document’s Ref and retrieves the entire document.
                   # Refs are similar to primary/foreign keys and serve as unique pointers to documents.
                   lambda x: q.get(x),
                   # q.index returns the Ref for an index. Note that all entities in FaunaDB are documents and have Refs.
                   # q.match takes the ref of an index, along with search parameters, and returns something called a SetRef.
                   # Finally, q.paginate takes a SetRef and returns a page with iterable data.
                   q.paginate(q.match(
                       q.index("products_by_category"), category)))   
                   )
           response = [i["data"] for i in data["data"]]
           for i in range(0, len(response)):response[i]['id'] = str(data["data"][i]["ref"].id())
           return jsonify(data=response)



       data = client.query(
           q.map_(
               lambda x: q.get(x), q.paginate(q.documents(q.collection("products")))
           )
       )
     # We use q.documents to return all document refs in a collection
       response = [i["data"] for i in data["data"]]
       for i in range(0, len(response)):response[i]['id'] = str(data["data"][i]["ref"].id())
       return jsonify(data=response)

   elif request.method == "POST":

       request_data = request.get_json()
       response = client.query(
           q.create(q.collection("products"), {"data": request_data})
       )
     # FQL: q.create(collection, data_to_create) - Used to create a new document in a collection
       return jsonify(id=response["ref"].id())


@bp.route("/<id>", methods=["GET", "PUT", "DELETE"])
def product(id):
   if request.method == "GET":
       try:
           response = client.query(q.get(q.ref(q.collection("products"), id)))
           return jsonify(data=response["data"])

       except errors.NotFound:
           return jsonify(data=None)

   elif request.method == "PUT":
       # FQL: update a products document
       try:
           response = client.query(
               q.update(
                   q.ref(q.collection("products"), id), {"data":request.get_json()}
               )
           )
      # FQL: we use q.update(doc_ref, data) to update an existing document in a collection 
       except Exception as e:
           return jsonify(error=str(e))

   elif request.method == "DELETE":
       # FQL: delete a product document matching the specified product Ref.
       try:
           response = client.query(q.delete(q.ref(q.collection("products"), id)))
           return response
       except Exception as e:
           return jsonify(error=str(e))

Reviews Microservice.

As described above, this microservice will be responsible for creating, reading, updating and deleting reviews for our application.

Add the following code to the reviews microservices file.
[./app/services/reviews.py]

from flask import (
   Blueprint, render_template, jsonify, request
   )
from app.db import client

# Fauna Imports
from faunadb import query as q
from faunadb import errors
from faunadb.objects import Ref

bp = Blueprint('reviews', __name__, url_prefix='/reviews')

@bp.route("", methods=["GET", "POST"])
def reviews():
   if request.method == "GET":
       product = request.args.get('product')
       if product:
      # FQL: Query the reviews_by_product index for products matching the category using the match function.
           data = client.query(
               q.map_(
                   lambda x: q.get(x), q.paginate(q.match(
                       q.index("reviews_by_product"), product))) 
                   )
           response = [i["data"] for i in data["data"]]
           for i in range(0, len(response)):response[i]['id'] = str(data["data"][i]["ref"].id())
           return jsonify(data=response)

       data = client.query(
           q.map_(lambda x: q.get(x), q.paginate(q.documents(q.collection("reviews"))))
       )
       response = [i["data"] for i in data["data"]]
       for i in range(0, len(response)):response[i]['id'] = str(data["data"][i]["ref"].id())
       return jsonify(data=response)

   elif request.method == "POST":

       request_data = request.get_json()
       response = client.query(
           q.create(q.collection("reviews"), {"data": request_data})
       )
       return jsonify(id=response["ref"].id())

@bp.route("/<id>", methods=["GET", "PUT", "DELETE"])
def review(id):
   if request.method == "GET":
       # Retrieve a reviews document that matched a specified id.
       try:
           response = client.query(q.get(q.ref(q.collection("reviews"), id)))
           return jsonify(data=response["data"])

       except errors.NotFound:
           return jsonify(data=None)

   elif request.method == "PUT":
       # FQL: update a reviews document that match the id specified
       try:
           response = client.query(
               q.update(
                   q.ref(q.collection("reviews"), id), {"data":response.get_json()}
               )
           )
       except Exception as e:
           return jsonify(error=str(e))

   elif request.method == "DELETE":
       # FQL: delete a reviews document that match the id specified
       try:
           response = client.query(q.delete(q.ref(q.collection("reviews"), id)))
           return response
       except Exception as e:
           return jsonify(error=str(e))

Register Services.

Our application has it’s microservices written as blueprints. In Flask, we will need to register each of these blueprints to make them available to our application.
To do this, add the following code to the ./app/init.py file

[./app/init.py]

import os
from flask import Flask

def create_app():

   app = Flask( __name__, instance_relative_config=True,)
   app.config.from_mapping( SECRET_KEY=os.environ.get('SECRET_KEY') or 'you-never-guess',)

   from app.services import index, products, reviews

   app.register_blueprint(index.bp)
   app.register_blueprint(products.bp)
   app.register_blueprint(reviews.bp)

   return app

app = create_app()

TESTING.

To run our application in testing and debug mode run the following in the application directory.

python3 run.py

Since our microservices exist as REST API endpoints that are decomposed by paths and routes, if you’re interested, you can make use of Postman to test how well they work. Here’s a Postman Collection you can use to test the various routes.

The following Github repo has sample data you can use to test each of the endpoints.

DEPLOYING TO VERCEL.

Vercel allows for configuration files to describe how we’d like our microservices deployed.
The template we used comes with one by default so you will not need to do so; below is what it looks like.

[./vercel.json]

{
    "version": 2,
    "builds": [
        {
            "src": "./index.py",
            "use": "@vercel/python"
        }
    ],
    "routes": [
        {
            "src": "/(.*)",
            "dest": "/"
        }
    ]
}

To deploy our app to Vercel, we first need to install the Vercel CLI by running the following command.

npm i -g vercel

Ensure you have a Vercel account, or head over to Vercel.com to create one.

Once registered, run the following command to login to the CLI with your account.

vercel login

Follow the prompts to confirm your email.
Once you successfully login, run the following command to setup and deploy the app to Vercel.

vercel

You will receive a similar prompt screen.

Since Vercel still doesn’t know your FAUNA SECRET KEY, you will get an error while using the app. To rectify this, add the secret key to your Vercel environment as follows.

vercel env add

Follow the prompts as shown below.

Add the key for the production environment and if you wish, the dev environment.

Now redeploy your app using

vercel --prod

And now your application should be live.
You can preview your app by visiting the link provided by Vercel.

CONCLUSION

From the tutorial, we can see how easy it is to get started with Fauna for our microservices. Fauna is indeed a powerful database; through FQL we have a flexible query language to support a myriad of queries and data models. It also comes with great features suited for microservices, like multi-tenancy and granular permissions.

Combined together with Vercel, we can have a truly serverless and fully managed microservice architecture. This is a great boost to productivity as we can focus on implementing features that make our application and UX unique, instead of getting hung up on the DevOps of old-school servers and databases.

I hope you find Fauna to be exciting, like I do, and that you enjoyed this article. Feel free to follow me on Twitter @theAmolo if you enjoyed my perspective!

BONUS:
All code written for this tutorial can be found in the following Github Repo

Comments

[Curious Paul]

Really great piece here. Although i should mention that flask-blueprints aren't standalone apps they are instead used to extend a single flask app into components (for the sake of separation of concerns). These blueprints cannot and shouldn't be referenced as 'services' they can't be deployed independently and are still a part of a single flask app. Microservice architecture would feature standalone and independent applications as opposed to blueprints in a single app.

[Amolo]

I totally agree. That's not what the article insinuates.
The whole idea of using blueprints is to act as an oversimplified "API gateway" for the two microservices.
They can as well be separated easily.
Thanks for that tho.

[gravesli]

I think you are great! I built a state monitor using Flask!
Flask State Github：github.com/yoobool/flask-state .
Should i can get some improvement suggestions from anyone? Thanks~

[Amolo]

Interesting.