DEV Community: Vortico

Releasing Flama 1.10

Vortico — Wed, 05 Mar 2025 12:30:02 +0000

We’re happy to announce Flama 1.10 🎉, packed with exciting features that have been brewing since our last major update. This release rolls up the goodness from 1.8 and 1.9, which we didn’t get around to shouting about earlier, and tops it off with some shiny new additions in 1.10. Let’s dive into the highlights!

This release brings a slew of new capabilities to make your ML API development smoother, smarter, and more powerful. Here’s what’s new across these versions:

Access and Refresh Token Components (1.8.0): Say hello to enhanced security and flexibility! Flama now includes components for access and refresh tokens, making it easier to implement robust authentication flows and manage user sessions seamlessly.
Support for Python 3.13 and Dropping 3.8 (1.8.0): We’re keeping up with the times—Flama now supports Python 3.13, bringing you the latest language features and performance boosts. To keep things lean, we’ve waved goodbye to Python 3.8 compatibility.
Adapted Serialization for New TensorFlow Version (1.8.0): Machine learning devs, rejoice! Flama’s serialization has been updated to play nicely with the latest TensorFlow version, ensuring your models integrate effortlessly.
Typing.Annotated for Schema Types (1.9.0): We’ve leveled up our type system by adopting typing.Annotated for schema definitions. This makes your code more expressive and type-safe, giving you clearer control over your API schemas.
Allow Importing DDD Without SQLAlchemy Installed (1.9.0): Flexibility is key! You can now use Flama’s ddd module without needing SQLAlchemy, making it lighter and more adaptable to your project’s needs.
Compatibility Lib for Older Python Versions (1.9.0): While we’ve embraced the future with 3.13, we’ve also added a compatibility library to ensure smooth sailing on older Python versions where needed.
Path Params Component (1.10.0): Dynamic routing just got better! The new path parameters component lets you handle URL variables with ease, giving your APIs more versatility.
Replace Black and Isort with Ruff (1.10.0): We’ve swapped out Black and Isort for Ruff, a faster and more comprehensive tool for code formatting and linting. Your development workflow just got a speed boost!
Replace Poetry with UV (1.10.0): Dependency management is now powered by UV instead of Poetry, offering a snappier and more efficient way to handle your project’s packages.
Module for Gathering Telemetry Data (1.10.0): Get insights like never before! Flama now includes a telemetry module to help you track usage and performance, empowering data-driven decisions for your APIs.
Enrich OpenAPI Specification (1.10.0): Your API docs are about to get a glow-up. We’ve enriched the OpenAPI spec generation, making it more detailed and user-friendly for consumers of your APIs.
Docs Page Using Scalar (1.10.0): Speaking of docs, we’ve introduced a sleek new documentation page powered by Scalar. It’s prettier, more interactive, and makes exploring Flama’s features a breeze.

To help you make the most of these updates, we’ll be rolling out a few detailed follow-up posts with examples showcasing how to leverage telemetry, path params, and the enriched OpenAPI specs in your projects. Keep an eye out!

Stay tuned for more, and happy coding! 🚀

References

About the authors

Vortico: We’re specialised in software development to help businesses enhance and expand their AI and technology capabilities.

Native Domain-Driven Design with Flama

Vortico — Mon, 28 Oct 2024 08:03:09 +0000

You've probably already heard about the recent release of Flama 1.7, which brought some exciting new features to help you with the development and productionalisation of your ML APIs. This post is precisely devoted to one of the main highlights of that release: Support for Domain-Driven Design. We recommend you to consider the following resources (and, get familiar with them if you aren't already):

Official Flama documentation: Flama documentation
Post introducing Flama for ML APIs: Introduction to Flama for Robust Machine Learning APIs

Now, let's get started with the new feature and see how you can leverage it to build robust and maintainable ML APIs.

This post is structured as follows:

What is Domain-Driven Design?
- Brief Overview
- Key Concepts
Implementing DDD with Flama
- Setting up the development environment
- Base application
- DDD in action
Conclusion
Support our work
References
About the authors

What is Domain-Driven Design?

Brief Overview

In modern software development, aligning business logic with the technical design of an application is essential. This is where Domain-Driven Design (DDD) shines. DDD emphasizes building software that reflects the core domain of the business, breaking down complex problems by organizing code around business concepts. By doing so, DDD helps developers to create maintainable, scalable, and robust applications. In what follows we introduce what we consider the most important concepts of DDD that you should be aware of. Let's remark that this post is not intended to be a comprehensive guide to DDD, nor a substitute of the main references on the topic. Indeed, we recommend the following resources to get a deeper understanding of DDD:

Cosmic Python by Harry Percival and Bob Gregory: This book is a great resource to learn how to apply DDD in Python.
Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans: This is the book that introduced DDD to the world, and it's a must-read for anyone interested in developing a deep understanding of DDD.

Key Concepts

Before exploring any of the key concepts of DDD, we recommend you to have a look at a quite useful figure by Cosmic Python where these are shown in the context of an app, thus showing how they're interconnected: figure.

Domain Model

The concept of domain model can be explained by a simplistic definition of its terms:

domain refers to the specific subject area of activity (or knowledge) that our software is being built to support for.
model refers to a simple representation (or abstraction) of the system or process that we are trying to encode in our software.

Thus, the domain model is a fancy (but standard and useful) way to refer to the set of concepts and rules that business owners have in their mind about how the business works. This is what we also, and commonly, refer to as the business logic of the application, including the rules, constraints, and relationships that govern the behavior of the system.

We'll refer to the domain model as the model from now on.

Repository pattern

The repository pattern is a design pattern that allows for the decoupling the model from the data access. The main idea behind the repository pattern is to create an abstraction layer between the data access logic and the business logic of an application. This abstraction layer allows for the separation of concerns, making the code more maintainable and testable.

When implementing the repository pattern, we typically define an interface that specifies the standard methods that any other repository must implement (AbstractRepository). And, then, a particular repository is defined with the concrete implementation of these methods where the data access logic is implemented (e.g., SQLAlchemyRepository). This design pattern aims at isolating the data manipulation methods so that they can be used seamlessly elsewhere in the application, e.g. in our domain model.

Unit of work pattern

The unit of work pattern is the missing part to finally decouple the model from the data access. The unit of work encapsulates the data access logic and provides a way to group all the operations that must be performed on the data source within a single transaction. This pattern ensures that all the operations are performed atomically.

When implementing the unit of work pattern, we typically define an interface that specifies the standard methods that any other unit of work must implement (AbstractUnitOfWork). And, then, a particular unit of work is defined with the concrete implementation of these methods where the data access logic is implemented (e.g., SQLAlchemyUnitOfWork). This design allows for a systematic handling of the connection to the data source, without the need to change the implementation of the business logic of the application.

Implementing DDD with Flama

After the quick introduction to the main concepts of DDD, we're ready to dive into the implementation of DDD with Flama. In this section, we'll guide you through the process of setting up the development environment, building a base application, and implementing DDD concepts with Flama.

Prior to the example, let's have a look at Flama's naming convention regarding the main DDD concepts we've just reviewed:

As you can see in the figure above, the naming convention is quite intuitive: Repository refers to the repository pattern; and, Worker refers to the unit of work. Now, we can now move on to the implementation of a Flama API which uses DDD. But, before we start, if you need to review the basics on how to create a simple API with flama, or how to run the API once you've already the code ready, then you might want to check out the quick start guide. There, you'll find the fundamental concepts and steps required to follow this post. Now, without further ado, let's get started with the implementation.

Setting up the development environment

Our first step is to create our development environment, and install all required dependencies for this project. The good thing is that for this example we only need to install flama to have all the necessary tools to implement JWT authentication. We'll be using poetry to manage our dependencies, but you can also use pip if you prefer:

poetry add "flama[full]" "aiosqlite"

The aiosqlite package is required to use SQLite with SQLAlchemy, which is the database we'll be using in this example.

If you want to know how we typically organise our projects, have a look at our previous post here, where we explain in detail how to set up a python project with poetry, and the project folder structure we usually follow.

Base application

Let's start with a simple application that has a single public endpoint. This endpoint will return a brief description of the API.

# src/app.py
from flama import Flama

app = Flama(
    title="Domain-driven API",
    version="1.0.0",
    description="Domain-driven design with Flama 🔥",
    docs="/docs/",
)

@app.get("/", name="info")
def info():
    """
    tags:
        - Info 
    summary:
        Ping
    description:
        Returns a brief description of the API
    responses:
        200:
            description:
                Successful ping.
    """
    return {"title": app.schema.title, "description": app.schema.description, "public": True}

If you want to run this application, you can save the code above in a file called app.py under the src folder, and then run the following command (remember to have the poetry environment activated, otherwise you'll need to prefix the command with poetry run):

flama run --server-reload src.app:app

INFO:     Started server process [3267]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

where the --server-reload flag is optional and is used to reload the server automatically when the code changes. This is very useful during development, but you can remove it if you don't need it. For a full list of the available options, you can run flama run --help, or check the documentation.

Alternatively, you can also run the application by running the following script, which you can save as __main__.py under the src folder:

# src/__main__.py
import flama

def main():
    flama.run(
      flama_app="src.app:app", 
      server_host="0.0.0.0", 
      server_port=8000, 
      server_reload=True
    )

if __name__ == "__main__":
    main()

And then, you can run the application by executing the following command:

poetry run python src/__main__.py

INFO:     Started server process [3267]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

DDD in action

Now, having set up a minimal skeleton for our application, we can start implementing the DDD concepts we've just reviewed within the
context of a simple example which tries to mimic a real-world scenario. Let's assume we are requested to develop an API to manage users, and we are provided with the following requirements:

We want to create new users via a POST request to /user/, providing the user's name, surname, email, and password.
Any user created will be stored in a database with the following schema:
- id: unique identifier for the user.
- name: user's name.
- surname: user's surname.
- email: user's email.
- password: user's password. This should be hashed before storing it in the database.
- active: a boolean flag to indicate whether the user is active or not. By default, users are created as inactive.
Users created must activate their account by sending a POST request to /user/activate/ with their email and password. Once the user is activated, the user's status must be updated in the database to active.
Users can sign in by sending a POST request to /user/signin/ with their email and password. If the user is active, the API must return all user's information. Otherwise, the API must return an error message.
Users that want to deactivate their account can do so by sending a POST request to /user/deactivate/ with their email and password. Once the user is deactivated, the user's status must be updated in the database to inactive.

This set of requirements constitute what we've previously referred to as the domain model of our application, which essentially is nothing but a materialisation of the following user workflow:

A user is created via a POST request to /user/.
The user activates their account via a POST request to /user/activate/.
The user signs in via a POST request to /user/signin/.
The user deactivates their account via a POST request to /user/deactivate/.
The user can repeat steps 2-4 as many times as they want.

Now, let's implement the domain model using the repository and worker patterns. We'll start by defining the data model, and then we'll implement the repository and worker patterns.

Data model

Our users data will be stored in a SQLite database (you can use any other database supported by SQLAlchemy). We'll use the following data model to represent the users (you can save this code in a file called models.py under the src folder):

# src/models.py
import uuid

import sqlalchemy
from flama.sqlalchemy import metadata
from sqlalchemy.dialects.postgresql import UUID

__all__ = ["user_table", "metadata"]

user_table = sqlalchemy.Table(
    "user",
    metadata,
    sqlalchemy.Column("id", UUID(as_uuid=True), primary_key=True, nullable=False, default=uuid.uuid4),
    sqlalchemy.Column("name", sqlalchemy.String, nullable=False),
    sqlalchemy.Column("surname", sqlalchemy.String, nullable=False),
    sqlalchemy.Column("email", sqlalchemy.String, nullable=False, unique=True),
    sqlalchemy.Column("password", sqlalchemy.String, nullable=False),
    sqlalchemy.Column("active", sqlalchemy.Boolean, nullable=False),
)

Besides the data model, we need a migration script to create the database and the table. For this, we can save the following code in a file called migrations.py at the root of the project:

# migrations.py
from sqlalchemy import create_engine

from src.models import metadata

if __name__ == "__main__":
    # Set up the SQLite database
    engine = create_engine("sqlite:///models.db", echo=False)

    # Create the database tables
    metadata.create_all(engine)

    # Print a success message
    print("Database and User table created successfully.")

And then, we can run the migration script by executing the following command:

> poetry run python migrations.py

Database and User table created successfully.

Repository

In this example we are going to need only one repository, namely the repository which will handle the atomic operations on the user table, the name of which will be UserRepository. Thankfully, flama provides a base class for repositories related to SQLAlchemy tables, called SQLAlchemyTableRepository.

The class SQLAlchemyTableRepository provides a set of methods to perform CRUD operations on the table, specifically:

create: Creates new elements in the table. If the element already exists, it will raise an exception (IntegrityError), otherwise it will return the primary key of the new element.
retrieve: Retrieves an element from the table. If the element does not exist, it will raise an exception (NotFoundError), otherwise it will return the element. If more than one element is found, it will raise an exception (MultipleRecordsError).
update: Updates an element in the table. If the element does not exist, it will raise an exception (NotFoundError), otherwise it will return the updated element.
delete: Deletes an element from the table.
list: Lists all the elements in the table that match the clauses and filters passed. If no clauses or filters are given, it returns all the elements in the table. If no elements are found, it returns an empty list.
drop: Drops the table from the database.

For the purposes of our example, we don't need any further action on the table, so the methods provided by the SQLAlchemyTableRepository are sufficient. We can save the following code in a file called repositories.py under the src folder:

# src/repositories.py
from flama.ddd import SQLAlchemyTableRepository

from src import models

__all__ = ["UserRepository"]

class UserRepository(SQLAlchemyTableRepository):
    _table = models.user_table

As you can see, the UserRepository class is a subclass of SQLAlchemyTableRepository, and it only requires the table to be set in the _table attribute. This is the only thing we need to do to have a fully functional repository for the user table.

If we wanted to add custom methods beyond the standard CRUD operations, we could do so by defining them in the UserRepository class. For example, if we wanted to add a method to count the number of active users, we could do so as follows:

# src/repositories.py
from flama.ddd import SQLAlchemyTableRepository

from src import models

__all__ = ["UserRepository"]

class UserRepository(SQLAlchemyTableRepository):
    _table = models.user_table

    async def count_active_users(self):
        return len((await self._connection.execute(self._table.select().where(self._table.c.active == True))).all())

Although we won't be using this method in our example, it's good to know that we can add custom methods to the repository if needed, and how they are implemented
in the context of the repository pattern. This is a powerful design pattern as we can already see, since we can implement here all the data access logic without having to change the business logic of the application (which is implemented in the corresponding resource methods).

Worker

The unit-of-work pattern is used to encapsulate the data access logic and provide a way to group all the operations that must be performed on the data source within a single transaction. In flama the UoW pattern is implemented with the name of Worker. In the same way as with the repository pattern, flama provides a base class for workers related to SQLAlchemy tables, called SQLAlchemyWorker. In essence, the SQLAlchemyWorker provides a connection and a transaction to the database, and instantiates all its repositories with the worker connection. In this example, our worker will only use a single repository (namely, the UserRepository) but we could add more repositories if needed.

Our worker will be called RegisterWorker, and we can save the following code in a file called workers.py under the src folder:

# src/workers.py
from flama.ddd import SQLAlchemyWorker

from src import repositories

__all__ = ["RegisterWorker"]


class RegisterWorker(SQLAlchemyWorker):
    user: repositories.UserRepository

Thus, if we had more repositories to work with, for instance ProductRepository and OrderRepository, we could add them to the worker as follows:

# src/workers.py
from flama.ddd import SQLAlchemyWorker

from src import repositories

__all__ = ["RegisterWorker"]

class RegisterWorker(SQLAlchemyWorker):
    user: repositories.UserRepository
    product: repositories.ProductRepository
    order: repositories.OrderRepository

As simple as that, we have implemented the repository and worker patterns in our application. Now, we can move on to implement the resource methods that will provide the API endpoints needed to interact with the user data.

Resources

Resources are one of the main building blocks of a flama application. They are used to represent application resources (in the sense of RESTful resources) and to define the API endpoints that interact with them.

In our example, we will define a resource for the user, called UserResource, which will contain the methods to create, activate, sign in, and deactivate users. Resources need to derive, at least, from the flama built-in Resource class, although flama provides more sophisticated classes to work with such as RESTResourceand CRUDResource.

We can save the following code in a file called resources.py under the src folder:

# src/resources.py
import hashlib
import http
import uuid

from flama import types
from flama.ddd.exceptions import NotFoundError
from flama.exceptions import HTTPException
from flama.http import APIResponse
from flama.resources import Resource, resource_method

from src import models, schemas, worker

__all__ = ["AdminResource", "UserResource"]

ENCRYPTION_SALT = uuid.uuid4().hex
ENCRYPTION_PEPER = uuid.uuid4().hex

class Password:
    def __init__(self, password: str):
        self._password = password

    def encrypt(self):
        return hashlib.sha512(
            (hashlib.sha512((self._password + ENCRYPTION_SALT).encode()).hexdigest() + ENCRYPTION_PEPER).encode()
        ).hexdigest()

class UserResource(Resource):
    name = "user"
    verbose_name = "User"

    @resource_method("/", methods=["POST"], name="create")
    async def create(self, worker: worker.RegisterWorker, data: types.Schema[schemas.UserDetails]):
        """
        tags:
            - User
        summary:
            User create
        description:
            Create a user
        responses:
            200:
                description:
                    User created in successfully.
        """
        async with worker:
            try:
                await worker.user.retrieve(email=data["email"])
            except NotFoundError:
                await worker.user.create({**data, "password": Password(data["password"]).encrypt(), "active": False})

        return APIResponse(status_code=http.HTTPStatus.OK)

    @resource_method("/signin/", methods=["POST"], name="signin")
    async def signin(self, worker: worker.RegisterWorker, data: types.Schema[schemas.UserCredentials]):
        """
        tags:
            - User
        summary:
            User sign in
        description:
            Create a user
        responses:
            200:
                description:
                    User signed in successfully.
            401:
                description:
                    User not active.
            404:
                description:
                    User not found.
        """
        async with worker:
            password = Password(data["password"])
            try:
                user = await worker.user.retrieve(email=data["email"])
            except NotFoundError:
                raise HTTPException(status_code=http.HTTPStatus.NOT_FOUND)

            if user["password"] != password.encrypt():
                raise HTTPException(status_code=http.HTTPStatus.UNAUTHORIZED)

            if not user["active"]:
                raise HTTPException(
                    status_code=http.HTTPStatus.BAD_REQUEST, detail=f"User must be activated via /user/activate/"
                )

        return APIResponse(status_code=http.HTTPStatus.OK, schema=types.Schema[schemas.User], content=user)

    @resource_method("/activate/", methods=["POST"], name="activate")
    async def activate(self, worker: worker.RegisterWorker, data: types.Schema[schemas.UserCredentials]):
        """
        tags:
            - User
        summary:
            User activate
        description:
            Activate an existing user
        responses:
            200:
                description:
                    User activated successfully.
            401:
                description:
                    User activation failed due to invalid credentials.
            404:
                description:
                    User not found.
        """
        async with worker:
            try:
                user = await worker.user.retrieve(email=data["email"])
            except NotFoundError:
                raise HTTPException(status_code=http.HTTPStatus.NOT_FOUND)

            if user["password"] != Password(data["password"]).encrypt():
                raise HTTPException(status_code=http.HTTPStatus.UNAUTHORIZED)

            if not user["active"]:
                await worker.user.update({**user, "active": True}, id=user["id"])

        return APIResponse(status_code=http.HTTPStatus.OK)

    @resource_method("/deactivate/", methods=["POST"], name="deactivate")
    async def deactivate(self, worker: worker.RegisterWorker, data: types.Schema[schemas.UserCredentials]):
        """
        tags:
            - User
        summary:
            User deactivate
        description:
            Deactivate an existing user
        responses:
            200:
                description:
                    User deactivated successfully.
            401:
                description:
                    User deactivation failed due to invalid credentials.
            404:
                description:
                    User not found.
        """
        async with worker:
            try:
                user = await worker.user.retrieve(email=data["email"])
            except NotFoundError:
                raise HTTPException(status_code=http.HTTPStatus.NOT_FOUND)

            if user["password"] != Password(data["password"]).encrypt():
                raise HTTPException(status_code=http.HTTPStatus.UNAUTHORIZED)

            if user["active"]:
                await worker.user.update({**user, "active": False}, id=user["id"])

        return APIResponse(status_code=http.HTTPStatus.OK)

Base application with DDD

Now that we have implemented the data model, the repository and worker patterns, and the resource methods, we need to modify the base application we introduced before, so that everything works as expected. We need to:

Add the SQLAlchemy connection to the application, and this is achieved by adding the SQLAlchemyModule to the application constructor as a module.
Add the worker to the application, and this is achieved by adding the RegisterWorker to the application constructor as a component.

This will leave the app.py file as follows:

# src/app.py

from flama import Flama
from flama.ddd import WorkerComponent
from flama.sqlalchemy import SQLAlchemyModule

from src import resources, worker

DATABASE_URL = "sqlite+aiosqlite:///models.db"

app = Flama(
    title="Domain-driven API",
    version="1.0.0",
    description="Domain-driven design with Flama 🔥",
    docs="/docs/",
    modules=[SQLAlchemyModule(DATABASE_URL)],
    components=[WorkerComponent(worker=worker.RegisterWorker())],
)

app.resources.add_resource("/user/", resources.UserResource)

@app.get("/", name="info")
def info():
    """
    tags:
        - Info
    summary:
        Ping
    description:
        Returns a brief description of the API
    responses:
        200:
            description:
                Successful ping.
    """
    return {"title": app.schema.title, "description": app.schema.description, "public": True}

It should be apparent to you already how the DDD pattern has allowed us to separate the business logic of the application (which is easily readable in the resource methods) from the data access logic (which is implemented in the repository and worker patterns). It's also worth noting how this sepration of concerns has made the code more maintainable and testable, and how the code is now more aligned with the business requirements we were given at the beginning of this example.

Running the application

Before running any command, please check that your development environment is set up correctly, and that the folder structure is as follows:

.
├── README.md
├── migration.py
├── models.db
└── src
    ├── __init__.py
    ├── __main__.py
    ├── app.py
    ├── models.py
    ├── repositories.py
    ├── resources.py
    ├── schemas.py
    └── worker.py

If everything is set up correctly, you can run the application by executing the following command (remember to run the migration script before running the application):

> poetry run flama run --server-reload src.app:app

INFO:     Will watch for changes in these directories: [...]
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [35369] using WatchFiles
INFO:     Started server process [35373]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

Now we can try the business logic we've just implemented. Remember, you can try this either by using a tool like curl or Postman, or by using the auto-generated docs UI provided by flama by navigating to http://localhost:8000/docs/ in your browser and trying the endpoints from there.

Create a user

To create a user, you can send a POST request to /user/ with the following payload:

{
  "name": "John",
  "surname": "Doe",
  "email": "john@doe.com",
  "password": "123456",
}

So, we can use curl to send the request as follows:

curl --request POST \
  --url http://localhost:8000/user/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "name": "John",
  "surname": "Doe",
  "email": "john@doe.com",
  "password": "123456"
}'

If the request is successful, you should receive a 200 response with an empty body, and the user will be created in the database.

Sign in

To sign in, you can send a POST request to /user/signin/ with the following payload:

{
  "email": "john@doe.com",
  "password": "123456"
}

So, we can use curl to send the request as follows:

curl --request POST \
  --url http://localhost:8000/user/signin/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "email": "john@doe.com",
  "password": "123456"
}'

Given that the user is not active, you should receive something like the following response:

{
  "status_code":400, 
  "detail":"User must be activated via /user/activate/",
  "error":"HTTPException"
}

We can also test what would happen if someone tries to sign in with the wrong password:

curl --request POST \
  --url http://localhost:8000/user/signin/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "email": "john@doe.com",
    "password": "1234567"
    }'

In this case, you should receive a 401 response with the following body:

{
  "status_code":401, 
  "detail":"Unauthorized",
  "error":"HTTPException"
}

Finally, we should also try to sign in with a user that doesn't exist:

curl --request POST \
  --url http://localhost:8000/user/signin/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "email": "foo@bar.com",
    "password": "123456"
    }'

In this case, you should receive a 404 response with the following body:

{
  "status_code":404, 
  "detail":"Not Found",
  "error":"HTTPException"
}

User activation

Having explored the sign in process, we can now activate the user by sending a POST request to /user/activate/ with the credentials of the user:

curl --request POST \
  --url http://localhost:8000/user/activate/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "email": "john@doe.com",
  "password": "123456" 
}'

With this request, the user should be activated, and you should receive a 200 response with an empty body.

As in the previous case, we can also test what would happen if someone tries to activate the user with the wrong password:

curl --request POST \
  --url http://localhost:8000/user/activate/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "email": "john@doe.com",
    "password": "1234567" 
}'

In this case, you should receive a 401 response with the following body:

{
  "status_code":401, 
  "detail":"Unauthorized",
  "error":"HTTPException"
}

Finally, we should also try to activate a user that doesn't exist:

curl --request POST \
  --url http://localhost:8000/user/activate/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "email": "foo@bar.com",
    "password": "123456"
}'

In this case, you should receive a 404 response with the following body:

{
  "status_code":404, 
  "detail":"Not Found",
  "error":"HTTPException"
}

User sign in after activation

Now that the user is activated, we can try to sign in again:

curl --request POST \
  --url http://localhost:8000/user/signin/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "email": "john@doe.com",
     "password": "123456"
    }'

Which, this time, should return a 200 response with the user's information:

{
  "email":"john@doe.com",
  "name":"John",
  "surname":"Doe",
  "id":"d73d4a62-dfe9-4907-91f4-f6b06f33c534",
  "active":true, 
  "password":"..."
}

User deactivation

Finally, we can deactivate the user by sending a POST request to /user/deactivate/ with the credentials of the user:

curl --request POST \
  --url http://localhost:8000/user/deactivate/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "email": "john@doe.com",
    "password": "123456"
}'

With this request, the user should be deactivated, and you should receive a 200 response with an empty body.

Conclusion

In this post we've ventured into the world of Domain-Driven Design (DDD) and how it can be implemented in a flama application. We've seen how DDD can help us to separate the business logic of the application from the data access logic, and how this separation of concerns can make the code more maintainable and testable. We've also seen how the repository and worker patterns can be implemented in a flama application, and how they can be used to encapsulate the data access logic and provide a way to group all the operations that must be performed on the data source within a single transaction. Finally, we've seen how the resource methods can be used to define the API endpoints that interact with the user data, and how the DDD pattern can be used to implement the business requirements we were given at the beginning of this example.

Although the sign-in process we've described here is not entirely realistic, you could combine the material of this and a previous post on JWT authentication to implement a more realistic process, in which the sign-in ends up returning a JWT token. If you're interested in this, you can check out the post on JWT authentication with flama.

We hope you've found this post useful, and that you're now ready to implement DDD in your own flama applications. If you have any questions or comments, feel free to reach out to us. We're always happy to help!

Stay tuned for more posts on flama and other exciting topics in the world of AI and software development. Until next time!

Support our work

If you like what we do, there is a free and easy way to support our work. Gifts us a ⭐ at Flama.

GitHub ⭐'s mean a world to us, and give us the sweetest fuel to keep working on it to help others on its journey to build robust Machine Learning APIs.

You can also follow us on 𝕏, where we share our latest news and updates, besides interesting threads on AI, software development, and much more.

References

About the authors

Vortico: We're specialised in software development to help businesses enhance and expand their AI and technology capabilities.

Protected ML APIs with Flama JWT Authentication

Vortico — Wed, 11 Sep 2024 08:30:34 +0000

You've probably heard about the recent release of Flama 1.7 already, which brought some exciting new features to help you with the development and productionalisation of your ML APIs. This post is precisely devoted to one of the main highlights of that release: Support for JWT Authentication. But, before we dive into the details with a hands-on example, we recommend you to bear in mind the following resources (and, get familiar with them if you haven't already):

Official Flama documentation: Flama documentation
Post introducing Flama for ML APIs: Introduction to Flama for Robust Machine Learning APIs

Now, let's get started with the new feature and see how you can secure your API endpoints with token-based authentication over headers or cookies.

This post is structured as follows:

What is JSON Web Token (JWT)?
JWT Authentication with Flama
- Set up the development environment
- Base application
- Flama JWT Component
- Protected endpoints
- Running the application
- Login endpoint
Conclusions
Support our work
References
About the authors

What is JSON Web Token?

If you're already familiar with the concept of JSON Web Token (JWT) and how it works, feel free to skip this section and jump straight to Implementing JWT Authentication with Flama. Otherwise, we're going to try to provide you with a succinct explanation of what JWT is and why it's so useful for authorisation purposes.

Brief introduction

We can start with the official definition given in the RFC 7519 document:

JSON Web Token (JWT) is a compact, URL-safe means of representing
claims to be transferred between two parties. The claims in a JWT
are encoded as a JSON object that is used as the payload of a JSON
Web Signature (JWS) structure or as the plaintext of a JSON Web
Encryption (JWE) structure, enabling the claims to be digitally
signed or integrity protected with a Message Authentication Code
(MAC) and/or encrypted.

So, in simple terms, JWT is an open standard that defines a way for transmitting information between two parties in the form of a JSON object. The information being transmitted is digitally signed to ensure its integrity and authenticity. This is why one of the main use cases of JWT is for authorisation purposes.

A prototypical JWT-based authentication flow would look something like this:

A user logs in to a system and receives a JWT token.
The user sends this token with every subsequent request to the server.
The server verifies the token and grants access to the requested resource if the token is valid.
If the token is invalid, the server denies access to the resource.

However, JWT tokens are not only useful to identify users, but they can also be used to share information between different services in a secure way via the payload of the token. Besides, given that the signature of the token is calculated using the header and the payload, the receiver can verify the integrity of the token and ensure that it hasn't been altered in transit.

JWT Structure

A JWT is represented as a sequence of URL-safe parts separated by dots (.), with each part containing a base64url-encoded JSON object. The number of parts in the JWT can vary depending on the representation being used, either JWS (JSON Web Signature) RFC-7515 or JWE (JSON Web Encryption) RFC-7516.
As of this point, we can assume we'll be using JWS, which is the most common representation for JWT tokens. In this case, a JWT token consists of three parts:

Header: Contains metadata about the token and the cryptographic operations applied to it.
Payload: Contains the claims (information) that the token is carrying.
Signature: Ensures the integrity of the token and is used to verify that the sender of the token is who it claims to be.

Thus, the syntax of a prototypical JWS JSON using the flattened JWS JSON Serialization is as follows (for more info, see RFC-7515 Sec. 7.2.2):



{
   "payload":"<payload contents>",
   "header":<header contents>,
   "signature":"<signature contents>"
}

In the JWS Compact Serialization, the JTW is represented as the concatenation:



BASE64URL(UTF8(JWS Header)) || '.' || BASE64URL(JWS Payload) || '.' || BASE64URL(JWS Signature)

An example of a JWT token would look like this (taken from one of the Flama tests here):



eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJkYXRhIjogeyJmb28iOiAiYmFyIn0sICJpYXQiOiAwfQ==.J3zdedMZSFNOimstjJat0V28rM_b1UU62XCp9dg_5kg=

The Header

For a general JWT object, the header can contain a variety of fields (e.g., describing the cryptographic operations applied to the token) depending on whether the JWT is a JWS or JWE. However, there are some fields which are common to both cases, and also the most commonly used:

alg: The algorithm used to sign the token, e.g. HS256, HS384, HS512. To see the list of supported algorithms in flama, have a look at it here.
typ: The type parameter is used to declare the media type of the JWT. If present, it is recommended to have the value JWT to indicate that the object is a JWT. For more information on this field, see RFC-7519 Sec. 5.1.
cty: The content type is used to communicate structural information about the JWT. For example, if the JWT is a Nested JWT, the value of this field would be JWT. For more information on this field, see RFC-7519 Sec. 5.2.

The Payload

The payload contains the claims (information) that the token is carrying. The claims are represented as a JSON object, and they can be divided into three categories, according to the standard RFC-7519 Sec. 4:

Registered claims: These are a set of predefined claims that are not mandatory but recommended. Some of the most common ones are iss (issuer), sub (subject), aud (audience), exp (expiration time), nbf (not before), iat (issued at), and jti (JWT ID). For a detailed explanation of each of these claims, see RFC-7519 Sec. 4.1.
Public claims: These are claims that are defined by the user and can be used to share information between different services.
Private claims: These are claims that are defined by the user and are intended to be shared between the parties that are involved in the communication.

The Signature

The signature is used to ensure the integrity of the token and to verify that the sender of the token is who it claims to be. The signature is calculated using the header and the payload, and it is generated using the algorithm specified in the header. The signature is then appended to the token to create the final JWT.

Implementing JWT Authentication with Flama

Having introduced the concept of JWT, its potential applications, besides a prototypical authentication flow, we can now move on to the implementation of a JWT-authenticated API using Flama. But, before we start, if you need to review the basics on how to create a simple API with flama, or how to run the API once you already have the code ready, then you might want to check out the quick start guide. There, you'll find the fundamental concepts and steps required to follow this post. Now, without further ado, let's get started with the implementation.

Setting up the development environment



poetry add flama[full]

Base application

Let's start with a simple application that has a single public endpoint. This endpoint will return a brief description of the API.



from flama import Flama

app = Flama(
    title="JWT protected API",
    version="1.0.0",
    description="JWT Authentication with Flama 🔥",
    docs="/docs/",
)

@app.get("/public/info/", name="public-info")
def info():
    """
    tags:
        - Public
    summary:
        Ping
    description:
        Returns a brief description of the API
    responses:
        200:
            description:
                Successful ping.
    """
    return {"title": app.schema.title, "description": app.schema.description, "public": True}

If you want to run this application, you can save the code above in a file called main.py under the src folder, and then run the following command (remember to have the poetry environment activated, otherwise you'll need to prefix the command with poetry run):



flama run --server-reload src.main:app

INFO:     Started server process [3267]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

Authentication: Flama JWT Component

Ok, now that we have our base application running, let's add a new endpoint that requires authentication. To do this, we'll need to use the following flama functionality:

Components (specifically JWTComponent): They're the building blocks for dependency injection in flama.
Middlewares (specifically AuthenticationMiddleware): They're used as a processing layer between the incoming requests from clients and the responses sent by the server.

Let's proceed and modify our base application to include the JWT authentication as intended, and then we'll explain the code in more detail.



import uuid

from flama import Flama
from flama.authentication import AuthenticationMiddleware, JWTComponent
from flama.middleware import Middleware

JWT_SECRET_KEY = uuid.UUID(int=0).bytes  # The secret key used to signed the token
JWT_HEADER_KEY = "Authorization"  # Authorization header identifie
JWT_HEADER_PREFIX = "Bearer"  # Bearer prefix
JWT_ALGORITHM = "HS256"  # Algorithm used to sign the token
JWT_TOKEN_EXPIRATION = 300  # 5 minutes in seconds
JWT_REFRESH_EXPIRATION = 2592000  # 30 days in seconds
JWT_ACCESS_COOKIE_KEY = "access_token"
JWT_REFRESH_COOKIE_KEY = "refresh_token"

app = Flama(
    title="JWT-protected API",
    version="1.0.0",
    description="JWT Authentication with Flama 🔥",
    docs="/docs/",
)

app.add_component(
    JWTComponent(
        secret=JWT_SECRET_KEY,
        header_key=JWT_HEADER_KEY,
        header_prefix=JWT_HEADER_PREFIX,
        cookie_key=JWT_ACCESS_COOKIE_KEY,
    )
)

app.add_middleware(
    Middleware(AuthenticationMiddleware),
)

# The same code as before here
# ...

Although we've decided to add the component and middleware after the initialisation of the app, you can also add them directly to the Flama constructor by passing the components and middlewares arguments:



app = Flama(
    title="JWT-protected API",
    version="1.0.0",
    description="JWT Authentication with Flama 🔥",
    docs="/docs/",
    components=[
        JWTComponent(
            secret=JWT_SECRET_KEY,
            header_key=JWT_HEADER_KEY,
            header_prefix=JWT_HEADER_PREFIX,
            cookie_key=JWT_ACCESS_COOKIE_KEY,
        )
    ],
    middlewares=[
        Middleware(AuthenticationMiddleware),
    ]
)

This is just a matter of preference, and both ways are completely valid.

With the modifications introduced above, we can proceed to add a new (and JWT protected) endpoint. However, before we do that, let's briefly explain in more detail the functionality we've just introduced, namely components and middlewares.

Flama Components

As you might've already noticed, whenever we create a new application we're instantiating a Flama object. As the application grows, as is the case right now, also grows the need to add more dependencies to it. Without dependency injection (DI), this would mean that the Flama class would have to create and manage all its dependencies internally. This would make the class tightly coupled to specific implementations and harder to test or modify. With DI the dependencies are provided to the class from the outside, which decouples the class from specific implementations, making it more flexible and easier to test. And, this is where components come into play in flama.

In flama, a Component is a building block for dependency injection. It encapsulates logic that determines how specific dependencies should be resolved when the application runs. Components can be thought of as self-contained units responsible for providing the necessary resources that the application's constituents need. Each Component in flama has a unique identity, making it easy to look up and inject the correct dependency during execution. Components are highly flexible, allowing you to handle various types of dependencies, whether they're simple data types, complex objects, or asynchronous operations. There are several built-in components in flama, although in this post we're going to exclusively focus on the JWTComponent, which (as all others) derives from the Component class.

The JWTComponent contains all the information and logic necessary for extracting a JWT from either the headers or cookies of an incoming request, decoding it, and then validating its authenticity. The component is initialised with the following parameters:

secret: The secret key used to decode the JWT.
header_key: The key used to identify the JWT in the request headers.
header_prefix: The prefix used to identify the JWT in the request headers.
cookie_key: The key used to identify the JWT in the request cookies.

In the code above, we've initialised the JWTComponent with some dummy values for the secret key, header key, header prefix, and cookie key. In a real-world scenario, you should replace these values with your own secret key and identifiers.

Flama Middlewares

Middleware is a crucial concept that acts as a processing layer between the incoming requests from clients and the responses sent by the server. In simpler terms, middleware functions as a gatekeeper or intermediary that can inspect, modify, or act on requests before they reach the core logic of your application, and also on the responses before they are sent back to the client.

In flama, middleware is used to handle various tasks that need to occur before a request is processed or after a response is generated. In this particular case, the task we want to handle is the authentication of incoming requests using JWT. To achieve this, we're going to use the built-in class AuthenticationMiddleware. This middleware is designed to ensure that only authorised users can access certain parts of your application. It works by intercepting incoming requests, checking for the necessary credentials (such as a valid JWT token), and then allowing or denying access based on the user's permissions which are encoded in the token.

Here’s how it works:

Initialization: The middleware is initialized with the Flama application instance. This ties the middleware to your app, so it can interact with incoming requests and outgoing responses.
Handling Requests: The __call__ method of the middleware is called every time a request is made to your application. If the request type is http or websocket, the middleware checks if the route (the path the request is trying to access) requires any specific permissions.
Permission Check: The middleware extracts the required permissions for the route from the route's tags. If no permissions are required, the request is passed on to the next layer of the application. If permissions are required, the middleware attempts to resolve a JWT token from the request. This is done using the previously discussed JWTComponent.
Validating Permissions: Once the JWT token is resolved, the middleware checks the permissions associated with the token against those required by the route. If the user’s permissions match or exceed the required permissions, the request is allowed to proceed. Otherwise, the middleware returns a 403 Forbidden response, indicating that the user does not have sufficient permissions.
Handling Errors: If the JWT token is invalid or cannot be resolved, the middleware catches the exception and returns an appropriate error response, such as 401 Unauthorized.

Adding a protected endpoint

By now, we should have a pretty solid understanding on what our code does, and how it does it. Nevertheless, we still need to see what we have to do to add a protected endpoint to our application. Let's do it:



@app.get("/private/info/", name="private-info", tags={"permissions": ["my-permission-name"]})
def protected_info():
    """
    tags:
        - Private
    summary:
        Ping
    description:
        Returns a brief description of the API
    responses:
        200:
            description:
                Successful ping.
    """
    return {"title": app.schema.title, "description": app.schema.description, "public": False}

And that's it! We've added a new endpoint that requires authentication to access. The functionality is exactly the same as the previous endpoint, but this time we've added the tags parameter to the @app.get decorator. The tag parameter can be used to specify additional metadata for an endpoint. But, if we use the special key permissions whilst using the AuthenticationMiddleware, we can specify the permissions required to access the endpoint. In this case, we've set the permissions to ["my-permission-name"], which means that only users with the permission my-permission-name will be able to access this endpoint. If a user tries to access the endpoint without the required permission, they will receive a 403 Forbidden response.

Running the application

If we put all the pieces together, we should have a fully functional application that has a public endpoint and a private endpoint that requires authentication. The full code should look something like this:



import uuid

from flama import Flama
from flama.authentication import AuthenticationMiddleware, JWTComponent
from flama.middleware import Middleware

JWT_SECRET_KEY = uuid.UUID(int=0).bytes  # The secret key used to signed the token
JWT_HEADER_KEY = "Authorization"  # Authorization header identifie
JWT_HEADER_PREFIX = "Bearer"  # Bearer prefix
JWT_ALGORITHM = "HS256"  # Algorithm used to sign the token
JWT_TOKEN_EXPIRATION = 300  # 5 minutes in seconds
JWT_REFRESH_EXPIRATION = 2592000  # 30 days in seconds
JWT_ACCESS_COOKIE_KEY = "access_token"
JWT_REFRESH_COOKIE_KEY = "refresh_token"

app = Flama(
    title="JWT-protected API",
    version="1.0.0",
    description="JWT Authentication with Flama 🔥",
    docs="/docs/",
)

app.add_component(
    JWTComponent(
        secret=JWT_SECRET_KEY,
        header_key=JWT_HEADER_KEY,
        header_prefix=JWT_HEADER_PREFIX,
        cookie_key=JWT_ACCESS_COOKIE_KEY,
    )
)

app.add_middleware(
    Middleware(AuthenticationMiddleware),
)

@app.get("/public/info/", name="public-info")
def info():
    """
    tags:
        - Public
    summary:
        Info
    description:
        Returns a brief description of the API
    responses:
        200:
            description:
                Successful ping.
    """
    return {"title": app.schema.title, "description": app.schema.description, "public": True}


@app.get("/private/info/", name="private-info", tags={"permissions": ["my-permission-name"]})
def protected_info():
    """
    tags:
        - Private
    summary:
        Info
    description:
        Returns a brief description of the API
    responses:
        200:
            description:
                Successful ping.
    """
    return {"title": app.schema.title, "description": app.schema.description, "public": False}

Running the application as before, we should see the following output:



flama run --server-reload src.main:app

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [48145] using WatchFiles
INFO:     Started server process [48149]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

If we navigate with our favourite browser to http://127.0.0.1:8000/docs/ we should see the documentation of our API as shown below:

As we can already expect, if we send a request to the public endpoint /public/info/ we should receive a successful response:



curl --request GET \
  --url http://localhost:8000/public/info/ \
  --header 'Accept: application/json'
{"title":"JWT protected API","description":"JWT Authentication with Flama 🔥","public":true}

However, if we try to access the private endpoint /private/info/ without providing a valid JWT token, we should receive a 401 Unauthorized response:



curl --request GET \
  --url http://localhost:8000/private/info/ \
  --header 'Accept: application/json'
{"status_code":401,"detail":"Unauthorized","error":null}%

Thus, we can say that the JWT authentication is working as expected, and only users with a valid JWT token will be able to access the private endpoint.

Adding the login endpoint

As we've just seen, we have a private endpoint which requires a valid JWT token to be accessed. But, how do we get this token? The answer is simple: we need to create a login endpoint that will authenticate the user and return a valid JWT token. To do this, we're going to define the schemas for the input and output of the login endpoint (feel free to use your own schemas if you prefer, for instance, adding more fields to the input schema such as username or email):



import uuid
import typing
import pydantic

class User(pydantic.BaseModel):
    id: uuid.UUID
    password: typing.Optional[str] = None


class UserToken(pydantic.BaseModel):
    id: uuid.UUID
    token: str

Now, let's add the login endpoint to our application:



import http

from flama import types
from flama.authentication.jwt import JWT
from flama.http import APIResponse

@app.post("/auth/", name="signin")
def signin(user: types.Schema[User]) -> types.Schema[UserToken]:
    """
    tags:
        - Public
    summary:
        Authenticate
    description:
        Returns a user token to access protected endpoints
    responses:
        200:
            description:
                Successful ping.
    """
    token = (
        JWT(
            header={"alg": JWT_ALGORITHM},
            payload={
                "iss": "vortico",
                "data": {"id": str(user["id"]), "permissions": ["my-permission-name"]},
            },
        )
        .encode(JWT_SECRET_KEY)
        .decode()
    )

    return APIResponse(
        status_code=http.HTTPStatus.OK, schema=types.Schema[UserToken], content={"id": str(user["id"]), "token": token}
    )

In this code snippet, we've defined a new endpoint /auth/ that receives a User object as input and returns a UserToken object as output. The User object contains the id of the user and the password (which is optional for this example, since we're not going to be comparing it with any stored password). The UserToken object contains the id of the user and the generated token that will be used to authenticate the user in the private endpoints. The token is generated using the JWT class, and contains the permissions granted to the user, in this case, the permission my-permission-name, which will allow the user to access the private endpoint /private/info/.

Now, let's test the login endpoint to see if it's working as expected. For this, we can proceed via the /docs/ page:

Or, we can use curl to send a request to the login endpoint:



curl --request POST \
  --url http://localhost:8000/auth/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "password": "string"
}'

which should return a response similar to this:



{"id":"497f6eca-6276-4993-bfeb-53cbbbba6f08","token":"eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJkYXRhIjogeyJpZCI6ICI0OTdmNmVjYS02Mjc2LTQ5OTMtYmZlYi01M2NiYmJiYTZmMDgiLCAicGVybWlzc2lvbnMiOiBbIm15LXBlcm1pc3Npb24tbmFtZSJdfSwgImlhdCI6IDE3MjU5ODk5NzQsICJpc3MiOiAidm9ydGljbyJ9.vwwgqahgtALckMAzQHWpNDNwhS8E4KAGwNiFcqEZ_04="}

That string is the JWT token that we can use to authenticate the user in the private endpoint. Let's try to access the private endpoint using the token:



curl --request GET \
  --url http://localhost:8000/private/info/ \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJkYXRhIjogeyJpZCI6ICI0OTdmNmVjYS02Mjc2LTQ5OTMtYmZlYi01M2NiYmJiYTZmMDgiLCAicGVybWlzc2lvbnMiOiBbIm15LXBlcm1pc3Npb24tbmFtZSJdfSwgImlhdCI6IDE3MjU5ODk5NzQsICJpc3MiOiAidm9ydGljbyJ9.vwwgqahgtALckMAzQHWpNDNwhS8E4KAGwNiFcqEZ_04='

which now returns a successful response:



{"title":"JWT protected API","description":"JWT Authentication with Flama 🔥","public":false}

And that's it! We've successfully implemented JWT authentication in our Flama application. We now have a public endpoint that can be accessed without authentication, a private endpoint that requires a valid JWT token to be accessed, and a login endpoint that generates a valid JWT token for the user.

Conclusions

The privatisation of some endpoints (even all at some instances) is a common requirement in many applications, even more so when dealing with sensitive data as is often the case in Machine Learning APIs which process personal or financial information, to name some examples. In this post, we've covered the fundamentals of token-based authentication for APIs, and how this can be implemented without much of a hassle using the new features introduced in the latest release of flama (introduced in a previous post). Thanks to the JWTComponent and AuthenticationMiddleware, we can secure our API endpoints and control the access to them based on the permissions granted to the user, and all this with just a few modifications to our base unprotected application.

We hope you've found this post useful, and that you're now ready to implement JWT authentication in your own flama applications. If you have any questions or comments, feel free to reach out to us. We're always happy to help!

Stay tuned for more posts on flama and other exciting topics in the world of AI and software development. Until next time!

Support our work

If you like what we do, there is a free and easy way to support our work. Gift us a ⭐ at Flama.

GitHub ⭐'s mean a world to us, and give us the sweetest fuel to keep working on it to help others on its journey to build robust Machine Learning APIs.

You can also follow us on 𝕏, where we share our latest news and updates, besides interesting threads on AI, software development, and much more.

References

About the authors

Vortico: We're specialised in software development to help businesses enhance and expand their AI and technology capabilities.

Releasing Flama 1.7

Vortico — Wed, 04 Sep 2024 09:45:05 +0000

We're back from a much-needed and relaxing summer break, and we're ready to kick off the fall with a new release of Flama. Yup, you read that right, we're releasing Flama 1.7 🎉

This release is a big one, with new features that will make your life much easier when developing and productionalising your ML APIs. The main highlights of this release are:

Support for Python 3.12: Flama now supports Python 3.12, so you can take advantage of all the new features and improvements that come with the latest version of Python.
Support for Domain Driven Design (DDD): Flama now comes with built-in support for domain-driven design with a new module named ddd. DDD is a powerful approach that helps you manage the complexity of real-world projects, particularly when dealing with intricate business logic, and complex data models. By focusing on the business domain, DDD ensures that your codebase remains aligned with business needs, making it easier to maintain, extend, and scale over time. The new module ddd comes with the essential building blocks for you to hit the ground running at full speed. For a deeper understanding on DDD, you can check out the book Architecture Patterns with Python by Harry Percival and Bob Gregory.
Support for Authentication: In this release, we've added support for authentication in Flama. You can now secure your API endpoints with token-based authentication over headers or cookies. This will help you protect your data and ensure that only authorised users can access your API, making it more secure and reliable in a few simple steps.

To better showcase these new features, we'll be publishing a couple of additional posts with highly detailed examples that will guide you through the process of using DDD and authentication with Flama.

Stay tuned for more updates and happy coding! 🚀

References

About the authors

Vortico: We're specialised in software development to help businesses enhance and expand their AI and technology capabilities.

Introducing Flama for Robust Machine Learning APIs

Vortico — Mon, 18 Dec 2023 14:31:54 +0000

Every time we use an application such as Instagram, Facebook, or Twitter, we are using an application programming interface (API) without even noticing it. APIs are everywhere, and they are the glue that connects the different services that we use every day in a seamless way.

This is precisely the reason why APIs are so important for software development. APIs allow developers to encapsulate functionality and expose it in a standardised way, so that other systems can consume it without having to know the details of the implementation. This is what we call abstraction.

For many years, APIs were mostly thought for exposing one of the following types of functionality:

Data manipulation: APIs were also used to expose data, i.e. the information that is stored in a database. For example, the information about a user, or the information about a product.
Business logic: APIs were also used to expose business logic, i.e. the rules that define how a business operates. For example, the rules that define how a user can buy a product, or the rules that define how a user can register in a system.

There has been a considerable effort in the last few years to try and standardise the way in which these type of APIs are implemented via different frameworks. However, over the last few years, a new type of functionality has become more and more popular: machine learning (ML) models; and the existing frameworks for building APIs are not well suited for this type of functionality. In this series of posts, we will learn how to build APIs using a Framework for the development of Lightweight Applications and Machine-learning Automation, also known as Flama.

Flama is an open-source Python library which establishes a standard framework for development and deployment of APIs with special focus on ML products. The main aim of Flama is to make ridiculously simple the deployment of ML APIs, simplifying (when possible) the entire process to a single line of code.

In particular, in this first post we are going to learn how to build a simple API which exposes:

A simple function which returns a string, to show how to expose business logic via an API.
A simple machine learning model (actually an ML pipeline) which returns a prediction given a certain input, to show how to expose ML models via an API.

In order to do so, we will discuss the following topics:

Setting up the development environment
Creating the project structure
Installing flama and extra dependencies for ML
Creating an ML pipeline with scikit-learn
Packaging the ML pipeline with flama
Serving the ML pipeline without code
Interacting the ML pipeline without server
Building an ML API using flama

We will finish this post with a summary of the main takeaways, and a list of resources for further reading. Hope you enjoy it!

Development environment

Python version

When dealing with software development, reproducibility is key. This is why we encourage you to use Python virtual environments to set up an isolated environment for your project. Virtual environments allow the isolation of dependencies, which plays a crucial role to avoid breaking compatibility between different projects. We cannot cover all the details about virtual environments in this post, but we encourage you to learn more about venv, pyenv or conda for a better understanding on how to create and manage virtual environments.

To create and activate a virtual environment with a compatible Python version you can take the following code snippets. Before you start working on a new project, it is very important to activate the corresponding environment. The last command line in both code snippets precisely activate the environment with pyenv or conda, respectively.

Creation of virtual environment with pyenv

pyenv install 3.11.5
pyenv virtualenv 3.11.5 flama-dev
pyenv local flama-dev

Creation of virtual environment with conda

conda create --name flama-dev python=3.11.5 --yes
conda activate flama-dev

Whilst you can use any tool to create and manage your virtual environments, we will be using pyenv throughout this series of posts. This is because pyenv is a lightweight tool which allows us to manage different Python versions and virtual environments in a very simple way, and it works great with poetry (which we will discuss later).

Packaging and dependency management

Once you have created and activated your virtual environment, you can start installing the dependencies for your project. Whenever we start a new project, we typically need to install dependencies which are not part of the Python standard library. These dependencies can be manually installed using pip or conda (as you probably know already), but that means we have to keep track of the dependencies and their versions manually! This is not only tedious, but also error-prone, because third-party dependencies usually have their own requirements, which eventually can conflict with the requirements of other dependencies. This leads to the so-called dependency hell, which is practically impossible to solve manually. This is where depenedency management tools come into play. These tools allow us to declare the dependencies of our project in a file (or via a simple command line), and they take care of looking for the correct versions of the dependencies and installing them for us, keeping track of the dependencies and their versions automatically.

We believe that poetry is currently the best tool for this purpose, besides of being the most popular one at the moment. This is why we will use poetry to manage the dependencies of our project throughout this series of posts. Poetry allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. Poetry also allows you to package your project into a distributable format and publish it to a repository, such as PyPI. We strongly recommend you to learn more about this tool by reading the official documentation.

Thus, the first thing we need to do is to install poetry in our active virtual environment. Following the official instructions, we can run the following command line:

Linux, macOS, Windows (WSL)

curl -sSL https://install.python-poetry.org | python3 -

Windows (PowerShell)

(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -

Once installed, you should follow the official instructions on how to add poetry to your path if it is not already there. You can check this by running the following command line:

poetry --version

Project structure

Now that we have our development environment ready, we can start our project. For this, we need to create a new directory where our project will live. We will call this directory flama_demo for simplicity, but feel free to use any name you prefer. The following command line will create the directory and move you into it, and will initialise a new poetry project:

mkdir flama_demo \
  && cd flama_demo \
  && poetry init --python="~=3.11.5" --no-interaction \
  && touch README.md \
  && mkdir src

This will create a pyproject.toml file with the following information:

[tool.poetry]
name = "flama-demo"
version = "0.1.0"
description = ""
authors = ["YOUR USERNAME <your.user@email.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "~=3.11.5"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Inspecting the folder flama_demo you should see the following structure:

.
├── README.md
├── pyproject.toml
└── src

With this, we have created a new poetry project with the name flama-demo, which depends on Python version 3.11.5. We have also created a README.md file, which is a good practice to document your project. Finally, we have created a src folder, which is where we will put the source code of our project.

Installing dependencies

Installing flama

Now that we have set up our project folder, we can proceed and add the dependencies of our project. We will start by adding flama as a dependency of our project. However, and before we do so, we need to know more about the extras that flama provides. Indeed, when you install flama you are installing the core functionality of the library, but there are some extra dependencies that you might need depending on the type of functionality you want to expose via your API. For example, flama provides support for the following typed data schemas:

Pydantic: A library for data validation and settings management with support for type hints.
Typesystem: A comprehensive library for data validation typically used to define typed data schemas which provides data validation and object serialization & deserialization tools.
Marshmallow: A library for data validation and serialization/deserialization.

Besides, flama also provides support for SQL databases via SQLAlchemy, an SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL. Finally, flama also provides support for HTTP clients to perform requests via httpx, a next generation HTTP client for Python.

These extras (you can check them here) are not installed by default when you install flama, but can be installed by specifying them as extras. In our current example, we will install flama with support for all the extras, so we will run the following command line:

poetry add "flama[full]"

which will add the following lines to our pyproject.toml file:

[tool.poetry.dependencies]
python = "~=3.11.5"
flama = {extras = ["full"], version = "^1.6.0"}

To learn more about the different extras contained in flama, and how to pick up only those you need for a cleaner installation, you can check the documentation.

Extra dependencies for ML

In addition to the dependencies that flama provides, we will also need to install some extra dependencies for our project. In particular, we will use:

scikit-learn: A library for machine learning in Python
numpy: A library for scientific computing in Python
pandas: A library for data analysis in Python
pyarrow: A library for efficient data interchange between Python and other languages

We can install these dependencies by running the following command line:

poetry add scikit-learn numpy pandas pyarrow

With this, we should have something like this in our pyproject.toml file:

[tool.poetry.dependencies]
python = "~=3.11.5"
flama = {extras = ["full"], version = "^1.6.0"}
scikit-learn = "^1.3.2"
numpy = "^1.26.2"
pandas = "^2.1.4"
pyarrow = "^14.0.1"

You don't need to worry if your pyproject.toml shows different versions for the dependencies, since this is exactly what poetry is for: to find the correct versions of the dependencies at the time of installation (which might be different from the ones shown here) and keep track of them for you.

With the needed dependencies already added to our project, we can proceed and install them in the active virtual environment by running the following command line:

poetry install

You can check everything is correctly installed by running:

poetry run flama --version
Flama 1.6.0

In the following, we are going to assume you have the environment correctly set up, hence it is up to you to run commands in the active virtual environment via poetry run, or simply activate the virtual environment and run the commands directly. E.g., in case you want to run the flama command line, you can do it via activating the virtual environment with poetry shell and then simply running the command line:

flama --version

ML pipeline with scikit-learn

Since the main goal of this post is to showcase how to serve an ML model via an API (which we'll call ML-API) with flama, we need an ML model to be served, indeed. We could use any ML model developed with any of the main ML libraries in Python:

However, we think it is a much better idea to quickly develop a simple ML model from scratch, showing how to build robust ML pipelines which not only contain the ML model itself, but also the pre-processing steps much needed later on when we want to make predictions with the model inside the ML-API. Failing in doing so is one of the most common mistakes when building ML-APIs, and typically leads to a lot of headaches when trying to deploy the ML model in production. Why? (you might ask). Well, because the ML model is typically trained after certain cleaning and transformation steps which ensure the data is digestible by the ML model. Thus, if we only package the ML model (typically yielding a pickle file) without the pre-processing steps, when we try to make predictions with the model (by loading the corresponding artifact) we would end up with errors, since the data we want to use for making predictions is not in the same format as the data used for training the model. As you can already see, this is a reproducibility nightmare, and we want to avoid it at all costs.

Fortunately, the main ML libraries in Python provide tools to build ML pipelines to avoid this problem. In this post, we are going to use scikit-learn to build the pipeline we want to serve via our ML-API, but you can use any other library you prefer.

The data

The goal of this post is not to build a very complex ML model by itself, but we want to go further than the prototypical Iris classification problem. For this reason, we are going to use a dataset which is a bit more complex, but still simple enough to be able to focus on the ML pipeline and the ML-API. The problem we are going to address has to do with the prediction of customer churn, i.e. the prediction of whether a customer will leave a company or not, which is a very common problem in the industry. The dataset we are going to use is a public dataset, which you can download from here. For the sake of brevity, we are not going to discuss here the details of the dataset, we will just assume that the data exploration has already been done.

Before we proceed with the ML pipeline, we are going to prepare the repository for the data and the ML pipeline script. To do so, we can run the following command lines:

mkdir -p data
mkdir -p src/pipeline
touch src/pipeline/__main__.py

And, later:

wget https://github.com/vortico/flama-demo/raw/master/data/data.parquet -O data/data.parquet
wget https://github.com/vortico/flama-demo/raw/master/data/artifact.json -O data/artifact.json
wget https://github.com/vortico/flama-demo/raw/master/data/input.json -O data/input.json

The project folder should look like this:

.
├── README.md
├── data
│   ├── artifact.json
│   ├── data.parquet
│   └── input.json
├── poetry.lock
├── pyproject.toml
└── src
    └── pipeline
        └── __main__.py

The ML pipeline

Having all the data in place, we can proceed and build the ML pipeline. As we mentioned before, the aim of the post is not the building and training a perfect ML pipeline. Instead, we want to focus our attention on how to structure the pipeline, and the project, with the ultimate goal of packaging the pipeline and serving it via an ML-API. For this reason, we are going to build a very simple ML pipeline, which will consist of the following steps:

Data loading: We will load the data from the data.parquet file into a pandas.DataFrame object.
Train-test split: We will split the data into training and test sets.
Preprocessing numerical and categorical features: We will apply some simple transformations to the numerical and categorical features of the dataset.
Training: We will train a simple ML model using the training data.
Evaluation: We will evaluate the performance of the model using the test data.

The code for this goes in the src/pipeline/__main__.py file, and it looks like this:

import numpy as np
import pandas as pd

from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import roc_auc_score, f1_score

# Set random seed for reproducibility:
np.random.seed(123_456)

# Loading data:
dataset = pd.read_parquet("data/data.parquet")

X = dataset.drop(columns=["Exited"]).values

y = dataset["Exited"].values

columns = dataset.columns

# Preprocessing numerical features:
numeric_transformer = Pipeline(
    [
        ("imputer", SimpleImputer(strategy="median")),
        ("scaler", StandardScaler()),
    ]
)
numeric_features = [
    columns.get_loc(c)
    for c in dataset.select_dtypes(include=["int64", "float64"])
    .drop(["RowNumber", "CustomerId", "Exited"], axis=1)
    .columns.values
]

# Preprocessing categorical features:
categorical_transformer = Pipeline(
    [
        ("imputer", SimpleImputer(strategy="constant", fill_value="missing")),
        ("onehot", OneHotEncoder(handle_unknown="ignore")),
    ]
)
categorical_features = [
    columns.get_loc(c)
    for c in dataset.select_dtypes(include=["object"])
    .drop(["Surname"], axis=1)
    .columns.values
]

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)
preprocessor = ColumnTransformer(
    [
        ("numerical", numeric_transformer, numeric_features),
        ("categorical", categorical_transformer, categorical_features),
    ]
)

# Model train:
mlp = MLPClassifier(
    hidden_layer_sizes=(8, 6, 1),
    max_iter=300,
    activation="tanh",
    solver="adam",
    random_state=123_456,
)
pipeline = Pipeline(
    [
        ("preprocessing", preprocessor),
        ("mlp_classifier", mlp),
    ]
)
pipeline.fit(X_train, y_train)

# Model evaluation:
pipeline.score(X_test, y_test)
pipeline.predict(X_test)

print(f"""
Model trained successfully:
    * Accuracy: {pipeline.score(X_test, y_test)}
    * Predictions: {pipeline.predict(X_test)}
""")

You can run this script by simply running the following command line:

python -m src.pipeline

Model trained successfully:
    * Accuracy: 0.848
    * Predictions: [0 1 0 ... 0 0 0]

As you can see, the script trains a simple MLP classifier with 3 hidden layers, and it evaluates the performance of the model using the test data. The accuracy of the model is not great, but that's beyond the scope of this post. The important thing here is that we have a simple ML pipeline which we can use to make predictions when new data comes in, and we have tested it by evaluating the performance of the model with the test data.

Packaging the ML pipeline: FLM files

Why packing ML pipelines and models?

Before we proceed, it is worth discussing why we need to pack ML pipelines and models in the first place. There are several reasons for this, but we will focus on the following ones:

Persistence: The ability to persist ML pipelines and models allows us to reuse them, either for further training, or for making predictions on new data.
Efficiency: Binary files are compact, efficient, and easy way to store pipelines and models.
Portability: Having ML pipelines self-contained in a single file allows us to transfer and use them across different environments.

How to pack ML pipelines?

Let us assume we are where we are after careful experimentation, cross-validation, testing, and so on, and we have found the optimal ML model for our problem. Great job!

Now, we want to take our model out of our Jupyter Notebook, or Python script, and offer it as a service to make predictions on demand. The first thing we think about is pickling (i.e., using pickle.dump) the model, and pass the resulting file to the corresponding team (or colleague) to develop the wrapper API which will have to eventually unpickle (i.e., using pickle.load) the object, and expose the predict method. It seems like a very repetitive and boring task, doesn't it?

Flama comes equipped with a very convenient CLI which does all the boring part for you seamlessly, just with a single line of code. For this, we only need our models to be packaged with the Flama counterparts of pickle's dump and load commands, namely: flama.dump and flama.load.

Dumping the ML pipeline

Flama's dump method uses optimal compression with the aim of making the packing process more efficient, and faster. In our case, we can dump the ML pipeline prepared before by adding the following lines to the src/pipeline/__main__.py file:

# Add these new imports:
import flama 

import datetime
import uuid

from sklearn.metrics import roc_auc_score, f1_score

# Here the script continues as before...

# Model dump:
id_ = uuid.UUID("e9d4a470-1eb2-423e-8fb3-eaf236158ab3")
path_ = "data/model.flm"
flama.dump(
    pipeline,
    path_,
    model_id=id_,
    timestamp=datetime.datetime.now(),
    params={"solver": "adam", "random_state": 123_456, "max_iter": 300},
    metrics={
        "accuracy": pipeline.score(X_test, y_test),
        "roc_auc_score": roc_auc_score(y_test, pipeline.predict(X_test)),
        "f1_score": f1_score(y_test, pipeline.predict(X_test)),
    },
    extra={
        "model_author": "Vortico",
        "model_description": "Churn classifier",
        "model_version": "1.0.0",
        "tags": ["loss", "churn"],
    },
    artifacts={"artifact.json": "./data/artifact.json"},
)

print(f"""
Model saved successfully:
    * File: {path_}
    * Id: {id_}
""")

The first two parameters are the ML pipeline object itself, and the path where the resulting file will be stored, respectively. The remaining parameters are optional, and are used to add metadata to the resulting file which might be quite useful for model management purposes. For further details on the use of these parameters, you can check the official documentation.

At this point, if you run the script again, you should see a new file in the data folder called model.flm. This is the file we will use later on to serve the ML pipeline via an ML-API. Indeed, the running of the script should produce the following output:

python -m src.pipeline

Model trained successfully:
    * Accuracy: 0.848
    * Predictions: [0 1 0 ... 0 0 0]

Model saved successfully:
    * File: data/model.flm
    * Id: e9d4a470-1eb2-423e-8fb3-eaf236158ab3

FLM files

OK, so we have the ML pipeline packaged as an FLM file, but what's that? FLM stands for Flama Lightweight Model. This comes from the fact that, FLM files are a lightweight representation of ML models, which come with useful metadata needed for later purposes, e.g. building a wrapper Flama API containing the model. To know more about FLM files, you can check the official documentation.

The structure of an FLM file is thought to be as simple as possible, and aims at keeping in a single file all the information needed to load and use the model. The structure of an FLM file is as follows:

├── model.flm
│   └── model
│       ├── model (python object)
│       └── meta
│           ├── id
│           ├── timestamp
│           ├── framework
│           ├── model
│           │   ├── obj
│           │   ├── info
│           │   ├── params
│           │   └── metrics
│           └── extra
└── artifacts
    ├── foo.json
    └── bar.csv

As you can see, the FLM file contains a wide range of information about the pipeline and model.

Loading the ML pipeline

We have an FLM file, and we would like to load it to make predictions, and so on. How can we do that? Well, we can use Flama's load method, which is the counterpart of dump. In our case, we can load the ML pipeline by adding the following lines to the src/pipeline/__main__.py file:

# Here the script continues as before...

# Load the ML pipeline:
pipeline = flama.load("./data/model.flm")
print(f"""
Model loaded successfully:
    * Id: {pipeline.meta.id}
    * Trained at: {pipeline.meta.timestamp}
    * Metrics: {pipeline.meta.model.metrics}
    * Author: {pipeline.meta.extra['model_author']}
""")

If you run the script again, you should see the following output:

python -m src.pipeline

Model trained successfully:
    * Accuracy: 0.848
    * Predictions: [0 1 0 ... 0 0 0]


Model saved successfully:
    * File: data/model.flm
    * Id: e9d4a470-1eb2-423e-8fb3-eaf236158ab3


Model loaded successfully:
    * Id: e9d4a470-1eb2-423e-8fb3-eaf236158ab3
    * Trained at: 2023-12-15 17:27:18.983044
    * Metrics: {'accuracy': 0.848, 'roc_auc_score': 0.7050107038056138, 'f1_score': 0.5571095571095571}
    * Author: Vortico

Hooray! 🥳🥳🥳 We have all the ingredients needed to build our ML API, and we have tested them to make sure everything works as expected. Now, we can proceed and see the magic of Flama in action.

Serving the ML pipeline: Flama CLI

The challenge

Once we have an ML pipeline trained and saved as a binary file, the process of serving the model usually requires addressing the following points:

Develop a service wrapping the model
Tackle the problem of the model lifecycle
Define an interaction interface

This is a common pattern present in (almost) every ML project, and it lacks a standard to follow so far.

This challenge can materialise in two different ways:

Synchronous: The client requests a prediction to the server and waits for the response. This can be solved with an HTTP service.
Asynchronous: An event occurs and the server communicates the prediction to the client. This can be solved with a streaming service, or even with a websocket.

Good news? Flama brings all the tools needed to address both cases.

The solution: Flama CLI

As we already saw at the very beginning of this post, Flama comes with the convenient flama command-line interface (CLI) at your disposal. Indeed, we already used it before when checking if all was correctly installed. In this section, we are going to deep dive a bit more into the CLI. For thi purpose, you only need to run the following command line:

flama --help

Usage: flama [OPTIONS] COMMAND [ARGS]...

  Fire up your models with Flama 🔥

Options:
  --version  Check the version of your locally installed Flama
  --help     Get help about how to use Flama CLI

Commands:
  model  Interact with an ML model without server.
  run    Run a Flama Application based on a route.
  serve  Serve an ML model file within a Flama Application.
  start  Start a Flama Application based on a config file.

The `serve` command

The command serve comes to the rescue of those who are looking for an instantaneous serving of an ML model without having to write an app. This command, as we are about to see, only requires the file with the ML model to be served. The model will have to be saved as a binary file beforehand by using the tools offered by Flama, as we have just seen.

Without much further ado, let's see how to serve the model we have just trained and saved:

flama serve "data/model.flm"
INFO:     Started server process [64069]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

What just happened? As you can see we have a Flama application up and running on http://127.0.0.1 and listening the port 8000. But, we have not written a single line of code! This is the codeless approach of Flama to ML models deployment, which makes possible the deployment of an already trained-and-tested ML model as an API ready to receive requests and return estimations almost without any effort, as you can see.

Before we proceed and make some requests to the ML-API, we can click on the link http://127.0.0.1:8000/ (or, simply, open it with your favourite browser) and check what is there waiting for us. You should see something like this:

{"meta":{"id":"e9d4a470-1eb2-423e-8fb3-eaf236158ab3","timestamp":"2023-12-15T17:27:18.983044","framework":{"lib":"sklearn","version":"1.3.2"},"model":{"obj":"Pipeline","info":{"memory":null,"steps":[["preprocessing",null],["mlp_classifier",null]],"verbose":false,"preprocessing":null,"mlp_classifier":null,"preprocessing__n_jobs":null,"preprocessing__remainder":"drop","preprocessing__sparse_threshold":0.3,"preprocessing__transformer_weights":null,"preprocessing__transformers":[["numerical",null,[3,6,7,8,9,10,11,12]],["categorical",null,[4,5]]],"preprocessing__verbose":false,"preprocessing__verbose_feature_names_out":true,"preprocessing__numerical":null,"preprocessing__categorical":null,"preprocessing__numerical__memory":null,"preprocessing__numerical__steps":[["imputer",null],["scaler",null]],"preprocessing__numerical__verbose":false,"preprocessing__numerical__imputer":null,"preprocessing__numerical__scaler":null,"preprocessing__numerical__imputer__add_indicator":false,"preprocessing__numerical__imputer__copy":true,"preprocessing__numerical__imputer__fill_value":null,"preprocessing__numerical__imputer__keep_empty_features":false,"preprocessing__numerical__imputer__missing_values":null,"preprocessing__numerical__imputer__strategy":"median","preprocessing__numerical__scaler__copy":true,"preprocessing__numerical__scaler__with_mean":true,"preprocessing__numerical__scaler__with_std":true,"preprocessing__categorical__memory":null,"preprocessing__categorical__steps":[["imputer",null],["onehot",null]],"preprocessing__categorical__verbose":false,"preprocessing__categorical__imputer":null,"preprocessing__categorical__onehot":null,"preprocessing__categorical__imputer__add_indicator":false,"preprocessing__categorical__imputer__copy":true,"preprocessing__categorical__imputer__fill_value":"missing","preprocessing__categorical__imputer__keep_empty_features":false,"preprocessing__categorical__imputer__missing_values":null,"preprocessing__categorical__imputer__strategy":"constant","preprocessing__categorical__onehot__categories":"auto","preprocessing__categorical__onehot__drop":null,"preprocessing__categorical__onehot__dtype":null,"preprocessing__categorical__onehot__feature_name_combiner":"concat","preprocessing__categorical__onehot__handle_unknown":"ignore","preprocessing__categorical__onehot__max_categories":null,"preprocessing__categorical__onehot__min_frequency":null,"preprocessing__categorical__onehot__sparse":"deprecated","preprocessing__categorical__onehot__sparse_output":true,"mlp_classifier__activation":"tanh","mlp_classifier__alpha":0.0001,"mlp_classifier__batch_size":"auto","mlp_classifier__beta_1":0.9,"mlp_classifier__beta_2":0.999,"mlp_classifier__early_stopping":false,"mlp_classifier__epsilon":1e-08,"mlp_classifier__hidden_layer_sizes":[8,6,1],"mlp_classifier__learning_rate":"constant","mlp_classifier__learning_rate_init":0.001,"mlp_classifier__max_fun":15000,"mlp_classifier__max_iter":300,"mlp_classifier__momentum":0.9,"mlp_classifier__n_iter_no_change":10,"mlp_classifier__nesterovs_momentum":true,"mlp_classifier__power_t":0.5,"mlp_classifier__random_state":123456,"mlp_classifier__shuffle":true,"mlp_classifier__solver":"adam","mlp_classifier__tol":0.0001,"mlp_classifier__validation_fraction":0.1,"mlp_classifier__verbose":false,"mlp_classifier__warm_start":false},"params":{"solver":"adam","random_state":123456,"max_iter":300},"metrics":{"accuracy":0.848,"roc_auc_score":0.7050107038056138,"f1_score":0.5571095571095571}},"extra":{"model_author":"Vortico","model_description":"Churn classifier","model_version":"1.0.0","tags":["loss","churn"]}},"artifacts":{"artifact.json":"/var/folders/h4/vc_99fk53b93ttsv18ss8vhw0000gn/T/tmpomibzc6l/artifacts/artifact.json"}}

Indeed, we can get the same response by running the command line:

curl http://127.0.0.1:8000/

{"meta":{"id":"e9d4a470-1eb2-423e-8fb3-eaf236158ab3","timestamp":"2023-12-15T17:27:18.983044","framework":{"lib":"sklearn","version":"1.3.2"},"model":{"obj":"Pipeline","info":{"memory":null,"steps":[["preprocessing",null],["mlp_classifier",null]],"verbose":false,"preprocessing":null,"mlp_classifier":null,"preprocessing__n_jobs":null,"preprocessing__remainder":"drop","preprocessing__sparse_threshold":0.3,"preprocessing__transformer_weights":null,"preprocessing__transformers":[["numerical",null,[3,6,7,8,9,10,11,12]],["categorical",null,[4,5]]],"preprocessing__verbose":false,"preprocessing__verbose_feature_names_out":true,"preprocessing__numerical":null,"preprocessing__categorical":null,"preprocessing__numerical__memory":null,"preprocessing__numerical__steps":[["imputer",null],["scaler",null]],"preprocessing__numerical__verbose":false,"preprocessing__numerical__imputer":null,"preprocessing__numerical__scaler":null,"preprocessing__numerical__imputer__add_indicator":false,"preprocessing__numerical__imputer__copy":true,"preprocessing__numerical__imputer__fill_value":null,"preprocessing__numerical__imputer__keep_empty_features":false,"preprocessing__numerical__imputer__missing_values":null,"preprocessing__numerical__imputer__strategy":"median","preprocessing__numerical__scaler__copy":true,"preprocessing__numerical__scaler__with_mean":true,"preprocessing__numerical__scaler__with_std":true,"preprocessing__categorical__memory":null,"preprocessing__categorical__steps":[["imputer",null],["onehot",null]],"preprocessing__categorical__verbose":false,"preprocessing__categorical__imputer":null,"preprocessing__categorical__onehot":null,"preprocessing__categorical__imputer__add_indicator":false,"preprocessing__categorical__imputer__copy":true,"preprocessing__categorical__imputer__fill_value":"missing","preprocessing__categorical__imputer__keep_empty_features":false,"preprocessing__categorical__imputer__missing_values":null,"preprocessing__categorical__imputer__strategy":"constant","preprocessing__categorical__onehot__categories":"auto","preprocessing__categorical__onehot__drop":null,"preprocessing__categorical__onehot__dtype":null,"preprocessing__categorical__onehot__feature_name_combiner":"concat","preprocessing__categorical__onehot__handle_unknown":"ignore","preprocessing__categorical__onehot__max_categories":null,"preprocessing__categorical__onehot__min_frequency":null,"preprocessing__categorical__onehot__sparse":"deprecated","preprocessing__categorical__onehot__sparse_output":true,"mlp_classifier__activation":"tanh","mlp_classifier__alpha":0.0001,"mlp_classifier__batch_size":"auto","mlp_classifier__beta_1":0.9,"mlp_classifier__beta_2":0.999,"mlp_classifier__early_stopping":false,"mlp_classifier__epsilon":1e-08,"mlp_classifier__hidden_layer_sizes":[8,6,1],"mlp_classifier__learning_rate":"constant","mlp_classifier__learning_rate_init":0.001,"mlp_classifier__max_fun":15000,"mlp_classifier__max_iter":300,"mlp_classifier__momentum":0.9,"mlp_classifier__n_iter_no_change":10,"mlp_classifier__nesterovs_momentum":true,"mlp_classifier__power_t":0.5,"mlp_classifier__random_state":123456,"mlp_classifier__shuffle":true,"mlp_classifier__solver":"adam","mlp_classifier__tol":0.0001,"mlp_classifier__validation_fraction":0.1,"mlp_classifier__verbose":false,"mlp_classifier__warm_start":false},"params":{"solver":"adam","random_state":123456,"max_iter":300},"metrics":{"accuracy":0.848,"roc_auc_score":0.7050107038056138,"f1_score":0.5571095571095571}},"extra":{"model_author":"Vortico","model_description":"Churn classifier","model_version":"1.0.0","tags":["loss","churn"]}},"artifacts":{"artifact.json":"/var/folders/h4/vc_99fk53b93ttsv18ss8vhw0000gn/T/tmpomibzc6l/artifacts/artifact.json"}}

As you can see, the response is a JSON object containing all the information about the model, including the metadata we added when saving the model as an FLM file. This is very useful, since it allows us to check the model is the one we want to serve, and also to check the metadata of the model, which might be useful for model management purposes.

Auto documentation

Besides, the serve command also provides an interactive documentation of the ML-API which we can access and test by opening the following link in our browser: http://127.0.0.1:8000/docs/. You should see something like this:

The documentation is automatically generated by Flama based on the information contained in the FLM file, and on the parameters passed to the serve command. The list of parameters accepted by the serve command can be found in the documentation. In short, we can find the following groups of parameters:

Model parameters
- model-url: Route of the model (default: /)
- model-name: Name of the model (default: model)
App parameters
- app-debug: Enable debug mode (default: False)
- app-title: Name of the application (default: Flama)
- app-version: Version of the application (default: 0.1.0)
- app-description: Description of the application (default: Fire up with the flame)
- app-docs: Description of the application (default: Fire up with the flame)
- app-schema: Description of the application (default: Fire up with the flame)

The parameter app-debug brings with it useful tools which make the debugging of the code easier, e.g. highly-detailed error messages, and interactive error webpages.

Server parameters: All uvicorn options can be passed to the command serve with the format server-, as we discussed for the command run, e.g.:
- server-host: Bind socket to this host (default: 127.0.0.1)
- server-port: Bind socket to this port (default: 8000)

Knowing this, we can fine tune a bit our previous command line to make the automatically-generated documentation more appropriate for our ML-API:

flama serve "data/model.flm" \
  --app-title "Churn classifier" \
  --app-description "Predict whether a customer will leave a company or not" \
  --app-version "1.0.0"

which should produce the following output:

We encourage you to play around with the different parameters of the serve command to see how they work and help you customise the ML-API.

Making predictions

With the ML-API up and running, we can proceed and make some predictions. For this, we can use the predict endpoint, which is automatically generated by Flama. We can make predictions by sending a POST request to the endpoint http://127.0.0.1:8000/predict/ (or, directly using the documentation of the predict endpoint). As an example, we can use the input data which we downloaded before under data/input.json as follows:

curl --request POST \
  --url http://127.0.0.1:8000/predict/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "input": [[10000, 15628319, "Walker", 792, "France", "Female", 28, 4, 130142.79, 1, 1, 0, 38190.78]]
}'

{"output":[0]}

or, with the documentation page:

As you can see, the response is a JSON object containing the prediction of the model. In this case, the model predicts the customer will not leave the company.

Serverless interaction with the model

There is an alternative way to interact with the model stored as FLM file, which is by using the model command of the Flama CLI. This command allows us to interact with the model without having to start a server. Even though the main purpose of this post is to show how to serve an ML model via an API, we think it iws worth mentioning this alternative way of interacting with the model, since it might be useful in some cases. In any case, we plan to discuss this topic in a much more detailed way in future posts, so stay tuned for a more in-depth discussion on this. Also, you can check the official documentation if you want to deep dive by yourself in the meantime.

The `model` command

In some cases, we want to interact with the model directly without the overhead of a server, e.g.:

Development & testing: We are working with a model locally and want to try it out on some data to quickly see whether everything is working as expected. This is typically what happens when we're in the development stage of the ML lifecycle.
Streaming workflow: This is also the case when we want to use a model as part of a larger pipeline where the model acts as a data processor in a stream of data, in which case we want to be able to pipe data into the model and get the output back.

The command model allows us to interact with models directly from the command line without the need for a server. To inspect the command options, run:

flama model --help

Usage: flama model [OPTIONS] FLAMA_MODEL_PATH COMMAND [ARGS]...

  Interact with an ML model without server.

  This command is used to directly interact with an ML model without the need
  of a server. This command can be used to perform any operation that is
  supported by the model, such as inspect, or predict. <FLAMA_MODEL_PATH> is
  the path of the model to be used, e.g. 'path/to/model.flm'. This can be
  passed directly as argument of the command line, or by environment variable.

Options:
  --help  Show this message and exit.

Commands:
  inspect  Inspect an ML model.
  predict  Make a prediction using an ML model.

We can check readily, by simply running:

flama model "data/model.flm" inspect

that the inspect subcommand gives us exactly the same information we got when we made a request to the root endpoint of the ML-API.

Very likely, we are much more interested in the predict subcommand, which allows us to make predictions using the model. We can use an input file to be passed to the model, as we did manually before with the /predict/ path of the ML-API. To do this, we only need to run:

flama model data/model.flm predict --file data/input.json --pretty

[
    0
]

The flag --pretty is used to print the output in a more human-readable way. The main advantage of this approach is that we can make predictions on a bunch of input data by simply passing the input file path. This is useful for local development and quick checks.

The previous command line is completely equivalent to the following one, which is more suitable for streaming services, or pipe several models together:

echo '[[10000, 15628319, "Walker", 792, "France", "Female", 28, 4, 130142.79, 1, 1, 0, 38190.78]]' | \
    flama model data/model.flm predict --pretty

[
    0
]

This, as just mentioned, is pretty useful if we want to chain several models, and make the output of one the output of the next in chain. E.g., imagine we have n models in files: model_1.flm, ..., model_n.flm, then we can pipe them as follows:

echo '[[<INPUT_DATA>]]' | \
    flama model model_1.flm predict | \
    flama model model_2.flm predict | \
    ...
    flama model model_n.flm predict

Custom ML APIs with Flama

So far, we have seen how to serve an ML pipeline with Flama CLI in a very simple way, without the need of writing a single line of code. This is a very powerful feature we cannot stress enough, not only because it allows to quickly have an ML-API up and running, but also (and more importantly) because it establishes a standard of communication between those who develop, train and test the ML pipeline; and those who have to put it into production. This is a very important point, since it allows to reduce the friction between the different teams involved in the ML lifecycle, and to make the process of putting ML pipelines into production much more efficient.

Why custom ML APIs?

Despite all the good things we have seen so far, there are cases where the ML-API provided by the Flama CLI is not enough, and we need to develop a custom ML-API. This is the case, for instance, when we need:

Further functionality: The ML-API provided by the Flama CLI comes with a set of endpoints which are enough for most cases. However, there might be some cases where we need to add further functionality to the ML-API, e.g. we might need to add a new endpoint to the ML-API to perform some specific task.
Fine control: The ML-API provided by the Flama CLI is a "one size fits all" solution, which means that it is not possible to fine tune the behaviour of the ML-API. For instance, we might need to add some startup or shutdown events to the ML-API.
Learning by doing: We might want to learn how to develop an ML-API from scratch, and Flama provides the perfect framework to do so.

Flama Application

At the very beginning of this post we mentioned that Flama is a framework for the development of lightweight applications. Indeed, when we run the serve command before, a Flama application was created, equipped with the set of endpoints we saw before (i.e., /, /docs/, /schema/, and /predict/), and run under the hood, without you ever noticing. Specifically:

/ is the root endpoint of the ML-API, which returns the inspection of the model (as we saw before with flama model inspect).
/docs/ is the endpoint which provides the automatically-generated documentation of the ML-API.
/predict/ is the endpoint which allows us to make predictions using the corresponding predict() method of the ML pipeline. This endpoint is in charge of unpacking the FLM file, and calling the predict() method.

If we want to develop a custom ML-API, we need to start by creating a Flama application, and we will do so taking some inspiration from the official documentation. For this, we will create a new folder called src/api:

mkdir src/api

and will add a new file called src/api/app.py with the following content:

from flama import Flama

app = Flama(
    title="Flama ML",
    version="1.0.0",
    description="Machine learning API using Flama 🔥",
    docs="/docs/"
)

@app.get("/")
def home():
    """
    tags:
        - Home
    summary:
        Returns warming message
    description:
        The function returns a hello message
    """
    return "Hello 🔥"

And we can run the application by simply running:

flama run src.api.app:app

INFO:     Started server process [68609]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

To know more about the run command of the Flama CLI, please have a look at the documentation.

If we want to run the application as a Python script, we can add the following lines to the file src/api/__main__.py:

import flama

def main():
    flama.run(flama_app="src.api.app:app", server_host="0.0.0.0", server_port=8000, server_reload=True)

if __name__ == "__main__":
    main()

and run the application by simply running:

python -m src.api

INFO:     Started server process [69083]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

Wonderful! 🥳🥳🥳 We have our Flama application up and running, with a custom endpoint which returns a warming message.
Now, we can proceed and add the ML pipeline we trained before to the application.

In case you did not notice, the fact that we used the parameter server_reload=True with flama.run in the __main__.py file activated the hot reloading feature of Flama, which means that the application will be reloaded every time we make a change to the code. This is very useful for development purposes, since it allows us to make changes to the code without having to restart the application every time.

Add the ML resource

Let us assume we want our ML-API to satisfy the following requirements:

The churn pipeline must be exposed under the route /churn/
We want to have exposed the predict method of the pipeline as we already had before
We want to have a new endpoint which returns the estimation of financial loss due to churn of a given customer (more details below).

For the first two requirements, we are going to use three built-in ingredients of Flama which we haven't seen yet, namely:

ModelResource: This is a built-in class which derives from the most fundamental resource class BaseResource. These classes can be seen as the interface that any custom model class will need to adhere to. In case you're not familiar with the interface design pattern, it can be very succinctly described as a blueprint for designing classes.
ModelResourceType: This is a built-in metaclass which derives from the most fundamental resource type metaclass ResourceType. The concept of metaclass is a bit less intuitive, and some highly recognised pythonists recommend to avoid them at all costs. We agree with such a claim, which is why you won't need to worry about them, apart from having to import the ModelResourceType, and use it as an argument when building your custom model class.
resource_method decorator: This is a built-in decorator which allows us to convert a given class method into an app endpoint, with the following main arguments:
- path: Route path through which we'll be able to call the decorated method, e.g. /predict/
- methods: HTTP methods associated with the endpoint, e.g. GET or POST
- name: Name which uniquely determines the route being added to the app.

Let's add the ML pipeline to the application by adding the following lines to the file src/api/app.py:

# Add these imports:
from flama.models import ModelResource, ModelResourceType

# Here the script continues as before...

# Add the ML pipeline:

class ChurnModel(ModelResource, metaclass=ModelResourceType):
    name = "churn_model"
    verbose_name = "Churn model"
    model_path = "data/model.flm"

app.models.add_model_resource(path="/churn", resource=ChurnModel)

As you can see, we have created a new class called ChurnModel which derives from ModelResource, and we have added it to the application by using the add_model_resource method of the models attribute of the application. This method can take several arguments, but we are mostly interested in the following ones:

path: Route path where the method predict will be placed, e.g. path="/churn/" means that the method predict will be placed under the route /churn/predict/.
resource: Resource class to be added to the application.

Given that the class ChurnModel inherits from ModelResource and uses ModelResourceType as metaclass, it will have the following methods:

/: Returns the inspection of the model.
/predict/: Returns the prediction of the model.

We can check this by running the application again:

flama run src.api.app:app

python -m src.api

and opening the documentation page at http://127.0.0.1:8000/docs/.

With the previous code, we are already serving the ML pipeline as we wanted, and satisfying the requirements (1) and (2).
What about the custom endpoint we wanted to add?

Add a custom endpoint to the Churn model resource

It seems we have everything almost ready to just add the custom endpoint we were told to add. Let us imagine our data-science colleague has developed a formula to estimate the financial loss due to the churn of a given customer, which looks like:

loss[$]=P(churn)×(N×Income[$]−OPEX[$])\text{loss}[\$] = \mathbb{P}({\text{churn}}) \times \left(N \times \text{Income}[\$] - \text{OPEX}[\$]\right)

where:

Average income ( $Income[$]\text{Income}[\$]$ ): Estimated average annual income of the customer
Churn probability ( $P(churn)\mathbb{P}({\text{churn}})$ ): Probability of churn of the customer which we get from the ML pipeline (i.e., the output of the predict method).
Agents per client ( $N$ ): Number of agents assigned to the customer
Operational cost ( $OPEX[$]\text{OPEX}[\$]$ ): Operational cost of the company

Out of the four parameters of the formula, we only have the first two: the estimated annual income of the customer (which will be passed as the ninth element of the input data), and the churn probability (which we get from the predict method of the ML pipeline). The other two parameters are not part of the input, but are parameters which were known/computed during the training stage, and were stored together with the ML pipeline as part of the FLM file, under the artifacts attribute of the dump method of the pipeline (see, artifacts={"artifact.json": "./data/artifact.json"}). This is very useful, since it allows us to use such information which was computed during the training stage, and use it in the production stage without having to recompute it again, or having to introduce it manually in the code of the ML-API (typically as constants, or in a configuration file).

We can add the custom endpoint to the ChurnModel class by adding the following lines to the file src/api/app.py:

# Add these imports:
import json

import numpy as np

from flama import schemas, types
from flama.models import ModelResource, ModelResourceType
from flama.resources import resource_method

# Here the script continues as before...

# Add the ML pipeline:
class ChurnModel(ModelResource, metaclass=ModelResourceType):
    name = "churn_model"
    verbose_name = "Churn model"
    model_path = "data/model.flm"

    @resource_method("/loss/", methods=["POST"], name="model-predict-loss")
    def predict_loss(
        self, data: types.Schema[schemas.schemas.MLModelInput]
    ) -> types.Schema[schemas.schemas.MLModelOutput]:
        """
        tags:
            - Churn model
        summary:
            Get loss from churn
        description:
            Computes the loss amount estimated according to the model parameters
            provided in the model artifacts, with loss being the product of the
            churn probability and the estimated salary of the client minus the
            operational cost of the company.
        """
        with open(self.model.artifacts["artifact.json"]) as f:
            params = json.load(f)

        x = np.array(data["input"])
        proba = self.model.model.predict_proba(x)[:, 0]

        loss = proba * (
            params["agents_per_client"] * x[:, 8].astype(float) - params["operational_cost"]
        )
        return types.Schema[schemas.schemas.MLModelOutput]({"output": loss.tolist()})

app.models.add_model_resource(path="/churn", resource=ChurnModel)

After adding the new code to the application source file, Flama will reload the application automatically, and you should see the new endpoint /churn/loss/ in the documentation page. To test it, you can either use the documentation page, or run the following command line:

curl --request POST \
  --url http://127.0.0.1:8000/churn/loss/ \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "input": [[10000, 15628319, "Walker", 792, "France", "Female", 28, 4, 130142.79, 1, 1, 0, 38190.78]]
}'

{"output":[5212.2945950392195]}%

Great news! 🥳🥳🥳 We have successfully added a custom endpoint to the ML-API, and we have tested it to make sure it works as expected. In this case, we have an estimated loss of $5212.29 due to the churn of the customer.

Conclusions

In this post we have seen how to build and serve a robust ML pipeline with Flama. We have seen how to train and save an ML pipeline, and how to serve it as an API with the Flama CLI. We have also seen how to interact with the model without the need of a server, and how to develop a custom ML-API with Flama. Finally, we have also shown how to add custom endpoints to the ML-API, and how to use the information stored in FLM files, e.g. to compute the output of a custom endpoint.

We certainly believe that Flama is a very powerful tool which can help you to put your ML pipelines into production in a very efficient way. We encourage you to try it out, and to give us feedback on how to improve it. We are looking forward to hearing from you! 🤗

Finally, we would like to thank you for reading this post, and we hope you enjoyed it. If you have any questions, please do not hesitate to contact us. We will be more than happy to help you! 🤓

References

About the authors

Vortico: We are specialised in software development to helps businesses enhance and expand their AI and technology capabilities.

DEV Community: Vortico

Releasing Flama 1.10

References

About the authors

Native Domain-Driven Design with Flama

Table of contents

What is Domain-Driven Design?

Brief Overview

Key Concepts

Domain Model

Repository pattern

Unit of work pattern

Implementing DDD with Flama

Setting up the development environment

Base application

DDD in action

Data model

Repository

Worker

Resources

Base application with DDD

Running the application

Create a user

Sign in

User activation

User sign in after activation

User deactivation

Conclusion

Support our work

References

About the authors

Protected ML APIs with Flama JWT Authentication

Table of contents

What is JSON Web Token?

Brief introduction

JWT Structure

The Header

The Payload

The Signature

Implementing JWT Authentication with Flama

Setting up the development environment

Base application

Authentication: Flama JWT Component

Flama Components

Flama Middlewares

Adding a protected endpoint

Running the application

Adding the login endpoint

Conclusions

Support our work

References

About the authors

Releasing Flama 1.7

References

About the authors

Introducing Flama for Robust Machine Learning APIs

Development environment

Python version

Creation of virtual environment with pyenv

Creation of virtual environment with conda

Packaging and dependency management

Linux, macOS, Windows (WSL)

Windows (PowerShell)

Project structure

Installing dependencies

Installing flama

Extra dependencies for ML

ML pipeline with scikit-learn

The data

The ML pipeline

Packaging the ML pipeline: FLM files

Why packing ML pipelines and models?

How to pack ML pipelines?

Dumping the ML pipeline

FLM files

Loading the ML pipeline

Serving the ML pipeline: Flama CLI

The challenge

The solution: Flama CLI

The serve command

The `serve` command

The `model` command