DEV Community: Adejumo Ridwan Suleiman

Build a Serverless Database for an SMS Spam Classifier with FaunaDB and FastAPI.

Adejumo Ridwan Suleiman — Fri, 23 Jun 2023 08:09:55 +0000

Introduction

A serverless database is a database that is hosted and managed by a third-party provider. This means you do not need to worry about the hardware, software, or maintenance of the database. You can simply connect to the database and start using it.

Serverless databases are a good option for businesses that do not have the resources to manage their database. They are also a good option for businesses that need to scale their database quickly. Such examples of serverless databases are FaunaDB, CockroachDB, Amazon DynamoDB, and so on.

In this tutorial, you will learn how to build a serverless database to store SMS messages sent to your WhatsApp number, and also their classification, if they are spam or not. You will use Twilio MessagingX SMS API to receive SMS messages on your Fast API backend which will send these data into FaunaDB.

Prerequisites

Python 3.9+ is installed on your machine.
FaunaDB free account, if you dont have one, you can set it up here.
Twilio free account, you can set up one here.
Ngrok account, follow this guide to set up one.
An IDE or text editor, such as VS code.
Hugging Face account, you can set one up here

Setting up the Development Environment

Before you start, you need to set up the development environment by creating the directory and files you will need.

mkdir sms_classifier
cd sms_classifier
touch requirements.txt models.py utils.py main.py .env

requirements.txt contains the required libraries to build the database.
model.py contains the code connecting the Fast API app to FaunaDB
utils.py contains the code to connect to the sms classification model on Hugging Face.
main.py contains the code to build the Fast API server
sms_classifier is the directory for all our files.

Next, you create and activate a virtual environment and update the Python package manager pip to the most recent version, to know more about the benefits of using virtual environments. Ensure you read this post.

python -m venv venv; ./venv/Scripts/Activate; pip - upgrade pip

If you are on a Linux machine use,

pyton -m venv venv; venv\\Scripts\\activate.bat; pip - upgrade pip

Next, fill the requirements.txt file with the following dependencies,

fastapi
uvicorn
faunadb
pyngrok
requests
dotenv

fastapi is a Python framework for building APIs quickly and easily
uvicorn is a lightning-fast server implementation for Python
pyngrok enables you to tunnel a local server to a public URL
requests allows you to send HTTP requests using Python
dotenv loads the environment variables from the .env file.
faunadb is a Python driver allowing you to connect to the fauna server.

Install these dependencies by running the following command on your terminal

pip install -r requirements.txt

Connecting to Fauna Database

On the Fauna home page, click on Create Database ,

Next, give the name of your database, which is SMS, and select a Region Group then click Create.

Go to Security and click on New Key to create your fauna key

Select the role as Server and Save

Copy the fauna key displayed to you, then go to the .env file and create a variable called FAUNA_SECRET and paste it there.

FAUNA_SECRET=<secret>

Go to the models.py and paste the following code

from faunadb import query as q
from faunadb.client import FaunaClient
from dotenv import load_dotenv
import os

load_dotenv()

client = FaunaClient(secret=os.getenv("FAUNA_SECRET"),
endpoint="https://db.us.fauna.com/")
collection_name = "sms"

# Check if collection exists
try:
  client.query(q.get(q.collection(collection_name)))
  print(f"Collection '{collection_name}' already exists.")
except:
  # Collection doesn't exist, create it
  client.query(q.create_collection({"name": collection_name}))
  print(f"Collection '{collection_name}' created successfully.")

load_dotenv() loads the environment variables into models.py

The FaunaClient() function is used to create a client object that can be used to interact with the FaunaDB database. It creates a fauna client using the secret key and fauna endpoint.

collection_name stores the name of the collection, q.get checks if a collection with the name sms exists, and if false, client.query creates a new collection sms, collections are like tables in SQL.

Run the code above and go back to the database you created earlier on. You will see the already created collection sms.

The dashboard shows no document in the collection, documents are NoSQL versions of rows in tables. Each SMS and its data will make up a document in the SMS collection.

Building the Spam Classifier Model Function

An SMS spam classifier classifies an SMS as spam or ham (not spam). One can build a spam classifier from scratch, but this needs a lot of data and computing resources to get an accurate model. You are going to set up a hugging face account and use an already pre-built model from hugging face. Thanks to hugging face, you dont need to build an SMS spam classifier from scratch.

To get a Hugging Face API key, go to Settings and click on Access Tokens

Create a new token with READ role by clicking on New Token

Copy this token and paste it to the .env file, and store it as HUGGING_API_KEY

HUGGING_API_KEY="<api_key>"
FAUNA_SECRET="<fauna_secret>"

Copy and paste the following code below into utils.py

from dotenv import load_dotenv
import os
import requests
load_dotenv()
api_key=os.getenv("API_KEY")
# SMS Classification
def sms_spam_class(sms):
  API_URL = "https://api-inference.huggingface.co/models/mrm8488/bert-tiny-finetuned-sms-spam-detection"
  headers = {"Authorization": f"Bearer {api_key}"}
  def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()
  output = query({
    "inputs": "f{sms}",
  })
  # Find the scores for 'LABEL_0' and 'LABEL_1'
  label_0_score = None
  label_1_score = None
  for item in output[0]:
    if item['label'] == 'LABEL_0':
      label_0_score = item['score']
    elif item['label'] == 'LABEL_1':
      label_1_score = item['score']
  # Check if 'LABEL_1' score is greater than 'LABEL_0' score
  if label_1_score is not None and label_0_score is not None:
    if label_1_score > label_0_score:
      spam_class = "spam"
      else:
      spam_class = "ham"
    else:
      spam_class = "Unable to determine spam status."
    return spam_class

The sms_spam_class() function checks if a given sms is spam or not, API_URL contains the reference link to the spam classification model API. The headers contain the Hugging Face api_key which allows you to make requests from the API using the requests library.

The function query() takes in the payload which is sms. This function makes a POST request using the requests.post function and returns a JSON response response.json() stored in the variable output.

The output contains the score of the word for various labels, the for loop checks each label score and stores the values into their respective variables, label_0_score and label_1_score for Label_0 and Label_1 respectively. Label_0 shows an SMS as ham (not spam) while Label_1 shows an SMS as spam.

If the score in Label_0 is greater than Label_1 then the SMS is spam else the SMS is ham , and it saves the resulting classification in spam_class.

Setting up the webhook with ngrok

Inside the main.py, set up a basic Fast API application

from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def index():
  return {"message": "I love Serverless"}

The code below sets up a basic fastapi backend, creating a new instance of the Fast API class and assigning it to the app variable. The @app.get decorator creates a new endpoint that you can access with an HTTP GET request. The endpoint is at the root URL / and returns a JSON response with a single key-value pair: message: I love Serverless.

To run the app, run the following command in your terminal:

uvicorn main:appreload

On your browser, open the host http://127.0.0.1:8000 , you will see a JSON response of {message: I love Serverless}, you can also access an interactive API doc provided by swagger on the host http://127.0.0.1:8000/doc which allows you to interact with your API and see if you have any errors.

To receive Twilio messages on the backend, you are going to use ngrok to host the local host on a public server. Read this post to learn how to set up ngrok on your machine.

On ngrok administrator, run the command ngrok http 8000, this makes your host public and you can receive messages on your backend.

Next, go to Active Numbers to see your number, and click on the number. On the dashboard displayed to you, click on Configure and go to Message Configuration , paste the ngrok address into the URL under When a message comes in and click Save configuration , this allows Twilio to request your fast API app anytime you receive an incoming message, congratulations you just set up a Webhook.

Building the Fast API Application

Update the main.py with the code below

from fastapi import FastAPI, Form, Request
from utils import sms_spam_class
from dotenv import load_dotenv
import os
from faunadb import query as q
from model import client
load_dotenv()
app = FastAPI()
@app.post("/sms")
async def reply(request: Request):
  form_data = await request.form()

  MessageSid = form_data["MessageSid"]
  AccountSid = form_data["AccountSid"]
  From = form_data["From"]
  Body = form_data["Body"]
  To = form_data["To"]

  spam_class = sms_spam_class(Body)

  response = client.query(q.create(q.collection("sms"), {"data": {
    "MessageSid": MessageSid,
    "AccountSid": AccountSid,
    "From": From,
    "Body": Body,
    "To": To,
    "spam_classification": spam_class
  }}))
  return ""

The code above sends a POST request at the endpoint /sms, the reply function takes the Twilio request as the query parameter. This will give us information on what and what was in the message you receive on your Twilio number.

await request.form() stores the message data into form_data, there are a lot of parameters when Twilio requests your webhook URL.

In this tutorial, you will just make use of some of the parameters. You can explore other parameters in their documentation.

MessageSid is a unique identifier assigned to every message
AccountSid is a unique identifier assigned to every Twilio account
From is the phone number sending the message
To is the phone number receiving the message
Body is the body of the SMS

sms_spam_class(Body) is the function you built earlier on classifying if a model is spam or not, storing the value in spam_class.

The response variable stores all the parameters gotten from the SMS data in the database collection SMS.

Getting the Server up and running

Now your fast API app is ready to receive and store messages in the fauna database,

Here is an example of an SMS sent to a Twilio number:

On Fauna, you will see the text messages reflected as follows:

Conclusion

In this tutorial, you learned how to create a Fauna serverless database that stores SMS messages and also their spam classification built on Fast API. You also learned about how to connect to pre-built models on Hugging Face.

You can extend this app by adding more parameters from the Twilio request made to your webhook such as media URL, date a message was sent and received, and so on.

You can also try another serverless database like Cassandra, Redis, MongoDB and so on.

9 Things You Do That Shows You Are Not A Professional Python Developer

Adejumo Ridwan Suleiman — Thu, 01 Jun 2023 13:24:01 +0000

Photo by Timothy Dykes on Unsplash

Show me your code and I will tell you who you are.

This article will fix the bad habits you have stuck to over the years or brought from other programming languages.

Manual String Formatting

Most of the time you see beginner Pythonistas using the + sign when combining two strings.

>>> name = "Ridwan"
>>> age = "22"
>>> print("My Name is " + name + " and I am " + age + " years old")
My Name is Ridwan and I am 22 years old

Instead of using the + sign, use the f string which makes your code readable, concise, and less prone to errors.

>>> print(f"My Name is {name} and I am {age} years old")
My Name is Ridwan and I am 22 years old

Using Default Mutable Arguments

In Python, anytime you pass a mutable value as an argument in a function, the default argument is mutated anytime the function is called. These mutable arguments are usually lists or dictionaries.

Look at the example below;

>>> def append(n, l=[]):
... l.append(n)
... return l
...

After defining the append function with a mutable argument l = [], anytime you call the function with a value of n, it changes the default value of l.

>>> l1 = append(0)
>>> l1
[0]

When next you call the function with a different value of n, you are going to see the previous value you used appended to the empty list argument.

>>> l2 = append(1)
>>> l2
[0, 1]

You can solve this problem by rewriting the code as:

>>> def append(n, l = None):
... if l is None:
... l = []
... l.append(n)
... return l
...
>>> l1 = append = [0]
>>> l2 = append = [1]
>>> l1,l2
([0], [1])

Now the argument l is set to None, anytime the function is called even if l is mutated, it is reassigned as None and then given the value of an empty list.

Not Using Comprehensions

Python comprehensions provides you a short and concise way of way of constructing sequences, last time I checked, Python supports 4 types of comprehension;

List Comprehensions
Dictionary Comprehensions
Set Comprehensions
Generator Comprehensions

You can read more about them here.

The code below divides the values in a dictionary by 2,

>>> numbers = {}
>>> for i in range(10):
... numbers[i] = i/2
...
>>> numbers
{0: 0.0, 1: 0.5, 2: 1.0, 3: 1.5, 4: 2.0, 5: 2.5, 6: 3.0, 7: 
3.5, 8: 4.0, 9: 4.5}

The above code can be written in a single line as,

>>> {i: i/2 for i in range(10)}
{0: 0.0, 1: 0.5, 2: 1.0, 3: 1.5, 4: 2.0, 5: 2.5, 6: 3.0, 7: 
3.5, 8: 4.0, 9: 4.5}

So stop making life difficult for yourself and start using comprehension.

Checking for Equality instead of Identity

Given,

a = [1, 2, 3]
b = [1, 2, 3]

If I ask you to check if the two variables are identical, the first thing to come to your mind is,

>>> a == b
True

The thing is that you need to know the difference between Identity and Equality.

Two variables can be equal but not identical.

If you check the memory addresses of a and b,

>>> id(a), id(b)
(1838093945856, 1838093487488)

You can see that they both have different addresses despite the fact that they have the same object.

This is the reason why, when you run,

>>> a == b
True

You get True, but when you run

>>> a is b
False

You get False, a and b are equal but not identical.

In the case where you have,

>>> c = [1,2,3]
>>> d = c
>>> id(c), id(d)
(1838089019712, 1838089019712)

You can see, c and d are both equal and identical, the object in c is also assigned to d.

>>> c == d
True
>>> c is d
True

This implies c and d have both the same value and memory addresses.

Hence you can say c is identical and equal to d.

All this epistle is to make you know the difference between is and ==, the former is used in checking identity while the latter is used in checking equality.

All Identical variables are equal but not all Equal variables are identical

Not Using Tuple Unpacking

Anytime you create a tuple in Python, it is known as packing a tuple,

>>> a_tuple = 1,2,3
>>> a_tuple
(1, 2, 3)

These values can be extracted back into various variables by unpacking

>>> x = a_tuple[0]
>>> y = a_tuple[1]
>>> z = a_tuple[2]
>>> print(x, y, z)
1, 2, 3

Instead of unpacking the elements in a tuple using multiple lines of code, you can do it in a single line of code.

>>> x,y,z = a_tuple
>>> print(x, y, z)
1, 2, 3

Creating Your Own Index Counter Variable

This one is common from those coming from other programming languages, you are asked to create a index counter variable and you type something like;

>>> a_list = [1,2,3,4,5,6,7,8,9,10]
>>> index = 0
>>> for elem in a_list:
... print(index, elem)
... index += 1
...
0 1
1 2
2 3
3 4
4 5
5 6
6 7
7 8
8 9
9 10

Instead, use the enumerate function to make your code look Pythonic;

>>> for index, elem in enumerate(a_list):
... print(index, elem)
...
0 1
1 2
2 3
3 4
4 5
5 6
6 7
7 8
8 9
9 10

Using Print Statement Instead of the Logging Module

This might not matter in small projects, but will surely help you in larger projects.

Instead of littering your code with print statements, use the logging instead.

>>> print('This is a warning message')
This is a warning message
>>> print('This is an error message')
This is an error message
>>> print('This is a critical message')
This is a critical message

Logging helps display useful messages to your users to add more context and understanding to what is happening in the code base.

>>> import logging
>>> logging.warning('This is a warning message')
WARNING:root:This is a warning message
>>> logging.error('This is an error message')
ERROR:root:This is an error message
>>> logging.critical('This is a critical message')
CRITICAL:root:This is a critical message

Importing functions and classes in a named module with import *

This bad habit most at times is common among newbies.

Importing using import * corrupts your namespace by importing all the functions and classes from that named module into your code which is likely to conflict with the functions you define or functions of other libraries imported.

Not Following pep8

Most of us are guilty of this,

Before I get canceled 😂, I know some of my codes in this article might have broken the rules of PEP-8, but the truth is bitter and needs to be told, following PEP-8 style and guidelines makes it easier for others to read and understand your code.

Not Recommended

>>> def function():
... x = [1,2,3]
... y= [2,3,5]
... z = [1, 2,3]
...
>>> def value(x = 7):
... ...
...

Recommended

>>> def function():
... x = [1, 2, 3]
... y = [2, 3, 5]
... z = [1, 2, 3]
...
>>> def number(x=7):
... ...
...

To read more about the PEP-8 style and guidelines, check out this article.

Thanks for reading. Hope the article was worth your time.