Django Authentication system - GitHub Actions, Automated testing, Static Analysis And Deployment on Vercel

Introduction

We have so far built some awesome API endpoints for authenticating and authorizing users securely and in a performant way. However, apart from testing with Postman (or Thunder Client on VS Code) and — for those who went through building the frontend from the previous series — via the frontend application, we haven't made our app simple enough for anyone to just issue a simple command that runs through the app and reports whether or not the app works as expected. Also, Python is forgiving though better in that regard than JavaScript. We are bound to write our codes in styles alien to the accepted styles adopted by the community. Since Python is a dynamically typed language, we need a way to ENFORCE types on all variables used so that we won't assign a string to an integer variable. We also need to deploy our application so that everyone else can access the beauty we have built. All these are what we will address in this article.

Assumption and Recommendation

It is assumed that you are familiar with Django. I also recommend you go through previous articles in this series so you can keep up to speed with this one.

Source code

The source code for this series is hosted on GitHub via:

Sirneij / django-auth-backend

Django session-based authentication system with SvelteKit frontend

django-auth-backend

Django session-based authentication system with SvelteKit frontend and GitHub actions-based CI.

This app uses minimal dependencies (pure Django - no REST API framework) to build a secure, performant and reliable (with 100% automated test coverage, enforced static analysis using Python best uniform code standards) session-based authentication REST APIs which were then consumed by a SvelteKit-based frontend Application.

Users' profile images are uploaded directly to AWS S3 (in tests, we ditched S3 and used Django's InMemoryStorage for faster tests).

A custom password reset procedure was also incorporated, and Celery tasks did email sendings.

Run locally

• To run the application, clone it:

  git clone https://github.com/Sirneij/django-auth-backend.git

  You can, if you want, grab its frontend counterpart.

• Change the directory into the folder and create a virtual environment using either Python 3.9, 3.10 or 3.11 (tested against the three versions). Then activate it:

  ~django-auth-backend$ virtualenv -p python3.11 virtualenv
  ~django-auth-backend$ source virtualenv/bin/activate 

• …

View on GitHub

Implementation

Step 1: Static analysis and testing setup

You need to install pytest-cov, pytest-django, pytest-bdd, pyflakes, pylint, pylint-celery,pytest-xdist,django-stubs, and other packages. To relieve you of that burden, I have installed them and made them available in the project's requirements_dev.txt.

First off, let's create some bash scripts for running our tests and static analysis automatically. One will run the tests, another will enforce static analysis, and the last will help delete all test databases so that they won't cluster our machines:

At the very root of our project, create a scripts folder and in it create test.sh, drop_test_dbs.sh and static_validation.sh:

# scripts/tests.sh
#!/usr/bin/env bash
py.test -n auto --nomigrations --reuse-db -W error::RuntimeWarning --cov=src --cov-report=html tests/

This command uses pytest-xdist to allow distributed testing. We used auto here to denote the number of workers that will be spawned for test. auto equals to the number of available CPUs on your machine. Instead of auto, you can use a number such as 2, 4 or any integer. Just ensure that the number is less than or equal to the number of CPUs your machine possesses. --nomigrations uses pytest-django to disable running migrations for our tests. This makes the test suites faster. Also, --reuse-db uses pytest-django to create databases without deleting them after the tests ran. Hence the reason we need drop_test_dbs.sh. --cov=src --cov-report=html uses pytest-cov to help report our test stats. -W error::RuntimeWarning turns our runtime warnings into errors. Next is drop_test_dbs.sh:

# src/drop_test_dbs.sh

#!/bin/bash

PREFIX='test' || '_sqlx_test'
export PGPASSWORD=<your_db_password>
export PGUSER=<your_db_user>
export PGHOST=<your_db_host>
export PGPORT=<your_db_port>

TEST_DB_LIST=$(psql -l | awk '{ print $1 }' | grep '^[a-z]' | grep -v template | grep -v postgres)
for TEST_DB in $TEST_DB_LIST ; do
    if [ $(echo $TEST_DB | sed "s%^$PREFIX%%") != $TEST_DB ]
    then
        echo "Dropping $TEST_DB"
        dropdb --if-exists $TEST_DB
    fi
done

It uses your database credentials to delete all DBs that start with "test".

Next:

# scripts/static_validation.sh


#!/usr/bin/env bash

# checks whether or not the source files conform with black and isort formatting
black --skip-string-normalization --check tests
black --skip-string-normalization --check src
isort --atomic --profile black -c src
isort --atomic --profile black -c tests

cd src

# Exits with non-zero code if there is any model without a corresponding migration file
python manage.py makemigrations --check --dry-run

# Uses prospector to ensure that the source code conforms with Python best practices
prospector  --profile=../.prospector.yml --path=. --ignore-patterns=static

# Analysis and checks whether or not we have common security issues in our Python code. 
bandit -r . -ll

# Checks for correct annotations
mypy .

It is well commented. To ensure that your code passes them, you must run the following after each code modification:

black --skip-string-normalization  src tests

isort --atomic --profile black src tests

--skip-string-normalization prevents black from replacing '' with "" or vice versa.

The repo has other important files. Moving on, we can't afford to use S3 for testing. We prefer to use the filesystem or better still, an in-memory storage. Therefore, we'll be overriding the STORAGES settings and others during tests. A convenient way to do this is to create a test_settings.py file in src/django_auth:

# src/django_auth/test_settings.py
from django.test import override_settings

common_settings = override_settings(
    STORAGES={
        'default': {
            'BACKEND': 'django.core.files.storage.InMemoryStorage',
        },
        'staticfiles': {
            'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage',
        },
    },
    DEFAULT_FROM_EMAIL='[e-mail]',
    PASSWORD_HASHERS=[
        'django.contrib.auth.hashers.MD5PasswordHasher',
    ],
)

We used Django's override_settings to set faster storage, a faster password hasher and a default DEFAULT_FROM_EMAIL. We'll use this next.

Step 2: Testing

Let's start with our models.py. In the tests package, create a users package and in it, test_models.py:

from tempfile import NamedTemporaryFile

import pytest
from django.test import TestCase
from factory.django import DjangoModelFactory

from django_auth.test_settings import common_settings
from users.models import Articles, Series, User, UserProfile


class UserFactory(DjangoModelFactory):
    first_name = 'John'
    last_name = 'Doe'
    is_active = True

    class Meta:
        model = User
        django_get_or_create = ('email',)


class UserProfileFactory(DjangoModelFactory):
    class Meta:
        model = UserProfile
        django_get_or_create = ('user',)


class SeriesFactory(DjangoModelFactory):
    name = 'Some title'
    image = NamedTemporaryFile(suffix='.jpg').name

    class Meta:
        model = Series


class ArticlesFactory(DjangoModelFactory):
    title = 'Some article title'
    url = 'https://dev.to/sirneij/authentication-system-using-python-django-and-sveltekit-23e1'

    class Meta:
        model = Articles
        django_get_or_create = ('series',)


@common_settings
class UserModelTests(TestCase):
    def setUp(self):
        """Test Setup."""
        self.user = UserFactory.create(email='[e-mail]')

    def test_str_representation(self):
        """Test __str__ of user."""
        self.assertEqual(str(self.user), f'{self.user.id} {self.user.email}')


@common_settings
class UserProfileModelTests(TestCase):
    def setUp(self):
        """Test Setup."""
        self.user = UserFactory.create(email='[e-mail]')
        self.user_profile = UserProfileFactory.create(user=self.user)

    def test_str_representation(self):
        """Test __str__ of user."""
        self.assertEqual(
            str(self.user_profile),
            f'<UserProfile {self.user_profile.id} {self.user_profile.user.email}>',
        )

    def test_create_user_success(self):
        """Test create_user method."""
        user = User.objects.create_user(email='[e-mail]', password='123456Data')
        self.assertEqual(user.email, '[e-mail]')

    def test_create_user_failure(self):
        """Test create_user method fails."""

        with pytest.raises(ValueError, match='The Email must be set'):
            User.objects.create_user(email='', password='123456Data')

    def test_create_super_user_success(self):
        """Test create_user method."""
        user = User.objects.create_superuser(email='[e-mail]', password='123456Data')
        self.assertEqual(user.email, '[e-mail]')

    def test_create_super_user_failure(self):
        """Test create_user method fails."""

        with pytest.raises(TypeError, match='Superusers must have a password.'):
            User.objects.create_superuser(email='[e-mail]', password=None)


@common_settings
class SeriesAndArticlesModelTests(TestCase):
    def setUp(self):
        """Test Setup."""
        self.series = SeriesFactory.create()
        self.articles = ArticlesFactory.create(series=self.series)

    def test_str_representation(self):
        """Test __str__ of series and articles."""
        self.assertEqual(str(self.series), self.series.name)
        self.assertEqual(str(self.articles), self.articles.title)

We are using Factoryboy to initialize our models. With that, we can use ModelName.create() to create a model instance. If we want one or more fields of the model to be supplied at creation, we use django_get_or_create = (<tuple_of_the_fields>) in the Meta class. I added some other models, Series and Articles, to help hold my articles for this project. To provide a default to an image field, I used NamedTemporaryFile which does exactly that. On each of the test cases, I added the @common_settings decorator which we imported from test_settings.py so that the tests will use the faster settings variables. In each test case, we tested all the important things — __str__ of the models and other ones. We also tested our custom UserManager.

Next, let's test our celery task:

# tests/users/test_tasks.py

from unittest.mock import patch

from django.test import TestCase

from django_auth.test_settings import common_settings
from tests.users.test_models import UserFactory
from users.tasks import send_email_message


@common_settings
class SendMessageTests(TestCase):
    @patch('users.tasks.send_mail')
    def test_success(self, send_mail):
        user = UserFactory.create(email='[e-mail]')

        send_email_message(
            subject='Some subject',
            template_name='test.html',
            user_id=user.id,
            ctx={'a': 'b'},
        )
        send_mail.assert_called_with(
            subject='Some subject',
            message='',
            from_email='[e-mail]',
            recipient_list=[user.email],
            fail_silently=False,
            html_message='',
        )

We used patch from unittest.mock to mock send_mail from Django so that during testing, we don't really send any mail by mimicking sending it. This is a nice approach to make your tests predictable. We also test part of our validate_email util:

# tests/users/test_utils.py

from django.test import TestCase

from users.utils import validate_email


class ValidateEmailTests(TestCase):
    def test_email_empty(self):
        """Test when even is empty."""
        is_valid, message = validate_email('')
        self.assertFalse(is_valid)
        self.assertEqual(message, 'Enter a valid email address.')

We won't talk about other tests aside from the profile_update tests:

# tests/users/views/test_profile_update.py
from shutil import rmtree
from tempfile import NamedTemporaryFile, mkdtemp

from django.test import Client, TestCase
from django.test.client import BOUNDARY, MULTIPART_CONTENT, encode_multipart
from django.test.utils import override_settings
from django.urls import reverse
from django.utils import timezone

from django_auth.test_settings import common_settings
from tests.users.test_models import UserFactory


@common_settings
class UserUpdateViewTests(TestCase):
    def setUp(self) -> None:
        """Set up."""
        self.url = reverse('users:profile_update')
        self.client = Client()
        self.media_folder = mkdtemp()

    def tearDown(self):
        rmtree(self.media_folder)

    def test_update_user_not_authenticated(self):
        """Test when user is not authenticated."""
        response = self.client.patch(self.url)

        self.assertEqual(response.status_code, 401)
        self.assertEqual(
            response.json()['error'],
            'You are not logged in. Kindly ensure you are logged in and try again',
        )

    def test_update_user_success_first_name(self):
        """Test update user success with first_name."""
        # First login
        user = UserFactory.create(email='[e-mail]')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'first_name': 'Owolabi'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )

        self.assertEqual(response.status_code, 200)

        user.refresh_from_db()
        self.assertEqual(user.first_name, data['first_name'])

    def test_update_user_success_last_name(self):
        """Test update user success with last_name."""
        # First login
        user = UserFactory.create(email='[e-mail]')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'last_name': 'Idogun'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )

        self.assertEqual(response.status_code, 200)

        user.refresh_from_db()

        self.assertEqual(user.last_name, data['last_name'])

    def test_update_user_success_thumbnail(self):
        """Test update user success with thumbnail."""
        # First login
        user = UserFactory.create(email='[e-mail]')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # Update user
        with override_settings(MEDIA_ROOT=self.media_folder):
            with NamedTemporaryFile() as f:
                f.write(b'some file data')
                f.seek(0)
                data = {'thumbnail': f}
                encoded_data = encode_multipart(BOUNDARY, data)
                response = self.client.patch(
                    self.url, encoded_data, content_type=MULTIPART_CONTENT
                )
                self.assertEqual(response.status_code, 200)

        user.refresh_from_db()

        self.assertIsNotNone(user.thumbnail)

    def test_update_user_success_phone_number(self):
        """Test update user success with phone_number."""
        # First login
        user = UserFactory.create(email='[e-mail]')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'phone_number': '[phone]'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

        user.userprofile.refresh_from_db()

        self.assertEqual(user.userprofile.phone_number, data['phone_number'])

    def test_update_user_success_birth_date(self):
        """Test update user success with birth_date."""
        # First login
        user = UserFactory.create(email='[e-mail]')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'birth_date': timezone.localdate()}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

        user.userprofile.refresh_from_db()

        self.assertEqual(user.userprofile.birth_date, data['birth_date'])

    def test_update_user_success_github_link(self):
        """Test update user success with github_link."""
        # First login
        user = UserFactory.create(email='[e-mail]')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'github_link': 'https://github.com/Sirneij'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

        user.userprofile.refresh_from_db()

        self.assertEqual(user.userprofile.github_link, data['github_link'])

We used some nifty tricks here. At setUp, we created a temporary media_folder to hold the uploaded test image. The folder gets deleted immediately after the tests finish running using the tearDown method. Since this endpoint expects a FormData from the request, we used Django's encode_multipart to encode our data. It's important to use the corresponding BOUNDARY from the same Django model so that the FormData is properly encoded. Else, our endpoint will have issues parsing the form properly and the input data will not be what is stored in the DB. For uploading an image, we did this:

...
# Update user
with override_settings(MEDIA_ROOT=self.media_folder):
    with NamedTemporaryFile() as f:
        f.write(b'some file data')
        f.seek(0)
        data = {'thumbnail': f}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

Again, NamedTemporaryFile was used to generate a temporary file and we overrode our app's MEDIA_ROOT to be the temporary folder we created in the setUp method. For each request, we ensured that we were logged in.

The testing concepts discussed here are enough to take a look at the final code test suite and not be lost.

Step 3: Setting GitHub Actions for testing and static analysis

I assume you have a GitHub account and have been pushing your codes so far to the platform. Let's add an action to our project that runs every time we create a pull request or push to the main branch. You can also set it so that until a pull request passes, you cannot merge such a request to the main branch. Let's create a flow. To do that, create a .github/workflows/django.yml file:

# .github/workflows/django.yml
name: Django-auth-backend CI

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 4
      matrix:
        python-version: [3.9.13, 3.10.11, 3.11]

    services:
      postgres:
        image: postgres:latest
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: github_actions
        ports:
          - 5432:5432
        # needed because the postgres container does not provide a healthcheck
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

      redis:
        image: redis:7
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v3
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements_dev.txt
      - name: Run static analysis
        run: chmod +x ./scripts/static_validation.sh && ./scripts/static_validation.sh
      - name: Run tests
        run: chmod +x ./scripts/test.sh && ./scripts/test.sh

We gave it a name and we want it to run when there is a push to the main branch or when there is a pull request. Then, we specified our build jobs which use the latest version of Ubuntu to build our application against three main versions of Python, [3.9.13, 3.10.11, 3.11]. To run our jobs, we need a PostgreSQL database and a redis instance. Those were properly configured. Notice that we specified our database credentials.

Next, we specified the build steps which use the important actions/checkout@v3. The steps involve setting up Python, installing our project's dependencies, and running static analysis and tests.

Step 4: Deployment on Vercel

We used to use Heroku for free and hobby deployments until they stopped it in October 2022. Vercel has come to the rescue and we'll deploy our Django application there. You have two options:

• Install Vercel CLI and deploy using it
• Connect your repository to Vercel and allow it to automatically deploy after each push to your repo's main branch or any branch of your choice.

You're at liberty to choose any method but ensure you create a file, vercel.json, in, for our app structure, src:

// src/vercel.json
{
  "version": 2,
  "builds": [
    {
      "src": "django_auth/wsgi.py",
      "use": "@vercel/python",
      "config": { "maxLambdaSize": "15mb" }
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "django_auth/wsgi.py"
    }
  ]
}

We are using wsgi but you can use asgi as well. Ensure you have requirements.txt in the folder too. You can check the repo for details.

Since our application needs a database, you can use Railway to provision free PostgreSQL and Redis instances. Ensure you get their credentials and set them accordingly as your application's environment variables on Vercel.

If you use the CLI, you can run migrations and create a super user using the following steps:

• SSH into your Vercel instance using the command vercel ssh.
• Navigate to your app's directory and run the command python manage.py migrate to apply any pending migrations.
• Create a superuser by running the command python manage.py createsuperuser and following the prompts.

For the last step, you can set DJANGO_SUPERUSER_PASSWORD and DJANGO_SUPERUSER_EMAIL environment variables and then issue python manage.py createsuperuser --no-input instead.

If you decide to use Vercel UI instead, you can create a build_files.sh script with this content:

# build_files.sh
pip install -r requirements.txt

python3.9 manage.py migrate
python3.9 manage.py createsuperuser --no-input

and then modify vercel.json:

{
  "version": 2,
  "builds": [
    {
      "src": "django_auth/wsgi.py",
      "use": "@vercel/python",
      "config": { "maxLambdaSize": "15mb" }
    },
    {
      "src": "build_files.sh",
      "use": "@vercel/static-build",
      "config": {
        "distDir": "staticfiles_build"
      }
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "django_auth/wsgi.py"
    }
  ]
}

This is some hack that will lead to deployment failure because we don't use file storage for our static files but those commands will run. You can then remove that segment and redeploy it.

Using the CLI allows you to incorporate the deployment process as one of the build processes of our GitHub workflow.

That's it for this series. Don't hesitate to drop your reactions, comments and contributions. You may also like me to write about anything and if I can, I will definitely oblige. I am also available for gigs.

Outro

Enjoyed this article? Consider contacting me for a job, something worthwhile or buying a coffee ☕. You can also connect with/follow me on LinkedIn and Twitter. It isn't bad if you help share this article for wider coverage. I will appreciate it...

Comments

[Machele Alhassan]

It seems there's no vercel ssh command, does Vercel have any way of allowing users to access their deployment's terminal?