DEV Community: Joshua Masiko

How to deploy a Django app to Google Cloud Run using Terraform

Joshua Masiko — Mon, 01 Jan 2024 18:45:42 +0000

Introduction

Deploying a Django application to a production environment may require you to perform the following tasks:

Provision hardware or virtual machines to run the application and database server
Install the required software on the provisioned servers
Purchase a domain from a registar for and setup the relevant DNS records to associate the domain with your app
Setup a webserver to terminate HTTP(S) traffic to your application
Obtain and congiure SSL certificate to secure traffic to the application

Goole Cloud Run

Cloud Run is a managed platform that enables you to run container based workloads on top of Google infrastructure.

Cloud Run automates many of the above steps and allows you to focus on developing and deploying updates to your application.

Some of the benefits include:

Run code written in any programming language on cloud run if you can package into a container
Source based deployments using build packs for supported languages making an explicit container packaging step optional.
Support for services which respond to web requests, and one-off jobs which run and exit after doing some work
A generous free tier and pay-per-use based pricing.
Services automatically receive unique https endpoints on a run.app sub domain with support for custom domains
Auto-scaling which automatically adds and removes running container instances in response to web traffic or resource utilization

Infrastructure as Code

You have 3 options when installing an application on Google Cloud:

Using the web based console
Using the gcloud tool
Using an automated provisioning tool such as Terraform

This tutorial uses the Terraform tool to create a repeatable recipe for deploying a Django application to Google Cloud Run and provisioning the backing services.

Google Cloud Components

You will use the following Google Cloud components:

Cloud Run: managed compute platform to run container based services which respond to web requests or jobs which run to completion
Artifact Registry: artifact storage to manage container images.
Cloud SQL: managed relational database service for MySQL, PostgreSQL, and SQL Server.
Cloud Storage: blog storage for static assets and media files
Secret Manager: secure storage for sensitive data e.g passwords.

Step 0 - Installing the supporting tools

Install the Google Cloud CLI by following the intructions here.

Initialize the gcloud CLI by run the following command:

gcloud init

Install Terraform by following the intructions here.

Step 1 - Creating a Google Cloud project

Create a new Google Cloud project:

gcloud projects create --name django-on-cloud-run --set-as-default

Type y at the prompt and press enter:

No project id provided.

Use [django-on-cloud-run-409907] as project id (Y/n)?  y

Enable billing on the project to allow usage of the required Google Cloud APIs within the project.

Set an environment variable for the generated project ID by running the following command.

PROJECT_ID=$(gcloud config get-value project)

Obtain credentials to enable Terraform to make authenticated requests to Google Cloud

gcloud auth application-default login --project $PROJECT_ID

Step 2 - Creating a starter Django project

Create a directory to host the project source code

mkdir django-on-cloudrun && cd django-on-cloudrun

Create 2 subfolders for the Django application code and the Terraform infrastructure code

mkdir {djangocloudrun,terraform}

The djangocloudrun subfolder will contain the Django/Python code and the terraform folder will contain the infrastructure provisioning code.

Create a python virtual enviroment to isolate the project dependencies

python -m venv venv

Activate the Python virtual environment

source venv/bin/activate

Go to the djangocloudrun subfolder

cd djangocloudrun

Create a requirements.txt file with the following content

# django-on-cloudrun/djangocloudrun/requirements.txt
Django==5.0
django-environ==0.10.0
django-storages[google]==1.13.2
google-cloud-run==0.10.1
gunicorn==20.1.0
psycopg2-binary==2.9.9

django-environ: used to read application settings from the environment.
django-storages[google]: used to integrate with Google Cloud Storage for static assets management
google-cloud-run: used to read service metadata on startup
gunicorn: WSGI application server for the Django application
psycopg2-binary: used to communicate with the managed Cloud SQL PostgreSQL instance

Install the project dependencies

 pip install -r requirements.txt

After the dependencies are installed create a starter Django project using the django-admin tool

django-admin startproject djangocloudrun .

View your project structure by running this command

tree

You project structure should look like this

.
├── djangocloudrun
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
└── requirements.txt

Rename the default generated settings.py file to basesettings.py

mv djangocloudrun/settings.py djangocloudrun/basesettings.py

Create a new settings.py file with the following content

# django-on-cloudrun/djangocloudrun/djangocloudrun/settings.py
import io
import os
from urllib.parse import urlparse

import environ
import google.auth
import requests
from google.cloud.run_v2.services.services.client import ServicesClient


from .basesettings import *


env = environ.Env()
if env("APPLICATION_SETTINGS", default=None):
    env.read_env(io.StringIO(os.environ.get("APPLICATION_SETTINGS", None)))
else:
    env_file = BASE_DIR / ".env"
    env.read_env(env_file)


SECRET_KEY = env("SECRET_KEY")
DEBUG = env("DEBUG", default=False)
DATABASES = {"default": env.db()}
SERVICE_NAME = env("SERVICE_NAME", default=None)

try:
    _, PROJECT_ID = google.auth.default()
except google.auth.exceptions.DefaultCredentialsError:
    PROJECT_ID = env("PROJECT_ID", default=None)

try:
    response = requests.get(
        "http://metadata.google.internal/computeMetadata/v1/instance/region",
        headers={"Metadata-Flavor": "Google"},
    )
    REGION = response.text.split("/")[-1]
except requests.exceptions.ConnectionError:
    REGION = None

if all((PROJECT_ID, REGION, SERVICE_NAME)):
    service_path = f"projects/{PROJECT_ID}/locations/{REGION}/services/{SERVICE_NAME}"
    client = ServicesClient()
    service_uri = client.get_service(name=service_path).uri
    ALLOWED_HOSTS = [urlparse(service_uri).netloc]
    CSRF_TRUSTED_ORIGINS = [service_uri]
else:
    ALLOWED_HOSTS = ["*"]

if GS_BUCKET_NAME := env("STATICFILES_BUCKET_NAME", default=None):
    STATICFILES_DIRS = []
    GS_DEFAULT_ACL = "publicRead"
    STORAGES = {
        "default": {
            "BACKEND": "storages.backends.gcloud.GoogleCloudStorage",
        },
        "staticfiles": {
            "BACKEND": "storages.backends.gcloud.GoogleCloudStorage",
        },
    }

Create a .env file in the Django project root with the following content

# django-on-cloudrun/djangocloudrun/.env
SECRET_KEY="django-insecure-secret-key"
DATABASE_URL=sqlite:////tmp/db.sqlite3
DEBUG=on

Create a Procfile file in the Django project root with the following content

# django-on-cloudrun/djangocloudrun/Procfile
web: gunicorn --bind 0.0.0.0:$PORT --workers 1 --threads 8 --timeout 0 djangocloudrun.wsgi:application
migrate_collectstatic: python manage.py migrate && python manage.py collectstatic --noinput --clear
create_superuser: python manage.py createsuperuser --username admin --email noop@example.com --noinput

A Procfile lists the web application entry points that are executed by Cloud Run service and jobs.

The web default entry point runs the application
The migrate_collecstatic entry point is executed by a job to run database migrations and copy static files to Cloud Storage
The create_superuser entry point is executed by a job to create a Django superuser

Step 3 - Running the application locally

Initialize the sqlite database by running migrations

python manage.py migrate

You should see output similar to this if migrations are run successfully

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK

Create a superuser account for logging into the Django admin interface

python manage.py createsuperuser --username admin --email admin@example.com

When prompted enter a password of your choice and confirm

Password:
Password (again):
Superuser created successfully.

Run the developement server

python manage.py runserver

The server starts up and prints output similar to this:

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
January 01, 2024 - 08:32:11
Django version 5.0, using settings 'django_cloudrun.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Navigate to http://127.0.0.1:8000 in your web browser and you will see the Django welcome page

Navigate to the Django admin site http://127.0.0.1:8000/admin and login using the admin username and the superuser password you set.

The Django admin dashboard is displayed

Step 4 - Deploying the app to Cloud Run using Terraform

Create a Cloud Storage bucket to store Terraform state using the gsutil tool which is bundled with the gcloud installation

gsutil mb gs://${PROJECT_ID}-tfstate

Enable Object Versioning on the storage bucket to keep a history of Terraform state:

gsutil versioning set on gs://${PROJECT_ID}-tfstate

Go to the terraform subfolder

cd ../terraform

Create a variables.tf file with the following content

# django-on-cloudrun/terraform/variables.tf
variable "project" {
  type        = string
  description = "Google Cloud Project ID"
}

variable "region" {
  type        = string
  default     = "us-central1"
  description = "Google Cloud Region"
}

variable "service_name" {
  type        = string
  description = "Name of Cloud Run service"
}

Create a terraform.tfvars file with the following content.
Replace the PROJECT_ID placeholder with the value of the PROJECT_ID environment variable you set earlier.

# django-on-cloudrun/terraform/terraform.tfvars
project      = "PROJECT_ID"
service_name = "django-on-cloudrun"

Create a templates subfolder

mkdir templates

Crete a application_settings.tftpl within the templates subfolder with the following content

# django-on-cloudrun/terraform/templates/application_settings.tftpl
DATABASE_URL="postgres://${user.name}:${user.password}@//cloudsql/${instance.project}:${instance.region}:${instance.name}/${database.name}"
STATICFILES_BUCKET_NAME="${staticfiles_bucket}"
SECRET_KEY="${secret_key}"
DEBUG=on

Create a main.tf file with the following content.
Replace the PROJECT_ID placeholder with the value of the PROJECT_ID environment variable you set earlier.

# django-on-cloudrun/terraform/main.tf
terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.9.0"
    }

    random = {
      version = "~> 3.6.0"
    }
  }
}

# Store Terraform state in a Google Cloud Storage bucket
terraform {
  backend "gcs" {
    bucket = "PROJECT_ID-tfstate"
  }
}

provider "google" {
  project = var.project
}

# Enable the required Google Cloud APIs
resource "google_project_service" "all" {
  for_each = toset([
    "artifactregistry.googleapis.com",
    "cloudbuild.googleapis.com",
    "compute.googleapis.com",
    "run.googleapis.com",
    "secretmanager.googleapis.com",
    "sql-component.googleapis.com",
    "sqladmin.googleapis.com",
  ])
  project            = var.project
  service            = each.key
  disable_on_destroy = false
}

# Create a service account for the Cloud Run service
resource "google_service_account" "django_cloudrun" {
  account_id = "django-run-sa"
  project    = var.project
}

# Create a Artifact Repository to store the application image
resource "google_artifact_registry_repository" "main" {
  format        = "DOCKER"
  location      = var.region
  project       = var.project
  repository_id = "django-app"

  depends_on = [
    google_project_service.all
  ]
}

# Provision a database server instance for the application
resource "google_sql_database_instance" "main" {
  name             = "django"
  database_version = "POSTGRES_14"
  region           = var.region
  settings {
    tier = "db-f1-micro"
  }
  deletion_protection = true

  depends_on = [
    google_project_service.all
  ]
}

# Create a database within the instance
resource "google_sql_database" "main" {
  name     = "django"
  instance = google_sql_database_instance.main.name
}

# Create a random password for the app database user
resource "random_password" "db_password" {
  length  = 32
  special = false
}

# Create the Django application database user
resource "google_sql_user" "django" {
  name     = "django"
  instance = google_sql_database_instance.main.name
  password = random_password.db_password.result
}

# Define local variables
locals {
  service_account = "serviceAccount:${google_service_account.django_cloudrun.email}"
  repository_id   = google_artifact_registry_repository.main.repository_id
  ar_repository   = "${var.region}-docker.pkg.dev/${var.project}/${local.repository_id}"
  image           = "${local.ar_repository}/${var.service_name}:bootstrap"
}

# Assign the Cloud Run service the required roles to connect to the DB and fetch service metadata
resource "google_project_iam_member" "service_roles" {
  for_each = toset([
    "cloudsql.client",
    "run.viewer",
  ])
  project = var.project
  role    = "roles/${each.key}"
  member  = local.service_account
}

# Create a Cloud Storage bucket to store static files
resource "google_storage_bucket" "staticfiles" {
  name     = "${var.project}-staticfiles"
  location = "US"
}

# Grant the Cloud Run service account admin access to the staticfiles bucket
resource "google_storage_bucket_iam_binding" "staticfiles_bucket" {
  bucket = google_storage_bucket.staticfiles.name
  role   = "roles/storage.admin"
  members = [
    local.service_account
  ]
}

# Create a random string to use as the Django secret key
resource "random_password" "django_secret_key" {
  special = false
  length  = 50
}

resource "google_secret_manager_secret" "application_settings" {
  secret_id = "application_settings"

  replication {
    auto {}
  }
  depends_on = [google_project_service.all]

}

# Replace the Terraform template variables and save the rendered content as a secret
resource "google_secret_manager_secret_version" "application_settings" {
  secret = google_secret_manager_secret.application_settings.id

  secret_data = templatefile("${path.module}/templates/application_settings.tftpl", {
    staticfiles_bucket = google_storage_bucket.staticfiles.name
    # media_bucket       = google_storage_bucket.media.name
    secret_key = random_password.django_secret_key.result
    user       = google_sql_user.django
    instance   = google_sql_database_instance.main
    database   = google_sql_database.main
  })
}

# Grant the Cloud Run service account access to the application settings secret
resource "google_secret_manager_secret_iam_binding" "application_settings" {
  secret_id = google_secret_manager_secret.application_settings.id
  role      = "roles/secretmanager.secretAccessor"
  members   = [local.service_account]
}

# Generate a random password for the superuser
resource "random_password" "superuser_password" {
  length  = 32
  special = false
}

# Save the superuser password as a secret
resource "google_secret_manager_secret" "superuser_password" {
  secret_id = "superuser_password"
  replication {
    auto {}
  }
  depends_on = [google_project_service.all]
}

resource "google_secret_manager_secret_version" "superuser_password" {
  secret      = google_secret_manager_secret.superuser_password.id
  secret_data = random_password.superuser_password.result
}

# Grant the Cloud Run service account access to the superuser password secret
resource "google_secret_manager_secret_iam_binding" "superuser_password" {
  secret_id = google_secret_manager_secret.superuser_password.id
  role      = "roles/secretmanager.secretAccessor"
  members   = [local.service_account]
}

# Build the application image that the Cloud Run service and jobs will use
resource "terraform_data" "bootstrap" {
  provisioner "local-exec" {
    working_dir = "${path.module}/../djangocloudrun"
    command     = "gcloud builds submit --pack image=${local.image} ."
  }

  depends_on = [
    google_artifact_registry_repository.main,
    google_project_service.all
  ]
}

# Create the migrate_collectstatic Cloud Run job
resource "google_cloud_run_v2_job" "migrate_collectstatic" {
  name     = "migrate-collectstatic"
  location = var.region

  template {
    template {
      service_account = google_service_account.django_cloudrun.email

      volumes {
        name = "cloudsql"
        cloud_sql_instance {
          instances = [google_sql_database_instance.main.connection_name]
        }
      }

      containers {
        image   = local.image
        command = ["migrate_collectstatic"]

        env {
          name = "APPLICATION_SETTINGS"
          value_source {
            secret_key_ref {
              version = google_secret_manager_secret_version.application_settings.version
              secret  = google_secret_manager_secret_version.application_settings.secret
            }
          }
        }

        volume_mounts {
          name       = "cloudsql"
          mount_path = "/cloudsql"
        }

      }
    }
  }

  depends_on = [
    terraform_data.bootstrap,
  ]
}

# Create the create_superuser Cloud Run job
resource "google_cloud_run_v2_job" "create_superuser" {
  name     = "create-superuser"
  location = var.region

  template {
    template {
      service_account = google_service_account.django_cloudrun.email

      volumes {
        name = "cloudsql"
        cloud_sql_instance {
          instances = [google_sql_database_instance.main.connection_name]
        }
      }

      containers {
        image   = local.image
        command = ["create_superuser"]

        env {
          name = "APPLICATION_SETTINGS"
          value_source {
            secret_key_ref {
              version = google_secret_manager_secret_version.application_settings.version
              secret  = google_secret_manager_secret_version.application_settings.secret
            }
          }
        }

        env {
          name = "DJANGO_SUPERUSER_PASSWORD"
          value_source {
            secret_key_ref {
              version = google_secret_manager_secret_version.superuser_password.version
              secret  = google_secret_manager_secret_version.superuser_password.secret
            }
          }
        }

        volume_mounts {
          name       = "cloudsql"
          mount_path = "/cloudsql"
        }

      }
    }
  }

  depends_on = [
    terraform_data.bootstrap
  ]
}

# Run the migrate_collectstatic the Cloud Run job
resource "terraform_data" "execute_migrate_collectstatic" {
  provisioner "local-exec" {
    command = "gcloud run jobs execute migrate-collectstatic --region ${var.region} --wait"
  }

  depends_on = [
    google_cloud_run_v2_job.migrate_collectstatic,
  ]
}

# Run the create_superuser the Cloud Run job
resource "terraform_data" "execute_create_superuser" {

  provisioner "local-exec" {
    command = "gcloud run jobs execute create-superuser --region ${var.region} --wait"
  }

  depends_on = [
    google_cloud_run_v2_job.create_superuser,
  ]
}

# Create and deploy the Cloud Run service
resource "google_cloud_run_service" "app" {
  name                       = var.service_name
  location                   = var.region
  autogenerate_revision_name = true

  lifecycle {
    replace_triggered_by = [terraform_data.bootstrap]
  }

  template {
    spec {
      service_account_name = google_service_account.django_cloudrun.email
      containers {
        image = local.image

        env {
          name  = "SERVICE_NAME"
          value = var.service_name
        }

        env {
          name = "APPLICATION_SETTINGS"
          value_from {
            secret_key_ref {
              key  = google_secret_manager_secret_version.application_settings.version
              name = google_secret_manager_secret.application_settings.secret_id
            }
          }
        }
      }
    }

    metadata {
      annotations = {
        "autoscaling.knative.dev/maxScale"      = "1"
        "run.googleapis.com/cloudsql-instances" = google_sql_database_instance.main.connection_name
        "run.googleapis.com/client-name"        = "terraform"
      }
    }


  }

  traffic {
    percent         = 100
    latest_revision = true
  }

  depends_on = [
    terraform_data.execute_migrate_collectstatic,
    terraform_data.execute_create_superuser,
  ]

}

# Grant permission to unauthenticated users to invoke the Cloud Run service
data "google_iam_policy" "noauth" {
  binding {
    role    = "roles/run.invoker"
    members = ["allUsers"]
  }
}

resource "google_cloud_run_service_iam_policy" "noauth" {
  location = google_cloud_run_service.app.location
  project  = google_cloud_run_service.app.project
  service  = google_cloud_run_service.app.name

  policy_data = data.google_iam_policy.noauth.policy_data
}

# Print the Cloud Run service url
output "service_url" {
  value = google_cloud_run_service.app.status[0].url
}

Initialize Terraform by running the following command

terraform init

You will see printed output similar to this on successfuly initialization

Initializing the backend...

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google versions matching "~> 5.9.0"...
- Finding hashicorp/random versions matching "~> 3.6.0"...
- Installing hashicorp/google v5.9.0...
- Installed hashicorp/google v5.9.0 (signed by HashiCorp)
- Installing hashicorp/random v3.6.0...
- Installed hashicorp/random v3.6.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Run terraform plan to review the resource terraform will create in Google Cloud

terraform plan

The plan will display the number of resources that will be added

Plan: 32 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + service_url = (known after apply)

Run terraform apply to create the resources in Google Cloud and deploy the Cloud Run service

terraform apply

Type yes at the prompt and press enter:

Plan: 32 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + service_url = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

The process will take a while to create all the resources and display output similar to this on completion.

Apply complete! Resources: 32 added, 0 changed, 0 destroyed.

Outputs:

service_url = "https://django-on-cloudrun-qfes6ko4lq-uc.a.run.app"

Copy the printed value of service_url

Step 5 - Logging into the application

Visit the service url in your browser and you will see the Django welcome page

Navigate to the Django Admin login page by appending /admin to your service URL.

https://django-on-cloudrun-qfes6ko4lq-uc.a.run.app/admin

Retrieve the superuser password using the following command:

gcloud secrets versions access latest --secret superuser_password && echo ""

Step 6 - Disabling DEBUG mode

Modify the templates/application_settings.tftpl file and change the DEBUG value from on to off.

The updated file should look like this

DATABASE_URL="postgres://${user.name}:${user.password}@//cloudsql/${instance.project}:${instance.region}:${instance.name}/${database.name}"
STATICFILES_BUCKET_NAME="${staticfiles_bucket}"
SECRET_KEY="${secret_key}"
DEBUG=off

Run teraform plan to review the changes

terraform plan

Terraform will display a plan of the actions to make the desired change

...

  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

....

Plan: 1 to add, 3 to change, 1 to destroy.

Run teraform apply to perform the desired changes

terraform apply

Type yes at the prompt and press enter:

...

Plan: 1 to add, 3 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Terraform will update the secret and redeploy the service then print the service url on completion.

Apply complete! Resources: 1 added, 3 changed, 1 destroyed.

Outputs:

service_url = "https://django-on-cloudrun-qfes6ko4lq-uc.a.run.app"

Visit the service URL and confirm that the welcome page is not displayed anymore when DEBUG mode is disabled.

Step 7 - Cleaning Up

In order not incur unnecessary cost you can delete the project.

This action will delete all the resources you created in the previous steps.

gcloud projects delete ${PROJECT_ID}

Type y at the prompt and press enter:

Your project will be deleted.

Do you want to continue (Y/n)?  y

After the project is deleted you will see a message like this

Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/django-on-cloud-run-409909].
...

The Source code for this article can be found here.

Google Cloud Run Service-to-Service Authentication using Go

Joshua Masiko — Sat, 06 May 2023 12:34:17 +0000

In this tutorial, we will demonstrate how to set up service-to-service authentication for Google Cloud Run services using the Go programming language.

We will cover the following steps:

Deploy two services - a sender and a receiver. Initially, the receiver will not require any authentication.
Add authentication to the receiver and demonstrate that the sender's requests will fail.
Authorize the sender identity to the receive and demonstrate that the sender's requests will succeed

Prerequisites

Before you can begin this tutorial, you will need the following:

A Google Cloud Platform (GCP) account and a new project
Enable billing for the new project
Install and configure the Google Cloud SDK (gcloud) on your local machine
Enable the required APIs

Creating a new project

Go to the Google Cloud Console.
Click the project drop-down and select or create the project you want to use for this tutorial.
Take note of your project ID, as you will need it later.

Enable billing

In the Cloud Console, open the main menu and select "Billing".
Click "Link a billing account" and follow the steps to set up billing for your project.

Install and configure the Google Cloud SDK (gcloud)

Download and install the Google Cloud SDK for your operating system.
Authenticate to your Google Cloud account using the gcloud tool:

$ gcloud auth login

This command will open a new browser window, asking you to log in with your Google account and grant permission to access your GCP resources.

Set up the default project and region:

$ gcloud config set project <YOUR_PROJECT_ID>
$ gcloud config set compute/region <YOUR_REGION>

Accept to enable the compute.googleapis.com API on the project if prompted.

Replace <YOUR_PROJECT_ID> with the project ID you noted earlier and <YOUR_REGION> with a desired region, such as us-central1.

Set the PROJECT_ID and REGION variables using the currently configured project and region:

$ PROJECT_ID=$(gcloud config get-value project)
$ REGION=$(gcloud config get-value compute/region)

Enable the required APIs

In the Cloud Console, open the main menu and select "APIs & Services" > "Library".
Search for "Cloud Run" and click on "Cloud Run API".
Click "Enable" to enable the Cloud Run API for your project.
Repeat the process for "Cloud Build API" and "Identity Token Service API".

You can also enable the required APIs using the gcloud command-line tool by running the following commands:

Enable the Cloud Run API:

$ gcloud services enable run.googleapis.com

Enable the Cloud Build API:

$ gcloud services enable cloudbuild.googleapis.com

Enable the Identity Token Service API (also known as the IAM Service Account Credentials API):

$ gcloud services enable iamcredentials.googleapis.com

These commands will enable the respective APIs for your current GCP project.

Each of the three APIs serves a specific purpose, and enabling them ensures that the required services are accessible in your project.

Cloud Run API: The Cloud Run API is necessary for deploying, managing, and scaling your containerized applications on Google Cloud Run.
Cloud Build API: The Cloud Build API allows you to build, test, and deploy your source code on Google Cloud. In this tutorial, you use the gcloud builds submit command to build your Docker images and push them to Google Container Registry.
Identity Token Service API (IAM Service Account Credentials API): This API enables you to generate access tokens, sign JSON Web Tokens (JWTs), and create OIDC ID tokens for service accounts. In the tutorial, you use service-to-service authentication, which requires generating identity tokens for your services.

Now you're ready to proceed with the tutorial on service-to-service authentication for Google Cloud Run using Go.

Deploy sending and receiving services

Create a directory for the project source code

$ mkdir ${PROJECT_ID} && cd ${PROJECT_ID}

Receiving Service

First, let's create the receiving service. Create a new directory named receiving-service and navigate to it:

$ mkdir receiving-service && cd receiving-service && go mod init receiver

Create a file named main.go and add the following code:

package main

import (
  "fmt"
  "net/http"
)

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello from the receiving service!")
  })

  http.ListenAndServe(":8080", nil)
}

Create a Dockerfile in the same directory with the following content:

# Build stage
FROM golang:1.17 AS builder

# Set the working directory
WORKDIR /app

# Copy Go module files
COPY go.* ./

# Download dependencies
RUN go mod download

# Copy Go files
COPY main.go .

# Build the binary
RUN CGO_ENABLED=0 GOOS=linux go build -o server .

# Final stage
FROM alpine:3.15

# Set the working directory
WORKDIR /app

# Copy the binary from the builder stage
COPY --from=builder /app/server /app/server

# Expose the listening port
EXPOSE 8080

# Run the server
CMD ["/app/server"]

Now, build and push the Docker image to Google Container Registry (GCR):

$ gcloud builds submit --tag gcr.io/${PROJECT_ID}/receiving-service

After the build is complete, deploy the service to Cloud Run:

$ gcloud run deploy receiving-service --image gcr.io/${PROJECT_ID}/receiving-service --region ${REGION} --platform managed --allow-unauthenticated

Set the RECEIVING_SERVICE_URL variable to the value of the receiving service URL after deployment.

$ RECEIVING_SERVICE_URL=$(gcloud run services describe receiving-service --region $REGION --format='value(status.url)')

Make a request to the receiving service URL and confirm that it returns the expected response

$ curl ${RECEIVING_SERVICE_URL}
Hello from the receiving service!

Sending Service

Create a new directory named sending-service and navigate to it:

$ cd ../ && mkdir sending-service && cd sending-service && go mod init sender

Create a file named main.go and add the following code:

package main

import (
  "fmt"
  "io/ioutil"
  "log"
  "net/http"
  "os"
)

func main() {
  receivingServiceURL := os.Getenv("RECEIVING_SERVICE_URL")
  if receivingServiceURL == "" {
    log.Fatal("RECEIVING_SERVICE_URL environment variable is not set")
  }

  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    resp, err := http.Get(receivingServiceURL)
    if err != nil {
      log.Printf("Failed to make request: %v", err)
      http.Error(w, "Failed to make request", http.StatusInternalServerError)
      return
    }
    defer resp.Body.Close()

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
      log.Printf("Failed to read response body: %v", err)
      http.Error(w, "Failed to read response body", http.StatusInternalServerError)
      return
    }

    fmt.Fprintf(w, "Response from receiving service: %s", string(body))
  })

  http.ListenAndServe(":8080", nil)
}

Create a Dockerfile in the same directory with the following content:

# Build stage
FROM golang:1.17 AS builder

# Set the working directory
WORKDIR /app

# Copy Go module files
COPY go.* ./

# Download dependencies
RUN go mod download

# Copy Go files
COPY main.go .

# Build the binary
RUN CGO_ENABLED=0 GOOS=linux go build -o server .

# Final stage
FROM alpine:3.15

# Install CA certificates for HTTPS calls
RUN apk --no-cache add ca-certificates

# Set the working directory
WORKDIR /app

# Copy the binary from the builder stage
COPY --from=builder /app/server /app/server

# Expose the listening port
EXPOSE 8080

# Run the server
CMD ["/app/server"]

Now, build and push the Docker image to Google Container Registry (GCR):

$ gcloud builds submit --tag gcr.io/${PROJECT_ID}/sending-service

After the build is complete, deploy the service to Cloud Run:

$ gcloud run deploy sending-service --image gcr.io/${PROJECT_ID}/sending-service --region ${REGION} --platform managed --allow-unauthenticated --set-env-vars RECEIVING_SERVICE_URL=${RECEIVING_SERVICE_URL}

Set the SENDING_SERVICE_URL variable to the value of the sending service URL after deployment.

$ SENDING_SERVICE_URL=$(gcloud run services describe sending-service --region $REGION --format='value(status.url)')`

Make a request to the receiving service URL and confirm that it returns the expected response

$ curl ${RECEIVING_SERVICE_URL}
Hello from the receiving service!

Visit the sending service URL in your browser, and you should see the message from the receiving service, indicating that the call to the receiving service succeeded.

$ curl ${SENDING_SERVICE_URL}
Response from receiving service: Hello from the receiving service!

Update the receiving service

First, set the --no-allow-unauthenticated flag while deploying the receiving service to require authentication:

$ gcloud run deploy receiving-service --image gcr.io/${PROJECT_ID}/receiving-service --region ${REGION} --platform managed --no-allow-unauthenticated

Visit the sending service URL in your browser, and you should see an error message, indicating that the call to the receiving service failed.

$ curl ${SENDING_SERVICE_URL}
Response from receiving service: 
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>403 Forbidden</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Forbidden</h1>
<h2>Your client does not have permission to get URL <code>/</code> from this server.</h2>
<h2></h2>
</body></html>

Update the sending service

In the sending-service/main.go file, add import the required packages:

import (
  ...
  "context"
  "time"
  "google.golang.org/api/idtoken"
)

Then create a new function httpClientWithIDToken to generate an authenticated HTTP client:

func httpClientWithIDToken(ctx context.Context, audience string) (*http.Client, error) {
  client, err := idtoken.NewClient(ctx, audience)
  if err != nil {
    return nil, err
  }
  return client, nil
}

Run go mod tidy to download the required package.

Modify the main function to use the authenticated HTTP client:

func main() {
  receivingServiceURL := os.Getenv("RECEIVING_SERVICE_URL")
  if receivingServiceURL == "" {
    log.Fatal("RECEIVING_SERVICE_URL environment variable is not set")
  }

  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
    defer cancel()

    client, err := httpClientWithIDToken(ctx, receivingServiceURL)
    if err != nil {
      log.Printf("Failed to create authenticated client: %v", err)
      http.Error(w, "Failed to create authenticated client", http.StatusInternalServerError)
      return
    }

    resp, err := client.Get(receivingServiceURL)
    if err != nil {
      log.Printf("Failed to make request: %v", err)
      http.Error(w, "Failed to make request", http.StatusInternalServerError)
      return
    }
    defer resp.Body.Close()

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
      log.Printf("Failed to read response body: %v", err)
      http.Error(w, "Failed to read response body", http.StatusInternalServerError)
      return
    }

    fmt.Fprintf(w, "Response from receiving service: %s", string(body))
  })

  http.ListenAndServe(":8080", nil)
}

Rebuild and redeploy the sending service:

$ gcloud builds submit --tag gcr.io/${PROJECT_ID}/sending-service`

Now, deploy the sending service

$ gcloud run deploy sending-service --image gcr.io/${PROJECT_ID}/sending-service --region ${REGION} --platform managed --allow-unauthenticated --set-env-vars RECEIVING_SERVICE_URL=${RECEIVING_SERVICE_URL}

Test the authentication

Visit the sending service URL in your browser, and you should now see a successful response from the receiving service

curl ${SENDING_SERVICE_URL}                                                                                                                                                                               
Response from receiving service: Hello from the receiving service!

We have identified our calling service to the receiving service, but we haven't set up the receiving service to accept requests from the calling service.

By default, Cloud Run services utilize the Compute Engine default service account, which has the Project > Editor IAM role. This grants your Cloud Run revisions read and write access to all resources in your GCP project.

To adhere to the principle of least privilege, we will create a new service account that has limited permissions.

Create a new service account for the calling service

Run the following command to create a new service account:

$ gcloud iam service-accounts create calling-service-sa --display-name "Calling Service Account"

Redeploy the calling service with the new service account

Redeploy the calling service using the new service account:

$ gcloud run deploy sending-service --image gcr.io/${PROJECT_ID}/sending-service --region ${REGION} --platform managed --allow-unauthenticated --service-account calling-service-sa@${PROJECT_ID}.iam.gserviceaccount.com --set-env-vars RECEIVING_SERVICE_URL=${RECEIVING_SERVICE_URL}

Now, if you visit the sending service URL in your browser, you should see a failed response because the calling service doesn't have permission to invoke the receiving service.

$ curl ${SENDING_SERVICE_URL}                                                                   
Response from receiving service: 
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>403 Forbidden</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Forbidden</h1>
<h2>Your client does not have permission to get URL <code>/</code> from this server.</h2>
<h2></h2>
</body></html>

Grant the calling service identity permission to invoke the receiving service

To grant the calling service account permission to invoke the receiving service, run the following command:

$ gcloud run services add-iam-policy-binding receiving-service --region ${REGION} --member=serviceAccount:calling-service-sa@${PROJECT_ID}.iam.gserviceaccount.com --role=roles/run.invoker

Step 4: Test the authentication

Visit the sending service URL in your browser, and you should now see a successful response from the receiving service, authenticated using the new service account with limited permissions.

curl ${SENDING_SERVICE_URL}
Response from receiving service: Hello from the receiving service!

Conclusion

In this tutorial, we demonstrated how to set up service-to-service authentication for Google Cloud Run services using the Go programming language.

Customizing Django Authentication using AbstractBaseUser

Joshua Masiko — Fri, 20 Mar 2020 08:34:35 +0000

Introduction

Django has a builtin app that provides much of the required authentication machinery out of the box.
It also allows you to customize authentication if the defaults don't match your requirements.
There are multiple approaches you can take including:

Using a Proxy Model based on User.
Adding a OneToOneField that points to User on a Profile model.
Extending AbstractUser and adding fields for additional profile information.
Extending AbstractBaseUser and implementing the required functionality.

You can choose among the first 3 options if you wan't to add minimal customization to the Django defaults.
If, however, you wan't to use an email address instead of a username as the user identifier, you have to extend AbstractBaseUser.
This article will explain how to do this using a bare-bones listings application as an example.

Prerequisites

You will need to have working knowledge of Python and the Django Framework.
The Django version used in this tutorial is 3.04.
A recent version of Python 3.
The instructions in this tutorial were performed on Ubuntu 18.04 LTS with Python 3.6.

High Level Description of the App

Unauthenticated users can view all listings.
Users have to register to post listings.
The registration information includes:

Mandatory fields: email, password, phone.
Optional fields: date_of_birth, photo.

Superusers can create staff users using the Django admin interface.
Staff can flag listings using the Django admin interface.

The most recent 30 unflagged listings are displayed on the home page.
Users can click on a listing to view its details.

Getting Started

On Ubuntu 18.04 you might need to add the universe repository and install python3-venv.

$ sudo add-apt-repository universe
$ sudo apt install -y python3-venv

Project Setup

Create and activate the virtual environment.

$ mkdir ~/.virtualenvs
$ python3 -m venv ~/.virtualenvs/listings
$ source ~/.virtualenvs/listings/bin/activate

Then install Django and create a project using the django-admin command.

(listings) $ pip install django==3.0.4
(listings) $ django-admin startproject listings
(listings) $ cd listings

You can now run the dev server and navigate to http://127.0.0.1:8000 in your browser.
You should see the Django Welcome Page.

(listings) $ python manage.py runserver

Ignore the warning about unapplied migrations in the console.
The recommended procedure is to apply migrations after setting up the custom user model.

The accounts app

Let's add an app to host our authentiation customizations.

(listings) $ python manage.py startapp accounts

The model and Manager

# accounts/models.py
from django.db import models
from django.contrib.auth.models import AbstractBaseUser, BaseUserManager, PermissionsMixin
from django.contrib.auth import get_user_model
from django.utils import timezone


class AccountManager(BaseUserManager):
    use_in_migrations = True

    def _create_user(self, email, name, phone, password, **extra_fields):
        values = [email, name, phone]
        field_value_map = dict(zip(self.model.REQUIRED_FIELDS, values))
        for field_name, value in field_value_map.items():
            if not value:
                raise ValueError('The {} value must be set'.format(field_name))

        email = self.normalize_email(email)
        user = self.model(
            email=email,
            name=name,
            phone=phone,
            **extra_fields
        )
        user.set_password(password)
        user.save(using=self._db)
        return user

    def create_user(self, email, name, phone, password=None, **extra_fields):
        extra_fields.setdefault('is_staff', False)
        extra_fields.setdefault('is_superuser', False)
        return self._create_user(email, name, phone, password, **extra_fields)

    def create_superuser(self, email, name, phone, password=None, **extra_fields):
        extra_fields.setdefault('is_staff', True)
        extra_fields.setdefault('is_superuser', True)

        if extra_fields.get('is_staff') is not True:
            raise ValueError('Superuser must have is_staff=True.')
        if extra_fields.get('is_superuser') is not True:
            raise ValueError('Superuser must have is_superuser=True.')

        return self._create_user(email, name, phone, password, **extra_fields)


class Account(AbstractBaseUser, PermissionsMixin):
    email = models.EmailField(unique=True)
    name = models.CharField(max_length=150)
    phone = models.CharField(max_length=50)
    date_of_birth = models.DateField(blank=True, null=True)
    picture = models.ImageField(blank=True, null=True)
    is_staff = models.BooleanField(default=False)
    is_active = models.BooleanField(default=True)
    date_joined = models.DateTimeField(default=timezone.now)
    last_login = models.DateTimeField(null=True)

    objects = AccountManager()

    USERNAME_FIELD = 'email'
    REQUIRED_FIELDS = ['name', 'phone']

    def get_full_name(self):
        return self.name

    def get_short_name(self):
        return self.name.split()[0]

USERNAME_FIELD is the name of the field on the user model that is used as the unique identifier.
REQUIRED_FIELDS are the mandatory fields other than the unique identifier

The create_user and create_superuser functions should accept the username field, plus all required fields as positional arguments.
Django’s provides a PermissionsMixin which we can include in the class hierarchy for our user model to support Django’s permissions.

Add the accounts app to the list of installed apps in settings.py

# listings/settings.py
...
INSTALLED_APPS = [
    ...
    'accounts.apps.AccountsConfig',
]

Specify the custom model as the default user model using the AUTH_USER_MODEL setting in settings.py

# listings/settings.py
...
AUTH_USER_MODEL = 'accounts.Account'

Install Pillow which is required by Django's ImageField

(listings) $ pip install pillow

Generate migrations for the model now that the model is setup.

(listings) $ python manage.py makemigrations accounts

And create the database.

(listings) $ python manage.py migrate

The forms

Subclassing AbstractBaseUser enables us to re-use the Django auth app built-in forms and views.

The following forms are compatible with any subclass of AbstractBaseUser:
AuthenticationForm: Uses the username field specified by USERNAME_FIELD.
SetPasswordForm: Allows users to change their password without entering the old password.
PasswordChangeForm: Allows users to change their password by entering the old password and a new password.
AdminPasswordChangeForm: Allows users to change their password from the Django admin.
PasswordResetForm: Assumes users to reset their passwords using a reset link sent to the email address.
These forms are by the built-in auth views to which we delegate authentication and password management.

UserCreationForm and UserChangeForm: Can be used to create users and change user details.
These forms are tied to default Django User model so we will provide our own implementations.
We will also add a registration form for signing up users.

# accounts/forms.py
from django import forms
from django.contrib.auth.models import Group
from django.contrib.auth.forms import ReadOnlyPasswordHashField

from .models import Account


class RegistrationForm(forms.ModelForm):
    password = forms.CharField(label='Password', widget=forms.PasswordInput)
    class Meta:
        model = Account
        fields = ('email', 'name', 'phone', 'date_of_birth', 'picture', 'password')

    def save(self, commit=True):
        # Save the provided password in hashed format
        user = super().save(commit=False)
        user.set_password(self.cleaned_data["password"])
        if commit:
            user.save()
        return user


class UserCreationForm(forms.ModelForm):
    password1 = forms.CharField(label='Password', widget=forms.PasswordInput)
    password2 = forms.CharField(label='Password confirmation', widget=forms.PasswordInput)

    class Meta:
        model = Account
        fields = ('email', 'name', 'phone', 'date_of_birth', 'picture', 'is_staff', 'is_superuser')

    def clean_password2(self):
        # Check that the two password entries match
        password1 = self.cleaned_data.get("password1")
        password2 = self.cleaned_data.get("password2")
        if password1 and password2 and password1 != password2:
            raise forms.ValidationError("Passwords don't match")
        return password2

    def save(self, commit=True):
        # Save the provided password in hashed format
        user = super().save(commit=False)
        user.set_password(self.cleaned_data["password1"])
        if commit:
            user.save()
        return user


class UserChangeForm(forms.ModelForm):
    password = ReadOnlyPasswordHashField()

    class Meta:
        model = Account
        fields = ('email', 'name', 'phone', 'date_of_birth', 'picture', 'password', 'is_active', 'is_superuser')

    def clean_password(self):
        # Regardless of what the user provides, return the initial value.
        # This is done here, rather than on the field, because the
        # field does not have access to the initial value
        return self.initial["password"]

The views

We implement views for user registration and profile editing.

from django.shortcuts import render

from django.shortcuts import render, redirect
from django.contrib.auth import authenticate, login as auth_login, logout
from django.contrib import messages
from django.contrib.auth.forms import AuthenticationForm
from django.shortcuts import render

from django.contrib.auth.mixins import LoginRequiredMixin
from django.views.generic import ListView
from django.views.generic.edit import CreateView, UpdateView
from django.urls import reverse

from .models import Account
from .forms import RegistrationForm


class RegistrationView(CreateView):
    template_name = 'registration/register.html'
    form_class = RegistrationForm

    def get_context_data(self, *args, **kwargs):
        context = super(RegistrationView, self).get_context_data(*args, **kwargs)
        context['next'] = self.request.GET.get('next')
        return context

    def get_success_url(self):
        next_url = self.request.POST.get('next')
        success_url = reverse('login')
        if next_url:
            success_url += '?next={}'.format(next_url)

        return success_url


class ProfileView(UpdateView):
    model = Account
    fields = ['name', 'phone', 'date_of_birth', 'picture']
    template_name = 'registration/profile.html'

    def get_success_url(self):
        return reverse('index')

    def get_object(self):
        return self.request.user

The admin

Now we register the custom user model with Django’s admin.

# accounts/admin.py
from django.contrib import admin
from django.contrib.auth.admin import UserAdmin as BaseUserAdmin

from .models import Account
from .forms import UserCreationForm, UserChangeForm


class AccountAdmin(BaseUserAdmin):
    form = UserChangeForm
    add_form = UserCreationForm

    list_display = ('email', 'name', 'phone', 'date_of_birth', 'is_staff',  'is_superuser')
    list_filter = ('is_superuser',)

    fieldsets = (
        (None, {'fields': ('email', 'is_staff', 'is_superuser', 'password')}),
        ('Personal info', {'fields': ('name', 'phone', 'date_of_birth', 'picture')}),
        ('Groups', {'fields': ('groups',)}),
        ('Permissions', {'fields': ('user_permissions',)}),
    )
    add_fieldsets = (
        (None, {'fields': ('email', 'is_staff', 'is_superuser', 'password1', 'password2')}),
        ('Personal info', {'fields': ('name', 'phone', 'date_of_birth', 'picture')}),
        ('Groups', {'fields': ('groups',)}),
        ('Permissions', {'fields': ('user_permissions',)}),
    )

    search_fields = ('email', 'name', 'phone')
    ordering = ('email',)
    filter_horizontal = ()


admin.site.register(Account, AccountAdmin)

Testing the app so far

We should now able to create users and edit their details.

Let's create a superuser.

(listings) $ python manage.py createsuperuser
Email: admin@example.com
Name: Admin User
Phone: 123456
Password:
Password (again):
Superuser created successfully.

Run the Django dev server then navigate to http://127.0.0.1:8000/admin in your browser and login with the credentials you specified.

(listings) $ python manage runserver

You should be able to add and edit users.

The core app

Now let's implement the core app functionality which includes:

Anyone can view listings.
Users can register to create accounts.
Registered users can post listings
Logged in users can change their passwords.
Logged in users can edit their profiles.
Staff users can flag listings.

We will use Bootstrap 4 and django-crispy-forms for styling the application interface and forms.

Install django-crispy-forms

Install django-crispy-form using pip

(listings) $ pip install django-crispy-forms

Then add crispy_forms the the installed apps list in your settings file and set the default styling to bootstrap4:

# listings/settings.py
...
INSTALLED_APPS = [
    ...
    'crispy_forms',
]

CRISPY_TEMPLATE_PACK = 'bootstrap4'

Create the app

(listings) $ python manage.py startapp core

Then add it to the installed apps list in settings.py

# listings/settings.py
INSTALLED_APPS = [
...
    'core.apps.CoreConfig',
]

Since the user model has a picture field let's set up MEDIA_URL and MEDIA_ROOT to tell Django where to save uploaded images.

# listings/settings.py
...
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

The auth app password reset view sends an email to the user containing instructions and a URL.
We will use the console email backend for testing purposes.

# listings/settings.py
...
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

We also want to redirect to the home view on login and logout.

LOGIN_REDIRECT_URL = 'home'
LOGOUT_REDIRECT_URL = 'home'

The models

Each listing is associated with the user and category.
We also track the date and time that the listing was posted.
Listings may be flagged. We record the timestamp and user who flagged the listing.
By default listings will be sorted in reverse chronological order.

# core/models.py
from django.db import models
from django.conf import settings
from django.utils import timezone
from django.urls import reverse
from django.template.defaultfilters import slugify


class Category(models.Model):
    name = models.CharField(max_length=150)

    def __str__(self):
        return self.name

    class Meta:
        verbose_name_plural = "Categories"


class Listing(models.Model):
    title = models.CharField(max_length=150)
    content = models.TextField()
    category = models.ForeignKey(Category, on_delete=models.PROTECT)
    expiry_date = models.DateField(null=True, blank=True)
    location = models.CharField(max_length=150, null=True, blank=True)
    slug = models.SlugField()

    created_by = models.ForeignKey(
        settings.AUTH_USER_MODEL,
        on_delete=models.PROTECT,
        related_name='listings'
    )
    created_at = models.DateTimeField(default=timezone.now)

    flagged = models.BooleanField(default=False)
    flagged_by = models.ForeignKey(
        settings.AUTH_USER_MODEL,
        on_delete=models.PROTECT,
        null=True,
        blank=True
    )
    flagged_at = models.DateTimeField(null=True, blank=True)

    def __str__(self):
        return self.title

    class Meta:
        ordering = ['-created_at']

    def save(self, *args, **kwargs):
        if not self.id:
            self.slug = slugify(self.title)

        return super(Listing, self).save(*args, **kwargs)

    def get_absolute_url(self):
        return reverse('listing',
                       args=[self.slug])

We can now create migrations for the app and sync the database with the models.

(listings) $ python manage.py makemigrations core
(listings) $ python manage.py migrate core

The admin

We register the Category and Listing models with the Django admin.

# core/admin.py
from django.contrib import admin
from django.utils import timezone

from .models import Category, Listing


class ListingAdmin(admin.ModelAdmin):
    list_display = ('category', 'title', 'content', 'created_by', 'flagged', 'flagged_by', 'flagged_at')
    list_filter = ('flagged',)
    exclude = ['slug', 'created_by', 'created_at', 'flagged_at', 'flagged_by']

    def save_model(self, request, obj, form, change):
        if not obj.pk:
            obj.created_by = request.user
        if obj.flagged:
            obj.flagged_by = request.user
            obj.flagged_at = timezone.now()
        else:
            obj.flagged_by = obj.flagged_at = None

        super().save_model(request, obj, form, change)


admin.site.register(Category)
admin.site.register(Listing, ListingAdmin)

Navigate to http://127.0.0.1:8000/admin.
Login as the superuser and you will see new entries for Categories and Listings displayed.

Clicking the Add button and create some test categories.

The views

The application functionality consists of 3 views:

The default view displays the most recent 30 listings.
A view to post a listing, restricted to logged-in users.
A view to display a listings details.

# core/views.py
from django.shortcuts import render

from django.contrib.auth.mixins import LoginRequiredMixin
from django.http import Http404
from django.views.generic import DetailView
from django.views.generic.list import ListView
from django.views.generic.edit import CreateView
from django.urls import reverse

from .models import Listing


class HomeView(ListView):
    template_name = 'home.html'
    queryset = Listing.objects.filter(flagged=False)
    context_object_name = 'listings'
    paginate_by = 30


class CreateListing(LoginRequiredMixin, CreateView):
    model = Listing
    fields = ['title', 'content', 'category', 'expiry_date', 'location']
    template_name = 'add_listing.html'

    def get_success_url(self):
        return reverse('home')

    def form_valid(self, form):
        form.instance.created_by = self.request.user
        return super().form_valid(form)


class ListingView(DetailView):
    template_name = 'listing.html'
    model = Listing

    def get_object(self):
        obj = super(ListingView, self).get_object()
        if obj.flagged:
            raise Http404()
        return obj

The URLs

Let's start with th project-level URLs. We will re-use the auth app views for Login, Logout, Password Change & Password Reset.

# listings/urls.py
from django.contrib import admin
from django.urls import path, include
from django.contrib.auth import views as auth_views

from accounts import views
from accounts.views import RegistrationView, ProfileView

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('core.urls')),

    path('register/', RegistrationView.as_view(), name='register'),
    path('profile/', ProfileView.as_view(), name='profile'),
    path('accounts/login/', auth_views.LoginView.as_view(), name='login'),
    path('accounts/logout/', auth_views.LogoutView.as_view(), name='logout'),

    path('password_change/', auth_views.PasswordChangeView.as_view(), name='password_change'),
    path('password_change/done/', auth_views.PasswordChangeDoneView.as_view(), name='password_change_done'),

    path('password_reset/', auth_views.PasswordResetView.as_view(), name='password_reset'),
    path('password_reset/done/', auth_views.PasswordResetDoneView.as_view(), name='password_reset_done'),
    path('reset/<uidb64>/<token>/', auth_views.PasswordResetConfirmView.as_view(), name='password_reset_confirm'),
    path('reset/done/', auth_views.PasswordResetCompleteView.as_view(), name='password_reset_complete'),
]

Then the core app URLs

# core/urls.py
from django.urls import path, include

from .views import HomeView, CreateListing, ListingView

urlpatterns = [
    path('', HomeView.as_view(), name='home'),
    path('add-listing', CreateListing.as_view(), name='add_listing'),
    path('listings/<slug:slug>/', ListingView.as_view(), name='listing'),
]

The auth views render templates from a registration folder.
We will create a project level templates folder for all our templates and a registration subfolder.

(listings) $ mkdir -p templates/registration

Then add the templates folder to the DIRS setting in settings.py

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

The templates

`auth` templates

These templates will be used by the auth app views.
Auth templates will exend a base template.

<!--templates/registration/auth_base.html-->
<!doctype  html>
<html  lang="en">
<head>
  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">
<title>Listings</title>
</head>
<body>
  <div  class="container">
    <div  class="row justify-content-center">
      <div  class="col-4">
      <h1  class="text-center">Listings</h1>
      {% block content %}

      {% endblock %}
      </div>
    </div>
  </div>
</body>
</html>

<!--templates/registration/login.html-->
{% extends "registration/auth_base.html" %}
{% load crispy_forms_tags %}

{% block content %}

<div class="card">
  <div class="card-body">
    <h4 class="card-title">Log in to your account</h4>
    <form method="post">
      {% csrf_token %}
      <input type="hidden" name="next" value="{{ next }}">{{ form|crispy }}<button type="submit"
        class="btn btn-primary btn-block">Log in</button>
    </form>
  </div>
  <div class="card-footer">
    Forgot your password? <a href="{% url "password_reset" %}">click here</a><br>
    New user? <a href="{% url "register" %}?next={{ next }}">create new account</a>
  </div>
</div>

{% endblock %}

<!--templates/registration/register.html-->
{% extends "registration/auth_base.html" %}
{% load crispy_forms_tags %}

{% block content %}

<div class="card">
    <div class="card-body">
        <h4 class="card-title">Signup for an account</h4>
        <form method="post" enctype="multipart/form-data">
            {% csrf_token %}
            {{form|crispy}}
            <input type="hidden" name="next" value="{{ next }}">
            <button type="submit" class="btn btn-primary btn-block">Register</button>

        </form>
    </div>
</div>
{% endblock %}

Change password template used by PasswordChangeView.

<!--templates/registration/password_change_form.html-->
{% extends 'base.html' %}
{% load crispy_forms_tags %}
{% block content %}

<div class="row justify-content-center">
    <div class="col-4">
        <div class="card">
            <div class="card-body">
                <h4 class="card-title">Change your password</h4>
                <form method="POST" enctype="multipart/form-data">
                    {% csrf_token %}
                    {{form|crispy}}
                    <button type="submit" class="btn btn-primary btn-block">Change Password</button>

                </form>
            </div>
        </div>
    </div>
</div>
{% endblock %}

Password change done template used by PasswordChangeDoneView.

<!--templates/registration/password_change_done.html-->
{% extends 'base.html' %}
{% load crispy_forms_tags %}
{% block content %}

<div class="row justify-content-center">
    <div class="col-8">
        <div class="card">
            <div class="card-body">
                <h2>Your password has changed</h2>
            </div>
        </div>
    </div>
</div>
{% endblock %}

Password Reset template used by PasswordResetView.

<!--templates/registration/password_reset_form.html-->
{% extends "registration/auth_base.html" %}
{% load crispy_forms_tags %}

{% block content %}

<div class="card">
    <div class="card-body">
        <h4 class="card-title">Reset your password</h4>
        <form method="post">
            {% csrf_token %}
            {{form|crispy}}
            <button type="submit" class="btn btn-primary btn-block">Reset</button>

        </form>
    </div>
</div>
{% endblock %}

Password reset email template used by PasswordResetView.

<!--templates/registration/password_reset_email.html-->
{% autoescape off %}
You're receiving this email because you requested a password reset for your user account at {{ site_name }}.

Please go to the following page and choose a new password:
{% block reset_link %}
{{ protocol }}://{{ domain }}{% url 'password_reset_confirm' uidb64=uid token=token %}
{% endblock %}
Your username, in case you’ve forgotten: {{ user.get_username }}

Thanks for using our site!

The {{ site_name }} team

{% endautoescape %}

Password reset done template used by PasswordResetDoneView.

<!--registration/templates/password_reset_done.html-->
{% extends "registration/auth_base.html" %}

{% block content %}

<div class="card">
    <div class="card-body">


        <p>We’ve emailed you instructions for setting your password, if an account exists with the email you entered.
            You should receive them shortly.</p>

        <p>If you don’t receive an email, please make sure you’ve entered the address you registered with, and check
            your spam folder.</p>
    </div>
</div>
{% endblock %}

Password reset confirm template used by PasswordResetConfirmView.

<!--templates/registration/password_reset_confirm.html-->
{% extends "registration/auth_base.html" %}
{% load crispy_forms_tags %}

{% block content %}

<div class="card">
    <div class="card-body">

        {% if validlink %}

        <p>Please enter your new password twice so we can verify you typed it in correctly.</p>

        <form method="post">{% csrf_token %}
            {{form|crispy}}
            <button type="submit" class="btn btn-primary btn-block">Change my password</button>

        </form>

        {% else %}

        <p>The password reset link was invalid, possibly because it has already been used. Please request a new password
            reset.</p>

        {% endif %}
    </div>
</div>
{% endblock %}

Password reset complete template used by PasswordResetCompleteView.

<!--templates/registration/password_reset_complete.html-->
{% extends "registration/auth_base.html" %}
{% block content %}

<div class="card">
    <div class="card-body">

        <p>Your password has been set. You may go ahead and log in now.</p>

        <p><a href="{{ login_url }}">Log in</a></p>
    </div>
</div>
{% endblock %}

Edit Profile template used by ProfileView.

<!--templates/registration/profile.html-->
{% extends 'base.html' %}
{% load crispy_forms_tags %}
{% block content %}

<div class="row justify-content-center">
    <div class="col-4">
        <div class="card">
            <div class="card-body">
                <h4 class="card-title">Update your profile</h4>
                <form method="POST" enctype="multipart/form-data">
                    {% csrf_token %}
                    {{form|crispy}}
                    <button type="submit" class="btn btn-primary btn-block">Update Profile</button>

                </form>
            </div>
        </div>
    </div>
</div>
{% endblock %}

`core` templates

These templates will be used by the core app and should be saved in the project level templates folder.

The Base Template

<!--templates/base.html-->
<!doctype html>
<html lang="en">

<head>
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css">

    <title>Listings</title>
    <style>
        body {
            padding-top: 5rem;
        }
    </style>
</head>

<body>

    <nav class="navbar navbar-expand-md navbar-dark fixed-top bg-dark">
        <a href="{% url "home" %}" class="navbar-brand">Listings</a>
        <button type="button" class="navbar-toggler" data-toggle="collapse" data-target="#navbarCollapse">
            <span class="navbar-toggler-icon"></span>
        </button>
        {% if user.is_authenticated %}
        <div id="navbarCollapse" class="collapse navbar-collapse">

            <ul class="nav navbar-nav ml-auto">
                <li class="nav-item dropdown">
                    <a href="#" class="nav-link dropdown-toggle" data-toggle="dropdown">{{user.name}}</a>
                    <div class="dropdown-menu dropdown-menu-right">
                        <a href="{% url "profile" %}" class="dropdown-item">Profile</a>
                        <a href="{% url "password_change" %}" class="dropdown-item">Change Password</a>
                        <div class="dropdown-divider"></div>
                        <a href="{% url "logout" %}" class="dropdown-item">Logout</a>
                    </div>
                </li>
            </ul>
        </div>
        {% endif %}
    </nav>

    <main role="main" class="container">
        {% block content %}

        {% endblock %}

    </main>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js"></script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js"></script>
</body>

The Home template

<!--templates/home.html-->
{% extends 'base.html' %}
{% block content %}

<a href="{% url "add_listing" %}">Add Listing</a>
<h3 class="mt-2">Latest Listings</h3>
<div class="col-md-8 mt-3">    
    {% for listing in listings %}
    <div class="card">   
      <div class="card-body">


      <blockquote class="blockquote mb-0">
        <p>{{listing.title}}<small><a href="{{listing.get_absolute_url}}" >&nbsp;View Details &rarr;</a></small></p>
        <footer class="blockquote-footer"><small>Posted on {{listing.created_at|date:"M d, Y"}} by
            <a href="#">{{listing.created_by.name}}</a> in <a href="">{{listing.category}}</a></small></footer>
      </blockquote>
    </div>
    {% endfor %}
</div>
{% endblock %}

Listing detail template

<!--templates/listing.html-->
{% extends 'base.html' %}
{% block content %}

<div class="card mb-3">
    <div class="card-body">
      <h5 class="card-title">{{listing.title}}</h5>
      <p class="card-text">{{listing.content}}</p>
      <p class="card-text"><small class="text-muted">Posted by {{listing.created_by.name}} on {{listing.created_at}}</small></p>
    </div>
  </div>

{% endblock %}

Add Listing template

<!--templates/add_listing.html-->
{% extends 'base.html' %}
{% load crispy_forms_tags %}
{% block content %}
<div class="row">
<form method="POST" enctype="multipart/form-data">
    {% csrf_token %}
    {{form |crispy}}
    <button type="submit" class="btn btn-primary btn-block">Add Listing</button>

</form>
</div>
{% endblock %}

Testing the app

Navigate to http://127.0.0.1:8000 in your browser.

Clicking Add Listing should take you to the login form.

Click create new account and signup for an account

Upon creation of the account you will be redirected to the login form.
Login using the credentials you specified and you will be taken to the Listing creation form.

Fill in some details and creating a listing and the listing. You will be redirected to the home page.

You can test password change and profile edit by clicking using the dropdown menu in the upper right corner

To test password reset, logout then click the forgot password link.
Enter your email and check the console for the email containing the reset link.

Paste the reset URL in your browser and you will be prompted to enter your new password.
Enter your new password and a confirmation is displayed upon successful change.

You can then click the Log in link and enter your new password to login.

Conclusion

This article has shown how to customize Django authentication to implement the following features:

Using an email address instead of a username as the identification token.
Django admin integration for the custom user model.
User signup, login.
Password change and reset.

The Source code for this article can be found here.

How to secure a Web Application running on a private network.

Joshua Masiko — Fri, 13 Mar 2020 18:11:23 +0000

Introduction

This article describes how to secure a web app running on a private network under these constraints:

The server is not reachable via the public internet.
The server is only reachable by clients on a LAN using private IP addresses.
The internet gateway for the server may not have a Static Public IP.
We do not want to use a self-signed certificate.
The certificate generation process should be automated.
The certificate renewal process should also be automated.

Prerequisites

Ubuntu 18.04 LTS running on a server reachable over the local network.
The server should be able to connect to the internet to download the required software.
A registered domain with any of the popular registrars with a panel where you can update the Name Server entries.

In this article:
The network segment is 192.168.1.0/24 and the server IP address is 192.168.1.4
We will use the domain private.hamsterdam.me as an example.

The components in the stack are:

Ubuntu Linux 18.04 LTS will run the Webserver, Firewall and certificate generation and renewal tool.

NGINX web server will terminate HTTPS traffic. It can proxy traffic to an application running on the same server or another machine on the LAN

Let's Encrypt Certbot will automate TLS/SSL certificate generation and renewal.

Digital Ocean will handle management of the DNS zone. Certbot has a plugin for the Digital Ocean API that uses the dns-01 challenge which involves creating TXT records. The dns-01 challenge allows us to get around the http-01 challenge requirement to have a public IP reachable via the internet.

These steps were performed on a fresh installation of Ubuntu 18.04 server.

First, we install NGINX

sudo add-apt-repository universe
sudo apt update
sudo apt install -y nginx

We will use the ufw firewall to manage access to the server. The firewall is disabled by default.

sudo ufw status

Status: inactive

Let's list ufw default application profiles.

sudo ufw app list
Available applications:
  Nginx Full
  Nginx HTTP
  Nginx HTTPS
  OpenSSH

We will allow HTTP and HTTPS and SSH traffic through the firewall.
HTTP maps to port 80 (normal, unencrypted web traffic) and HTTPS to port 443 (TLS/SSL encrypted traffic).

We will also allow OpenSSH since we need to login to the server over the network.
The Nginx Full profile allows both HTTP and HTTPS traffic.

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'

Now enable the firewall

sudo ufw enable

Check the status again and confirm the firewall is active and SSH and HTTP(S) are allowed.

sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
Nginx Full                 ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
Nginx Full (v6)            ALLOW       Anywhere (v6)

We can confirm that that web server is reachable by navigating to http://192.168.1.4 from another machine on the local network.
We should see the NGINX welcome page.

The next step is to set up the NGINX web root and server block for our domain.
First create the web root folder and apply the necessary permissions.

sudo mkdir -p /var/www/private.hamsterdam.me/html
sudo chown -R $USER:$USER /var/www/private.hamsterdam.me/html/
sudo chmod -R 755 /var/www/private.hamsterdam.me/

Add an index file using your favorite editor.

vi /var/www/private.hamsterdam.me/html/index.html

Paste this content into the file and save.

<h1>Welcome to private.hamsterdam.me</h1>

Set up server block.
The ubuntu NGINX convention is to add configuration files to the sites-available folder
and then enable them by creating symbolic links in the sites-enabled folder.

sudo vi /etc/nginx/sites-available/private.hamsterdam.me

Add the following content to the file.

server {
        listen 80;
        listen [::]:80;

        root /var/www/private.hamsterdam.me/html;
        index index.html index.htm index.nginx-debian.html;

        server_name private.hamsterdam.me;

        location / {
                try_files $uri $uri/ =404;
        }
}

Now create a symbolic link.

sudo ln -s /etc/nginx/sites-available/private.hamsterdam.me /etc/nginx/sites-enabled/

Verify that the NGINX settings are ok and reload the configuration.

nginx -t
sudo nginx -s reload

Setup DNS

This step requires you to update the DNS NS records at your registrar to point to the Digital Ocean servers.

Digital Ocean provides detailed instructions at this link.

To confirm the DNS settings for our domain lets install whois.

sudo apt install -y whois

whois hamsterdam.me

The response will have multiple line lines. We are interested in lines that start with Name Server to confirm that NS records are set up correctly

Name Server: NS1.DIGITALOCEAN.COM
Name Server: NS2.DIGITALOCEAN.COM
Name Server: NS3.DIGITALOCEAN.COM

In the Digital Ocean control panel for your domain add an A record for the domain that will serve the web app.
Normally we would use the public IP of the server but here we use the private IP of the server.

After adding the record ping private.hamsterdam.me from a machine on the Local network until it resolves successfully.

ping private.hamsterdam.me

Pinging private.hamsterdam.me [192.168.1.4] with 32 bytes of data:
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.1.4:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

Now confirm that the NGINX block is correctly set up by navigatiing to http://private.hamsterdam.me
You should see the welcome page we added earlier.

Let's Encrypt Certbot

We will now install the certificate for private.hamsterdam.me using the certbot client.

Add the Certbot repository for Ubuntu.

sudo add-apt-repository ppa:certbot/certbot

Press ENTER at the prompt.

Then install Certbot’s Nginx package.

sudo apt install -y python-certbot-nginx

We will use the Digital Ocean certbot plugin to automate certificates generation and renewal.

Install the plugin.

sudo apt install -y python3-certbot-dns-digitalocean

The plugin authenticates to the Digital Ocean API using a token obtained from the Digital Ocean account’s Applications & API Tokens page.

Click Generate New Token.

Then enter a name for the token and click Generate Token.

The token is displayed in the list as a long hexadecimal value.
Make sure you copy this value as it is only displayed once.

Next, create a folder for the credentials file.

mkdir -p .secrets/certbot/

Use your favorite editor to create the credentials file.

vi .secrets/certbot/digitalocean.ini

Then add an entry to the credentials file that looks like this

dns_digitalocean_token = 0000111122223333444455556666777788889999aaaabbbbccccddddeeeeffff

The value after the equals sign should be the token you copied from the Digital Ocean panel.

Update the file permissions to prevent the file from being read by other users.

chmod 600 ~/.secrets/certbot/digitalocean.ini

Generate the certificate.

sudo certbot --authenticator dns-digitalocean --installer nginx --dns-digitalocean-credentials ~/.secrets/certbot/digitalocean.ini

When prompted, Enter you email address, Agree to the terms of service and Choose whether to share your email or not.

You will then be prompted to select the domain.

Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: private.hamsterdam.me

In this case there is only one entry so select 1 and press ENTER.

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

We choose option 2 so that unencrypted requests are redirected to the secure port.

If all goes well you will see a congratulatory message and further NOTES.

Congratulations! You have successfully enabled https://private.hamsterdam.me

If we inspect our NGINX conf we will see that certbot has made various changes to
enable HTTPS and redirect HTTP traffic to the secure port.

less /etc/nginx/sites-enabled/private.hamsterdam.me

server {

        root /var/www/private.hamsterdam.me/html;
        index index.html index.htm index.nginx-debian.html;

        server_name private.hamsterdam.me;

        location / {
                try_files $uri $uri/ =404;
        }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/private.hamsterdam.me/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/private.hamsterdam.me/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = private.hamsterdam.me) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80;
        listen [::]:80;

        server_name private.hamsterdam.me;
    return 404; # managed by Certbot


}

Now reload the NGINX conf.

sudo nginx -s reload

Now for the moment of truth.

Navigate to http://private.hamsterdam.me
The browser will be redirected to https://private.hamsterdam.me and you should see the lock icon displayed

Certbot also adds a cron entry for renewing the certificate which you can inspect.

less /etc/cron.d/certbot

How to Keep Secrets out of Django Settings

Joshua Masiko — Fri, 13 Mar 2020 18:02:45 +0000

Introduction

The Django startproject command creates a settings file that includes values that should be kept secret. These include:

SECRET_KEY is a value used when generating sessions and password reset tokens.
DATABASE_SETTINGS specifies the database connection settings and may include database credentials.

The DEBUG setting is also of interest:
DEBUG is a boolean value which determines whether Django displays detailed tracebacks are displayed when exceptions are raised.

The deployment checklist recommends that:

DEBUG be set to False to avoid leaking sensitive information from the detailed tracebacks that Django displays
Secrets and database credentials be kept out of source control.

This is also consistent with the Twelve-factor recommendation to keep Config separate from Code.

Twelve-factor is a useful set of recommendations for building web applications.

Django-environ

Django-environ is a third-party Django library that can be used to manage environment variables.
Values are read on startup instead of hardcoded in the settings file. The benefits include:

Your don't have to use multiple settings files to manage different deployments (e.g staging, production, developer).
Secrets can be kept out of source control.

It reads values from a .env file in the same folder as settings.py.

Instructions

Note: In the instructions below .env.example and .env should be saved in the same folder as settings.py

Install django-environ

pip install django-environ

Add a .env.example file with these contents:

DEBUG=on
SECRET_KEY=your-secret-key
DATABASE_URL=psql://urser:urpassword@127.0.0.1:8458/database

This file is an example for collaborators to follow when creating the .env file for their local environment.

Add .env to your .gitignore file to prevent it from being added to the repository

At the top of your settings.py add the following:

import environ
env = environ.Env(
    # set casting, default value
    DEBUG=(bool, False)
)
# reading .env file
environ.Env.read_env()

Change the DEBUG, SECRET_KEY and DATABASES lines in settings.py to read their values from the environment.

DEBUG = env('DEBUG')
...
SECRET_KEY = env('SECRET_KEY')
...

DATABASES = {
    'default': env.db(),
}

Your can then add .env.example and the the modified settings file to the Git repository

git add .

git commit -m "Extracted secrets from the settings file"

For each deployment (staging, production, developer environment) you then:

Create a .env file with the settings specific to that environment. These include:

Your desired DEBUG setting.
The secret key.
Your DATABASE connection string and credentials.

Django has a utility function that the startproject command uses to generate the secret
You can use it from the command line.
With your virtual environment activated run this command:

python -c 'from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())'

It will spit out a random string like:

a1$%jr29f2(u^l=r1q1a2$sztg%x7%g8s@!ne#_(^5$woi#wi$

Assuming your DATABASE setting was

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'blog',
        'USER': 'dbuser',
        'PASSWORD': 'dbpassword',
        'HOST': '127.0.0.1',
        'PORT': '5432',
    }
}

Your .env file would look like

DEBUG=on
SECRET_KEY=a1$%jr29f2(u^l=r1q1a2$sztg%x7%g8s@!ne#_(^5$woi#wi$
DATABASE_URL=psql://dbuser:dbpassword@127.0.0.1:5432/blog

Note that there are not quotes around the values in the .env file

For your production deployment change DEBUG=on to DEBUG=off

You can confirm that your app starts up successfully by running the Django Dev Server

python manage runserver

If there are any errors consult the Django-environ documentation for guidance.

How to serve private media files with Django

Joshua Masiko — Fri, 13 Mar 2020 17:35:56 +0000

In this tutorial, we will create a barebones document manager with these features:

Logged in users can upload files.
Superusers can view all files.
Regular users can view only the files they uploaded.

These steps were performed on Ubuntu 18.04 LTS.

Getting Started

First lets add the universe repository and install python3-venv.

sudo add-apt-repository universe
sudo apt install -y python3-venv

The Project and App

Now create the project.

mkdir ~/.virtualenvs
python3 -m venv ~/.virtualenvs/docman
source ~/.virtualenvs/docman/bin/activate
pip install django
django-admin startproject docman
cd docman

Now create the database and tables by running migrate.

./manage.py migrate

After the migration process completes create an app

./manage.py startapp core

The model

Each document is associated with the user.
We also track the date and time that the document was uploaded.
By default documents will be sorted in reverse chronological order.

# core/models.py

from django.db import models
from django.contrib.auth.models import User
from django.utils import timezone
from django.template.defaultfilters import slugify


class Document(models.Model):
    title = models.CharField(max_length=200)
    description = models.TextField()
    file = models.FileField()
    created_by = models.ForeignKey(User, on_delete=models.PROTECT)
    created_at = models.DateTimeField(default=timezone.now)
    slug = models.SlugField(max_length=255, editable=False)

    class Meta:
        ordering = ['-created_at']

    def save(self, *args, **kwargs):
        if not self.id:
            self.slug = slugify(self.title)

        return super(Document, self).save(*args, **kwargs)

Now add the app to the INSTALLED_APPS list in settings.py

# docman/settings.py

INSTALLED_APPS = [
...
    'core.apps.CoreConfig',
]

Generate migrations for the model.

./manage makemigrations core

Then update the database to add the table for the model.

./manage migrate

The views

The application functionality consists of 3 views:

The default view displays a list of documents.
A view to upload a document.
A view to download a document.

All views are restricted to logged-in users.

# core/views.py

import os

from django.shortcuts import render, get_object_or_404
from django.contrib.auth.mixins import LoginRequiredMixin
from django.views.generic import ListView
from django.views.generic.edit import CreateView
from django.http import FileResponse, HttpResponseForbidden, HttpResponse
from django.views import View
from django.urls import reverse
from django.conf import settings

from .models import Document


class DocumentList(LoginRequiredMixin, ListView):
    model = Document

    def get_queryset(self):
        queryset = Document.objects.all()
        user = self.request.user

        if not user.is_superuser:
            queryset = queryset.filter(
                created_by=user
            )

        return queryset


class DocumentCreate(LoginRequiredMixin, CreateView):
    model = Document
    fields = ['title', 'description', 'file']

    def get_success_url(self):
        return reverse('home')

    def form_valid(self, form):
        form.instance.created_by = self.request.user
        return super().form_valid(form)


class DocumentDownload(View):
    def get(self, request, relative_path):
        document = get_object_or_404(Document, file=relative_path)
        if not request.user.is_superuser and document.created_by != request.user:
            return HttpResponseForbidden()
        absolute_path = '{}/{}'.format(settings.MEDIA_ROOT, relative_path)
        response = FileResponse(open(absolute_path, 'rb'), as_attachment=True)
        return response

The URLs

Lets wire up the urls for the views.

# docman/urls.py

from django.contrib import admin
from django.urls import path
from django.contrib.auth import views as auth_views

from core.views import DocumentList, DocumentCreate, DocumentDownload

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', DocumentList.as_view(), name='home'),
    path('document-add/', DocumentCreate.as_view(), name='document-add'),
    path('media/<path:relative_path>', DocumentDownload.as_view(), name='document-download'),

    path('accounts/login/', auth_views.LoginView.as_view()),
    path('accounts/logout/', auth_views.LogoutView.as_view(), name='logout'),
]

The templates

Create a login template.

# core/templates/registration/login.html

<form method="POST">
    {% csrf_token %}
    {{form.as_p}}
    <button type="submit">Login</button>
</form>

Create a base template.

# core/templates/base.html

<h1>Document Manager</h1>
<p>Logged in as {{user.get_full_name}}</p>
<p><a href="{% url "logout" %}">Logout</a></p>
{% block content %}
{% endblock %}

The document list displays documents in reverse chronological order.

# core/templates/core/document_list.html

{% extends "base.html" %}

{% block content %}
    <h2>Documents</h2>
    <a href="{% url "document-add" %}">Add Document</a>
    <table width="50%">
        <thead>
            <tr>
                <th>Title</th>
                <th>Description</th>
                <th>Created by</th>
                <th>Created at</th>
                <th>File</th>
            </tr>
        </thead>
        {% for document in object_list %}
            <tr>
            <td>{{ document.title }}</td>
            <td>{{document.description}}</td>
            <td>{{document.created_by}}</td>
            <td>{{document.created_at}}</td>
            <td><a href="{{document.file.url}}">{{document.file.name}}</a></td>
            </tr>
        {% endfor %}
    </table>
{% endblock %}

Files are downloaded by clicking on the filename link.
The Add Document button links to the document upload page.

# core/templates/core/document_form.html

{% extends 'base.html' %}
{% block content %}
<h1>Add Document</h1>
<form method="POST" enctype="multipart/form-data">
    {% csrf_token %}
    {{form.as_p}}
    <button type="submit">Submit</button>
</form>
{% endblock %}

Django stores uploaded files on the local file system using paths relative to MEDIA_ROOT
Lets define media root in settings.py

# docman/settings.py

MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

Also add a line at the bottom of settings.py that sets the URL to redirect to when a user logs out

# docman/settings.py

LOGOUT_REDIRECT_URL = '/'

The users

We will create two users:

A superuser who can view all uploaded documents.
A regular user who can view only the documents they uploaded.

We will use the django-createuser app to add a convenient management command for creating users.

Install django-createuser

pip install django-createuser

Add django_createuser to your installed apps in settings.py

# docman/settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'core.apps.CoreConfig',
    'django_createuser',  #new
]

Now create the superuser.

python manage.py createuser --email admin@example.com --first_name Administrator --is-superuser --password test1234 admin

Then the regular user.

python manage.py createuser --email user@example.com --first_name Test --last_name User --password test1234 user

Run the application

We will use the Django the dev server.

python manage.py runserver

Navigate to http://127.0.0.1:8000 in your browser.

You will get prompted to login using username and password.

Click the Add Document link and upload a document

Then:
Logout from the superuser account.
Login as the regular user using the username user and password test1234.
The document uploaded by the superuser will not be visible in the list.
Upload a document as the regular user.

If you login as the superuser both documents are displayed in the list.
You should be able to download both documents as the superuser by clicking on the links.

Testing access control

Copy the both document links and paste them in a text editor

Test access for logged out users

Logout and try downloading the files using the links you copied.
Access will be denied

Test access to superuser file for regular user.

Next Steps

The setup described in this article is not recommended for production.
Serving media files is best handled by a web server like NGINX or a CDN backed Object Storage service like AWS S3.

A future article will evolve our document manager to use NGINX and Object Storage.

DEV Community: Joshua Masiko

How to deploy a Django app to Google Cloud Run using Terraform

Introduction

Goole Cloud Run

Infrastructure as Code

Google Cloud Components

Step 0 - Installing the supporting tools

Step 1 - Creating a Google Cloud project

Step 2 - Creating a starter Django project

Step 3 - Running the application locally

Step 4 - Deploying the app to Cloud Run using Terraform

Step 5 - Logging into the application

Step 6 - Disabling DEBUG mode

Step 7 - Cleaning Up

Google Cloud Run Service-to-Service Authentication using Go

Prerequisites

Creating a new project

Enable billing

Install and configure the Google Cloud SDK (gcloud)

Enable the required APIs

Deploy sending and receiving services

Receiving Service

Sending Service

Update the receiving service

Update the sending service

Rebuild and redeploy the sending service:

Test the authentication

Create a new service account for the calling service

Redeploy the calling service with the new service account

Grant the calling service identity permission to invoke the receiving service

Step 4: Test the authentication

Conclusion

Customizing Django Authentication using AbstractBaseUser

Introduction

Prerequisites

High Level Description of the App

Getting Started

Project Setup

The accounts app

The model and Manager

The forms

The views

The admin

Testing the app so far

The core app

Install django-crispy-forms

The models

The admin

The views

The URLs

The templates

auth templates

core templates

Testing the app

Conclusion

How to secure a Web Application running on a private network.

Introduction

Prerequisites

The components in the stack are:

First, we install NGINX

Setup DNS

Let's Encrypt Certbot

How to Keep Secrets out of Django Settings

Introduction

Django-environ

Instructions

How to serve private media files with Django

Getting Started

The Project and App

The model

The views

The URLs

The templates

The users

Run the application

Testing access control

Test access for logged out users

Test access to superuser file for regular user.

Next Steps

`auth` templates

`core` templates