Memcached for Django with docker

Why Memcached?

Are you tired of fighting with the performance of your application? Do you want to improve how you consume your resources? If so, applying a caching solution will help you, and Memcached is ideal for that.

While there are other well spread caching solutions, like Redis or Varnish, Memcached combines the ability to handle the cache programmatically, and a native support from the Django framework (where it is defined as “The fastest, most efficient type of cache supported natively by Django”).

What we will do?

For testing better the performance that Memcached can provide to your applications, you are going to build a middleware that calls four external APIs. Those calls are consolidated and presented in an endpoint of your application.

I will try to keep this article updated, but if for any reason the links don’t work, you can use any other external open API of your choice. In this GitHub repository, you can find several open APIs.

The code

I assume you have a basic knowledge of Django. Also that you have at least a basic installation of Django ready. If not, you can visit the Django official tutorial or clone an example from this repository (note that the necessary migrations are already done).

Preparing your docker-compose

First, create a Dockerfile at the root of your project.

FROM python:3
RUN mkdir /python
WORKDIR /python
COPY requirements.txt /python/
RUN pip install -r requirements.txt
COPY . .

What you are doing here is creating a docker container with the last version of Python 3, and loading the packages listed in the requirements file.
Speaking about the requirements, let’s create a requirements.txt file, also in the root of your project with the following lines:

python-memcached>=1.3
Django>=2.0,<3.0

You are just adding Django and the needed libraries for Memcached. If your project is bigger and needs other packages, add them after these two lines.
Next, let’s create the docker-compose.yml file. You will include the configuration for Django and Memcached.

version: '3'
services:
  api:
    build: .
    container_name: django-memchached
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/python
    ports:
      - "8000:8000"
    depends_on:
      - cache
  cache:
   image: memcached
   ports:
     - "11211:11211"
   entrypoint:
    - memcached
    - -m 64
Django settings
You have to add the following configuration lines for Memcached to the settings.py file of your Django app.
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': 'cache:11211',
    }
}

The main detail of this configuration that you have to be aware of is the entry ‘LOCATION’. It is composed of a direction and a port. Here the direction value is cache because it was named like that before in the docker-compose file (take a look at it). If you change that value in the Docker configuration, remember to change it here also. Usually, this entry has a DNS or an IP as a value.

Here is magic!

First, create a connector.py file in your app folder.

import http.client
import json
class Connector(object):
    ENDPOINTS = [
        '/awardees.json',
        '/newspapers.json',
        '/lccn/sn86069873/1900-01-05/ed-1/seq-2.json',
        '/lccn/sn86069873/1900-01-05/ed-1/seq-3.json'
    ]
    DOMAIN = 'chroniclingamerica.loc.gov'
def __init__(self, endpoints = ENDPOINTS):
        self.endpoints = endpoints
def call_endpoints(self):
        response = []
        for endpoint in self.endpoints:
            response.append(self.request(endpoint))
        return response
def request(self, endpoint):
        headers = {
            'content-type': 'application/json',
            'Cache-Control': 'no-cache',
        }
        connection = http.client.HTTPSConnection(self.DOMAIN)
        connection.request('GET', endpoint, '', headers)
        response = connection.getresponse()
        json_string = response.read().decode('utf-8')
        data = json.loads(json_string)
        connection.close()
        return data

There are three functions in the class:
init (self, endpoints = ENDPOINTS):
The initialization of the class. Assigns to self.endpoints the value provided during the creation of the instance, or takes by default the value of the constant ENDPOINTS.
call_endpoints(self):
Iterates on the self.endpoints variable, calls the request function and aggregates all the responses of the calls.
request(self, endpoint):
It makes an HTTP request to the endpoint provided.
Furthermore, you have the constants DOMAIN and ENDPOINTS. You can play with your preferred APIs changing these values.
After the connector, you have to create the view. Add the following to the views.py file of your app:

from django.shortcuts import render
from django.http import HttpResponse
from django.views import View
from django.core.cache import cache
import json
from thing.connector import Connector
class Middleware(View):
    def get(self, request):
        cache_key = 'sample_key' #identificative value
        cache_time = 7200 #time in seconds of the cache
        data = cache.get(cache_key)
        if not data:
            connector = Connector().call_endpoints()
            data = json.dumps(connector)
            cache.set(cache_key, data, cache_time)
        response = HttpResponse(data)
        response ['content-type'] = 'application/json'
        response ['Cache-Control'] =  'no-cache'
        return response

Here is where you apply the caching. The workflow is quite easy. The code checks if the data is cached for the key provided. If there is data in Memcached, then it gives that data. If not, it calls the Connector for taking the data and then cache it adding a key and the time that the resource should be stored.
The next step is to create a urls.py also in the folder of your app.

from django.urls import path
from . import views
from thing.views import Middleware
urlpatterns = [
    path('chroniclingamerica', Middleware.as_view(), name='middleware_endpoint'),
]

Note that in the import path of the view, you must substitute “thing” by the name of your app (if you have cloned the example repository, you don’t need to change it).
Then, you have to update the urls.py file of your project. Add there the path to the URLs file of your app (the line in bold below).

from django.contrib import admin
from django.urls import include, path
urlpatterns = [
    path('middleware/v1/', include('thing.urls')),
    path('admin/', admin.site.urls),
]

Let’s run it!!!

If you have followed all the instructions, now you have everything ready. Let’s up and build the docker containers. Execute the following command in the root of your Django App:
docker-compose up --build -d
With the webserver up and running, it’s the moment to check the response time of our endpoint. Go to the terminal and run this curl:

curl -I -X GET  http://127.0.0.1:8000/middleware/v1/chroniclingamerica -w %{time_total}\\n

You will obtain a time between 400 and 500 milliseconds. Now, repeat the command. You will have a response in around 15 milliseconds. Awesome, right!
As you can see, the improvement in performance is remarkable. While in this example, we have caught the result of calls to a third party, you can use it for any element: responses from the database, inner processes, calls to your other microservices, etc.
You can find all the code exposed in this article in this Github repository.