DEV Community: Charlesu49

HTTPS: A Conversation between the Client and Server

Charlesu49 — Tue, 28 Mar 2023 21:20:38 +0000

Act I Scene I

Client:

Hi, good morning.

Server:

Good morning, how can I help you today?

Client:

I have a question about website security. Why is it that some websites start with "https://" while others start with just "http://"?

Server:

That's a great question! Website security is incredibly important in today's digital age, and understanding the difference between "http://" and "https://" is a crucial step in protecting your personal information online. Allow me to explain.

Client:

Please do.

Server:

When you connect to a website using the "http://" protocol, all data that is transmitted between your browser and the server is sent in plain text. This means that anyone who is able to intercept the data, such as a hacker or someone on the same network as you, can read it. This can include sensitive information like passwords, credit card numbers, and other personal details.

Client:

That's worrying.

Server:

It is! That's why websites that handle sensitive information, such as banks or e-commerce sites, use the "https://" protocol instead. This protocol uses a secure connection called SSL/TLS to encrypt all data transmitted between your browser and the server, making it much more difficult for anyone to intercept and read your data.

Client:

I see. How does SSL/TLS work?

Server:

Well, SSL/TLS works by establishing a secure connection between your browser and the server. When you connect to a website using SSL/TLS, your browser and the server engage in a process called a handshake, which allows them to exchange information and establish a secure connection.

Client:

Okay, how does that work?

Server:

During the handshake, we first agree on a protocol version to use for the connection. Then, I send you my public key, which is included in the digital certificate.

Client:

What's a digital certificate?

Server:

A digital certificate is an electronic document that contains information about the website's identity and public key. It is issued by a trusted third-party organization called a Certificate Authority (CA).

Client:

And how does my browser know which CA to trust?

Server:

Your browser has a list of trusted CAs built in. When it receives the certificate from our server, it checks to make sure that it was issued by a trusted CA and that it has not expired or been revoked.

Client:

I see. And then what happens?

Server:

Once your browser has verified the certificate, you use my public key to encrypt a small piece of data that I can decrypt with my private key. This is called the client key exchange, and it allows us to securely exchange a session key that we will use for the rest of the connection. The session key is generated using a process called key exchange, which uses some randomness from both of us to derive a shared secret. From that point on, all data sent between your browser and our server is encrypted using the session key.

Client:

That's fascinating. So using SSL/TLS is really important for protecting sensitive information online?

Server:

Absolutely!

Encrypting a Django Rest Framework Response

Charlesu49 — Mon, 27 Mar 2023 19:51:06 +0000

Introduction

Encryption is the process data goes through to get transformed from a readable format(plaintext) to an unreadable format(ciphertext). After encryption, the ciphertext appears to be unreadable random data and anyone attempting to read the original data would require an encryption/decryption key. The purpose of encryption is to help protect sensitive information from unauthorized entities and to also enhance secure communication between two entities. In this article, we will talk about encryption and how we can handle response encryption in Django Rest Framework(DRF) with a detailed step on how to achieve this.

Why encrypting DRF responses with AES is important?

Encrypting responses is important to protect sensitive data from being accessed or intercepted by unauthorized parties. By encrypting the response with a strong encryption algorithm like AES, the data can only be accessed by authorized parties with the correct key.

AES Encryption

Advanced Encryption Standard (AES) is a specification for data encryption by the National Institute of Standards and Technology (NIST). AES uses a block cipher that works on fixed block sizes of 128 bits and has three key sizes: 128 bits, 192 bits, and 256 bits. The larger the key size, the stronger the encryption. AES encryption works by repeatedly applying a complex set of mathematical operations to the plaintext data, using the encryption key as input. This process, called rounds, converts the plaintext into ciphertext.

Setting Up AES Encryption in DRF

Now I am going to walk you through setting up AES encryption in DRF and ensuring that all responses from your DRF APIs are encrypted.

Let us assume we work for the Secret Service and we need to build an API that fetches information about our agents and returns this information as a response, this is sensitive information so we need to encrypt this to ensure that even if the transmitted data is intercepted it will make no sense to the unauthorized party. We will build the project from scratch, create an endpoint that returns a list of agents details in plaintext format, create another endpoint that returns the data as ciphertext and another on that accepts the ciphertext and decrypts it to plaintext. You can follow along with the following outlined steps and note that a repository of the completed project can be found at this github repository.

Code Along

Create a new project directory mkdir encryption_project.
Change the directory into the newly created directory cd encryption_project.
Create a virtual environment virtualenv venv.
Activate the virtual environment source venv/bin/activate.
Download Django, DRF, Pycryptodome(which provides the AES encryption class) and python-dotenv by running the following command pip install django && pip install djangorestframework && pip install pycryptodome && pip install python-dotenv.
Add this to the requirements.txt file by running pip freeze > requirements.txt.
Start a new Django project django-admin startproject secret_service.
Create a new Django app django-admin startapp agents.
Navigate to your settings.py file under the secret_service project directory and add the following line to register the newly created app(agents) and DRF;
```
# secret_service/settings.py
INSTALLED_APPS = [
    ...
    # newly added below
    'rest_framework',
    'agents'
]
```

Navigate to the models.py file in the agents app directory and create a model for the agent's information;

# agents/models.py

from django.db import models

class Agent(models.Model):
    first_name = models.CharField(max_length=100)
    last_name = models.CharField(max_length=100)
    badge_no = models.CharField(max_length=10, unique=True)
    code_name = models.CharField(max_length=100)
    location = models.CharField(max_length=100)
    secret_mission = models.TextField()
    active = models.BooleanField()
    no_of_assignments = models.IntegerField()

Create a serializer for the Agent model, create a file named serializers.py under the agents app directory and add the below code;
```
# agents/serializers.py
from rest_framework import serializers
from .models import Agent

class AgentSerializer(serializers.ModelSerializer):
    class Meta:
        model = Agent
        fields = ('first_name', 'last_name', 'badge_no', 'code_name', 'location', 'secret_mission')
```
A serializer is a tool in Django that converts data from your application into a format that can be easily shared with other applications, usually in JSON format. In this example, the AgentSerializer is used to convert Agent model instances into JSON format so that they can be returned as API responses(note that we will be encrypting this json before returning it). The serializer defines which fields from the model should be included in the JSON output.
Set up the Agent model to be managed through the Django admin site.
```
# agents/admin.py
from django.contrib import admin
from .models import Agent

class AgentAdmin(admin.ModelAdmin):
    list_display = ("first_name", "last_name", "code_name", "location", "secret_mission")

admin.site.register(Agent, AgentAdmin)
```
The AgentAdmin class customizes the appearance of the Agent model in the admin site by defining which fields to display in the list view (list_display). The admin.site.register(Agent, AgentAdmin) line registers the Agent model with the admin site and applies the customizations defined in the AgentAdmin class.
At this point, we can make migrations by running python manage.py makemigrations on the terminal to create the table on our database. Note that I am using the default Sqlite DB so this command automatically creates the database if you are using a non-file based database like Postgres/Oracle DB for instance you will have to create your database and manually configure the connection to it in your settings.py file in the project directory. Running this command should generate a response like the one below confirming that a new migrations file has been created;
Now we run python manage.py migrate to apply the created migrations, we should have a file like the below if it runs successfully;
We create a superuser to enable us to access the admin dashboard and add some dummy data to our agents table by running python manage.py createsuperuser and populating our data as below;
We can start our development server by running python manage.py runserver.
Navigate to your admin dashboard via the URL http://localhost/admin and sign in with the credentials created earlier. You should see the Agents models and a click on '+Add' should bring up a page that looks like the one below;

Populate this page to maintain some dummy agents' data.

Let us create a view that returns the agents information in plain text

In views.py file in the agents directory, add the below codeagents/views.py

# agents/views.py
from rest_framework.views import APIView 
from rest_framework.response import Response 
from rest_framework import status 
from .serializers import AgentSerializer 
from .models import Agent 

class AgentListPlainView(APIView):
    def get(self, request):
        agents = Agent.objects.all()
        if agents:
            serializer = AgentSerializer(agents, many=True)
            data = serializer.data
        data = {
            'status': 'success',
            'code' : status.HTTP_200_OK,
            'data': data
            }
        return Response(data, status=status.HTTP_200_OK)

In DRF, the default renderer is JSONRenderer. This renderer is responsible for converting Python objects to JSON format before sending the response to the client. It is used when no other renderer is specified or when the Accept header of the request explicitly specifies application/json as the expected response format and because we wish to return this particular response as plaintext we will let the default renderer handle this.

Next, we register our app's URL routes in the urls.py file in the project directory by adding the below code;

# secret_service/urls.py

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),

    # newly added line below
    path('', include('agents.urls'))
]

Next, we proceed to our agents directory and add a new file named urls.py

Navigate to the newly created urls.py file and add the below lines of code to register our endpoint for fetching agents' lists as plain text.

# agents/urls.py

from django.urls import path
from . import views

urlpatterns = [
    path('agent-list-plain/', views.AgentListPlainView.as_view(), name='agent-list-plain'),
]

At this point, we can use postman, insomnia or curl to test our API using the URL http://localhost:8000/agent-list-plain/

This is returned as plaintext because we are making use of DRF's default renderer in our view, while this may serve in many cases where encryption is not required, that is not our case here as we need to ensure the returned data is encrypted.
We can now create an endpoint that overrides DRF's default JSONRenderer with a custom renderer we will build following the below steps;
In the root of your project add a .env file and add the lines;
```
# .env
AES_IV=whsbdhgntkgngmhk
AES_SECRET_KEY=somerandomsecret
```
Ensure that the key used is 16, 24 or 32 bytes long (respectively for AES-128, AES-192 or AES-256) with AES-256 being the most secure since it employs more rounds resulting in more complex encryption than the others.
In the agents directory create a new file renderers.py and add the below;
```
# agents/renderers.py

import base64
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad
from rest_framework.renderers import BaseRenderer
import json

load_dotenv()

# here the string gotten from the environmental variable is converted to bytes
AES_SECRET_KEY = bytes(os.getenv('AES_SECRET_KEY'), 'utf-8')
AES_IV = bytes(os.getenv('AES_IV'), 'utf-8')

class CustomAesRenderer(BaseRenderer):
    media_type = 'application/octet-stream'
    format = 'aes'

    def render(self, data, media_type=None, renderer_context=None):
        plaintext = json.dumps(data)
        padded_plaintext = pad(plaintext.encode(), 16)
        cipher = AES.new(AES_SECRET_KEY, AES.MODE_CBC, AES_IV)
        ciphertext = cipher.encrypt(padded_plaintext)
        ciphertext_b64 = base64.b64encode(ciphertext).decode()
        response = {'ciphertext': ciphertext_b64}
        return json.dumps(response)
```
Here we define a custom renderer for DRF that can be used to encrypt responses using the AES encryption algorithm. The CustomAesRenderer class is defined as a subclass of DRF's BaseRenderer class. It sets the media_type attribute to application/octet-stream and the format attribute to aes, indicating that this renderer will be used to render responses in binary format using AES encryption.

The render() method is where the actual encryption takes place. It takes the data that needs to be encrypted as an argument, which is expected to be in JSON format. The data is first serialized to JSON format using the json.dumps() method. The serialized data is then padded with additional bytes using the pad() function from the Crypto.Util.Padding module. The data is then encrypted using AES with a secret key and initialization vector specified by AES_SECRET_KEY and AES_IV respectively. Finally, the encrypted data is encoded in base64 format and returned as the encrypted response.

This custom renderer can be used in DRF views by specifying the renderer_classes attribute and including the CustomAesRenderer as we will see next.

Next, navigate to views.py in the agents directory and add the following;

# agents/views.py

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from .renderers import CustomAesRenderer # newly added
from .serializers import AgentSerializer
from .models import Agent

class AgentListPlainView(APIView):
    def get(self, request):
        agents = Agent.objects.all()
        if agents:
            serializer = AgentSerializer(agents, many=True)
            data = serializer.data
        data = {
            'status': 'success',
            'code' : status.HTTP_200_OK,
            'data': data
            }
        return Response(data, status=status.HTTP_200_OK)

# newly added
class AgentListEncryptedView(APIView):
    renderer_classes = [CustomAesRenderer]

    def get(self, request):
        agents = Agent.objects.all()
        if agents:
            serializer = AgentSerializer(agents, many=True)
            data = serializer.data
        data = {
            'status': 'success',
            'code' : status.HTTP_200_OK,
            'data': data
            }
        return Response(data, status=status.HTTP_200_OK)

The AgentListEncryptedView performs a similar function as AgentListPlainView except that the response is routed through the renderer class CustomAesRenderer which handles the encryption before returning the ciphertext to the client.

Add this endpoint to the URL routes by navigating to the urls.py file in the agents directory and add the below;

# agents/urls.py

from django.urls import path
from . import views

urlpatterns = [
    path('agent-list-plain/', views.AgentListPlainView.as_view(), name='agent-list-plain'),

# newly added below
    path('agent-list-encrypted/', views.AgentListEncryptedView.as_view(), name='agent-list-encrypted'),

Testing the endpoint using the URL http://localhost:8000/agent-list-encrypted/ we get the below;

We have now completed the endpoint that returns the data as plaintext and one that returns the data as ciphertext, next, we will implement an endpoint that takes the ciphertext and decrypts it using our KEY and Initialization Variable(IV) by following the below steps;

Navigate to the views.py file in the agents directory and add the below code;


# agents/views.py

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from .renderers import CustomAesRenderer
from .serializers import AgentSerializer
from .models import Agent

# newly added below

import base64
from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
from .renderers import AES_SECRET_KEY, AES_IV
import json

class AgentListPlainView(APIView):
    def get(self, request):
        agents = Agent.objects.all()
        if agents:
            serializer = AgentSerializer(agents, many=True)
            data = serializer.data
        data = {
            'status': 'success',
            'code' : status.HTTP_200_OK,
            'data': data
            }
        return Response(data, status=status.HTTP_200_OK)

class AgentListEncryptedView(APIView):
    renderer_classes = [CustomAesRenderer]

    def get(self, request):
        agents = Agent.objects.all()
        if agents:
            serializer = AgentSerializer(agents, many=True)
            data = serializer.data
        data = {
            'status': 'success',
            'code' : status.HTTP_200_OK,
            'data': data
            }
        return Response(data, status=status.HTTP_200_OK)

# newly added below

class DecryptAgentList(APIView):
    def post(self, request, *args, **kwargs):
        # Decode the request data from base64
        encrypted_data = request.data['ciphertext']
        enc = base64.b64decode(encrypted_data)
        cipher = AES.new(AES_SECRET_KEY, AES.MODE_CBC, AES_IV)
        try:
            decrypted_data = unpad(cipher.decrypt(enc),16)
            decrypted_data = json.loads(decrypted_data)
            data = {
                "data" : decrypted_data
            }
            return Response(data)
        except Exception as e:
            return Response({"data": f"An error- {e}"})

The DecryptAgentList class is a subclass of the DRF APIView class and defines a post method that handles HTTP POST requests.

Inside the post method, the encrypted data is extracted from the request body in base64 format and then decoded back to binary format using base64.b64decode(). Next, an AES cipher object is created using the same secret key and initialization vector (IV) that were used to encrypt the data. The unpad() function from the Crypto.Util.Padding module is used to remove any padding that was added to the data during encryption. The decrypted data is then loaded from JSON format using the json.loads() method and returned in a structured response that includes the decrypted response as a value of a key called data.

Finally, the plaintext response is returned using the DRF Response class if decryption is successful else an error message is returned.

Register the DecryptAgentList endpoint on the URL routes by adding the below code to the urls.py of the agents directory;


# agents/urls.py

from django.urls import path
from . import views

urlpatterns = [
    path('agent-list-plain/', views.AgentListPlainView.as_view(), name='agent-list-plain'),
    path('agent-list-encrypted/', views.AgentListEncryptedView.as_view(), name='agent-list-encrypted'),

# newly added

    path('decrypt-agent-list/', views.DecryptAgentList.as_view(), name='decrypt-agent-list')
]

Proceed to test this endpoint

Conclusion

We have successfully implemented AES encryption in Django Rest Framework to secure our API responses. We have discussed the importance of data encryption and the benefits it provides in securing sensitive data. By utilizing the AES encryption algorithm, we have ensured that our data is protected from unauthorized access, making our application more secure. With these techniques, we can confidently deploy our Django Rest Framework application, knowing that our data is protected from prying eyes.