DEV Community: Andrea Bertoloni

Define and serve an HTML page using a single Dockerfile

Andrea Bertoloni — Wed, 02 Apr 2025 17:49:18 +0000

This is a quick and easy way to write an HTML page directly in the same Dockerfile where you define the Nginx server.

Use case:
I needed a very basic placeholder page for a project before it was actually put online.
Since I use Coolify, which allows you to use a Dockerfile as an application's source, I could easily achieve the goal by simply editing the file directly within the dashboard and hitting "Deploy".

Below is the complete Dockerfile setup:

FROM nginx:alpine

# Create the HTML file
RUN echo '<!DOCTYPE html>\
<html>\
<head>\
<title>My new project</title>\
</head>\
<body>\
<p>Lorem ipsum</p>\
</body>\
</html>' > /usr/share/nginx/html/index.html

# Expose port 80
EXPOSE 80

# Run nginx in foreground
CMD ["nginx", "-g", "daemon off;"]

Python: upload multiple files concurrently with aiohttp and show progress bars with tqdm

Andrea Bertoloni — Mon, 15 Nov 2021 12:55:31 +0000

I had to quickly implement this functionality in a command line tool and tried to look for some online ready-to-use example.
Since I wasn't able to find any, I came up with this solution (perhaps someone can find it helpful).

import os
import asyncio
import aiohttp
import aiofiles
from tqdm import tqdm


class FileManager():
    def __init__(self, file_name: str):
        self.name = file_name
        self.size = os.path.getsize(self.name)
        self.pbar = None

    def __init_pbar(self):
        self.pbar = tqdm(
            total=self.size,
            desc=self.name,
            unit='B',
            unit_scale=True,
            unit_divisor=1024,
            leave=True)

    async def file_reader(self):
        self.__init_pbar()
        chunk_size = 64*1024
        async with aiofiles.open(self.name, 'rb') as f:
            chunk = await f.read(chunk_size)
            while chunk:
                self.pbar.update(chunk_size)
                yield chunk
                chunk = await f.read(chunk_size)
            self.pbar.close()


async def upload(file: FileManager, url: str, session: aiohttp.ClientSession):
    try:
        async with session.post(url, data=file.file_reader()) as res:
            # NB: if you also need the response content, you have to await it
            return res
    except Exception as e:
        # handle error(s) according to your needs
        print(e)


async def main(files):
    url = 'https://httpbin.org/post'
    files = [FileManager(file) for file in files]

    async with aiohttp.ClientSession() as session:
        res = await asyncio.gather(*[upload(file, url, session) for file in files])

    print(f'All files have been uploaded ({len(res)})')


if __name__ == "__main__":
    # In a real application you may may want to handle files
    # in a more robust way, do some validation etc.
    files = ['./file_1', './file_2', './file_3']

    asyncio.run(main(files))

Obfuscate Django models ids by encoding them as non-sequential non-predictable strings

Andrea Bertoloni — Mon, 15 Jun 2020 17:53:34 +0000

TL;DR
Replace urls like example.com/users/1/ with something like example.com/users/zpA1nRdJbwG/ without changing the actual ids in your db.
(Also applicable to stuff other than urls, e.g. replace api responses like { "id": 1, "email": "user@example.com" } with something like { "id": "zpA1nRdJbwG", "email": "user@example.com" })

Sometimes, whatever the reasons, you may want to change the ids of your models, which usually are incremental integers, with some non-sequential non-predictable codes instead.
So, instead of having - for example - an url like example.com/users/1/, you would have something like example.com/users/zpA1nRdJbwG/.

A common solution for this is to use UUIDs as the primary keys for your models. Similarly, you could generate some other [random] unique strings when you create an object in the db.
The problem here is that we're going to lose all the benefits that come from using incremental integers as primary keys.

So, a solution might be to encode/decode the ids in the application layer, without touching the actual incremental numeric ids in the database.
To do so we're going to integrate Hashids, a library "that generates short, unique, non-sequential ids from numbers", into Django.

Install Hashids with

$ pip install hashids
# or pipenv install hashids
# or poetry add hashids

URLs

The first step is to register a custom path converter, so that we can then define a url as path('users/<hashids:user_id>/', ...) that will automatically pass the decoded integer id in the related view.

Create a folder named ids_encoder at the same level as any other Django app, with two empty files inside: __init__.py converters.py.

# inside your project root directory
$ mkdir ids_encoder
$ cd ids_encoder
$ touch __init__.py converters.py

You should end up with something like this

mysite/
|-- db.sqlite3
|-- ids_encoder
|   |-- __init__.py
|   `-- converters.py
|-- manage.py
|-- mysite
|   |-- __init__.py
|   |-- settings.py
|   |-- urls.py
|   `-- wsgi.py
`-- ...

In ids_encoder/converters.py, paste this code

class HashidsConverter():
    regex = '[0-9a-zA-Z]+'

    def to_python(self, value: str) -> int:
        hashids = Hashids()
        decoded_values = hashids.decode(value)
        # output of hashids.decode is always a tuple
        if len(decoded_values) != 1:
            raise ValueError
        return decoded_values[0]

    def to_url(self, value: int) -> str:
        hashids = Hashids()
        return hashids.encode(value)

This class has the shape required for a custom path converter:

a regex (as a string) that matches the parameter passed in the url
a method to_python which converts the matched string into the type that should be passed to the view function
a method to_url which converts the Python type into a string to be used in the URL

We now need to register the custom converter, so go in mysite/urls.py

from django.urls import register_converter

from ids_encoder import converters

register_converter(converters.HashidsConverter, 'hashids')

urlpatterns = []

To test that everything works, we just write a simple view that returns the exact params the it receives (remind: the view itself receives the already decoded value).

In mysite/urls.py

from django.urls import path, register_converter
from django.http import HttpResponse

from ids_encoder import converters

register_converter(converters.HashidsConverter, 'hashids')

def test_user_id(request, user_id):
    return HttpResponse(user_id)

urlpatterns = [
    path('users/<hashids:user_id>/', test_user_id, name='test_user_id'),
]

We are now ready to test it.
Generate a hashid like this

$ ./manage.py shell -c 'from hashids import Hashids; print(Hashids().encode(1))'
jR

Now, run the development server and send a request

$ ./manage.py runserver

and from another shell

$ curl localhost:8000/users/jR/
1

If everything is correct you should see that the curl command returns 1, which is the value we encoded in the previous command.

Ok, the custom path converter works just fine, yet the encoded values are still easily guessable by anyone with just as little as Hashids().decode(jR).
That is why Hashids - no big surprise - let us set a salt (also a min_length and a custom alphabet).

So now we are going to set our custom values in settings.py and doing a bit of refactoring, ending up with custom encode/decode utility functions to also being able to encode/decode ids in other parts of our project (for example in a DRF serializer).

Salt, min_length and encode/decode utilities

In mysite/settings.py add this dict

HASHIDS = {
    # SECURITY WARNING: keep the salt used in production secret!
    'SALT': 'Nel mezzo del cammin di nostra vita',
    'MIN_LENGTH': 11
}

Create a new file ids_encoder/utils.py and paste this code in it

from django.conf import settings

def get_params():
    try:
        HASHIDS = settings.HASHIDS
    except:
        HASHIDS = {}

    salt = HASHIDS.get('SALT')
    min_length = HASHIDS.get('MIN_LENGTH')
    res = {}
    if salt: res['salt'] = salt
    if min_length: res['min_length'] = min_length

    return res

def get_regex(params):
    min_length = params.get('min_length')
    if min_length is not None:
        return f'[0-9a-zA-Z]{{{ min_length },}}'
    return '[0-9a-zA-Z]+'

PARAMS = get_params()
REGEX = get_regex(PARAMS)

get_params reads our custom settings and convert that in dict that can then be passed to the Hashids initilizer.
get_regex returns the appropriate regex according to settings (e.g. if no min length is set the regex would be '[0-9a-zA-Z]+', if min length is 11 the regex would be '[0-9a-zA-Z]{11,}')

Now in ids_encoder/__init__.py

from .utils import PARAMS
from hashids import Hashids

hashids = Hashids(**PARAMS)

def encode_id(_id: int) -> str:
    return hashids.encode(_id)

def decode_id(_id: str) -> int:
    decoded_values = hashids.decode(_id)
    # output of hashids.decode is always a tuple
    if len(decoded_values) != 1:
        raise ValueError
    return decoded_values[0]

These are the encode/decode functions that we could later import anywhere in the project.

And finally, in ids_encoder/converters.py, replace the previous code with

from .utils import REGEX
from . import encode_id, decode_id


class HashidsConverter():
    regex = REGEX

    def to_python(self, value: str) -> int:
        return decode_id(value)

    def to_url(self, value: int) -> str:
        return encode_id(value)

Ok, now everything's done. We can test it as we did before.
Generate a hashid like this (note, we are now using our own encode_id function to generate the hashid)

$ ./manage.py shell -c 'from ids_encoder import encode_id; print(encode_id(1))'
zpA1nRdJbwG

Run the development server and send a request

$ ./manage.py runserver

and from another shell

$ curl localhost:8000/users/zpA1nRdJbwG/
1

Example within a DRF serializer

If you want to obfuscate the id field in the response of a Django REST Framework view, I would define a serializer like this

from rest_framework.serializers import ModelSerializer
from django.contrib.auth import get_user_model

from ids_encoder import encode_id


class UsersSerializer(ModelSerializer):
    class Meta:
        model = get_user_model()
        fields = ['id', 'email']

    def to_representation(self, instance):
        """Convert id to hashid"""
        res = super().to_representation(instance)
        res['id'] = encode_id(res['id'])
        return res

Note that you may also want to override the to_internal_value method of a DRF serializer (refer to the DRF docs).

Wait for the WebSocket connection to be open, before sending a message

Andrea Bertoloni — Thu, 09 Apr 2020 15:41:21 +0000

Sometimes when you open a new WebSocket connection and try to send a message, you can receive an Error reporting something like this: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state., because the connection is not open yet.

To avoid that, I saw some people suggesting to send the message on the onopen event; something like:

socket.onopen = () => socket.send(message)

This approach, though, only works if you need to trigger send() once (on open).

A more flexible approach would be to wait for the connection to be open (if it's not already).

The waitForOpenConnection function, in the example below, returns a Promise that resolves when the connection is finally open, checking it every n milliseconds.
If, after a given number of checks, the connection is still not open, the Promise is rejected.
In the sendMessage function, you can see an example of the usage.

const waitForOpenConnection = (socket) => {
    return new Promise((resolve, reject) => {
        const maxNumberOfAttempts = 10
        const intervalTime = 200 //ms

        let currentAttempt = 0
        const interval = setInterval(() => {
            if (currentAttempt > maxNumberOfAttempts - 1) {
                clearInterval(interval)
                reject(new Error('Maximum number of attempts exceeded'))
            } else if (socket.readyState === socket.OPEN) {
                clearInterval(interval)
                resolve()
            }
            currentAttempt++
        }, intervalTime)
    })
}

const sendMessage = async (socket, msg) => {
    if (socket.readyState !== socket.OPEN) {
        try {
            await waitForOpenConnection(socket)
            socket.send(msg)
        } catch (err) { console.error(err) }
    } else {
        socket.send(msg)
    }
}

Link to the GitHub gist