disclaimer: I am new to Docker so please take this with a grain of salt. at the time of writing, this setup worked on my computer.
also please let me know in the comments if I got something wrong
the files
These files assumes that your Dockerfile lives in the root of your Django project (i.e. in the same level as manage.py).
Dockerfile
FROM python:3.9-slim-buster
# set environment variables
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
# don't write .pyc files
ENV PYTHONDONTWRITEBYTECODE 1
# prevent Docker from buffering stdout
ENV PYTHONUNBUFFERED 1
# set working directory
WORKDIR /code
RUN apt-get update && apt-get install -y \
curl \
gnupg2 \
unzip \
wget
# chrome
RUN sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& apt-get update \
&& apt-get install google-chrome-stable -y
# chromedriver
RUN CHROME_VERSION="$(google-chrome --version)" \
&& export CHROMEDRIVER_RELEASE="$(echo $CHROME_VERSION | sed 's/^Google Chrome //')" \
&& export CHROMEDRIVER_RELEASE=${CHROMEDRIVER_RELEASE%%.*} \
&& CHROMEDRIVER_VERSION=$(curl http://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROMEDRIVER_RELEASE}) \
&& curl --output /tmp/chromedriver_linux64.zip "http://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VERSION/chromedriver_linux64.zip" \
&& cd /tmp \
&& unzip chromedriver_linux64.zip \
&& rm -rf chromedriver_linux64.zip \
&& mv chromedriver /usr/local/bin/chromedriver \
&& chmod +x /usr/local/bin/chromedriver \
&& cd /code
COPY ./requirements.txt .
# install dependencies
RUN apt-get update && apt-get install -y \
gcc \
less \
libmagickwand-dev \
libpq-dev \
vim \
&& rm -rf /var/lib/apt/lists/* \
&& pip install -r requirements.txt
# copy project
COPY . .
docker-compose.yml
version: "3.9"
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
environment:
- CHROME_DRIVER_PATH=/usr/local/bin/chromedriver
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- "POSTGRES_HOST_AUTH_METHOD=trust"
redis:
image: redis:alpine
celery:
build:
context: .
command: celery -A rises_ops worker -l INFO
environment:
- CHROME_DRIVER_PATH=/usr/local/bin/chromedriver
volumes:
# same volume as web
- .:/code
depends_on:
- db
- redis
- web
volumes:
postgres_data:
migration script
after running docker compose up --build, run the migration from your terminal
$ docker-compose exec web python manage.py migrate
(there is probably a smarter way to do this, let me know in the comments)
references
blogs that actually helped
https://learndjango.com/tutorials/django-docker-and-postgresql-tutorial
https://learndjango.com/tutorials/django-docker-and-postgresql-tutorial
debugging tips
# build image with no cache
$ docker build . --no-cache
# output build process to your terminal
$ docker build . --progress=plain
# re-build your services
$ docker compose up --build
# clean cache
$ docker system prune
$ docker image prune
note that the pg_data volume is actually saved in Host, so the DB data will persist even after you delete your conatiners. to clear even the DB,
# identify your volume
# format is project_db-data e.g. myproject_db-data
$ docker volume ls
# remove
$ docker volume rm <VOLUME ID>
ref: https://torajirousan.hatenadiary.jp/entry/2020/07/08/094325
docker-compose.yml
the docker-compose.yml is pretty straightforward. we have 4 services: web db redis celery.
- web: the Django werver
- db: PostgreSQL DB
- redis: Redis used in the Celery workers
- celery: worker process for Django
the commands written in the command: section of the file should be similar to what you execute in your local terminal.
one thing to note is that, web and celery share the same volume, since both of them need access to the Django code
Dockerfile
the bulk of this file is to install chrome and chromedriver. both of them are not in the default apt-get repository, so you have to
- add the repository for
google-chrome-stable, and - download the zip file for
chromedriver
in my past experience, Ubuntu has chromium and chromium-driver in the default repository, so this might be a better option if image size is not a problem for you.
chrome
to explain the process for installing Google Chrome,
# add the repository to the list
RUN sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' \
# add key for signed packages
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
# don't forget to update the list after adding the repo
&& apt-get update \
# finally install the package
&& apt-get install google-chrome-stable -y
chromedriver
getting chromedriver is even more complex.
this is because
- chrome and chromedriver needs to have compatible versions
- BUT chrome and chromedriver versions are not always the same
to get the compatible versions, you have to first download the chrome, and get the compatible version from a web page.
let's go through this with an example.
at the time of writing (Sep. 2022), the latest version of google-chrome is 105.0.5195.102.
$ google-chrome --version
# Google Chrome 105.0.5195.102
to get the corresponding chromedriver version, go to
https://chromedriver.storage.googleapis.com/LATEST_RELEASE_105
(if your chrome version was 80.x.x.x, you would instead go to https://chromedriver.storage.googleapis.com/LATEST_RELEASE_80 )
the page would show you that the corresponding chromedriver version is 105.0.5195.52
Dockerfile does this in the script by first getting the 105 part from the chrome version 105.0.5195.102.
# get chrome version
RUN CHROME_VERSION="$(google-chrome --version)" \
# remove string "Google Chrome " from the output
&& export CHROMEDRIVER_RELEASE="$(echo $CHROME_VERSION | sed 's/^Google Chrome //')" \
# get the major version
# `%%` removes the longest match of the following pattern `.*`
# so it removes `.0.5195.52` and stores `105` in the variable
&& export CHROMEDRIVER_RELEASE=${CHROMEDRIVER_RELEASE%%.*} \
next, it accesses the LATEST_RELEASE_ page, and stores the response in the variable CHROMEDRIVER_VERSION.
&& CHROMEDRIVER_VERSION=$(curl http://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROMEDRIVER_RELEASE}) \
# 105.0.5195.52
# coincindentally, chrome & chromedriver versions match
then, download the zip via curl --output, and unzip in the /tmp folder
# specify output location to /tmp/chromedriver_linux64.zip
&& curl --output /tmp/chromedriver_linux64.zip "http://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VERSION/chromedriver_linux64.zip" \
# go to `/tmp`
&& cd /tmp \
# unzip in `/tmp`
&& unzip chromedriver_linux64.zip \
finally, we move chromedriver to /usr/local/bin and make sure chromedriver is executable
# remove zip file because we don't need it anymore
&& rm -rf chromedriver_linux64.zip \
# move executable to /usr/local/bin
&& mv chromedriver /usr/local/bin/chromedriver \
# add permission
&& chmod +x /usr/local/bin/chromedriver \
as a clean up step, we move back to the WORKDIR
&& cd /code
put together, we have
RUN CHROME_VERSION="$(google-chrome --version)" \
&& export CHROMEDRIVER_RELEASE="$(echo $CHROME_VERSION | sed 's/^Google Chrome //')" \
&& export CHROMEDRIVER_RELEASE=${CHROMEDRIVER_RELEASE%%.*} \
&& CHROMEDRIVER_VERSION=$(curl http://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROMEDRIVER_RELEASE}) \
&& curl --output /tmp/chromedriver_linux64.zip "http://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VERSION/chromedriver_linux64.zip" \
&& cd /tmp \
&& unzip chromedriver_linux64.zip \
&& rm -rf chromedriver_linux64.zip \
&& mv chromedriver /usr/local/bin/chromedriver \
&& chmod +x /usr/local/bin/chromedriver \
&& cd /code
(same result as docker-compose.yml in the first section)
Top comments (0)