I've successfully deployed my Australian postcodes API project to https://railway.app. I did have some problem during deployment. I'm describing how I've addressed this problem. In the process, we're also covering the following: β΅ running Railway's own Nixpacks Docker build tool locally on Ubuntu 22.10. βΆ Override the Nixpacks-built Docker image's CMD
: we look at three (3) ways to run the Flask CLI command venv/bin/flask update-postcode
, and similarly, we look at how to override the start command gunicorn wsgi:app --preload
specified in the Nixpacks required Procfile.
These are the two (2) API endpoints hosted on Railway:
-
Swagger UI Documentation: https://web-production-ed7a.up.railway.app/api/v0/ui .
-
Endpoint API:
web-production-ed7a.up.railway.app/api/v0/aust-postcode/
.
E.g. To search for localities which containspring
: https://web-production-ed7a.up.railway.app/api/v0/aust-postcode/spring .
π Full source code and documentation: https://github.com/behai-nguyen/bh_aust_postcode. This repo is now Railway-deployment ready. It includes Railway's required files requirements.txt
, Procfile
and runtime.txt
.
As noted in the README.md file, the current version supports PostgreSQL, instead of SQLite as it did originally, since https://railway.app does not support SQLite.
Related posts on this Australian postcodes API project. Please note, except for changing to support PostgreSQL, there was no change to functionalities:
- Python: A simple web API to search for Australian postcodes based on locality aka suburb.
- Ubuntu 22.10: hosting a Python Flask web API with Gunicorn and Nginx.
- jQuery plugin: bhAustPostcode to work with the search Australian postcodes web API.
βΆ Railway deployment problem.
On a side note, I stumbled upon the Railway website. I was able to set up a PostgreSQL database fairly quickly, I can connect to it using pgAdmin 4
version 6.18
, Windows 10. The documentation is easy to understand, I like it. I played around with it for awhile. I was on a free plan. But the next day, they stated that they can't verify who I am using my GitHub account, so I joined the Hobby Plan. It is only fair, we need to pay for the services.
Before deployment, I was actually thinking that I already have it in the bag π, since the database is my biggest concern, and I am pretty sure I have no problem with it. But before talking to the database, the project needs to be successfully deployed.
Railway reported error, these are the last lines of my second deployment log:
...
File "/app/wsgi.py", line 1, in <module>
from app import app
File "/app/app.py", line 3, in <module>
from bh_aust_postcode import create_app
ModuleNotFoundError: No module named 'bh_aust_postcode'
[2023-07-05 10:36:48 +0000] [9] [INFO] Worker exiting (pid: 9)
[2023-07-05 10:36:48 +0000] [1] [INFO] Shutting down: Master
[2023-07-05 10:36:48 +0000] [1] [INFO] Reason: Worker failed to boot.
To recap, the directory structure of the project is as follows:
/home/behai/webwork/bh_aust_postcode
βββ app.py
βββ Hosting.md
βββ instance
βββ LICENSE
βββ omphalos-logging.yml
βββ Procfile
βββ pyproject.toml
βββ pytest.ini
βββ README.md
βββ requirements.txt
βββ runtime.txt
βββ src
β βββ bh_aust_postcode
β β βββ api
β β β βββ bro.py
β β β βββ __init__.py
β β β βββ postcode_pool.py
β β β βββ routes.py
β β βββ commands
β β β βββ schema.sql
β β β βββ update_postcode.py
β β βββ config.py
β β βββ __init__.py
β β βββ utils
β β βββ __init__.py
βββ tests
β βββ conftest.py
β βββ __init__.py
β βββ test_api_endpoints.py
β βββ test_bro.py
β βββ test_postcode_pool.py
βββ wsgi.py
Seeing the error, I did verify that the name of the project root directory bh_aust_postcode
is not important, it can be anything. Looking at Railway's build log, I understand that the root /app
directory is the value of the Docker image environment variable WORKDIR
-- and that should not be a problem!
What I did next was installing Railway's own build tool Nixpacks to my Ubuntu 22.10 machine and did my own build: I did not supply my own Dockerfile
, I want to use the default to closely match the Railway's image.
-- My own built image using Nixpacks produced the same error! Which is somehow... a good thing!
I started to fully qualify all import
s across the entire project. I.e. changing from:
from bh_aust_postcode.config import get_database_connection
to:
from src.bh_aust_postcode.config import get_database_connection
And it finally deployed!
-- Only then I remember addressing this very issue in Python: Docker image build β install required packages via requirements.txt vs editable install! And that was nearly one (1) year ago! But not all wasted, I learn about Nixpacks and a bit more about Docker.
I did not want to use absolute import, but I did in this case. Perhaps if I supply my own Dockerfile
, then I can use editable install and do not have to use absolute import?
β· Installing and running Railwayβs own Nixpacks Docker build tool locally on Ubuntu 22.10.
Nixpacks installation is simple. Download the appropriate installation file from Nixpacks Releases.
For HP Pavilion 15 Notebook PC, Born On Date
of 04/October/2014
, it is nixpacks-v1.9.2-amd64.deb
, and I copied to it /home/behai/Public/
. Then run the following command to install:
$ sudo dpkg -i /home/behai/Public/nixpacks-v1.9.2-amd64.deb
We need to set the values of the environment variables in the .env
file appropriate for the Docker image.
Content of /home/behai/webwork/bh_aust_postcode/.env:
SECRET_KEY=">s3g;?uV^K=`!(3.#ms_cdfy<c4ty%"
FLASK_APP=app.py
FLASK_DEBUG=True
SOURCE_POSTCODE_URL="http://192.168.0.17/australian_postcodes.json"
KEEP_DOWNLOADED_POSTCODES=False
DB_CREATE_SCRIPT="schema.sql"
SCHEMA_NAME='bh_aust_postcode'
POSTCODE_TABLE_NAME='postcode'
PGHOST=192.168.0.17
PGDATABASE=ompdev
PGUSER=postgres
PGPASSWORD=pcb.2176310315865259
PGPORT=5432
Just to save a bit of data usage, I don't want to download the postcodes from https://www.matthewproctor.com/Content/postcodes/australian_postcodes.json every time I do a test run, I store a copy in the default Nginx site, it can be accessed as http://192.168.0.17/australian_postcodes.json
. Where 192.168.0.17
is the IP address of the Ubuntu 22.10 machine.
The PostgreSQL database server used is the Official Docker image running on Ubuntu 22.10, please see this post for how to set it up. Environment variables PGHOST
, PGDATABASE
, PGUSER
, PGPASSWORD
and PGPORT
specify database connection information.
To build, run the below command. Please note, β΅ the present working directory is /home/behai/
, βΆ the name of the resultant Docker image is bh-aust-postcode
:
$ sudo nixpacks build webwork/bh_aust_postcode --name bh-aust-postcode
The partial build log is shown in the first two (2) screenshots, the resultant Docker image listing is in the last one:
βΈ Override the Docker image's CMD
.
β΅ Three (3) ways to run the Flask CLI command venv/bin/flask update-postcode
for the Docker image.
As noted in the README.MD file, we need to run the following command to download postcodes and populate the database:
$ venv/bin/flask update-postcode
The Railway's equivalence, using its own CLI is:
$ railway run flask update-postcode
I can verify that it works, because I've successfully run it to populate the database hosted by Railway. I've never thought about this before, till now: how do we run commands such as this from a Docker image?
The obvious answer is to run the target Docker image in bash
mode, then run application's commands inside it. Command to get to bash
interactive mode:
$ sudo docker run -it --rm bh-aust-postcode bash
The below screenshot shows bh-aust-postcode
in bash
mode:
Note: it does not show the Python virtual environment directory venv
, I did supply a local .gitignore
file, and Nixpacks uses it for the build.
And running the venv/bin/flask update-postcode
equivalent command:
root@598d49469064:/app# flask update-postcode
Output:
The second approach is to override Docker CMD
. I.e.:
$ sudo docker run -it bh-aust-postcode "flask update-postcode"
Output:
The third method is to override ENTRYPOINT
. I found the command a little nonintuitive:
$ sudo docker run -it --entrypoint /opt/venv/bin/flask bh-aust-postcode update-postcode
On /opt/venv/bin/flask
, just flask
it would not work, the error is about executable not found. And its output is identical to the previous two (2):
βΆ Override the start command gunicorn wsgi:app --preload
specified in the Nixpacks required Procfile.
In a similar manner to the previous section, we can run bh-aust-postcode
image with a specific port and non-routable 0.0.0.0
IP address:
$ sudo docker run -it bh-aust-postcode "gunicorn --bind 0.0.0.0:5000 wsgi:app"
$ sudo docker run -it --entrypoint /opt/venv/bin/gunicorn bh-aust-postcode --bind 0.0.0.0:5000 wsgi:app
While the container is running, both of the following endpoints API will work locally:
$ curl http://172.17.0.3:5000/api/v0/ui
$ curl http://172.17.0.3:5000/api/v0/aust-postcode/spring
Please note, 172.17.0.3
is the IPv4Address
of the container. We run the image without specifying the container name. To find the container IP address we need to know the container name.
The following command shows all available containers, and whether or not they've stopped or still are running:
$ sudo docker ps -a
Since we're using the default network bridge
:
$ sudo docker network inspect bridge
Look for the container name in the output, the value of IPv4Address
is the IP address which we should use.
It has been an interesting exercise for me. The deployment process in itself is not that complicated. Involving a database, we'll need to carry a few steps, but that's to be expected. I would like to write about it in a later post. I hope you find the information in this post useful. Thank you for reading and stay safe as always.
βΏβΏβΏ
Feature image sources:
- https://in.pinterest.com/pin/337277459600111737/
- https://www.omgubuntu.co.uk/2022/09/ubuntu-2210-kinetic-kudu-default-wallpaper
- https://railway.app/design
- https://icon-icons.com/download/170045/PNG/512/
- https://seeklogo.com/vector-logo/332789/python
- https://flask-restx.readthedocs.io/en/latest/
- https://www.vectorstock.com/royalty-free-vector/australia-map-with-flag-blue-red-background-vector-25323215
- https://logos-world.net/australia-post-logo/
Top comments (0)