DEV Community: tigerfanxiao

Build Python Docker Development Environment on Win11 from Scratch

tigerfanxiao — Mon, 12 May 2025 13:35:46 +0000

Guideline

This guide is to help build python docker development environment on brand new Win11.

Install WSL on win11, which used to create Linux container on windows.
Install Docker Desktop on Windows
Install VSCode on Windows
Create Dockerfile, build docker image and run docker container
Create Docker-compose to automate the process of build and render by code modification # Step 1 Install WSL on win11 WSL, aka Windows Sub-system Linux is a tool to help build virtual environment on windows and share the file system. By WSL, Docker desktop on windows can build Linux core container inside of it, which can help the same behavior as we deploy our container to Linux server.

Let's see how to enable WSL on win11
Open Powershell to install wsl

wls --install 
# By default widnows will install Ubuntu as WSL. 
# You need to configure user / password to complete the installation.

After the installation complete, you can test wsl Ubuntu by search in "Ubuntu" in start menu or open in windows terminal

Or in terminal with following command

wsl -d Ubuntu

Step 2 Install Docker Desktop

You can download and install Docker Desktop on windows from the following URL
https://docs.docker.com/desktop/setup/install/windows-install/
After you complete the installation of Docker Desktop, you need to go to the setting of docker desk to enable WSL integration, which basically will allow you to run docker daemon inside of WSL

![[Pasted image 20250509171217.png]]
To check if docker desktop run well, go to WSL Ubuntu environment and run

docker --version
docker run hello-world # it is the test docker image for testing

Step 3 Install VSCode

In this guide, we suppose to use VScode as our IDE. Actually I found that you need to pay for Pycharm Professional to have remote Python interpreter configuration.

Download VSCode through the following URL
https://code.visualstudio.com/

The following extension as well

WSL - Microsoft
Dev Container - Microsoft
Python - Microsoft
Docker - Microsoft Now we can develop inside of VSCode and run the code inside of docker container which is running WSL Ubuntu

Install python virtual environment on Ubuntu

Install python3 and pip3

# check which version of python3 installed on your os
python3 --version
# Install pip3 
sudo apt install python3-pip
pip3 --version # check pip3 installed correctly

install venv for python virtual-env

sudo apt-get update
sudo apt-get install libpython3-dev
sudo apt-get install python3-venv

create python virtual environment

python3 -m venv .venv
# Activate virtual environment
source ./venv/bin/activate
# Install python package in virtual environment
pip install -r requirements.txt

# to exit virtual environment
deactivate

Configure Python interpreter inside of virtual environment as Python interpreter for the code in the VSCode
ctrl + shift + p to select new python interpreter

select virtual env you just created

Now we have configured our local python development environment well. Then let's create docker container and make the same environment in the container. The objective here is we develop locally but render our server inside of docker. The code is locally saved on your windows, including requirements.txt, Dockerfile, docker-compose, .gitignore, all of them will help use to rebuild the container or environment from scratch in regardless of docker container life cycle.

Step 4 Create Dockerfile

You need to have some basic knowledge to understand the following tutorial. I personally refer the video of Arjan on youtube https://www.youtube.com/watch?v=zkMRWDQV4Tg&t=405s. Cannot recommend more. The following example and code is just an example from him.
you can refer the code here https://github.com/ArjanCodes/2022-docker/blob/main/README.md?plain=1

Create a folder under your home directory on your WSL Ubuntu for your FastApi app

- app
    - requirements.txt
    - main.py
    - Dockerfile
    - channels.json

Allow explain what are those files used for one by one

`requirements.txt`

fastapi # popular python API framework
pydantic # python package of python typing
uvicorn # python package used in fastapi for server
watchfiles # python package to watch any file change in the main directory,as uvicorn, the server of FastAPI, only detect the change of .py file

main.py is a basic FastAPI script to provide basic function for testing

import json

from dataclasses import dataclass, field
from fastapi import FastAPI, HTTPException, Response

app = FastAPI()

@dataclass
class Channel:
    id: str
    name: str
    tags: list[str] = field(default_factory=list) # default value is a list
    description: str = ""

channels: dict[str, Channel] = {}

with open("channels.json", encoding="utf8") as file:
    channels_raw = json.load(file)
    for channel_raw in channels_raw:
        channel = Channel(**channel_raw)
        channels[channel.id] = channel

@app.get("/")
def read_root() -> Response:
    return Response("The server is running! fan xiao")

@app.get("/channels/{channel_id}", response_model=Channel)
def read_item(channel_id: str) -> Channel:
    if channel_id not in channels:
        raise HTTPException(status_code=404, detail="Channel not found")
    return channels[channel_id]

channels.json is a list of json object

[
    {
        "id": "codestackr",
        "name": "codeSTACKr",
        "tags": ["web development", "typescript"],
        "description": "My tutorials are generally about web development and include coding languages such as HTML, CSS, Sass, JavaScript, and TypeScript."
    },
    {
        "id": "jackherrington",
        "name": "Jack Herrington",
        "tags": ["frontend", "technology"],
        "description": "Frontend videos from basic to very advanced; tutorials, technology deep dives. You'll love it!"
    },
    {
        "id": "arjancodes",
        "name": "ArjanCodes",
        "tags": ["software design", "Python"],
        "description": "ArjanCodes focuses on helping you become a better software developer."
    }
]

`Dockerfile`

You can think Dockerfile is the handbook Docker Daemon used to build docker image.
When you run docker build command, Docker Daemon will find Dockerfile in current directory to build docker image.
Several points need to be mentioned

# Base image to specify the python version
FROM python:3.14.0b1-slim-bullseye
# Make a working directory in the container
WORKDIR /app
# Copy and install requirements.txt to the container 
# The installation will be cached by docker itself. Suppose we don't change the requirement as frequently as code, each time we change the docker file it, this package installation would be re-run
# --no-cache-dir means not save the python package inside of container
COPY ./requirements.txt /app
RUN pip3 install --no-cache-dir -r requirements.txt
# copy the local source code to container
COPY . /app
# run flask server on port 80
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

Build image from the Dockerfile

# . means find Dcokerfile in the current directory
# -t means tag. If you didn't assign a tag, docker daemon will randomly assign it, which helps you later on easily find this image
docker build -t xiao/channel_api:0.1 .

run docker container

# channel-api is image name
# -d means detached 
docker run -d -p 8080:80 channel_api 

# kill container
docker kill <container_id>

Test on local browser http://localhost:8080/

Step 5 Create Docker-compose

To automate the process of build and render by code modification, we create docker compose
Create a docker-compose.yaml file

services:
  app:
    build: . # Directory of Dockerfile
    container_name: simple-python-server
    # the command will replace the CMD in Dockerfile restart the server 
    # uvicorn only detect the change on .py file
    command: uvicorn main:app --host 0.0.0.0 --port 80 --reload
    ports:
      - 8080:80 # Local on windows port 8080 mapping to docker container port 80
    # Any change in volumes will trigger reload uvicorn server, overwrite the copy command in Dockerfile
    volumes:
      - .:/app

run docker container based on docker compose

docker compose up --build # first time you need to use --build

How to use Python Configparser Package

tigerfanxiao — Wed, 01 Feb 2023 16:31:51 +0000

import configparser
import os

def get_config(section, key):
    config = configparser.ConfigParser()
    config.read(os.path.join(APP_PATH, 'config.ini'))
    return config

CONFIG = get_config()

# You can use it as CONFIG['section']['key']

How to Process Large EXCEL File with Python Win32com

tigerfanxiao — Tue, 10 Jan 2023 17:05:46 +0000

When you have a large excel file to handle, say 100M, it is not efficient to read it directly through pandas or openpyxl.
I have tested myself, to open one 100M size excel file and convert to pickle format, pandas takes around 8 minutes, even in readonly mode in my laptop.

Although later you can access pickle file with pandas quickly, the first 8 minutes was not acceptable for some task.

In this blog, we leverage the Excel Application itself to help us to convert excel file to csv format, which is also very easy to be handled by pandas.

Sometimes you have to admit Microsoft is still the best man to handle excel:)

Using Win32com library, you could reduce the time to around 100 seconds, which is only a fifth of time using pandas.

from time import time
import win32com.client
from os import path
import os

cwd = os.getcwd()
your_excel_file_path = path.join(cwd, "your_excel_file.xlsx")
export_csv_file_path = path.join(cwd, "new_file.csv")

start = time()

excel_app = win32com.client.Dispatch("Excel.Application")
excel_app.DisplayAlerts = False
wb = excel.Workbooks.Open(Filename=your_excel_file_path , ReadOnly=1)

# make target sheet active
ws = wb.Worksheets("active_sheet_name").Activate() 

wb.SaveAs(Filename=export_csv_file_path , FileFormat="6")  # 6 means csv
wb.Close(False)
excel.Application.Quit()
wb = None
excel_app = None


end = time()
print(start - end)  # around 100 seconds

Do not use append/extend in Python Comprehension expression

tigerfanxiao — Fri, 02 Dec 2022 11:17:36 +0000

One mistake which for me almost one hour to debug is that I used list method append in the python comprehension expression.

Take a look at belows to assignment, check which one will have output of [[1, 4], [2, 4]]

# first 
[subset.append(4) for subset in [[1], [2]] 
# second
[subset + [4] for subset in [[1], [2]]

Actually the second one is correct, and with the first one, you will get [None]. Why? Because list method like append or extend does not have return value, they just modify the original list. For that reason you got None when you tried to do some operation on a list.

Be careful of those methods which don't have return value when you are trying to use them in comprehension expression in Python.

Pyinstaller or Python eel generated .exe refer external files errors

tigerfanxiao — Wed, 30 Nov 2022 09:59:05 +0000

When we are trying to generate .exe file for our python project, normally we use pyinstaller. If you are writing python eel project, basically in the background, eel encapsulate pyinstaller.

But sometimes we want to refer the external file in our project, for example for config or setting files. In development environment, we didn't found any issue, but when we run generated .exe file, we may encounterd error saying that cannot find the specific external files. The cause of this error is when you run .exe file, the app is referring the local python interpreter not the one packaged inside of .exe file.

In this blog, I wrote a function to refer the differrent project path for dev env and production.

Commands we use to generate .exe file

# eel command to generate .exe file
# where web is the dir for static files
python -m eel --onefile main.py web

# pyinstaller command to generate .exe file
pyinstaller --onefile -w main.py

function refer different path for dev env and production.

def get_app_running_path():
    # for .exe file
    if getattr(sys, 'frozen', False):
        app_path = os.path.dirname(sys.executable)
    else:
        # for dev environment
        app_path = os.path.dirname(__file__)
    return app_path

How to test port accessibility on windows 10

tigerfanxiao — Wed, 16 Mar 2022 09:33:03 +0000

We all know we can use Ping to test IP accessibilty some windows platform. However, sometimes we not only want to test if we can access some ip address or domain, we would like also want to test if we could access some specified port. For example, if you would like to access https://www.some.site, but you cannot open it in browser, it could be because you have been blocked by firewall for your local ip address. You need some tools to test port accessibilty. Or you may want to test some email server address with port number.

Here we introduce a way to test port accessibility on windows 10 with telnet.

As Telnet by default is not opened in windows, for some security reason. You could refer this link to open telnet on your window 10 device.

After that, open one cmd window

telnet <ip/domain> <port>