DEV Community

Peijin Zhang (张沛锦)
Peijin Zhang (张沛锦)

Posted on

Use a container for astronomy data processing

#astronomy #radio #hpc #computing

Radio astronomy needs a lot of software: CASA, WSClean, DP3, LOFAR tools, custom scripts—and installing all of them cleanly on every machine is painful.

Containers let you ship a complete, working environment that runs the same on your laptop, workstation, and HPC.

Below is a short, practical workflow using Podman and the astronrd/linc image, and then an example of building your own image like peijin/lwa-solar-pipehost.

Install Podman

https://podman.io/docs/installation

MacOS, (need brew https://brew.sh/)

brew install podman
Enter fullscreen mode Exit fullscreen mode

Windows, need WSL (follow instructions here:https://learn.microsoft.com/en-us/windows/wsl/install) then inside WSL is a ubuntu.

Or linux

# Debian/Ubuntu
sudo apt-get update
sudo apt-get install -y podman

# Fedora
sudo dnf install -y podman
Enter fullscreen mode Exit fullscreen mode

Pull and run image

The astronrd/linc image (from ASTRON) is a great “batteries included” environment for radio interferometric calibration and imaging.

podman pull docker.io/astronrd/linc
Enter fullscreen mode Exit fullscreen mode

Then try to start an interactive shell inside the container:

podman run --rm -it \
  -v /your/data/path:/data \
  docker.io/astronrd/linc \
  /bin/bash
Enter fullscreen mode Exit fullscreen mode

Explanation:

--rm – delete the container when you exit

-it – interactive terminal

-v /your/data/path:/data – mount your local data directory into /data inside the container

docker.io/astronrd/linc – the image to run

/bin/bash – start a bash shell inside

Now you’re inside a fully configured environment with tools like DP3 and WSClean available, without installing them on your host system.

Try WSClean and DP3 inside astronrd/linc

Once inside the container shell, you can test the tools:

wsclean --version
DP3 --help
Enter fullscreen mode Exit fullscreen mode

Then exit to escape the shell.

(optional) Build your own Docker/Podman image

Although there are so many plug-and-play-ready prebuilt container images, sometimes when a more customized package is needed, it's more convenient to create an image.

Features:

  • Pin exact versions (WSClean v3.x, DP3 commit XYZ, CASA version, etc.)

  • Add your own pipelines/scripts

  • Pre-configure environment variables, paths, and helper tools

The following is a minimal example of how you might build an image like peijin/lwa-solar-pipehost for an OVRO-LWA–style pipeline host.

Step 1: create Dockerfile

Create a file called Dockerfile:

# Start from a base astronomy image (or just Ubuntu)
FROM docker.io/astronrd/linc:latest
# or: FROM ubuntu:22.04

# Set a working directory
WORKDIR /opt/pipeline

# Install extra system packages (example)
RUN apt-get update && apt-get install -y --no-install-recommends \
    git \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

# Copy your pipeline scripts into the image
COPY pipeline/ /opt/pipeline/

# Install Python dependencies
RUN pip3 install --no-cache-dir -r requirements.txt

# Set environment variables (example)
ENV PYTHONUNBUFFERED=1 \
    OMP_NUM_THREADS=8

# Default entrypoint (can be a shell, or a pipeline script)
ENTRYPOINT ["/bin/bash"]
Enter fullscreen mode Exit fullscreen mode

Of course, you can do other optimizations to include more packages.

Step 2: Build the image

From the directory with your Dockerfile:

podman build -t peijin/lwa-solar-pipehost:latest .
Enter fullscreen mode Exit fullscreen mode

Step 3: Run container with the image 

podman run --rm -it \
  -v /your/data/path:/data \
  peijin/lwa-solar-pipehost:latest \
  /bin/bash
Enter fullscreen mode Exit fullscreen mode

Inside, your pipeline code is already in /opt/pipeline.

Top comments (0)