DEV Community: Yhary Arias

Docker Offload and the advantages of using it in IA projects

Yhary Arias — Wed, 16 Jul 2025 02:43:11 +0000

Docker Offload is the new release of Docker released this July 10, 2025, this new functionality allows us to offload the execution of containers remotely, taking advantage of the cloud without configuring complex environments.

Docker describes it as a way to run containers “as if it were local, but running on another machine”, all controlled from your own docker run as we have always done.

How can this be an advantage in an IA project?
In general we know that IA projects usually need:

Powerful GPU's to train or run models.
Lots and lots of RAM (otherwise the machine shuts down 😬)
Scalability and efficiency in model testing

With Docker Offload, you can:

Use your local machine to develop, but running models on a more powerful infrastructure.
Accelerate ML pipelines without manually mounting servers
Test giant models without crashing your machine

Let's get to the point! here are the advantages:

Simplicity: No need to set up complex clusters or remote servers.
Scalability: You can easily leverage cloud or external servers
Portability: You keep using the usual Docker ecosystem.
Ideal for AI testing and development without overloading your team 😎

Let's get to the good stuff, the practice:

🚀 Step by step: basic Docker Offload configuration.

Before you start, you need to have installed:

Docker Desktop updated (I recommend you to remove the app from your machine and reinstall it, to go fixed).
Docker Offload enabled (requires Docker account with beta program access).

Enable Docker Offload

In Docker Desktop:

Go to Settings > Features in development
Enable the Docker Offload option
Restart Docker Desktop

Let's follow the "quickstart" of the official Docker documentation

According to the documentation as of today (07/15/2025) we must open Docker Desktop and log in.
Now open the terminal of your machine (whatever it is) and run the following command:

$ docker offload start.

It should show you the following message:

Hit Enter if the account you are going to use is correct. Then it will ask you if you need GPU support. For this example we will say “yes”:

We are now ready to use Docker Offload! 🚀

Docker Offload will run on an instance with an NVIDIA L4 GPU, which is useful for machine learning or resource intensive workloads.

Now if you go to Docker Desktop you are going to see a cloud icon ☁️ in the header of the panel, this tells us that you have successfully enabled Docker Offload.

To check the status of Docker Offload let's run the following command:

$ docker offload status

Finally we are going to run a container with Docker Offload.

The documentation suggests the following example:

$ docker run --rm --gpus all hello-world

You should see the following after running the command:

We are doing quite well 🥳 now the following example is optional, so you can see how it works with another command creating a container in a simple way:

$ docker run --platform linux/amd64 python:3.11-slim python -c "print('Hola desde Docker Offload!')"

Wait ‼️ don't go away, we must stop the use of Docker Offload to avoid unnecessary consumption of resources in the cloud and return to a local environment by running the following command:

$ docker offload stop

🛠️ And that's it!
I hope this Docker Offload journey has saved you CPU, RAM... and headaches! 🧠💻

Love you guys, thanks for making it this far. Bye.
[Yhary Arias / @ia.fania]

Orchestrating Models: Machine Learning with Docker Compose

Yhary Arias — Tue, 25 Mar 2025 18:19:50 +0000

Docker Compose is a powerful tool for defining and managing multi-container Docker applications in a simple and efficient way. In this article, we will explore the basics of Docker Compose and how you can start using it to orchestrate your applications.

What is Docker Compose?

Docker Compose is a tool that allows you to define and run multi-container Docker applications using a YAML file to configure your application's services. Then, with a single command, you can create and run all the defined containers. This makes it easier to create and configure complex development and production environments where multiple services need to interact with each other.

Optional
Installing Docker Compose

Before getting started, make sure you have Docker Compose installed on your machine. You can install it by following the official Docker instructions.

If your OS is Mac, you can use the following command to install Docker Compose. Before running it, ensure that Docker Desktop is installed on your machine.

$ brew install docker-compose

Now, check the installed version:
$ docker-compose --version

Before testing Docker Compose with a Machine Learning project, let's clarify the difference between Docker Compose and Kubernetes.

Docker Compose:

What it does: Docker Compose is a tool that allows you to run multiple containers together. It is mainly designed for development and testing. It is ideal if you want to quickly spin up multiple services on your machine, such as a database, an API, and a web application, all running locally.

How it works: You use a file called docker-compose.yml to define which containers you will use and how they connect to each other. For example, you can say: "I want to launch my application and connect it to a database." Compose will take care of that for you with a single command.

Ideal for: Small or development projects where you don't need a very complex system and just want to test things quickly on your machine.

Kubernetes:

What it does: Kubernetes is much larger and more powerful than Docker Compose. Not only does it help you launch containers, but it also helps manage applications in production, on real servers, efficiently and at scale.

How it works: Kubernetes ensures that your application is always running, has enough resources, and can handle many users. If one of your containers fails, Kubernetes automatically replaces it. It can also scale (increase or decrease the number of containers) based on what your application needs at any moment.

Ideal for: Large-scale production applications that need to be always available and handle heavy traffic. Big companies or projects expecting significant growth often use Kubernetes.

In summary:

Docker Compose is for launching and managing multiple containers quickly on your local machine, ideal for development or testing.

Kubernetes is for managing large-scale applications in production, where you need more control, stability, and scalability.

Compose is like a small engine that helps you work on your project. Kubernetes is like a massive system that keeps your application running smoothly, even with many users and high traffic.

Now, let's get to the real deal. 😎

I'll show you how to apply Docker Compose in an ML project. We will create a simple application that trains a machine learning model and exposes a web service for making predictions.

Project Objective

Our project will consist of two services:

ML Service: A trained machine learning model using scikit-learn that is exposed via a Flask web API.

Database Service: A PostgreSQL database to store prediction results.

Project Structure

The basic file structure will be as follows:

ml_project/
│
├── docker-compose.yml
├── ml_service/
│   ├── Dockerfile
│   ├── app.py
│   ├── model.py
│   ├── requirements.txt
└── db/
    ├── init.sql

1. Define docker-compose.yml

The first step is to define the services in a docker-compose.yml file.

version: '3'
services:
  ml_service:
    build: ./ml_service
    ports:
      - "5000:5000"
    depends_on:
      - db
  db:
    image: postgres
    environment:
      POSTGRES_DB: ml_results
      POSTGRES_USER: ml_user
      POSTGRES_PASSWORD: ml_password
    volumes:
      - ./db/init.sql:/docker-entrypoint-initdb.d/init.sql
    ports:
      - "5432:5432"

2. Create the Machine Learning service
Inside the ml_service/ folder, we create a Dockerfile that will install the necessary Python dependencies, train a model, and expose the service.

Dockerfile (for the ML service)

FROM python:3.8

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]

requirements.txt
Here we define the dependencies we will use, such as Flask for the web server and scikit-learn for the ML model.

Flask==2.1.0
scikit-learn==1.0.2
psycopg2-binary==2.9.3  # Para conectar con PostgreSQL

model.py
This file contains the code to train the machine learning model. We will use a simple classification model like Logistic Regression.

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
import pickle

def train_model():
    # Load sample dataset
    iris = load_iris()
    X, y = iris.data, iris.target

    # Train model
    model = LogisticRegression()
    model.fit(X, y)

    # Save model to file
    with open('model.pkl', 'wb') as f:
        pickle.dump(model, f)

if __name__ == "__main__":
    train_model()

This code trains a simple classification model and saves it as a model.pkl file.

app.py
This is the Flask file that creates the API to make predictions using the trained model. Additionally, it stores the predictions in the database.

from flask import Flask, request, jsonify
import pickle
import psycopg2

app = Flask(__name__)

# Load trained ML model
with open('model.pkl', 'rb') as f:
    model = pickle.load(f)

# Connect to PostgreSQL database
conn = psycopg2.connect(
    dbname="ml_results", user="ml_user", password="ml_password", host="db"
)
cur = conn.cursor()

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json
    X_new = [data['features']]

    # Make prediction
    prediction = model.predict(X_new)[0]

    # Save prediction to database
    cur.execute("INSERT INTO predictions (input, result) VALUES (%s, %s)", (str(X_new), int(prediction)))
    conn.commit()

    return jsonify({"prediction": int(prediction)})

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=5000)

This Flask service listens on port 5000 and, upon receiving a POST request with the input characteristics, returns a prediction and stores the result in PostgreSQL.

3. Configure the Database
In the db/ directory, we create an init.sql file to initialize the database with a table to store predictions.

init.sql

CREATE TABLE predictions (
    id SERIAL PRIMARY KEY,
    input TEXT,
    result INTEGER
);

This script will run automatically when the PostgreSQL container is started, and will create a table called predictions.

4. Run the Project
Now that everything is set up, you can run the entire project with Docker Compose. From the root project directory, run:

$ docker-compose up

This will make Docker Compose:

Build the machine learning service image.
Start the ML and database containers.
Run the Flask model and web server on port 5000.

5. Test the API
To make a prediction, send a POST request to http://localhost:5000/predict with a JSON body containing dataset features.

Command example curl:
$ curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"features": [5.1, 3.5, 1.4, 0.2]}'

If everything is configured correctly, you will get a response like:

{
  "prediction": 0
}

Conclusion
With Docker Compose, we have created a machine learning project that includes a web service to make predictions with an ML model, as well as a PostgreSQL database to store the results. Docker Compose simplifies the management of these services in development and production environments, allowing you to work with multiple containers in a coordinated manner.

This example is just a starting point, and you can expand it by adding more services, connecting other machine learning models or integrating tools like Redis for caching or Celery for asynchronous tasks.

Now you're ready to use Docker Compose in more complex machine learning projects!

Author: Yhary Arias.
GitHub: @yharyarias
LinkedIn: @yharyarias
Instagram: @ia.fania