DEV Community: Yue Su

Setting Up vGPU in Docker Container (with Specific Python Version)

Yue Su — Fri, 28 Jul 2023 18:42:17 +0000

Our team has recently been building a backend service using the Django framework. Alongside this, we've also developed an in-house machine learning command line utility tool. To run this tool efficiently, we utilize an instance with an enabled nVidia GPU. The challenge lies in setting up an environment where both Django and the utility are containerized with Docker, with the ability to harness the GPU power inside the container. In addition, the tool requires a specific Python version (3.9 in our case) to run.

This task was not as straightforward as we initially imagined. It seemed we could not just pull off a Python 3.9 office image, install the CUDA driver, and expect it to run smoothly. After a few attempts, we found this to be a considerable challenge. We then took a different approach, opting to use an official nvidia/cuda image and installing the needed Python version. After several tweaks, it finally worked.

Here are the main steps involved:

Ensure that the GPU driver is installed properly on the instance. This is relatively easy. We followed this guide. To check if everything is working as expected, use the nvidia-smi command line. If all is well, you will see the CUDA version and other related information. For additional guidance, refer to the official CUDA installation guide.

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.199.02   Driver Version: 470.199.02   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GRID V100D-8C       On   | 00000000:00:05.0 Off |                    0 |
| N/A   N/A    P0    N/A /  N/A |    560MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Use the base Dockerfile to build the backend. It's crucial to mention that the chosen image should be based on the CUDA driver version used in your environment. If they do not match, the container might throw errors. Ubuntu20.04 comes with Python 3.8 by default. If you do not set up symbolic links correctly, the packages will be installed with Python 3.8, resulting in errors. We used the official NVIDIA CUDA Docker images as a base.

# Use an official NVIDIA CUDA runtime as a parent image
FROM nvidia/cuda:11.4.3-cudnn8-devel-ubuntu20.04

# Set environment variables
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=America/Toronto

# Install Python and pip
RUN apt-get update && apt-get install -y \
    python3.9 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

# Create symbolic links for Python and pip
RUN ln -sf /usr/bin/python3.9 /usr/bin/python \
    && ln -sf /usr/bin/python3.9 /usr/bin/python3 \
    && ln -sf /usr/bin/pip3 /usr/bin/pip

# Set the working directory to /backend
WORKDIR /backend
COPY requirements.txt requirements.txt

RUN pip install --upgrade pip
RUN pip install -r requirements.txt

COPY . .

Use Docker Compose to manage our services. Here is a base yml file you can use if you're utilizing Django as the backend. Make sure to configure the runtime and environment settings correctly for the container to properly utilize the GPU.

version: '3'

services:
backend:
    build: 
    context: .
    dockerfile: Dockerfile
    ports:
    - 8000:8000
    container_name: backend
    runtime: nvidia
    environment:
    - NVIDIA_VISIBLE_DEVICES=all
    restart: unless-stopped
    volumes:
    - ./:/backend
    env_file:
    - .env
    command: >
    bash -c "python manage.py makemigrations
    && python manage.py migrate
    && python manage.py runserver 0.0.0.0:8000"

Test your configuration. If all the previous steps were successful, you should be able to run the nvidia-smi command within the container.

A simple database modelling for a web-based messenger with Sequelize and postgresDB

Yue Su — Mon, 28 Dec 2020 19:43:40 +0000

Goal

This is a simplified database modelling example, which aims to provide a basic structure of the database and demonstrate data persistence.

It will be used for an application that is designed to be a messenger clone, featuring real-time and offline messaging. All the messages and conversations are stored in the database so that a registered user could retrieve the information when logged in.

Tables and associations

A User table for storing username, email and hashed password.
Conversation table and UserToConversation tables for storing user's conversation and many-to-many relationships between user and conversations.
Message table for storing message including sender's id, conversation id and content.

A user can create many conversations, and a conversation can have many users, the UserToConversation table is used to storing this mapping info.

For example, when user_1 wants to have a conversation with user_2 and user_3 in a group chat, a conversation record will be created first, and three UserToConversation records would be created subsequently.

Here, the "users" column in the Conversation table is a string for recording all the users' IDs in a conversation. It could be used for eliminating duplicated conversations.

Connecting Postgres with Sequelize

I used to use Knex for this type of work, but when I learned Sequelize, I forget about Knex right away, as well as the trauma when setting up the Knex environment.

File structure

├── models
│   ├── index.js
│   ├── addAssociations.js
│   ├── syncModels.js
│   ├── user.model.js
│   └── conversation.model.js
    .
    .

First, for initializing the Sequlize instance, we could setup a 'models' folder and have an index.js file as such:

const { Sequelize } = require("sequelize");
const { addAssociations } = require("./addAssociations");
const { syncModels } = require("./syncModels");

const sequelize = new Sequelize(
  process.env.DB_NAME,
  process.env.DB_USERNAME,
  process.env.DB_PASSWORD,

  {
    host: process.env.DB_HOST,
    dialect: "postgres",
    operatorsAliases: false,

    pool: {
      max: 5,
      min: 0,
      acquire: 30000,
      idle: 10000,
    },
  }
);

const modelDefiners = [
  require("./user.model"),
  require("./conversation.model"),
  require("./message.model"),
  require("./userToConversation.model"),
];

for (const modelDefiner of modelDefiners) {
  modelDefiner(sequelize);
}

addAssociations(sequelize);
syncModels(sequelize);

//test the database connection
sequelize
  .authenticate()
  .then(() => console.log("Postgres Connected!"))
  .catch((err) => console.error(err));

module.exports = sequelize;

Once the connection is up, we could add tables and associations. Sequelize will take care of setting up foreign keys.

For users in 'users.modules.js':

const { DataTypes } = require("sequelize");

module.exports = (sequelize) => {
  sequelize.define("user", {
    username: {
      type: DataTypes.STRING,
      allowNull: false,
    },
    email: {
      type: DataTypes.STRING,
      allowNull: false,
      isEmail: true,
    },
    password: {
      type: DataTypes.STRING,
      allowNull: false,
    },
    photoURL: { type: DataTypes.STRING, allowNull: true },
  });
};

For conversations in 'conversation.model.js':

const { DataTypes } = require("sequelize");

module.exports = (sequelize) => {
  sequelize.define("conversation", {
    users: {
      type: DataTypes.STRING,
      unique: true,
    },
  });
};

For UserToConversation in 'userToConversation.module.js':

module.exports = (sequelize) => {
  sequelize.define("userToConversation");
};

For Messages in 'message.model.js':

const { DataTypes } = require("sequelize");

module.exports = (sequelize) => {
  sequelize.define("message", {
    content: {
      type: DataTypes.STRING,
      allowNull: false,
    },
    currentChatReceiverId: {
      type: DataTypes.INTEGER,
      allowNull: false,
    },
  });
};

For associations in 'addAssociations.js':

function addAssociations(sequelize) {
  const { user, conversation, message, userToConversation } = sequelize.models;

  user.hasMany(userToConversation);
  userToConversation.belongsTo(user);

  conversation.hasMany(userToConversation);
  userToConversation.belongsTo(conversation);

  conversation.hasMany(message);
  message.belongsTo(conversation);

  user.hasMany(message);
  message.belongsTo(user);
}

module.exports = { addAssociations };

At last, we will need to sync tables with the posgres server in 'syncModels.js':

const syncModels = async (sequelize) => {
  const { user, conversation, message, userToConversation } = sequelize.models;

  try {
    await user.sync();
    await conversation.sync();
    await userToConversation.sync();
    await message.sync();
    console.log("synced");
  } catch (error) {
    console.error(error);
  }
};

module.exports = { syncModels };

The modelling part is done, and the models will be used in the routes for querying data and so on.

Thanks for reading.

Making concurrent API calls in Node

Yue Su — Thu, 24 Dec 2020 01:30:01 +0000

The problem

When building up a backend API, it is common that we need to fetch data from a third-party API, clean, format and merge them, and then forward it to the front-end.

For instance, NASA's public could be used to fetch
APOD (Astronomy Photo of the Day) with any given date. However, it doesn't support fetching multiple photos with a range of dates. Now suppose we were asked to build a backend API that can return a list of APOD with a given number of days, what should we do?

The first thought I came up with is to generate an array that contains a range of dates. Then I can do a forEach method or a for loop to iterate through the array and make API calls one by one, get the data, push it into a result array, and finally return the result to the front-end. However, even this would work, it doesn't align with the goal, which requires to do the calls concurrently. Using forEach or a for loop still would do the job in order, not simultaneously. It's slow and not efficient.

After a little bit of research, I came across a library called async that perfectly fulfills the requirement of the task. The async library provides various types of functions for working with asynchronous JavaScript.

In this example, the method will be using is parallel, and it's mainly for flow control:

parallel(tasks, callback)

It allows us to run a number of tasks in parallel, without waiting until the previous function has completed. The results are passed to the callback as an array.

Let's get started.

The solution

First, we need to make a helper function, it takes the number of days as a parameter, and returns an array of dates. NASA's API can only take the date format as YYYY-MM-DD, so for example, if today's date is 2020-12-23, and the number of days is equal to 6, the returned array will be:

[
  '2020-12-18',
  '2020-12-19',
  '2020-12-20',
  '2020-12-21',
  '2020-12-22',
  '2020-12-23'
]

Here is what the function looks like:

function generatedates(numberOfDays) {
  const result = []
  const today = new Date()

  for (let i = 0; i < numberOfDays; i++) {
    let date = new Date(today)
    date.setDate(today.getDate() - i)
    let dd = date.getDate()
    let mm = date.getMonth() + 1
    let yyyy = date.getFullYear()

    if (dd < 10) {
      dd = "0" + dd
    }
    if (mm < 10) {
      mm = "0" + mm
    }
    date = yyyy + "-" + mm + "-" + dd
    result.unshift(date)
  }

  return result
}

Then we need to add an endpoint to the node server.

/api/photos

The parallel function takes an array of function as the first argument, so we could use the map method to iterate through the dates array and returns the function array. Each function in the array fires an Axios call to the NASA API and get the picture of that date.

The second argument of the parallel function is a callback function. In this case, since the API calls return promises, the callback function will return two items. The first one is the possible error, and the second one is the array of the result.

If we don't need to further process the data, we can simply pass them to the front-end. We can also use the forEach method to clean the data and only extract the information we need.

Here is the logic of the endpoint:

const URL = "https://api.nasa.gov/planetary/apod"

server.get("/api/photos", (req, res) => {
  const days = req.query.days
  const dates = generateDates(days)

  const functionArray = dates.map((date) => {
    return async function () {
      const data = await axios.get(`${URL}?api_key=${api_key}&date=${date}`)
      return data.data
    }
  })

  async.parallel(functionArray, (err, result) => {
    res.status(200).json({ items: result.length, photos: result })
  })
})

Now the user can make an API request to fetch any number of photos, such as:

//fetch photos of the past week
api/photos?days=7

//fetch photos of the past month
api/photos?days=30

And the result will be shown as:

{
    "items": 6,
    "photos": [...]
}

Code

Check the GitHub repo for this example
Repo