DEV Community: Batkhuu

Install Docker and Docker Compose on Linux

Batkhuu — Thu, 07 Apr 2022 08:38:05 +0000

Official pages

Install Docker Engine on Ubuntu: https://docs.docker.com/engine/install/ubuntu/#installation-methods
Docker install script for most Linux distros: https://github.com/docker/docker-install
NVIDIA Docker installation guide for most Linux distros: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#setting-up-nvidia-container-toolkit
Post-installation steps for Linux: https://docs.docker.com/engine/install/linux-postinstall
Install Docker Compose: https://docs.docker.com/compose/install/#install-compose
Docker Compose github releases: https://github.com/docker/compose/releases
Compose V2: https://docs.docker.com/compose/cli-command/#install-on-linux

1. Install Docker

1.1. Install Docker with the script

# Download docker installer script:
curl -fsSL https://get.docker.com -o get-docker.sh
# Install docker by 'get-docker.sh' script:
DRY_RUN=1 sudo sh get-docker.sh

# Remove downloaded script:
rm -vrf get-docker.sh

# To avoid using 'sudo' for docker commands:
# Create a new 'docker' group:
sudo groupadd docker
# Add current user to the 'docker' group:
sudo usermod -aG docker $(whoami)

# Apply new group changes to the new shell session:
newgrp docker
# Or reboot the system to apply docker group changes:
sudo shutdown -r now

# Configure docker to start on reboot:
sudo systemctl enable docker.service
sudo systemctl enable containerd.service

# Check docker is installed and running:
docker -v
docker info
docker images

1.2. [Only for GPU] Install NVIDIA Docker

Skip this step, if you don't have NVIDIA GPU!
[REQUIRED] Install NVIDIA GPU driver: https://github.com/bybatkhuu/wiki/blob/main/posts/manuals/installs/nvidia-driver-linux.md
For Ubuntu/Debian-based Linux distros:

## [IMPORTANT] Only for Ubuntu/Debian-based linux
# Download the nvidia-docker GPG key and setup the stable repository:
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

# Update repository list:
sudo apt-get update
# Install nvidia-docker2:
sudo apt-get install -y nvidia-docker2

Edit the /etc/docker/daemon.json file:

sudo nano /etc/docker/daemon.json

and add the following default-runtime into the /etc/docker/daemon.json file:

{
    "default-runtime": "nvidia"
}

For example:

{
    "default-runtime": "nvidia",
    "runtimes":
    {
        "nvidia":
        {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

save changes and exit from the file editor.

# Restart docker service:
sudo systemctl daemon-reload
sudo systemctl restart docker.service

2. [RECOMMENDED] Limit docker log file max size and max rotation

Edit the /etc/docker/daemon.json file:

sudo nano /etc/docker/daemon.json

and add the following log-opts into the /etc/docker/daemon.json file:

{
    "log-opts":
    {
        "max-size": "10m",
        "max-file": "10"
    }
}

save changes and exit from the file editor.

# Restart docker service:
sudo systemctl restart docker.service

3. Install Docker Compose V2

## Download docker-compose binary release:
# [RECOMMENDED] Compose v2.2.3 (Linux) release:
sudo curl -L "https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-$(uname -m)" -o /usr/local/bin/docker-compose
# [For old version] Compose v1.29.2 (Linux) release:
# sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
# [Only for experimental] Latest version release:
# sudo curl -L $(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep browser_download_url | grep linux-$(uname -m)\" | cut -d '"' -f 4) -o /usr/local/bin/docker-compose

# Apply executable permission to the docker-compose binary:
sudo chmod +x /usr/local/bin/docker-compose

# Create a symbolic link to '/usr/bin':
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

# Check docker-compose version:
docker-compose --version

👍 ✨

INSTALLATION COMPLETE

[OPTIONAL] Change the docker data directory to the external storage

# Stop the docker service:
sudo systemctl stop docker.service

Edit the /etc/docker/daemon.json file:

sudo nano /etc/docker/daemon.json

and add the following data-root into the /etc/docker/daemon.json file (don't forget to change [NEW_STORAGE_PATH] to your new directory):

{
   "data-root": "[NEW_STORAGE_PATH]/docker/data-root"
}

For example:

{
   "data-root": "/mnt/ssd/1/docker/data-root"
}

save changes and exit from the file editor.

# Copy the current docker data directory to the new directory:
sudo rsync -a /var/lib/docker/ [NEW_STORAGE_PATH]/docker/data-root
# For example:
sudo rsync -a /var/lib/docker/ /mnt/ssd/1/docker/data-root

# Rename the old docker data directory as a backup:
sudo mv /var/lib/docker /var/lib/docker.old

# Start docker service:
sudo systemctl start docker.service

# Test docker is working properly:
docker -v
docker info
docker images
docker ps -a

# Remove the old backup docker data directory:
sudo rm -rf /var/lib/docker.old

[OPTIONAL] Install command auto-completion for docker-compose

# Download docker-compose auto-completion script into '/etc/bash_completion.d/':
sudo curl \
    -L https://raw.githubusercontent.com/docker/compose/1.29.2/contrib/completion/bash/docker-compose \
    -o /etc/bash_completion.d/docker-compose

# Reload current bash shell session:
source ~/.bashrc

References

Docker supported platforms (OS): https://docs.docker.com/engine/install
Install NVIDIA GPU driver: https://github.com/bybatkhuu/wiki/blob/main/posts/manuals/installs/nvidia-driver-linux.md
Change docker data root directory: https://tienbm90.medium.com/how-to-change-docker-root-data-directory-89a39be1a70b

Install PM2 (Process Manager 2)

Batkhuu — Thu, 07 Apr 2022 08:29:16 +0000

PM2 (Process Manager 2)

Process manager tool based on Node.js.
It's a tool that allows you to manage your applications (Node.js/python/binary/shell script/etc...).

Features

Start/stop/restart applications on demand
Easy to manage multiple processes
Run applications in the background
Monitor logs and resources usage
Restart applications on source code changes or failure
Startup on system reboot (service)
Environment based configuration
PM2 Plus - Advanced features (Web interface, cluster mode, etc...)

Official pages

PM2: https://pm2.keymetrics.io
Quick start guide: https://pm2.keymetrics.io/docs/usage/quick-start
Single page guide: https://pm2.keymetrics.io/docs/usage/pm2-doc-single-page
PM2 npm: https://www.npmjs.com/package/pm2
PM2 logrotate: https://www.npmjs.com/package/pm2-logrotate

Prerequisites

Node.js
- [RECOMMENDED] NVM: https://github.com/bybatkhuu/wiki/blob/main/posts/manuals/installs/nvm.md

Installation

# Install PM2 through NPM:
npm install -g pm2

# Install pm2-logrotate to rotate logs for PM2:
pm2 install pm2-logrotate

# Check PM2 is running and check the version:
pm2 ping
pm2 -v

👍 ✨

INSTALLATION COMPLETE

Usage

### Run application:

## Start node.js application in the background:
pm2 start app.js

## Start shell script app with naming "script-app":
pm2 start script.sh -n script-app

## Start the python application and watch for changes:
pm2 start python-app.py --watch

## Start binary file with arguments (--port 8080):
pm2 start binary-file -- --port 8080

## Use ecosystem configuration file:
# Generate a simple ecosystem.config.js file:
pm2 init simple
# Update the ecosystem.config.js file according to your needs:
vi ecosystem.config.js
# Start ecosystem.config.js:
pm2 start ecosystem.config.js

## Or use JSON file and start in the foreground:
pm2 start pm2-process.json --no-daemon

## Or use YAML file and specify log file path:
pm2 start pm2-process.yml --log /path/to/logs


### Check logs:
pm2 logs app-name --lines 50


### Restart application:
pm2 restart app-name


### Shows a list of applications registered into PM2:
pm2 list


### Monitor applications:
pm2 monit


### Stop and delete the application:
pm2 stop app-name
pm2 del app-name


### Generate ecosystem.config.js file:
pm2 ecosystem


## Startup application on system reboot:
pm2 startup
#--------------------------------------------------------------------------------
## Run according to shown command!
# For example:
# [PM2] You have to run this command as root. Execute the following command:
#      $ sudo su -c "env PATH=$PATH:/home/user/.nvm/versions/node/v12.20.1/bin pm2 startup <distribution> -u <user> --hp <home-path>"
#--------------------------------------------------------------------------------
pm2 save


### Help:
pm2 --help

PM2 plus

PM2 plus overview: https://pm2.io/docs/plus/overview

References

Ecosystem file: https://pm2.keymetrics.io/docs/usage/application-declaration
Startup script: https://pm2.keymetrics.io/docs/usage/startup
Logs: https://pm2.keymetrics.io/docs/usage/log-management
Monitoring: https://pm2.keymetrics.io/docs/usage/monitoring
Custom metrics: https://pm2.keymetrics.io/docs/usage/process-metrics
Cluster mode: https://pm2.keymetrics.io/docs/usage/cluster-mode
Automate your python script with PM2: https://towardsdatascience.com/automate-your-python-script-with-pm2-463238ea0b65

Install Miniconda (Conda) and Python

Batkhuu — Thu, 07 Apr 2022 08:29:09 +0000

Miniconda (Conda)

The clean and light version of the anaconda.

Features

Cross-platform
Python
Conda
Miniforge
Multiple environments

Official pages

Miniconda: https://docs.conda.io/en/latest/miniconda.html
Install on Linux: https://conda.io/projects/conda/en/latest/user-guide/install/linux.html
Miniforge: https://github.com/conda-forge/miniforge

Install on Linux, macOS, or Windows

1. Download and install Miniconda (v3)

For Windows: https://docs.conda.io/en/latest/miniconda.html
For Linux or macOS:

## Recommended path to install Miniconda:
# For example: ${HOME}/workspaces/runtimes/miniconda3

## Linux (x86_64):
wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.11.0-Linux-x86_64.sh
bash Miniconda3-py38_4.11.0-Linux-x86_64.sh

## Linux (aarch64):
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
bash Miniforge3-Linux-aarch64.sh

## macOS (Intel):
wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.11.0-MacOSX-x86_64.sh
bash Miniconda3-py38_4.11.0-MacOSX-x86_64.sh

## macOS (Apple M1):
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh


# Remove downloaded file:
rm -vrf Mini*.sh

# For bash:
# Load .bashrc file to init conda into current bash session:
source ~/.bashrc

# For zsh:
# Load .zshrc file to init conda into current zsh session:
source ~/.zshrc


# Clean conda caches:
conda clean -y -av

# Update conda to the latest version:
conda update -y conda

# Check installed conda version:
conda -V

2. Install Python

# Create a new conda environment with python and pip:
conda create -y -n py38 python=3.8.12 pip

# Set default conda environment to .bashrc:
## Linux:
echo "conda activate py38" >> ~/.bashrc

## macOS:
echo "conda activate py38" >> ~/.zshrc

# Activate new conda environment:
conda activate py38

# Upgrade pip to the latest version:
pip install -U pip
pip install pytest

# Clean pip caches:
pip cache purge

# Clean conda caches:
conda clean -y -av

# Check installed python and pip version:
python -V
pip -V

👍 ✨

INSTALLATION COMPLETE

Usage

## Viewing a list of the packages in a conda environment:
conda list

# Or list packages of other conda environment:
conda list -n py38


## Viewing a list of conda environments:
conda env list


## Creating environment variables associated with Conda environments:
conda env config vars set PYTHONPATH="${PWD}:${PYTHONPATH}"


## Exporting conda environment to environment.yml file:
conda env export > environment.yml


## Creating conda environment from environment.yml file:
conda env create -f environment.yml


## Cloning a conda environment:
conda create -y -n py38_clone --clone py38


## Removing conda environment:
conda remove -y -n py38 --all


## Conda help:
conda -h

References

Miniconda installation: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
Managing conda environments: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

Install NVM (Node Version Manager) and Node.js

Batkhuu — Thu, 07 Apr 2022 08:27:46 +0000

NVM (Node Version Manager)

NVM is a tool for managing multiple versions of Node.js.

Features

Cross-platform
Node.js
Manager for multiple versions of Node.js, npm, and yarn
Multiple Node.js environments

Official pages

NVM: https://github.com/nvm-sh/nvm
NVM for Windows: https://github.com/coreybutler/nvm-windows

Install on Linux or macOS

1. Download and install NVM (v0.39.1)

## Recommended path to install NVM:
# Set and export `NVM_DIR` environment variable. For example:
mkdir -vp ~/workspaces/runtimes/.nvm
export NVM_DIR="${HOME}/workspaces/runtimes/.nvm"

# Install NVM:
wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash


## For bash:
# Load .bashrc file to init NVM into current bash session:
source ~/.bashrc

## For zsh:
# Load .zshrc file to init NVM into current zsh session:
source ~/.zshrc


# Check installed NVM version:
nvm --version

2. Install Node.js

# Install Node.js, update NPM to latest, and set default Node.js:
nvm install --latest-npm --alias=default 16.14.2
# Set to use default Node.js:
nvm use default

# Clean NVM caches:
nvm cache clear

# Check installed Node.js and NPM version:
node -v
npm -v

# Install essential extra packages:
npm install -g pm2 newman jshint
pm2 install pm2-logrotate

# Clean NPM caches:
npm cache clean --force

👍 ✨

INSTALLATION COMPLETE (Linux, macOS)

Install on Windows

1. Download and install NVM-Windows (v1.1.9)

Downloads: https://github.com/coreybutler/nvm-windows/releases

2. Install Node.js (Windows)

nvm install 16.14.2
nvm use 16.14.2

👍 ✨

INSTALLATION COMPLETE (Windows)

Usage

# NVM help:
nvm -h

References

Install NVM on Ubuntu 18.04: https://nbanzyme.medium.com/easy-way-to-install-nvm-on-ubuntu-18-04-2cfb19ee5391
Install NVM macOS: https://nodesource.com/blog/installing-node-js-tutorial-using-nvm-on-mac-os-x-and-ubuntu/

Install NVIDIA CUDA on Linux

Batkhuu — Thu, 07 Apr 2022 08:27:45 +0000

Official pages

CUDA: https://developer.nvidia.com/cuda-toolkit
CUDA archive: https://developer.nvidia.com/cuda-toolkit-archive
Install CUDA: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#runfile
cuDNN: https://developer.nvidia.com/cudnn
Install cuDNN: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#installlinux-tar
TensorRT: https://developer.nvidia.com/tensorrt-getting-started
Install TensorRT: https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-tar

Prerequisites

Check current Linux system specification

Check NVIDIA GPU exists:

## Use lshw:
lshw -C display

# Only for Ubuntu/Debian:
sudo apt-get update --fix-missing -o Acquire::CompressionTypes::Order::=gz
sudo apt-get install -y pciutils mesa-utils
# Or use lspci:
lspci | grep -i vga && lspci | grep -i nvidia
# Or use glxinfo:
glxinfo | egrep "OpenGL vendor|OpenGL renderer"

Check linux OS information:

## Check linux OS version and distro:
uname -a && cat /etc/*release

## Check kernel version:
uname -r
# Or:
hostnamectl

## Check GCC and make compiler version:
gcc --version && make -v

## Check glibc version:
ldd --version

Check CUDA version compatibility

[IMPORTANT] Search and check the latest stable NVIDIA driver version for your GPU model(s):
- NVIDIA driver downloads: https://www.nvidia.com/Download/Find.aspx?lang=en-us
[IMPORTANT] Check the compatibility of the NVIDIA driver version with CUDA versions:
- CUDA release note: https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html
- CUDA compatibility PDF: https://docs.nvidia.com/pdf/CUDA_Compatibility.pdf
- CUDA compatibility: https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility
[IMPORTANT] Check system requirements for CUDA versions:
- CUDA latest version: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#system-requirements
- CUDA older versions archive: https://developer.nvidia.com/cuda-toolkit-archive
[RECOMMENDED] Check the compute capability of your GPU model(s):
- GPU compute capability: https://developer.nvidia.com/cuda-gpus
- CUDA Wikipedia: https://en.wikipedia.org/wiki/CUDA
[RECOMMENDED] Check the cuDNN version with compute capability of your GPU model(s):
- cuDNN support matrix: https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html
[OPTIONAL] Check the compatibility of TensorRT versions with CUDA versions and cuDNN versions:
- TensorRT latest version: https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html
- TensorRT older versions archives: https://docs.nvidia.com/deeplearning/tensorrt/archives/index.html
[RECOMMENDED] Check compatible CUDA versions for required machine learning/deep learning frameworks/libraries you want to use:
- Tensorflow-gpu: https://www.tensorflow.org/install/source#gpu
- Pytorch: https://pytorch.org/get-started/previous-versions
- Etc...

Install NVIDIA GPU driver

[REQUIRED] Install NVIDIA GPU Driver: https://github.com/bybatkhuu/wiki/blob/main/posts/manuals/installs/nvidia-driver-linux.md

Install CUDA toolkit

Download CUDA

IMPORTANT! Download and use the .RUN file! It can prevent installing incompatible NVIDIA drivers and prevent overwriting the previous /usr/local/cuda (if you installed another CUDA already) symbolic link.
IMPORTANT! Don't download or use the .DEB file! It will overwrite the current default CUDA (if you installed it already) directory (/usr/local/cuda) and also it will overwrite the already installed latest compatible NVIDIA driver version into the old or incompatible driver. Moreover, if you want to use a different version of CUDA, never install CUDA.DEB file!
Search and download the required version of CUDA (cuda_***_linux.run) installer file from this link: https://developer.nvidia.com/cuda-toolkit-archive
If you're using a command-line interface, just copy the download link from the NVIDIA website and use wget to download. For example:

mkdir -v ~/Downloads
cd ~/Downloads
# 1. (Skip this, if you've already downloaded) Download CUDA .run installer file by wget:
wget https://developer.download.nvidia.com/compute/cuda/[CUDA_VERSION]/local_installers/cuda_[CUDA_VERSION]_[DRIVER_VERSION]_linux.run
# For example:
wget https://developer.download.nvidia.com/compute/cuda/11.0.3/local_installers/cuda_11.0.3_450.51.06_linux.run

Install CUDA.run file

When installing CUDA.run, follow the below steps:

Type 'accept' and ENTER.
IMPORTANT! Uncheck and disable NVIDIA driver installation to prevent overwriting already installed latest compatible version of NVIDIA driver into an old or incompatible driver.
IMPORTANT! Uncheck and disable CUDA symbolic link installation to prevent overwriting the current default CUDA (if you installed it already) directory (/usr/local/cuda).
Choose install and press it!

# Change directory to cuda_***.run file downloaded directory. For example:
cd ~/Downloads

# 2. IMPORTANT! Install cuda_***.run file:
sudo sh cuda_[CUDA_VERSION]_[DRIVER_VERSION]_linux.run
# For example:
sudo sh cuda_11.0.3_450.51.06_linux.run

# 3. Remove downloaded file:
rm -vrf cuda_[CUDA_VERSION]_[DRIVER_VERSION]_linux.run
# For example:
rm -vrf cuda_11.0.3_450.51.06_linux.run

Post-installation

Edit the ~/.bashrc file:

# 4. Add CUDA environment variables to current user .bashrc file:
# (It loads environment variables, whenever current user opens a new bash session/new terminal)
nano ~/.bashrc

and add the following lines into the ~/.bashrc file (don't forget to change [CUDA_VERSION] into the installed version):

## CUDA ##
# Change to installed CUDA version:
# export CUDA_VERSION=[CUDA_VERSION]
export CUDA_VERSION=11.0
export CUDA_HOME="/usr/local/cuda-${CUDA_VERSION}"
export CUDA_PATH="${CUDA_HOME}"
export PATH="${CUDA_PATH}/bin:${PATH}"
export LIBRARY_PATH="${CUDA_PATH}/lib64:${LIBRARY_PATH}"
export LD_LIBRARY_PATH="${CUDA_PATH}/lib64:${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH="${CUDA_PATH}/extras/CUPTI/lib64:${LD_LIBRARY_PATH}"
export NVCC="${CUDA_PATH}/bin/nvcc"
export CFLAGS="-I${CUDA_PATH}/include ${CFLAGS}"

save changes and exit from the file editor.

# 5. Load .bashrc file to init environment variables into the current bash session:
source ~/.bashrc

# 6. Check CUDA compiler version:
nvcc -V
# Or:
cat ${CUDA_PATH}/version.txt
# Or:
cat ${CUDA_PATH}/version.json

[RECOMMENDED] Install cuDNN (For Deep Learning)

Download cuDNN

IMPORTANT! Download and use .TGZ or .TAR.XZ file! You can install cuDNN into a custom directory, which means you can control which version of cuDNN you want to use and where to install it.
IMPORTANT! Don't download and use .DEB file! It will overwrite the cuDNN library into the current default CUDA directory (/usr/local/cuda).
Download cuDNN .TGZ or .TAR.XZ file from this link: https://developer.nvidia.com/rdp/cudnn-download

Install cuDNN .tgz or .tar.xz package files

# Change directory to cudnn_***.tgz or .tar.xz file downloaded directory. For example:
cd ~/Downloads

## Extract cuDNN .tgz or .tar.xz package file:
# New versions:
tar -xf cudnn-linux-x86_64-[CUDNN_VERSION]_cuda[CUDA_VERSION]-archive.tar.xz
# For example:
tar -xf cudnn-linux-x86_64-8.3.1.22_cuda11.5-archive.tar.xz

# Or older versions:
tar -xzf cudnn-[CUDA_VERSION]-linux-x64-v[CUDNN_VERSION].tgz
# For example:
tar -xzf cudnn-11.0-linux-x64-v8.0.5.39.tgz


# Copy extracted files into specific CUDA directory:
sudo cp -v cud*/include/cudnn*.h /usr/local/cuda-${CUDA_VERSION}/include
sudo cp -v cud*/lib*/libcudnn* /usr/local/cuda-${CUDA_VERSION}/lib64

# Add read permissions to extracted files:
sudo chmod a+r /usr/local/cuda-${CUDA_VERSION}/include/cudnn*.h /usr/local/cuda-${CUDA_VERSION}/lib64/libcudnn*

# Remove downloaded files:
rm -rf cud*

# Check installed cuDNN version:
cat /usr/local/cuda-${CUDA_VERSION}/include/cudnn.h | grep CUDNN_MAJOR -A 2
# Or:
cat /usr/local/cuda-${CUDA_VERSION}/include/cudnn_version.h | grep CUDNN_MAJOR -A 2

# cuDNN installation complete!

[OPTIONAL] Install TensorRT

Download TensorRT

IMPORTANT! Download and use the .TAR.GZ file! You can install TensorRT into a custom directory, which means you can control which version of TensorRT you want to use and where to install it.
IMPORTANT! Don't download and use .DEB file! You can't control TensorRT libraries.
Download the cuDNN .TAR.GZ file from this link: https://developer.nvidia.com/nvidia-tensorrt-download

Install TensorRT .tar.gz package files

# Change directory to TensorRT-***.tar.gz file downloaded directory. For example:
cd ~/Downloads

# Extract TensorRT .tar.gz package file:
tar -xzf TensorRT-[TENSORRT_VERSION].[OS_DISTRO].x86_64-gnu.cuda-[CUDA_VERSION].cudnn[CUDNN_VERSION].tar.gz
# For example:
tar -xzf TensorRT-7.2.0.14.Ubuntu-18.04.x86_64-gnu.cuda-11.0.cudnn8.0.tar.gz

# Create TensorRT base directory:
sudo mkdir -vp /opt/tensorrt

# Copy extracted files into TensorRT directory:
sudo mv -vf TensorRT-[TENSORRT_VERSION] /opt/tensorrt/TensorRT-[TENSORRT_VERSION]
# For example:
sudo mv -vf TensorRT-7.2.0.14 /opt/tensorrt/TensorRT-7.2.0.14

# Remove downloaded file:
rm -vrf TensorRT-[TENSORRT_VERSION].[OS_DISTRO].x86_64-gnu.cuda-[CUDA_VERSION].cudnn[CUDNN_VERSION].tar.gz
# For example:
rm -vrf TensorRT-7.2.0.14.Ubuntu-18.04.x86_64-gnu.cuda-11.0.cudnn8.0.tar.gz

Edit the ~/.bashrc file:

# Add TensorRT environment variables into current user .bashrc file:
# (It loads environment variables, whenever current user opens a new bash session/new terminal)
nano ~/.bashrc

and add the following lines into the ~/.bashrc file (don't forget to change [TENSORRT_VERSION] into the installed version):

## TensorRT ##
# Change to your installed TensorRT version:
# export TENSORRT_VERSION=[TENSORRT_VERSION]
export TENSORRT_VERSION=7.2.0.14
export TENSORRT_PATH="/opt/tensorrt/TensorRT-${TENSORRT_VERSION}"
export PATH="${TENSORRT_VERSION}/bin:${PATH}"
export LD_LIBRARY_PATH="${TENSORRT_PATH}/lib:${LD_LIBRARY_PATH}"

save changes and exit from the file editor.

# Load .bashrc file to init environment variables into current bash session:
source ~/.bashrc

# (Only for Python) Install TensorRT python packages:
cd ${TENSORRT_PATH}/python
pip install tensorrt-[TENSORRT_VERSION]-cp[PYTHON_VERSION]-none-linux_x86_64.whl
# For example:
pip install tensorrt-7.2.0.14-cp38-none-linux_x86_64.whl

cd ${TENSORRT_PATH}/uff
pip install uff-*-py2.py3-none-any.whl

cd ${TENSORRT_PATH}/graphsurgeon
pip install graphsurgeon-*-py2.py3-none-any.whl

cd ${TENSORRT_PATH}/onnx_graphsurgeon
pip install onnx_graphsurgeon-*-py2.py3-none-any.whl

# TensorRT installation complete!

Test CUDA

# Install sample source codes to test CUDA:
cuda-install-samples-[CUDA_VERSION].sh ~
# For example:
cuda-install-samples-11.0.sh ~

# Change directory to CUDA sample source codes installed directory:
cd ~/NVIDIA_CUDA-[CUDA_VERSION]_Samples
# For example:
cd ~/NVIDIA_CUDA-11.0_Samples

# Build sample source codes:
make clean && make -j8

# Change directory to built binary directory:
cd bin/x86_64/linux/release

# Run CUDA test binaries:
./deviceQuery
./bandwidthTest

cd ~
# Remove sample source codes:
rm -rf ~/NVIDIA_CUDA-[CUDA_VERSION]_Samples
# For example:
rm -rf ~/NVIDIA_CUDA-11.0_Samples

👍 ✨

INSTALLATION COMPLETE

Install NVIDIA GPU driver on Linux (Ubuntu/Debian)

Batkhuu — Thu, 07 Apr 2022 08:23:31 +0000

Official pages

NVIDIA driver downloads: https://www.nvidia.com/Download/Find.aspx?lang=en-us
NVIDIA driver installation: https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html#runfile
NVIDIA driver configuration instructions: https://download.nvidia.com/XFree86/Linux-x86_64/510.47.03/README/installdriver.html
NVIDIA persistence: https://docs.nvidia.com/deploy/driver-persistence/index.html#persistence-daemon
NVIDIA persistenced github: https://github.com/NVIDIA/nvidia-persistenced

Install NVIDIA GPU driver

Disable automatic updates on Ubuntu/Debian

# 1. Disable auto-update and auto-upgrade:
echo 'APT::Periodic::Update-Package-Lists "0";' | sudo tee /etc/apt/apt.conf.d/20auto-upgrades
echo 'APT::Periodic::Unattended-Upgrade "0";' | sudo tee -a /etc/apt/apt.conf.d/20auto-upgrades

Disable open-source nouveau driver

# 2. Write blacklist options for the nouveau driver (disable):
echo "blacklist nouveau" | sudo tee -a /etc/modprobe.d/blacklist-nouveau.conf
echo "options nouveau modeset=0" | sudo tee -a /etc/modprobe.d/blacklist-nouveau.conf

# 3. Update kernel initramfs for blacklist (modprobe.d) options:
sudo update-initramfs -u

Install essential packages

# 4. Clean apt package manager lists and caches:
sudo rm -vrf /var/lib/apt/lists/* /var/cache/apt/archives/* /tmp/*
sudo apt-get clean -y

# 5. Updating, indexing, and caching apt package manager app lists:
sudo apt-get update --fix-missing -o Acquire::CompressionTypes::Order::=gz

# 6. Upgrade and reinstall OS base packages:
sudo apt-get upgrade -y

# 7. Remove old or unnecessary leftover packages:
sudo apt-get autoremove -y

# 8. IMPORTANT! Install dkms and base packages:
sudo apt-get install -y build-essential pkg-config libglvnd-dev mesa-utils wget dkms

Download the appropriate Linux driver for the current NVIDIA GPU model

Search and download the latest verified stable NVIDIA GPU driver (NVIDIA-Linux-x86_64-***.run) installer file from this link: https://www.nvidia.com/Download/Find.aspx?lang=en-us
IMPORTANT! Download and use the .RUN file! It can install the NVIDIA driver with the DKMS module, which is really important!
IMPORTANT! Don't download or use the .DEB file! It will BREAK the kernel eventually, especially when the kernel header/version/module is upgraded, which is so FRUSTRATING! Therefore, sometimes conflict with other application dependencies (dependency hell)...
If you're using a command-line interface, just copy the download link and use wget to download. For example:

mkdir -v ~/Downloads
cd ~/Downloads

# 9. (Skip this, if you've already downloaded) Download appropriate Linux driver for current NVIDIA GPU model
wget https://us.download.nvidia.com/[GPU_MODEL_TYPE_URL]/NVIDIA-Linux-x86_64-[DRIVER_VERSION].run
# For example:
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/510.54/NVIDIA-Linux-x86_64-510.54.run

Preparation steps before installing NVIDIA GPU driver

[IMPORTANT] Change GUI environment to console environment

Skip this step to 11, if you're not using a GUI environment and already using a console environment or SSH
TIP: From here you're going to enter the console environment, so you need to write down these below commands on some text file to read from command-line or note on somewhere!
You can change the GUI environment to a virtual console environment by pressing these keyboard buttons: Ctrl + Alt + F1/F3
Or by entering the Linux command:

# 10. IMPORTANT! Entering into virtual console environment:
sudo chvt 1
# Or:
sudo chvt 3

Enter username and password to log in to a console environment

Stop GUI processes to prepare the installation of the NVIDIA GPU driver

# 11. Stop GUI service (lightdm/gdm/gdm3/kdm/slim/etc..):
sudo service lightdm stop
# Or:
sudo service gdm stop
# Or:
sudo service gdm3 stop

# 12. Initialize and refresh console environment without X11 (GUI) processes:
sudo init 3

# Change directory to NVIDIA-Linux-x86_64-***.run driver downloaded directory. For example:
cd ~/Downloads

# 13. Change permission of NVIDIA-Linux-x86_64-***.run driver installer file to executable:
chmod +x ./NVIDIA-Linux-x86_64-[DRIVER_VERSION].run
# For example:
chmod +x ./NVIDIA-Linux-x86_64-510.54.run

Install NVIDIA driver.run file

IMPORTANT! (--dkms) Register Nvidia driver to the DKMS for future kernel upgrade.
IMPORTANT! (--no-x-check) Disable the X11 config checker to prevent triggering X11 (GUI) processes while installing the driver.
IMPORTANT! (--no-cc-version-check) Disable the C/C++ version checker to prevent install error occurs when the DKMS C/C++ version is not matching with the current installed C/C++ version!

# 14. IMPORTANT! Install nvidia driver with appropriate arguments:
sudo ./NVIDIA-Linux-x86_64-[DRIVER_VERSION].run --no-x-check --no-cc-version-check --dkms
# For example:
sudo ./NVIDIA-Linux-x86_64-510.54.run --no-x-check --no-cc-version-check --dkms

Select 'Continue installation' on the warning message.
IMPORTANT! Select 'Yes' for the register of the DKMS kernel module.
Select 'No' for 32-bit compatibility installation OR select 'OK' when a 32-bit compatibility error appears.
IMPORTANT! Select 'No' when it asks to automatically update your X configuration file.

# 15. Remove downloaded file:
rm -vf ./NVIDIA-Linux-x86_64-[DRIVER_VERSION].run
# For example:
rm -vf ./NVIDIA-Linux-x86_64-510.54.run

# 16. Reboot system:
sudo shutdown -r now

# 17. Check nvidia driver is installed:
nvidia-smi

Post-installation

[Only for ML/DL server] Setup nvidia-persistence mode daemon

Not recommended: GPU that is used by X11/Wayland processes (for GUI rendering).

# Install git to clone:
sudo apt-get install -y ssh git

# Download nvidia-persistenced source code from github:
git clone https://github.com/NVIDIA/nvidia-persistenced.git

# Install nvidia-persistenced service daemon:
cd nvidia-persistenced/init
sudo ./install.sh

# Remove downloaded files:
cd ../.. && rm -rf nvidia-persistenced

# Check nvidia-persistence mode is ON:
nvidia-smi
# Or check nvidia-persistenced.service is running:
systemctl status nvidia-persistenced.service

👍 ✨

INSTALLATION COMPLETE

[OPTIONAL] Install the NVIDIA GPU monitoring tool

🌟📡 nvitop

# Install python3 and pip3 packages:
sudo apt-get install -y python3 python3-pip

# Install nvitop into system python3 packages:
sudo pip3 install nvitop

# Check nvitop:
nvitop -m full

📊 nvtop

# Install cmake, ncurses and git:
sudo apt-get install -y cmake libncurses5-dev libncursesw5-dev

# Download nvtop source code from github:
git clone https://github.com/Syllo/nvtop.git
mkdir -vp nvtop/build && cd nvtop/build

cmake ..
# If it errors with "Could NOT find NVML (missing: NVML_INCLUDE_DIRS)"
# try the following command instead, otherwise skip to the build with make:
#cmake .. -DNVML_RETRIEVE_HEADER_ONLINE=True

make
sudo make install

# Remove downloaded files:
cd ../.. && rm -rf nvtop

# Check nvtop:
nvtop

https://github.com/Syllo/nvtop

Re-install or upgrade the NVIDIA driver to the latest version

Remove DKMS NVIDIA modules

Skip this step, if you didn't install the NVIDIA driver with DKMS.

# Check current dkms nvidia modules, versions, and kernels:
# [TIP]: Use module versions to remove.
dkms status | grep nvidia

# Remove old dkms nvidia module/version:
sudo dkms remove nvidia/[DRIVER_VERSION] --all
# For example:
sudo dkms remove nvidia/510.54 --all

Disable the nvidia-persistenced service daemon

Skip this step, if you didn't setup the nvidia-persistenced daemon.

# Stop and disable nvidia-persistenced service daemon:
sudo systemctl stop nvidia-persistenced.service
sudo systemctl disable nvidia-persistenced.service

Uninstall all nvidia-* packages

sudo apt-get remove --purge '^nvidia-.*'
# Or:
sudo nvidia-uninstall

# Reboot system:
sudo shutdown -r now

Install the latest NVIDIA driver version

Install NVIDIA driver again (start from the beginning): #install-nvidia-gpu-driver