The shared storage feature provides the real-time consistent read function of the standby machine, and the page exchange between the active and standby machines can be accelerated through RDMA. This chapter briefly describes how to enable the RDMA communication function in shared storage.
Preconditions
Enabling the RDMA function depends on the CX5 network card and the OCK RPC dynamic library.
Steps
Get the OCK RPC package.
Obtained from the designated openGauss interface person.
Deploy the OCK RPC package.
Decompress the OCK RPC package.
tar -zxf OCK_platform_rpc_22.0.0_EulerOS-aarch64.tar.gz
cd OCK_platform_rpc_22.0.0_EulerOS-aarch64_release
tar -zxf OCK_platform_rpc_EulerOS_aarch64.tar.gz
tar -zxf OCK_platform_rpc_sbin.tar.gz
cp rpc_sbin/ock_rpc_opengauss.sh ./
Modify the parameters of the deployment script ock_rpc_opengauss.sh.
Deployment user, which is the same as the opengauss user.
It is a non-root user. Ensure that this user exists on all servers.
USER_NAME="${USER}"
Deployment server, which can contain native
HOST_IP=(30.30.xx.7 30.30.xx.8)
Deployment package. The package name starts with OCK_platform_rpc.
PACKAGES="OCK_platform_rpc_aarch64_EulerOS.tar.gz"
Specifies the path from the SCP package to the server.
If the path does not exist, the path is automatically created.
PATH_TO_SCP="/home/ock/test"
Opengauss environment variable
GAUSSHOME="/home/ock/mppdb_temp_install"
OCK_RPC environment variable. Do not change
OCK_RPC_LIB_PATH="${GAUSSHOME}/lib"
The parameters involved in the script are as follows:
USER_NAME: The user name of the machine where the DN is located.
HOST_IP: cluster machine ip, including local machine.
PACKAGES: The obtained ock rpc binary package name (matches the architecture of the machine operating system).
PATH_TO_SCP: copy path.
GAUSSHOME: Consistent with the environment variable GAUSSHOME configured by openGauss.
OCK_RPC_LIB_PATH: The path where the ock rpc binary is located. The shared storage will be used by dynamic loading. Please make sure to configure it correctly.
Execute the deployment script ock_rpc_opengauss.sh.
sh ock_rpc_opengauss.sh
Description: Realize the function:
Complete the deployment, decompression and copying of the OCK RPC binary package in the cluster.
Configure the relevant environment variable $OCK_RPC_LIB_PATH.
Configure the UCX environment variable of the cluster node. (UCX is a communication component developed based on RDMA, and OCK RPC is a communication component developed on the basis of UCX and adapted to shared storage.)
Configure other parameter environment variables.
export UCX_ZCOPY_THRESH="32768"
export UCX_USE_MT_MUTEX=n
export UCX_NET_DEVICES=mlx5_1:1
Parameter Description:
UCX_ZCOPY_THRESH optimizes parameters to reduce memory application during RDMA transmission. Unit: byte. Default value: 32768. (optional parameter, default value suggested).
UCX_USE_MT_MUTEX optimization parameter, the type of data consistency lock during RDMA transmission, n: atomic lock; y: mutex lock. (Optional parameter, it is recommended to use atomic lock, which has better performance).
UCX_NET_DEVICES, UCX configuration parameter, used to specify the network card port, you can install the mlnx_ofed driver, and use the ibdev2netdev command to view the current RDMA active port. (It is recommended to specify an active port).
Note: After configuring the environment variables, execute source ~/.bashrc
Execute 1~5 to complete gs_initdb database building.
Modify the postgresql.conf configuration file.
Configure ss_interconnect_type=RDMA, add configuration items ss_rdma_work_config and ss_ock_log_path.
Example:
ss_interconnect_type=RDMA
ss_rdma_work_config ='6 10' （OCK RDMA使用用户态poll方式，并绑定cpu [6 10]，空格分开）
ss_ock_log_path=“/home/ock_test/log”(OCK RDMA 消息通信过程日志记录)
Execute the gs_ctl start command to pull the database process.
gs_ctl start -D /data/ss_test/dn_primary
Top comments (0)