HarmonyOS Private Repository Construction in Practice

Background

In Android and iOS development, dependencies and collaborations are often carried out in the form of binary products. Android uses Maven as the repository, and iOS has Pod as the repository. We can use open libraries provided by others on the officially provided platforms, which greatly improves everyone's development efficiency. However, some libraries related to a company's business do not want to be used by external personnel. Uploading them to external common repositories is not very safe. Many companies have built private repositories internally. On the one hand, it is more secure, and on the other hand, pushing and pulling products is usually faster.

HarmonyOS has the same issue. We can conveniently use open products from third parties on the official repository platform, but for a company's internal business, it still needs to rely on private repositories. This article builds its own private repository based on the tools provided by the official.

Introduction to HarmonyOS Shared Packages

HarmonyOS shared packages are divided into static shared packages and dynamic shared packages:

• HAR (Harmony Archive) is a static shared package that can contain code, C++ libraries, resources, and configuration files. Through HAR, multiple modules or multiple projects can share relevant codes such as ArkUI components and resources. HAR is different from HAP and cannot be independently installed and run on devices. It can only be referenced as a dependency of application modules.
• HSP (Harmony Shared Package) is a dynamic shared package. Static shared packages will be packaged into each dependent HAP, resulting in a large package size and repeated packaging of multiple copies of public resources and codes into the application. Dynamic shared packages allow multiple HAPs to share the same public resource code. HSP only supports sharing within an application and does not support cross-application sharing.

Building a Private Repository with the ohpm-repo Private Repository Tool

The official provides the ohpm-repo tool to help developers quickly build lightweight ohpm private repositories. It is compatible with the ohpm package manager and caches all dependencies on demand to accelerate installation in a private network.

ohpm-repo supports single-point deployment and multi-instance deployment:

• Single-point deployment: ohpm-repo is only deployed and used on one machine.
• Multi-instance deployment: ohpm-repo will be deployed on multiple machines with the same configuration content and shared data storage space.

Installation of the Dependent Environment

1. ohpm-repo depends on node to run and supports node.js 16.x and above versions. Nodejs needs to be installed first, and the environment needs to be configured. NodeJs can be downloaded from the official website (https://nodejs.org/download/release/latest/).

Downloading the ohpm-repo Tool

1. Download the ohpm-repo tool. The download address is: https://developer.huawei.com/consumer/cn/doc/harmonyos-guides/ide-software-download-0000001507075446
2. Unzip the ohpm-repo tool.

Configuring the ohpm-repo Environment Variables

1. Configure the path of the bin directory in the unzipped directory of the ohpm-repo tool package to the system environment variable path: export PATH=$OHPM-REPO-PATH/bin:$PATH
2. Execute the command ohpm-repo -v to view the version number and verify that the unzipped package is intact.

Configuration of the ohpm-repo Service Configuration File

Enter the conf directory in the unzipped directory, open the config.yaml file. The default configuration is as follows:

##### server configuration section #####
listen: 0.0.0.0:8088
# listen:
# - localhost:8088            # 监听本机环回地址
# - http://localhost:8088     # 监听本机环回地址
# - 0.0.0.0:8088              # 监听本机所有地址 (INADDR_ANY)
# 协议可配置 http 或者 https，默认为 http
# port: 1-65535(Windows系统)/ 1024-65535(Linux或Mac系统）

# 可选 (listen 为 https 协议时必须配置)
https_key: ''                 # https 服务使用的 key 的路径  (不配置默认为'')
https_cert: ''                # https 服务使用的 crt 的路径  (不配置默认为'')

##### server deploy root section #####
deploy_root: ''                # 安装根目录 (不配置默认为 `<现有用户home目录>/ohpm-repo`)，只支持绝对路径，且路径目录必须存在

##### server numeric limit section #####
max_package_size: 100          # 上传包大小限制，单位是MB (0, 100]，不配置默认为 100
max_extract_size: 500          # 压缩包解压后大小限制，单位是MB [max_package_size, 500]，不配置默认为 500
max_extract_file_num: 10240    # 压缩包解压后文件个数限制 (0, 102400]，不配置默认为 10240
user_rate_limit: 100           # 用户访问频率控制，单位是次/s (0, 10000]，不配置默认为 100
fetch_timeout: 60              # 请求/响应的超时时间，单位是秒 (0, 3600]，不配置默认为 60
keep_alive_timeout: 60         # TCP 保持连接的超时时间，单位是秒 (0, 3600]，不配置默认为 60
api_timeout: 60                # api超时时间，单位是秒(0, 3600]，不配置默认为 60
upload_lock_hour: 24           # 下架某一三方包所有版本后，限时禁止同名三方包上传，单位是小时 (0, 168]，不配置，默认为 24
upload_max_times: 100          # 单用户24小时内上传次数限制 (0, 10000]，不配置默认为 100

##### metadata storage section #####
## 数据存储类型 filedb 和 mysql 二选一，不可都配置
db:                         # 必须用 yaml 数组形式写法
  type: filedb
  config:                   # 如果想修改存储路径且保留旧的数据，则需要把旧路径下的数据文件迁移至新路径
    path:./db              # 本地数据存储路径，不配置默认为<deploy_root>/db;

#db:                        # 必须用yaml数组形式写法
#  type: mysql
#  config:
#    host: "localhost"      # 数据库主机地址
#    port: 3306             # 数据库端口 (0,65535]
#    username: root         # 数据库的用户名
#    password: "password"   # 数据库的用户密码（请配置明文, 最终在部署目录中会转换为密文）
#    database: "repo"       # 数据库名

##### storage section #####
## 文件存储类型fs,sftp 和 custom 三选一，不可多选。

store:                               # 必须用 yaml 数组形式写法
  type: fs
  config:                            # 上传资源后如若要修改存储路径，则需要把旧路径下的数据迁移至新路径中
    path:./storage                  # 已上架三方库存储路径，不配置默认为 <deploy_root>/storage;
    #server: http://localhost:8088   # 三方库下载链接，不配置默认取值

# 文件存储类型为 sftp 时，最多配置三个 sftp服务
#store:                               # 必须用 yaml 数组形式写法
#  type: sftp                         # 当且仅当 db 的类型为 mysql 时，store 的类型才能为 sftp
#  config:
#    location:
#      -
#        name: test_one_sftp          # 主机名字，名字不能与其他sftp配置重复
#        host: "localhost"            # 主机地址
#        port: 22                     # 主机端口 (0,65535]
#        read_username: "read"        # 主机有读权限的用户名字
#        read_password: "password"    # 主机有读权限的用户密码（请配置明文, 最终在部署目录中会转换为密文）
#        write_username: "write"      # 主机有写权限的用户名字
#        write_password: "password"   # 主机有写权限的用户密码（请配置明文, 最终在部署目录中会转换为密文）
#        path: /source22              # 相对 sftp 根目录的文件路径，仅限/开头，且路径文件夹必须存在
#      -
#        name: test_two_sftp
#        host: "localhost"
#        port: 24
#        read_username: "read"
#        read_password: "password"
#        write_username: "write"
#        write_password: "password"
#        path: /source24
#    #server: http://localhost:8088   # 本地仓库下载链接地址，不配置默认取 listen 的值、

#store:
#  type: custom                                            # custom是自定义存储插件类型，自定义存储插件开发流程见指导文档
#  config:
#    export_name: CustomStorage                            # 插件export的类名
#    plugin_path:../plugins/CustomStorage.js              # 插件的绝对路径或者相对于ohpm-repo软件包的路径，建议将插件放在软件包的plugins目录下
#    custom_field: "test"                                  # 自定义字段，通过引入libs/common/getStorageConfigInfo.js的getStorageConfigInfo方法获取自定义字段的值
#    #server: http://localhost:8088                        # 本地仓库下载链接地址，不配置默认取listen的值
##### uplink section #####
uplink_cache_path:./uplink      # 缓存路径，不配置默认为 <deploy_root>/uplink
uplink_cache_time: 168           # 远程包 metadata 缓存时间，单位为小时，默认 168 小时，取值范围为 (0, 8760]

##### log section #####
logs_path:./logs                # 日志路径，不配置默认为 <deploy_root>/logs

##### log level section #####
# 日志级别: 级别由低到高分别是 all、trace、debug、info、warn、error、fatal、mark、off
# run，operate 和 access 不配置或者配置错误，默认为 info
loglevel_run: info
loglevel_operate: info
loglevel_access: info

It includes listening ports, https configuration, the private repository deployment directory deploy_root, service-related configurations, storage configurations, logs, etc. Just configure it according to the actual situation.

Regarding the storage module:
The db is the configuration item for metadata storage. The db supports local storage of fileDB and storage in the mysql database.
The store is the configuration item for file storage. The store supports local storage, sftp storage, and custom plugin storage.

Methods for Modifying the Configuration File After the Private Repository is Successfully Started:

• When starting the private repository for the first time, execute the install command to specify the configuration file: Find the specified configuration file to modify its content, then re-execute install to specify the modified configuration file, and then execute start to start the private repository.
• When starting the private repository for the first time, execute the install command without specifying the configuration file: The configuration file in the conf directory under the unzipped path of the private repository compression package is used by default. Modify the content of this file, and then re-execute the install and start operations.

Installation and Startup

Execute ohpm-repo install to install. After the installation is completed, set the environment variables according to the configuration. After the setting is completed, execute ohpm-repo start to start the service.

Using Private Repository Shared Packages

By default, the client ohpm tool only pulls dependency packages from the official public repository. To pull from a private repository, additional configuration is required. There are two configuration methods:

1. Configure this private repository for all projects: ohpm config set registry <配置的私仓服务地址>/repos/ohpm
2. Configure for a certain dependency installation: ohpm install @ohos/lottie --registry <配置的私仓服务地址>/repos/ohpm

The configured private repository service address above refers to the address information of store.config.server in the configuration file. For example, if store.config.server is http://127.0.0.1:8088, then the registry is: http://127.0.0.1:8088/repos/ohpm. If store.config.server is not configured, the default value will be taken.

Publishing Shared Packages

Locally developed shared packages, whether static shared packages or dynamic shared packages, can be published through the ohpm command tool or using the Web page. For convenience and efficiency in general development work, we usually use the command line to publish.

1. Generate an ssh key locally: ssh-keygen -m PEM -t RSA -b 4096 -f <your_key_path>
2. Log in to the ohpm-repo private repository management address, click on the personal center in the upper right corner of the home page, and add a new public key. Paste the content of the public key file (<your_key_path>.pub) into the public key input box.
3. Set the private key path: ohpm config set key_path <your_key_path>
4. Log in to the ohpm-repo private repository management address, click on the personal center in the upper right corner of the home page, and copy the publishing code.
5. Configure the publishing code to the.ohpmrc file: ohpm config set publish_id <your_publish_id>
6. Publish the static shared library: ohpm publish demo.har
7. Publish the dynamic shared package: ohpm publish demo.tgz
   1. The HSP package of the dynamic shared package cannot be directly published in the private repository and needs to be converted into a.tgz package first.
   2. Switching the compilation mode to the release mode will package out a.taz package.

Module Configuration

In the library module (at the same level as the src folder), add the following files:

• Newly create a README.md file: In the README.md file, it must include the introduction and reference method of the package, and more detailed introductions can also be added according to the content of the package.
• Newly create a CHANGELOG.md file: Fill in the version update record of HAR.
• Add a LICENSE file: The LICENSE license file.

The README.md file will eventually be displayed on the private repository web platform, and the dependent users can use it directly according to the description in README.md, so it is best to describe it clearly.

Explanation of the oh_package.json5 configuration file:

{  
    "parameterFile": "../dependencies.json5",  
    "keywords": [  
        "asr"  
    ],  
    "name": "@xx/base-asr",  
    "version": "1.0.0-rc.9",  
    "repository": "http://gerrit.google.com/mobile_harmony/base_asr",  
    "description": "asr sdk",  
    "main": "Index.ets",  
    "author": "qingkouwei",  
    "license": "Apache-2.0",  
    "dependencies": {  

    }  
}

The module name, version number, and description must meet the requirements. Otherwise, the upload to the private repository will fail. The version number can only increase sequentially and cannot be overwritten infinitely like a SNAPSHOT in Android.

Best Practices

In the actual development process, an engineering project may have multiple SDKs. After the SDKs are developed, they need to be integrated into other engineering projects. Sometimes, when locating problems and debugging, they may need to be frequently uploaded to the private repository, and other engineering projects rely on the packaged ones for debugging. In this way, it is particularly cumbersome and inefficient to compile the SDK each time and then execute commands to upload it, especially when an engineering project has multiple SDKs and these SDKs also have dependencies.

The best way is to be able to compile and upload to the private repository with one click through scripts, and upload them one by one according to the dependency relationship. It is also possible to package and upload a certain module separately.

First, create a version.json5 to store the SDK version:

{  
  "project": {  
    "sdk_version": "1.1.0-rc.1",  
  }  
}

Automatic packaging:

def run_commands(modulename, productname):  
    hvigor_home = '/Applications/DevEco-Studio.app/Contents/tools/hvigor'  

    # 打包命令
    command1 = 'node %s/bin/hvigorw.js --mode module -p product=default -p module=%s@default -p buildMode=debug assembleHar --analyze --parallel --incremental --daemon' %(hvigor_home,modulename)  

    # 等待第一个命令执行完成  
    process1.wait()  

    # 上传命令  
    ohpm publish productname
    process2.wait()

Automatically modify the version number:

python
def changeVersionAModule():  
    with open('version.json5', 'r') as f:  
        data = json5.load(f)  
        versionName = data['project']['sdk_version']  

        with open('AModule/oh-package.json5', 'r') as f:  
            aData = json5.load(f)  
            aData['version'] = versionName  
            with open('AModule/oh-package.json5', 'w') as f:  
                json.dump(aData, f, indent=4)  

        with open('dependencies.json5', 'r') as depf:  
            depData = json5.load(depf)  
            depData