How to install mmcv in an Npu Enviroment

#deeplearning #python #tutorial #ubuntu

Description

I want to run musetalk model in Ascend GPU today, but the mmcv caused a lot of trouble.I summarized some experience and would like to share it here.

My Environment

NPU: 910b2
os: aarch64 Ubuntu 22.04.5 LTS

Story

When I attempted to install mmcv with pip, an error No module named mmcv._ext error will be thrown. This error could only be resolved by compiling it from source code.However, the compilation succeeded in Cpu environment but failed when attempted to compiled it for npu environment.

Compile with CPU

Download mmcv code from github and the branch is npu-dev.

Modify setup.py

The 21 line

#cmd_class = {'build_ext': BuildExtension}
cmd_class={'build_ext': BuildExtension.with_options(use_ninja=False)}

From 273 line to 300 line

    elif (os.getenv('FORCE_NPU', '0') == '1'):
        print(f'Compiling {ext_name} only with CPU and NPU')
        try:
            from torch_npu.utils.cpp_extension import NpuExtension
            define_macros += [('MMCV_WITH_NPU', None)]
            extension = NpuExtension
        except Exception:
            raise ImportError('can not find any torch_npu')
        # src
        op_files = glob.glob('./mmcv/ops/csrc/pytorch/*.cpp') + \
            glob.glob('./mmcv/ops/csrc/pytorch/cpu/*.cpp') + \
            glob.glob('./mmcv/ops/csrc/common/npu/*.cpp') + \
            glob.glob('./mmcv/ops/csrc/pytorch/npu/*.cpp')
        include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common'))
        include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common/npu'))
        if 'cxx' not in extra_compile_args:
            extra_compile_args['cxx'] = {}
        extra_compile_args['cxx'] += ['-std=c++17', '-Wall']

    else:
        print(f'Compiling {ext_name} only with CPU')
        op_files = glob.glob('./mmcv/ops/csrc/pytorch/*.cpp') + \
            glob.glob('./mmcv/ops/csrc/pytorch/cpu/*.cpp')
        extension = CppExtension
        include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common'))
        if 'cxx' not in extra_compile_args:
            extra_compile_args['cxx'] = {}
        extra_compile_args['cxx'] += ['-std=c++17', '-Wall']

Run compile command

You must install torch and torchvision before run the following command.

MMCV_WITH_OPS=1 MAX_JOBS=40 python setup.py build_ext

Try to run following command if you`re luncky.

MMCV_WITH_OPS=1 MAX_JOBS=40 python setup.py develop

Note that this is just a test setup, since it will generate a symbolic link to site-packages folder. And run the command blow to verify the mmcv module.

from mmcv.ops import MultiScaleDeformableAttention

This is the actual installation command.

MMCV_WITH_OPS=1 MAX_JOBS=40 python setup.py install

Notice

The file _ext.cpython-310-aarch64-linux-gnu.so may not be located in the mmcv directory, instead, it resides in the build directory.You must create symbolic link manually.

DEV Community