What is MediaPipe
MediaPipe is an open-source cross-platform framework for building machine learning pipelines for processing sequential data like video and audio and deploying it on a wide range of target devices.
MediaPipe empowers your application with state-of-the-art machine learning algorithms, running with real-time speed on edge devices with low-code APIs or via no-code studio builder.
You are free to use any pre-built solution as a "black box" or to customize it to your needs. You can even fully reimplement the algorithm. This article will help you to make it, as it shows how to dive inside a solution as deep as you want.
Let's figure out how to access any intermediate result inside the solution graph from the Python API using the official code example from MediaPipe itself: mediapipe.solutions.pose.Pose.
The very first step we should do is to build the mediapipe python package from its source code.
Building MediaPipe python package from source code
To achieve our goal - building the MediaPipe python package from its sources - we follow the official build instructions.
It should be as easy as:
clone the source code
install all the dependencies and build tools
build the Python wheel/install the package into the virtual environment.
However, sometimes you get the following error while building the MediaPipe Python package:
mediapipe/tasks/cc/components/processors/proto/detection_postprocessing_graph_options.proto:39:12:
Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3,
simply remove the 'optional' label, as fields are 'optional' by default.
To fix it, you should change the file mediapipe/tasks/cc/components/processors/proto/detection_postprocessing_graph_options.proto:
find the line containing text:
syntax = "proto3";(for me it was line 16)change it to the following text:
syntax = "proto2";and try again.
Note: If you missed some steps and jumped directly into building the package, you may get annoying error messages even after returning back and following all the instructions. One possible solution is to execute thebazel clean --expunge command and try to rebuild the package. Sometimes, even this step is not enough and your best option is to delete your MediaPipe copy and start from the beginning of the process.
NOTE: you may have to manually fix __init__.py file of the built MediaPipe package after each rebuilding (delete duplicated code).
Accessing intermediate results
There are several possible situations you may get into while trying to extract some intermediate processing results from the MediaPipe solutions:
• intermediate results are exposed from the graph, but not passed to the Python code;
• you want to expose graph node inputs/outputs as the new outputs from the graph and Python code;
• you want to print some information from inside the C++ code of the graph nodes.
Let's take a look at all these situations one by one.
The Article continued on the OpenCV.ai blog...
Top comments (0)