loading...

Run PoseNet with Nodejs

kojikanao profile image Koji (he/him) ・2 min read

What is Posenet?

https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
https://github.com/tensorflow/tfjs-models/tree/master/posenet

As you can see, basically we don't need to use nodejs for PoseNet since we need to use canvas to feed data to tensorflowjs lol

Also, we can use tensorflow with python instead of js, but I did try because there are nodejs and tfjs-node(https://github.com/tensorflow/tfjs-node).

steps

  1. install packages/libs
  2. install npm packages
  3. write code & run it

Step1

As I mentioned, need to use canvas, so need to install libs.
By the way, this is for mac, but you can find out information for Linux and Windows on the internet. In addition, now we can use Homebrew on Linux(I tested it on Ubuntu)

$ brew install pkg-config cairo pango libpng jpeg giflib

Step2

I'm using yarn since npm didn't allow me to install tfjs. But, you can use/try npm instead of yarn.

$ yarn add @tensorflow-models/posenet @tensorflow/tfjs @tensorflow/tfjs-node botkit canvas rollup

Step3

The code is messy since just test tfjs-node with posenet.

const tf = require('@tensorflow/tfjs-node');
const posenet = require('@tensorflow-models/posenet');
const {
    createCanvas, Image
} = require('canvas')
const imageScaleFactor = 0.5;
const outputStride = 16;
const flipHorizontal = false;

const tryModel = async() => {
    console.log('start');
    const net = await posenet.load(0.75);
    const img = new Image();
    img.src = './test.jpg';
    const canvas = createCanvas(img.width, img.height);
    const ctx = canvas.getContext('2d');
    ctx.drawImage(img, 0, 0);
    const input = tf.browser.fromPixels(canvas);
    const pose = await net.estimateSinglePose(input, imageScaleFactor, flipHorizontal, outputStride);
    // console.log(pose);
    for(const keypoint of pose.keypoints) {
        console.log(`${keypoint.part}: (${keypoint.position.x},${keypoint.position.y})`);
    }
    console.log('end');
}



tryModel();

Result

Got 17 keypoints, yay!

Posted on by:

kojikanao profile

Koji (he/him)

@kojikanao

Looking for opportunities! Software Engineer in NYC/Research Resident@NYU-ITP18-19 Mostly I work with reactjs and sometimes python. code to do something weird/useless/helpful Japanese

Discussion

markdown guide
 
[deleted]
 

i just used npm install @tensorflow-models/posenet to use the model.

nodejs v10.15.0
npm v6.9.0

 

I am always receiving this error: UnhandledPromiseRejectionWarning: Error: Image given has not completed loading

 
 

Yes, after installing all the dependencies I ended up with this error code : FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal.

can you tell me your os version, nodejs version and your package.json's dependencies

linux subsystem on windows 10, "dependencies": {
"@tensorflow-models/posenet": "2.1.3",
"body-parser": "1.19.0",
"express": "4.17.1",
"morgan": "1.9.1",
"typescript": "3.6.3"
},
"devDependencies": {
"@tensorflow/tfjs": "1.2.9",
"@tensorflow/tfjs-node": "1.2.9",
"botkit": "4.5.0",
"canvas": "2.6.0",
"rollup": "1.21.2"
}

tensorflor-models/posenet has been updated so you need to update code.

just checked my package.json

"dependencies": {
    "@tensorflow-models/posenet": "^1.0.3",
    "@tensorflow/tfjs": "^1.1.2",
    "@tensorflow/tfjs-node": "^1.1.2",
    "botkit": "^4.0.2",
    "canvas": "^2.5.0",
    "rollup": "^1.13.1"
  }
 

I ended up uninstalling everything and reinstalling. But now when I run the code I get " (node:87485) UnhandledPromiseRejectionWarning: TypeError: Cannot create property 'architecture' on number '0.75'" which seems to be an error within posenet. maybe not everything is completely installed correctly? what do you think thank you for the help

 

It's because posenet api has been updated in version 2.0. See:
github.com/tensorflow/tfjs-models/...

Example new loading code:

const net = await posenet.load({
  architecture: 'MobileNetV1',
  outputStride: 16,
  inputResolution: 513,
  multiplier: 0.75
});
 

Can you please add video inference support for posenet?

 

sorry i don't have time now.
i recommend you to check tfjs repo and dan's repos dev.to/oveddan since he is the peron who has implemented posenet to tfjs.
my code is outdated.

 

@gaelguedia
can you tell me your os version, nodejs version, npm/yarn and your package.json's dependencies

 

The above error was on windows 10, js 10. The issue seems to be related to canvas dependency. It's not easy to have it running on windows. It's easier on ubuntu especially easier to install cairo dependencies for Jpg/jpeg support