DEV Community

Cover image for Add real-time pose estimation to a React Native app (Expo, no API key)
Fabrice Sepret
Fabrice Sepret

Posted on Edited on

Add real-time pose estimation to a React Native app (Expo, no API key)

If you are building a fitness, wellness or rehab app, sooner or later you want the phone to see the user move: count a squat, check a posture, measure a range of motion. The first step is always the same. You need the position of the body's joints, on every camera frame, without sending video to a server.

This tutorial gets you there in a fresh Expo project: 17 body keypoints per frame, computed on the device, with no API key and no backend. I am the founder of PoseTracker, the SDK used here, so take that into account; the code below is the same as in our public docs.

What you will build

A single screen that opens the camera, draws a skeleton over the person, and logs the keypoints in JavaScript. The pose model is MoveNet, running on the phone inside a WebView. Nothing is uploaded.

1. Install

npm install @pose-tracker/react-native-pose-estimation react-native-webview
npx expo install react-native-webview expo-camera
Enter fullscreen mode Exit fullscreen mode

expo-camera is only there for the permission API. The camera itself is captured by the SDK's view.

There are two packages with the same API. The one above bundles the model, so keypoints work offline (about 10 MB packed). @pose-tracker/react-native-pose-estimation-light is about 212 kB and downloads the model when the view starts, so it needs a network connection. Pick offline if your users train in a basement gym, light if install size matters more.

2. Declare the camera permission

In app.json:

{
  "expo": {
    "ios": {
      "infoPlist": {
        "NSCameraUsageDescription": "Camera access is required for on-device pose estimation."
      }
    },
    "android": {
      "permissions": ["android.permission.CAMERA"]
    },
    "plugins": [
      [
        "expo-camera",
        {
          "cameraPermission": "Camera access is required for on-device pose estimation."
        }
      ]
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

Request the permission in your app before you mount the camera screen.

3. Mount the view and read the keypoints

import {
  PoseTrackerProvider,
  WebViewPoseView,
  usePoseTracker,
} from '@pose-tracker/react-native-pose-estimation';

export default function App() {
  return (
    <PoseTrackerProvider>
      <CameraScreen />
    </PoseTrackerProvider>
  );
}

function CameraScreen() {
  usePoseTracker({
    onKeypoints: (e) => {
      console.log(e.keypoints.length, e.score);
    },
  });

  return (
    <WebViewPoseView
      style={{ flex: 1 }}
      drawSkeleton
      loadingText="AI Loading"
    />
  );
}
Enter fullscreen mode Exit fullscreen mode

That is the whole integration. Three pieces do the work:

  • PoseTrackerProvider creates the client. Without a token it runs in keypoints-only mode.
  • WebViewPoseView loads the model and opens the camera.
  • usePoseTracker({ onKeypoints }) gives you a typed event on each frame.

What you get on each frame

Field Type Meaning
keypoints Keypoint[] 17 joints, each with a name, x, y and a score
score number Mean confidence of the pose
timestampMs number Time of the frame

The 17 joints are the COCO set: nose, eyes, ears, shoulders, elbows, wrists, hips, knees and ankles. With those you can already compute a knee angle, detect that the hips went below the knees, or check that both shoulders are level.

Why a WebView

It is a fair question for an SDK. The model runs with TensorFlow.js on WebGL inside react-native-webview, which is what lets the same code run in Expo Go, on iOS and on Android without a native build step. The trade-off is that you depend on the WebView's GPU path. On the phones we test, that holds a real-time frame rate; measure it on the oldest device you intend to support.

From keypoints to repetitions

Keypoints are free and stay free. Turning them into "that was one clean squat" is a different job: you need to segment the movement, count, and score the form. You can write that yourself from the keypoints, or use PoseTracker's engine, which is the paid part. With an API key:

<PoseTrackerProvider
  apiToken="YOUR_API_KEY"
  options={{ features: { angles: true, progression: true, minGrade: 'B' } }}
>
  <WebViewPoseView drawSkeleton />
</PoseTrackerProvider>
Enter fullscreen mode Exit fullscreen mode
const { preload, startExercise } = usePoseTracker({
  onMessage: (msg) => {
    if (msg.type === 'counter') {
      // current_count and form_score: { score, avg_score, grade }
    }
  },
});

await preload();
startExercise('squat');
Enter fullscreen mode Exit fullscreen mode

The free plan covers 200 tracking calls a month, which is enough to prototype.

Try it without writing code

If you want to see it on your phone first, open the Expo Snack in Expo Go and scan the QR code. No API key needed.

Where to go next

One more thing. MoveNet stops at 17 keypoints and at the ankle. We are preparing our own model, PoseTrackerNet, with 39 keypoints, including the feet, the trunk and the neck. It is in beta, with a public release planned by December 1, 2026.

Top comments (0)