DEV Community: Toki Hirose

Exploring Feature Distributions from Pedestrian Trajectories

Toki Hirose — Sun, 29 Mar 2026 05:07:50 +0000

Motivation

In this article, I explore the statistical distributions of features extracted from pedestrian trajectories. Although the tracking accuracy still has room for improvement, analyzing the distributions of speed and related features allows me to characterize average pedestrian behavior from real video data. I have recently been studying information geometry, and I want to investigate how pedestrian behavior changes across locations by modeling it as probability distributions. By constructing a statistical manifold from these distributions across multiple locations, I aim to capture differences between pedestrian populations that conventional clustering methods such as k-means — which rely on Euclidean distance — cannot detect.

Note

I use AI assistance to draft and polish the English, but the analysis, interpretation, and core ideas are my own. Learning to write technical English is itself part of this project.

Introduction

In Article 1, I detected pedestrians in video footage and visualized their trajectories. In Article 2, I projected those trajectories onto a map using homography. Known limitations remain — detection accuracy and calibration point selection for the homography transform — but these are out of scope for this article and will be addressed in a future iteration. In this article, I analyze the speed and behavioral distributions of the detected pedestrians.

The data used in this analysis was collected at the Ginza pedestrian zone (歩行者天国) in Tokyo.

Feature Extraction

Each pedestrian trajectory stores a timestamp, pixel coordinates within the video frame, and the height of the detected bounding box. From a fixed camera, this is sufficient to measure speed, acceleration, dwell time, and several behavioral indicators.

Pixel to Meter Conversion

All trajectory measurements are initially in pixel units. To convert to real-world distances, I used the bounding box height as a depth proxy. For each track at each frame, the average bounding box height between consecutive frames is computed and used to derive a scale factor based on the assumed average human height of 1.7 m:

scale [m/px] = H_REAL / avg_bbox_height_px
real_dist [m] = pixel_dist [px] × scale

This approach effectively performs implicit monocular depth estimation using the known height of a pedestrian as a reference object.

Speed

Per-step real-world speed was computed by dividing the depth-normalized displacement between frames by the elapsed time. For each track, I extracted:

real_speed_mean: mean walking speed
real_speed_cv: coefficient of variation (std / mean), which normalizes variability relative to the mean and indicates how much a single trajectory fluctuates in pace

Acceleration

Acceleration was computed as the finite difference of per-step speed divided by elapsed time. I extracted the mean of signed acceleration (real_accel_mean). Since acceleration can be positive or negative, the signed mean captures whether there is a net directional bias across the trajectory.

I also computed decel_ratio — the fraction of acceleration steps where the value is negative (i.e., the pedestrian is decelerating). A value near 0.5 indicates balanced acceleration and deceleration (typical steady walking); values above 0.5 indicate dominant braking behavior. While real_accel_abs_mean captures the intensity of speed change, decel_ratio captures its frequency, making them complementary indicators.

Speed Skewness

For each trajectory, I computed the skewness of the per-step speed distribution (speed_skew). A value near zero indicates a symmetric speed profile — steady walking. A negative value indicates the presence of a slow phase within the trajectory (hesitation or stopping). A positive value indicates a fast phase (e.g., hurrying after a pause). This captures the temporal asymmetry of the speed profile in a single scalar, which mean and variance alone cannot express.

Stop Ratio

I computed stop_ratio as the fraction of steps where real-world speed falls below 0.3 m/s. A higher stop ratio indicates that the pedestrian spent more time engaging with the environment — reading a sign, looking at a shop window, or reacting to a stimulus.

Path Straightness

real_straightness is the ratio of the depth-normalized displacement from start to end point to the total real path length (range: 0–1). Values near 1 indicate a nearly straight trajectory; values near 0 indicate winding, reversal, or wandering. Both displacement and total path length are depth-normalized using the bounding box height at the respective trajectory points, correcting for the variation in pixel scale with camera distance.

Dwell Time

duration_sec is the observation duration of each track, computed as the number of frames from first to last detection multiplied by the frame interval.

Distribution Analysis

I plotted the frequency distribution of each feature across all tracks and fitted candidate probability distributions using the Kolmogorov-Smirnov (KS) test.

The KS statistic D measures the maximum difference between the empirical CDF of the data and the theoretical CDF of a candidate distribution. A smaller D indicates a better fit. The corresponding p-value represents the probability of observing a deviation as large as D by chance under the null hypothesis; a higher p-value indicates a better fit. When comparing multiple candidate distributions, I selected the one with the smallest KS statistic (breaking ties by p-value).

Candidates were assigned based on the support of each feature:

Positive-valued: Normal, Log-normal, Half-normal, Gamma, Exponential
Bounded [0, 1]: Beta, Uniform, Normal, Log-normal
Full real line: Normal, Laplace, t(df=5), Cauchy

Feature	Unit	Mean	Std	Best Distribution	KS	p-value	Accept (p>0.05)
`real_speed_mean`	m/s	1.360	0.420	Gamma	0.037	0.350	✓
`real_speed_cv`	—	0.894	0.383	Log-normal	0.061	0.019	✗
`real_accel_mean`	m/s²	0.026	5.362	Cauchy	0.034	0.474	✓
`stop_ratio`	—	0.085	0.089	Beta	0.118	0.000	✗
`speed_skew`	—	2.460	1.866	Gamma	0.031	0.591	✓
`real_straightness`	—	0.484	0.268	Beta	0.031	0.466	✓
`decel_ratio`	—	0.499	0.054	Normal	0.067	0.008	✗
`duration_sec`	s	3.910	3.076	Gamma	0.045	0.161	✓

Results

The eight features selected for downstream analysis and their associated distribution families are as follows:

Feature	Best Distribution	Accept (p>0.05)	Rationale
`real_speed_mean`	Gamma	✓	Best fit for walking speed distribution
`real_speed_cv`	Log-normal	✗	Lowest KS among all candidates; captures individual variation in pace
`real_accel_mean`	Cauchy	✓	Lowest KS; heavy tails reflect rare but strong braking and surging events
`stop_ratio`	Beta	✗	Lowest KS; bounded in [0,1]; sensitive indicator for behavioral intervention
`speed_skew`	Gamma	✓	Lowest KS; captures temporal asymmetry in the speed profile
`real_straightness`	Beta	✓	Lowest KS; bounded in [0,1]; captures path linearity
`decel_ratio`	Normal	✗	Lowest KS; captures deceleration frequency
`duration_sec`	Gamma	✓	Best fit for observation duration

For features where the KS test was rejected (p < 0.05), I adopted the distribution with the lowest KS statistic as the working approximation. The reasoning is described in the Discussion.

Interpretation

real_speed_mean

Why this distribution?
Speed is non-negative and right-skewed: most pedestrians walk at a typical pace, with a tail of faster walkers. The Gamma distribution naturally accommodates a lower bound at zero and a right tail, making it a better fit than the Normal distribution, which would assign probability mass to negative values.

What the data reveals
The analysis reveals that the mean walking speed is 1.36 m/s, consistent with typical pedestrian speeds in a busy shopping district like Ginza. Speeds above 2.5 m/s are present but rare, representing pedestrians in a hurry.

real_speed_cv

Why this distribution?
The coefficient of variation is a positive quantity representing normalized speed variability within a trajectory. In a dense urban environment like Ginza, pedestrians frequently adjust pace due to crowds, direction changes, and window shopping. Tracking ID switches — where the system reassigns an ID between two different individuals — can also cause artificial speed jumps. Both effects contribute to a right-skewed distribution with a heavy tail, which the Log-normal distribution captures well.

What the data reveals
The mean CV of 0.89 indicates that most pedestrians show substantial within-trajectory speed variation. This is expected given the high foot traffic and the current limitations of the CentroidTracker-based tracking system.

real_accel_mean

Why this distribution?
Signed mean acceleration can be positive or negative, so a bell-shaped distribution centered near zero is expected. In a pedestrian zone without traffic signals, pedestrians are not forced to stop — sharp decelerations more likely reflect voluntary behavior such as entering a shop or looking at a sign. The Cauchy distribution captures the same bell shape as the Normal but with much heavier tails, consistent with the occasional extreme acceleration events observed.

What the data reveals
The distribution is centered very close to zero (mean = 0.026 m/s²), confirming that pedestrians neither systematically accelerate nor decelerate over their trajectories. The heavy Cauchy tails reflect rare but strong braking or surging events.

stop_ratio

Why this distribution?
stop_ratio is bounded in [0, 1] and represents the fraction of steps below a speed threshold within a trajectory. The Beta distribution is the natural choice for proportions on this interval.

What the data reveals
The distribution peaks near zero and 0.05, indicating that most pedestrians rarely drop below 0.3 m/s. However, the long right tail — reaching up to approximately 0.4 — confirms that a meaningful minority paused significantly, likely engaging with shop displays or other environmental stimuli.

real_straightness

Why this distribution?
real_straightness is bounded in [0, 1], making the Beta distribution the natural candidate. In a shopping district, pedestrians follow moderately direct paths but with meaningful deviation — neither hugging a straight line nor wandering randomly. This intermediate behavior, away from both boundary extremes, is well captured by a Beta distribution with parameters that place probability mass in the interior of [0, 1].

What the data reveals
The distribution is spread across the interior of [0, 1] with a mean of 0.48, indicating that pedestrians follow paths that are neither fully straight nor fully winding. This intermediate behavior is consistent with movement through a busy shopping district, where pedestrians maintain a general direction but frequently deviate in response to storefronts, crowds, and signage.

speed_skew

Why this distribution?
Per-step speed is non-negative, so the within-trajectory speed distribution has a natural floor at zero. This floor creates an inherent right skew in the speed profile, which means speed_skew is almost always positive (98.4% of tracks in this dataset). The Gamma distribution, which is defined for positive values and flexible in shape, fits this behavior well.

What the data reveals
The mean speed_skew is 2.47 (std = 2.03), indicating that most pedestrians have a few brief high-speed moments within an otherwise slower trajectory. This is consistent with occasional surges in pace — stepping around another person, crossing a gap in the crowd — embedded in otherwise steady walking.

decel_ratio

Why this distribution?
decel_ratio measures the fraction of steps where acceleration is negative. For steady walking, this should be close to 0.5 — alternating acceleration and deceleration in roughly equal measure. The Normal distribution centered near 0.5 captures this well.

What the data reveals
The mean is 0.499, confirming that pedestrians in this dataset decelerate and accelerate in nearly equal proportions. There is no systematic braking bias in baseline behavior — an important reference for detecting shifts caused by an intervention in Article 12 and 13.

duration_sec

Why this distribution?
Observation duration is positive and right-skewed: most pedestrians cross the field of view in a few seconds, but some linger, pause, or follow longer paths. The Gamma distribution accommodates the peak at short durations and the heavy right tail.

What the data reveals
The median duration is approximately 2 seconds, consistent with pedestrians walking through the frame. The right tail extends to around 14 seconds. One limitation of this approach is that the maximum observed duration is relatively short — it is likely that some pedestrians who stopped for longer had their track IDs reassigned by the tracker, truncating their trajectories.

Discussion

KS Test Sensitivity at Large Sample Sizes

The Kolmogorov-Smirnov test becomes increasingly sensitive as sample size grows. With n ≈ 600, even small deviations from a theoretical distribution are detectable, leading to rejection even when the visual fit is clearly reasonable. The KS statistic D measures the maximum gap between empirical and theoretical CDFs; a small D is evidence of good fit regardless of whether the test is formally rejected. For this reason, I adopted the distribution with the smallest D as the working approximation for all features, including those where the null hypothesis was rejected.

Feature Correlation and Implications for Manifold Construction

The correlation matrix reveals that speed_skew and real_speed_cv are strongly correlated (r = 0.77). Highly correlated features introduce redundant dimensions into the statistical manifold, which may affect the geometry of the parameter space in Article 4. Both features are retained for now, and their effect on the manifold structure will be examined in that article.

The Cauchy Distribution and Fisher Information

The Cauchy distribution for real_accel_mean is a physically meaningful result: it reflects the occasional extreme acceleration events — sudden stops at a shop entrance, abrupt lane changes — that occur against a background of smooth walking. However, the Cauchy distribution has no defined mean or variance. Its Fisher information metric is defined only through the scale parameter γ, so in Article 4, the treatment of the location parameter μ in the manifold construction will require careful consideration.

Limitations of the Data Collection Method

One limitation of this approach is that the CentroidTracker used for pedestrian tracking is prone to ID switching when pedestrians cross paths. This produces artificial velocity jumps in some trajectories and likely inflates the values of real_speed_cv and speed_skew. Improving tracking accuracy — for example by switching to a re-identification-based tracker — would reduce this artifact.

A second limitation is that the depth normalization via bbox_height assumes full-body visibility. For occluded or partially cropped pedestrians, the scale factor may be unreliable.

Toward Detecting Behavioral Change

The eight distribution families established here define the coordinate system of the statistical manifold in Article 4. In future articles (Articles 12 and 13), I will investigate how the parameters of these distributions shift when a standing demonstration is present. A systematic shift in Beta parameters for stop_ratio, or a change in the Cauchy scale parameter for real_accel_mean, would provide a quantitative signature of behavioral disruption.

Conclusion

In this article, I extracted eight features from pedestrian trajectories obtained from video footage and fitted probability distributions to each feature using the KS test. The analysis reveals that mean walking speed, speed skewness, and dwell time follow Gamma distributions; speed variability follows a Log-normal distribution; signed mean acceleration follows a Cauchy distribution; stop ratio and path straightness follow Beta distributions; and deceleration ratio follows a Normal distribution. These eight features and their associated distribution families form the coordinate parameterization of the statistical manifold to be constructed in Article 4.

In the next article

In the next article, I will apply this parameterization schema to pedestrian trajectory data collected at multiple stations along the Yamanote Line. Each location will be represented as a point on a statistical manifold, where coordinates are the estimated parameters of the eight distributions identified here. Using the Fisher information metric and e/m geodesics, I will compute distances between locations in a way that reflects genuine differences in pedestrian behavior — differences that Euclidean distance in raw feature space cannot capture.

Code

https://github.com/TOKIHISA/people_trajectory_analysis/blob/main/analysis/explore_distributions_pixel.ipynb

Visualizing Pedestrian Trajectories on a Map with MapLibre

Toki Hirose — Wed, 18 Mar 2026 15:11:01 +0000

Note:I use AI assistance to draft and polish the English, but the analysis, interpretation, and core ideas are my own. Learning to write technical English is itself part of this project.

Motivation

When I stood alone in protest at stations along the Yamanote Line, I watched people walk past me. Some glanced at my sign and looked away with a frown. Others gave a small nod. But I had no way to measure how many people actually reacted to my message — or how.

That question is what started this project. I wanted to capture not just whether people reacted, but how their movement changed: did they slow down, step aside, or adjust their path? To do that, I needed to track pedestrian trajectories from video and place them on a real map. I filmed at several stations in Tokyo using a smartphone, and built the pipeline using open-source tools — YOLOX for detection, ByteTrack for tracking, and MapLibre for visualization.　

Introduction

In the previous article, I extracted pedestrian trajectories from street video as structured JSON. The coordinates in that output are pixel positions in image space means something relative to the camera frame, but nothing about the real world.

In this article, I transform those image-space trajectories into geographic coordinates (WGS84) using homography, then render them on an interactive map with MapLibre GL JS. The result is a browser-based viewer where you can watch each pedestrian's path play back in real time over an aerial orthophoto. The data used inthis article was filmed at Shnbashi station.

The Core Problem: Two Different Coordinate Systems

Video footage and maps live in fundamentally different spaces. A pixel at (x, y) in a camera frame corresponds to some location (lon, lat) on the ground — but the mapping is not a simple linear scale. Because the camera is tilted at an angle rather than looking straight down, perspective distortion means that objects farther from the camera appear compressed. A projective transformation, called a homography, is the correct model for this mapping when the scene is approximately planar (which a ground surface is).

A homography H is a 3×3 matrix that maps points in image space to points in world space via:

[lon, lat, 1]^T ∝ H · [x, y, 1]^T

To compute H, I need at least four Ground Control Points (GCPs): pairs of corresponding locations, one in image coordinates and one in geographic coordinates.

Collecting Ground Control Points

The GCP workflow requires two tools running in sequence.

Step 1: Selecting Map Coordinates

The first tool (gcp_selector_map.py) opens a browser-based map where I click on identifiable features — a road marking, a corner of a crosswalk, a utility cover — and record their geographic coordinates.

The tool uses Folium with multiple tile layer options: OpenStreetMap, Esri World Imagery, and the Geospatial Information Authority of Japan (GSI) aerial photos. For urban scenes in Japan, the GSI orthophoto is particularly useful because it is regularly updated and offers clear ground-level features.

# Aerial orthophoto from Japan's Geospatial Information Authority
folium.TileLayer(
    tiles='https://cyberjapandata.gsi.go.jp/xyz/seamlessphoto/{z}/{x}/{y}.jpg',
    attr='国土地理院',
    name='航空写真 (国土地理院)',
).add_to(m)

Clicking on the map adds a numbered marker and records the WGS84 coordinates. Once four or more points are selected, the tool exports a gcp_config.json:

{
  "gcp_wgs84": [
    {"lat": 35.68423, "lon": 139.76531},
    {"lat": 35.68418, "lon": 139.76547}
  ],
  "gcp_image": []
}

Step 2: Selecting Video Frame Coordinates

The second tool (gcp_selector_video.py) opens a frame from the same video and displays it in an OpenCV window. I click on the same physical features in the same order as I did on the map.

cap.set(cv2.CAP_PROP_POS_FRAMES, target_frame)
ret, frame = cap.read()

Using a stable, early frame from the video (default: 10 seconds in) avoids motion blur and ensures that the scene features are visible. Right-click removes the last point, Enter confirms. The tool writes the pixel coordinates back into gcp_config.json:

{
  "gcp_wgs84": [
    {"lat": 35.68423, "lon": 139.76531},
    ...
  ],
  "gcp_image": [
    {"x": 412, "y": 631},
    ...
  ]
}

The matching order matters: point 1 in gcp_wgs84 must correspond to point 1 in gcp_image.

Computing the Homography Matrix

With the GCP pairs collected, project_v2wgs84.py computes the homography using OpenCV's findHomography with RANSAC:

self.H, _ = cv2.findHomography(
    self.src_points,  # image coordinates (N, 2)
    self.dst_points,  # WGS84 coordinates (N, 2) as [lon, lat]
    cv2.RANSAC,
    ransacReprojThreshold=3.0
)

RANSAC is used here because a few GCPs may be slightly misclicked. The algorithm fits the best homography to the inlier set and discards outliers, making the result more robust than a direct least-squares fit.

Note that with exactly four GCPs — the minimum required to determine H — RANSAC has no effect: all four points are consumed by the initial sample, leaving none to evaluate as inliers or outliers. To benefit from outlier rejection, provide at least five GCPs, and preferably eight or more.

After computing H, the reprojection error is calculated by transforming each source GCP back through H and measuring the distance to the target:

projected = cv2.perspectiveTransform(src_reshaped, self.H)
errors = np.sqrt(np.sum((projected - self.dst_points) ** 2, axis=1))
self.reprojection_error = float(np.mean(errors))

Transforming Trajectories

The transform_trajectory method applies H to each point in a track's pixel-space trajectory:

def transform_points(self, points: np.ndarray) -> np.ndarray:
    pts = points.reshape(-1, 1, 2).astype(np.float64)
    transformed = cv2.perspectiveTransform(pts, self.H)
    return transformed.reshape(-1, 2)

Before transformation, points outside the convex hull of the GCPs are discarded. The homography is only well-defined within the region spanned by the control points; extrapolation beyond the convex hull produces increasingly distorted results.

def is_in_valid_region(self, x: float, y: float) -> bool:
    hull = cv2.convexHull(self.src_points.astype(np.float32))
    return cv2.pointPolygonTest(hull, (float(x), float(y)), False) >= 0

Trajectory Simplification

The raw trajectory from the tracker contains a point for every frame — typically 30 points per second. For visualization and downstream analysis, this is far more detail than needed. When drawing in Maplibre, dense point sequences slow down rendering without adding visual clarity. I simplify by keeping only points where something meaningful changes: a direction shift or a speed change.

# Keep a point if:
# - direction changed by more than angle_threshold_deg (default: 10°)
# - speed changed by more than speed_ratio_threshold (default: 30%)
if angle > angle_threshold_deg or speed_change > speed_ratio_threshold:
    result.append(curr)

This preserves turning points, stops, and accelerations while dropping the redundant intermediate frames during straight, constant-speed walking. A 60-second trajectory of ~1,800 frames typically reduces to 20–80 representative points.

The output for each track gains a trajectory_wgs84 field alongside the original pixel-space trajectory:

{
  "id": 7,
  "trajectory_wgs84": [
    {"lon": 139.76531, "lat": 35.68423, "frame": 120, "time_sec": 4.0},
    {"lon": 139.76545, "lat": 35.68419, "frame": 155, "time_sec": 5.17}
  ],
  "geometry_wgs84": {
    "type": "LineString",
    "coordinates": [[139.76531, 35.68423], [139.76545, 35.68419]]
  }
}

Rendering in MapLibre GL JS

The final step is generate_viewer.py, which reads the WGS84 JSON and generates a self-contained HTML file. The viewer uses MapLibre GL JS with an OpenStreetMap base layer.

Map Setup

const map = new maplibregl.Map({
  container: 'map',
  style: {
    version: 8,
    sources: {
      osm: {
        type: 'raster',
        tiles: ['https://tile.openstreetmap.org/{z}/{x}/{y}.png'],
        tileSize: 256,
      }
    },
    layers: [{ id: 'osm', type: 'raster', source: 'osm' }]
  },
  center: [center_lon, center_lat],
  zoom: 19
});

Two-Layer Rendering Strategy

The viewer maintains two GeoJSON sources:

Background (bg): Full path of each active track, drawn at low opacity (0.15). Updated only when tracks enter or leave the scene.
Trail (trail): A sliding window of the recent path for each active track, updated every animation frame.

// Background: full path at low opacity
map.addLayer({
  id: 'bg', type: 'line', source: 'bg',
  paint: { 'line-color': ['get', 'color'], 'line-width': 2, 'line-opacity': 0.15 }
});

// Trail glow + line
map.addLayer({
  id: 'trail-glow', type: 'line', source: 'trail',
  paint: { 'line-color': ['get', 'color'], 'line-width': 8, 'line-blur': 6, 'line-opacity': 0.3 }
});
map.addLayer({
  id: 'trail-line', type: 'line', source: 'trail',
  paint: { 'line-color': ['get', 'color'], 'line-width': 3 }
});

Each track gets a unique color computed with the golden-angle hue distribution, which maximizes perceptual separation between adjacent track indices.

Animation Loop

The animation loop advances a currentTime variable and slices each track's coordinates to the active window:

function animate(ts) {
  currentTime += (ts - lastTs) / 1000 * playbackSpeed;

  const trailFeatures = [];
  for (let i = 0; i < TRACKS.length; i++) {
    const track = TRACKS[i];
    const isActive = currentTime >= track.start && trailStart <= track.end;
    if (!isActive) continue;

    // Binary search for the relevant coordinate slice
    const headIdx = upperIdx(coords, currentTime);
    const tailIdx = lowerIdx(coords, currentTime - trailSec);
    const slice = coords.slice(tailIdx, headIdx + 1).map(([lon, lat]) => [lon, lat]);

    if (slice.length >= 2) {
      trailFeatures.push({ type: 'Feature', ... });
    }
  }
  map.getSource('trail').setData({ type: 'FeatureCollection', features: trailFeatures });
  requestAnimationFrame(animate);
}

Binary search on the time axis avoids scanning all coordinates each frame, keeping the animation smooth even with hundreds of simultaneous tracks.

The viewer includes a timeline scrubber, adjustable playback speed (1×–120×), and a configurable trail window length (1–30 seconds). The generated HTML is fully self-contained — no server required, just open in a browser.

Results

Running the full pipeline on the Shinbashi station video produces a self-contained HTML viewer. GCP collection takes around 10 minutes per video, split between map selection and video frame annotation.

The geographic alignment looks accurate at zoom level 19 — pedestrians appear to walk along the correct sidewalk, following the actual street geometry. The animation runs smoothly in the browser even with multiple tracks active simultaneously.

Trajectory simplification reduces the point count significantly, making the viewer responsive without any noticeable loss of visual detail.

Interpretation

Plotting the first video revealed an immediate artifact: trajectories near the camera showed erratic up-and-down distortion, diverging left and right rather than tracing a clean path. This is a known limitation of ground-level homography — people walking close to the camera lens experience the most perspective distortion, and the foot-point approximation breaks down when the viewing angle is steep.

The rest of the scene told a clearer story. Almost all trajectories moved in one direction: left to right across the frame, from the street toward the station entrance. This was an evening shoot, and the pattern reflects the commuter flow — people heading home from work. There were no trajectories crossing perpendicular to this flow.

To reduce the near-camera distortion in future shoots, I plan to mount the camera higher on a tripod, which flattens the viewing angle and extends the reliable region closer to the lens.

Discussion

Limitations of Homography

A homography assumes the scene is a flat plane. For a ground-level camera looking across a street, this is a reasonable approximation for people walking on flat pavement. It breaks down for:

Tall objects: A person's head and feet are at different heights, so their bounding box center (used as the tracking point) introduces a height-dependent offset. Using the foot point of the bounding box instead of the centroid reduces this effect.
Sloped terrain: Hills or stairs invalidate the planar assumption.
Wide GCP coverage: Reprojection error increases at points far from the GCP convex hull. The is_in_valid_region filter mitigates this by discarding trajectory points outside the reliable area.

For this project, I chose the foot point of the bounding box as the tracking coordinate, which approximates the actual ground contact point and keeps the homography error small for typical pedestrian heights.

GCP Selection Quality

The quality of the homography depends heavily on GCP selection. Good practices:

Use features clearly visible in both the orthophoto and the video frame
Distribute points across the full area of interest (not clustered in one corner)
Avoid features that might differ between the photo and video (temporary markings, vehicles)
Use more than four points and let RANSAC handle any mismatches

In practice, six to eight well-distributed GCPs produce significantly better results than the minimum four.

Simplification Trade-offs

The angle/speed-based simplification trades off detail for compactness. A threshold of 10° for direction changes and 30% for speed changes preserves the essential shape of each trajectory. Setting thresholds too high risks losing turns; too low retains excessive redundancy. For the information-geometry analysis in later articles, the full pixel-space trajectory (not the simplified WGS84 version) is used to preserve accurate timing information.

Conclusion

By combining GCP-based homography calibration with MapLibre GL JS rendering, I can place pedestrian trajectories extracted from street video onto an interactive geographic map. The pipeline — collect GCPs, compute H, transform coordinates, generate viewer — takes a few minutes per video and produces a self-contained HTML file that runs in any browser.

Bridge

Watching the trajectories animate on the map gives an intuitive picture of pedestrian flow — but comparing across videos or stations requires something more rigorous than visual inspection.

The next step is to construct a distribution from the trajectories in each video, representing the collective movement pattern as a probability distribution over features such as speed, direction, and acceleration. Each video then becomes a single point in a statistical manifold, where the geometry encodes how different two movement patterns are from each other. In the next article, I extract those features and begin constructing that manifold.

Extracting Pedestrian Trajectories from Street Video as JSON

Toki Hirose — Sun, 15 Mar 2026 01:43:41 +0000

Note: I use AI assistance to draft and polish the English, but the analysis, interpretation, and core ideas are my own. Learning to write technical English is itself part of this project.

Motivation

Why extract pedestrian trajectories from smartphone video footage? This approach serves multiple purposes in my research on urban social movements:

GIS-ready data: JSON output integrates seamlessly with geographic information systems and mapping tools
Cost-effective data collection: Eliminates the need for expensive GPS trackers or surveillance infrastructure
Understanding pedestrian behavior: Reveals how people move and interact in urban environments
Measuring protest reactions: Quantifies how standing demonstrations affect surrounding pedestrian flow

This project emphasizes rapid deployment for protest monitoring. The entire setup requires only a smartphone and tripod, enabling quick response to emerging events.

Introduction

In urban planning and transportation studies, understanding pedestrian movement patterns is crucial for designing safer and more efficient public spaces. Previous methods like manual observation or GPS tracking have limitations in coverage and cost. Computer vision offers a scalable alternative through video analysis.

This article demonstrates how to extract pedestrian trajectories from street video footage using open-source tools. I'll use YOLOX-Tiny for real-time person detection and implement a custom centroid-based tracker to generate structured JSON trajectory data. The sample videos used in this project were captured on a smartphone, which keeps the setup lightweight and easy to deploy.

Methodology

Person Detection with YOLOX-Tiny

YOLOX-Tiny is a lightweight object detection model optimized for real-time inference. I use the ONNX export for cross-platform compatibility with OpenCV and ONNX Runtime.

The detection pipeline:

Preprocessing: Letterbox resizing to maintain aspect ratio
Inference: YOLOX model processes the frame
Postprocessing: Convert detections to bounding boxes
Filtering: Confidence thresholding and non-maximum suppression

Centroid-Based Tracking

For tracking detected persons across frames, I implement a simple but effective centroid tracker:

Each detection's bounding box center becomes a centroid
Tracks are maintained by matching centroids between frames
New tracks are registered for unmatched detections
Lost tracks are deregistered after a maximum disappearance threshold
For visualize, footage tracks are also detected

Trajectory Analysis

For each complete trajectory, I extract:

Duration: Total time the person was tracked
Distance: Total pixels traveled
Direction: Movement angle in degrees
Start/End positions: Entry and exit points
Screen exit detection: Whether the person left the frame

Implementation

Setup and Dependencies

# Required packages
pip install opencv-python numpy onnxruntime

# Download YOLOX-Tiny ONNX model
# From: https://github.com/Megvii-BaseDetection/YOLOX

Core Detection Function

def detect_persons(frame, session):
    # Preprocess frame
    blob, ratio = preprocess_yolox(frame, 416, 416)

    # Run inference
    output = session.run(None, {session.get_inputs()[0].name: blob})[0]

    # Postprocess detections
    # ... (filter by confidence, apply NMS)

    return boxes, confidences

Tracking Implementation

Full CentroidTracker Implementation

from collections import defaultdict
import numpy as np

class CentroidTracker:
    """
    centroid-based tracking algorithm for associating detected bounding boxes across frames.
    In addition to tracking centroids, it also maintains trajectories based on the foot point of the bounding box (the point where the person touches the ground), which is more stable for movement analysis.
    """

    def __init__(self, max_disappeared=50):
        self.next_object_id = 0
        self.objects = {}  # ID: (centroid_x, centroid_y)
        self.disappeared = {}  # ID: disappeared_frame_count
        self.trajectories = defaultdict(list)  # ID: [(x, y, frame), ...]
        self.first_seen = {}  # ID: first frame detected
        self.last_seen = {}  # ID: last frame detected
        self.max_disappeared = max_disappeared

    def register(self, centroid, foot_point, frame_num):
        """register a new object with a unique ID"""
        self.objects[self.next_object_id] = centroid
        self.disappeared[self.next_object_id] = 0
        self.trajectories[self.next_object_id].append(
            (foot_point[0], foot_point[1], frame_num)
        )
        self.first_seen[self.next_object_id] = frame_num
        self.last_seen[self.next_object_id] = frame_num
        self.next_object_id += 1

    def deregister(self, object_id):
        """deregister an object and remove it from tracking"""
        del self.objects[object_id]
        del self.disappeared[object_id]

    def update(self, rects, frame_num):
        """
        update the tracker with new bounding box detections

        Args:
            rects: the list of detected bounding boxes [(x1, y1, x2, y2), ...]
            frame_num: the current frame number

        Returns:
            objects: a dictionary mapping object IDs to their current centroids {(cx, cy)}
        """
        # when no detections are present, mark existing objects as disappeared
        if len(rects) == 0:
            for object_id in list(self.disappeared.keys()):
                self.disappeared[object_id] += 1

                if self.disappeared[object_id] &gt; self.max_disappeared:
                    self.deregister(object_id)

            return self.objects

        # conpute centroids and foot points for the current detections
        input_centroids = np.zeros((len(rects), 2), dtype="int")
        input_feet = np.zeros((len(rects), 2), dtype="int")
        for i, (x1, y1, x2, y2) in enumerate(rects):
            cx = int((x1 + x2) / 2.0)
            input_centroids[i] = (cx, int((y1 + y2) / 2.0))
            input_feet[i] = (cx, y2)  # foot point is the bottom center of the bounding box

        # if no existing objects, register all input centroids
        if len(self.objects) == 0:
            for i in range(len(input_centroids)):
                self.register(input_centroids[i], input_feet[i], frame_num)

        # existing objects are present, match input centroids to existing object centroids
        else:
            object_ids = list(self.objects.keys())
            object_centroids = list(self.objects.values())

            # conpute distance matrix between existing object centroids and input centroids
            D = np.zeros((len(object_centroids), len(input_centroids)))
            for i, oc in enumerate(object_centroids):
                for j, ic in enumerate(input_centroids):
                    D[i, j] = np.linalg.norm(oc - ic)

            # find the smallest distance pairs (existing object to input centroid)
            rows = D.min(axis=1).argsort()
            cols = D.argmin(axis=1)[rows]

            used_rows = set()
            used_cols = set()

            for (row, col) in zip(rows, cols):
                if row in used_rows or col in used_cols:
                    continue

                # when distance is lower than a threshold, consider it a match
                if D[row, col] &gt; 100:  # if the distance is too large, ignore the match (this threshold can be tuned)
                    continue

                object_id = object_ids[row]
                self.objects[object_id] = input_centroids[col]  # using the centroid for tracking
                self.disappeared[object_id] = 0
                self.trajectories[object_id].append(
                    (input_feet[col][0], input_feet[col][1], frame_num)  # using the foot point for trajectory analysis
                )
                self.last_seen[object_id] = frame_num

                used_rows.add(row)
                used_cols.add(col)

            # not matched existing objects
            unused_rows = set(range(D.shape[0])) - used_rows
            for row in unused_rows:
                object_id = object_ids[row]
                self.disappeared[object_id] += 1

                if self.disappeared[object_id] &gt; self.max_disappeared:
                    self.deregister(object_id)

            # not matched input centroids
            unused_cols = set(range(D.shape[1])) - used_cols
            for col in unused_cols:
                self.register(input_centroids[col], input_feet[col], frame_num)

        return self.objects

JSON Output Structure

The trajectory data is saved as structured JSON:

{
  "video_name": "street_footage.mp4",
  "fps": 30,
  "resolution": "1920x1080",
  "tracks": [
    {
      "id": 1,
      "duration": 12.5,
      "total_distance": 320.4,
      "trajectory": [
        {"x": 100, "y": 200, "frame": 10, "time_sec": 0.333},
        {"x": 105, "y": 202, "frame": 11, "time_sec": 0.367}
      ],
      "geometry": {
        "type": "LineString",
        "coordinates": [[100, 200], [105, 202]]
      }
    }
  ]
}

Results

Processing a 5min video at 30 FPS typically yields:

The JSON output provides rich data for further analysis:

Spatial patterns of movement
Temporal distribution of pedestrian activity
Flow direction analysis

Discussion

Advantages of This Approach

Cost-effective: Uses commodity hardware and free software
Scalable: Can process hours of footage automatically
Structured output: JSON format integrates with GIS and analysis tools
Real-time capable: YOLOX-Tiny enables live processing

The sample videos in this project were captured on a smartphone, but the same pipeline can be applied to fixed surveillance cameras for longer-term monitoring.

Interpretation

Processing demonstration videos from Shinbashi station revealed insights about centroid tracking performance and pedestrian behavior during protests:

Commuter indifference: In Japan, individual protests are uncommon, so commuters typically ignore demonstrators. Additionally, most people are busy office workers who tend to focus on their commute rather than noticing activities around them.
Camera height issues: Using a smartphone camera with a low tripod created unreliable detections. People near the camera appeared with unnatural up-and-down trajectories due to the low-angle perspective.
ID swapping during interactions: When pedestrians crossed paths or interacted closely, their tracking IDs would swap, creating fragmented trajectories for the same individuals.

Overall, the system successfully captured general movement patterns. Future improvements could include filtering trajectories with sudden angle changes after intersections or removing outliers based on historical movement differences.

Limitations and Future Improvements

Occlusion handling: Simple centroid tracking fails in crowds
Camera motion: Assumes static camera position
Identity persistence: No re-identification across camera cuts
Stopping behavior: People who stop moving in videos sometimes lose their tracking ID due to centroid distance thresholds, leading to fragmented trajectories (e.g., ID 5 → 110 → 430 as the same person gets re-detected with new IDs)

For crowded scenes, more sophisticated trackers like DeepSORT or ByteTrack would improve performance. Camera motion compensation using optical flow could extend applicability to moving platforms.

In this project, I prioritized spending time on analysis and visualization rather than implementing the most advanced tracking pipeline; that tradeoff made it easier to iterate quickly with real data.

Applications

This trajectory data serves as input for:

Urban planning: Identifying pedestrian flow bottlenecks
Safety analysis: Detecting high-risk crossing patterns
Traffic engineering: Optimizing signal timing
Accessibility studies: Understanding mobility patterns

The structured JSON format makes it easy to integrate with mapping libraries like MapLibre GL JS for visualization, as I'll explore in the next article.

Conclusion

By combining YOLOX-Tiny detection with centroid tracking, I can extract meaningful pedestrian trajectory data from video footage. The resulting JSON structure provides a foundation for spatial analysis of urban movement patterns. While the current implementation works well for moderate-density scenarios, future enhancements could address occlusion and camera motion challenges.

In the next article, I'll visualize these trajectories on an interactive map using MapLibre GL JS.

reference

Download YOLOX-Tiny ONNX model
- From: https://github.com/Megvii-BaseDetection/YOLOX
Centroid tracker
- https://pyimagesearch.com/2025/07/14/people-tracker-with-yolov12-and-centroid-tracker/
my github project
- https://github.com/TOKIHISA/people_trajectory_analysis/blob/main/src/detects_people.py