On the cover, a conceptual rendering of a drone in an event stream alongside the edge of an industrial component.
On 15 June 2026, Prophesee launched Mantara, a system that uses event-based vision and onboard artificial intelligence to detect and track drones. The processing runs on the Hearth software platform. [1] At VISION 2026 in Stuttgart (6–8 October), IDS Imaging Development Systems demonstrated the same principle by tracking an air hockey puck with an event camera. IDS and Prophesee had announced the demonstration on 1 October. [2] [3] On a production line, this type of imaging can capture rapid motion such as a vibrating component or individual droplets ejected at high speed. [4]
Measuring the frequency of a vibration
In an example documented by Sony Semiconductor Solutions, a model car vibrates on a platform. Its moving edges produce changes in brightness, which the software uses to estimate and map vibration frequencies. The figure shows the principle using a flexible strip. For a regular oscillation, the interval between two successive peaks in the tip’s position gives the period. Frequency is the reciprocal of that period. [4] [5]
The vertical arrow shows how the tip moves. The illustrative curve shows its estimated position over time. The coral bracket between successive peaks marks one oscillation period. [4] [5]
The estimate relies on an edge with sufficient contrast. Reporting the vibration amplitude in millimetres requires the image scale, the viewing geometry and an assessment of camera movement relative to the part. Comparing the result with a reference instrument under operating conditions helps establish the measurement error. [4] [5]
What an event sensor records
A frame camera records light levels across the entire pixel array in each image. An event sensor produces an asynchronous sequence of changes reported by individual pixels. As a bright tile moves to the right over a dark background under steady lighting, pixels reached by the leading edge get brighter. Pixels left behind by the trailing edge get darker. Stationary parts of the scene are still visible in the frames and contribute little to the event stream under these conditions. [4] [5] [6]
As the plate moves right, its trailing edge reveals the dark background and produces OFF events. Its leading edge covers the background and produces ON events. The bottom row collects events between the two frames. [5] [6]
In the model described by Sony Semiconductor Solutions and Prophesee, a photoreceptor converts incident light to an electrical signal with a logarithmic response. The pixel circuit compares the signal with a reference set at its previous activation. When the accumulated change reaches the threshold in either direction, the pixel emits an event and updates its reference. Increases and decreases can have different thresholds. A slow change can produce an event if it reaches the required contrast, while a smaller change stays below the threshold. [4] [5] [6]
The white curve shows the pixel’s change in log brightness (ΔL) relative to its reference over time. The cyan dot marks when ΔL reaches the positive threshold +C and the pixel emits an ON event. [4] [5] [6]
The Metavision documentation represents an event as (X, Y, P, T). X and Y are pixel coordinates, P is polarity and T is the timestamp, expressed in microseconds in that format. Prophesee encodes ON as 1 and OFF as 0. Each event thus identifies a local change. Determining whether it belongs to a droplet edge, a vibrating part or a change in lighting requires further observations and an algorithm. [5] [6]
Timing, latency and data volume
Timestamp resolution is the granularity with which time is represented. To determine when a system can deliver a result, the pixel response, readout, transfer and processing also have to be considered. [4] [5] [6]
Three quantities to distinguish
| Quantity | What it measures |
|---|---|
| Timestamp resolution | The increments used to express an event’s time. |
| Acquisition and transfer time | The interval between the light change and the event becoming available to software. |
| Application latency | The time from a scene change to a usable result, including any wait for processing. |
When several pixels fire together, the readout circuitry has to determine the order in which their events leave the sensor. The photoreceptor also has a finite response. Gallego and colleagues describe a test in which a light-emitting diode (LED) with modulated intensity stimulated a single pixel in a DVS128 sensor. The pixel responded over a narrower frequency range when the background light was reduced. That measurement concerns the pixel under the test conditions. The latency of the complete system also depends on the interface and downstream computation. [4] [5]
To display the stream, software accumulates events on a grid. The length of the window determines how much history the image shows. The refresh rate determines how often the software rebuilds it. If the algorithm waits for the window to close before calculating a result, that wait contributes to latency. [5] [6]
In a stable scene, activity may be concentrated around a few edges. Widespread motion, reflections or illumination changes across many pixels can rapidly increase the flow. The bandwidth required depends on the number of events per unit of time, their encoding and the interface that carries them. Downstream memory use also depends on how many events the algorithm keeps and on the representation it builds. Comparisons with frame cameras therefore need to account for peaks as well as averages, using the same scene and task. [4] [5]
Tracking moving droplets
In another application, Sony tracks droplets ejected at high speed. Motion streaks overlap in the frame image in the comparison, while event processing separates the trajectories and assigns each droplet an identifier, position, apparent width and estimated speed. Separation depends on visible edges and on the algorithm’s ability to associate successive observations. Reflections, nearby droplets and overlapping projections make that association more ambiguous. Width in the image plane is distinct from physical diameter and needs calibration before it can be converted to a dimensional measurement. [4] [5]
From the event stream to an onboard result
In an industrial pipeline, the receiver decodes the stream and preserves coordinates, polarity and time information. A filter can suppress isolated events that have little support from nearby pixels or limit contributions from regions outside the measurement area. The algorithm can then work on small groups of events or on a grid that records the time of the most recent activation at each pixel. This representation, called a time surface, shows the recent history of edges and discards earlier activations at the same location. [4] [5] [6]
Position estimation, association and classification use these representations in different ways. A longer window can give an estimator more evidence, with the tradeoff that it can mix different phases of motion and delay the response. Grids make it easier to reuse algorithms designed for dense data, while direct processing can take advantage of sparse event distributions. The balance between accuracy, latency, memory and computation depends on how the result will be used. [5]
Operating conditions and limits
The stream responds to any change in the intensity reaching a pixel. A pulsing light source can activate large areas even if a part stays still. If the camera vibrates, fixed edges on the machine also move across the array. An edge with little contrast may produce few events even when it moves. Photoreceptor noise and differences in pixel thresholds affect the output, particularly at higher sensitivity. Filtering can reduce some of that noise and can also remove weak signals that matter to the measurement. [4] [5]
Combining frames with event-based sensing is one of the directions IDS and Prophesee set out in their letter of intent, signed on 11 March at Embedded World in Nuremberg. It builds on their work on uEye EVS industrial cameras, where EVS stands for event-based vision sensor. [7]
When a line needs to read a code or inspect a stationary defect, a frame preserves appearance details that can give the event map context. When the task is to distinguish closely spaced passages or estimate the period of a moving edge, the timing of the changes becomes especially valuable. Combining the two signals for mixed applications requires aligning images and events to the same scene and time base. [5] [7]
A comparison on the factory floor needs to use the same measurement task under the line’s lighting and motion conditions. The deciding factors are error against a reference instrument, time from motion to a usable result and data volume at peak activity. [5]
Sources and further reading
- Prophesee, Mantara launch announcement (15 June 2026).
- Prophesee and IDS, VISION 2026 demonstration announcement (1 October 2026).
- IDS, video of the VISION 2026 demonstration (from 0:43).
- Sony Semiconductor Solutions, Event-based Vision Sensor Technology.
- G. Gallego et al., Event-based Vision: A Survey (full text), IEEE Transactions on Pattern Analysis and Machine Intelligence. DOI 10.1109/TPAMI.2020.3008413.
- Prophesee, Event-Based Concepts.
- IDS, letter of intent and collaboration with Prophesee (11 March 2026).
Originally published on SiliconLogix.



Top comments (0)