Introduction
Over the previous three articles, we have looked at EIS (Electronic Image Stabilization) from multiple angles.
- The first article laid out the differences between OIS, EIS, and HIS, positioning EIS within the broader landscape of image stabilization.
- The second article took us inside the EIS pipeline through Gyroflow, an open-source project.
- The third article examined the collision between Rolling Shutter and EIS, explaining why RSC (Rolling Shutter Correction) is necessary. This final article closes the EIS series. So far we have focused on "how current EIS works." This time we focus on "where EIS is heading."
From a camera module developer's perspective, three major trends are palpable on the ground today — AI-based EIS, sensor technology evolution, and software-defined cameras. Predicting the future always carries the risk of being wrong, so instead of offering the rosy outlooks typical of marketing materials, I'll focus on changes I actually feel in module development work.
AI-Based EIS — From Gyros to Learning
The Limits of Conventional EIS
The EIS we have discussed so far was fundamentally a combination of gyro sensors + traditional computer vision + mathematical models.
- Read angular velocity from the gyro to estimate camera orientation
- Separate intended motion from shake using a low-pass filter
- Warp frames using rotation matrices
- Correct row-by-row timing differences with RSC This structure has been validated over many years and is still used in most commercial products. However, it has a few structural limitations.
First, it is weak against translation. Gyros only measure rotation. Vertical bounce while walking, or back-and-forth motion inside a vehicle, cannot be fully estimated from gyro data alone. Even with an accelerometer added, integration drift becomes a major problem.
Second, it does not account for scene context. Gyro-based EIS only knows "how much the camera moved," not "what is in the frame." It applies the same correction whether the subject is centered, near, or far.
Third, crop margin is wasted. A generous crop margin is reserved to handle worst-case shake, but such large shake rarely occurs in practice. As a result, FOV is sacrificed unnecessarily most of the time.
Where AI Steps In
Deep learning-based approaches tackle these limitations in a different way. There are three main directions.
1. Learned motion estimation
Instead of or alongside gyro data, pixel-level motion between frames (optical flow) is estimated using learned models. This provides a motion field that takes into account subject depth and scene structure. Google Pixel's video stabilization is said to have evolved in this direction.
2. Learned frame synthesis
Areas lost to cropping are filled in using information from surrounding frames, or heavily shaken frames are reconstructed from multiple frames. This can reduce crop margins or even retain most of the original FOV. Research tracks like "Motion-Aware Video Frame Interpolation" and "Neural Video Stabilization" fall in this direction.
3. End-to-end learning
A neural network learns directly from input video to stabilized output video, without a traditional pipeline. While many good results have been reported in research, commercial deployment remains limited due to challenges in real-time mobile processing and output consistency.
The Reality of AI EIS on the Ground
Reading academic papers and tech media, AI-based EIS looks like it has already solved everything. The reality in module development is a bit different.
Power and thermals. Running neural inference in real time on mobile devices requires NPUs or GPUs. Keeping them under high load throughout video capture causes serious heat and battery drain. The typical compromise is "high-quality AI EIS only in specific modes."
It does not fully replace the gyro. AI-based motion estimation works well when the scene has many feature points and lighting is sufficient, but performance degrades sharply in low light or against monochromatic backgrounds. Realistic products are moving toward a hybrid of gyro and AI: the gyro guarantees a baseline of stability, while AI adds scene awareness to push quality higher.
Training data bias. Neural network performance depends heavily on training data. It performs well in certain scenes (bright outdoors, urban environments), but in environments not represented in training data (underwater, extremely low light, industrial settings), unpredictable artifacts can emerge. From a camera module supplier's perspective, this translates into increased validation burden.
In summary, AI-based EIS is a clear direction, but rather than "gyro-based EIS fully exiting the stage," the more accurate picture from the field is that "a gyro + AI collaborative structure is becoming the standard."
Sensor Technology Evolution — Hardware for EIS
It is easy to focus solely on software algorithms, but much of EIS's progress owes itself to advances in sensor hardware. Here are a few changes I see on the ground.
Higher-Precision, Smaller Gyro Sensors
The performance of 6-axis IMUs built into smartphones has visibly improved in recent years. Sampling rates now exceed 1–2 kHz, and noise density and drift characteristics continue to improve. This directly lifts the quality of data that EIS depends on.
Perhaps more importantly, hardware synchronization between image sensor and gyro is becoming standard. The readout time synchronization issue discussed in the third article directly affects gyro timestamp accuracy, and recent platforms solve this at the hardware level.
Stacked CMOS and Fast Readout
Stacked CMOS technology, pioneered by Sony, separates the pixel layer and the circuit layer and stacks them. This allows more circuit area without increasing the overall sensor size, enabling significantly faster readout.
Faster readout directly reduces Rolling Shutter distortion. Rolling Shutter distortion is fundamentally proportional to "the time difference between the top and bottom rows of a frame," so reducing that time difference reduces the distortion, even with a Rolling Shutter architecture.
Going further, Stacked sensors with integrated DRAM use high-speed buffering to approximate Global Shutter behavior. They are not true Global Shutter, but they offer a realistic way to significantly mitigate Rolling Shutter's drawbacks.
Global Shutter on Mobile?
In theory, Global Shutter sensors would eliminate Rolling Shutter problems at the root. So will mobile eventually transition to Global Shutter?
From an image-capture camera perspective, not anytime soon. As discussed in the third article, the triple wall of pixel size, low-light performance, and cost remains high. This applies not only to the main camera but equally to the front camera. The primary use cases for front cameras (selfies, video calls) are more often in low-light conditions, and there is little high-speed distortion to solve, so there is weak motivation to switch to Global Shutter. As a result, cameras that users "shoot" with — whether main or front — are likely to continue using Rolling Shutter + RSC.
On the other hand, sensing-oriented cameras follow a different path. Receivers in ToF and structured light systems, tracking cameras in AR/VR headsets, barcode scanners in industrial devices — in the domain of sensors that the system uses to perceive the world, rather than image-makers for the user, Global Shutter is already close to standard. In this domain, low latency and distortion-free instantaneous capture matter more than low-light performance.
In other words, "Global Shutter is entering mobile" is accurate, but the path is sensing, not imaging. In the territory that EIS addresses — user-facing video — Global Shutter is unlikely to become mainstream in the near future.
Gyro-in-Sensor
There is movement toward integrating a gyro sensor within the image sensor package itself. This minimizes the physical distance between image sensor and gyro, allowing vibration phases to match more precisely and hardware synchronization to be cleaner.
From a module developer's perspective, this is welcome — the BOM simplifies, and the sensor vendor takes responsibility for gyro characteristics as well. The trade-off is reduced flexibility in sensor selection.
Summary: Hardware Sets the Ceiling for Algorithms
No matter how refined an EIS algorithm is, there is a clear ceiling if input data quality is poor. Conversely, when sensor and gyro quality improve, the same algorithm yields visibly better results.
From a software developer's standpoint, it's easy to say "the algorithm determines quality." But working in module development, you feel every day that hardware sets the ceiling for algorithms. Even in the age of AI-based EIS, this relationship will not change.
Software-Defined Camera — Boundaries Disappear
What is a Software-Defined Camera?
Software-Defined Camera (SDC) is not a term with a rigid definition, but the concept circulating in the industry can be summarized as:
"A structure in which a camera's final output quality is determined primarily by software processing (especially AI/ML), rather than by hardware such as sensors and lenses."
This concept is not new. Features like Google Pixel's HDR+, Night Sight, and Magic Eraser were early forms of SDC. The reason Pixel — whose hardware specs appeared unremarkable on paper — maintained top-tier photo quality was entirely due to software.
In the context of EIS, what SDC means is that "stabilization is no longer something software does after the sensor captures the frame; the entire capture pipeline is designed with stabilization as a built-in premise."
A Pipeline Without Boundaries
The traditional camera pipeline was divided into these stages:
Sensor readout → RAW → ISP (demosaic, noise reduction, tone mapping) → EIS/RSC → Encoder
Each stage received the output of the previous one and processed it independently, in a block structure. Recent pipelines, however, are rapidly blurring these boundaries.
Multi-frame Fusion. Instead of a single frame, multiple frames are taken in simultaneously, and noise reduction, HDR, super-resolution, and stabilization are all performed together. Here, EIS is no longer an independent stage — it becomes part of the multi-frame alignment process.
Neural ISP. There is a movement toward replacing or augmenting traditional ISP blocks with neural networks. Demosaicing, noise reduction, and detail recovery are being unified into a single network. If this trend expands to include EIS as well, an end-to-end network of "RAW input → stabilized and corrected output" becomes a real possibility.
Computational Viewfinder. The viewfinder image the user sees is already a computed result. It used to be "preview is light, high-end processing applies only to the final capture." Now, the preview is processed close to the final result. EIS has become a feature that must operate from the preview stage.
A Module Developer's Dilemma
The shift toward SDC places camera module developers in a peculiar position.
On one hand, because software compensates for hardware weaknesses, there is more headroom in module design. Optical defects that would have been classified as defects in the past can now be saved by software correction.
On the other hand, optimizing only for the module's "optical performance" is no longer enough. What matters is how well the module meshes with the entire software pipeline. Even with the same image sensor, the final result changes completely depending on ISP tuning, gyro synchronization accuracy, and calibration data quality.
As a result, the value chain of the camera module industry is shifting from "pure hardware supply" toward the integrated provision of "hardware + calibration data + drivers + ISP tuning." This is precisely the change I have been feeling firsthand since starting this series.
Where EIS Is Heading — Closing the Series
Closing this four-part EIS series, I want to organize a few thoughts that run through the whole. What started as a simple story about shake correction turned out to be a story that expanded to cover the entire camera system.
First, EIS is not just a "shake-correction feature." It is an integrated problem that spans the full camera system — gyro, sensor, ISP, lens correction, encoder. That is why this series covered OIS/HIS, the pipeline, Rolling Shutter, and AI.
Second, the evolution of EIS is the interaction of hardware and software. A good algorithm reaches its true potential only on a good sensor, and a good sensor only becomes meaningful when paired with well-tuned algorithms. Discussing EIS while looking at only one side is half an understanding.
Third, the role of the camera module developer is changing. In the past, the core was "supplying optical modules that meet specs." Going forward, competitiveness lies in "providing systems that integrate into a software pipeline." I write this series from the middle of that shift.
There are many topics on EIS that this series did not cover — actual tuning workflows, EIS in automotive cameras, the very different requirements of VR/AR, and more. Future articles may pick these up as the opportunity arises.
The next series will address another pillar of camera module development: ISP tuning. If EIS was "the domain of motion," ISP is "the domain of light and color."
This article is based on practical experience in camera module development. Statements about the future reflect a reading of publicly known technology trends and changes observed on the ground, and are not predictions or descriptions of any specific product roadmap. Comments, corrections, and opinions are always welcome.
Top comments (0)