DEV Community: Ross Kim

Light Does Not Land Evenly

Ross Kim — Wed, 05 Aug 2026 12:43:45 +0000

Uneven Light

Last time I looked at how AE decides the amount of light, through the frame of the two seats. But an amount being settled doesn't mean that light lands evenly across the frame. The edges receive less light than the center, and sometimes light of a different color. This installment looks at that unevenness, lens shading, through the same frame.

Lens shading refers both to the phenomenon, in which the amount and color of light vary by position as light passes through the lens and sensor, and to the correction that flattens it back out. Photograph the same white surface and the center comes out bright while the edges come out dark, the center neutral while the edges are faintly tinted. Lens shading correction fills in this positional mismatch so that a single light looks the same everywhere in the frame.

What Lens Shading Decides

The mismatch it addresses runs along two strands: light falling off toward the edges (luminance shading), and the color of the light shifting by position (color shading).

Luminance shading is also called vignetting. Light passing through the lens at an angle to reach the sensor's edges is diminished compared to light entering straight at the center. On top of optical falloff, when the angle at which the lens bends light diverges from the angle at which a pixel accepts it, the edges gather less light. So even a uniform surface darkens from center to edge.

Color shading is trickier. Light striking the edges at an angle passes through the infrared cut filter at a different angle, and that filter's transmission characteristics vary with angle, so the color at the edges shifts slightly. The behavior of the microlens and color filter above each pixel is also sensitive to the angle of incidence. So white that was neutral at the center takes on a faint green or magenta tint at the edges.

Correction is a matter of multiplying by a different gain at each position. The frame is divided into a grid, a surface of positional gains (a gain map) is built for each of the R, G, and B channels, and multiplying by that surface lifts the darkened edges and pulls the tinted color back to neutral. The map is built by photographing a uniform white surface under uniform light and measuring how it actually darkens and how it actually shifts. Measurement → gain map → correction → switching surfaces per illuminant. That is the skeleton of lens shading.

The Core Is Fixing the Camera, Not the Scene

Within that skeleton, what most reveals lens shading's character is that what it fixes is not the scene.

AE responds to how bright the scene is. White balance, coming later, responds to what light the scene sits under. Both follow the world outside. Lens shading is different. The edges are dark and tinted not because of the scene but because of how this camera's lens and sensor take in light. Whatever you photograph, that mismatch is stamped onto every frame in nearly the same shape. Lens shading correction doesn't fix the scene. It erases the mark the camera leaves on every photograph it takes.

So this work is unit-specific. The shape of the mark, which regions darken by how much and which edges shift in which direction, comes out of the lens, the sensor, the infrared cut filter, and the alignment where they meet, and that combination differs slightly from module to module. That said, the mark isn't entirely fixed. Color shading varies with the color of the light. The edge tint under daylight is not the same as the tint under incandescent, so when the light changes the shape of the mark changes too, and a correction fit under one light doesn't quite fit under another. It is the camera's own problem, and yet it remains tethered, by one thread, to the light outside.

In the Two Seats, This Work Diverges

The two seats grip the work of measuring and flattening this mark from different ends.

In the ISP seat, the asset is the shading algorithm itself. What model represents the surface of positional gains (a grid, or a radial function), how many reference illuminants the differing surfaces are measured at and how the space between them is filled, which surface gets selected in real time, and how that selection interlocks with the estimate of the light's color. These are the handles this seat moves. But this seat does not choose which lens or sensor it will be paired with. Falloff has a different shape from lens to lens, and edge behavior differs from sensor to sensor. That optical mark is a given input, and the model representing the gain surface must be able to hold not one mark but a wide range of them. In this seat, the module's optical characteristics are not a handle you can move but a wall you cannot cross.

In the module seat, that handle and that wall swap places. The shading algorithm is now a given: it arrives with the ISP this module will run on. And the module, already designed and in production with its lens, sensor, and infrared cut filter, is fixed. What can be moved are the surfaces the algorithm leaves open. Under each reference illuminant you photograph a uniform white surface, measure how this module actually darkens and shifts, and seat the gain map (calibration); you decide how many anchor points to place between illuminants; and you bring the residual tint inside the allowed range. The optical mark that one seat cannot cross as a wall, the other measures directly under uniform light and uses as the starting point of the gain map. The same mark is a wall in one seat and a handle in the other.

So "It Went Well" Means Different Things

Both seats share the goal of passing spec. What diverges is what each protects beyond it.

What the ISP seat protects is breadth. A shading algorithm that flattens the mark of one lens and one sensor isn't enough. It has to hold optical systems with differently shaped falloff in a single model, fill the space between illuminants smoothly, stay stable where it interlocks with the estimate of the light's color, and offer a clean calibration interface where the integrating side can seat its own module's surfaces. Only then does that algorithm get chosen again on the next project. In this seat, lens shading "going well" is closer to a generality reused across optical systems.

What the module seat protects is repeatability. This one module passing its shading spec (uniform brightness across the frame under defined illuminants, residual tint below threshold) isn't the end. In production, slight lens decentering, assembly tilt, and mismatch between the microlens and the chief ray angle (CRA) shake the shape of the mark from unit to unit. Fit the gain map too tightly to a single golden sample and the unit on the next line ships with edges visibly dark or visibly tinted. In this seat, lens shading "going well" is closer to a specificity that repeats on this module.

Before Asking What White Is

Only once lens shading has flattened the frame can we say the frame sits under a single light. And then the next question opens. What color is that light, and what do we call white? What's more, as long as color shading varies with the color of the light, even choosing which surface to apply leans somewhat on knowing what color the light is. The next seat can't be put off.

The seat that measures this module's mark and the seat that catches that mark in a model: I have watched from both sides where the correction surface ends up between them. Next time I move the frame to the color of that light: white balance (AWB).

Who Decides How Much Light

Ross Kim — Fri, 31 Jul 2026 11:03:12 +0000

The Decision at the Very Front

Last time I said that the same word, "tuning," names different work depending on which seat you sit in. One seat holds the ISP as its asset and makes that algorithm behave consistently across many inputs. The other holds a specific module as its asset and makes it deliver the required image on top of a given ISP. This installment looks at the first block, AE, through that frame.

AE, or auto exposure, is the job of deciding how much light the camera takes in from a scene. As the decision at the very front, where the image begins, it earns its place as the series' first block. Every block that follows sees only the light AE let through. Getting white right, moving color: all of it starts on top of the brightness AE has already set. Take in too little and there isn't enough signal to even discuss color. Take in too much and information is gone, beyond the reach of any correction downstream.

What AE Decides

AE deals with two things in the end. How much light to take in from the scene, and how to produce that amount.

"How much" comes from measuring the scene: how to divide the frame and where to put the weight when measuring brightness (metering), and what brightness to bring that measurement to (the reference brightness). In backlight, do you save the subject or the background? How far does "the subject" extend? That judgment decides the "how much."

"How" comes from splitting that amount between exposure time and gain. In a mobile module the aperture is usually fixed, so there are effectively only two roads to the same brightness: expose the sensor longer, or amplify the signal more. And finally, how quickly and smoothly should these values follow when the scene changes (convergence). Metering → reference brightness → allocation → convergence. That is the skeleton of the AE block.

The Core Is a Decision About Splitting a Budget

Within that skeleton, what most reveals AE's character is the "how": splitting the amount of light between exposure time and gain.

If you see the amount of light as a budget, AE is the job of allocating that budget across two line items. Fill the same brightness with exposure time and hand shake and subject motion smear. Fill it with gain and noise rises. Neither is free. So at the heart of AE tuning sits a rule that decides "at this brightness, how far do I allow time, and from where do I start raising gain?" It's commonly called the program line: a curve joining (exposure time, gain) pairs across brightness.

In the Two Seats, This Decision Diverges

The two seats grip this rule from different ends.

In the ISP seat, the asset is the AE algorithm itself. The metering engine that measures the scene, the convergence control that carries values through scene changes without shaking, and the shape of the program line. These are the handles this seat moves. But this seat does not choose which sensor or module its algorithm will be paired with. Each sensor's gain-versus-noise behavior and each module's response are given inputs, which is why an AE engine must hold up not on one sensor but across many. In this seat, the sensor's characteristics are not a handle you can move but a wall you cannot cross.

In the module seat, that handle and that wall swap places. The AE algorithm is now a given: it arrives with the ISP this module will run on. And the module, already designed and in production with its sensor, lens, and actuator, is fixed. What can be moved are the values the algorithm leaves open. You seat the program line against this sensor's actual noise and motion-blur behavior, and you adjust metering weights and reference brightness to the brightness and subject priority the client wants.

So "It Went Well" Means Different Things

Both seats share the goal of passing spec. What diverges is what each protects beyond it.

What the ISP seat protects is breadth. An AE engine converging well on one module isn't enough. It has to converge consistently, without hunting, across many sensors and many customers, and it has to expose a parameter surface that the integrating side can adjust to its own module. Only then does that engine get chosen again on the next project. In this seat, AE "going well" is closer to a generality that gets reused.

What the module seat protects is repeatability. This one module passing the client's AE spec (brightness accuracy under defined conditions, convergence speed, no flicker banding) isn't the end. That pass has to hold across the sensor variation that shifts from unit to unit in production. Fit the allocation too tightly to one unit and it falls apart on the next line. In this seat, AE "going well" is closer to a specificity that repeats on this module.

After the Amount of Light Is Set

Once AE sets the operating point, the next question opens on top of it. Does the light taken in land evenly across the frame? Even light whose amount is settled falls off at the edges, and sometimes changes color there.

Having passed through both seats, I've watched up close which handle and which wall these decisions get made between. Next time I move the frame to that unevenness: lens shading.

ISP Tuning as a Job

Ross Kim — Thu, 30 Jul 2026 12:20:46 +0000

Same Word, Different Work

In camera module work, the word "tuning" comes up constantly. But it points to entirely different jobs depending on which seat you're sitting in.

For one person, tuning means refining algorithms and parameters so that the ISP they built behaves consistently across many kinds of sensors and lenses. For another, tuning means making one specific module they designed and are about to mass-produce deliver the image a client asked for, on top of an ISP they were handed.

Both say they "went through hell tuning it." But what one moved and what the other moved are not the same thing. What they hold in their hands and what they accept as an immovable wall differ, and so does the standard by which they can say it's finished.

This series follows that difference. It looks at how the same word, "tuning," names two different jobs in two different seats, by opening up the core image-formation blocks one at a time.

What Separates the Seats Is the Asset

"Seat" here doesn't mean a job title or an employer. What separates the seats is what you hold as the asset you can sell.

One seat's asset is the ISP: the processing blocks that turn light into an image, the algorithms inside them, and the ability to run all that on many different pieces of hardware. What this seat sells, in the end, is a promise: "our ISP produces a good image no matter which sensor and lens it meets." In this seat, the module is not an asset. It is one of countless inputs the ISP must withstand.

The other seat's asset is the module: binding lens, sensor, and actuator into a single physical component, designing it, and mass-producing it at a defined quality. What this seat sells is the promise that "this module meets spec and can be produced reliably at volume." In this seat, the ISP is usually not an asset. It arrives with the SoC or comes from outside. It is a given condition.

So the same "tuning" becomes different work in the two seats. One refines its own asset, the ISP, to fit someone else's hardware. The other fits its own asset, the module, to someone else's processing chain. The direction the refining hand points is opposite from the start.

From here on, I'll call the first the ISP seat and the second the module seat.

Handles and Walls Swap Places

Tuning is, in the end, moving something in order to satisfy something. So: what can you move, and what can you not? The arrangement of handles and walls swaps between the two seats.

In the ISP seat, what moves is the ISP itself. There is freedom in how the processing blocks behave and in their parameters. The module, conversely, is a wall. And this seat isn't fitting to a single module. It assumes a range of modules as input and looks for behavior that won't collapse anywhere across that range.

In the module seat, the arrangement is reversed. Once design is finished and volume production begins, the module hardware is effectively frozen. The lens characteristics and the sensor's behavior are already settled. What can be moved are the ISP's settings sitting on top of it, and even those have less freedom than in the ISP seat, since the ISP itself is a condition handed down from outside.

Both seats aim at a spec the client has defined. Once a project is locked and the client and target product are fixed, a spec that must be met exists for either seat. The mere existence of a spec does not separate them.

What separates the two seats is not whether a spec exists, but what each can move toward it. Facing the same spec, what was a handle in one seat becomes a wall in the other.

"It Went Well" Means Different Things

If handles and walls are arranged differently, then what it means to say "the tuning went well" differs too. Both seats must pass the spec, as we've seen. Where they diverge is beyond that passing. Past the spec, each seat has something else it must protect.

What the ISP seat protects beyond passing is generality. Meeting this project's spec isn't the end. The ISP must not collapse anywhere across the range of hardware it is expected to carry. Only then can it say, to the next customer and the next project, "just use ours." Tuning in this seat hits the spec in front of it while simultaneously holding up a reusable breadth.

What the module seat protects beyond passing is repeatability. One unit passing the spec isn't the end. Every unit coming off the line must hold that result. Tuning in this seat is optimized toward binding fixed hardware stably to a defined spec.

Holding up breadth and holding up repeatability are different jobs. Even when touching the same block, the two seats touch it to pass the same spec while protecting different things.

The Cross-Section This Series Examines: Four Blocks

Many processing blocks are involved as an ISP turns light into an image. This series doesn't cover all of them. I chose only the cross-section that most sharply reflects the difference between the two seats.

There are four blocks. The order follows the pipeline, the order in which light becomes an image.

AE (Auto Exposure) — decides how much light to take in.
Lens Shading Correction — corrects the light falloff and the color shift that grow toward the edges of the frame.
AWB (Auto White Balance) — estimates the color of the light source and makes what people perceive as white come out white.
CCM (Color Correction Matrix) — moves all the colors beyond white closer to what a person sees. There's a reason lens shading sits before AWB. If the frame hasn't been flattened, the color cast at the edges muddies the very evidence AWB uses to estimate the illuminant. You have to flatten the frame before you can ask what white is.

Two criteria drove the selection. First, the block had to make the division of labor between the seats clearly visible. Second, it had to belong to the core of image formation: the amount of light, how evenly that light lands across the frame, the color of the illuminant, and the reproduction of color. All of these are work at the stage where the image is formed, which is different in character from post-processing that polishes an image already formed.

It's only honest to state why the other blocks are left out. This is not a ranking of importance. It's a question of whether they're cleanly captured by this series' frame.

AF (Auto Focus) belongs more to mechanics and calibration than to ISP tuning. Bringing something into focus is a different kind of problem from what happens inside the chain that turns light into an image.

Blocks like NR (Noise Reduction), Sharpening, and Gamma/Tone vary enormously in structure from one ISP to the next, and "how much is good" is deeply subjective. These blocks are handled differently in the two seats as well, but the difference comes less directly from asset structure and gets buried in per-ISP implementation and taste. The "two seats" frame doesn't cleanly separate them.

So this series looks at four blocks only. Not everything that could be covered, but only what this frame makes sharp.

The Distance Varies by Block, and That Belongs to Each Installment

One thing to nail down in advance: the distance between the two seats is not the same across the four blocks. In some, the seats do nearly the same work. In others, nearly opposite work.

But how that distance varies by block is not something I'll settle here in Part 1. If I concluded up front which block is farthest and which is closest, each installment that followed would become an exercise in retroactively justifying that conclusion. The distance comes not from the seats but from the nature of the block itself, and that nature only reveals itself when you actually look inside. So I leave that judgment to each installment.

About the seat I'm writing from, one line is enough. I have passed through both of these seats and watched, up close, how the same work is carried out on top of different decision structures. What this series rests on is that observation.

From the next installment, I open the blocks one at a time. The first is AE: how much light to take in. The decision at the very front, where the image begins.

The Present and Future of EIS — A Camera Module Developer's View on AI, Sensors, and Software-Defined Cameras

Ross Kim — Wed, 22 Apr 2026 11:43:32 +0000

Introduction

Over the previous three articles, we have looked at EIS (Electronic Image Stabilization) from multiple angles.

The first article laid out the differences between OIS, EIS, and HIS, positioning EIS within the broader landscape of image stabilization.
The second article took us inside the EIS pipeline through Gyroflow, an open-source project.
The third article examined the collision between Rolling Shutter and EIS, explaining why RSC (Rolling Shutter Correction) is necessary. This final article closes the EIS series. So far we have focused on "how current EIS works." This time we focus on "where EIS is heading."

From a camera module developer's perspective, three major trends are palpable on the ground today — AI-based EIS, sensor technology evolution, and software-defined cameras. Predicting the future always carries the risk of being wrong, so instead of offering the rosy outlooks typical of marketing materials, I'll focus on changes I actually feel in module development work.

AI-Based EIS — From Gyros to Learning

The Limits of Conventional EIS

The EIS we have discussed so far was fundamentally a combination of gyro sensors + traditional computer vision + mathematical models.

Read angular velocity from the gyro to estimate camera orientation
Separate intended motion from shake using a low-pass filter
Warp frames using rotation matrices
Correct row-by-row timing differences with RSC This structure has been validated over many years and is still used in most commercial products. However, it has a few structural limitations.

First, it is weak against translation. Gyros only measure rotation. Vertical bounce while walking, or back-and-forth motion inside a vehicle, cannot be fully estimated from gyro data alone. Even with an accelerometer added, integration drift becomes a major problem.

Second, it does not account for scene context. Gyro-based EIS only knows "how much the camera moved," not "what is in the frame." It applies the same correction whether the subject is centered, near, or far.

Third, crop margin is wasted. A generous crop margin is reserved to handle worst-case shake, but such large shake rarely occurs in practice. As a result, FOV is sacrificed unnecessarily most of the time.

Where AI Steps In

Deep learning-based approaches tackle these limitations in a different way. There are three main directions.

1. Learned motion estimation

Instead of or alongside gyro data, pixel-level motion between frames (optical flow) is estimated using learned models. This provides a motion field that takes into account subject depth and scene structure. Google Pixel's video stabilization is said to have evolved in this direction.

2. Learned frame synthesis

Areas lost to cropping are filled in using information from surrounding frames, or heavily shaken frames are reconstructed from multiple frames. This can reduce crop margins or even retain most of the original FOV. Research tracks like "Motion-Aware Video Frame Interpolation" and "Neural Video Stabilization" fall in this direction.

3. End-to-end learning

A neural network learns directly from input video to stabilized output video, without a traditional pipeline. While many good results have been reported in research, commercial deployment remains limited due to challenges in real-time mobile processing and output consistency.

The Reality of AI EIS on the Ground

Reading academic papers and tech media, AI-based EIS looks like it has already solved everything. The reality in module development is a bit different.

Power and thermals. Running neural inference in real time on mobile devices requires NPUs or GPUs. Keeping them under high load throughout video capture causes serious heat and battery drain. The typical compromise is "high-quality AI EIS only in specific modes."

It does not fully replace the gyro. AI-based motion estimation works well when the scene has many feature points and lighting is sufficient, but performance degrades sharply in low light or against monochromatic backgrounds. Realistic products are moving toward a hybrid of gyro and AI: the gyro guarantees a baseline of stability, while AI adds scene awareness to push quality higher.

Training data bias. Neural network performance depends heavily on training data. It performs well in certain scenes (bright outdoors, urban environments), but in environments not represented in training data (underwater, extremely low light, industrial settings), unpredictable artifacts can emerge. From a camera module supplier's perspective, this translates into increased validation burden.

In summary, AI-based EIS is a clear direction, but rather than "gyro-based EIS fully exiting the stage," the more accurate picture from the field is that "a gyro + AI collaborative structure is becoming the standard."

Sensor Technology Evolution — Hardware for EIS

It is easy to focus solely on software algorithms, but much of EIS's progress owes itself to advances in sensor hardware. Here are a few changes I see on the ground.

Higher-Precision, Smaller Gyro Sensors

The performance of 6-axis IMUs built into smartphones has visibly improved in recent years. Sampling rates now exceed 1–2 kHz, and noise density and drift characteristics continue to improve. This directly lifts the quality of data that EIS depends on.

Perhaps more importantly, hardware synchronization between image sensor and gyro is becoming standard. The readout time synchronization issue discussed in the third article directly affects gyro timestamp accuracy, and recent platforms solve this at the hardware level.

Stacked CMOS and Fast Readout

Stacked CMOS technology, pioneered by Sony, separates the pixel layer and the circuit layer and stacks them. This allows more circuit area without increasing the overall sensor size, enabling significantly faster readout.

Faster readout directly reduces Rolling Shutter distortion. Rolling Shutter distortion is fundamentally proportional to "the time difference between the top and bottom rows of a frame," so reducing that time difference reduces the distortion, even with a Rolling Shutter architecture.

Going further, Stacked sensors with integrated DRAM use high-speed buffering to approximate Global Shutter behavior. They are not true Global Shutter, but they offer a realistic way to significantly mitigate Rolling Shutter's drawbacks.

Global Shutter on Mobile?

In theory, Global Shutter sensors would eliminate Rolling Shutter problems at the root. So will mobile eventually transition to Global Shutter?

From an image-capture camera perspective, not anytime soon. As discussed in the third article, the triple wall of pixel size, low-light performance, and cost remains high. This applies not only to the main camera but equally to the front camera. The primary use cases for front cameras (selfies, video calls) are more often in low-light conditions, and there is little high-speed distortion to solve, so there is weak motivation to switch to Global Shutter. As a result, cameras that users "shoot" with — whether main or front — are likely to continue using Rolling Shutter + RSC.

On the other hand, sensing-oriented cameras follow a different path. Receivers in ToF and structured light systems, tracking cameras in AR/VR headsets, barcode scanners in industrial devices — in the domain of sensors that the system uses to perceive the world, rather than image-makers for the user, Global Shutter is already close to standard. In this domain, low latency and distortion-free instantaneous capture matter more than low-light performance.

In other words, "Global Shutter is entering mobile" is accurate, but the path is sensing, not imaging. In the territory that EIS addresses — user-facing video — Global Shutter is unlikely to become mainstream in the near future.

Gyro-in-Sensor

There is movement toward integrating a gyro sensor within the image sensor package itself. This minimizes the physical distance between image sensor and gyro, allowing vibration phases to match more precisely and hardware synchronization to be cleaner.

From a module developer's perspective, this is welcome — the BOM simplifies, and the sensor vendor takes responsibility for gyro characteristics as well. The trade-off is reduced flexibility in sensor selection.

Summary: Hardware Sets the Ceiling for Algorithms

No matter how refined an EIS algorithm is, there is a clear ceiling if input data quality is poor. Conversely, when sensor and gyro quality improve, the same algorithm yields visibly better results.

From a software developer's standpoint, it's easy to say "the algorithm determines quality." But working in module development, you feel every day that hardware sets the ceiling for algorithms. Even in the age of AI-based EIS, this relationship will not change.

Software-Defined Camera — Boundaries Disappear

What is a Software-Defined Camera?

Software-Defined Camera (SDC) is not a term with a rigid definition, but the concept circulating in the industry can be summarized as:

"A structure in which a camera's final output quality is determined primarily by software processing (especially AI/ML), rather than by hardware such as sensors and lenses."

This concept is not new. Features like Google Pixel's HDR+, Night Sight, and Magic Eraser were early forms of SDC. The reason Pixel — whose hardware specs appeared unremarkable on paper — maintained top-tier photo quality was entirely due to software.

In the context of EIS, what SDC means is that "stabilization is no longer something software does after the sensor captures the frame; the entire capture pipeline is designed with stabilization as a built-in premise."

A Pipeline Without Boundaries

The traditional camera pipeline was divided into these stages:

Sensor readout → RAW → ISP (demosaic, noise reduction, tone mapping) → EIS/RSC → Encoder

Each stage received the output of the previous one and processed it independently, in a block structure. Recent pipelines, however, are rapidly blurring these boundaries.

Multi-frame Fusion. Instead of a single frame, multiple frames are taken in simultaneously, and noise reduction, HDR, super-resolution, and stabilization are all performed together. Here, EIS is no longer an independent stage — it becomes part of the multi-frame alignment process.

Neural ISP. There is a movement toward replacing or augmenting traditional ISP blocks with neural networks. Demosaicing, noise reduction, and detail recovery are being unified into a single network. If this trend expands to include EIS as well, an end-to-end network of "RAW input → stabilized and corrected output" becomes a real possibility.

Computational Viewfinder. The viewfinder image the user sees is already a computed result. It used to be "preview is light, high-end processing applies only to the final capture." Now, the preview is processed close to the final result. EIS has become a feature that must operate from the preview stage.

A Module Developer's Dilemma

The shift toward SDC places camera module developers in a peculiar position.

On one hand, because software compensates for hardware weaknesses, there is more headroom in module design. Optical defects that would have been classified as defects in the past can now be saved by software correction.

On the other hand, optimizing only for the module's "optical performance" is no longer enough. What matters is how well the module meshes with the entire software pipeline. Even with the same image sensor, the final result changes completely depending on ISP tuning, gyro synchronization accuracy, and calibration data quality.

As a result, the value chain of the camera module industry is shifting from "pure hardware supply" toward the integrated provision of "hardware + calibration data + drivers + ISP tuning." This is precisely the change I have been feeling firsthand since starting this series.

Where EIS Is Heading — Closing the Series

Closing this four-part EIS series, I want to organize a few thoughts that run through the whole. What started as a simple story about shake correction turned out to be a story that expanded to cover the entire camera system.

First, EIS is not just a "shake-correction feature." It is an integrated problem that spans the full camera system — gyro, sensor, ISP, lens correction, encoder. That is why this series covered OIS/HIS, the pipeline, Rolling Shutter, and AI.

Second, the evolution of EIS is the interaction of hardware and software. A good algorithm reaches its true potential only on a good sensor, and a good sensor only becomes meaningful when paired with well-tuned algorithms. Discussing EIS while looking at only one side is half an understanding.

Third, the role of the camera module developer is changing. In the past, the core was "supplying optical modules that meet specs." Going forward, competitiveness lies in "providing systems that integrate into a software pipeline." I write this series from the middle of that shift.

There are many topics on EIS that this series did not cover — actual tuning workflows, EIS in automotive cameras, the very different requirements of VR/AR, and more. Future articles may pick these up as the opportunity arises.

The next series will address another pillar of camera module development: ISP tuning. If EIS was "the domain of motion," ISP is "the domain of light and color."

This article is based on practical experience in camera module development. Statements about the future reflect a reading of publicly known technology trends and changes observed on the ground, and are not predictions or descriptions of any specific product roadmap. Comments, corrections, and opinions are always welcome.

Rolling Shutter and EIS — Why Image Stabilization Can Make Your Video Worse

Ross Kim — Fri, 17 Apr 2026 11:47:23 +0000

Introduction

In the previous article, we looked at the internal structure of the EIS pipeline through Gyroflow: gyro data acquisition → integration → smoothing → frame warping. But when you actually implement this pipeline, you hit a problem that's easy to overlook.

"I applied EIS correction, but the video looks even more wobbly — like jello."

The culprit is Rolling Shutter. Rolling Shutter is a physical characteristic of the sensor, unrelated to EIS itself. But when EIS transforms frames without accounting for it, the distortion gets amplified. This article covers what Rolling Shutter is, how it conflicts with EIS, and why a separate correction step is necessary.

What Is Rolling Shutter?

Global Shutter vs Rolling Shutter

Image sensors capture a frame in one of two fundamental ways.

Global Shutter exposes all pixels simultaneously — every pixel starts and stops exposure at the same instant. Every pixel in a single frame captures the same moment in time. This method is used in CCD sensors and some industrial CMOS sensors.

Rolling Shutter exposes each row sequentially. The first row begins exposure, then the second row starts slightly later, then the third, and so on. By the time the full frame is read out, there's a time difference of several to tens of milliseconds between the top row and the bottom row.

Nearly all CMOS sensors in smartphones and consumer cameras today use Rolling Shutter. The reason is straightforward — lower manufacturing cost, simpler circuit design, and easier high-resolution implementation.

Distortions Caused by Rolling Shutter

When the camera moves during capture with a Rolling Shutter sensor, each row captures the scene at a slightly different point in time. This produces three characteristic distortions.

1. Skew

When the camera pans horizontally at speed, vertical lines tilt diagonally. The top rows capture the scene at one horizontal position, while the bottom rows capture it after the camera has shifted. Buildings and poles appearing to lean is a classic example.

2. Jello/Wobble

When the camera vibrates during video recording, each row captures a slightly different position. The result is a jello-like wobbling distortion across the entire frame. This is commonly seen in dashcam footage or video shot while running.

3. Partial Exposure

Very brief light sources like a flash or lightning illuminate only certain rows, leaving the rest dark. This shows up as a bright band across part of the frame in still photos.

For video recording and EIS, the most problematic distortions are skew and jello — especially the jello effect.

What Happens When EIS Meets Rolling Shutter

The Core Assumption of EIS: "All Pixels in a Frame Were Captured at the Same Instant"

Recall the EIS pipeline from the previous article. EIS estimates camera rotation from gyro data and warps the entire frame in the opposite direction to cancel out that rotation. There's a critical assumption embedded in this process.

"One frame represents one moment in time."

In other words, EIS applies the same correction value (rotation matrix) to every pixel in the frame. With a Global Shutter sensor, this assumption holds perfectly. With a Rolling Shutter sensor, it doesn't.

What Goes Wrong

Consider a concrete scenario.

Assume the camera is vibrating clockwise. EIS reads the gyro data and determines: "This frame was captured with a 2-degree clockwise rotation, so I'll apply a 2-degree counter-clockwise correction."

But due to Rolling Shutter, the actual situation is different:

Top rows: captured at 1 degree of clockwise rotation
Middle rows: captured at 2 degrees of clockwise rotation
Bottom rows: captured at 3 degrees of clockwise rotation

When EIS uniformly applies "2 degrees counter-clockwise" to the entire frame:

Top: 1° rotation + (-2° correction) = -1° overcorrected
Middle: 2° rotation + (-2° correction) = 0° correct
Bottom: 3° rotation + (-2° correction) = +1° undercorrected

The result is that the top and bottom of the frame shift in opposite directions, creating a new jello effect that wasn't there before, or amplifying an existing one.

This is the real explanation behind "I turned on EIS and the video looks even shakier."

What Is Rolling Shutter Correction (RSC)?

Why Row-Level Correction Is Needed

To solve this problem, the EIS warping stage must not apply a single correction value to the entire frame. Instead, each row (or group of rows) needs its own correction value based on the exact moment that row was captured.

This is Rolling Shutter Correction (RSC).

How RSC Works

The key to RSC is leveraging the temporal resolution of gyro data. As mentioned in the EIS pipeline article, gyro data is typically sampled at 1kHz or higher. At 30fps, one frame spans about 33ms. With a 1kHz gyro, that's 33 gyro samples within a single frame.

If the Rolling Shutter readout time (the total time to sequentially read all rows) is, say, 20ms, then the exact capture time of each row can be calculated, and the camera orientation at that time can be estimated from the corresponding gyro data.

In summary:

Calculate the capture time of each row within the frame (from readout time and row number)
Interpolate gyro data at that time to determine camera orientation
Apply a different correction transform to each row

Effectively, warping is performed on a "per-row" basis rather than a "per-frame" basis.

How Do You Know the Readout Time?

One of the most critical parameters for RSC is the sensor's readout time. This value varies depending on the sensor's row count, clock speed, and resolution mode.

In commercial products, readout time is either specified in the sensor datasheet or passed as part of the frame metadata through the ISP driver. Internally at manufacturers, this value is often hardcoded into the ISP pipeline.

In the open-source Gyroflow, the value can be entered manually by the user, automatically extracted from video metadata, or estimated through a calibration video. Since cameras and sensors vary enormously unlike commercial products, this process is essential.

RSC Implementation in Commercial Products

Smartphones

In smartphone ISPs, EIS and RSC typically operate as a single integrated pipeline. The ISP receives readout time information from the sensor, synchronizes it with gyro data timestamps, and performs row-level correction. All of this runs in real time within the ISP hardware accelerator.

Major mobile ISPs including Qualcomm Spectra, Samsung SLSI, and MediaTek Imagiq all have built-in RSC capabilities. However, RSC quality varies between ISP vendors and depends on the tuning state.

Action Cameras / Drones

Products like GoPro and DJI handle EIS+RSC through proprietary ISPs or dedicated processors. Action cameras face particularly harsh vibration environments, making RSC critically important. GoPro's HyperSmooth and DJI's RockSteady are both pipelines that include RSC.

Post-Processing

Post-processing software like Gyroflow and Adobe Premiere's Warp Stabilizer also support RSC. The advantage of post-processing is unlimited look-ahead since the entire video can be analyzed, but the challenge lies in knowing the readout time accurately.

Practical Issues with Rolling Shutter and EIS

Readout Time Synchronization

RSC precision is directly determined by the synchronization accuracy between gyro data timestamps and sensor readout timing. Even a few milliseconds of sync error can cause corrections to misalign, leaving residual jello effects or creating new distortions.

In commercial products, the gyro sensor and image sensor share a hardware timer, or the ISP aligns gyro data using frame sync signals as a reference. In cases like Gyroflow where external gyro logs are used, this synchronization must be estimated in software, which can reduce precision.

Resolution and Readout Time

Even with the same sensor, readout time changes depending on the resolution mode (4K vs 1080p), binning settings, and frame rate. In 4K mode, more rows need to be read, so readout time increases and Rolling Shutter distortion grows. Conversely, binning reduces the effective row count and shortens readout time.

In practice, these values are managed per sensor mode and must be accurately passed to the ISP. If the value is wrong, RSC applies corrections at the wrong timing, resulting in distortion that's subtle but clearly visible.

Summary

Rolling Shutter is a physical characteristic arising from the sequential row-by-row exposure of the sensor. On its own, it causes skew and jello effects, but the problem becomes more severe when combined with EIS. When EIS assumes "the entire frame was captured at the same instant" and applies uniform correction, the per-row timing differences from Rolling Shutter get amplified into visible distortion.

RSC (Rolling Shutter Correction) solves this by leveraging the high temporal resolution of gyro data to apply individual corrections matched to each row's capture time. Today's major mobile ISPs process EIS and RSC as an integrated pipeline, and post-processing software like Gyroflow also supports RSC.

In the next article — the final installment of the EIS series — I plan to cover EIS from a camera module developer's perspective: its present and future, including AI-based EIS, sensor technology evolution, and the direction of software-defined cameras.

This article was written based on hands-on experience in camera module development. It describes publicly available technical principles and general industry practices, not the internal implementation of any specific product. If you find any errors or have feedback, please leave a comment.

Understanding the EIS Pipeline Through Gyroflow — A Look Inside Image Stabilization via Open Source

Ross Kim — Mon, 13 Apr 2026 12:16:51 +0000

Introduction

In the previous article, I covered the differences between OIS, EIS, and HIS. For EIS, I explained it roughly as "correcting frames based on gyro data," but what actually happens inside is more complex than that suggests.

EIS is not simply "cropping a shaky video to stabilize it." It is a pipeline that reads data from gyro sensors, estimates camera motion, decides which movements to keep and which to remove, and finally transforms each frame.

In this article, I will reference an open-source project called Gyroflow to examine what each stage of the EIS pipeline actually does. While commercial EIS implementations are proprietary, Gyroflow implements the same principles as open source, making it an excellent learning resource.

What Is Gyroflow

Gyroflow is an open-source video stabilization software that uses gyroscope data. It was originally built for post-processing stabilization of action camera and drone footage.

Here are its key features:

Gyro data-based: Estimates motion vectors from gyro sensor data rather than image analysis (optical flow)
Wide camera support: Can directly read gyro data from major action cameras including GoPro, DJI, Sony, and Insta360
Lens profiles: Includes built-in distortion models for various lenses, including fisheye
GPU acceleration: Supports near-real-time processing speeds via OpenCL/wgpu
Rust-based: Core logic is written in Rust, making the code relatively readable

The source code is available on GitHub, and the community is active. Beyond practical use, it holds significant value as a reference for understanding how EIS works internally.

EIS Pipeline Overview

The EIS pipeline can be broken down into five stages. The diagram below shows the overall flow:

Let's walk through each stage.

Stage 1: Gyro Data Acquisition

A gyroscope sensor measures the camera's angular velocity. In simple terms, it records in which direction and how fast the camera is rotating across three axes (pitch, yaw, roll).

This data is typically recorded alongside video frames at the camera firmware level. Action cameras like GoPro embed gyro data in the video file's metadata. Gyroflow parses this metadata to extract the gyro data.

There are some critical points here:

Timestamp Synchronization

The timestamps of gyro data and video frames must match precisely. Since the gyro sensor's sampling rate (typically 200Hz–8kHz) and the video frame rate (30fps, 60fps, etc.) differ, time synchronization is essential to correctly match gyro data to specific frames.

To illustrate how sensitive this synchronization is: an offset of just 10ms can cause the correction direction to reverse relative to the actual shake, effectively doubling the shake amplitude. At 30fps, one frame is approximately 33ms, so 10ms is only one-third of a frame — yet it is fatal to the result. Gyroflow allows adjusting this offset either automatically or manually, while commercial products typically handle synchronization at the hardware level.

Gyro Data Noise

Gyro sensor data always contains noise. There is drift — a slow accumulation of error — as well as high-frequency noise. Using this noisy data directly results in jittery or unstable corrections. Therefore, preprocessing of the gyro data itself (such as a low-pass filter) is necessary.

Stage 2: Motion Estimation — Camera Orientation Calculation

Integrating gyro data (angular velocity) over time yields the camera's orientation. The direction the camera was pointing at each frame is expressed as a quaternion or rotation matrix.

Angular velocity data → Time integration → Per-frame camera orientation (quaternion)

Several practical issues arise in this process:

Integration Drift

Integrating angular velocity accumulates error. This is gyro drift. It is negligible over short intervals, but over tens of seconds to several minutes of footage, it builds up to significant levels. Gyroflow offers an option to use optical flow-based auxiliary estimation alongside gyro data to correct this drift.

Coordinate System Transformation

The gyro sensor's coordinate system does not always match the camera (image) coordinate system. The axis mapping varies depending on how the sensor is mounted on the circuit board. This is why Gyroflow has an IMU orientation setting. In commercial products, this mapping is fixed in firmware.

Stage 3: Smoothing — Which Movements to Keep, Which to Remove

This stage is the core of EIS and has the greatest impact on output quality.

Camera movements fall into two categories:

Intended movements: Smooth camera motion the operator deliberately performs, such as panning and tilting
Unintended movements: Unwanted shake from hand tremor, walking vibration, etc.

The goal of EIS is to remove the latter while preserving the former. The challenge is that perfectly distinguishing between the two automatically is impossible.

Limitations of a Simple Low-Pass Filter

The most intuitive approach is to apply a low-pass filter (LPF) to the camera orientation data — remove high-frequency shake while passing through low-frequency motion.

However, this method has a fundamental problem:

Set the cutoff frequency low → Hand shake is well suppressed, but fast panning introduces lag (the video feels like it is trailing behind)
Set the cutoff frequency high → Panning response is good, but hand shake remains

Ultimately, a fixed LPF cannot simultaneously achieve "stable yet responsive" results.

Gyroflow's Smoothing Algorithms

Gyroflow provides several smoothing algorithms, notably:

Plain 3D: Gaussian smoothing in quaternion space. Simple but produces basic results.
Horizon lock: A special mode that locks the horizon. Frequently used for drone footage.
Fixed camera: Removes all movement as if the camera were fixed. Tripod effect.
Velocity dampened: Dynamically adjusts smoothing intensity based on camera movement speed. Strong smoothing when stationary, weak smoothing during panning.

Among these, velocity dampened is the most practical. It takes an adaptive approach to overcome the limitations of a simple LPF: "reduce smoothing when the camera is moving fast, strengthen smoothing when stationary."

Commercial EIS products also use similar adaptive smoothing in principle, but the specific implementations vary by manufacturer and are not publicly disclosed.

Stage 4: Frame Warping

Once a "corrected camera orientation" has been obtained through smoothing, the actual video frames need to be transformed.

The frame is rotated/transformed by the difference between the original camera orientation and the corrected camera orientation. This process is called frame warping.

How Warping Works

Simplified, the flow looks like this:

Original frame's camera orientation: Q_original
Smoothed target orientation: Q_smoothed
Correction rotation: Q_correction = Q_smoothed × Q_original⁻¹
Apply this correction rotation to the image

Multiplying by Q_original⁻¹ (the inverse of the quaternion) means "reversing the original frame's rotation." It is an operation that cancels out the original shaky orientation and moves the frame to the desired smooth orientation.

In practice, lens distortion correction is also performed alongside the rotation. If the image is rotated while wide-angle fisheye distortion remains, the edges exhibit a warping artifact. This is why Gyroflow requires a lens profile — the distortion model must be known for accurate warping.

FOV Loss

Rotating/transforming a frame creates empty areas at the edges. These empty areas must be hidden by cropping the video, which reduces the field of view (FOV).

This is the most fundamental trade-off in EIS:

Stronger correction → more cropping → greater FOV loss
Preserving FOV → correction intensity must be reduced → shake remains

Gyroflow allows users to directly adjust this crop ratio and offers a Dynamic Zoom feature that automatically calculates the optimal crop per frame. Commercial products typically use a fixed crop (around 10–15%) or resolution oversampling (4K capture → 1080p stabilized output) to make FOV loss imperceptible.

Stage 5: Output

Encoding the warped frames produces the final stabilized video. Since Gyroflow is a post-processing tool, this occurs as a separate rendering step, whereas commercial real-time EIS executes this entire pipeline simultaneously with each frame capture.

Real-time processing introduces additional constraints:

Look-ahead limitation: Future frames available for smoothing are limited
Computational resources: Processing must complete within GPU/DSP resource budgets
Thermal/power: Must operate within the mobile device's thermal and power budget

Because of these constraints, even when using the same principles, there is a quality gap between post-processing (Gyroflow) and real-time (commercial EIS) output.

Differences Between Gyroflow and Commercial EIS

While Gyroflow helps us understand EIS principles, there are several important differences from commercial EIS:

Aspect	Gyroflow (Post-processing)	Commercial EIS (Real-time)
Processing timing	After recording	Simultaneous with recording
Smoothing look-ahead	Entire video available	A few to tens of frames
Lens distortion correction	Separate lens profile	Built into ISP
Gyro synchronization	Manual/auto estimation	Hardware synchronization
Computational resources	PC GPU (unlimited)	Mobile DSP (limited)
FOV handling	Dynamic crop available	Typically fixed crop

In commercial products, this entire pipeline runs with hardware acceleration inside the ISP (Image Signal Processor), often operating in hybrid mode coordinated with OIS.

Summary

On the surface, EIS can be summarized as "cropping to suppress shake," but internally, a complex pipeline exists:

Gyro sensor data acquisition and time synchronization
Camera orientation estimation through angular velocity integration
Smoothing to separate intended movements from shake
Frame warping that accounts for lens distortion
Managing the trade-off between FOV loss and correction intensity

Gyroflow implements this pipeline transparently as open source, making it an excellent reference for understanding how EIS works. For those interested in reading the actual code, I recommend checking out the GitHub repository.

In the next article, I plan to cover the relationship between Rolling Shutter and EIS — why rolling shutter correction is additionally required in EIS.

This article was written based on hands-on experience in camera module development. The descriptions of Gyroflow reference its publicly available source code and documentation, and do not describe the internal implementation of any specific product. If you find any errors or have feedback, please leave a comment.

OIS vs EIS vs HIS — A Camera Module Developer's Guide to Image Stabilization

Ross Kim — Fri, 10 Apr 2026 06:48:03 +0000

Why Image Stabilization Matters

Smartphone camera megapixel counts keep climbing year after year, but what truly determines video quality lies elsewhere: Image Stabilization.

In video recording, frame-to-frame shake accumulates, making footage uncomfortable to watch no matter how high the resolution. For photos, faster shutter speeds can mitigate the effects of hand shake in bright conditions, but in low light, slower shutter speeds make motion blur an unavoidable problem.

There are three main approaches to solving this: OIS (Optical), EIS (Electronic), and HIS (Hybrid), which combines the two. Let's break down the principles, pros and cons, and real-world differences in how each actually works.

OIS — Optical Image Stabilization

How It Works

OIS physically moves the lens or image sensor to counteract hand shake. When the gyro sensor inside the camera module detects movement, an actuator shifts the lens assembly or sensor in the opposite direction to correct the optical axis.

Implementation Methods

OIS is broadly divided into two approaches:

Lens-shift: Moves a portion of the lens group along the X and Y axes. This is the traditional approach with a long history in the camera industry.
Sensor-shift: Moves the image sensor itself. Apple introduced this with the iPhone 12 Pro Max and has since expanded it across the entire iPhone lineup. Samsung also adopted sensor-shift OIS with the Galaxy S26 Ultra. Chinese manufacturers like Huawei (Pura 70 Ultra, Pura 80 Ultra) and OPPO have also implemented it in their flagships. Since the sensor is lighter than the lens, it offers faster response times and a wider correction range, and adoption is expanding across the flagship market.

Actuator types include VCM (Voice Coil Motor), SMA (Shape Memory Alloy), and ball-guide mechanisms, selected based on tradeoffs between module size, power consumption, and correction range.

Pros

Optical correction means no image quality loss
Stabilizes the light reaching the sensor, making it strong in low-light conditions
Effective for both still photos and video

Cons

Additional components (actuators, gyros, driver ICs) increase module cost and size
Correction range has physical limits (typically around ±1 degree)
Durability concerns from drops and impacts (OIS lock mechanisms required)
Additional power consumption

EIS — Electronic Image Stabilization

How It Works

EIS corrects hand shake through software rather than hardware. It calculates inter-frame shake based on gyro sensor data and applies cropping and warping (geometrically transforming and aligning the image) to each frame to produce stabilized video output.

In simple terms, the actual output uses a narrower area than the full sensor capture, and the position of this output region is adjusted frame by frame according to the detected shake.

Pipeline

A typical EIS processing pipeline works as follows:

Gyro data acquisition — Angular velocity data is read from the IMU (Inertial Measurement Unit)
Motion estimation — Gyro data is combined with (in some cases) image-based motion vectors to estimate camera movement
Target path generation — Intentional panning is separated from unintentional shake. A smoothing filter creates a stabilized target path
Frame transformation — Affine (transformation preserving parallelism, including rotation, scaling, and translation) or homography (transformation that also accounts for perspective) transforms are applied to each frame for correction
Crop and output — The final frame is cropped to remove black borders caused by the transformation

Pros

No additional hardware required, favorable for cost reduction
No impact on module size
Algorithm improvements possible through software updates
No physical correction range limits, enabling compensation for large movements (practically effective up to around ±3 degrees)

Cons

FOV (field of view) loss due to cropping (typically 10–20%)
Resolution degradation from the crop + warp process
Difficult to apply to still photos (requires inter-frame comparison)
Weak at correcting high-frequency vibrations
Cannot reduce motion blur itself in low light (since it's not optical correction)
Combined with rolling shutter distortion, correction results can appear unnatural

HIS — Hybrid Image Stabilization

Concept

HIS combines OIS and EIS. The term "HIS" itself is more of a marketing and technical convenience label used by manufacturers rather than an industry standard, but the concept of using OIS and EIS together is adopted by most flagship smartphones today.

Actual Operating Structure: OIS First-Pass Correction → EIS Post-Processing

It's easy to imagine HIS as OIS and EIS equally sharing the workload, but the actual implementation is closer to a serial pipeline.

OIS corrects first, physically — When the gyro sensor detects shake, the actuator moves the lens or sensor to optically counteract the movement.
EIS reads the OIS correction results and post-processes — Residual micro-shake, rolling shutter distortion, and motion blur remaining after OIS correction are refined through software to produce smooth final video.

The key point is that EIS knows what OIS corrected and by how much. By passing OIS actuator motion data to the EIS pipeline, over-compensation is avoided and the strengths of each system can be leveraged.

The best example of this structure is Google Pixel 2's (2017) Fused Video Stabilization. Google extracts gyro signals and OIS motion data together to precisely estimate camera movement, then synthesizes frames through a 3-stage pipeline with machine learning-based motion filtering. Sony Xperia's Optical SteadyShot also officially employs a structure that drives OIS and EIS simultaneously.

Angle-Based Role Division

As an approach to OIS-EIS collaboration, dividing roles based on correction angle is also discussed.

OIS's physical correction limit is approximately ±1 degree
EIS can provide practical correction up to approximately ±3 degrees

Using this difference, shake within ±1 degree is handled optically by OIS without quality loss, while larger movements exceeding ±1 degree are handled by EIS through software. This minimizes EIS crop amount while securing a wide correction range.

However, the extent to which this approach is implemented in actual commercial products varies by manufacturer, and most do not disclose their detailed internal structures.

Differences by Camera and Mode

One important point to note is that even on the same smartphone, the stabilization method changes depending on the camera and shooting mode.

Still photos → OIS only (EIS cannot be applied as it requires inter-frame comparison)
Video recording (main camera) → OIS + EIS combination often operates
Ultra-wide camera → EIS only (no OIS hardware)
Telephoto camera → Primarily OIS

Even modes marketed as providing powerful video stabilization, such as Samsung's "Super Steady" and Apple's "Action Mode," actually work by switching to the ultra-wide camera and maximizing EIS crop. Rather than sophisticated OIS+EIS collaboration, they are closer to modes that aggressively apply EIS.

Pros

Adding EIS software post-processing on top of OIS optical correction produces smoother video than either alone
Since OIS handles first-pass correction, EIS crop amount can be reduced
Most flagship smartphones adopt this combination, making it effectively the standard configuration for video stabilization

Cons

Synchronization between OIS motion data and the EIS pipeline is challenging to implement
OIS hardware costs are still included
Behavior varies by camera and mode, making it difficult for users to experience consistency
Implementation levels vary significantly between manufacturers, and most do not disclose their detailed structures

Comparison Summary

Attribute	OIS	EIS	HIS
Correction Method	Hardware (Physical)	Software (Digital)	Hardware + Software
Image Quality Impact	No loss	FOV loss, resolution degradation	OIS-level maintained + minimized EIS crop
Low-Light Performance	Excellent	Limited	Excellent
Still Photos	Effective	Ineffective	Same as OIS
Video Stabilization	Effective (range limited)	Effective (quality tradeoff)	Most effective
Module Cost	High	No additional cost	High
Module Size	Increases	No impact	Increases
Implementation Difficulty	Medium	Medium	High
Correction Range	±1° (physical limit)	~±3°	OIS ±1° + extended by EIS

Future Trends

The Rise of AI-Based EIS

While traditional EIS has been based on gyro data and conventional computer vision, deep learning-based motion estimation and frame synthesis technologies have been advancing rapidly in recent years. Google's Pixel series leads this field, evolving toward reducing crop loss by combining inter-frame interpolation with super-resolution.

Expansion of Sensor-Shift OIS

Sensor-shift was pioneered by Apple, followed by Chinese manufacturers like Huawei and OPPO, and Samsung finally adopted it with the Galaxy S26 Ultra. As major flagship manufacturers adopt it one after another, sensor-shift is becoming the new standard in the flagship market. Sensor-shift increases lens design flexibility, creating strong synergy with lens module miniaturization.

Software-Defined Camera

In the long term, the weight of hardware correction will likely decrease, with software correction combining sensor data and AI processing becoming more important. However, the limits of physics — particularly insufficient light in low-light conditions — cannot be fully overcome by software alone, so OIS is unlikely to disappear entirely.

Conclusion

Image stabilization is not a simple feature but a system design challenge where hardware, software, physics, and cost are complexly intertwined. There is no single "best" answer among OIS, EIS, and HIS. The optimal combination varies depending on product positioning, target price, and primary shooting scenarios.

In the next article, I plan to dive deeper into the internal structure of the EIS pipeline, covering gyro data processing, frame transformation, and smoothing algorithms.

This article was written based on hands-on experience in camera module development. What type of image stabilization does your smartphone use? If you find any errors in this article, please point them out in the comments. Feedback and opinions are always welcome.

What It's Like Being a Software Engineer at a Hardware Company — The Reality of SW Development Dragged Along by HW Schedules

Ross Kim — Tue, 07 Apr 2026 14:58:33 +0000

I used to work in software development in the mobile industry, and now I develop software at a camera module company. We make camera modules for smartphones and industrial devices. The core of the product is hardware — lenses, sensors, mechanical design — and software's role is to make that hardware work or to develop test programs.
Working as a software engineer in this kind of environment, you run into some unique situations. Things that would be hard to imagine at a pure software company (though, to be fair, I've never actually worked at one, so I can't say for sure). Here are a few experiences I've put together.
"We Can't Change the Hardware, So Fix It in Software"
This is the phrase you hear the most as a software engineer at a hardware company.
Once, the hardware team physically mounted a display in the wrong orientation. With assembly already complete, tearing it off and reattaching it didn't make sense in terms of cost or time. So the request that came to software was simple: "Just flip the display output direction in software."
This wasn't a one-time incident. When problems come up in hardware design, the cost of fixing them is high, so it's an everyday occurrence for the fix to land on software's plate. From software's perspective, work that was never part of the plan suddenly appears. And yet, the deadline doesn't change.
An Organization That Treats Software Like Hardware
The opposite pattern exists too.
Sometimes a fix that would clearly take just a few lines of code gets met with: "That's already been finalized, so changes aren't possible." Hardware-centric organizations tend to be very conservative about changes after design sign-off. Recutting molds or redesigning boards costs real money, so the mindset itself makes sense.
The problem is when that same standard gets applied to software. Software's essential strength is its flexibility — but when you lock it into the same process as hardware, inefficiency follows. Instead of a straightforward fix, you end up adding workaround logic or tacking on extra features. The code gets more complex with every iteration.
When Hardware Is Delayed, It's the Software Schedule That Shrinks
Product development has deadlines. Delivery dates promised to clients don't move easily.
The development sequence is usually hardware first, software second. You need the sensor and mechanical parts ready before you can load firmware, and tuning and validation have to happen on actual hardware — so this order makes sense.
But when the hardware phase gets delayed due to design changes or component issues, the overall deadline stays the same while the time given to software shrinks. Once the hardware team finishes their part, their stance becomes: "We've handed it off, so figure out the rest within the remaining time." (Or, as mentioned above, it becomes: "Just find a way to fix it in software.")
A software development period that was supposed to be a month can turn into two weeks. But the amount of work doesn't shrink. If this happened once or twice, you could push through with overtime. But when it repeats on every project, it just starts to feel normal.
So Is This Environment All Bad?
I'm not trying to just list complaints.
Working in this kind of environment, you naturally become a software engineer who understands hardware. You learn to read circuit diagrams and develop an instinct for designing software on top of physical constraints. I think that's an experience that's hard to gain in a pure software environment.
And when you keep responding to requests like "solve this hardware problem with software," the range of your problem-solving ability grows. Implementing defined requirements and creatively working around physical constraints are two different kinds of skills.
Being a software engineer at a hardware company is uncomfortable, but it's also a uniquely rewarding environment for growth.

I develop software at a camera module company. I write about real experiences from the field.