If you have ever taken a photo of a spinning propeller or a fast-moving train out of a car window using your smartphone, you have likely witnessed the “Rolling Shutter” effect. Propellers look like warped boomerangs; vertical poles look slanted; cars look like they are leaning forward.

This phenomenon, often called the “Jello effect,” occurs because most modern consumer cameras (CMOS sensors) do not capture the entire image at the exact same instant. Instead, they scan the scene line by line, usually from top to bottom. If the camera or the object moves during that scanning process, the geometry of the image breaks.

For photographers, this is an annoyance. For computer vision engineers and students trying to perform 3D reconstruction or Structure-from-Motion (SfM), it is a mathematical nightmare. The standard laws of geometry used for “Global Shutter” cameras (which capture everything at once) simply do not apply.

In this post, we will deep-dive into the research paper “Order-One Rolling Shutter Cameras,” which proposes a groundbreaking unified theory to make sense of these distortions. The authors identify a specific, highly practical class of Rolling Shutter (RS) cameras—called Order-One (\(RS_1\))—that allows for elegant mathematical solutions to problems that were previously incredibly difficult to solve.

The Problem: When Geometry Drifts

To understand the contribution of this paper, we first need to look at the difference between how we usually model cameras and how RS cameras actually behave.

In a classic Perspective Camera (Global Shutter), every pixel in the image is exposed at time \(t\). All rays of light pass through a single optical center \(C\) and hit the image plane. This projects a point in 3D space to a unique point on the 2D image.

In a Rolling Shutter Camera, every row of pixels is exposed at a slightly different time.

  • Row 1 is exposed at \(t_1\).
  • Row 2 is exposed at \(t_2\).
  • …
  • Row \(N\) is exposed at \(t_N\).

If the camera is moving during this time, the “optical center” \(C\) is effectively moving while the photo is being taken.

Fig 1a shows points seen multiple times by general RS cameras. Fig 1b shows Order-One cameras seeing points exactly once.

As shown in Figure 1(a) above, a general RS camera is complex. Because the camera moves, a single point in 3D space could theoretically be “seen” by the camera multiple times. Imagine the camera sweeps past a point, then rotates back and sweeps past it again during the same frame readout. This creates a multi-valued mapping that is mathematically messy.

However, the authors of this paper asked a crucial question: Are there Rolling Shutter cameras that behave more like perspective cameras? Specifically, is there a class of RS cameras that projects every generic point in space to exactly one image point?

They call these Order-One Rolling Shutter (\(RS_1\)) cameras. As illustrated in Figure 1(b), these cameras maintain a one-to-one relationship between points in space and points in the image, making them much easier to work with mathematically while still modeling the real-world distortion.

The Mathematical Model

To formalize this, the authors constructed a back-projection model. Instead of just thinking about pixels, they think about Rolling Planes.

The Geometry of Rolling Planes

Since each line of the image corresponds to a specific time, and the camera has a specific center and orientation at that time, all the light rays captured by that single line of pixels form a plane in 3D space. This is the Rolling Plane, denoted as \(\Sigma(r)\).

  • \(r\): The rolling line index (which row of the image we are on).
  • \(C(r)\): The position of the camera center at the time row \(r\) is read.
  • \(\Sigma(r)\): The plane in 3D space passing through \(C(r)\) that corresponds to row \(r\).

The set of all rays captured by the camera forms a geometric structure called a Line Congruence.

Overview of RS notation showing the camera center trajectory C(r) and the rolling planes.

Figure 2 visualizes this setup. As the camera center \(C(r)\) moves along a trajectory (the curved line), the projection plane \(\Pi(r)\) shifts. The rolling line \(r\) sweeps across the projection plane.

The authors derive a map, \(\Lambda\), which links the 2D image coordinates back to the 3D light rays (elements of the Grassmannian, \(\text{Gr}(1, \mathbb{P}^3)\)).

The equation for the back-projection map Lambda.

For a camera to be Order-One, this map \(\Lambda\) must be birational. In simpler terms, this means the relationship between the image coordinates and the 3D rays must be invertible using rational functions (polynomial fractions). You give me a ray, I give you a unique pixel; you give me a pixel, I give you a unique ray.

The Fundamental Characterization of \(RS_1\)

The paper provides a powerful theorem (Theorem 4) that characterizes exactly what these cameras look like geometrically. For an RS camera to be of Order One (\(RS_1\)), two main conditions must be met:

  1. Intersection Line \(K\): The most visually intuitive finding is that all rolling planes \(\Sigma(r)\) must intersect in a single line \(K\) in 3D space.
  2. Rational Motion: The movement of the camera center \(C(r)\) and the rotation must follow algebraic constraints (specifically, they must be rational maps).

This geometric constraint—that all planes associated with the scanlines meet at a common axis \(K\)—is a massive simplification. It allows the authors to classify these cameras into specific types (Type I, II, and III) based on how the camera center’s path \(\mathcal{C}\) interacts with this intersection line \(K\).

Constructing Practical \(RS_1\) Cameras

You might be wondering: “Is this just math theory, or do real cameras behave this way?”

It turns out that Linear Rolling Shutter cameras—a standard model used in robotics and autonomous driving—are often \(RS_1\) cameras.

Linear \(RS_1\) Cameras

A “linear” RS camera is one that moves in a straight line with constant speed and does not rotate (or rotates constantly). This is a very good approximation for a car driving down a highway or a drone flying a straight path.

The authors prove a specific condition for this: A linear RS camera has Order One if and only if its motion line is parallel to the projection plane.

If the camera moves towards the scene (perpendicular to the sensor), it is Order Two (points might be seen twice). But if it sideslips (moves parallel to the sensor, like looking out a train window), it is Order One. This covers the vast majority of “drive-by” footage used in mapping and localization.

Illustration of the parameter space for linear RS1 cameras showing the relationships between K, C, and infinity.

Figure 3 illustrates the geometry of such a camera setup. The diagram shows the relationship between the camera path \(C\), the intersection line \(K\), and the rolling planes \(\Sigma\) relative to the plane at infinity \(H^\infty\). The vector \(B\) represents the normal vector of the rolling planes as they sweep.

The Straight-Cayley Model

The paper also analyzes the Straight-Cayley model, a popular practical model for RS cameras that uses a specific parameterization for rotation.

Diagrams showing the complex geometry of camera centers and rolling planes for Straight-Cayley models.

Figure 6 (from the image deck) visualizes an example of a Straight-Cayley camera that qualifies as \(RS_1\). The cyan curve is the trajectory of the camera center (a twisted cubic curve), and the black fans are the rolling planes. Notice how they all converge on the magenta line \(K\). This visual confirmation proves that the abstract definition of “all planes meeting at a line” actually corresponds to complex, realistic camera motions used in engineering.

Images of Lines

In a standard Global Shutter photograph, a straight line in the 3D world looks like a straight line in the 2D image.

In a Rolling Shutter image, we know this isn’t true (remember the warped propeller). But what shape is it?

The authors prove that for \(RS_1\) cameras, the image of a 3D line is a rational curve of a specific degree. For the common case of a camera moving at constant speed along a line (Linear \(RS_1\)), the image of a 3D line is a conic section (like a hyperbola or ellipse).

This is a crucial insight. If you are trying to write software to recognize power lines or lane markings from a moving car, you shouldn’t look for lines—you should look for conics. The paper provides the exact degree of these curves for different camera setups:

Table showing the degrees of the image curves for different camera types.

(Note: The table referenced describes that for Linear \(RS_1\), the degree is 2, meaning a conic).

Structure-from-Motion: The Minimal Problems

The “Holy Grail” of geometric computer vision is solving for Structure-from-Motion (SfM). Given a set of matching points between two or more images, can we calculate where the cameras were and where the 3D points are?

For standard cameras, we have the famous “5-point algorithm.” For Rolling Shutter, it’s much harder because we have extra unknowns (velocity).

A “Minimal Problem” is the smallest set of data points required to produce a finite number of solutions for the camera poses. For example, “Can I solve this if I have 7 points seen by 2 cameras?”

Using their new theory, the authors classified all possible minimal problems for Linear \(RS_1\) cameras observing points and lines.

The 31 Problems

They discovered exactly 31 minimal problems for 2, 3, 4, and 5 cameras.

Illustration of all minimal problems showing camera configurations and point/line requirements.

Figure 4 acts as a map of these discoveries. Each entry represents a solvable geometric problem.

  • The Code: The numbers like 2100101 represent the number of features: free points, points on lines, lines, etc.
  • The Result: The number below the drawing (e.g., 60, 320, 140) is the degree of the problem—essentially, how many mathematical solutions exist.

Why does this matter?

  1. Efficiency: Problems with small degrees (like 28 or 48) are easy for computers to solve in milliseconds using “minimal solvers.”
  2. Feasibility: Before this paper, engineers might have tried to solve relative pose with random numbers of points, not knowing if a solution was even mathematically possible. This table tells you exactly how many points and lines you need.
  3. New Tools: The authors highlighted several “practical” problems for 2 cameras (using 7 or 9 points) that are suitable for real-time applications like autonomous vehicle navigation.

Conclusion: A New Foundation for RS Geometry

The paper “Order-One Rolling Shutter Cameras” does something rare in computer vision: it simplifies a complex problem by adding more structure.

By defining the class of \(RS_1\) cameras, the authors bridged the gap between the simplicity of perspective cameras and the chaos of general rolling shutters. They showed that:

  1. Geometry: \(RS_1\) cameras are defined by rolling planes intersecting in a line.
  2. Mappings: They project 3D points to 2D points uniquely (one-to-one).
  3. Reality: Practical setups, like cars moving sideways, often fit this model.
  4. Solvability: There are 31 specific minimal problems that can be used to compute relative pose, paving the way for better SfM algorithms.

For students and researchers, this opens the door to building “Rolling Shutter aware” 3D reconstruction pipelines that are mathematically sound rather than just approximations. As RS cameras remain dominant in smartphones and cheap robotics, this theory provides the blueprint for handling the “Jello effect” correctly.