The Approach
Our approach combines the processes of smoothing, segmentation, 3D
motion and structure estimation. New constraints have been developed
which are defined directly on the image derivatives, leading to a
geometrical and statistical estimation problem. The main idea is based
on the interaction between 3D motion and shape, which allows us to
estimate the 3D motion while at the same time segmenting the scene. If
we use a wrong 3D motion estimate to compute depth, then we obtain a
distorted version of the depth function. The distortion, however, is
such that the worse the motion estimate, the more likely we are to
obtain depth estimates that vary locally more than the correct
ones. Local variability of depth is due either to the existence of a
discontinuity or to a wrong 3D motion estimate; by exploiting the
statistics of the raw image measurements (derivatives) these two cases
can be differentiated. Clearly, at the end of the process a good
estimate of correspondence can also be made.
Since at the beginning of the process correspondence or flow is not
available, we cannot utilize the epipolar constraint that has been
traditionally used. Instead, we utilize the positive depth constraint
and geometric constraints arising from understanding the distortion
function, as explained before. More specifically, if 
is the actual scene model and
, with 
a distortion function depending on the errors in the 3D transformation and image measurements. Understanding this function provides the insight that human visual space is a non-Euclidean space; further, it explains a number of illusions and predicts others (see the section on biology). At the same time, this understanding gives rise to algorithms for 3D motion estimation, motion segmentation and scene reconstruction from video sequences, producing results not obtainable by correspondence-based approaches. Theoretical results also demonstrate that existing approaches are special cases of our approach; that is, our approach is provably better than the state-of-the-art, correspondence-based schemes.
Return to the front page
Revised 1999/04/15
Send questions about these Web pages to
larson@cfar.umd.edu