CMSC 828Z::

Class Description :: Syllabus :: Reading List :: Projects :: Helpful Links ::
 
 

CMSC828Z - 3D Photography and Inverse Rendering

News:

10/07/02: The lecture slides are now accessible from the syllabus page.
10/01/02: All 10 project descriptions are online!

Summary of Content:

This class will be about the interaction between computer vision and computer graphics. Specifically, we will study four related problems:
  1. 3D photography: recovery of shape and texture of the scene objects to create a view independent representation of the scene
  2. Inverse rendering: recovery of scene parameters like surface reflection properties and the ambient illumination that are important to rerender the scene under novel lighting conditions and spatial scene configurations
  3. Motion Capture: recovery of the motion models and parameters of the objects in the scene to enable retargeting of motions to other objects and animate the scene objects using new motion parameters.
  4. Camera Design: What camera design optimally facilitates the solution to the three problems above?
As witnessed by the ubiquity of "digital magic" in today's movies, there is a large need in computer graphics to integrate artificial imagery with real objects (or their images). In this class we will study how computer vision techniques can be used to accurately recreate "reality" and where the theoretical limits of its contribution lie and depend on the design of the camera employed. We will approach this by interpreting the computer vision problems named above as being inverse operators to the operators used in nature and computer graphics when transforming a scene description into an image. If we know the shape and reflection properties of all the objects in the scene, as well as all the illumination information, then accurate models exist of the physical process that transforms the electromagnetic energy that passes through the scene into the image we finally capture of the scene. In computer vision now, given a number of images of this scene, we would like to recover the scene parameters that were used to fill the surrounding space with light rays of varying intensity and color based on the captured images. Unfortunately, in contrast to the well-defined image formation process, the inversion of these physical process is not well-defined, it is ill-posed. Many sets of scene parameters can give rise to the same image!

Goals for this Class:

The goal of this class is to give the student a feeling for the difficulties involved when one tries to extract information about the world from image sequences. Therefore, we will study the image formation pipeline, starting from representations for the shape of objects and their surface properties, up to the accurate models to the lens system and ccd architecture of modern cameras. We will not limit our study to conventional pinhole cameras, but we will also study more general images and cameras that are better suited for processing by computers then conventional images. Having understood the image formation process, we will then study in a series of projects to what extent and under what assumptions it is possible to "invert" the image formation pipeline and recover the scene parameters. More specifically during this class the union of all the projects (see the projects page ) will allow us to recover spatio-temporal models from multiple calibrated video sequences which we will supply. For previous work you can take a look at Spatio-temporal Reconstruction of Human Head using Multi-resolution Surfaces.

Administrative Issues, Workload and Grading Policy:

Instructors: Yiannis Aloimonos and Jan Neumann with help by Patrick Baker and Abhijit Ogale

Location: (CSI 1121)

Time: Friday 11:30am- 2:00pm

Workload: Each student in the class will be asked to give a presentation (see the preliminary syllabus for a list of topics, the topics will be finalized during the first week of class) and do a project that will be a part of the spatio-temporal reconstruction problem. At the end of the class we expect everyone to turn in a report that integrates the result of the project with the presentation to illustrate how the presented concepts were a part of the solution to the "inverse rendering" problem.

Grading Policy: The grade for this class will be based on this report and the quality of the presentation given. Since there are no exams in this class, this class will NOT count for the Phd or MS qualifying sequence nor for the MS Comps!

Prerequisites: This course will deal with problems in computer graphics and computer vision on a research level. Therefore, familiarity with the concepts of computer vision and/or computer graphics, as well as good knowledge of multivariate calculus and linear algebra is recommended. The programming language for the projects will be MATLAB and C++, so previous experience with both is very helpful.

Contact: For questions please email cmsc828z at videogeometry.com.


 

  
   © 2002 by Jan Neumann •  jn at videogeometry dot com