![[Up]](/Images/Up.gif){kind=small}
![[Top]](/Images/Top.gif){kind=small}
![[Search]](/Images/Search.gif){kind=small}
The University of Maryland is developing (a) and (b). The problem of stabilization is an illposed problem, like many other visual recovery problems. A two-tiered approach is being followed. On the one hand particular solutions to the problem are being developed by imposing constraints on the vehicle and the environment. Using an image registration algorithm, is is possible to compensate for image plane displacements due to unknown camera motion. This method has been tested on a limited set of real images acquired from a HMMWV with encouraging results. As this technique is based on local convolutions, real-time implementations are possible using dedicated hardware. It is planned to extend this method to incorporate full 3D motion information as well as FLIR data. On the other hand, an active solution to the problem is being developed through dynamic fixation, which is basically the way humans stabilize their images. It is planned to implement on a recently acquired head-eye system, equipped with inertial sensors, the capability of fixation as the system is moving. Fixation for a short time interval stabilazes the imager. It can then be followed by a saccade and a new fixation.
Regarding the problem of egomotion estimation a two-pronged approach is again being pursued. An end-to-end algorithm for feature based estimation of egomotion has been developed and tested on six indoor and outdoor real monocular and binocular image sequences. The algorithm is based on image plane compensation of unknown camera motion, feature extraction, establishing correspondence, and a recursive nonlinear Kalman filter. Although the problems of establishing correspondence and egomotion are known to be ill-posed, by using the constraints on vehicle motion and additional sensor data from FLIR and LADAR (expected to be available from Martin Marietta), it is planned to develop robust methods for feature detection, target acquisition, tracking and egomotion estimation. The availability of FLIR should make feature extraction easier, and the availability of target range information from a LADAR should make the structure from motion equations well-posed. Work is also being done on an algorithm for the estimation of the vhiecle's motion that is not based on correspondance, but works by locating global patterns on the changing images. It is planned to implement the algorithm in real time and demonstrate it on NIST sequences.
A model-based egomotion estimation algorithm for an autonomous vehicle navigating through rough terrain has also been developed. Due to the uneven terrain, the vehicle undergoes bouncing, pitch and roll motion. To reliably accomplish other tasks such as tracking and obstacle avoidance using visual inputs, it is essential to consider these disturbances. Two vehicle models available in the literature are being used for egomotion estimation. The Half Vehicle Model (HVM) takes into account the bouncing and pitch motion of the vehicle, and the Full Vehicle Model (FVM) also considers the roll motion. The dynamics of the vehicle are formulated using standard equations of motion. Assuming that depth information is known for some landmarks in the scene (e.g., obtained from a laser range finder), a feature-based approach is proposed to estimate vehicle motion parameters such as the vertical movement of the center of mass and the instantaneous angular velocity. An Iterated Extended Kalman filter (IEKF) is used for recursive parameter estimation. Simulations have been performed for both known and unknown terrain.
The University of Rochester is using (a) and (b) to develop (c). In particular, methods have been developed for detecting independently moving objects from a moving vehicle in real time. The primary method uses an estimate of the vehicle motion parameters to produce a constraint map that can be used to flag motion that is inconsistent with what could be produced by motion of the vehicle. The current version uses a model of allowed vehicle motions in order to quickly estimate these motion parameters from the global motion field. This was effective in the situations tested so far, but does not handle arbitrary vehicle motions. Use of the Maryland results will remove this limitation. A second independent motion detector does not need motion parameters, since it only detects objects that maneuver, e.g., humans, animals, or vehicles that are turning or accelerating. This information will ultimately be incorporated into the threat map.
The second aspect of the Rochester contribution is identification of the independently moving objects detected. The algorithms will flag such a motion as to whether it arises from a person, a vehicle, or from trees blowing in the wind. In order to enhance the information content of the threat map, techniques for classifying motion will be applied. In particular, primary classification techniques have been formulated that can flag a motion as smooth, periodic, episodic, or statistical in nature. Smooth motions include those due to mostly rigid objects such as vehicles. Periodic motions arise primarily from locomoting people or animals and from certain sorts of machinery. Episodic movements are time-bounded motions such as an explosion or throwing an object. Statistical movements arise from non-rigid motions of complex objects such as waves on water, or vegetation blowing in the wind. Second tier classification techniques for periodic and statistical motions are being developed which have already exhibited some success. Work is also being done on classification techniques for episodic motion. Smooth motion does not have much distinguishing character, so object recognition techniques seem the best bet for identifying smoothly moving objects.
The University of Pennsylvania is using (a) to partially develop (d). In order to develop (d), one needs to work out the technical details involved in the particular problems of modelling physics-based material classification. The work is currently concentrating on:
Regarding (2), range data will be introduced in order to distinguish among potentially ambiguous cases of shading, shadow and albedo variation. In conjunction with color processing, depth maps enable a search for objects which may cast visible shadows, resulting in shadow projection or verification. Ideally, it will also be possible to distinguish shading and shadow with the help of information about gross characteristics of the geometry of the surface. The specific plan for this year is to integrate range information acquired by the vehicle with the physics-based approaches to shadow detection that have already been developed, and demonstrate the capability on data acquired by the NIST HMMWV. This capability will be one step toward the development of (d).
NIST is developing a hardware and software infrastructure on the HMMWV testbed so that it will be ready to accept and integrate software dealing with (a), (b), (c), and (d) by the end of 1994. The data used in the experimental development of (aĞd) is being collected by the NIST vehicle.