[Home]
[Courses]
Finding DNA in Human Motion
You are in Home->Projects->Gait and Activity DNA
Motivation
Human motion analysis has been an active research area for the last two decades. The popularity is due to a boost of applications in surveillance, clinical and sports video analysis. Although the traditional approaches are based on markers or feature points on human body, we are interested in the tracking free methods.
Background
Methodology
We present a compact characterization for human gait and activity subspace. Traditionally, activities are studied as signals in X-Y domain (images) or X-Y-t volume domain. Our analysis is based on decomposing the activity subspace into X-t slices, which generates a twisted pattern, the Double Helical Signature (DHS). It is shown that the patterns sufficiently characterize gait and associated activities. The features of the DHS are: (1) It naturally codes appearance and spatio-temporal parameters for human gait; (2) It reveals an inherent geometrical symmetry (Frieze Group), leading to a distance definition as a temporal skeleton; and (3) It is compact for recovering gait and activity related parameters. We also study the DHS matching across cameras and time. A local curve embedding algorithm is applied to extract the geometrical structure under different activities and poses. We illustrate the effectiveness of DHS for several applications. The DHS is first used for simultaneously pedestrian segmenting and parts labeling. Our algorithm is robust to object size, moving directions, shadows as well as severe occlusion. Preliminary experiments indicate that the proposed approach is superior to many existing methods in terms of segmentation accuracy. We then apply it to match gait signatures in different views and time. Finally we classify activities such as carrying a backpack, briefcase etc. By recognizing various symmetries, we provide a reliable solution for load carrying event detection that does not depend on silhouettes and tracking landmarks.
Results and Future Directions
We presented a method for understanding the gait and activity subspace using DHS in layered slices. The proposed method naturally integrates temporal body dynamics with 2D shape information. It does not require silhouettes and feature tracking. Our approach has two main discoveries. First, the twisted pattern belongs to a Frieze Group, enabling separation of self-intersecting curves for robust and efficient learning. Second, only a finite set of DHS is needed for compact and sufficient representation for activity subspace topology reconstruction and articulation parameters estimation such as cadence, step/stride length and style for different individuals and a class of events related to carrying objects. Some of the demo video are shown below. Each video shows the event (carrying objects) detection with the insets showing the gait DNA at different DHS. For more details, pls refer to University of Maryland Invention Disclosure IS-2005-108, 2005.
Moreover, we have implemented a pedestrian monitoring system capable of simultaneously segmenting and labeling body parts, matching across various cameras and time as well as recognizing load carrying events. The experimental results demonstrate the effectiveness under lighting changes, shadows, camera motion, various viewing angles as well as severe occlusions. The sensitivity analysis shows the robustness for several key factors such as body movement direction, viewing angle and target size. The work indicates that considering human motion in spatio-temporal domain is a significantly efficient method to analyze gait and activities.
Publications, Patents and Awards
Gait DNA and its applications in surveillance systems. Yang Ran, Rama Chellappa, Qinfen Zheng, University of Maryland Invention Disclosure IS-2005-108, 2005