SBIR-STTR Award

Recognition of human intention using infrared video sequences has applications in surveillance. Although motion trajectories extracted from IR sequences contain significant information about the underlying event, we need a methodology to connect trajectories and intentions. ImageCorp, Inc proposes to implement two deformable-shape based descriptions for characterizing human intentions. The first one is based on the factorization of motion trajectories in terms of basis shapes such that every activity is represented as a weighted sum of basis shapes. When a video is given, the method extracts motion trajectories and identifies the closest activity using a similarity measure. If an unforeseen event occurs, the corresponding motion trajectory will be poorly represented by the existing basis shapes and so will be seen as an outlier. The second approach uses statistical shape theory for modeling the mean shape of the configurations of moving humans and its deformation over time. The mean shapes are represented using Markov processes, which enable us to derive conditions for identifying similar intentions as well as abnormal ones. Both approaches lend themselves for operator intervention and on-line training. We will evaluate the effectiveness of these two approaches using in house data and any sponsor provided data

SET Associates proposes to design, develop and evaluate an assisted/automated system for recognition of intents of humans in urban environments. We propose a modular architecture that involves solving the following problems: detection of stationary and moving humans in different cluttered backgrounds, state of the human (whether armed or not) and recognition of the particular weapon being carried and the intent of the human(s) carrying the weapon. In addition, the proposed system will determine if the detected humans are carrying harmless objects. The approach taken for detection of stationary and moving humans and the weapons they may carry uses a combination of shape and motion driven matching techniques. Patterns of motion exhibited by detected humans prior to conducting a hostile act are analyzed using deformable shape analysis methods. Finally, hidden Markov models are trained to recognize one of many hostile acts that the detected target may be planning to undertake. Our design methodology allows for operator intervention and on-line training. The system will be evaluated using data collected by SET Associates and also supplied by the sponsor. Issues such as real-time implementation and robustness to variations in modeled intents will be addressed.

Keywords:
Activity Modeling, Abnormal Activity, Factorization Theorem, Statistical Shape Modeling, Shape Space, Intention Recognition, Deformable Shapes, Hidde