Pattern of Life characterization involves studying how people interact with the world around them on a daily basis: places they visit; people they interact with; actions they perform. Effective tracking of such patterns of life has direct implications to defense intelligence, cyber intelligence and corporate security. To solve this problem, ObjectVideo in collaboration with Dr. Leman Akoglu (Stony Brook), proposes to build a system that mines multiple data sources for relational information between people and places, detects anomalies and provides the tools to track the behaviors of anomalous entities. We will study patterns of life on four data domains: surveillance video, satellite imagery, text streams and non-traditional data sources such as weather feeds. We will use image / video analytics libraries to identify events from surveillance videos and detect change in satellite imagery. We will invoke Natural Language Processing tools to perform named entity recognition, entity resolution and relation estimation. Additionally, we will fit time-series models to weather feeds to study anomaly. We propose a unified graph representation to combine aforementioned data. Finally, we will invoke anomaly detection algorithms that scale up with large dynamically changing graphs and will build a visualization tool that helps analyst track anomalies.
Benefit: Pattern of life characterization has direct implications to military intelligence, corporate security, cyber intelligence and competitive intelligence. Patterns of life can be studied from a variety of datasets: videos, text, imagery, public records. Since data availability is abundant, an analyst aimed at understanding the dynamics of a group of individuals of his interest can tap into day to day behavior. Anomalies detected in patterns of life can be as a result of benign changes to ones life (job-loss, health issues) or more serious changes such as growing resentment against the government, sudden associations with terrorist groups. Studying patterns of life provides the following advantages: It implicitly provides situational awareness. By studying patterns of life, we can provide real-time alerts for key events and newly emerging issues. Such situational awareness plays a critical role in dispatching relief measures to regions affected by natural or man-made calamities. We issue threat forecasts, whereby we predict events, describe the anticipated risks and identify vulnerabilities if any within a network of people. For instance, by studying patterns of life one may gauge a growing resentment against the ruling authority and simmering unrest among public. Under such instances, safety measures could be taken by the ruling authority to (a) calm the subjects (b) increase police protection to critical infrastructures (c) evacuate foreign nationals for their safety. Covert networks are typically difficult to discover as they blend very well with the society. Carefully monitoring patterns of life provides the opportunity to detect anomalies that an entity from the covert network displayed. Upon detecting an anomalous entity, by fine tuning data sources that focus on the anomalous entity we can detect the entitys hidden network. Thus detecting an anomalous entity typically leads to the detection of the entitys support structure. It leads the analyst to the entity within the covert network who plays a critical role in keeping the network together. F3EAD protocols address such entities as High Value Individuals (HVI). Most military missions are about thwarting HVIs, rather than anomalous entities. Finally, studying patterns of life often helps discover hitherto unknown relationships between different communities. Such a discovery can have significant security implications.
Keywords: trajectory, trajectory, Eigenbehavior, Visualization., PEGASUS, time-series, named-entity, OddBall
