SBIR-STTR Award

Primordial, Panoptic, and intuVision propose the development of Tentacle—a multi-camera immersive surveillance system that synthesizes information from hundreds of video streams into an intuitive 3D representation. Team Primordial will start with our existing state-of-the-art baseline components—Panoptic C-Thru, intuVision Panoptes, and intuVision Video Recall—which already fully meet 6 and partially meet 7 of the topic’s 13 requirements. Fully met requirements include automatically extracting metadata for tracked entities, scaling to hundreds of sensors, enabling real-time queries, supporting customizable event notification, and enabling diverse sensor integration. Partially met requirements include supporting adaptive multi-spectral sensor fusion, dynamically moving multiple cameras, and combining feeds into a single representation. From this baseline, Team Primordial will develop Tentacle to fully meet 100% of the topic’s requirements. In phase I, we will update Panoptes to support geolocating entities and transmitting reduced-bandwidth streams. We will also prototype the Panoptic C-Thru 3D surveillance display, conduct a demonstration, and deliver a final report. Upon phase I’s completion, Tentacle will fully meet 9 of the requirements. In phase II, we will update Panoptes to support moving cameras, sensor fusion, and detecting construction activities. Upon phase II’s completion, Tentacle will fully meet all 13 of the topic’s requirements.

Benefit:
By fusing hundreds of video streams into a single synthesized 3D display and automatically flagging potential security events, Team Primordial anticipates that Tentacle will 1) increase an analyst’s event detection probability, 2) reduce response time, 3) decrease the analyst to sensor ratio, 4) reduce fatigue, 5) increase job satisfaction, and 6) reduce turnover. Tentacle will benefit large surveillance systems such as those found at airports, military bases, hotels, casinos, entertainment venues, federal buildings, hospitals, and large retail operations. Northrop Grumman has expressed an interest in deploying Tentacle to 114 United States embassies under an existing State Department contract. Raytheon has expressed an interest in deploying Tentacle to airports under an existing contract with the Chile Ministry of Defense and Administration.

Keywords:
Realtime Video Surveillance, People And Vehicle Tracking, Multi-Camera Multisensor Fusion Algorithms, Automated Identification And Tagging, Animated Graphical Interface, Metad

The technological capacity to gather raw sensor data has far exceeded our capacity to exploit and understand it. There are neither enough people available nor intelligent computer algorithms developed to process all incoming sensor data in real-time. To address this problem, Team Primordial (Primordial plus partners Panoptic, intuVision, and Carnegie Mellon University) proposes Tentacle, a multi-camera immersive surveillance system. Simply put, Tentacle is software that facilitates quickly understanding what is happening now and in the past. For its eyes, Tentacle uses the ever-growing number of battlefield sensors. Tentacle fuses sensor feeds into a 3D virtual world comprised of imagery, terrain, and avatars representing people and vehicles. Tentacle works by integrating real-time feature extraction and tagging algorithms for detecting, tracking, and identifying entities with rule-based database query and dynamic alert tools for de-cluttering and highlighting relevant events. In phase I, Primordial demonstrated Tentacle’s ability to display a 3D world mimicking reality while supporting queries and alerts. In phase II, Primordial will update Tentacle by improving its static tracking, entity correlation, metadata extraction, and behavior analysis algorithms. We will also add support for tracking entities from moving platforms (e.g. UAVs) and performing spatial queries.

Benefit:
By fusing hundreds of video streams into a single synthesized 3D display and automatically flagging potential security events, Team Primordial anticipates that Tentacle will 1) increase an analyst’s event detection probability, 2) reduce response time, 3) decrease the analyst to sensor ratio, 4) reduce fatigue, 5) increase job satisfaction, and 6) reduce turnover. Tentacle will benefit large surveillance systems such as those found at airports, military bases, hotels, casinos, entertainment venues, federal buildings, hospitals, and large retail operations. Northrop Grumman has expressed an interest in deploying Tentacle to 114 United States embassies under an existing State Department contract. Raytheon has expressed an interest in deploying Tentacle to airports under an existing contract with the Chile Ministry of Defense and Administration.

Keywords:
Realtime Video Surveillance, People And Vehicle Tracking, Multi-Camera Multisensor