SBIR-STTR Award

To address the CBD’s need for an augmented reality (AR) display that shows chemical, biological, radiological, nuclear (CBRN), and electro-optical/infrared (CBRNE) threats, Intellisense Systems, Inc. (Intellisense) proposes to develop a new CBRNE Augmented Reality Display and Device Integration Network (CARDDIN), based on a modular network that incorporates CBRNE and geolocation information from all sensors and nodes in the network, fuses that data in real-time at each node, and streams AR video to a heads-up display (HUD) worn by on-the-move users while minimizing the size, weight, and power consumption (SWaP) of the electronics carried by each user. CARDDIN is platform agnostic, enabling integration with any reconnaissance vehicle, command and control (C2) systems, CBRNE sensor (whether vehicle?mounted or handheld), and HUD. As a result, CARDDIN is compatible with existing and future platforms and can be easily upgraded and tailored for specific concepts of operation (CONOPs). Situational awareness is maintained by using intuitive warning indicators, and CARDDIN’s scene management algorithm addresses issues of overlapping threat regions, ensuring situational awareness without increasing the cognitive load on the user. In Phase I, Intellisense will demonstrate the feasibility of CARDDIN by showing direct communication of mature radioisotope identification device (RIID) and mass spectrometer data to a HUD device to generate an AR display of sensor data with localization of threats. By the conclusion of Phase II, Intellisense plans to demonstrate the CARDDIN system in a relevant platform combining the sensors on a vehicle exterior with the real-time AR view on a HUD device from the interior of a vehicle to demonstrate real-time CBRNE situational awareness.

To address the CBD’s need for an augmented reality (AR) display that shows chemical, biological, radiological, nuclear (CBRN), and explosive (CBRNE) threats, Intellisense Systems, Inc. (Intellisense) proposes to advance the development of its new CBRNE Augmented Reality Display and Device Integration Network (CARDDIN), demonstrated to be feasible in Phase I. CARDDIN is based on a modular network that incorporates CBRNE and geolocation information from all sensors and nodes in the network, fuses that data in real-time at each node, and streams AR video at 60+ fps to a heads-up display (HUD) worn by on-the-move users while minimizing the size, weight, and power consumption (SWaP) of the electronics carried by each user. Each module in the network can independently provide all available CBRNE data to users and the network incorporates handoff algorithms to enable on-the-move users to move between module spatial coverage areas without interruption to the AR scene. CARDDIN is platform agnostic, enabling integration with any reconnaissance vehicle, command and control (C2) system (including inertial navigation systems (INSs) and the Android Team Awareness Kit (ATAK)), CBRNE sensor (whether vehicle-mounted, downrange, or handheld), and HUD. As a result, CARDDIN is compatible with existing and future platforms and can be easily upgraded and tailored for specific concepts of operation (CONOPS). Situational awareness is maintained by using intuitive warning indicators using Department of Defense (DoD)-standardized symbology, and CARDDIN’s scene management algorithm addresses issues of overlapping threat regions, ensuring situational awareness without increasing the cognitive load on the user. CARDDIN also incorporates built-in network security to ensure CONOPS integrity and to eliminate excess resource drain, improving the latency and operational time. In Phase II, Intellisense will mature the CARDDIN concept, support integration of the system onto a representative platform, and show integration with geo-referenced, off-board sensor systems, greater fidelity on the display, and increased usability for on-the-move users. By the end of Phase II, Intellisense will demonstrate CARDDIN in a relevant platform by combining the sensors on a vehicle exterior with the real-time AR view on a HUD device from the interior of the vehicle to demonstrate real-time CBRNE situational awareness. These Phase II results will be used to transition the technology to Phase III production.