SBIR-STTR Award

Effective Anti-Submarine Warfare (ASW) operations against near-peer threats require increasing numbers of sensors and data sources, more data from each sensor, and a means of developing a live, consistent, and accurate Unified Operational Picture from this this torrent of data. Even well-trained and experienced Operators can be overwhelmed by the increasing cascade of signals from a growing number of sensors and data feeds with varying data formats and certainty levels. Our team at ThayerMahan, led by Vice Admiral (ret) Mike Connor, has extensive experience with this challenge of synthesizing a clear target picture from masses of data from multiple sensors and domains. We deploy and operate 24/7 maritime surveillance-as-a-service operations, using long-dwell autonomous sensing systems. We process data streams into threat representations on-board those systems and escalate/upload potential target contact reports to our operations center. We have performed or are currently performing on contracts with the Office of Naval Research (ONR) and the Joint Staff on behalf of US Southern Command (SOUTHCOM) and US Indo-Pacific Command (INDOPACOM) to process large volumes of sensor data from passive acoustic arrays and RF signal detection systems, which we integrate with AIS data and environmental data to form a Unified Operational picture of the activity in maritime surveillance areas. We will develop Acoustic Sense and Track (ASSENT) for this SBIR an intelligent automation and machine learning (ML) system which fuses data and sensor streams into a persistent, 3D unified picture of the operational and threat environment. We will prototype and demonstrate this capability in Phase I using our existing data streams (acoustic, RF, AIS) to show the value of our approach. Our approach is a multi-layered AI system with four critical components: a) Featurization of sensor data our teams expert knowledge informs our ML pipeline and algorithms which power transformations of sensor data into data features which facilitate target identification and tracking; b) 3D Projection for Data Fusion - we project our featurized sensor outputs into entity-centric representations in 3D space, using deep associative vectors to match and enhance each threat entity representation with data from multiple sources; c) Forward Projection of Target Location we generate trajectory projections for each of the entities in the operational environment to bridge discontinuous detections and identify signals pertaining to the same object across sensors and time; d) Flexible Data packages and Interfaces for Operators, Fire Control, and C2 providing the right information in the right format to each user of the system for maximum ASW effectiveness, from the SQQ-89 operator to fire control and C2. Our solution delivers a clear, entity-based, fused operational picture to facilitate improved C2 understanding and tactical decision-making by system operators and ASW decision-makers.

Benefit:
ThayerMahan leverages acoustic data in its day-to-day operations as it grows a network of unmanned acoustic sensing systems, providing search as a service to enhance maritime security and support interdiction of illicit maritime trafficking. Customer demand for this service is growing rapidly, such that the Office of Naval Research has asked us to consider how we can support 100x as many hydrophones per acoustician as we do presently. Based on customer demand, it is clear the rapid growth of these sensor networks will outpace our ability to hire and train the acoustic and tactical analysts we will need in order to scale our operations to 100x their current size. As in the Navy, our acoustic beams-to-operator ratio is increasing rapidly. We need intelligent systems that empower acoustic operators to process vastly more data, to track targets more efficiently, and to track them across much larger sensing networks and territories of the ocean. We are one of a very few companies that uses these training- and operations-enhancing tools in our day-to-day operations. We are able to provide the data, the acoustic and AI expertise, and the operational experience required to build these algorithms, evaluate their performance on real data, compare with existing standards, and iterate towards a robust and actionable unified operational and tactical picture. Beyond the Navy, government and commercial demand is growing for services in maritime and port security, sensor and vehicle research and development (R&D); oil and gas; fisheries monitoring and enforcement; offshore wind energy; and operations of ports and shipping lanes. We are engaged in discussions with commercial entities including rsted Offshore North America about the use of acoustic detection to detect, track, and report endangered whales and other marine mammals around their wind farm operations. Undersea acoustics as a commercial service is building from a current estimated market size of $638M to $1.64B over the next 5 years. To meet expected market demand, our company will need 20 new staff over the coming 5 years - even with increasing automation and intelligent pre-processing of sensor data. Technology developed in this project will enable us to cover four times more area by 2025, with direct benefits to national security, transportation safety, and the environment, including marine mammals. The specific AI tool and underlying acoustic data set will be of interest to other ASW communities including Surface and Submarine Forces. We have significant revenue in this domain already, and with the tools we propose to develop here, we will grow this business substantially. In addition, we anticipate commercialization results from continued expansion of our services to support the missions of other US government agencies (NOAA, DHS) and commercial entities, particularly in the emerging offshore wind market.

Keywords:
data fusion, data fusion, Advanced Teaming, AI, ASW, Sonar, tracking, Unified Operating Picture, Common Operating Picture

During Phase I, ThayerMahan was able to build, test, and validate on relevant internal data sets an integrated, multi-layered platform for detecting, tracking, and correlating multi-modal data to deliver an integrated track for each vessel in a given operating area. The work we propose for Phase II moves our Phase I-validated architecture and modules, which were trained and proven on prototype-scale training data, to a fully operational system trained and optimized on vastly larger training data and more diverse scenarios. The key technical/data gap in Phase I was active acoustic data, which will be our top priority focus during the the Phase II Base. The proposed work will lead to the delivery of a state-of-the-art, automated system which delivers robust tracking and a Unified Operational Picture (UOP) for Anti-Submarine Warfare (ASW) which operates vastly faster and with higher accuracy than current systems and workflows can provide, while retaining the full value of spectral and kinematic data collected by sensors. The expanded solution positions the Navy to rapidly declutter the tactical ASW picture, target threats faster, and operate at a higher level of Situational Awareness for C2 and fires. We will also deliver on the additional IWS-5 objectives outlined to our team, e.g., a product which can detect an acoustic energy signal, track it across bearings and through time, and re-identify it on the same or another sensor at a future point in time. Ongoing internal research and development at ThayerMahan since the end of Phase I shows continued improvement in auto-scissoring and target featurization capabilities, each of which will deliver enhanced functionality to these IWS-5 priority areas. During the course of Phase II, we will evolve this capability into a fully operational system which performs this entire task automatically in concert with the UOP for ASW capability outlined in the original SBIR. Our work is divided into three key areas: a) Optimization of neural networks and AI systems including extending our proven architecture to encompass active sonar returns and outputs from Navy classification systems; b) Data pipeline engineering and neural network retraining this work focuses on delivering end-to-end data flows which ensure robust operational performance, and extending data sets to retrain algorithms and improve overall performance, and; c) Software engineering and integration this effort focuses on the display of our Unified Operational Picture, first using ThayerMahan tools, and then integrating with Navy interface systems, as well as the operationalization of our platform and its containerization and integration with Navy systems.

Benefit:
As data volumes generated by U.S. Navy platforms and ThayerMahans own maritime acoustic surveillance platforms continue to rise rapidly, there has arisen an urgent need for automated systems which can process complex acoustic signals into a comprehensive picture of all of the vessel and threat tracks in the operating environment. As was identified in this SBIR, there is an need for novel systems which can deliver this capability reliably and instantly at scale. This need translates into commercial opportunity, which ThayerMahan has demonstrated its ability to capture. ThayerMahan already generate millions of dollars in annual revenue from acoustic data in its day-to-day operations as it grows a network of unmanned acoustic sensing systems, providing search as a service to enhance maritime security, protect marine mammals, and support interdiction of illicit maritime trafficking. Customer demand for this service is growing rapidly, such that the Office of Naval Research has asked us to consider how we can support 100x as many hydrophones per acoustician as we do presently. We are currently in discussions for multi-system, extended deployments in the Pacific and Atlantic, as well as for allies. The Navy can use our technology to improve shore-side and on-ship operations across the SQQ-89 (surface ships), BQQ-10 (submarines), and in shore-side operations, where hundreds of acousticians currently process and correlate data across hundreds of incoming beams simultaneously. Our AI-powered system is 1,700x faster than our current, manual internal processes for manual correlation of acoustic and other data for target tracking. These solutions will bring immense value to the Navy. We see and are capturing this value internally. A perfect commercial use case has just been launched during April 2022 we are operating a marine mammal detection operation mandated by the regulatory body overseeing the construction of rsted Offshore North Americas offshore wind installations. We provide multi-sensor-based tracking of North Atlantic Right Whales to prevent disruption to this species during offshore wind construction. This is a multi-million-dollar deployment which will benefit from instantaneous processing of acoustic information. To provide these services at scale, automation becomes mission-critical. Undersea acoustics as a commercial service is building from a current estimated market size of $700M to $1.8B over the next 5 years. To meet expected market demand in the sensor- and data-processing domain, our company will need 20 new staff over the coming 5 years - even with increasing automation and intelligent pre-processing of sensor data. Technology developed in this project will enable us to cover four times more area by 2025, with direct benefits to national security, transportation safety, and the environment, including marine mammals.

Keywords:
Acoustic, Artificial Intelligence (AI). automation, accelerate detection-to-tracking workflows, Anti-Submarine Warfare, algorithm, unified operational picture, data featurization