SBIR-STTR Award

The accelerated pace of todays battlespace requires military leaders to build strategies and make decisions in a much shorter timeframe. The ability to exploit 3D battlespace data and intuitively visualize in 3D all elements of the operational environment will improve the operators ability to act with precision, certainty, and expediency. Emerging field of light display (FoLD) technology provides a glasses-free 3D aerial image with all the depth cues expected by the human visual system. As these systems mature, there is a requirement to develop the best protocol to display data and to interact with the 3D aerial image above and below the image plan. FoVI3D will investigate a common 2D touch tablet to allow for high-resolution text to be displayed to each viewer independently and provide for easy button access, and a 3D wand for the manipulation of 3D transforms and provide a virtual cursor within the 3D view volume. Both interfaces will be developed based on applied human factors engineering to optimize a natural human interaction. This is a required building block for the technical insertion of a FoLD central to the AEGIS combat information center improving the human-machine interface to enable simple, natural, collaborative data visualization.

Benefit:
With the emergence of light-field displays, viewers will have the ability to exploit the vast amounts of 3D data available and improve situational awareness and understanding. The ability to quickly observe, orient, decide, and act with certainty will enable the warfighter to engage, react, and respond in a confidently. With the accelerated pace of todays battlespace, the introduction of a new visualization paradigm that promotes a deeper and more accurate understanding of the environment is necessary. Federal agencies, non-governmental organizations, and commercial industries will benefit from the incorporation of natural 3D visualization. FoVI3D is currently working with the Army, Air Force, and Navy to develop and implement light-field display technologies to improve warfighter performance and reduce the manpower requirements associated with training, performing, and executing tasks associated with the decision cycle. The development and implementation of an interactive strategy for the light-field display is a necessary building block for the integration of this technology to the DoD and to commercial markets. Direct DoD Applications The Department of Defense has actively been supporting the research and development of light-field display technologies for the past 20 years. The vast benefits of improved situational awareness and understanding has applications for the strategic, operational, and tactical levels of command. The Air Force has defined requirements for a light-field display central to the future Air and Space Operation Centers as well as the joint effort with the Space and Missile Systems Center, the Enterprise Ground Services. The Air Force Materiel Command, 711 Human Performance Wing, Battlespace Visualization Branch continues to monitor, support, and validate the development supporting these visualization requirements. Air Force is also interested in the use of light-field displays in the cockpit of rotary and fixed wing aircraft. For rotary-wing aircraft, an enhanced visualization tool to aid in night, blackout, or rough terrain landings will save lives, protect resources, and increase the likelihood of a successful mission. An Air Force evaluation utilizing light-field holograms in close air support mission planning and execution concluded that the advanced visualization capability provided Joint Terminal Attack Controllers (JTACs) with greater confidence in the tested mission scenarios. All nine JTACs cited a better understanding of relative height information, enhanced collateral damage estimation, and determining lines of sight/ lines of fire as the greatest benefit. The US Navy warfare domains, sub-surface, surface, and air rely on data collected from a variety of radar, lidar, and sonar sensors. Light-field display technology would allow seaman to exploit this data for better situational awareness and understanding. Manned and unmanned subsurface vessels would be able to more easily and confidently maneuver through contested environments. Fast attack craft would be able to execute raids, landings, and extractions more quickly increasing the element of surprise on the adversary. Government and non-governmental organizations could execute natural disaster relief and recovery exercises as well as humanitarian efforts with the aid of the augmented light-field windshield. Whether through darkness of night or severe weather activity, the benefits of true spatial awareness in real-time could greatly speedup recovery efforts as well as protect the cargo. FoVI3Ds relationships with Honeywell, Lockheed Martin, and General Dynamics provides entry into the transition of this innovative technology into the services mission systems. Existing multiple-year IDIQs will help facilitate the test, evaluation, and integration required to successfully insert new technology. The integration of light-field display technology within all services will increase in situ and direct decision making and operations of unmanned vessels, vehicles, and drones. Medical Visualization Pre and Post-Operative Review The ability to natively visualize the 3D information generated from MRIs, CTs, PET/CTs, ultrasounds, etc. will help doctors, staff, and surgical centers to plan and prepare for patient specific procedures. Whether it is targeting radiation treatment in cancer patients, reducing critical surgical time in heart and brain procedures, or a precise plan for graft/implant requirements, the operating time is reduced resulting in a quicker recovery, fewer infections, and lowering the costs to surgical centers and patients. FoVI3D has had initial conversations with Barco, Christie, and LG introducing light-field displays for use in medical applications.

Keywords:
Field of light displays, Field of light displays, HETEROGENEOUS DISPLAY ECOSYSTEM, ENABLE QUICK DECISIONS WITH CONFIDENCE, MEMORY RETENTION IMPROVEMENT, 3D INTERFACE, Human Machine Interface, LIGHTFIELD DISPLAYS, COGNITIVE BURDEN REDUCTION

The accelerated pace of todays battlespace requires military leaders to build strategies and make decisions in a much shorter timeframe. The ability to exploit 3D battlespace data and intuitively visualize in 3D all elements of the operational environment will improve the operators ability to act with precision, certainty, and expediency. Emerging field of light display (FoLD) technology provides a glasses-free 3D aerial image with all the depth cues expected by the human visual system. As these systems mature, the Navy requires a mechanism for interacting with the 3D aerial image above and below the image plan that will provide an optimized ability for the user to engage with 3D information in a high-stress environment. FoVI3D, with support from the Naval Postgraduate School (NPS) Modeling Virtual Environments and Simulation (MOVES) Institute and Human Systems Integration (HIS) Program, will develop a Common Operational Picture (COP) Workstation prototype consisting of a central, large format, square LCD panel surrounded by 4 wide aspect ratio, 2D touch panels. The 2D touch panels provide task specific interaction for individual users while the central 3D visualization provides a COP for collaboration. A 3D wand system will enable users the ability to interact with the 3D information. After year one, the prototype 3D COP visualization workstation will be delivered to NPS for design validation, human factors evaluation, and user integration/experience feedback. Year three will validate the principle benefits of face-to-face interaction for battlespace management, mature the application demonstration, and support the Land Based Test Site demonstration. The work performed in this phase II effort is a required step for the future technical insertion of a FoLD central to the AEGIS combat information center thereby improving the human machine interface to enable an optimized ability to engage with complex 3D information in a high-stress environment allowing the warfighter to increase task accuracy, reduce response time, lower the cognitive strain and improve overall situational awareness/understanding.

Benefit:
Advanced visualization capabilities able to exploit, understand, and communicate complex 3D data is highly desired in multiple verticals across a variety of industries. Beyond NAVSEA AEGIS Integrated Combat System but still within the DoD, U. S. Air Force Air Operation Center and the Enterprise Ground Services Office (associate with Space Command and Control), U.S Central Command (CENTCOM) Military Information Support Operation (MISO) team, and the U.S. Army Futures Command are actively investigating advanced 3D visualization opportunities. The DoDs requirements translates to NASA and commercial space travel and exploration; disaster planning, response, recovery, and casualty evacuation services/functions for DoD and non-government organizations (NGOs); and the scouting and surveillance needs of the Secret Service, Federal Bureau of Investigators, and U.S. Intelligent Agencies. Beyond traditional ISR needs, both the medical and oil and gas industries have a high value need to better understand complex environments quickly, accurately, and effectively. FoVI3D plans to leverage current DoD contracts and relationships with large prime contractors to further the integration and acceptance within the Defense Programs. Continuing to present and speak at Defense sponsored workshops such as those supported at AFWERX, SOFWERX, and Industry Days, FoVI3D will solicit interest within other commands of the armed services, intelligence agencies, NATO organizations, and defense related contractors. On the commercial side, FoVI3D has been engaged with LG and Barco who are major distributors of commercial medical displays, as well as a host of other industries. The commercialization goals for the Common Operational Picture Workstation being developed during the phase II effort include defining the human factors evaluation/validation, defined acceptance criteria and metrology methodology for emerging 3D displays, published studies on cognitive performance, and engagement from the military medical community. Beyond the 36-month phase II base and option, FoVI3D to support the ruggedization process, further develop and refine the 3D metrology application (developed under FA8450-16-C-6672), and the integration of the next generation field of light display currently being developed under program FA8649-20-C-0029.

Keywords:
LIGHTFIELD DISPLAYS, COGNITIVE BURDEN REDUCTION, MEMORY RETENTION IMPROVEMENT, HETEROGENEOUS DISPLAY ECOSYSTEM, 3D INTERFACE, ENABLE QUICK DECISIONS WITH CONFIDENCE, Field of light displays, Human Machine Interface