SBIR-STTR Award

The Marine Corps has taken an initiative towards transformation of Maintenance Training from the Industrial age to the Information age. However, traditional classroom training techniques with student handouts and presentations coupled with stove-piped legacy material results in training being out-of-date with system operations. This limits the modernization of learning techniques required to deliver and sustain improved readiness of Naval platforms. As more complex systems are fielded, not only will maintaining them be more challenging, but keeping training up to date will be even more challenging. There is a need for digital-twin driven immersive maintenance training techniques that provide up-to-date knowledge and expertise to successfully maintain systems. Qualtech Systems, Inc. (QSI), in collaboration with its industry partners, proposes an enterprise training platform, Training and E-Learning for Maintenance-tasks using a Modern Platform for Operational-efficiencies (TEMM-PO) , leveraging its combined expertise in model-based training, photogrammetry and machine learning (ML) applied to computer vision, and learning sciences, to provide a continuously improvable content authoring pipeline that provides just-in-time and just-in-place adaptive maintenance training with a varied set of immersive experiences. The capability will be provided as sharable web-content that provides on-demand, anytime/anywhere learning. TEMM-PO will reduce the time to create training curricula by using a seamless immersive content creation pipeline that automates the steps required to refine the source content. The training pipeline will be continuously improvable and will be in sync with the system tech-refreshes. The solution is not tied to any specific system and can be adapted across varied maintenance domains. TEMM-PO will lead to improvements in student proficiencies, increased retention of maintenance skills and concepts through tailored and adaptive training backed by expert reasoner that enables efficient troubleshooting. The in turn will result in a a better trained military personnel who end up maintaining the equipment, thereby increasing its operational availability.

Benefit:
This project will result in a maintenance training ecosystem that would streamline immersive content creation, reduce training time and improve learning gains. Some of the transition partners and opportunities are identified below: Marine Corps System Command (MCSC): QSI will work with the MCSC to capture vehicle troubleshooting guides for some of its combat systems into the web-based solution for Technician Training and Troubleshooting. QSI has had Phase IIs with Air Force to provide its Manufacturing Bases with a Guided Troubleshooting solution through adaptation, integration and deployment of the TEAMS tool suite for improving the operational availability of complex mission-critical depot maintenance infrastructure equipment. Incorporating AR/VR into technician training through the Guided Troubleshooting interface would be a natural extension of the effort. Immersive Naval Training Systems. The expected TRL at the end of Phase II will be TRL7. QSI plans to add a licensable module to its Product Suite that enables immersive guided troubleshooting. Applications of this commercial technology is expected in Military AR (MAR), Energy and Utilities, MRO (Maintenance, Repair and Overhaul), Healthcare training, Manufacturing, etc.

Keywords:
Continuous Improvement Cycle, Continuous Improvement Cycle, Automated Metrics capture, Adaptive Training, Immersive Content Pipeline, Easy-to-use AR/VR content creation interface, Distributed Learning, Digital twin, Domain agnostic

The Marine Corps has taken an initiative towards advancing Maintenance Training from the Industrial age into the Information age. However, traditional classroom training techniques with student handouts and presentations, coupled with stove-piped legacy material, results in training being out-of-date with system operations. This learning techniques are inefficient and limit the capability to deliver and sustain improved readiness of Naval platforms. As more complex systems are fielded, not only will maintaining them be more challenging, but keeping training up to date will be even more difficult. Hence, there is a need for digital-twin driven adaptive and immersive maintenance training techniques that provide up-to-date knowledge and expertise to successfully maintain systems. During recent conversations with PM TRASYS stakeholders, QSI learnt that the main challenge in creating immersive content is lack of ready 3D models in the schoolhouses. This necessitates development of a content creation pipeline that is able to generate 3D models from common means such as available 2D pictures, or taking new pictures using common cellphone cameras and photogrammetry techniques. Qualtech Systems, Inc. (QSI), in collaboration with Charles River Analytics (Charles River) and Aptima Inc. (Aptima) (the QSI team) proposes an enterprise training platform, Training and E-Learning for Maintenance using a Modern Platform for Operational-efficiencies (TEMM-PO) , leveraging its combined expertise in photogrammetry and machine learning (ML) applied to computer vision, to provide a continuously improvable immersive content authoring pipeline that provides just-in-time and just-in-place maintenance training with a varied set of immersive experiences. The content creation pipeline will use open-source tools to convert legacy training content into interactive and immersive 3D content. The proposed effort will result in a novel immersive content creation pipeline that sources legacy 2D images and 3D-scanning techniques and generates ready 3D models for use in the web-based AR/VR immersive scene designer. The proposed capabilities for converting legacy material to immersive content can accelerate the Marine Corps strategic goal of training technicians to be efficient maintainers.

Benefit:
The proposed effort will result in a novel immersive content creation pipeline that sources legacy 2D images and 3D-scanning techniques, and generates ready 3D models for use in the web-based AR/VR immersive scene designer. The proposed capabilities for converting legacy material to immersive content can accelerate the Marine Corps strategic goal of training technicians to be efficient maintainers. The research and technology development of the proposed enterprise maintenance training solution will benefit the Marine Corps in the following ways: Improving Marine Corps maintenance training effectiveness: TEMM-PO will provide just-in-time and just-in-place training with a varied set of immersive experiences. Automated content creation: The solution will reduce the time to create training curricula by using a seamless immersive content creation pipeline that automates the steps required to refine the source content into 3D models. Increase in Learning Gains: Improvements in student proficiencies, increased retention of maintenance skills and concepts through tailored and adaptive immersive training backed by the digital twin that enables efficient troubleshooting. Continuously improvable training pipeline: As the single-source-of-truth digital twin changes with system tech-refreshes, the agile training curricula automatically is kept up to date. Improving Marine Corps operational availability: Some of the indirect benefits stem from having better trained personnel who end up maintaining the equipment, thereby increasing its operational availability. Reduced Training Resource Footprint: Since the training is virtual, there is less reliance on physical equipment. Distributed learning: Sharable web-content that provides on-demand, anytime/anywhere learning. Objective Metrics: Measurable metrics in the creation of immersive content. Domain Agnostic: Not tied to any specific system, and can be adapted across varied maintenance domains. Multiple Learning Modalities: The immersive content can be instantly deployed remotely in multiple targets: tablets, smartphones, PMAs, AR devices, workstations, etc. QSI TEMM-POs ML driven content generation pipeline has the potential for few personnel to be able to perform the 3D object generation on a large scale, thus saving time and contracting costs. The proposed effort will result in a novel capability that combines realistic, interactive visualizations to seamlessly bridge the gap between traditional classroom training methodologies and hands-on practical applications. The proposed AR pipeline can help achieve the Navys strategic goal of 21st Century distributed/remote learning.

Keywords:
21st Century Learning, AI/ML, 2D to 3D conversion, Computer Vision, Democratization of Immersive Content Generation, Immersive Content Generation from Legacy Data, Photogrammetry, Remote Content Authoring