SBIR-STTR Award

This proposal is in response to rapid modeling need in our military force. We plan to develop and demonstrate a framework that will allow warfighters to rapidly acquire and edit geospecific terrain databases, including both building exterior and interiors. The output of our models will be in standard 3D format, which can be used in a variety of simulation and training software, including VBS2. We propose a two-pronged approach to achieve this goal. One is a novel hand-held 3D camera that supports many modes of operations, from passive stereo image to active 3D scanning, under a variety of operation conditions, from indoor to outdoor. The other is a novel reconstruction-by-recognition modeling pipeline. The outcome of our system is a visually complete 3D model consisting of common objects in a terrain database, such as houses, plants, street lights, mailbox, etc. These objects are also labeled with semantic information, allowing quick editing by novice users. Our team has combined experiences of over 25 years in hardware design and 3D modeling. We leverage our prior results in stereo matching, scene understanding, and model reconstruction to quickly achieve the technical goals by the end of this Phase I project.

Benefit:
The success of this project will address a major hurdle in military simulation and training: the rapid creation and editing of high-fidelity training databases by warfighters, not professional modelers, in the field. This is expected to enable more realistic and effective training for the ever changing environment in a war zone. In addition to military use, we believe our unique hardware and novel software will be attractive for many other applications in which 3D visualization is valuable, from entertainment to e-commerce.

Keywords:
reconstruction by synthesis, reconstruction by synthesis, 3D range camera, semantic reconstruction, Scene Understanding, terrain modeling.

The follow up research has two primary tasks: (1) to minimize the camera design by integrating both the sensors and the computing platform into a portable package, which a target cost of 2000 dollars; (2) to continue simulator-friendly 3D model reconstruction using semantic Analysis. Raw data from sensors contain too many points or triangles for direct use in VBS2, and the data are unstructured. Novel algorithms developed in Phase I detect important semantic features to reconstruct models with semantic labels. In phase II, we will optimize the algorithm and increase its robustness to develop a commercial quality high fidelity modeling product which meets the needs of VBS2 or other simulation engine. Monitoring the progress of ongoing projects and product development addressing indoor 3D acquisition and modeling, we will develop additional software interfaces, allowing input from other types of sensors possibly developing porting ability of modeling modules onto cellphones. We plan to develop interface with our own low-cost rapidly deployable immersive display. Our unique display uses built-in cameras for automatic calibration and provides up to 360-degree field-of-view. Our goal: a complete visual simulation/training system which directly uses 3D models acquired in the operational theaters, improving training, realism and efficiency.

Benefit:
Our company has a long history of developing product and services related to 3D visualization. One of the major problems we often hear from our customers is that they would like to have easy and quick means to create virtual worlds based on real world images and video. In addition to the military training and simulation applications targeted in this proposal, we believe realistic 3D models can have many other applications such as entertainment and web commerce. For example, having an accurate and realistic interior models can be used in home decoration and real estate sales, both are huge mulit-billion dollar market. Having a low-cost solution will enable us to pursue commercial applications in these areas.

Keywords:
DTED, Modeling, Terrain, Simulation, VBS2, Synthetic Natural Environment