Traditional methods for recovering 3D scene and object structure have suffered from a serious lack of robustness due to errors in computing optical flow or errors in tracking feature points or other tokens. In addition, due to computational complexity, fast algorithms for these applications are not known. We propose to commercialize a fast technique using FFT computations for dense scene and structure recovery using unconstrained video. Since the quality of structure estimates can be significantly improved by using more frames, we implement a fusion technology for integrating the two-frame based depth maps without significantly affecting the computational efficiency. The fusion technique uses the covariance matrix of the errors in two-frame based estimates and hence is statistically sound. In Phase I, we will integrate a suite of algorithms to demonstrate the utility of our approach for two commercial applications: face and terrain modeling. In the process of fusing two-frame depth estimates, we will be able to integrate a generic model of the object being recovered as well as generate a measure of quality of depth recovery. During Phase II, a prototype system will be built by integrating generic models and unconstrained videos. The primary military applications for the proposed work are in generating improved terrain models, and building target and scene models for simulation of battlefields. Commercial applications are in 3-D model building for facial animation, face recognition, real estate walkthroughs and virtual reality applications.
Keywords: Depth Recovery, Face Modeling, View Synthesis, Camera Self-Calibration, Scene Modeling, Object Modeling, Structure From Motion, Terrain Modelin
