SBIR-STTR Award

Deep Neural Networks have become ubiquitous in the modern analysis of voluminous datasets with geometric symmetries. In the field of Particle Physics, experiments such as DUNE require the detection of particle signatures interacting within the detector, with analyses of over a billion 3D event images per channel each year; with typical setups containing over 150,000 different channels.  In an analogous data intensive field, satellites continually produce datasets requiring the detection of millions of objects per 1000 sq km over the full surface of Earth. Understanding the uncertainty induced by the underlying Machine Learning Algorithm is important to such analyses. This error has not been included in analyses in a fundamental way and is currently included exclusively in sophisticated and costly empirical studies. We will develop a theoretical bounds on this error utilizing Fourier analysis (Xu, Zhang, Luo, Xiao, & Ma, 2019) and will build upon the a priori generalization bound established for shallow networks (Xu, Zhang, Zhang, & Zhao, 2019) by considering deep Rectified Linear Unit (ReLU) neural networks of minimal width and disparate test and train domains. We will then work on extending our bounds to simple Deep Convolutional Neural Networks, to simple empirical studies on disparate test and train domains, and to empirical studies for object detection.

Deep Convolutional Neural Networks (DCNNs) have become ubiquitous in the analysis of large datasets with geometric symmetries. These datasets are common in medicine, science, intelligence, autonomous driving and industry. While analysis based on DCNNs have proven powerful, uncertainty estimation for such analyses has required sophisticated empirical studies. This has negatively impacted the effectiveness DCNN, motivating the development of a bound on generalization risk. Uncertainty estimation is a crucial component of physical science. Models can be trusted only to the degree their limitations and potential inaccuracies are fully understood and accurately characterized. Best estimates can be wrong. Accordingly, the scientific community has invested a great deal of effort into understanding and benchmarking methods of uncertainty quantification, and we will bring a deep knowledge of those tools and traditions to bear on the the predictions of uncertainty for DCNN. There are parallels between astrophysics and computer vision in that many of the ground truth labels in real-world data are established by human inspection. We will bring the tools and expertise of science to bear on benchmark data in computer vision as well, both for rigor and to create points of reference for a broad community of practitioners and researchers. Generalization risk or error is the difference between the error as found in training or validation and the error that exists in application. In empirical studies, this risk is estimated using a blind test sample. However, doing so is costly when data is limited, and such studies are necessarily incomplete since the blind test sample does not include all the data that the DCNN will be applied to. This motivates the ascertainment and study of a mathematical bound on on generalization risk. In the norm based approach, we have found that the interplay between the frequency behavior of the target function and the NN depth determines it’s approximation error. In this project we will develop a toolbox which returns a bound on the a priori generalization risk when provided a DCNN topology and example data or some functional description. (This risk will be independent of the training of the sample.) This bound will be applied for ResNet101 and similar DCNN on tasks in computer vision and astrophysics, and the impact of the bound on astrophysics uncertainty analyses will be evaluated. In particular, we will evaluate this uncertainty for classification and regression tasks on images of strong lenses and galaxy mergers. This toolbox will allow improved uncertainty estimation in the domains where DCNNs are used, such as in astrophysics, particle physics and computer vision.