SBIR-STTR Award

In recent decades, High Performance Computing (HPC) has risen as one of the key enablers of innovation in science and engineering. However, industrial adoption has been impeded by the complexity and cost of deploying applications. Problems include the quality, speed, and flexibility of software tools and the specialized expertise required to adapt solutions given the profusion of hardware configurations. Efficient data reading and writing remains a key bottleneck in standard approaches, with poor I/O often limiting the scalability of application codes, reducing the scale and scope of problems that can be addressed with simulation. This SBIR Phase I will support the deployment of PIDX, a high- performance parallel I/O library that has been shown to scale efficiently to a million cores on DoE flagship HPC systems. The key innovation enabling high-performance I/O is a proprietary hierarchical space-filling data layout that delivers optimal performance for memory hierarchies. Furthermore, the ViSUS framework enables fast visualization and analysis, even for petabytes of data stored on remote systems. However, achieving this performance, until now, has been a complex process requiring deep integration of the PIDX library, custom tuning of parameters for HPC systems, and designed for the open data typical of scientific environments. This SBIR will make this technology accessible for commercial use by reducing the time and complexity of installing and using the PIDX library and ViSUS framework. The first component of the work will be to harden the existing PIDX library and develop a simple API that is easy to use and integrate in simulation software. Next, software tools will be developed to simplify the installation and tuning of the library to achieve high performance on commercial HPC systems. Furthermore, the library will be integrated with mainstream data analysis and visualization clients in the form of plugins to enable fast remote visualization and simulation monitoring without disrupting existing workflows. Finally, The ViSUS data server will be extended to account for restrictions and permissions required for private use of data. Make the data generated immediately available for remote streaming. Commercial impact and societal

Benefit:
A simple and efficient I/O ecosystem is a prerequisite component of engineering and industrial use of HPC systems. Removing the I/O bottleneck will enable more efficient use of expensive HPC resources, allow larger and higher-fidelity simulations for more accurate modeling, and reduce the cost and complexity of writing scaling simulation codes. The kind of data addressed by this project is already being produced and utilized commercially, such as reservoir simulation in oil and gas exploration, fluid dynamics studies in aerospace industries, and computational electromagnetics in manufacturing processes. The industrial HPC market saw a 10% growth in the first quarter of 2015, which is further expected to expand as additional applications are adopted. The I/O library supported by this SBIR will furthermore ease the burden of scaling, as today’s scientific research capacity runs become tomorrow’s commercial capability processing.

In recent decades, High Performance Computing (HPC) has risen as one of the key enablers of innovation in science and engineering. However, complexity and cost of deploying applications impedes industrial adoption. Problems include the quality, speed, and flexibility of software tools and the specialized expertise required to adapt solutions given the profusion of hardware configurations. Efficient data reading and writing (I/O) remain a key bottleneck in standard approaches, often limiting the scalability of application codes, reducing the scale and scope of problems that can be addressed with simulation. How the problem is begin addressed: This SBIR/STTR Phase II will support the deployment of PIDX, a high-performance parallel I/O library, which scales efficiently to a million cores on DoE flagship HPC systems. The key innovation enabling high-performance I/O is a proprietary hierarchical space-filling data layout that delivers optimal performance for memory hierarchies. PIDX and the ViSUS framework enables fast visualization and analysis, even for petabytes of data stored on remote systems. This SBIR/STTR will make this technology accessible for commercial use by reducing the time and complexity of installing and using the PIDX library and ViSUS framework. Phase I results and Phase II course of action: Phase I hardening and deploying the PIDX library in the Uintah project resulted in reducing the I/O cost from 49% to 2% of the compute time, effectively dou- bling scientific productivity. With a very conservative estimate of just 2 cents per core compute hour, this simulation was completed with over $3M in savings for the DOE, already returning more than double the investment costs for the proposed Phase II SBIR/STTR grant. This SBIR/STTR Phase II will continue to harden the existing PIDX library and deploy performance improvements. The PIDX library will be lever- aged to enhance post-simulation analysis and visualization workflows. Finally, the breadth of simulations accelerated by PIDX will be expanded by developing support for adaptive resolution meshes. Commercial impact and societal

Benefit:
A simple and efficient I/O ecosystem is a prerequisite component of engineering and industrial use of HPC systems. Removing I/O bottleneck will enable more efficient use of expensive HPC resources, allow larger and higher-fidelity simulations for more accurate modeling, and reduce the cost and complexity of writing scaling simulation codes. The data addressed by this project is already being produced and utilized commercially in reservoir simulations in oil and gas exploration, fluid dynamics studies in aerospace industries, and computational electromagnetics in manufacturing processes. The industrial HPC market saw a 10% growth in the first quarter of 2015, which is further expected to expand as additional applications are adopted. The I/O library supported by this SBIR/STTR will furthermore ease the burden of scaling, as today’s scientific research capacity runs become tomorrow’s commercial capability processing.