SBIR-STTR Award

For many classes of programming problems that require parallel libraries, years of experience are required to effectively use those libraries, and frequently they are difficult to use, requiring complicated programming interfaces. The large time investment and limited usability prohibit typical domain specialists, who have limited programming expertise, from creating parallel codes and benefiting from parallel resources. Programs that can make this time investment typically have huge resources at their disposal, such as the Advanced Simulation & amp; Computing (ASC) campaign, or the Scientific Discovery through Advanced Computing (SciDAC) program. The motivation for industry users to use parallel libraries will only grow with time, as multi-core systems are becoming more widely available on commodity desktop and laptop systems, and it is not far off before hundred-core desktops and laptops are common. We propose to design a high-level interface to the Trilinos Tpetra parallel linear algebra library in the expressive and user-friendly Python language. This interface will make parallel linear algebra (1) easier to use via a simplified user interface, (2) more intuitive through features such as advanced indexing, and (3) more useful by enabling access to it from the already extensive Python scientific software stack. Commercial applications and other

Benefits:
The project will result in better utilization of existing HPC resources and will facilitate a decrease in the total time to market from concept to implementation, improving the competitiveness of US industries, universities and defense organizations. The benefit to the public will be observed in diverse applications that use parallel linear algebra libraries for simulation and modeling: product design and manufacturing, pharmaceuticals, thermal management, network and data transmission design, medical imaging, operational logistics, environmental project planning, and risk and performance assessment, to name some examples.

The need to access parallel computing resources is ever expanding, as both the scale of simulations and the size of data sets are quickly growing. Also, the increasing ubiquity of parallel hardwaremulti-core chips, parallel co-processors and accelerators, near effortless cluster computing on cloud infrastructurespush new and existing high-performance computing (HPC) users to move to parallel programming. Unfortunately, the gap is increasing between the available performance offered by HPC systems and the available usability of the software packages used to program them. We propose to develop a distributed HPC framework composed of three open-source software packages in the user-friendly and expressive Python language. This framework builds on the success of Phase I by (i) continuing the development of improved PyTrilinos wrappers, (ii) by implementing a general distributed array package modeled after NumPy and, (iii) by providing functionality to easily import external routines into Python, including parallel code. This Python HPC framework will make parallel computing easier to use by bringing the user-friendly NumPy interface to HPC, more intuitive by reducing the mental overhead required to work with parallel distributed arrays, and more useful by enabling access to parallel distributed solvers. PyTrilinos wrappers were expanded to wrap fundamental components of the second-generation Tpetra package, providing a basis for further expansion of PyTrilinos wrappers. The second Phase I success was that the Optimized Distributed NumPy package was designed and deemed feasible from both a technical and business perspective. A distributed HPC framework will be created to provide commonality for parallel programming in Python and allow easy access to external HPC libraries. This includes building on the research and development of Phase I by continuing the PyTrilinos expansion and implementing the distributed array package designed in Phase I, as well as a third package that easily integrates external code into Python and compiles specific routines. Commercial Applications and Other

Benefits:
Enthought will use the framework to (i) enhance its software product offerings, (ii) to provide its consulting clients with large simulation and modeling based consulting and data analysis and, (iii) to provide HPC-with-Python software training. In addition, the distributed Python HPC framework will be made freely available as open-source software, providing a basis for others on which to build commercial applications and with which to solve complex problems.