SBIR-STTR Award

Representing networks and complex associations of all types, graphs are a critically important data structure within the Department of Defense. Applying graph analytics and mining to large quantities of graph data enables conducting operations, gathering intelligence, and defending the nations critical capabilities. Existing capabilities for graph mining are limited to modest data scales, batch processing, and have iterative algorithms that are complex to express in existing graph pattern languages or in data-parallel graph constructs. These algorithms require a procedural (vs declarative) definition. They form an important class of techniques that enable community detection, deep-learning, and other techniques. Apache Spark is a rapidly emerging platform for complex analytics, which uses the Scala functional language. GPUs have been demonstrated to provide 1000X acceleration graph and linear algebra problems. DASL is a Scala based DSL plugin for GPU accelerated graph analytics on Spark. DASL will be 100-1000x faster than Spark, 10,000x faster than Hadoop, and scale from laptops to clusters. DASL will reduce the barrier of use for GPUs on graphs, enable simpler and smarter algorithms, and deliver the performance of GPUs through a library of fast linear algebra primitives and scalable runtime. We will demonstrate DASL on Cyber domain problems.

Representing networks and complex associations, graphs are an important data structure within the Department of Defense. Applying graph analytics and mining to large data sets enables conducting operations, gathering intelligence, and defending the nations critical capabilities. Graphics Processing Units (GPUs) are demonstrated to provide 10-1000X acceleration for graphs. Building analytics for GPUs is challenging due to the complexities of parallel programming and specialized languages. Phase I developed a Scala-based domain-specific language for graphs called DASL (dazzle) using Linear Algebra and a runtime to execute DASL on GPUs. On 140 million Netflows, DASLs performance was comparable to current state of the art low-level graph frameworks, with code 3,000% less complex. DASL provided the best complexity to performance ratio, from 400% to 3000% faster, and delivered over 163,000 traversed edges per second (TEPS) per line of algorithmic code. Analytic executions were 10-100X faster than non-GPU approaches. Phase II will develop DASL focusing on increased scale and performance optimization. The research will be augmented with techniques for rapid data integration and visualization. The results will achieve 10-100X acceleration of graph analytic workflows from data to visualization. Commercial applications include cyber defense, life sciences, precision medicine, financial compliance, and social networking/e-commerce.Representing networks and complex associations, graphs are an important data structure within the Department of Defense. Applying graph analytics and mining to large data sets enables conducting operations, gathering intelligence, and defending the nations critical capabilities. Graphics Processing Units (GPUs) are demonstrated to provide 10-1000X acceleration for graphs. Building analytics for GPUs is challenging due to the complexities of parallel programming and specialized languages. Phase I developed a Scala-based domain-specific language for graphs called DASL (dazzle) using Linear Algebra and a runtime to execute DASL on GPUs. On 140 million Netflows, DASLs performance was comparable to current state of the art low-level graph frameworks, with code 3,000% less complex. DASL provided the best complexity to performance ratio, from 400% to 3000% faster, and delivered over 163,000 traversed edges per second (TEPS) per line of algorithmic code. Analytic executions were 10-100X faster than non-GPU approaches. Phase II will develop DASL focusing on increased scale and performance optimization. The research will be augmented with techniques for rapid data integration and visualization. The results will achieve 10-100X acceleration of graph analytic workflows from data to visualization. Commercial applications include cyber defense, life sciences, precision medicine, financial compliance, and social networking/e-commerce.