MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing

TitleMagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing
Publication TypeConference Paper
Year of Publication2019
AuthorsNichols, D., N-S. Tomov, F. Betancourt, S. Tomov, K. Wong, and J. Dongarra
Conference NameISC High Performance
Date Published2019-06
PublisherSpringer International Publishing
Conference LocationFrankfurt, Germany
Abstract

In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.

DOI10.1007/978-3-030-34356-9_37
Project Tags: 
External Publication Flag: