Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs

TitleDesign, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs
Publication TypeTech Report
Year of Publication2020
AuthorsBrown, C., A. Abdelfattah, S. Tomov, and J. Dongarra
Technical Report Series TitleInnovative Computing Laboratory Technical Report
Date Published2020-08
InstitutionUniversity of Tennessee
KeywordsAMD GPUs, GPU computing, HIP Runtime, HPC, numerical linear algebra, Portability

Dense linear algebra (DLA) has historically been in the vanguard of software that must be adapted first to hardware changes. This is because DLA is both critical to the accuracy and performance of so many different types of applications, and because they have proved to be outstanding vehicles for finding and implementing solutions to the problems that novel architectures pose. Therefore, in this paper we investigate the portability of the MAGMA DLA library to the latest AMD GPUs. We use auto tools to convert the CUDA code in MAGMA to the HeterogeneousComputing Interface for Portability (HIP) language. MAGMA provides LAPACK for GPUs and benchmarks for fundamental DLA routines ranging from BLAS to dense factorizations, linear systems and eigen-problem solvers. We port these routines to HIP and quantify currently achievable performance through the MAGMA benchmarks for the main workload algorithms on MI25 and MI50 AMD GPUs. Comparison with performance roofline models and theoretical expectations are used to identify current limitations and directions for future improvements.

Project Tags: 
External Publication Flag: