2023 Workshop: HPC on Heterogeneous Hardware (H3)
The HPC on Heterogeneous Hardware (H3) Workshop is intended as an in-person event in Hamburg, Germany. It does so by providing a platform for pioneering work on algorithmic research, software library design, programming models, and workflow development for increasingly heterogeneous hardware. In the workshop context, such hardware spans from ARM processors featuring long-vector extensions through GPU-accelerated systems to architectures deploying special function units, FPGAs, or deep learning processors. The workshop will compose of a well-balanced mix of invited talks, peer-reviewed conference contributions, and a panel bringing together worldwide experts in heterogeneous computing.
The DOE Report on Productive Computational Science in the Era of Extreme Heterogeneity identified 8 areas that would affected by the inevitable arrival and eminence of heterogeneity: programming environments, O/S, SysOps, productivity metrics/tools, software methodology, I/O, workflows, and modeling. These themes rang particularly true at our own Heterogeneity Panel featured recently at SC21 with hybrid online/virtual attendance of about 180 participants.
Very few companies, now including only Intel, Samsung, and TSMC, manage to mass produce CMOS device at the single-nm scale: a somewhat whimsically called Angstrom era of lithography entering more convenient unit of measurement Angstrom for atomic-scale transistor features. With Dennard Scaling long gone and Moore's Law at cross-roads, heterogeneity became the prevailing paradigm to maximally exploit the efficiency of the on-chip transistors at the 10s of Angstroms scale in what now may be considered new era of chip design.
Perhaps the most challenging aspect is to limit the workshop's scope to the very few thematic areas that currently dominate the efforts of the community. This year, these include the following topics of interest:
- Heterogeneity in programming approaches including language solutions and DSL-friendly middleware libraries.
- Heterogeneous workloads that rely on convergence of scientific modeling, data analytics, and scientific AI/ML data models
- Heterogeneity in data representation including hierarchical, randomized, compressive, and mixed-precision methods
Topics of Interest
A more specific list of topics of interest to focus the submissions and draw specific speakers and invite broad participation of attendees will be the following:
- Heterogeneous algorithms that scale not just in terms of the system size but across diverse hardware kinds.
- Heterogeneity in data approaches that incorporate mixed-precision storage and compute include data compression as well as hierarchical and randomized projections.
- Software systems and libraries that support heterogeneous compute hardware and networking.
- Programming models and tools that incorporate heterogeneity of both on-node compute and cross-node networking.
- Paper submission: March 21, 2023 (AOE)
- Notification to authors: April 4, 2023 (AOE)
- Workshop date: May 25, 2023
- Final Presentation Slides: May 26, 2023 (AOE)
- Camera-Ready Workshop with Proceedings: June 22, 2023 (AOE)
- Hartwig Anzt, University of Tennessee, USA
- Bilel Hadri, King Abdullah University of Science and Technology (KAUST), Saudi Arabia
- Hatem Ltaief, King Abdullah University of Science and Technology (KAUST), Saudi Arabia
- Piotr Luszczek, University of Tennessee, USA
- Andrey Alekseenko, KTH Royal Institute of Technology, Sweden
- Qinglei Cao, University of Tennessee, USA
- Pedro Diniz, University of Porto, Portugal
- Alfredo Goldman, São Paulo University, Brazil
- Mehdi Goli, Codeplay, UK
- Jiali Li, University of Tennessee, USA
- Neil Lindquist, University of Tennessee, USA
- Ravi Reddy Manumachu, University College Dublin, Ireland
- Max Melnichenko, University of Tennessee, USA
Format of the Workshop
H3 Workshop is initially meant as a half-day workshop to enable a selection of a good set of contributed manuscripts and talks. We would also like to ensure a programmatically balanced program and maintain a reasonable burden on the reviewers in order to provide sufficient quality of the paper reviews and informative feedback for the authors. The workshop will be held as a in-person event.
A sample schedule of the afternoon workshop to accommodate in-person attendees in Hamburg, Germany, and potential online attendees across multiple time zones will follow this general outline:
- May 25, 2023
- 14:00 - 14:10 | Introduction (Anzt, Luszczek)
- 14:10 - 14:50 | Keynote talk: Mixed-precision scientific computing with Tensor Cores on NVIDIA GPUs: Exceeding the performance characteristics of single precision while maintaining numerical accuracy by Harun Bayraktar, Director of Engineering, Math & Quantum Computing Libraries, NVIDIA [Abstract]
- 14:50 - 15:00 | Q&A session
- 15:00 - 15:20 | Talk 1: GEMM-Like Convolution for Deep Learning Inference on the Xilinx Versal by Jie Lei
- 15:20 - 15:40 | Talk 2: OpenACC unified programming environment for multi-hybrid acceleration with GPU and FPGA by Taisuke Boku
- 15:40 - 16:00 | Talk 3: Towards Quantum Acceleration of a Classical MCAE Application by Sophia Kolak
- 16:00 - 16:30 | Coffee break
- 16:30 - 16:50 | Talk 4: Observed Memory Bandwidth and Power Usage on Intel FPGA Platforms with oneAPI: A Comparison with GPUs by Chris Siefert
- 16:50 - 17:10 | Talk 5: Exploring the Use of Dataflow Architectures for Graph Neural Network Workloads by Sanjif Shanmugavelu
- 17:10 - 17:30 | Talk 6: An Investigation into the Performance and Portability of SYCL Compiler Implementations by Steven Wright
- 17:30 H3 Workshop concludes
Paper Submission and Publication
Papers should be submitted to the workshop with EasyChair. All papers must be original and not simultaneously submitted to another journal or conference. They will be reviewed and should include abstract, keywords, the e-mail address of the corresponding author, and must not exceed 12 pages, including text, tables, figures, and references at a main font size no smaller than in LNCS style. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the workshop to present the work.
Accepted papers will be published in a Springer LNCS volume (SCOPUS indexed). The format must be according to the Springer LNCS Style. Initial submissions are in PDF but the authors of accepted papers will be requested to provide source files. Extra page allotment will be provided to accommodate reviewers comments.
Any inquires should be directed to organizers.
- Mixed-precision computing leveraging Tensor Cores on GPUs can exceed numerical and performance characteristics of IEEE754 single precision for scientific computing by Harun Bayraktar
- Recent increases in computational throughput of GPUs have come from reduced and mixed-precision matrix multiplication units known as Tensor Cores primarily targeting applications in artificial intelligence. This has motivated the development of mixed-precision algorithms that leverage these capabilities while preserving a level of accuracy required by applications in science and engineering. The HPL-MxP TOP500 benchmark based on the iterative refinement method is a notable example of this. These methods, while often useful, are not universally applicable due to their numerical sensitivity and inability to guarantee convergence. The goal of this work is to address such shortcomings by developing a "drop-in" replacement for single-precision matrix multiplications and tensor contractions that leverage Tensor Cores to meet or exceed the numerical accuracy of IEEE754 based implementations while delivering significant performance benefits. In addition to explaining the mixed-precision techniques used, we demonstrate numerical accuracy claims through error analysis and two scientific applications: weather forecast and quantum computing simulations.