This project is developing PAPI, which will provide tool designers and application engineers with a consistent interface and methodology for the use of low-level performance counter hardware found across the entire compute system (i.e. CPUs, GPUs, on/off-chip memory, interconnects, I/O system, energy/power, etc.). PAPI will enable users to see, in near real time, the relations between software performance and hardware events across the entire computer system.

Exa-PAPI builds on the latest PAPI project and will be extended with:

  1. Performance counter monitoring capabilities for new and advanced ECP hardware, and software technologies.
  2. Fine-grained power management support.
  3. Functionality for performance counter analysis at "task granularity" for task-based runtime systems.
  4. "Software-defined Events" that originate from the ECP software stack and are currently treated as black boxes (i.e., communication libraries, math libraries, task-based runtime systems, etc.)

The objective is to enable monitoring of both types of performance events—hardware- and software-related events—in a uniform way, through one consistent PAPI interface. Third-party tools and application developers will have to handle only a single hook to PAPI in order to access all hardware performance counters in a system, including the new software-defined events.

2018 ECP Meeting Poster
2018 Poster
Sponsored By
Exascale Computing Project
National Nuclear Security Administration
The United States Department of Energy


Haidar, A., H. Jagode, P. Vaccaro, S. Tomov, and J. Dongarra, Investigating Power Capping toward Energy-Efficient Scientific Applications,” Concurrency and Computation: Practice and Experience (CCPE): Special Issue on Power-Aware Computing 2017, Submitted.
Haidar, A., H. Jagode, A. YarKhan, P. Vaccaro, S. Tomov, and J. Dongarra, Power-aware Computing: Measurement, Control, and Performance Analysis for Intel Xeon Phi,” 2017 IEEE High Performance Extreme Computing Conference (HPEC'17), Best Paper Finalist, Waltham, MA, IEEE, September 2017.  (908.84 KB)


  • Presented at the 2017 IEEE High Performance Extreme Computing Conference (HPEC '17) (September 12-14, 2017) in Waltham, MA, USA: Haidar, A., Jagode, H., YarKhan, A., Vaccaro, P., Tomov, S., Dongarra, J. "Power-aware Computing: Measurement, Control, and Performance Analysis for Intel Xeon Phi" (Best Paper Finalist)
  • Presented at the 11th Parallel Tools Workshop (September 11-12, 2017) in Dresden, Germany: Anthony Danalis, Heike Jagode, Hanumanth Hanumantharayappa, Jack Dongarra. "Counter Inspection Toolkit: Making Sense out of Hardware Performance Events"
  • Presented an update on the NVIDIA support in PAPI update at the CORAL Tools working group meeting on June 15, 2017.
  • Presented at the VI-HPS Tools Workshop (June 23, 2017): "EXA-PAPI: The exascale performance application programming interface"

ICL Team Members

Anthony Danalis

Jack Dongarra

John Henry

Heike Jagode

Frank Winkler

Exascale Computing Project

Exa-PAPI is part of ICL's involvment in the Exascale Computing Project (ECP). The ECP was established with the goals of maximizing the benefits of high-performance computing (HPC) for the United States and accelerating the development of a capable exascale computing ecosystem. Exascale refers to computing systems at least 50 times faster than the nation’s most powerful supercomputers in use today.

The ECP is a collaborative effort of two U.S. Department of Energy organizations – the Office of Science (DOE-SC) and the National Nuclear Security Administration (NNSA).