This project is developing PAPI, which will provide tool designers and application engineers with a consistent interface and methodology for the use of low-level performance counter hardware found across the entire compute system (i.e. CPUs, GPUs, on/off-chip memory, interconnects, I/O system, energy/power, etc.). PAPI will enable users to see, in near real time, the relations between software performance and hardware events across the entire computer system.

Exa-PAPI builds on the latest PAPI project and will be extended with:

  1. Performance counter monitoring capabilities for new and advanced ECP hardware, and software technologies.
  2. Fine-grained power management support.
  3. Functionality for performance counter analysis at "task granularity" for task-based runtime systems.
  4. "Software-defined Events" that originate from the ECP software stack and are currently treated as black boxes (i.e., communication libraries, math libraries, task-based runtime systems, etc.)

The objective is to enable monitoring of both types of performance events—hardware- and software-related events—in a uniform way, through one consistent PAPI interface. Third-party tools and application developers will have to handle only a single hook to PAPI in order to access all hardware performance counters in a system, including the new software-defined events.

2019 ECP Meeting Poster
2019 Poster


Looking for beta-testers for PAPI's latest PCP-component

Exa-PAPI offers a new PAPI component, called "pcp", which interfaces to the Performance Co-Pilot (PCP). It enables PAPI users to monitor IBM Power9 hardware performance events, particularly shared "NEST" events without root access. We are looking for beta-testers and feedback of the new PAPI PCP component.

For access to the code, follow these steps:

		git clone https://bitbucket.org/icl/papi.git
cd papi/src
./configure --with-components=pcp
make && make install

Please email your questions and comments to [javascript protected email address].


Haidar, A., H. Jagode, P. Vaccaro, A. YarKhan, S. Tomov, and J. Dongarra, Investigating Power Capping toward Energy-Efficient Scientific Applications,” Concurrency Computation: Practice and Experience, vol. 2018, issue e4485, pp. 1–14, April 2018.  (1.2 MB)
Haidar, A., H. Jagode, A. YarKhan, P. Vaccaro, S. Tomov, and J. Dongarra, Power-aware Computing: Measurement, Control, and Performance Analysis for Intel Xeon Phi,” 2017 IEEE High Performance Extreme Computing Conference (HPEC'17), Best Paper Finalist, Waltham, MA, IEEE, September 2017.  (908.84 KB)


  • Presented at the 2017 IEEE High Performance Extreme Computing Conference (HPEC '17) (September 12-14, 2017) in Waltham, MA, USA: Haidar, A., Jagode, H., YarKhan, A., Vaccaro, P., Tomov, S., Dongarra, J. "Power-aware Computing: Measurement, Control, and Performance Analysis for Intel Xeon Phi" (Best Paper Finalist)
  • Presented at the 11th Parallel Tools Workshop (September 11-12, 2017) in Dresden, Germany: Anthony Danalis, Heike Jagode, Hanumanth Hanumantharayappa, Jack Dongarra. "Counter Inspection Toolkit: Making Sense out of Hardware Performance Events"
  • Presented an update on the NVIDIA support in PAPI update at the CORAL Tools working group meeting on June 15, 2017.
  • Presented at the VI-HPS Tools Workshop (June 23, 2017): "EXA-PAPI: The exascale performance application programming interface"

ICL Team Members

Rizwan Ashraf

Tony Castaldo

Anthony Danalis

Jack Dongarra

Heike Jagode

Frank Winkler

Exascale Computing Project

Exa-PAPI is part of ICL's involvment in the Exascale Computing Project (ECP). The ECP was established with the goals of maximizing the benefits of high-performance computing (HPC) for the United States and accelerating the development of a capable exascale computing ecosystem. Exascale refers to computing systems at least 50 times faster than the nation’s most powerful supercomputers in use today.

The ECP is a collaborative effort of two U.S. Department of Energy organizations – the Office of Science (DOE-SC) and the National Nuclear Security Administration (NNSA).

Sponsored By
Exascale Computing Project
National Nuclear Security Administration
The United States Department of Energy