Workshop Scope

The standardization of an interface for dense linear algebra operations in the BLAS standard has enabled interoperability between different linear algebra libraries, thereby boosting the success of scientific computing, in particular in scientific HPC. In particular, from this point on, applications could interface highly optimized implementations through a standardized API. Despite numerous efforts in the past, the community has not yet agreed on a standardization for sparse linear algebra operations due to numerous reasons. One is the fact that sparse linear algebra objects allow for many different storage formats, and different hardware may favor different storage formats. This makes the definition of a FORTRAN-style all-circumventing interface extremely challenging. Another reason is that opposed to dense linear algebra functionality, in sparse linear algebra, the size of the sparse data structure for the operation result is not always known prior to the information. Furthermore, as opposed to the standardization effort for dense linear algebra, we are late in the technology readiness cycle, and many production-ready software libraries using sparse linear algebra routines have implemented and committed to their own sparse BLAS interface. At the same time, there exists a demand for standardization that would improve interoperability, and sustainability, and allow for easier integration of building blocks. In an inclusive, cross-institutional effort involving numerous academic institutions, US National Labs, and industry, we spent two years designing a hardware-portable interface for basic sparse linear algebra functionality that serves the user needs and is compatible with the different interfaces currently used by different vendors. Defining an interface is an iterative process, and in this third edition of the workshop on defining an interface for sparse linear algebra functionality, we focus on the experience and feedback from the community in terms of adopting the interface we proposed in November 2024. This includes discussing how the set of functionality needs to be adopted or expanded to meet user needs. We also plan a hackathon-style coding session focusing on the wrapping of existing libraries into the proposed interface.


Topics to be covered include but are not limited to:

  • Feedback and experience with the proposed sparse BLAS standard

  • Experiences wrapping existing functionality into the proposed interface

  • Extensions of the scope in terms of functionality and programming languages

  • Algorithm implementation discussions


Defining an API standard for sparse linear algebra functionality is a community effort. If you are interested in participating in the discussions and the workshop, please reach out to hanzt@icl.utk.edu.

Position Paper

Interface for Sparse Linear Algebra Operations


@misc{abdelfattah2024interfacesparselinearalgebra,

title={Interface for Sparse Linear Algebra Operations},

author={Ahmad Abdelfattah and Willow Ahrens and Hartwig Anzt and Chris Armstrong and Ben Brock and Aydin Buluc and Federico Busato and Terry Cojean and Tim Davis and Jim Demmel and Grace Dinh and David Gardener and Jan Fiala and Mark Gates and Azzam Haider and Toshiyuki Imamura and Pedro Valero Lara and Jose Moreira and Sherry Li and Piotr Luszczek and Max Melichenko and Jose Moeira and Yvan Mokwinski and Riley Murray and Spencer Patty and Slaven Peles and Tobias Ribizel and Jason Riedy and Siva Rajamanickam and Piyush Sao and Manu Shantharam and Keita Teranishi and Stan Tomov and Yu-Hsiang Tsai and Heiko Weichelt},

year={2024},

eprint={2411.13259},

archivePrefix={arXiv},

primaryClass={cs.MS},

url={https://arxiv.org/abs/2411.13259},

}

Reference Implementation

Workshop Agenda

This talk is about dense BLAS. The main goal of the talk is to steer discussion between Dense BLAS and Sparse BLAS community. Thanks for inviting me. BLAS stands for “Basic Linear Algebra Subroutines” and it represents a set of standardized interfaces to access high performance linear algebra functionalities. An archetypical example is matrix-matrix multiplication. Workflow is as follows. At the foundation, there is an agreed-upon interface, a reference implementation and an associated test suite. Then teams (Nvidia, AMD, Intel, ARM, university labs, etc.) specialize in developing and providing highly optimized implementations of these interfaces for given computing architectures. Then the world can benefit from it. There is a need for restarting conversation with vendors (Nvidia, AMD, Intel, etc.) and users so as to develop new interfaces that reflect the evolving landscapes. One push is AI and all the new low precision arithmetics that are instantiated, (8-bit, 16-bit arithmetics, etc.), another push is new functionalities that have emerged and proven useful in many kernels, they ought to be integrated in the standard so as to be converted to high performance implementations. We can also mention batched BLAS. BLAS is a critical piece of infrastructure for many workflows. This creates some inertia and resistance to change. Indeed BLAS did not evolve for 35 years (between 1990 and 2025). In 2025 and 2026, a few BLAS functionalities were added in ad-hoc ways to the BLAS. In the near future, we would like to move forward in a more consensus, community-based fashion. This talk will present the need for new BLAS routines, challenges, potential designs and some ideas to move us forward.



Monday, March 23th
8:00 - 10:30 Arrival & Breakfast
10:30 - 11:00 Workshop Opening (Hartwig Anzt): State of the effort and next steps
11:00 - 11:30 Discussions: What do we need to prioritize?
12:00 - 13:30 Lunch and Walk over the Campus
13:30 - 15:00 Progress of the Reference Implementation Focus Group
Spencer Patty (Intel)
15:00 - 15:30 Coffee break
15:30 - 17:00 Hackathon
Tuesday, March 24th
8:00 - 9:00 Breakfast
9:00 - 9:10 Tuesday Opening: Hartwig Anzt
9:10 - 9:40 Tuesday Keynote: TBD TBD
9:40 - 10:00 Coffee break
10:00 - 12:00 Discussions - breakout groups
12:00 - 13:30 Workshop photo and Lunch
13:30 - 15:00 Hackathon
15:00 - 15:30 Coffee break
15:30 - 17:30 Hackathon
18:00 - 21:00 Informal workshop evening event
Wednesday, March 25th
8:00 - 9:00 Breakfast
9:00 - 9:10 Wednesday Opening: Hartwig Anzt
9:10 - 9:40 Building momentum for a community-based (dense) BLAS evolution
Julien Langou (UC Denver)
9:40 - 10:00 Coffee break
10:00 - 12:00 Discussions - breakout groups
12:00 - 13:30 Lunch
13:30 - 15:00 Hackathon
15:00 - 15:30 Coffee break
15:30 - 17:00 Hackathon
18:00 - 21:00 Workshop dinner at Vida Knoxville
Chartreuse Room, located at 531 S Gay St.
Thursday, March 26th
8:00 - 9:00 Breakfast
9:00 - 9:10 Thursday Opening: Hartwig Anzt
9:10 - 12:00 Plenary discussions and paper finalization
12:00 - 13:30 Farewell Lunch
13:30 - 17:00 Open End Hackathon

Logistics

The workshop will take place at the University of Tennessee, Knoxville, 1122 Volunteer Blvd., Claxton Building


Map and directions to Claxton Building: ICL Location

You should be able to walk from Knoxville Downtown (hotel area) to the Claxton building. The distance is about 1 mile.

There is also a free trolley to campus. Take the Orange Line from the corner of Locust and Clinch to the Hodges Library stop. Claxton is the building with a courtyard; enter through the large glass doors next to the courtyard, and take the first left in the building. Orange Line runs Mon-Fri 7am-8pm (everyday: every 15 mins)

Remote participants can attend the workshop under https://bit.ly/zoom_anzt

Contact Information

Hartwig Anzt

hanzt@icl.utk.edu

Innovative Computing Lab
University of Tennessee
USA