Class Info

We will use C / C++ for programming. Some knowledge of C/C++ programming is necessary.

The lecture slides will be collected in the git repo

The homework assignments will be available in the git repo
(You need to request access to the repository)

Class Schedule

Topics to be covered include but are not limited to:

  • August 23: Class introduction

    Scientific Computing, High Performance Computing, State of the Art & Hardware architecture trends, Compute power, speedup, parallel efficiency

  • August 30: Parallel computing principles

    Performance, Bandwidth, Arithmetic Intensity, Machine Balance, Roofline Model, Parallel Execution, Flynn’s Taxomony, Amdahl’s law, Fork-join, Bulk Synchronous Programming, Task-based execution, Shared & distributed memory, Weak & strong scalability

  • September 6: Sustainable software development

    Versioning systems, git, automated testing, unit testing with Googletest, continuous integration, Build systems, Makefiles

  • September 13: Shared memory programming: Multicore

    Multicore architectures, OpenMP, parallel for, tasking

  • September 20: Shared memory programming: GPUs

    GPU architectures, CUDA, HIP, SYCL, GPU kernel design

  • September 27: Computational dense linear algebra

    BLAS, LU factorization, Cholesky, QR, LINPACK, LAPACK, MAGMA

  • October 4: Distributed computing (Bosilca)

    Distributed architectures, Networks, MPI programming

  • October 11: Performance modeling (Danalis)

    Performance modeling, Performance evaluation

  • October 18: Computational sparse linear algebra

    SpMV, SpMM, SpGeMM, Iterative solvers, Relaxation methods, Krylov solvers

  • October 25: Preconditioning

    Preconditioning principles, Left and right preconditioning, Jacobi, Gauss-Seidel, Incomplete LU, Sparse triangular solves, approximate sparse triangular solves

  • November 1: Discretization methods & Multigrid Methods

    Taylor Series, Finite Differences, Geometric Multigrid, Algebraic Multigrid

  • November 8: Resilience in HPC (Boutellier)

  • November 15: Projection and its Importance in Scientific Computing (Tomov)

  • November 22: No class

  • November 29: Machine Learning (Luszczek)

    SVM, Neural Networks, Training and Classification

  • December 6: Project presentations

    Student presentations

  • December 13: Oral Exams

Class Grading

The success control takes place in the form of a project presentation and an oral exam of at least 30 minutes duration and evaluation of the written project report and grading of the performance in the exercises.

A total of 200 points can be achieved, that compose of

  • a maximum of 60 points for the exercise sheets (10 per exercise sheet),

  • a maximum of 60 points for the final presentation including an oral examination,

  • a maximum of 80 points for the project implementation and project report.

  • To pass the success control, at least 140 points must be collected.

Class Material

The following references are useful material for understanding the class content and deeper insight:
  • The Sourcebook of Parallel Computing, Edited by Jack Dongarra, Ian Foster, Geoffrey Fox, William Gropp, Ken Kennedy, Linda Torczon, Andy White, 2002, 760 pages, ISBN 1-55860-871-0, Morgan Kaufmann Publishers.

  • Introduction to High-Performance Computing for Scientists and Engineers, by Georg Hager, Gerhard Wellein, CRC Press, 2010.

  • Introduction to High-Performance Scientific Computing, Victor Eijkhout,