Class Info

We will use C / C++ for programming. Some knowledge of C/C++ programming is necessary.

The lecture slides will be collected in the git repo

The homework assignments will be available in the git repo
(You need to request access to the repository)

Class Schedule

Topics to be covered include but are not limited to:

  • August 23: Class introduction

    Scientific Computing, High Performance Computing, State of the Art & Hardware architecture trends, Compute power, speedup, parallel efficiency

  • August 30: Sustainable software development

    Versioning systems, git, automated testing, unit testing with Googletest, continuous integration, Build systems, Makefiles

  • September 6: Parallel computing principles

    Performance, Bandwidth, Arithmetic Intensity, Machine Balance, Roofline Model, Parallel Execution, Flynn’s Taxomony, Amdahl’s law, Fork-join, Bulk Synchronous Programming, Task-based execution, Shared & distributed memory, Weak & strong scalability

  • September 13: Shared memory programming: Multicore

    Multicore architectures, OpenMP, parallel for, tasking

  • September 20: Shared memory programming: GPUs

    GPU architectures, CUDA, HIP, SYCL, GPU kernel design

  • September 27: Distributed computing

    Distributed architectures, Networks, MPI programming

  • October 4: Computational dense linear algebra

    BLAS, LU factorization, Cholesky, QR, LINPACK, LAPACK, MAGMA

  • October 11: Computational sparse linear algebra

    SpMV, SpMM, SpGeMM, Iterative solvers, Relaxation methods, Krylov solvers

  • October 18: Performance enginering

    Performance modeling, Performance profiling

  • October 25: Preconditioning

    Preconditioning principles, Left and right preconditioning, Jacobi, Gauss-Seidel, Incomplete LU, Sparse triangular solves, approximate sparse triangular solves

  • November 1: Discretization methods

    Taylor Series, Finite Differences, Finite Elements, Load balancing, Domain decomposition

  • November 8: Multigrid

    Geometric Multigrid, Algebraic Multigrid

  • November 15: Mixed precision

    Precision formats, rounding theory, mixed precision algorithms

  • November 22: No class

  • November 29: Ginkgo

    Overview of the Ginkgo math software stack

  • December 6: Project presentations

    Student presentations

  • December 13: Oral Exams

Class Grading

The success control takes place in the form of a project presentation and an oral exam of at least 30 minutes duration and evaluation of the written project report and grading of the performance in the exercises.

A total of 200 points can be achieved, that compose of

  • a maximum of 60 points for the exercise sheets (10 per exercise sheet),

  • a maximum of 60 points for the final presentation including an oral examination,

  • a maximum of 80 points for the project implementation and project report.

  • To pass the success control, at least 140 points must be collected.

Class Material

The following references are useful material for understanding the class content and deeper insight:
  • The Sourcebook of Parallel Computing, Edited by Jack Dongarra, Ian Foster, Geoffrey Fox, William Gropp, Ken Kennedy, Linda Torczon, Andy White, 2002, 760 pages, ISBN 1-55860-871-0, Morgan Kaufmann Publishers.

  • Introduction to High-Performance Computing for Scientists and Engineers, by Georg Hager, Gerhard Wellein, CRC Press, 2010.

  • Introduction to High-Performance Scientific Computing, Victor Eijkhout,