Class Info


We will use C / C++ for programming. Some knowledge of C/C++ programming is necessary.

The lecture slides will be collected in the git repo https://gitlab.com/cosc594/lecture_slides.git.

The homework assignments will be available in the git repo https://gitlab.com/cosc594/exercises.git.
(You need to request access to the repository)

Class Schedule


Topics to be covered include but are not limited to:

  • August 23: Class introduction

    Scientific Computing, High Performance Computing, State of the Art & Hardware architecture trends, Compute power, speedup, parallel efficiency

  • August 30: Sustainable software development

    Versioning systems, git, automated testing, unit testing with Googletest, continuous integration, Build systems, Makefiles

  • September 6: Parallel computing principles

    Performance, Bandwidth, Arithmetic Intensity, Machine Balance, Roofline Model, Parallel Execution, Flynn’s Taxomony, Amdahl’s law, Fork-join, Bulk Synchronous Programming, Task-based execution, Shared & distributed memory, Weak & strong scalability

  • September 13: Shared memory programming: Multicore

    Multicore architectures, OpenMP, parallel for, tasking

  • September 20: Shared memory programming: GPUs

    GPU architectures, CUDA, HIP, SYCL, GPU kernel design

  • September 27: Distributed computing

    Distributed architectures, Networks, MPI programming

  • October 4: Computational dense linear algebra

    BLAS, LU factorization, Cholesky, QR, LINPACK, LAPACK, MAGMA

  • October 11: Computational sparse linear algebra

    SpMV, SpMM, SpGeMM, Iterative solvers, Relaxation methods, Krylov solvers

  • October 18: Performance enginering

    Performance modeling, Performance profiling

  • October 25: Preconditioning

    Preconditioning principles, Left and right preconditioning, Jacobi, Gauss-Seidel, Incomplete LU, Sparse triangular solves, approximate sparse triangular solves

  • November 1: Discretization methods

    Taylor Series, Finite Differences, Finite Elements, Load balancing, Domain decomposition

  • November 8: Multigrid

    Geometric Multigrid, Algebraic Multigrid

  • November 15: Mixed precision

    Precision formats, rounding theory, mixed precision algorithms

  • November 22: No class

  • November 29: Ginkgo

    Overview of the Ginkgo math software stack

  • December 6: Project presentations

    Student presentations

  • December 13: Oral Exams

Class Grading

The success control takes place in the form of a project presentation and an oral exam of at least 30 minutes duration and evaluation of the written project report and grading of the performance in the exercises.


A total of 200 points can be achieved, that compose of

  • a maximum of 60 points for the exercise sheets (10 per exercise sheet),

  • a maximum of 60 points for the final presentation including an oral examination,

  • a maximum of 80 points for the project implementation and project report.

    • To pass the success control, at least 140 points must be collected.

Class Material

The following references are useful material for understanding the class content and deeper insight:

  • The Sourcebook of Parallel Computing, Edited by Jack Dongarra, Ian Foster, Geoffrey Fox, William Gropp, Ken Kennedy, Linda Torczon, Andy White, 2002, 760 pages, ISBN 1-55860-871-0, Morgan Kaufmann Publishers.

  • Introduction to High-Performance Computing for Scientists and Engineers, by Georg Hager, Gerhard Wellein, CRC Press, 2010.

  • Introduction to High-Performance Scientific Computing, Victor Eijkhout, http://theartofhpc.com/