When to checkpoint at the end of a fixed-length reservation?

TitleWhen to checkpoint at the end of a fixed-length reservation?
Publication TypeConference Paper
Year of Publication2023
AuthorsBarbut, Q., A. Benoit, T. Herault, Y. Robert, and F. Vivien
Conference NameFault Tolerance for HPC at eXtreme Scales (FTXS) Workshop
Date Published2023-08
Conference LocationDenver, United States
Other Numbershal-04215554f

This work considers an application executing for a fixed duration, namely the length of the reservation that it has been granted. The checkpoint duration is a stochastic random variable that obeys some well-known probability distribution law. The question is when to take a checkpoint towards the end of the execution, so that the expectation of the work done is maximized. We address two scenarios. In the first scenario, a checkpoint can be taken at any time; despite its simplicity, this natural problem has not been considered yet (to the best of our knowledge). We provide the optimal solution for a variety of probability distribution laws modeling checkpoint duration. The second scenario is more involved: the application is a linear workflow consisting of a chain of tasks with IID stochastic execution times, and a checkpoint can be taken only at the end of a task. First, we introduce a static strategy where we compute the optimal number of tasks before the application checkpoints at the beginning of the execution. Then, we design a dynamic strategy that decides whether to checkpoint or to continue executing at the end of each task. We instantiate this second scenario with several examples of probability distribution laws for task durations.

External Publication Flag: