Computational scientists and engineers face many challenges when writing codes for high-end computing (HEC) systems. With the movement towards peta-scale machines and the increasing focus on improving development productivity, writing a good code is even more challenging. The DARPA High Productivity Computing Systems (HPCS)1 project is developing new machine architectures, programming languages, and software tools to improve the productivity of scientists and engineers 2. Although the existence of these new technologies is important for improving productivity, they will not achieve their stated goals if individual scientists and engineers are not able to effectively use them to solve their problems. A necessary first step in determining the usefulness of new architectures, languages and tools is to gain a better understanding of what the scientists and engineers do, how they do it, and what problems they face in the current HEC development environment. Because the HEC community is very diverse 3, it is necessary to sample different application domains to be able to draw any meaningful conclusions about the commonalties and trends in software development in this community.
This paper discusses some of the similarities and differences found in ten projects taken from different domains as understood by researchers in the HPCS project. We have undertaken a series of case studies to gain deeper insight into the nature of software development for scientific and engineering software. These studies have focused on two different types of projects, (characterized in Table 1):
- ASC-Alliance projects, DOE-sponsored computational science centers based at University of Illinois Urbana-Champaign, California Institute of Technology, University of Utah, Stanford University, and University of Chicago. In the rest of this paper, the codes are referred to as ASC Codes.
- Codes from DARPA HPCS mission partners (organizations that have a vested interest in the outcome of and often financial sponsorship of the project), some of which are classified and would therefore be inaccessible to other researchers. Due to the sensitive nature of many of these projects, they must remain anonymous in this paper. In the rest of this paper, these codes are referred to as MP Codes.
ASC Codes | MP Codes | |
# of projects | 5 | 5 |
Environment | Academia (ASC-Alliance projects) | Mission Partners (DoD, DOE, NASA) |
Classified | No | Some |
Code size | 200-600 KLOC | 80-760 KLOC |
Type | Coupled multi-physics applications | Single physics to coupled multi-physics and engineering |
While these case studies are still ongoing, the results to date allow for some cross-project analysis to provide a deeper understanding of the state of the practice 4 5 6 7. In this paper, we discuss some observations gained from this analysis.