November 2006 A
High Productivity Computing Systems and the Path Towards Usable Petascale Computing
Jeffrey C. Carver, Mississippi State University
Lorin M. Hochstein, University of Nebraska, Lincoln
Richard P. Kendall, Information Sciences Institute, University of Southern California
Taiga Nakamura, University of Maryland, College Park
Marvin V. Zelkowitz, University of Maryland, College Park; Fraunhofer Center for Experimental Software Engineering
Victor R. Basili, University of Maryland, College Park; Fraunhofer Center for Experimental Software Engineering
Douglass E. Post, DoD High Performance Computing Modernization Office

2. Goals & methodologies

While the ultimate goal for the case studies of the ASC codes and the MP codes is the same (to improve the productivity of computational scientists and engineers), we have used a different approach for each type of code. The object of study in the ASC codes has been the individual programmer (e.g., “Where does the computational scientist spend her time?”), while the object of study in the MP studies has been the project (e.g. “Which factors determine project success?”). Table 2 shows a comparison of the goals of the two types of case studies.

ASC Codes MP Codes
  • Characterize which scientific programming activities are time-consuming and problematic
  • Characterize the common problems encountered by programmers
  • Characterize the impact of technologies on developer effort
  • Identify project success factors
  • Identify ways that successful projects manage risk
  • Identify productivity barriers that should be addressed by vendors
  • Develop a reference body of case studies
Table 2. Case study goals.

Table 3 provides an overview of the methodology used for each type of case study. In general, the approach for the MP codes was more comprehensive (longer questionnaire, on-site interviews, multiple subjects interviewed independently), and the approach for the ASC codes was more lightweight, which permitted quicker turnaround time when running the studies. Furthermore, each type of study collected different types of information. The focus in the ASC was lower level (i.e., more details about fewer things), while the focus in the MP codes was higher level (i.e., less details about more things).

ASC Codes MP Codes
Type Ongoing Retrospective
Interviewees Technical leads Projects leads, project staff
  1. Pre-interview questionnaire
  2. Telephone interview
  3. Generate summary document
  4. Send summary document for approval/comments
  5. Generate synthesis report across all projects
  6. Send synthesis report to all centers for approval/ comments
  1. Identify project and sponsors
  2. Negotiate case study participation
  3. Pre-interview questionnaire
  4. On-site interview
  5. Initial list of findings
  6. Follow-up
  7. Write report
  • Product: attributes, machine target, history
  • Project organization: structure, staff, configuration management
  • Development activities: adding new features, testing, tuning, debugging, porting, effort distribution, bottlenecks, achieving performance
  • Programming models and productivity: choice of model, adoption of language, productivity measures
  • Goals, requirements, deliverables
  • Project characteristics, structure, organization and risks
  • Code Characteristics
  • Staffing
  • Workflow Management
  • V&V, Testing
  • Success Measures
  • Lessons Learned
Table 3. Methodology.

Pages: 1 2 3 4 5

Reference this article
Carver, J. C., Hochstein, L. M., Kendall, R. P., Nakamura, T., Zelkowitz, M. V., Basili, V. R., Post, D. E. "Observations about Software Development for High End Computing," CTWatch Quarterly, Volume 2, Number 4A, November 2006 A. http://www.ctwatch.org/quarterly/articles/2006/11/observations-about-software-development-for-high-end-computing/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.