In this study we address the above conjectures by evaluating consult tickets and job logs and further verifying the preliminary assessments, based on the previously mentioned means, with user surveys and by personal interviews of developers using current TeraGrid Sites. More in depth explanation of the studies conducted can be found at SDSC's Performance Modeling and Characterization (PMaC) web site.3
The initial assessment, to quantify and qualify HPC productivity, was derived from evaluating the user-submitted help tickets. The ticket sampling included all help tickets submitted to the SDSC help desk from March 2004 to March 2006. These tickets span numerous architectures. The consulting tickets enabled the identification of possible HPC resources productivity bottlenecks.
Because only 307 of the 920 registered users submitted support tickets during the time span we investigated, it was clear that ticket analysis alone did not account for all users. Attempting to include a broader set of users, we looked at system job logs of the SDSC DataStar supercomputer, 4 a 2,368-processor IBM Power4 system. We evaluated high-level trends for all 59,030 jobs run on the DataStar P655 nodes at SDSC from January 2003 to April 2006. The jobs ranged in size from 1 to 128 eight-processor nodes.
To further address some of these questions raised in the previous two studies we embarked upon a survey and interviewing campaign. We developed an interview strategy based on available references 5 and examples 6to avoid common pitfalls in design. The questions included a general background and experience section, a development section, and a development practices and process section. The full interview script is available on the SDSC PMaC web site.3 This strategy was extended to create a user survey. The survey had two goals; first, to get answers from questions similar to the interview script from a larger sample size, and second, to find potential future interview subjects for further investigation. The full survey is also available on the PMaC web site. 3