November 2006 B
High Productivity Computing Systems and the Path Towards Usable Petascale Computing
Declan Murphy, Sun Microsystems, Inc.
Thomas Nash, Sun Microsystems, Inc.
Lawrence Votta, Jr., Sun Microsystems, Inc.
Jeremy Kepner, MIT Lincoln Laboratory

3. Different Perspectives on Productivity

Individuals with different professional responsibilities look at productivity with naturally different perspectives. These include, for example, acquisition decision makers, project managers, individual programmers (“users”), researchers, system administrators, service engineers, operators, vendors, and system designers. It is useful to focus on two of these, project managers, and decision makers.

In principle, project manager perspectives are aligned with their institutional management, the decision makers. In practice, they differ because at least our stereotype of a project manager is only concerned with project personnel costs and not either machine operating or capital costs. And similarly the project manager can only address certain terms in the utility numerator of the productivity ratio. So, the project manager's perspective on productivity is a subset of the decision maker in Eqs.1 and 2,


The decision maker's (acquisition) productivity we developed in earlier sections is then


Here Eorg=EadmAsys is the organization's and system's multi-project resource utilization efficiency.

We can get considerably more simplification and, perhaps, also more insight, if we now think in terms of comparing the new, next generation (HPCS) system that we are evaluating to a standard reference machine. This reference can be a defined traditional MPI cluster configuration for which there is some level of understanding and experience regarding productivity in its environment. All measurements and terms in Eq. 8 can be ratios of the new system to the reference. In most organizations, for budget and institutional reasons, the ratio of project (development personnel) costs to machine and operations costs is typically a constant. In this situation we conclude that


The normalization to the reference system is indicated by bars. We assume that management evaluation of utility obtainable per effectively utilized resource is identical for the two systems (=1). The not-surprising conclusion is that we can consider the relative improvement in productivity to be just the product of productivity measurables for the new system normalized to the old.

4. Productivity Benchmarks and Operating Points

We could get an accurate figure of merit as part of a post-mortem – after the life cycle is over. At that point, we could have access to real experience. But that's not what we want; we need to predict and estimate in advance. So where do our productivity estimators come from? We have been assuming there are two classes of productivity benchmarking activities: 1) at the job- and project-level, measuring development and production activities; and, 2) at the system-level, measuring administration activities and the effect of the differences in system designs and configuration options on administration overhead and system-level resource utilization.

Development, job- and project-level, productivity benchmarks would aim to measure - for the problem mix of a specific environment - the development time required to attain different levels of productive resource utilization. The simplest example is how much programming time it takes to attain various levels of speedup. Curves of productive resource utilization vs. development and other project personnel time will increase, probably like step functions and will usually present an obvious point of diminishing returns. This can be taken as the “operating point.” Averaged over the work flow and job mix, it provides an estimator of and ECPU, Emem, EBW, and EIO.

Similarly, administration, system-level, benchmarks can aim to measure Eadm. For aspects involving effective scheduling, this could be accomplished, specific to a given environment and system configuration, by creating a simulated scheduling environment and allowing working administrators to attempt to optimize the allocation of a list of prioritized (value assigned) jobs for some simulated time period. An ideal allocation would be defined for each environment as attaining the full value of each project including time considerations. The ratio of the measured to the ideal would give an estimator for this aspect of Esys. Just as for the development benchmarks, this can be treated as a curve where better allocation, more efficient use of resources, and a general increase in output value, uses more administrator time, and a reasonable operating point of diminishing returns is selected. The cost of the administrator time at this operating point would be included as all or part of cadm (depending on how complete a benchmark simulation is used).

Pages: 1 2 3 4 5 6 7

Reference this article
"A System-wide Productivity Figure of Merit," CTWatch Quarterly, Volume 2, Number 4B, November 2006 B. http://www.ctwatch.org/quarterly/articles/2006/11/a-system-wide-productivity-figure-of-merit/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.