Individuals with different professional responsibilities look at productivity with naturally different perspectives. These include, for example, acquisition decision makers, project managers, individual programmers (“users”), researchers, system administrators, service engineers, operators, vendors, and system designers. It is useful to focus on two of these, project managers, and decision makers.

In principle, project manager perspectives are aligned with their institutional management, the decision makers. In practice, they differ because at least our stereotype of a project manager is only concerned with project personnel costs and not either machine operating or capital costs. And similarly the project manager can only address certain terms in the utility numerator of the productivity ratio. So, the project manager's perspective on productivity is a subset of the decision maker in Eqs.1 and 2,

(2)

The decision maker's (acquisition) productivity we developed in earlier sections is then

(3)

Here E_org=E_admA_sys is the organization's and system's multi-project resource utilization efficiency.

We can get considerably more simplification and, perhaps, also more insight, if we now think in terms of comparing the new, next generation (HPCS) system that we are evaluating to a standard reference machine. This reference can be a defined traditional MPI cluster configuration for which there is some level of understanding and experience regarding productivity in its environment. All measurements and terms in Eq. 8 can be ratios of the new system to the reference. In most organizations, for budget and institutional reasons, the ratio of project (development personnel) costs to machine and operations costs is typically a constant. In this situation we conclude that

(4)

The normalization to the reference system is indicated by bars. We assume that management evaluation of utility obtainable per effectively utilized resource is identical for the two systems (=1). The not-surprising conclusion is that we can consider the relative improvement in productivity to be just the product of productivity measurables for the new system normalized to the old.

We could get an accurate figure of merit as part of a post-mortem – after the life cycle is over. At that point, we could have access to real experience. But that's not what we want; we need to predict and estimate in advance. So where do our productivity estimators come from? We have been assuming there are two classes of productivity benchmarking activities: 1) at the job- and project-level, measuring development and production activities; and, 2) at the system-level, measuring administration activities and the effect of the differences in system designs and configuration options on administration overhead and system-level resource utilization.

Development, job- and project-level, productivity benchmarks would aim to measure - for the problem mix of a specific environment - the development time required to attain different levels of productive resource utilization. The simplest example is how much programming time it takes to attain various levels of speedup. Curves of productive resource utilization vs. development and other project personnel time will increase, probably like step functions and will usually present an obvious point of diminishing returns. This can be taken as the “operating point.” Averaged over the work flow and job mix, it provides an estimator of and E_CPU, E_mem, E_BW, and E_IO.

Similarly, administration, system-level, benchmarks can aim to measure E_adm. For aspects involving effective scheduling, this could be accomplished, specific to a given environment and system configuration, by creating a simulated scheduling environment and allowing working administrators to attempt to optimize the allocation of a list of prioritized (value assigned) jobs for some simulated time period. An ideal allocation would be defined for each environment as attaining the full value of each project including time considerations. The ratio of the measured to the ideal would give an estimator for this aspect of E_sys. Just as for the development benchmarks, this can be treated as a curve where better allocation, more efficient use of resources, and a general increase in output value, uses more administrator time, and a reasonable operating point of diminishing returns is selected. The cost of the administrator time at this operating point would be included as all or part of c_adm (depending on how complete a benchmark simulation is used).