The government, or the owners or stockholders, establish how activities are to be valued by the institution management. Management valuation appears in the largely subjective U_sys variable, which includes, for example, what constitutes “success” for a computing activity. They may well value a successfully completed activity higher than the cost of the resources used. (It would be unusual for them not to do so at budget proposal time.) U_sys assumes, as managers might often wish, that all resources are available and fully allocated at all times during the life of their expensive system.⁸ We define it as a multiplier to the peak system resources in CPU units. System utilization efficiency and project level effectiveness is included in E_adm and E_proj as described below.

Once a system is delivered, the system administrators (and vendor) strive to meet management expectations for availability, A_sys and system level resource utilization efficiency (including allocation), E_adm.

More detailed descriptions of the individual variables follow and they are expanded further in the Appendix. We indicate dimensional units in brackets.

P, the productivity, is dimensionless, as required by the common economic interpretation of the productivity concept we use, output/input, [$]/[$].

C is total cost of ownership, including all costs for developing software and running it on the computer, over a defined lifetime of the system (T). This is expanded into components in the Appendix. [$]

T is the lifetime of the system as defined in the typical budget approval process. It does not show up explicitly in the top-level equation, but it is important to the definitions of the variables as well as for the measurements. It will depend on considerations specific to each environment, including, for example, whether the procurement and justification (and cost) involve continuing upgrades. Those responsible for budget submissions at the proposing institution are the source of this variable. [yrs]

E_job is the ratio of U_job to the total system cost C_sys. U_job is the total productively used resources over the lifetime by all individual jobs in all project activities, normalized to the assumption of 100% availability and 100% resource allocation. To a large extent, this efficiency measures how well programs use parallel resources. We include other costly resources (e.g., memory, bandwidth, I/O) besides CPU. In order not to favor one class of resources over another, we weight the resources by their relative costs.⁹ We say “productively used” so that we only include resources that would be consumed in direct support of maximizing utility for the specified problem. Estimators for E_job and for , the corresponding average total project personnel time (including development and production) per project, are what Job-level Productivity Benchmarks should aim to measure for different environments and systems. More detailed definitions of these can be found in.² [Dimensionless]

A_sys is the availability; the fraction of total resource time available to the jobs making up R_job. This accounts for all planned and unplanned downtimes, but does not include job restarts due to failures, which is included in E_sys below. Note this is not fraction of system time, but fraction of resource time, so that it accounts for portions of the large system being available when other portions are down. This parameter is based on vendor estimates of MTTF, maintenance requirements, and other RAS considerations. [Dimensionless]

The product E_sys≡E_admE_proj is the effectiveness of the workload resource allocation at meeting the institution's priorities. System architecture, software, and administrative tools can make a big difference to this important metric. This is where utility (or value) enters the picture and we go beyond just looking at job-level optimization. We assume that the institution will have a process for establishing the priority or the utility or the value of the individual projects and jobs. E_sys is the ratio of the actual total value of the jobs run over the lifetime to the optimal total that would come from maximally efficient management of a reference HPCS platform,¹⁰ for a given job mix, a given installed system configuration, and a given time pattern of resource downtime. As shown in [²], we allow for time dependent values, since allocation efficiency that completes a job or project long after it is needed has little or no value. We recognize explicitly (see further discussion in [²]) that not all the factors that go into E_sys are amenable, even in principle, to before or after the fact quantitative measurement. Some will need to be evaluated subjectively, and primarily have to do with the effectiveness with which projects (management assigned and valued tasks) are able to accomplish management goals and schedules. This is why we have written E_sys as a product with E_adm as the measurable part and E_proj the project effectiveness part requiring subjective evaluation. E_adm is the estimator that, along with the cost of administration, is what System-level and Administration Productivity Benchmarks should aim to measure, for different systems, configurations, and user environments. E_proj represents other considerations that result in reduced efficiency at delivering utility. [Dimensionless]

U_sys is the summed utility (or value) of all projects or activities, per unit of total available resources, R, that would complete successfully during the lifetime T if the system administrators given the systems tools and capabilities of the HPCS Reference Platform were able to attain the optimum level of resource utilization and the system was perfectly available. The values may be assigned by whatever process the institution uses to prioritize its projects and justify its budget proposals. This variable only includes the local institution's evaluation of the value of the projects it intends for the installation. In some environments, it may be ignored and set to a constant such as C_sys. We convert all resource units to CPU operations, weighting by their relative costs. [$/ops]