To some people it seems premature to talk of the next generation of the Grid when in many cases the Grid has yet to deliver according to its original vision. Grid research has come a long way since it was originally mooted–in terms analogous to the electric power grid–as an infrastructure that was always-on and delivered chargeable access to compute, data and other resources when and wherever they were required. Pioneering projects, largely science-based, in Europe, the US and Asia have demonstrated the positive benefits afforded by large-scale, widely distributed computation and data access and such projects are now undertaking previously impracticable scientific research. This is particularly true in the health sector where some large cancer research projects are now gathering speed and will hopefully afford real benefits and breakthroughs across society.

However, although the Grid can be said to be delivering in a scientific context, the same is not true in the business domain. Visit any investment bank in the United Kingdom and they will (privately) talk proudly of the success of their Grid. In reality, they are actually talking about the success of their clustered computing approach. There are two main reasons for this. Firstly, the hijacking of the “Grid” word by over-eager vendor marketing departments following the dot com bubble in the early part of this decade has confused may potential users about what the Grid is really for–inter-enterprise, joined-up computing. Secondly, and more importantly, the Grid used and promulgated by the science and research communities does not take into account the typical regulatory and management issues faced by many industries. Unless the Grid can be seen to offer real benefits to business it will remain a powerful tool for science and will be largely ignored by business, except in its simplest application server and clustered computing form. In the worst case we will see a complete divergence in Grid computing between science and business.

It doesn’t have to be like this.

It is very easy to complain that the Grid to date has failed to link its developments to the real needs of its users. In the scientific domain this simply is not true. Wide-ranging requirements-gathering activities have taken place and will continue. These activities have helped to guide the development of the tools most needed by these programmes of scientific research. In most cases these are programmes of research where a specific end-point is reasonably clear, and the main motivation for using Grids is to collaborate in order to pool resources. In the business domain, the requirements that Grids have to meet are far broader and more varied. A wide variety of projects, notably in the UK and Europe, have been undertaken, and there have been many notable demonstrations of the efficacy of Grids both in the cluster and broader Grid contexts. However, these projects did not produce universal solutions spanning many business applications, because different solutions were required in each case. For example, the GRASP project used “traditional” academic Grid principles to support resource sharing within a cooperative of application service providers, providing higher performance and reliability for ASP services, but requiring mutual trust between the providers. The GRIA project implemented an inter-enterprise collaboration infrastructure allowing the users to pool resources obtained on commercial terms from independent service providers. The GEMSS project took a similar approach for medical simulation services, but resources from different service providers cannot be pooled in a single application because that would make it very difficult to meet European privacy regulations for processing patient data.

All of these commercial Grid prototypes support the application requirements typically found in academic Grids, but they all had to be specialised in some way to meet specific business requirements in the sector or scenarios they were designed to support. There remains a lack of consensus on what constitutes a usable business Grid, and without this, the impact of Grids on business will remain very limited. To overcome these barriers, it is not enough to simply implement solutions that meet the business requirements of individual users. The World Wide Web was not delivered by studying the requirements of business – but it continues to be truly transformational in the way all of us live and work. The next generation of Grid must therefore be truly transformational. It must go well beyond the stated requirements of science and business and it must also be prepared to challenge current orthodoxies. Those of us who believe in the Grid understand that, as it delivers over the next decade, we will begin to see the true value of this computing revolution.

Getting from where we are now to where we want to be will not be easy. Neither is it easy to visualise that end point clearly. We just know it will be there. In some business sectors, enormous resistance to change has built up and the Grid is perceived to have failed to deliver or it has delivered in a very constrained way – such as cluster computing. For instance, a common refrain in the financial services sector is “we’ll never deploy wide-area Grids – they’re alright for scientists but what about our regulatory and security requirements?” The purpose of the NextGRID Project is to challenge some of these attitudes and to undertake the necessary thinking to make the Grid truly able to deliver.

NextGRID,¹ funded by the European Union’s Information Society & Media directorate, is a three year, €16.5million research project with partners drawn from across European academia and business. With 13 of the 22 partners coming from the business domain, including SAP, Microsoft, Fujitsu, BT and NEC, the project has a strong focus on tackling the “Grid for business” agenda. The project is currently the only project worldwide that is specifically focused on driving the architecture of the Grid forward.

Information and communication technologies are recognised as having a key role in Europe’s transformation into a dynamic, competitive knowledge-based economy. Sustained success is increasingly reliant on flexibility in business processes, which allows businesses to adapt to a changing global environment. IT applications and services are an essential enabler for this flexibility. Largely to meet this need there is an ongoing clear shift in the market towards a service-oriented approach to IT systems. This is allowing consumers to obtain a wide range of services as required from a range of providers, delivered via a ubiquitous telecommunications infrastructure. The emergence of this infrastructure has allowed users to enjoy permanent global connectivity from a range of wired and wireless devices without needing to be concerned with the technologies and networks involved.

The Grid has the potential to make a significant advance beyond the World Wide Web, by turning it from a passive information medium into an active tool for creating and exploring new knowledge and thereby fuelling business and industry. Today, as discussed above, this potential is unrealised and, without far more cost-effective and universally applicable technology, will remain so. A crucial missing element is the ability to compose services from independent sources in a standardised and cost-effective way. To go beyond current business use of the Grid, applications should be capable of executing on an inter-enterprise Grid infrastructure.

Current Grid systems do not address this service composition challenge–they impose business models on users and application developers, usually based on the “traditional” virtual organisation model for collaboration between mutually trusting parties. Until the Grid can support a wide range of dynamically evolving business models, while maintaining stability as seen by each stakeholder, it is hard to see how the Grid can support third-party application development, which is one of the key drivers behind the success of non-Grid computing platforms.

The separate interests of independent stakeholders cannot be resolved a priori as is the case for non-Grid applications designed to execute in a single domain. This implies that a Grid infrastructure must be capable of combining the different business models used by different stakeholders at run time, so the Grid presents a stable interface to each stakeholder. This is of course analogous to the World Wide Web today where a multitude of Web servers and Web browsers (mostly) happily coexist with each other. Furthermore, commercial business models are essential for the Grid’s long-term viability.

The broad NextGRID vision is of a networked IT infrastructure to support an unlimited range of applications and business processes throughout their lifecycle. This includes all resources– hardware, software, data and services, available from a complex ecosystem of providers. The primary goal of NextGRID is to define the architecture that will lead to the emergence of the Next Generation Grid. This will prepare the way for the mainstream use of Grid technologies and their widespread adoption by organisations and individuals from across the business and public domains. In addition to new architectural designs, NextGRID will contribute to the key middleware components, application support mechanisms, know-how and standards that underpin the Next Generation Grid.

Of course, NextGRID cannot address these objectives alone. The participants in NextGRID are the representatives of a much larger community of researchers, technology vendors, service providers and users. We inspire and work with this wider community, providing critical input and thought leadership to the development of the architecture for future Grids, incorporating our results into widely accepted standards, and so encompassing a much larger body of work within our own organisations and in the community at large. We also understand that parts of our work will be incremental and parts revolutionary.

The project structure is built around the architectural design process. This process is informed by the development work, business and operational activities and application experimentation. At the end of each six-month design cycle, the results are fed into the development activities, which focus on Grid foundations, dynamics and interactions. The consolidated outputs of the project are exploited up by its standardisation activity and the business partners in the project. The work of the project is very broad and this article is too short to relate all of it here. The project has now completed two architecture cycles and the broad thrust of our activities has been defined. Rather than detailing our many activities there is more value to be gained by focusing on one particularly important innovation–the idea that Grids can be built up from pairwise inter-enterprise relationships governed by Service Level Agreements (SLAs) that capture the mutual interests of each pair of participants.

Our initial architectural designs assume that applications will be constructed by composing services, each of which has some common properties and behaviours. When executing applications, we can assume that certain core “infrastructure” services or properties are available in the environment of the application. In the context of NextGRID we are building on the HTTP(S), SOAP, WSDL1.1, WS-Addressing, WS-Security, SAML1.1 and X.509 protocols and the OGSA WSRF Basic Profile 1.0 and SLA Template interfaces. A key architectural requirement is that service composition should result in self-similar structures that are themselves amenable to NextGRID composition rules.

A key aspect of the current NextGRID conceptual architecture is that all interactions will be governed through bipartite “partnership” SLAs. NextGRID believes that SLAs should be used to build relationships between service providers and consumers, and provide the necessary information to set up the environment and components to manage the service. The SLA should outline details that are agreed by both parties, and allow for the service to be operated and monitored in accordance with the consumer requirements and in an economically sustainable manner.

Neither the service provider nor the consumer will gain a significant advantage by violating an SLA. The customer will not get the service they require, and the provider’s reputation will be damaged –perhaps to the stage where the customer will not use the service again. It is therefore proposed to have a framework that is less focused on monitoring of every element of every transaction and see the relationship between provider and consumer as a partnership within a context – with that context being provided by the SLA. Where necessary, monitoring and enforcement are provided for, but the aim of the SLA idea is to focus on the partnership and agreement side rather than the violation side of the contract.

The benefit of such a proposal is that the NextGRID architecture can support both community minded approaches as well as the commercial offering of services. SLAs allow for services to be provided in exchange for an equivalent set of services or a cash purchase. In a commercial context where services are provided for a fee, and the fact the service is provided on a Grid infrastructure is irrelevant to the end user, it is more important to provide specific QoS levels that need to be communicated, agreed upon and upheld.

Approaching the issue of SLAs in this partnership model allows for a lighter-weight monitoring infrastructure and avoids having a monitoring system that is more expensive to provide in economic, computational or time terms than some of the services it is tasked with monitoring.

We believe that an SLA is a key component to be considered at all stages in the lifecycle of a service. The policies for managing the service, the probes for monitoring it and the acceptable quality of service terms to offer to a consumer should be produced at the same time as the service is designed and developed. This ensures the correct information is available to be able to guarantee the QoS levels necessary that a consumer will consider entering into an agreement with a provider to use the service.

Having completed the initial architectural design work with regard to SLAs, the Grid foundations activity is now producing a prototype implementation of an SLA template interface. Other tasks are analysing how “collective” inter-enterprise computing business issues can be addressed through pairwise SLAs between participants, by studying how accountability and billing can be represented in SLA terms, for example. These ideas will then be implemented, so that NextGRID industrial partners can experiment with using this technology for real world applications, to prove its efficacy or, alternatively, show where we have gone wrong in our design and thinking. It is through the many strands of work within NextGRID such as this that we are building the next generation of Grid that will meet the needs of business and commerce.

Gartner Group–the well known IT analysis company–have widely publicised the idea of a “hype cycle for emerging technologies” In the Gartner Hype Cycle Special Report for 2005, Grid Computing finds itself halfway down the slope from the “peak of inflated expectations” to the “trough of disillusionment.” Interestingly however, Gartner indicates that they expect it to reach the “plateau of productivity” within 2-5 years.

Achieving our goal of productive Grid computing for business and science requires that we focus both on incremental improvements of current technologies and also that we are prepared to think beyond the status quo and try out new ideas. This is a key premise of the NextGRID project.