Science is a global enterprise. Its conduct transcends geographic, discipline, and educational levels. The routine ability to work with experts around the world, to use resources distributed in space across international boundaries, and to share and integrate different types of data, knowledge and technology is becoming more realistic. It is the development and deployment of compatible cyberinfrastructures (a.k.a Grid) that link together computers, data stores, and observational equipment via networks and middleware that form the operative IT backbone of international science teams. While, large community projects exist that exploit the Grid (e.g. Large Hadron Collider)¹ , international collaboration can and most likely will take place at scales of smaller teams. For example, a multidisciplinary, distributed team of researchers from the University of Zurich, the University of California San Diego, and Monash University in Australia are synthesizing application and grid middleware, using distributed computational resources from the PRAGMA testbed ², to gain understanding of complex biochemical reactions than can impact the design of new drugs ^{3 4 5 5a 6}. This example and others ^{7 8 9} demonstrate the value and potential of working with the emerging cyberinfrastructure. Yet, significant effort was required to bring these tools, people and resources together. The current challenge for the Grid community is to make this potential and demonstration a reality, on a routine basis.

Established in 2002, the Pacific Rim Application and Grid Middleware Assembly ¹⁰ is an open organization whose focus is how to practically create, support and sustain international science and technology collaborations. Specific experiments are postulated, candidate technologies and people are identified to support these experiments, evaluation is performed in our trans-Pacific routine-use laboratory, and successful solutions are integrated into country-specific software stacks or Global Grid Forum ¹¹ standards. The group harnesses the ingenuity of more than 100 individuals from 25 institutions to create and sustain these long-term activities. PRAGMA plays a critical role as an international conduit for personal interactions, ideas, information, and grid technology. Our multi-faceted framework for collaboration catalyzes and enables new activities because of a culture of openness to new ideas. Our pragmatic approach has lead to new scientific insights ³, enhanced technology ^{12 13 14}, and a fundamental sharing of experiences manifest in our routine-use laboratory.

PRAGMA began with the following observations: global science communities were emerging in increasing numbers; grid software had entered its second phase of implementation; and international networks were expanding rapidly in capacity as fundamental high-speed enablers for data and video communication. But, the integration and productive use of these potential tools was “out of reach” to many scientists. To address the issue of making this technology routinely accessible, a founding set of Pacific Rim Institutions began to work together to share ideas, challenges, software, and possible end-to-end solutions.

Our common-sense approach begins with prospective collaborative science-driven projects (like whole genome annotation, quantum chemistry dynamics, Australian Savannah wildfire simulation, and remote control of large electron microscopes coupled with 3D tomographic reconstruction) so that both people and candidate technologies can be identified to address the scientific needs. Identification is through people-to-people networking, progressively more sophisticated demonstrations, tutorials on software components (e.g. gFarm, MOGAS¹⁵, Nimrod, Rocks¹⁶, Ninf-G ¹⁷ and others) and a consistent face-to-face workshop schedule. When enough ingredients are available to start down the pathway of using the Grid, integrating software to be grid-aware, and/or sharing data, then software is instanced onto our routine-use laboratory. This lab (described in more detail below with its evolution and management challenge described in ¹⁸) is where technologists from multiple organizations work together to provide a baseline infrastructure for evaluation. Successful science projects can move to larger resource pools if needed. The entire end-to-end process is possible because of an active international steering committee that continually focuses the group’s multiple efforts for tangible results. Below we describe and illustrate these key components of PRAGMA, together with software distribution and community building.

PRAGMA brings multidisciplinary, multi-institutional teams together, driven by application needs. In addition to the computational chemistry application described above, another team of researchers from the US, Japan, China, and Singapore, integrated a protein annotation pipeline (iGAP ¹⁹) developed at UCSD, a distributed file system (Gfarm ²⁰) developed at the National Institute for Advanced Industrial Science and Technology (AIST), and a metascheduler (CSF) being extended by researchers at Jilin University in China to schedule iGAP testing ²¹. This software/middleware synthesis effort has led to improvements of Gfarm. In particular, the metadata server design is changed to meet the requirements of high throughput file creation and registration. Automatic replication of data and deployment of applications to remote sites become fully supported for most common architectures ⁷. Finally, a successful annotation of the bacteria, Burkholderia mallei, a known bioterrorism agent, has been conducted with this infrastructure and the PRAGMA testbed (the annotation will be publicly available pending publication of analysis results).

A final example integrates expertise of IPv6 networking at the Cybermedia Center of Osaka University, remote control of a microscope at UCSD, and use of a computational grid to build tomographic reconstructions of subcellular structures, and the development of visualization modules from the National Center of High-performance Computing (NCHC), to provide an enhanced suite of tools for researchers to use ²². Not only did the team benefit, but each group did as well. UCSD researchers were able to better access the machine in Japan and distributed compute resources at the three sites. Osaka researchers were able to control the machines and make codes available to their users, and NCHC colleagues were able to take the concept and knowledge of remote control of a microscope and retarget the application to that of sensors in the environment, creating EcoGrid ²³ in Taiwan.

Each of these international science and technology teams has shared technology and experience to significantly enhance their research agendas. The structure of PRAGMA, with its culture of openness to new ideas and technologies coupled with a recurring series of focused workshops²⁴, provided the essential glue for these teams. Each of these accomplishments has resulted in ongoing collaborations that now span years.

These and other examples ⁷ have driven the use of PRAGMA’s evolving grass-root grid testbed. The overall goal of the PRAGMA testbed is to provide a stable platform to allow these and other application/middleware codes to be tested, and to understand how to make applications run on a routine basis without the superhuman efforts that many of these examples currently require.

The current testbed consists of resources and participants from 19 institutions from five continents and 13 countries. It is an instantiation of a useful, interoperable, and consistently available grid system that is neither dedicated to the needs of a single science domain nor funded by a single national agency. This testbed is heterogeneous in equipment and connectivity (bandwidth as well as persistence) between machines, reflecting both funding realities and the future global cyber-infrastructure.

The testbed has been grown using a minimum set of requirements. The initial software stack comprised of Globus plus local scheduling software. Additional middleware is added based on the needs of the applications. For example, a remote procedure call middleware, Ninf-G developed at AIST, became part of the testbed since it was required by two applications being used in PRAGMA: one is a time dependent density functional theory calculation; the other a Quantum Mechanical / Molecular Dynamics (QM/MD) code ⁷.

These routine use experiments have produced results through strong feedback between application and middleware developers. Codes have had to be improved to operate in a network environment where connections fail, in particular to be more fault tolerant. The testbed development is being driven by many application areas, allowing examination of different requirements. In addition, a richer monitoring set (SCMSWeb)²⁵ developed at Kasetsart University and accounting tools (MOGAS) ²⁶ of the Nanyang Technological University were introduced to the testbed.

PRAGMA has demonstrated that these grids can be very useful to improve codes and to conduct meaningful and otherwise unachievable science.

As software is tested in these real-world environments and used in multiple applications, ways of disseminating software is needed. In the case of Ninf-G, we have successfully developed a procedure that allowed it to be integrated into the US NMI software stack for version 8 and subsequent releases. This was the first instance of a non-US code being introduced into that stack. Another dissemination vehicle we are using is the Rocks Rolls mechanism (i.e. RPM packages configured for automatic (re)deployment to Rocks-based clusters). PRAGMA partners also released codes with changes, and Rocks was localized by the partners at the Korea Institute for Science and Technology Information (KISTI) to allow for easier access and use by the Korean Grid community ²⁷. And with broader dissemination comes broader use. In one case this led to a set of standards proposed in GGF for remote procedure calls ²⁸.

Finally, as the broader community produces new codes or standards, PRAGMA will adopt them. Examples include the CA from Naregi²⁹ or the PMA lead by AIST, which then are used by the broader PRAGMA community.

PRAGMA itself focuses on a grass roots approach in an effort to enable new communities to form and assemble expertise not available at any single institution Some global issues require the ability to rapidly form international teams. Responses to epidemics such as SARS (PRAGMA played a crucial role in helping to pull together an international team to aid Taiwan in their efforts to combat the disease) ³⁰ and emerging threats like Avian Flu often require teams to assemble in hours or days. Other groups with a small set of geographically dispersed experts simply do not have the personnel resources to independently build a complete cyberinfrastructure. In this model, PRAGMA has played a leading role in catalyzing GLEON³¹, the Global Lake Ecological Observatory Network, a grassroots network of limnologists, information technology experts, and engineers with a common goal of building a scalable, persistent network of lake ecology observatories. Data from these observatories will help this community to better understand key processes such as the effects of climate and land use change on lake function, the role of episodic events such as typhoons in resetting lake dynamics, and carbon cycling within lakes. These teams are, by nature and expertise, international in scope.

We have built a community by focusing on concrete projects to build trust and on an on-going series of semi-annual working meetings. The meetings rotate among PRAGMA member sites to engage a broader group of researchers at each site and to allow the PRAGMA community to appreciate its members’ cultural richness.

Students are an essential component of the PRAGMA community. Pacific Rim Undergraduate Experiences (PRIME)³² provides UCSD undergraduate students summer research experiences at four PRAGMA sites; Osaka, Monash, NCHC, and the Computer Network Information Center of the Chinese Academy of Sciences (CNIC). The students conduct research and contribute to the infrastructure. Further, they have helped expand the collaborations between scientists at the institutions. In addition, the Japanese government has awarded Osaka University funds to create Pacific Rim International UniverSities (PRIUS), a program designed to improve education for graduate students interested in grid technology by supporting a series of activities, including exchanges. Both PRIME and PRIUS build on and enhance the PRAGMA community and would not exist without PRAGMA.

PRAGMA is both a multifaceted organization and an experiment. It is open to institutions who wish to actively participate in projects and contribute and share resources, middleware, applications and expertise with other members. The value of the structure is that it allows for transfer of technology among institutions, allowing in some cases for rapid start ups by acquiring technology, and in other cases for user feedback on technologies that have been developed. The structure also allows for a transfer of technologies between disciplines where, for example, control of an instrument is being moved from neuroscience and a microscope to ecology and a sensor.

Collectively, we have built a human network that allows for new activities to begin. We have built a stable, persistent grass-roots grid testbed on which codes can be tested and science conducted. We have shared our experiences via publications and the improved codes via a variety of software dissemination vehicles, allowing the broader community to benefit. Finally, we have used the structure to build a legacy where researchers will work, collaborate, and educate internationally.