Grid Standards and Interoperability

White paper "Implementing Grid Standards" by KnowARC
An inventory of standards and experiences from the development of the ARC middleware

Overview

During the past years, numerous Grid and Grid-like middleware products have emerged, to list some:

They are capable of providing (some of) the fundamental Grid services, such as Grid job submission and management, Grid data management and Grid information services. The emergence and broad deployment of the different middlewares brought up the problem of interoperability. It took a while for the Grid community to start delivering widely accepted, usable and implementable standards. Recent developments show welcoming advances in this direction.

ARC is named among the three core European middlewares by the European Grid Initiative (EGI) design study in the blueprint document released in end-2008. These three middlewares (ARC, gLite and Unicore) are the basis of the future Universal Middleware Distribution - a set of middleware components enabling the European Grid infrastructure.

The NorduGrid Collaboration believes in coexistence of interoperable Grid middlewares and the diversity of Grid solutions. We don't think that a single Grid middleware is the solution, neither we think such a single solution that suits all is achievable. NorduGrid strongly opposes any kind of monopolistic approach; instead we'd like to see well-defined, broadly accepted open interfaces of the various Grid middleware components. Interoperability should be achieved by establishing these interfaces based upon community standards. Interoperability is understood on the service level, on the level of fundamental Grid services and their interfaces.

Therefore NorduGrid aspires to play an active role in several standardization processes and invests efforts in the implementation and support of emerging standards. The Collaboration participated in the Rome CRM initiative, contributed to the GLUE2 specification, closely follows the OGF developments, and cooperates with the major middleware development projects. We are proud to be a major contributor to the GLUE 2.0 information model specification, recently endorsed as a proposed standard by the Open Grid Forum (OGF). As one of the founders of the GLUE working group, the NorduGrid Collaboration and KnowARC are naturally committed to adopting GLUE 2.0 through its ARC middleware.

Implemented standards

ARC implements and contributes to development of a number of standards, mostly in the framework of the Open Grid Forum (OGF). ARC Grid middleware design and development follows web service-based approach. Below is a brief list of most relevant to ARC standards and related activities:

ARC and interoperability

An interoperability snapshot of the NorduGrid/ARC middleware is presented below, organized by middleware components.

Security system

The security infrastructure of ARC fully complies with and relies on the Grid Security Infrastructure (GSI) for authentication. GSI is a de facto community standard. Authorization procedures support or rely on such solutions as VOMS, GACL and LCAS/LCMAPS. None of these are community standards though. The new ARC security framework under development makes use of more conventional standards, like XACML.

Job Description

Currently ARC uses the extended Resource Specification Language (xRSL) for describing Grid job requests. OGF standard JSDL is also supported, although it is less versatile and does not enjoy wide acceptance by users.

Data Management

ARC data management components support and are compatible with the most common solutions, such as the GridFTP protocol, as well as storages based on traditional FTP and HTTP servers. ARC is also capable of interfacing to most commonly accepted open data indexing catalogues such as the Globus RLS and the LHC File Catalogue (LFC). SRB is not supported yet, mostly due to its original restrictive license. ARC data management solutions aim to support the SRM standard, and it is supported by all the relevant data movement tools in ARC.

Information Services

A community-accepted information model and representation of Grid resources and entities is a cornerstone of interoperability. The major middlewares make use of different incompatible information models. ARC implemented and for long time relied on its own model, while other large deployments make use of various modifications of the Glue model. A major re-design of Glue started in 2007, and resulted with a GLUE2 specification proposal in spring 2009. The NorduGrid Collaboration was a very significant player in that process, providing a co-chair for the standard development group, and becoming one of the first middlewares to implement the specification.

Job submission interface

There is no standard job submission interface commonly accepted by the Grid community. Current Grid systems make use of very different solutions for job submission: some of them rely on a particular GRAM implementation from Globus, others make use of Condor functionalities, or have their own proprietary protocol for that. The original ARC implemented job submission via GridFTP channel, while the new ARC execution service A-REX is implemented in a WS-RF framework. OGF's Basic Execution Service (BES) has emerged recently as a good foundation for job submission standardization, but not unlike JSDL, it is rather limited in functionality relevant to high-throughput computing. Still, new ARC execution service A-REX supports BES to a large extent.

Usage statistics & accounting

Until recently, ARC collected usage information via the experimental ARC logger service. Each Grid job ran in the ARC system is described described by a Usage Record, which recently moved from an ad hoc format to the one compliant with the OGF specification (and thus inevitably minimalistic). ARC provides no own accounting service, being compatible with other solutions like SGAS.

Relevant documents