Advanced Resource Connector
- Trac browser
- Direct access to ARC v0.8.x tree and older
- Direct access to ARC Nox tree
| Current production release of
ARC middleware (available via download site) |
ARC v0.8.2.2 |
Background
The Advanced Resource Connector (ARC) middleware by
NorduGrid (sometimes called the NorduGrid middleware)
is a software solution that uses Grid technologies
to enable sharing and federation of computing and storage
resources distributed across different administrative and
application domains. ARC is used to create Grid
infrastructures of various scope and complexity, from campus
to national Grids. It is distributed under the Apache v2.0 license. 
Since the first release (May 2002) the middleware is deployed and being used in production environments (see e.g. Atlas Data-Challenge 1 on NorduGrid). Emphasis is put on scalability, stability, reliability and performance of the middleware. A significant number of national and international Grid infrastructures, like Swegrid, DCSC, NDGF or Ukrainian Academic Grid chose ARC as their middleware.
In 2006-2009, the EU KnowARC project brought ARC to a new level, providing it with Web Service based components compliant with Open Standards. This technology preview was released in end-2009 as ARC Nox
In 2009, the European Grid Initiative (EGI) Blueprint named ARC among the major middlewares to be supported. Current ARC development is largely done in the framework of the EU European Middleawre Initiative (EMI) project, in close collaboration with EGI.
Middleware description
ARC provides a reliable implementation of the fundamental Grid services, such as information services, resource discovery and monitoring, job submission and management, brokering and data management and resource management. Most of these services are provided through the security layer of the GSI. The middleware builds upon standard Open Source solutions like the OpenLDAP, OpenSSL, SASL and Globus Toolkit® (GT) libraries. All the external sofware is provided in the download area and the complete list of external software dependencies can be found here. NorduGrid provides innovative solutions essential for a production quality middleware: the Grid Manager, gridftpd (the ARC/NorduGrid GridFTP server), the information model and providers (NorduGrid schema), User Interface and broker (a "personal" broker integrated into the user interface), extended Resource Specification Language (xRSL), and the monitoring system.
The ARC middleware integrates computing resources (usually, commodity computing clusters managed by a batch system or standalone workstations) and Storage Elements, making them available via a secure common Grid layer. An overview of the architecture can be found, e.g. in the NorduGrid architecture and tools presentation. Complete documentation, including component descriptions, installation details, usage instructions, articles and presentations, can be found at the documentation page.
ARC main components are:
- Grid services running on the resources: the Grid Manager, gridftpd and the information services. Grid jobs are submitted to the cluster through gridftpd and a separate session directory is created for each job. The Grid session directories are made available through the gridftpd server during and after job execution. The Grid Manager is a service running on a computing resource taking care of jobs, session directories and the input data cache area. Information services are implemented as efficient scripts populating the NorduGrid information database stored in the Globus-specific OpenLDAP backends.
- Indexing services for the resources and data: a special usage of an OpenLDAP database allows to build a hierarchical mesh of Grid-connected sites. For data indexing, ARC middleware relies on 3rd-party solutions, such as e.g. the LHC File Catalog (LFC). ARC client tools and the Grid Manager are capable of interacting with these services.
- Clients making intelligent use of the distributed information and data available on the Grid. ARC comes with a light-weight command line client, the User Interface. The ARC User Interface is a set of tools to submit, monitor and manage jobs on the Grid, move data around and query resource information. The User Interface comes with a built-in broker, which is able to select the best matching resource for a job. Grid job requirements are expressed via xRSL. Another special client is the Grid Monitor, which uses any Web browser as an agent to periodically query the distributed information system and present the results as a set of inter-linked Web pages.
ARC is designed to be a scalable, non-intrusive and portable solution. The development is user- and application-driven, with the main requirements being those of performance, stability, useability and portability. As a result of this approach, the standalone client is available for a dozen of platforms and can be installed in a few minutes. The server installation does not require a full site reconfiguration. The middleware can be built on any platform where the external software packages (like GT libraries) are available. While being deployed on a large production Grid and being used by real users, the middleware is naturally undergoing continous real life tests.
Last, but not least, ARC is designed with international users in mind: the client tools are available in several languages. The Grid Monitor is translated to more than 10 languages, and the User Interface is available not just in English, but also in Russian and Swedish. Volunteers to provide more translations are always welcomed!
.Packaging, distribution
The middleware is available in binary tarballs, RPMs and DEBs. The sources come as tarballs or source RPMs, while the pre-built binaries are divided into the following modules:
| nordugrid-arc-client | Client programs |
| nordugrid-arc-server | Server-side meta-package with dependencies on all components |
| nordugrid-arc-gridftpd | Gridftpd package |
| nordugrid-arc-gridmanager | Grid-manager package |
| nordugrid-arc-infosys-ldap | Information system and information index server |
| nordugrid-arc-gridmap-utils | Utilities for managing gridmap-files |
| nordugrid-arc-ca-utils | Utilities for mainting CA files |
| nordugrid-arc-monitor | Grid monitor web interface (should be installed on a Web server) |
| nordugrid-arc-doc | Documentation |
| nordugrid-arc-libs | Runtime libraries |
| nordugrid-arc-libs-devel | Development files |
| nordugrid-arc-python | Python bindings of ARClib |
Furthermore, nordugrid-arc-standalone client binary tarballs are available for a variety of Linuxes. This client comes with all the required external software (e.g., Globus) and configuration, thus providing a complete, out-of-the-box client package that can be used by other Grids, e.g., for data transfer or data indexing.
Downloads and installation
The middleware is free to be deployed anywhere by anybody. Pre-built binary releases for a dozen of GNU/Linux distributions can be downloaded from this site, and installation instructions are available from the "Documentation" area.
Software releases are available under the Apache v2.0 license. The NorduGrid software repository hosts binary packages and the source code, and provides all the required external software.
Research and development
The Open Source development of the middleware is coordinated by the NorduGrid Collaboration. The Collaboration is open to new members. Contributions from the community to the software and the documentation are most welcomed, please contact the technical coordinator or write to the discussion list for details. Sources can be downloaded from this Web site or SVN.
User support, feature/bug reports, documentation
User support and site installation assistance is provided via the request tracking system available at . In addition, NorduGrid runs several mailing lists among which nordugrid-discuss is a general forum for all kind of issues related to the ARC middleware.
NorduGrid deploys the Bugzilla problem tracking system. Feature and enhancement requests, as well as discovered problems, should be reported there.
Research papers, overview talks, reference manuals, user guides, installation instructions, conference presenations, FAQ and even tutorial materials can be fetched from the documents section.