Advanced Resource Connector (ARC) version 0.6 is the second stable
release of this grid middleware. The first stable release (v0.4) was
announced on April 13 2004. After more than three years of
development, NorduGrid Collaboration presents this new release that
contains some new features and hundreds of bug fixes.
ARC release 0.6 is not substantially different from version 0.4, and
middleware is based on the same technology and consists of the same
key components. While underlying protocols are the same, there is only
minimal backwards compatibility between client tools and services of
the two stacks, hence an upgrade is strongly recommended.
ARC release 0.6 was available for preview and tests since December 15
2006; during this period it was shown to perform substantially better
than previous versions, offering more functionality.
Description
The Advanced Resource Connector as of version 0.6 is an open source
software solution that enables production quality computational grids
of wide range in size and purpose. The middleware integrates computing
resources (commodity computing clusters managed by a batch system or
standalone workstations) and, to a lesser extent, storage services,
making them available via a common secure grid layer. The middleware
builds upon standard open source solutions like the OpenLDAP, OpenSSL,
and Globus Toolkit 4 pre-WS libraries. It relies on well-tested
pre-OGSA grid technologies in creating unique ARC-specific services
and tools. ARC developers strive to achieve simplicity,
non-invasiveness, high performance, stability and reliability. With
release 0.6 come enhanced scalability and better conformance to community standards. ARC
middleware is officialy supported on all major Linux flavors, and is
known to operate smoothly on other Linux systems, with a variety of
batch job management systems.
Features
ARC provides a reliable implementation of fundamental grid services,
such as information services, resource discovery and monitoring, job
submission and management, brokering, basic data and resource
management. Most of these services are provided through the security
layer of GSI.
ARC implements several services and tools essential for a production
quality middleware:
Computing Service, consisting of ARC-specific grid-manager and
GridFTP server.
Storage Service integrated with data indexing system (Smart Storage
Element).
Information system, consisting of NorduGrid information model and
providers.
Client tool implemented as a CLI with integrated resource
discovery, matchmaking and brokering, making use of ARC xRSL
language for job description; it also has basic data manipulation
functionalities.
Real-time monitoring system that relies entirely on the information
system.
ARC main features are:
Acknowledged simplicity in deployment and maintenance on a wide
range of computing resources of various configurations.
Highly sophisticated grid gatekeeper for computing resources
(grid-manager), capable of most complex job and data management.
operations, with comprehensive security features and extensibility
Absence of centralized workload management service.
Reliable information system that accurately reflects up-to date
system status, suitable for both resource discovery and monitoring.
Easy to install, small and yet very powerful client tool making
intelligent use of the distributed information.
Changes since ARC v0.4
New components:
Client C++ library for unified and easy access to ARC-enabled
resources for third party applications. Bindings for Python
and Perl are provided as well. Basic Java library is also available.
Optional: minimal HTTPs/g server, used by Web Services.
Optional: Smart Storage Element with SRM-compliant interface. A Web
Service. Uses GSI-secured HTTP protocol for data transer and can
be integrated with data indexing systems, such as Globus RLS.
Optional: Globus RLS is adopted as the default data indexing system.
Optional: Logger service to store job information. A Web
Service. Complete with client.
New functionalities and features:
Support for more sources in authorization framework - VOMS, LCAS,
LCMAPS, MyProxy. Extensible to any third-party authorization
framework.
Support for JSDL 1.0 with ARC-specific extensions.
Major re-write of the client code base introduce modularity,
multithreaded implementation and extensibility and localization.
Unified configuration for ARC services and clients.
Support for more service types in all data management tools -
SRM v1.1 with both HTTPg implementations, LFC, Fireman.
Support for more LRMS types - Condor, SGE, LSF, fork.
Enhanced job processing algorithm - better handling of faults.
Many new attributes and some schema changes in information system
which enhance client functionality and LDAP compatibility.
Major rewrite of Grid Monitor increases it's robustness and
performance. Multiple localizations are added.
New packaging and distribution: changed package names, complete
re-arrangement of package contents, new Yum repositories, upgraded
external dependencies.
Possibility to run services as a non-privileged user.
Overall improved usability: server logs, notifications, client
command line options.
Multiple documentation updates.
Fixed bugs:
In total, 351 bugs were reported and fixed during development, of
them 24 blocking, 14 critical, 25 major, 256 normal, 32 minor and
trivial ones. While some were introduced during development, most
existed in ARC 0.4. See NorduGrid bug tracking system for details.
System requirements
The middleware is tested on a variety of Linux systems. While it
should work on other Unix-like systems, this release was not tested on
non-Linux machines.
For a computing service, you will need a Linux cluster running a
Local Resource Management System or a standalone Linux box
configured with "fork" job submission. It can be shared with other
services.
For a storage service, you will need a conventional disk array
with Linux front-end, or simply a Linux box with some storage
capacity. It can be shared with other services.
For all the optional services, a shared Linux box is sufficient.
For a client any recent Linux box will do.
Availability
The middleware is free to be deployed anywhere by anybody. Pre-built
binary releases for a dozen of Linux platforms can be downloaded from
http://download.nordugrid.org or via NorduGrid RPM repositories.
The software is released under GNU General Public License (GPL).
The NorduGrid repository hosts the source code, and provides all the
essential external software which are not part of a standard Linux
distribution.
Installation notes
NorduGrid ARC packages fall into two main kinds: the server and the
client part. Server is typically installed on a computing resource by
a system administrator, while the client can be set up anywhere and
needs no system administrator privileges. Detailed installation
instructions are distributed with the middleware documentation and are
available at the Web site:
Non-interoperable with other known grid middlewares.
Portability is limited by the Globus Toolkit 4 libraries which
means the middleware is restricted to few Unixes and is tested
only on Linux.
Needs non-standard OpenLDAP and OpenSSL libraries (Globus legacy).
Only partial support for clusters without a shared file system.
No support for routing queues in PBS.
Storage Element is limited to POSIX file systems.
No cluster-level sandboxing is provided for grid jobs apart from
the standard Unix access controls. No easily implementable fine
grained authorization for computing resources.
No access control for the information system.
Information system represents only two services: cluster and
storage.
Inconsistent treatment for node-sharing queues.
LDAP port and information schema hardwired in the client libary.
No bulk data manipulation tools.
Ad-hoc data management tools with insufficient resource
description.
Inconsistencies in service discovery and authorization.
No official workload management system.
No user management system.
No GUI.
For community authorization services, only VOMS is tested.
For data management services, only RLS is tested.
Support
User support and site installation assistance is provided via the
request tracking system available at .
In addition, NorduGrid runs a couple of mailing lists, among which the
nordugrid-discuss mailing list is a general forum for all kind of
issues related to ARC middleware.
NorduGrid deploys the Bugzilla problem tracking system. Feature and
enhancement requests, as well as problems, should be reported there.
Contact information is kept updated on the NorduGrid web site.