ARC Compute Element

EMI Compute Elements comparison chart

ARC Compute Element (CE) is a Grid front-end on top of a conventional computing resource (e.g. a Linux cluster or a standalone workstation). ARC CE is sometimes also called ARC server. This page presents a quick overview of ARC CE functionalities and corresponding internal components.

For more details on architecture, design, deployment and maintenance, please see:

ARC CE is:

  • Publishing information about itself
  • Performing user authorisation based on Grid credentials and local policies
  • Mapping Grid users to local accounts
  • Accepting Grid jobs from authorised users:
    • Job description documents in JSDL or ARC xRSL languages
    • Job input files
  • Downloading additional input files from Grid storages on behalf of the authorised user
  • Caching input files
  • Converting Grid job descriptions to local batch system job submission scripts (most known batch systems are supported)
  • Preparing pre-installed application runtime environment if specified in job description
  • Submitting jobs to the local batch systems and their consequent handling: providing job outputs upon user request, providing job status information, interrupting jobs upon user request, restarting failed jobs upon user request, cleaning up user session
  • Uploading user-specified job output to Grid storages on behalf of the user
  • Collecting accounting information (usage record)

ARC CE interfaces

GridFTP + LDAP (pre-Web Service)

Can be used in either pre-Web Service or in a Web Service enabled configuration

Pre-WS interface: GridFTP for job description and files; anonymous LDAP for information

WS-based ARC CE: unifies all interfaces


Web Service

Authorisation

Delegation of credentials

Inspects users' credentials, makes authorisation decisions

Requires delegation of credentials to act on behalf of users

Grid users are commonly mapped to local accounts


User mapping

Interaction with batch (LRMS) systems

Front-end to LRMS with shared session directories

Interacts with LRMS via specialised scripts

Job information and all relevant files are stored in dedicated directories

Directories commonly shared with worker nodes: session, RTE and cache


Persistent directories needed by ARC CE

File staging

Stage-in

Key feature #1: download input files on behalf of the user (stage-in) and upload output files (stage-out)

Key feature #2: input files are cached to avoid multiple transfers in case a file is processed by several jobs


Stage-out


NorduGrid homepage