Installing ARC Client Tools

Step 1. Enable NorduGrid ARC repos

Prepare your system to install via the NorduGrid Repositories.

If you need the latest client tools that are not yet released, configure nightly builds repository following Using ARC packages from nightly builds instructions.

Note

On several operating systems you can install ARC without using the NorduGrid Repositories (e.g. using EPEL for CentOS/RHEL). But we recommend that you even on those systems install from the NorduGrid Repositories as they contains IGTF certificates and get updated faster on new releases.

Step 2. Install packages

Client tools

The main client tools package is nordugrid-arc-client. Install it using your OS packet manager [1]:

[root ~]# dnf -y install nordugrid-arc-client

This package brings necessary plugins as dependency and allows you to submit jobs and perform data transfers in the ARC ecosystem, e.g.:

  • submit jobs via ARC REST interface defined by xRSL or EMI-ES ADL descriptions

  • query ARCHERY registry and ARC REST GLUE2 local information system

  • support local and HTTP(S) data transfers

  • support Rucio and SRM

If you need more features, additional plugins are available (see below).

ARCCTL

ARC Control Tool is one-stop-shop for sysadmins and users running ARC that automate many operations.

For the client side it will allow you to easily deploy CA certificates and VOMS configuration:

[root ~]# dnf -y install nordugrid-arc-arcctl

Additional plugins

Note

In ARC 7, the nordugrid-arc-plugins-arcrest is included in the nordugrid-arc-plugins-needed, so you no longer need to install it specially as you had to in ARC 6.

  • Data transfers:

    • gridftp - support for gridftp data transfers [globus]

    • xrootd - support for xroot protocol

    • s3 - support for s3 protocol

    • gfal - add support for numerous data transfer protocols and file catalogues via installed GFAL2 plugins

[root ~]# dnf -y install nordugrid-arc-plugins-<PLUGIN NAME>

Step 3. Setting up credentials

e-Science distributed computing networks (grid) heavily relies on cryptography and uses either personal X.509 certificates/keys or OIDC tokens to identify entities and sets of dedicated Certification Authorities (CA).

In the most common workflow you are also required to be a member of some Virtual Organization (VO) to get access to infrastructure resources. On the technical level the VO membership is currently handled by infrastructure VOMS services.

ARC supports both x509 and OIDC tokens, and both can be used by the same ARC-CE (provided the ARC-CE is configured to) or ARC client.

A new feature in ARC 7 is to support grids that only rely on the default system CA bundle, or other non-igtf CA’s. For details how the ARC-CE must be set up, and how to submit jobs to such a CE, please refer to: ARC-CE only accepting non-igtf tokens.

CA certificates bundle

The CA certificates are used to verify entities (e.g. users as well as compute and storage services) and should be installed on the client host as well for infrastructure security reasons.

In the distributed grid environment a set of dedicated CAs is used as a part of IGTF.

The IGFT CA certificates bundle can easily be installed with ARC Control Tool [2]:

[root ~]# arcctl deploy igtf-ca classic

Personal authentication

As mentioned ARC 7 supports both OIDC tokens and x509 user certificates. To interact with an ARC-CE you must obtain one of these.

Personal X.509 certificate

For production infrastructure usage you need a personal certificate signed by one of the IGTF accredited CAs.

Typically there is at least one IGTF accreditage CA in each country that you can find on map.

Oganizational and technical procedures varies from CA to CA, so you should read the instructions on a choosen CA web-site.

Once you get your certificate, install it to your client host and ensure the permissions are set correctly. For historical reasons the default location for certificate and key is .globus unless redefined in client configuration file:

[user@client ~]$ ls -l ~/.globus/
total 12
-rw-r--r-- 1 user user 6353 Oct  1 11:55 usercert.pem
-r-------- 1 user user 1854 Oct  1 11:55 userkey.pem

Note

For testing and development purposes it is possible to use local testing CA. ARC Control Tool provides built-in test CA capabilities that allows you to bootstrap own test CA and generate host and user certificates.

OIDC token

To interact with an ARC-CE using tokens you must obtain a bearer token from a token issuer that your grid supports. In the case of WLCG this can be for instance the WLCG-IAM: https://wlcg.cloud.cnaf.infn.it/login, EGI Check-in: https://aai.egi.eu/registry/ or SciTokens: https://scitokens.org/. In other grids, it will be the token issuer that your community trusts.

Once you have created an identity at the required Identity Provider, and you have obtained a bearer token, you store the token in the environment variable BEARER_TOKEN before issuing the client requests.

export BEARER_TOKEN=<token-string>

For more details on the OIDC token support, please see: ARC support for OIDC.

Virtual Organization memberhip

In most cases you do have some implicit VO affiliation on your workplace. If not, you can search for VOs using e.g. EGI VO(s) search tool.

Every Virtual Organisation (VO) has own procedures and policies. Please contact VO support team for membership instructions.

To prove that you are a member of the particular VO you need to obtain a special token from the VOMS server as a part of your authentication process.

Unfortunately, the EGI API is no longer publicly accessible. For information how to obtain the token, please visit the EGI Database for your experiment, for instance for CMS: https://operations-portal.egi.eu/vo/view/voname/cms#VOV_section. and consult your experiment about how to install the necessary vomses.

Step 4. Try it out

When all is set, you can try to submit a job to the grid infrastructure.

Create a proxy certificate

To submit a job, or perform any other action you need a so-called proxy-certificate which is a Single Sign-On token for distributed grid-infrastructure. It is generated in the following way:

[user ~]$ arcproxy
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Proxy generation succeeded
Your proxy is valid until: 2019-11-28 02:06:57

If you need to include you VO membership confirmation token (attribute certificate), specify the VO name as well:

 [user ~]$ arcproxy -S area51
 Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
 Contacting VOMS server (named area51): voms.example.org on port: 15001
 Proxy generation succeeded
 Your proxy is valid until: 2019-11-28 02:08:27


.. or obtain a bearer token from your token Issuer

export BEARER_TOKEN=<token-string>

Submit a test job

ARC client tools includes the arctest utility that comes with several test jobs on board. So you can try to submit a job without the need to write a job description.

You can use the NorduGrid top-level ARCHERY service registry (nordugrid.org) to find available CEs automatically [3]:

[user ~]$  arctest -J 2 --registry nordugrid.org
Submitting test-job 2:
&( executable = "/usr/bin/env" )( stdout = "stdout" )( stderr = "stdout" )( gmlog = "gmlog" )( jobname = "arctest2" )( clientxrsl = "&( executable = ""/usr/bin/env"" )( jobname = ""arctest2"" )( stdout = ""stdout"" )( join = ""yes"" )( gmlog = ""gmlog"" )" )
Client version: nordugrid-arc-6.5.0
Test submitted with jobid: https://arc.example.org:443/arex/e932ad551df1
Computing service: KNU ARC

The job status can be than checked with the arcstat tool:

[user ~]$ arcstat https://arc.example.org:443/arex/e932ad551df1
Job: https://arc.example.org:443/arex/e932ad551df1
 Name: arctest2
 State: Running

Status of 1 jobs was queried, 1 jobs returned information

To fetch the job’s stdout run arccat tool:

[user ~]$ arccat https://arc.example.org:443/arex/e932ad551df1
GRIDMAP=/dev/null
HOSTNAME=arc.example.org
TMPDIR=/tmp
<output omitted>