Installing ARC Client Tools

Warning

Information in this document is fully applicable to ARC 6.5.0 and above only!

Step 1. Enable NorduGrid ARC6 repos

Prepare your system to install via the NorduGrid Repositories.

If you need the latest client tools that are not yet released, configure nightly builds repository following Using ARC packages from nightly builds instructions.

Note

On the several operating systems you can install ARC without NorduGrid Repositories (e.g. using EPEL for CentOS/RHEL). But even on that system you will benefit from NorduGrid Repositories as they contains IGTF certificates and gets updated faster on new release.

Step 2. Install packages

Client tools

The main client tools package is nordugrid-arc-client. Install it using your OS packet manager [1]:

[root ~]# yum -y install nordugrid-arc-client
[1]Examples are shown for YUM-based systems. You can use APT or any other packet managers in the similar way.

This package brings necessary plugins as dependency and allows you to submit jobs and perform data transfers in the ARC ecosystem, e.g.:

  • submit jobs via EMI-ES interface defined by XRSL or ADL descriptions
  • query ARCHERY registry and LDAP information system
  • support local and HTTP(S) data transfers
  • support Rucio, SRM and ACIX

If you need more features, additional plugins are avaialable (see below).

ARCCTL

arcctl is one-stop-shop for sysadmins and users running ARC6 that automate many operations.

For the client side it will allows you to easiely deploy CA certificates and VOMS configuration:

[root ~]# yum -y install nordugrid-arc-arcctl

Warning

The arcctl is available for stand-alone usage without ARC CE since version 6.5. For ealier ARC versions CA certificates and VOMS configuration should be installed manually. You can find some hints in the old client installation instructions

Additional plugins

You can consider installing additional plugins for extra functionality:

  • Job submission:
    • arcrest - submit jobs via ARC REST interface
    • gridftpjob - submit jobs via GRIDFTPJOB interface (installs gridftp as a dependency) [globus]
    • internal - submit jobs via local filesystem Internal interface (requires A-REX on the same host)
  • Data transfers:
    • gridftp - support for gridftp data transfers [globus]
    • xrootd - support for xroot protocol
    • s3 - support for s3 protocol
    • gfal - add support for numerous data transfer protocols and file catalogues via installed GFAL2 plugins
[globus](1, 2) In the ARC < 6.5 there was an all-in-one plugin globus instead of gridftp, gridftpjob and some other components.
[root ~]# yum -y install nordugrid-arc-plugins-<PLUGIN NAME>

Step 3. Setting up credentials

Currently users authentication in e-Science distrbuted computing networks (grid) heavily relies on cryptography and uses personal X.509 certificates/keys to identify entities and set of dedicated Certification Athorities (CA).

In the most common workflow you are also required to be a member of some Virtual Organization (VO) to get access to infrastructure resources. On the technical level the VO membership is currently handled by infrastructure VOMS services.

CA certificates bundle

CA certificates used to verify entities (e.g. users as well as compute and storage services) and should be installed on the client host as well for infrastructure security reasons.

In the distributed grid environment set of dedicated CAs used as a part of IGTF.

IGFT CA certificates bundle can be easiely installed with arcctl [2]:

[root ~]# arcctl deploy igtf-ca classic
[2]In case ARC is not installed from the NorduGrid repositories, use --installrepo argument to enable third-party repositories with IGTF CA certificates

Personal X.509 certificate

For production infrastructure usage you shoud obtain a personal certificate signed by one of the IGTF accredited CAs.

Typically there is at least one IGTF accreditage CA in each country that you can find on map.

Oganizational and technical procedures varies from CA to CA, so you should read the instructions on a choosen CA web-site.

Once you get your certificte, install it to your client host and ensure the permissions are set correctly. For historical reasons the default location for certificate and key is .globus unless redefined in client configuration file:

[user@client ~]$ ls -l ~/.globus/
total 12
-rw-r--r-- 1 user user 6353 Oct  1 11:55 usercert.pem
-r-------- 1 user user 1854 Oct  1 11:55 userkey.pem

Note

For testing and development purposes it is possible to use local testing CA. arcctl provides built-in test CA capabilities that allows you to bootstrap own test CA and generate host and user certificates.

Virtual Organization memberhip

In most cases you do have some implicit VO affiliation on your workplace. If not, you can search for VOs using e.g. EGI VO(s) search tool.

Every Virtual Organisation (VO) has own procedures and policies. Please contact VO support team for membership instructions.

To prove that you are a member of the particular VO you need to obtain a special token from the VOMS server as a part of your authentication process.

The arcproxy tool will get this token automatically, but requires VOMS server enpoint details to be configured.

With arcctl you only need to know the VO name (for VOs in EGI database) or VOMS URL [3]:

[root ~]# arcctl deploy vomses --egi-vo atlas
[root ~]# arcctl deploy vomses --voms  https://voms.ndgf.org:8443 --use-client-cert nordugrid.org
[3]Most of VOMS-Admin services by default prohibit access without client certificate even to configuration page. If this is your case, the --use-client-cert option will instruct arcctl to use your personal certificate to establish connection.

Step 4. Try it out

When all is set, you can try to submit a job to the grid infrastructure.

Create proxy certificate

To submit a job, or perform any other action you need a so-called proxy-certificate which is a Single Sign-On token for distributed grid-infrastructure. It is generated in the following way:

[user ~]$ arcproxy
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Proxy generation succeeded
Your proxy is valid until: 2019-11-28 02:06:57

If you need to include you VO membership confirmation token (attribute certificate), specify the VO name as well:

[user ~]$ arcproxy -S area51
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named area51): voms.example.org on port: 15001
Proxy generation succeeded
Your proxy is valid until: 2019-11-28 02:08:27

Submit test job

ARC client tools includes arctest utility that come with several test jobs on board. So you can try to submit job without the need to write a job description.

You can use NorduGrid top-level ARCHERY service registry (nordugrid.org) to find available CEs automatically [4]:

[user ~]$  arctest -J 2 --registry nordugrid.org
Submitting test-job 2:
&( executable = "/usr/bin/env" )( stdout = "stdout" )( stderr = "stdout" )( gmlog = "gmlog" )( jobname = "arctest2" )( clientxrsl = "&( executable = ""/usr/bin/env"" )( jobname = ""arctest2"" )( stdout = ""stdout"" )( join = ""yes"" )( gmlog = ""gmlog"" )" )
Client version: nordugrid-arc-6.5.0
Test submitted with jobid: https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
Computing service: KNU ARC
[4]HINT: you can add -d INFO to get mode dateails of CE selection process

The job status can be than checked with the arcstat tool:

[user ~]$ arcstat https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
Job: https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
 Name: arctest2
 State: Running

Status of 1 jobs was queried, 1 jobs returned information

To fetch the job’s stdout run arccat tool:

[user ~]$ arccat https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
GRIDMAP=/dev/null
HOSTNAME=arc.example.org
TMPDIR=/tmp
<output omitted>