Overview of ARC client tools

This document contains general overview of available ARC client tools. For more detailed information about possible tool usage options, please refer to the --help messages and corresponding man pages.

Submission endpoint types

Table 7 The submission types short, long and legacy style
short long legacy
emies org.nordugrid.emies org.ogf.glue.emies.activitycreation
gridftpjob org.nordugrid.gridftpjob org.nordugrid.gridftpjob
arcrest org.nordugrid.arcrest N/A
internal org.nordugrid.internal org.nordugrid.internal

Example of use:

arcsub -T emies -C some-ce-endpoint

Credentials

arcproxy

The arcproxy command creates a proxy certificate from a key/certificate pair which can then be used to access resources.

It is all-in-one tool for all kind of operations with proxy certificates, including VOMS extension generation and communications with MyProxy services.

[user ~]$ arcproxy -S myVO
Enter pass phrase for private key:
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named myVO): voms.example.org on port: 15004
Proxy generation succeeded
Your proxy is valid until: 2020-01-22 22:50:44

[user ~]$ arcproxy -I
Subject: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith/CN=2123720174
Issuer: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Time left for proxy: 11 hours 56 minutes 45 seconds
Proxy path: /tmp/x509up_u1000
Proxy type: X.509 Proxy Certificate Profile RFC compliant impersonation proxy - RFC inheritAll proxy
Proxy key length: 2048
Proxy signature: sha256
====== AC extension information for VO myVO ======
VO        : myVO
subject   : /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
issuer    : /DC=org/DC=AccreditatedCA/O=hosts/CN=voms.example.org
uri       : voms.example.org:15004
attribute : /myVO/Role=NULL/Capability=NULL
Time left for AC: 11 hours 56 minutes 47 seconds

[user ~]$ arcproxy -L myproxy.example.org -M put -S myVO -U myproxyuser
Enter pass phrase for private key:
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named myVO): voms.example.org on port: 15004
Proxy generation succeeded
Your proxy is valid until: 2020-01-29 10:51:52
Enter pass phrase for MyProxy server (new):
Verifying - Enter pass phrase for MyProxy server (new):
Succeeded to put a proxy onto MyProxy server

arcrenew

The arcrenew command is used for renewing the proxy of jobs that have been submitted to computing resources.

[user ~]$ arcrenew https://arc.example.org:443/arex/7LqLDmKESEwnf5481mks8bjnABFKDmABFKDmhNFKDmHGFKDm9BpmKn
Jobs processed: 1, renewed: 1

Job submission and management

arcsub

The arcsub command is used for submitting jobs to computing resources.

Note

Submission endpoint selection options are changed in ARC 6.5.0 release

[user ~]$ arcsub -T emies -C arc.example.org myJob.xrsl
Job submitted with jobid: https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn

arcstat

The arcstat command is used for obtaining the status of jobs that have been submitted to computing resources.

[user ~]$ arcstat https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Job: https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
 Name: myJob
 State: Running

Status of 1 jobs was queried, 1 jobs returned information

arccat

The arccat command performs the cat command on the stdout, stderr or A-REX error log of the job.

It can also show the specified file from job’s session directory.

[user ~]$ arccat https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
SHELL=/bin/bash
HISTSIZE=1000
<output omitted>

[user ~]$ arccat -l https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
2020-01-22T09:57:45Z Job state change UNDEFINED -> ACCEPTED   Reason: (Re)Accepting new job
2020-01-22T09:57:45Z Job state change ACCEPTED -> PREPARING   Reason: Starting job processing
2020-01-22T09:57:45Z Job state change PREPARING -> SUBMIT   Reason: Pre-staging finished, passing job to LRMS
----- starting submit_pbs_job -----
PBS jobname: myJob
<output omitted>

arckill

The arckill command is used to kill running jobs.

[user ~]$ arckill https://arc.example.org:443/arex/wOOKDmT2REwnf5481mks8bjnABFKDmABFKDmhNFKDmWFFKDm1FSSRn
Jobs processed: 1, successfully killed: 1, successfully cleaned: 1

arcclean

The arcclean command removes a job from the computing resource.

[user ~]$ arckill --keep https://arc.example.org:443/arex/UcBLDm32REwnf5481mks8bjnABFKDmABFKDmhNFKDmiFFKDm0zk4un
Jobs processed: 1, successfully killed: 1

[user ~]$ arcclean https://arc.example.org:443/arex/UcBLDm32REwnf5481mks8bjnABFKDmABFKDmhNFKDmiFFKDm0zk4un
Jobs processed: 1, deleted: 1

arcresub

The arcresub allows to resubmit job to the same or other compute resources.

[user ~]$ arcresub --same https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Job submitted with jobid: https://arc.example.org:443/arex/KSeNDmz5REwnf5481mks8bjnABFKDmABFKDmhNFKDmsFFKDmp4MPWn

arcresume

The arcresume command is used for resuming a job that was submitted to compute resources and then subsequently failed. The job will be resumed at the last ok state reported by the cluster.

[user ~]$ arcresume https://arc.example.org:443/arex/vIGMDm58REwnf5481mks8bjnABFKDmABFKDmhNFKDm6FFKDmOBYWDm
Jobs processed: 1, resumed: 1

arcget

The arcget command is used for retrieving the results from a job.

[user ~]$ arcget --keep https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Results stored at: 5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Jobs processed: 1, successfully retrieved: 1

arctest

The arctest command is used for testing resources. It is able to:

  • submit several test job
  • prints info about installed user- and CA-certificates

Note

Submission endpoint selection options are changed in ARC 6.5.0 release

[user ~]$ arctest -J 2 -C arc.example.org
Job submitted with jobid: https://arc.example.org:443/arex/rRVLDmSBSEwnf5481mks8bjnABFKDmABFKDmhNFKDm8FFKDmGoZf3m

[user ~]$ arctest -E
Certificate information:
  Certificate: /home/john/.globus/usercert.pem
  Subject name: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
  Valid until: 2020-04-21 18:48:15

Proxy certificate information:
  Proxy: /tmp/x509up_u1000
  Proxy-subject: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith/CN=2123720174
  Valid for: 11 hours 37 minutes 33 seconds

Certificate issuer: /DC=org/DC=AccreditatedCA

CA-certificates installed:
  /DC=org/DC=AccreditatedCA
<output omitted>

Data manipulation

arcls

The arcls command is used for listing files in storage elements and replica catalogs.

[user ~]$ arcls srm://se.example.org/dpm/example.org/home/myVO/
data.1
data.2

[user ~]$ arcls -lL rucio://rucio-lb-prod.cern.ch/replicas/mc15_13TeV/EVNT.19785567._000729.pool.root.1
<Name>                           <Type>  <Size>       <Modified>      <CheckSum>        <Latency>
EVNT.19785567._000729.pool.root.1  file 167468788       (n/a)         adler32:f7332aeb      (n/a)
     root://xrootd.lcg.triumf.ca:1094//atlas/atlasdatadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
     srm://srm.ndgf.org:8443/srm/managerv2?SFN=/atlas/disk/atlasdatadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
     gsiftp://gridftp.echo.stfc.ac.uk:2811/atlas:datadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1

arccp

The arccp command copies files to, from and between storage services.

[user ~]$ arccp /mnt/data/data.123 srm://se.example.org/dpm/example.org/home/myVO/data.123

[user ~]$ arccp rucio://rucio-lb-prod.cern.ch/replicas/mc15_13TeV/EVNT.19785567._000729.pool.root.1 /tmp/

arcmkdir

The arcmkdir command creates directories on storage elements and catalogs.

[user ~]$ arcmkdir srm://se.example.org/dpm/example.org/home/myVO/myData

arcrename

The arcrename command renames files on storage elements.

[user ~]$ arcrename srm://se.example.org/dpm/example.org/home/myVO/data.123 srm://se.example.org/dpm/example.org/home/myVO/data.124

arcrm

The arcrm command deletes files on storage elements.

[user ~]$ arcrm srm://se.example.org/dpm/example.org/home/myVO/data.124

Information services

arcinfo

The arcinfo command is used for obtaining the status of computing resources and detailed information about the resource according to published data in the information system.

The summary optput includes the information endpoint type starting from ARC 6.5.0.

[user ~]$ arcinfo arc.example.org
Computing service:  (production)
Information endpoint: ldap://arc.example.org:2135/Mds-Vo-Name=local,o=grid (org.nordugrid.ldapng)
Information endpoint: ldap://arc.example.org:2135/o=glue (org.nordugrid.ldapglue2)
Information endpoint: https://arc.example.org:443/arex (org.nordugrid.arcrest)
Information endpoint: https://arc.example.org:443/arex (org.ogf.glue.emies.resourceinfo)
Submission endpoint: https://arc.example.org:443/arex (status: ok, interface: org.nordugrid.arcrest)
Submission endpoint: https://arc.example.org:443/arex (status: ok, interface: org.ogf.glue.emies.activitycreation)
Submission endpoint: gsiftp://arc.example.org:2811/jobs (status: ok, interface: org.nordugrid.gridftpjob)

[user ~]$ arcinfo -l arc.example.org | grep Implementation
    Implementation name: nordugrid-arc-6.5.0

arcsync

The arcsync command synchronizes your local jobs database with the information at a given computing element(s).

[user ~]$ $ arcsync -C arc.example.org
Synchronizing the local list of active jobs with the information in the
information system can result in some inconsistencies. Very recently submitted
jobs might not yet be present, whereas jobs very recently scheduled for
deletion can still be present.
Are you sure you want to synchronize your local job list? [y/n] y
Total number of new jobs found: 7