Overview of ARC client tools

This document contains general overview of available ARC client tools. For more detailed information about possible tool usage options, please refer to the --help messages and corresponding man pages.

Submission endpoint types

In ARC 7 only the arcrest job submission interface is available, and therefore you do not have to specify the interface explisitly, as arcrest is the default interface.

Table 11 The submission types short and long

short	long
arcrest	org.nordugrid.arcrest
internal	org.nordugrid.internal

Example of use:

arcsub -T arcrest -C some-ce-endpoint

Credentials

arcproxy

The arcproxy command creates a proxy certificate from a key/certificate pair which can then be used to access resources.

It is all-in-one tool for all kind of operations with proxy certificates, including VOMS extension generation and communications with MyProxy services.

[user ~]$ arcproxy -S myVO
Enter pass phrase for private key:
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named myVO): voms.example.org on port: 15004
Proxy generation succeeded
Your proxy is valid until: 2024-01-22 22:50:44

[user ~]$ arcproxy -I
Subject: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith/CN=2123720174
Issuer: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Time left for proxy: 11 hours 56 minutes 45 seconds
Proxy path: /tmp/x509up_u1000
Proxy type: X.509 Proxy Certificate Profile RFC compliant impersonation proxy - RFC inheritAll proxy
Proxy key length: 2048
Proxy signature: sha256
====== AC extension information for VO myVO ======
VO        : myVO
subject   : /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
issuer    : /DC=org/DC=AccreditatedCA/O=hosts/CN=voms.example.org
uri       : voms.example.org:15004
attribute : /myVO/Role=NULL/Capability=NULL
Time left for AC: 11 hours 56 minutes 47 seconds

[user ~]$ arcproxy -L myproxy.example.org -M put -S myVO -U myproxyuser
Enter pass phrase for private key:
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named myVO): voms.example.org on port: 15004
Proxy generation succeeded
Your proxy is valid until: 2024-01-29 10:51:52
Enter pass phrase for MyProxy server (new):
Verifying - Enter pass phrase for MyProxy server (new):
Succeeded to put a proxy onto MyProxy server

arcrenew

The arcrenew command is used for renewing the proxy of jobs that have been submitted to computing resources.

[user ~]$ arcrenew https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
Jobs processed: 1, renewed: 1

Job submission and management

arcsub

The arcsub command is used for submitting jobs to computing resources.

Note

Submission endpoint selection options are changed in ARC 6.5.0 release

[user ~]$ arcsub -T arcrest -C arc.example.org myJob.xrsl
Job submitted with jobid: https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb

arcstat

The arcstat command is used for obtaining the status of jobs that have been submitted to computing resources.

[user ~]$ arcstat https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
Job: https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
 Name: myJob
 State: Running

Status of 1 jobs was queried, 1 jobs returned information

arccat

The arccat command performs the cat command on the stdout, stderr or A-REX error log of the job.

It can also show the specified file from job’s session directory.

[user ~]$ arccat https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
SHELL=/bin/bash
HISTSIZE=1000
<output omitted>

[user ~]$ arccat -l https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
2024-01-22T09:57:45Z Job state change UNDEFINED -> ACCEPTED   Reason: (Re)Accepting new job
2024-01-22T09:57:45Z Job state change ACCEPTED -> PREPARING   Reason: Starting job processing
2024-01-22T09:57:45Z Job state change PREPARING -> SUBMIT   Reason: Pre-staging finished, passing job to LRMS
----- starting submit_pbs_job -----
PBS jobname: myJob
<output omitted>

arckill

The arckill command is used to kill running jobs.

[user ~]$ arckill https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
Jobs processed: 1, successfully killed: 1, successfully cleaned: 1

arcclean

The arcclean command removes a job from the computing resource.

[user ~]$ arckill --keep https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
Jobs processed: 1, successfully killed: 1

[user ~]$ arcclean https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
Jobs processed: 1, deleted: 1

arcresub

The arcresub allows to resubmit job to the same or other compute resources.

[user ~]$ arcresub --same https://arc.example.org:443/arex/rest/1.0/jobs/f77b3d1b1efb
Job submitted with jobid: https://arc.example.org:443/arex/rest/1.0/jobs/28c62c12a26b

arcresume

The arcresume command is used for resuming a job that was submitted to compute resources and then subsequently failed. The job will be resumed at the last ok state reported by the cluster.

[user ~]$ arcresume https://arc.example.org:443/arex/rest/1.0/jobs/28c62c12a26b
Jobs processed: 1, resumed: 1

arcget

The arcget command is used for retrieving the results from a job.

[user ~]$ arcget --keep https://arc.example.org:443/arex/rest/1.0/jobs/28c62c12a26b
Results stored at: 8c62c12a26b
Jobs processed: 1, successfully retrieved: 1

arctest

The arctest command is used for testing resources. It is able to:

• submit several test job

• prints info about installed user- and CA-certificates

Note

Submission endpoint selection options are changed in ARC 6.5.0 release

[user ~]$ arctest -J 2 -C arc.example.org
Job submitted with jobid: https://arc.example.org:443/arex/rest/1.0/jobs/28c62c12a26b

[user ~]$ arctest -E
Certificate information:
  Certificate: /home/john/.globus/usercert.pem
  Subject name: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
  Valid until: 2024-04-21 18:48:15

Proxy certificate information:
  Proxy: /tmp/x509up_u1000
  Proxy-subject: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith/CN=2123720174
  Valid for: 11 hours 37 minutes 33 seconds

Certificate issuer: /DC=org/DC=AccreditatedCA

CA-certificates installed:
  /DC=org/DC=AccreditatedCA
<output omitted>

Data manipulation

arcls

The arcls command is used for listing files in storage elements and replica catalogs.

[user ~]$ arcls srm://se.example.org/dpm/example.org/home/myVO/
data.1
data.2

[user ~]$ arcls -lL rucio://rucio-lb-prod.cern.ch/replicas/mc15_13TeV/EVNT.19785567._000729.pool.root.1
<Name>                           <Type>  <Size>       <Modified>      <CheckSum>        <Latency>
EVNT.19785567._000729.pool.root.1  file 167468788       (n/a)         adler32:f7332aeb      (n/a)
     root://xrootd.lcg.triumf.ca:1094//atlas/atlasdatadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
     srm://srm.ndgf.org:8443/srm/managerv2?SFN=/atlas/disk/atlasdatadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
     gsiftp://gridftp.echo.stfc.ac.uk:2811/atlas:datadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1

arccp

The arccp command copies files to, from and between storage services.

[user ~]$ arccp /mnt/data/data.123 srm://se.example.org/dpm/example.org/home/myVO/data.123

[user ~]$ arccp rucio://rucio-lb-prod.cern.ch/replicas/mc15_13TeV/EVNT.19785567._000729.pool.root.1 /tmp/

arcmkdir

The arcmkdir command creates directories on storage elements and catalogs.

[user ~]$ arcmkdir srm://se.example.org/dpm/example.org/home/myVO/myData

arcrename

The arcrename command renames files on storage elements.

[user ~]$ arcrename srm://se.example.org/dpm/example.org/home/myVO/data.123 srm://se.example.org/dpm/example.org/home/myVO/data.124

arcrm

The arcrm command deletes files on storage elements.

[user ~]$ arcrm srm://se.example.org/dpm/example.org/home/myVO/data.124

Information services

arcinfo

The arcinfo command is used for obtaining the status of computing resources and detailed information about the resource according to published data in the information system.

[user ~]$ arcinfo arc.example.org
Computing service:  (production)
Information endpoint: https://arc.example.org:443/arex (org.nordugrid.arcrest)
Submission endpoint: https://arc.example.org:443/arex (status: ok, interface: org.nordugrid.arcrest)


[user ~]$ arcinfo -l arc.example.org | grep Implementation
    Implementation name: nordugrid-arc

arcsync

The arcsync command synchronizes your local jobs database with the information at a given computing element(s).

[user ~]$ $ arcsync -C arc.example.org
Synchronizing the local list of active jobs with the information in the
information system can result in some inconsistencies. Very recently submitted
jobs might not yet be present, whereas jobs very recently scheduled for
deletion can still be present.
Are you sure you want to synchronize your local job list? [y/n] y
Total number of new jobs found: 7