Overview of ARC client tools
This document contains general overview of available ARC client tools. For more detailed information about possible tool usage options, please refer to the --help
messages and corresponding man pages.
Submission endpoint types
short |
long |
legacy |
---|---|---|
emies |
org.nordugrid.emies |
org.ogf.glue.emies.activitycreation |
gridftpjob |
org.nordugrid.gridftpjob |
org.nordugrid.gridftpjob |
arcrest |
org.nordugrid.arcrest |
N/A |
internal |
org.nordugrid.internal |
org.nordugrid.internal |
Example of use:
arcsub -T emies -C some-ce-endpoint
Credentials
arcproxy
The arcproxy
command creates a proxy certificate from a key/certificate pair which can
then be used to access resources.
It is all-in-one tool for all kind of operations with proxy certificates, including VOMS extension generation and communications with MyProxy services.
[user ~]$ arcproxy -S myVO
Enter pass phrase for private key:
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named myVO): voms.example.org on port: 15004
Proxy generation succeeded
Your proxy is valid until: 2020-01-22 22:50:44
[user ~]$ arcproxy -I
Subject: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith/CN=2123720174
Issuer: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Time left for proxy: 11 hours 56 minutes 45 seconds
Proxy path: /tmp/x509up_u1000
Proxy type: X.509 Proxy Certificate Profile RFC compliant impersonation proxy - RFC inheritAll proxy
Proxy key length: 2048
Proxy signature: sha256
====== AC extension information for VO myVO ======
VO : myVO
subject : /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
issuer : /DC=org/DC=AccreditatedCA/O=hosts/CN=voms.example.org
uri : voms.example.org:15004
attribute : /myVO/Role=NULL/Capability=NULL
Time left for AC: 11 hours 56 minutes 47 seconds
[user ~]$ arcproxy -L myproxy.example.org -M put -S myVO -U myproxyuser
Enter pass phrase for private key:
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named myVO): voms.example.org on port: 15004
Proxy generation succeeded
Your proxy is valid until: 2020-01-29 10:51:52
Enter pass phrase for MyProxy server (new):
Verifying - Enter pass phrase for MyProxy server (new):
Succeeded to put a proxy onto MyProxy server
arcrenew
The arcrenew
command is used for renewing the proxy of jobs that have been submitted to computing resources.
[user ~]$ arcrenew https://arc.example.org:443/arex/7LqLDmKESEwnf5481mks8bjnABFKDmABFKDmhNFKDmHGFKDm9BpmKn
Jobs processed: 1, renewed: 1
Job submission and management
arcsub
The arcsub
command is used for submitting jobs to computing resources.
Note
Submission endpoint selection options are changed in ARC 6.5.0 release
[user ~]$ arcsub -T emies -C arc.example.org myJob.xrsl
Job submitted with jobid: https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
arcstat
The arcstat
command is used for obtaining the status of jobs that have
been submitted to computing resources.
[user ~]$ arcstat https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Job: https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Name: myJob
State: Running
Status of 1 jobs was queried, 1 jobs returned information
arccat
The arccat
command performs the cat command on the stdout, stderr or A-REX error log of the job.
It can also show the specified file from job’s session directory.
[user ~]$ arccat https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
SHELL=/bin/bash
HISTSIZE=1000
<output omitted>
[user ~]$ arccat -l https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
2020-01-22T09:57:45Z Job state change UNDEFINED -> ACCEPTED Reason: (Re)Accepting new job
2020-01-22T09:57:45Z Job state change ACCEPTED -> PREPARING Reason: Starting job processing
2020-01-22T09:57:45Z Job state change PREPARING -> SUBMIT Reason: Pre-staging finished, passing job to LRMS
----- starting submit_pbs_job -----
PBS jobname: myJob
<output omitted>
arckill
The arckill
command is used to kill running jobs.
[user ~]$ arckill https://arc.example.org:443/arex/wOOKDmT2REwnf5481mks8bjnABFKDmABFKDmhNFKDmWFFKDm1FSSRn
Jobs processed: 1, successfully killed: 1, successfully cleaned: 1
arcclean
The arcclean
command removes a job from the computing resource.
[user ~]$ arckill --keep https://arc.example.org:443/arex/UcBLDm32REwnf5481mks8bjnABFKDmABFKDmhNFKDmiFFKDm0zk4un
Jobs processed: 1, successfully killed: 1
[user ~]$ arcclean https://arc.example.org:443/arex/UcBLDm32REwnf5481mks8bjnABFKDmABFKDmhNFKDmiFFKDm0zk4un
Jobs processed: 1, deleted: 1
arcresub
The arcresub
allows to resubmit job to the same or other compute resources.
[user ~]$ arcresub --same https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Job submitted with jobid: https://arc.example.org:443/arex/KSeNDmz5REwnf5481mks8bjnABFKDmABFKDmhNFKDmsFFKDmp4MPWn
arcresume
The arcresume
command is used for resuming a job that was submitted to compute resources and then subsequently
failed. The job will be resumed at the last ok state reported by the cluster.
[user ~]$ arcresume https://arc.example.org:443/arex/vIGMDm58REwnf5481mks8bjnABFKDmABFKDmhNFKDm6FFKDmOBYWDm
Jobs processed: 1, resumed: 1
arcget
The arcget
command is used for retrieving the results from a job.
[user ~]$ arcget --keep https://arc.example.org:443/arex/5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Results stored at: 5SwNDmmwREwnf5481mks8bjnABFKDmABFKDmhNFKDmKFFKDmNPeLgn
Jobs processed: 1, successfully retrieved: 1
arctest
The arctest command is used for testing resources. It is able to:
submit several test job
prints info about installed user- and CA-certificates
Note
Submission endpoint selection options are changed in ARC 6.5.0 release
[user ~]$ arctest -J 2 -C arc.example.org
Job submitted with jobid: https://arc.example.org:443/arex/rRVLDmSBSEwnf5481mks8bjnABFKDmABFKDmhNFKDm8FFKDmGoZf3m
[user ~]$ arctest -E
Certificate information:
Certificate: /home/john/.globus/usercert.pem
Subject name: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Valid until: 2020-04-21 18:48:15
Proxy certificate information:
Proxy: /tmp/x509up_u1000
Proxy-subject: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith/CN=2123720174
Valid for: 11 hours 37 minutes 33 seconds
Certificate issuer: /DC=org/DC=AccreditatedCA
CA-certificates installed:
/DC=org/DC=AccreditatedCA
<output omitted>
Data manipulation
arcls
The arcls
command is used for listing files in storage elements and replica catalogs.
[user ~]$ arcls srm://se.example.org/dpm/example.org/home/myVO/
data.1
data.2
[user ~]$ arcls -lL rucio://rucio-lb-prod.cern.ch/replicas/mc15_13TeV/EVNT.19785567._000729.pool.root.1
<Name> <Type> <Size> <Modified> <CheckSum> <Latency>
EVNT.19785567._000729.pool.root.1 file 167468788 (n/a) adler32:f7332aeb (n/a)
root://xrootd.lcg.triumf.ca:1094//atlas/atlasdatadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
srm://srm.ndgf.org:8443/srm/managerv2?SFN=/atlas/disk/atlasdatadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
gsiftp://gridftp.echo.stfc.ac.uk:2811/atlas:datadisk/rucio/mc15_13TeV/ee/3b/EVNT.19785567._000729.pool.root.1
arccp
The arccp
command copies files to, from and between storage services.
[user ~]$ arccp /mnt/data/data.123 srm://se.example.org/dpm/example.org/home/myVO/data.123
[user ~]$ arccp rucio://rucio-lb-prod.cern.ch/replicas/mc15_13TeV/EVNT.19785567._000729.pool.root.1 /tmp/
arcmkdir
The arcmkdir
command creates directories on storage elements and catalogs.
[user ~]$ arcmkdir srm://se.example.org/dpm/example.org/home/myVO/myData
arcrename
The arcrename
command renames files on storage elements.
[user ~]$ arcrename srm://se.example.org/dpm/example.org/home/myVO/data.123 srm://se.example.org/dpm/example.org/home/myVO/data.124
arcrm
The arcrm
command deletes files on storage elements.
[user ~]$ arcrm srm://se.example.org/dpm/example.org/home/myVO/data.124
Information services
arcinfo
The arcinfo
command is used for obtaining the status of computing resources and detailed information about the resource according to published data in the information system.
The summary optput includes the information endpoint type starting from ARC 6.5.0.
[user ~]$ arcinfo arc.example.org
Computing service: (production)
Information endpoint: ldap://arc.example.org:2135/Mds-Vo-Name=local,o=grid (org.nordugrid.ldapng)
Information endpoint: ldap://arc.example.org:2135/o=glue (org.nordugrid.ldapglue2)
Information endpoint: https://arc.example.org:443/arex (org.nordugrid.arcrest)
Information endpoint: https://arc.example.org:443/arex (org.ogf.glue.emies.resourceinfo)
Submission endpoint: https://arc.example.org:443/arex (status: ok, interface: org.nordugrid.arcrest)
Submission endpoint: https://arc.example.org:443/arex (status: ok, interface: org.ogf.glue.emies.activitycreation)
Submission endpoint: gsiftp://arc.example.org:2811/jobs (status: ok, interface: org.nordugrid.gridftpjob)
[user ~]$ arcinfo -l arc.example.org | grep Implementation
Implementation name: nordugrid-arc-6.5.0
arcsync
The arcsync
command synchronizes your local jobs database with the information at a given computing element(s).
[user ~]$ $ arcsync -C arc.example.org
Synchronizing the local list of active jobs with the information in the
information system can result in some inconsistencies. Very recently submitted
jobs might not yet be present, whereas jobs very recently scheduled for
deletion can still be present.
Are you sure you want to synchronize your local job list? [y/n] y
Total number of new jobs found: 7