Installing ARC Client Tools
Warning
Information in this document is fully applicable to ARC 6.5.0 and above only!
Step 1. Enable NorduGrid ARC6 repos
Prepare your system to install via the NorduGrid Repositories.
If you need the latest client tools that are not yet released, configure nightly builds repository following Using ARC packages from nightly builds instructions.
Note
On the several operating systems you can install ARC without NorduGrid Repositories (e.g. using EPEL for CentOS/RHEL). But even on that system you will benefit from NorduGrid Repositories as they contains IGTF certificates and gets updated faster on new release.
Step 2. Install packages
Client tools
The main client tools package is nordugrid-arc-client
. Install it using your OS packet manager [1]:
[root ~]# yum -y install nordugrid-arc-client
This package brings necessary plugins as dependency and allows you to submit jobs and perform data transfers in the ARC ecosystem, e.g.:
submit jobs via EMI-ES interface defined by XRSL or ADL descriptions
query ARCHERY registry and LDAP information system
support local and HTTP(S) data transfers
support Rucio, SRM and ACIX
If you need more features, additional plugins are avaialable (see below).
ARCCTL
ARC Control Tool is one-stop-shop for sysadmins and users running ARC6 that automate many operations.
For the client side it will allows you to easiely deploy CA certificates and VOMS configuration:
[root ~]# yum -y install nordugrid-arc-arcctl
Warning
The ARC Control Tool is available for stand-alone usage without ARC CE since version 6.5. For ealier ARC versions CA certificates and VOMS configuration should be installed manually. You can find some hints in the old client installation instructions
Additional plugins
You can consider installing additional plugins for extra functionality:
Job submission:
arcrest
- submit jobs via ARC REST interface
gridftpjob
- submit jobs via GRIDFTPJOB interface (installsgridftp
as a dependency) [globus]
internal
- submit jobs via local filesystem Internal interface (requires A-REX on the same host)Data transfers:
gridftp
- support for gridftp data transfers [globus]
xrootd
- support for xroot protocol
s3
- support for s3 protocol
gfal
- add support for numerous data transfer protocols and file catalogues via installed GFAL2 plugins
[root ~]# yum -y install nordugrid-arc-plugins-<PLUGIN NAME>
Step 3. Setting up credentials
Currently users authentication in e-Science distrbuted computing networks (grid) heavily relies on cryptography and uses personal X.509 certificates/keys to identify entities and set of dedicated Certification Athorities (CA).
In the most common workflow you are also required to be a member of some Virtual Organization (VO) to get access to infrastructure resources. On the technical level the VO membership is currently handled by infrastructure VOMS services.
CA certificates bundle
CA certificates used to verify entities (e.g. users as well as compute and storage services) and should be installed on the client host as well for infrastructure security reasons.
In the distributed grid environment set of dedicated CAs used as a part of IGTF.
IGFT CA certificates bundle can be easiely installed with ARC Control Tool [2]:
[root ~]# arcctl deploy igtf-ca classic
In case ARC is not installed from the NorduGrid repositories, use --installrepo
argument to enable third-party repositories with IGTF CA certificates
Personal X.509 certificate
For production infrastructure usage you shoud obtain a personal certificate signed by one of the IGTF accredited CAs.
Typically there is at least one IGTF accreditage CA in each country that you can find on map.
Oganizational and technical procedures varies from CA to CA, so you should read the instructions on a choosen CA web-site.
Once you get your certificte, install it to your client host and ensure the permissions are set correctly.
For historical reasons the default location for certificate and key is .globus
unless redefined in client configuration file:
[user@client ~]$ ls -l ~/.globus/
total 12
-rw-r--r-- 1 user user 6353 Oct 1 11:55 usercert.pem
-r-------- 1 user user 1854 Oct 1 11:55 userkey.pem
Note
For testing and development purposes it is possible to use local testing CA. ARC Control Tool provides built-in test CA capabilities that allows you to bootstrap own test CA and generate host and user certificates.
Virtual Organization memberhip
In most cases you do have some implicit VO affiliation on your workplace. If not, you can search for VOs using e.g. EGI VO(s) search tool.
Every Virtual Organisation (VO) has own procedures and policies. Please contact VO support team for membership instructions.
To prove that you are a member of the particular VO you need to obtain a special token from the VOMS server as a part of your authentication process.
The arcproxy
tool will get this token automatically, but requires VOMS server enpoint details to be configured.
With ARC Control Tool you only need to know the VO name (for VOs in EGI database) or VOMS URL [3]:
[root ~]# arcctl deploy vomses --egi-vo atlas
[root ~]# arcctl deploy vomses --voms https://voms.ndgf.org:8443 --use-client-cert nordugrid.org
Most of VOMS-Admin services by default prohibit access without client certificate even to configuration page. If this is your case, the --use-client-cert
option will instruct arcctl
to use your personal certificate to establish connection.
Step 4. Try it out
When all is set, you can try to submit a job to the grid infrastructure.
Create proxy certificate
To submit a job, or perform any other action you need a so-called proxy-certificate which is a Single Sign-On token for distributed grid-infrastructure. It is generated in the following way:
[user ~]$ arcproxy
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Proxy generation succeeded
Your proxy is valid until: 2019-11-28 02:06:57
If you need to include you VO membership confirmation token (attribute certificate), specify the VO name as well:
[user ~]$ arcproxy -S area51
Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith
Contacting VOMS server (named area51): voms.example.org on port: 15001
Proxy generation succeeded
Your proxy is valid until: 2019-11-28 02:08:27
Submit test job
ARC client tools includes arctest
utility that come with several test jobs on board. So you can try to submit job without the need to write a job description.
You can use NorduGrid top-level ARCHERY service registry (nordugrid.org
) to find available CEs automatically [4]:
[user ~]$ arctest -J 2 --registry nordugrid.org
Submitting test-job 2:
&( executable = "/usr/bin/env" )( stdout = "stdout" )( stderr = "stdout" )( gmlog = "gmlog" )( jobname = "arctest2" )( clientxrsl = "&( executable = ""/usr/bin/env"" )( jobname = ""arctest2"" )( stdout = ""stdout"" )( join = ""yes"" )( gmlog = ""gmlog"" )" )
Client version: nordugrid-arc-6.5.0
Test submitted with jobid: https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
Computing service: KNU ARC
HINT: you can add -d INFO
to get mode dateails of CE selection process
The job status can be than checked with the arcstat
tool:
[user ~]$ arcstat https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
Job: https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
Name: arctest2
State: Running
Status of 1 jobs was queried, 1 jobs returned information
To fetch the job’s stdout run arccat
tool:
[user ~]$ arccat https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo
GRIDMAP=/dev/null
HOSTNAME=arc.example.org
TMPDIR=/tmp
<output omitted>