Installing ARC Client Tools¶
Information in this document is fully applicable to ARC 6.5.0 and above only!
Step 1. Enable NorduGrid ARC6 repos¶
Prepare your system to install via the NorduGrid Repositories.
If you need the latest client tools that are not yet released, configure nightly builds repository following Using ARC packages from nightly builds instructions.
On the several operating systems you can install ARC without NorduGrid Repositories (e.g. using EPEL for CentOS/RHEL). But even on that system you will benefit from NorduGrid Repositories as they contains IGTF certificates and gets updated faster on new release.
Step 2. Install packages¶
The main client tools package is
nordugrid-arc-client. Install it using your OS packet manager :
[root ~]# yum -y install nordugrid-arc-client
|||Examples are shown for YUM-based systems. You can use APT or any other packet managers in the similar way.|
This package brings necessary plugins as dependency and allows you to submit jobs and perform data transfers in the ARC ecosystem, e.g.:
- submit jobs via EMI-ES interface defined by XRSL or ADL descriptions
- query ARCHERY registry and LDAP information system
- support local and HTTP(S) data transfers
- support Rucio, SRM and ACIX
If you need more features, additional plugins are avaialable (see below).
arcctl is one-stop-shop for sysadmins and users running ARC6 that automate many operations.
For the client side it will allows you to easiely deploy CA certificates and VOMS configuration:
[root ~]# yum -y install nordugrid-arc-arcctl
The arcctl is available for stand-alone usage without ARC CE since version 6.5. For ealier ARC versions CA certificates and VOMS configuration should be installed manually. You can find some hints in the old client installation instructions
You can consider installing additional plugins for extra functionality:
- Job submission:
arcrest- submit jobs via ARC REST interface
gridftpjob- submit jobs via GRIDFTPJOB interface (installs
gridftpas a dependency) [globus]
internal- submit jobs via local filesystem Internal interface (requires A-REX on the same host)
- Data transfers:
gridftp- support for gridftp data transfers [globus]
xrootd- support for xroot protocol
s3- support for s3 protocol
gfal- add support for numerous data transfer protocols and file catalogues via installed GFAL2 plugins
|[globus]||(1, 2) In the ARC < 6.5 there was an all-in-one plugin |
[root ~]# yum -y install nordugrid-arc-plugins-<PLUGIN NAME>
Step 3. Setting up credentials¶
Currently users authentication in e-Science distrbuted computing networks (grid) heavily relies on cryptography and uses personal X.509 certificates/keys to identify entities and set of dedicated Certification Athorities (CA).
In the most common workflow you are also required to be a member of some Virtual Organization (VO) to get access to infrastructure resources. On the technical level the VO membership is currently handled by infrastructure VOMS services.
CA certificates bundle¶
CA certificates used to verify entities (e.g. users as well as compute and storage services) and should be installed on the client host as well for infrastructure security reasons.
In the distributed grid environment set of dedicated CAs used as a part of IGTF.
IGFT CA certificates bundle can be easiely installed with arcctl :
[root ~]# arcctl deploy igtf-ca classic
|||In case ARC is not installed from the NorduGrid repositories, use |
Personal X.509 certificate¶
For production infrastructure usage you shoud obtain a personal certificate signed by one of the IGTF accredited CAs.
Typically there is at least one IGTF accreditage CA in each country that you can find on map.
Oganizational and technical procedures varies from CA to CA, so you should read the instructions on a choosen CA web-site.
Once you get your certificte, install it to your client host and ensure the permissions are set correctly.
For historical reasons the default location for certificate and key is
.globus unless redefined in client configuration file:
[user@client ~]$ ls -l ~/.globus/ total 12 -rw-r--r-- 1 user user 6353 Oct 1 11:55 usercert.pem -r-------- 1 user user 1854 Oct 1 11:55 userkey.pem
For testing and development purposes it is possible to use local testing CA. arcctl provides built-in test CA capabilities that allows you to bootstrap own test CA and generate host and user certificates.
Virtual Organization memberhip¶
In most cases you do have some implicit VO affiliation on your workplace. If not, you can search for VOs using e.g. EGI VO(s) search tool.
Every Virtual Organisation (VO) has own procedures and policies. Please contact VO support team for membership instructions.
To prove that you are a member of the particular VO you need to obtain a special token from the VOMS server as a part of your authentication process.
arcproxy tool will get this token automatically, but requires VOMS server enpoint details to be configured.
With arcctl you only need to know the VO name (for VOs in EGI database) or VOMS URL :
[root ~]# arcctl deploy vomses --egi-vo atlas [root ~]# arcctl deploy vomses --voms https://voms.ndgf.org:8443 --use-client-cert nordugrid.org
|||Most of VOMS-Admin services by default prohibit access without client certificate even to configuration page. If this is your case, the |
Step 4. Try it out¶
When all is set, you can try to submit a job to the grid infrastructure.
Create proxy certificate¶
To submit a job, or perform any other action you need a so-called proxy-certificate which is a Single Sign-On token for distributed grid-infrastructure. It is generated in the following way:
[user ~]$ arcproxy Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith Proxy generation succeeded Your proxy is valid until: 2019-11-28 02:06:57
If you need to include you VO membership confirmation token (attribute certificate), specify the VO name as well:
[user ~]$ arcproxy -S area51 Your identity: /DC=org/DC=AccreditatedCA/O=people/CN=John Smith Contacting VOMS server (named area51): voms.example.org on port: 15001 Proxy generation succeeded Your proxy is valid until: 2019-11-28 02:08:27
Submit test job¶
ARC client tools includes
arctest utility that come with several test jobs on board. So you can try to submit job without the need to write a job description.
You can use NorduGrid top-level ARCHERY service registry (
nordugrid.org) to find available CEs automatically :
[user ~]$ arctest -J 2 --registry nordugrid.org Submitting test-job 2: &( executable = "/usr/bin/env" )( stdout = "stdout" )( stderr = "stdout" )( gmlog = "gmlog" )( jobname = "arctest2" )( clientxrsl = "&( executable = ""/usr/bin/env"" )( jobname = ""arctest2"" )( stdout = ""stdout"" )( join = ""yes"" )( gmlog = ""gmlog"" )" ) Client version: nordugrid-arc-6.5.0 Test submitted with jobid: https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo Computing service: KNU ARC
|||HINT: you can add |
The job status can be than checked with the
[user ~]$ arcstat https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo Job: https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo Name: arctest2 State: Running Status of 1 jobs was queried, 1 jobs returned information
To fetch the job’s stdout run
[user ~]$ arccat https://arc.example.org:443/arex/oIlKDmiOCuvnjwO5upha6lOqABFKDmABFKDmEFHKDmPBFKDmUYtvNo GRIDMAP=/dev/null HOSTNAME=arc.example.org TMPDIR=/tmp <output omitted>