Setting up ARCHERY for small research community

This tutorial demonstrates the registry setup for a small community.

The following scenario defines the use-case: a research community is using a set of ARC-enabled resources (HPC/grid/cloud) to run computations. There is no central job-server, but reseachers want to submit jobs to all available ARC CEs, without the need to choose and specify a particular CE for each job.

With ARCHERY we can achieve this without introducing an extra service, relying solely on DNS.

Please refer to the ARCHERY documentation for more details beyond the scope of this tutorial.

Prerequisites

• Basic knowledge of DNS records

• Have a set of ARC-CE’s already configured

Example infrastructure

Our example infrastructure topology is flat and consist of 3 ARC CEs:

Small research community e-Infrastructure topology - a set of ARC-CE’s forming a community grid “myorg”.

A researcher submitting a job from the remote client machine will use ARCHERY as a unified endpoint to submit jobs to any of the ARC-CE’s in the grid (3 in our example), without the need of knowing what resources are available and choosing a particular resource.

Define ARCHERY in DNS

This tutorial uses myorg.example.org as the DNS domain name of the registry.

Step 1 - Collect ARC CE endpoints information

We will assume ARC 7, and are therefore only interested in the arcrest endpoints. This reduces the required endpoints to just 1 per ARC CE and makes the task of defining ARCHERY records simpler.

You might know the endpoint for ARC CE configuration already, but otherwise you can use arcinfo to get it:

[user@client ~]# arcinfo ce01.myorg.example.org
Computing service: ce01.myorg.example.org (production)
  Information endpoint: https://ce01.myorg.example.org:443/arex (org.ogf.glue.emies.resourceinfo)
  Information endpoint: https://ce01.myorg.example.org:443/arex (org.nordugrid.arcrest)
  Submission endpoint: https://ce01.myorg.example.org:443/arex (status: ok, interface: org.ogf.glue.emies.activitycreation)
  Submission endpoint: https://ce01.myorg.example.org:443/arex (status: ok, interface: org.nordugrid.arcrest)

We are interested in the org.nordugrid.arcrest endpoint. So our endpoint list for the example topology is:

• https://ce01.myorg.example.org:443/arex

• https://ce02.myorg.example.org:443/arex

• https://arc7.anotherexample.no:443/arex

Step 2 - Construct DNS records

According to the ARCHERY data model, this topology can be described with:

1. An Entry point Group Object (representing our set of CE’s). DNS name should be _archery.myorg.example.org. It will contain pointers to Service Objects for every ARC CE.

2. Service Objects for each ARC CE. The DNS name is arbitrary and for simplicity we just name them ce01.myorg.example.org, ce02.myorg.example.org and ce03.myorg.example.org.

DNS records for the example topology are therefore as follows:

Name                              Record-type Record
-----------------------------------------------------------------------------------------------------------------
_archery.myorg.example.org.       TXT         "u=dns://ce01.myorg.example.org. t=archery.service"
_archery.myorg.example.org.       TXT         "u=dns://ce02.myorg.example.org. t=archery.service"
_archery.myorg.example.org.       TXT         "u=dns://ce03.myorg.example.org. t=archery.service"

ce01.myorg.example.org.           TXT         "o=service t=org.nordugrid.arex id=ce01.myorg.example.org"
ce01.myorg.example.org.           TXT         "u=https://ce01.myorg.example.org:443/arex t=org.nordugrid.arcrest"

ce02.myorg.example.org.           TXT          "o=service t=org.nordugrid.arex id=ce02.myorg.example.org"
ce02.myorg.example.org.           TXT          "u=https://ce02.myorg.example.org:443/arex t=org.nordugrid.arcrest"

ce03.myorg.example.org.           TXT          "o=service t=org.nordugrid.arex id=arc7.anotherexample.no"
ce03.myorg.example.org.           TXT          "u=https://arc7.anotherexample.no:443/arex t=org.nordugrid.arcrest"

Step 3 - Add records to DNS

In this example we will add DNS records through an openstack DNS API. Depending on your DNS management interface available, you need to adjust accordingly.

To add DNS records for the myorg.example.org entry point _archery group:

[user ~]$ openstack recordset create --type TXT myorg.example.org. _archery.myorg.example.org. \
   --record "\"u=dns://ce01.myorg.example.org. t=archery.service\"" \
   --record "\"u=dns://ce02.myorg.example.org. t=archery.service\"" \
   --record "\"u=dns://ce03.myorg.example.org. t=archery.service\""

Similarly add DNS records for each of the three ARC CE services, defining the REST endpoints:

[user ~]$ openstack recordset create --type TXT  myorg.example.org. ce01.myorg.example.org. \
  --record "\"o=service t=org.nordugrid.arex id=ce01.myorg.example.org\""
  --record "\"u=https://ce01.myorg.example.org:443/arex t=org.nordugrid.arcrest\""

[user ~]$ openstack recordset create --type TXT  myorg.example.org. ce02.myorg.example.org. \
  --record "\"o=service t=org.nordugrid.arex id=ce02.myorg.example.org\""
  --record "\"u=https://ce02.myorg.example.org:443/arex t=org.nordugrid.arcrest\""

[user ~]$ openstack recordset create --type TXT  myorg.example.org. ce03.myorg.example.org. \
  --record "\"o=service t=org.nordugrid.arex id=arc7.anotherexample.no\"" \
  --record "\"u=https://arc7.anotherexample.no:443/arex t=org.nordugrid.arcrest\""

That is all that is needed to define small community registry to use distributed ARC CE resources.

Test submission using ARCHERY endpoint

Now you can use the --registry option in arcsub to submit your jobs to any of the ARC-CE from your small community grid:

[user ~]$ arcsub --registry myorg.example.org myjob.xrls
Job submitted with jobid: https://ce02.myorg.example.org:443/arex/rest/1.0/jobs/361a038bdc51

[user ~]$ arcsub --registry myorg.example.org myjob.xrls
Job submitted with jobid: https://ce01.myorg.example.org:443/arex/rest/1.0/jobs/36171e49dc51

[user ~]$ arcsub --registry myorg.example.org myjob.xrls
Job submitted with jobid: https://arc7.anotherexample.no:443/arex/rest/1.0/jobs/361db27023c6

(Optional alternative) Use archery-manage instead of Step 1 and 2

The archery-manage tool aims to automate DNS management of ARCHERY records on scale. It supports dynamic DNS updates and addresses more complex cases beyond the small community infrastructure with just a couple of ARC CEs.

In this example we are using archery-manage only as a help-tool to discover service endpoints and output the necessary DNS records that we in turn will feed into our DNS service. It is up to you to decide whether the manual or archery-manage way is a better fit for you.

The tool is distributed along other ARC packages, so you just need to install it with (for example on RHEL):

[root ~]# dnf install -y nordugrid-arc-archery-manage

First you need to define the list of ARC CEs you want to include, one CE per line. In our exampe it is:

[user ~]# cat celist.txt
ce01.myorg.example.org
ce02.myorg.example.org
arc7.anotherexample.no

Then you simply need to run archery-manage in records output mode.

Note

Currently archery-manage uses the LDAP service on the ARC-CE for endpoints discovery, so it has to be enabled to use this method. In the future, archery-manage will use the REST interface as well without the need for LDAP.

Exact command for our example use-case is:

[root ~]# archery-manage -s arcce-list:celist.txt -o zonefile -f type:org.nordugrid.arcrest --domain myorg.example.org
2b40751af1.myorg.example.org.     TXT "o=service t=org.nordugrid.arex id=ce02.myorg.example.org"
2b40751af1.myorg.example.org.     TXT "u=https://ce02:443/arex t=org.nordugrid.arcrest"
2e670cff3b.myorg.example.org.     TXT "o=service t=org.nordugrid.arex id=arc7.anotherexample.no"
2e670cff3b.myorg.example.org.     TXT "u=https://arc7.anotherexample.no:443/arex t=org.nordugrid.arcrest"
53f2071f0a.myorg.example.org.     TXT "o=service t=org.nordugrid.arex id=ce01.myorg.example.org"
53f2071f0a.myorg.example.org.     TXT "u=https://ce01:443/arex t=org.nordugrid.arcrest"
_archery.myorg.example.org.       TXT "u=dns://2b40751af1.myorg.example.org. t=archery.service"
_archery.myorg.example.org.       TXT "u=dns://2e670cff3b.myorg.example.org. t=archery.service"
_archery.myorg.example.org.       TXT "u=dns://53f2071f0a.myorg.example.org. t=archery.service"

Notice we use the option -f in the example to filter for only REST endpoints. You can add other endpoints as well if you like.

This gives us the constructed DNS records we need to add to DNS service. And we can now either add these manually as in the Step 3 above, or user archery-manage with the --ddns-update option to update the DNS, if you have access to do so.

Navigation

• Go back to the Overview