Deploying ARCHERY for Country/Organization¶
Organizations and/or countries that maintain a distributed computing infrastructure based on Nordugrid ARC middleware are expected to deploy their own instances of ARCHERY to support distributed computing workflows of their infrastructure resource consumers.
Note
Administrative access to the DNS server is required during the initial bootstrap of the DNS-based ARCHERY registry.
Further data maintenance inside the registry DOES NOT require administrative access.
This document shows the example configuration of an ARCHERY registry in the DNS zone owned by an organization. This applies to all kind of setup flavors, e.g. country-level, project/experiment, virtual organization, etc.
Note
Configuration examples in this guide are provided for BIND, however you can use any name server implementation configured in a similar way.
Entry point¶
An entry point to an ARCHERY-based registry is a domain name. In this guide example.org
is used as an entry point.
To submit a job using ARCHERY run:
[user ~]$ arcsub -g example.org myjob.xrsl
The nordugrid.org
represents the whole Nordugrid infrastructure entry point. It holds references to the country-level
endpoints named upon country code top-level domain (ccTLD) identifiers: <ccTLD>.archery.nordugrid.org
Transaction signature key to manage endpoints¶
Though it is possible to manage records manually, it is strongly advised to use Dynamic DNS updates with archery-manage
.
This eliminates typos, maintains up-to-date information and keeps it simple to manage endpoints without administrative access to the DNS server itself.
Generate key¶
Use the following command to generate the key:
[user ~]$ dnssec-keygen -a HMAC-MD5 -b 256 -n USER archery
From the generated files you need a secret part to be included in both BIND and archery-manage
configuration.
Define key in BIND¶
Create the /etc/named/archery.key
and put the secret key generated in the previous step here:
key archery_key {
algorithm hmac-md5;
secret "S0Me+SecRet+keYgener@tedwithdnssec==";
};
Include the key definition into named.conf
using the config line:
include "/etc/named/archery.key";
Keyfile for archery-manage¶
Create a file (let it be named archery-manage.key
) with the generated key in the following format:
archery_key:S0Me+SecRet+keYgener@tedwithdnssec==
Configure DNS zone to host endpoints¶
It is possible to put records directly into the example.org
zone, however for manageability and security [1]
reasons it is advised to have dedicated zone configured for ARCHERY records.
[1] | The content will be updated dynamically and it is better to eliminate the possiblity of changing non-archery related records. A dedicated subzone is the easiest way. |
You can use any name for dedicated zone, this example uses index.example.org
. [2]
[2] | Use <ccTLD>.archery.nordugrid.org if you are setting up country-level index for the Nordugrid infrastructure |
Addresses used in the following examples:
- Primary (master) DNS:
ns1.example.org (192.0.2.100)
- Seondary (slave) DNS:
ns2.example.org (192.0.2.200)
Define zone in DNS¶
Add zone definition to master named.conf
:
zone "index.example.org." IN {
type master;
file "master/index.example.org.db";
notify yes;
also-notify {
# slave DNS IP address
192.0.2.200;
};
allow-transfer {
# slave DNS IP address
192.0.2.200;
};
allow-update {
key archery_key;
};
};
Please note allow-update
directive that authorize dynamic DNS update queries signed by archery-manage
key.
The secondary DNS should be configured without any special options:
zone "index.example.org." IN {
type slave;
file "slave/index.example.org.db";
masters {
192.0.2.100;
};
allow-transfer {
192.0.2.100;
};
};
Create zonefile with basic structure¶
Zonefile requires only basic SOA
record and will be filled with data by archery-manage
.
The zonefile template (timers are subject to arrange depending on planned update frequency):
$ORIGIN example.org.
$TTL 3600
index IN SOA ns1.example.org. hostmaster.example.org. (
2018082401 ; serial
1200 ; refresh (20 minutes)
180 ; retry (3 minutes)
604800 ; expire (1 week)
60 ; minimum (1 minute)
)
NS ns1.example.org.
NS ns2.example.org.
Define records in parent zone¶
Note
If you setup a country-level index for the Nordugrid infrastructure such records are defined in the parent nordugrid.org
zone.
Please provide your setup information for the top-level index instead of this section.
Define NS
records [3] to refer defined subzone:
$ORIGIN example.org.
# dedicated ARCHERY zone
index NS ns1.example.org.
index NS ns2.example.org.
[3] | If you plan to use a different out-of-scope domain names in NS don’t forget to add glue A records. |
To create an ARCHERY entry point in the parent zone you can:
define
CNAME
record to useexample.org
as an entry point:$ORIGIN example.org. # ARCHERY entry point _archery CNAME _archery.indexOR define
TXT
resource record with ARCHERY data pointing to group:$ORIGIN example.org. # ARCHERY entry point _archery TXT “u=index.example.org t=archery.group”
The same technique can be used to define any other entry points (e.g. in the complete different domain). CNAME
is recommended if you point to only one ARCHERY group.
Define static list of Computing Elements¶
The list of CE hostnames used by archery-manage
to fetch the endpoints from ARC CE directly [4] and push it to DNS zone with an incremental DDNS update.
CE hostanames should be defined line-by-line in a plain text file:
ce01.example.org
ce02.example.org
arc6.example.org
[4] | At the moment infosys LDAP GLUE2 should be enabled on ARC CE |
Populate DNS with information¶
To populate DNS zone with endpoint information you should run archery-manage
in the following way:
[user ~]$ archery-manage -s arcce-list:ces.list --ddns-update \
--domain index.example.org --ddns-master-ip 192.0.2.100 \
--ddns-tsig-keyfile archery-manage.key
Note
Updates are performed over the network, so you can run archery-manage
on any host. It SHOULD NOT be the DNS server itself.
Consider to add filters to archery-manage
:
Filter: Port connectivity¶
Check network connectivity to endpoint TCP port and filter endpoints that do not pass this test:
[user ~]$ archery-manage -s arcce-list:ce.list -f portscan ...
Filter: Endpoint type¶
By default any endpoints available on the CE are published. If you want to filter endpoints based on type there is a dedicated filter.
For example, if you want only EMI-ES ResourceInfo endpoints for EMI-ES only operation:
[user ~]$ archery-manage -s arcce-list:ce.list -f type:org.ogf.glue.emies.resourceinfo ...
EMI-ES ResourceInfo and LDAP GLUE2 endpoints:
[user ~]$ archery-manage -s arcce-list:ce.list -f type:org.ogf.glue.emies.resourceinfo,org.nordugrid.ldapglue2 ...
There is a special endpoint filter resourceinfo
that is equivalent to EMI-ES ResourceInfo, LDAP GLUE2 and LDAP Nordugrid.
Filter: VO¶
For project-based ARCHERY deployment it is also useful to filter endpoints based on VO access policy. Only endpoints that advertise specified VO support will be added [5]:
[user ~]$ archery-manage -s arcce-list:ce.list -f vo:exampleVO ...
[5] | Resource information (GLUE2PolicyRule in GLUE2 and nordugrid-cluster-acl in Nordugrid LDAP) will be used as a source of supported VOs |
Checking the operation¶
General DNS queries¶
Any DNS client tool can be used, for example:
[uaer ~]$ host -t TXT _archery.index.moldyngrid.org
_archery.index.moldyngrid.org descriptive text "u=dns://d91b974761._archery.index.moldyngrid.org. t=archery.service"
_archery.index.moldyngrid.org descriptive text "u=dns://d9b05b0008._archery.index.moldyngrid.org. t=archery.service s=0"
<output omitted>
[user ~]$ host -t TXT d91b974761._archery.index.moldyngrid.org.
d91b974761._archery.index.moldyngrid.org descriptive text "o=service t=org.nordugrid.arex id=arc.univ.kiev.ua"
d91b974761._archery.index.moldyngrid.org descriptive text "u=https://arc.univ.kiev.ua:443/arex t=org.ogf.glue.emies.activitycreation"
d91b974761._archery.index.moldyngrid.org descriptive text "u=ldap://arc.univ.kiev.ua:2135/o=glue t=org.nordugrid.ldapglue2"
<output omitted>
Submit a job¶
Use arcsub -g example.org -d DEBUG
to check the process of endpoints discovery from ARCHERY.
Setup regular updates to ARCHERY¶
To keep information about endpoints up to date setup a CRON job or Systemd Timer to run archery-manage
regularly.
In combination with at least port filtering this allows to eliminate stale endpoints and actualize information in registry.
Configure LDAP-monitor to use ARCHERY¶
NorduGrid LDAP monitor support fetching endpoints [6] from ARCHERY with the following configuration in settings.inc
:
<?php
$archery_list = array (
array (
"endpoint" => "example.org",
)
);
?>
[6] | Nordugrid monitor only works with org.nordugrid.ldapng data (and LDAP GLUE2 is experimental with known issues), so Nordugrid LDAP schema publishing is mandatory to use monitor. |