Deploying ARCHERY for Country/Organization

Organizations and/or countries that maintain a distributed computing infrastructure based on Nordugrid ARC middleware are expected to deploy their own instances of ARCHERY to support distributed computing workflows of their infrastructure resource consumers.

Note

Administrative access to the DNS server is required during the initial bootstrap of the DNS-based ARCHERY registry.

Further data maintenance inside the registry DOES NOT require administrative access.

This document shows the example configuration of an ARCHERY registry in the DNS zone owned by an organization. This applies to all kind of setup flavors, e.g. country-level, project/experiment, virtual organization, etc.

Note

Configuration examples in this guide are provided for BIND, however you can use any name server implementation configured in a similar way.

Entry point

An entry point to an ARCHERY-based registry is a domain name. In this guide example.org is used as an entry point. To submit a job using ARCHERY run:

[user ~]$ arcsub -g example.org myjob.xrsl

The nordugrid.org represents the whole Nordugrid infrastructure entry point. It holds references to the country-level endpoints named upon country code top-level domain (ccTLD) identifiers: <ccTLD>.archery.nordugrid.org

Transaction signature key to manage endpoints

Though it is possible to manage records manually, it is strongly advised to use Dynamic DNS updates with archery-manage.

This eliminates typos, maintains up-to-date information and keeps it simple to manage endpoints without administrative access to the DNS server itself.

Generate key

Use the following command to generate the key:

[user ~]$ dnssec-keygen -a HMAC-MD5 -b 256 -n USER archery

From the generated files you need a secret part to be included in both BIND and archery-manage configuration.

Define key in BIND

Create the /etc/named/archery.key and put the secret key generated in the previous step here:

key archery_key {
      algorithm hmac-md5;
      secret "S0Me+SecRet+keYgener@tedwithdnssec==";
};

Include the key definition into named.conf using the config line:

include "/etc/named/archery.key";

Keyfile for archery-manage

Create a file (let it be named archery-manage.key) with the generated key in the following format:

archery_key:S0Me+SecRet+keYgener@tedwithdnssec==

Configure DNS zone to host endpoints

It is possible to put records directly into the example.org zone, however for manageability and security [1] reasons it is advised to have dedicated zone configured for ARCHERY records.

[1]The content will be updated dynamically and it is better to eliminate the possiblity of changing non-archery related records. A dedicated subzone is the easiest way.

You can use any name for dedicated zone, this example uses index.example.org. [2]

[2]Use <ccTLD>.archery.nordugrid.org if you are setting up country-level index for the Nordugrid infrastructure

Addresses used in the following examples:

  • Primary (master) DNS: ns1.example.org (192.0.2.100)
  • Seondary (slave) DNS: ns2.example.org (192.0.2.200)

Define zone in DNS

Add zone definition to master named.conf:

zone "index.example.org." IN {
    type master;
    file "master/index.example.org.db";
    notify yes;
    also-notify {
        # slave DNS IP address
        192.0.2.200;
    };
    allow-transfer {
        # slave DNS IP address
        192.0.2.200;
    };
    allow-update {
        key archery_key;
    };
};

Please note allow-update directive that authorize dynamic DNS update queries signed by archery-manage key.

The secondary DNS should be configured without any special options:

zone "index.example.org." IN {
    type slave;
    file "slave/index.example.org.db";
    masters {
        192.0.2.100;
    };
    allow-transfer {
        192.0.2.100;
    };
};

Create zonefile with basic structure

Zonefile requires only basic SOA record and will be filled with data by archery-manage. The zonefile template (timers are subject to arrange depending on planned update frequency):

$ORIGIN example.org.
$TTL 3600
index   IN SOA    ns1.example.org. hostmaster.example.org. (
             2018082401 ; serial
             1200       ; refresh (20 minutes)
             180        ; retry (3 minutes)
             604800     ; expire (1 week)
             60         ; minimum (1 minute)
             )
        NS    ns1.example.org.
        NS    ns2.example.org.

Define records in parent zone

Note

If you setup a country-level index for the Nordugrid infrastructure such records are defined in the parent nordugrid.org zone. Please provide your setup information for the top-level index instead of this section.

Define NS records [3] to refer defined subzone:

$ORIGIN example.org.
# dedicated ARCHERY zone
index   NS ns1.example.org.
index   NS ns2.example.org.
[3]If you plan to use a different out-of-scope domain names in NS don’t forget to add glue A records.

To create an ARCHERY entry point in the parent zone you can:

  • define CNAME record to use example.org as an entry point:

    $ORIGIN example.org.
    # ARCHERY entry point
    _archery CNAME _archery.index
    
  • OR define TXT resource record with ARCHERY data pointing to group:

    $ORIGIN example.org.
    # ARCHERY entry point
    _archery TXT “u=index.example.org t=archery.group”
    

The same technique can be used to define any other entry points (e.g. in the complete different domain). CNAME is recommended if you point to only one ARCHERY group.

Define static list of Computing Elements

The list of CE hostnames used by archery-manage to fetch the endpoints from ARC CE directly [4] and push it to DNS zone with an incremental DDNS update.

CE hostanames should be defined line-by-line in a plain text file:

ce01.example.org
ce02.example.org
arc6.example.org
[4]At the moment infosys LDAP GLUE2 should be enabled on ARC CE

Populate DNS with information

To populate DNS zone with endpoint information you should run archery-manage in the following way:

[user ~]$ archery-manage -s arcce-list:ces.list --ddns-update \
            --domain index.example.org --ddns-master-ip 192.0.2.100 \
            --ddns-tsig-keyfile archery-manage.key

Note

Updates are performed over the network, so you can run archery-manage on any host. It SHOULD NOT be the DNS server itself.

Consider to add filters to archery-manage:

Filter: Port connectivity

Check network connectivity to endpoint TCP port and filter endpoints that do not pass this test:

[user ~]$ archery-manage -s arcce-list:ce.list -f portscan ...

Filter: Endpoint type

By default any endpoints available on the CE are published. If you want to filter endpoints based on type there is a dedicated filter.

For example, if you want only EMI-ES ResourceInfo endpoints for EMI-ES only operation:

[user ~]$ archery-manage -s arcce-list:ce.list -f type:org.ogf.glue.emies.resourceinfo ...

EMI-ES ResourceInfo and LDAP GLUE2 endpoints:

[user ~]$ archery-manage -s arcce-list:ce.list -f type:org.ogf.glue.emies.resourceinfo,org.nordugrid.ldapglue2 ...

There is a special endpoint filter resourceinfo that is equivalent to EMI-ES ResourceInfo, LDAP GLUE2 and LDAP Nordugrid.

Filter: VO

For project-based ARCHERY deployment it is also useful to filter endpoints based on VO access policy. Only endpoints that advertise specified VO support will be added [5]:

[user ~]$ archery-manage -s arcce-list:ce.list -f vo:exampleVO ...
[5]Resource information (GLUE2PolicyRule in GLUE2 and nordugrid-cluster-acl in Nordugrid LDAP) will be used as a source of supported VOs

Checking the operation

General DNS queries

Any DNS client tool can be used, for example:

[uaer ~]$ host -t TXT _archery.index.moldyngrid.org
_archery.index.moldyngrid.org descriptive text "u=dns://d91b974761._archery.index.moldyngrid.org. t=archery.service"
_archery.index.moldyngrid.org descriptive text "u=dns://d9b05b0008._archery.index.moldyngrid.org. t=archery.service s=0"
<output omitted>

[user ~]$ host -t TXT d91b974761._archery.index.moldyngrid.org.
d91b974761._archery.index.moldyngrid.org descriptive text "o=service t=org.nordugrid.arex id=arc.univ.kiev.ua"
d91b974761._archery.index.moldyngrid.org descriptive text "u=https://arc.univ.kiev.ua:443/arex t=org.ogf.glue.emies.activitycreation"
d91b974761._archery.index.moldyngrid.org descriptive text "u=ldap://arc.univ.kiev.ua:2135/o=glue t=org.nordugrid.ldapglue2"
<output omitted>

Submit a job

Use arcsub -g example.org -d DEBUG to check the process of endpoints discovery from ARCHERY.

Setup regular updates to ARCHERY

To keep information about endpoints up to date setup a CRON job or Systemd Timer to run archery-manage regularly. In combination with at least port filtering this allows to eliminate stale endpoints and actualize information in registry.

Configure LDAP-monitor to use ARCHERY

NorduGrid LDAP monitor support fetching endpoints [6] from ARCHERY with the following configuration in settings.inc:

<?php
$archery_list = array (
  array (
    "endpoint" => "example.org",
  )
);
?>
[6]Nordugrid monitor only works with org.nordugrid.ldapng data (and LDAP GLUE2 is experimental with known issues), so Nordugrid LDAP schema publishing is mandatory to use monitor.