Tutorial Part 3 - Production ready setup with remote ARC datadelivery service

For this part of the tutorial you need at least one remove datadelivery server set up. It makes sense that the session directory and the cache directory is hosted on this server and shared to the ARC-CE and the compute nodes.

The EGI tutorial test clusters are set up in this way.

Install ARC Datadelivery service

Setup nightly repo

On the remote data delivery server, install the nordugrid nightly repo following the same procedure as for the ARC-CE, namely this one.

Install ARC packages

dnf install -y nordugrid-arc-arcctl nordugrid-arc-datadelivery-service nordugrid-arc-plugins-needed
arcctl deploy igtf-ca classic --installrepo igtf

Configure ARC Datadelivery service

To set up remote datadelivery we must configure both the ARC-CE and the remote ARC datadelivery server.

On the remote Data Delivery server

On the remote data delivery server you must create the following minimal arc.conf:

[common]
hostname = 158.37.63.219

[datadelivery-service]
allowed_ip = 158.37.63.230
transfer_dir = /grid/cache1
transfer_dir = /grid/session1

secure=no

In our case we have made it simple, we do not require a host certificate (secure=no) and we have not set any FQDN either for this remote datadelivery server, so we use the ip-adress as hostname. The allowed_ip is the ip-adress of the ARC-CE(s) that we will allow requests from.

On the ARC-CE

For the ARC-CE we must add the following option to the [arex/data-delivery] block - using the FQDN is you have that, or the ip-adress if not:

deliveryservice=http://<ipv-remote-server>:<configured-port>/datadeliveryservice

For instance:

deliveryservice=http://158.37.63.219:443/datadeliveryservice

Start the remote Datadelivery service

$ systemctl start arc-datadelivery-service

That is it. Now you have a remote data delivery server set up, and the ARC-CE only takes care of the high level logic of the transfers, and not the actual transfers themselves.

For more details on the A-REX data transfer framework, see here, and for the remote data deliver service, see here.