Tutorial Part 3 - Production ready setup with remote ARC datadelivery service
For this part of the tutorial you need at least one remove datadelivery server set up. It makes sense that the session directory and the cache directory is hosted on this server and shared to the ARC-CE and the compute nodes.
The EGI tutorial test clusters are set up in this way.
Install ARC Datadelivery service
Setup nightly repo
On the remote data delivery server, install the nordugrid nightly repo following the same procedure as for the ARC-CE, namely this one.
Install ARC packages
dnf install -y nordugrid-arc-arcctl nordugrid-arc-datadelivery-service nordugrid-arc-plugins-needed
arcctl deploy igtf-ca classic --installrepo igtf
Configure ARC Datadelivery service
To set up remote datadelivery we must configure both the ARC-CE and the remote ARC datadelivery server.
On the remote Data Delivery server
On the remote data delivery server you must create the following minimal arc.conf:
[common]
hostname = 158.37.63.219
[datadelivery-service]
allowed_ip = 158.37.63.230
transfer_dir = /grid/cache1
transfer_dir = /grid/session1
secure=no
In our case we have made it simple, we do not require a host certificate (secure=no) and we have not set any FQDN either for this remote datadelivery server, so we use the ip-adress as hostname. The allowed_ip is the ip-adress of the ARC-CE(s) that we will allow requests from.
On the ARC-CE
For the ARC-CE we must add the following option to the [arex/data-delivery] block - using the FQDN is you have that, or the ip-adress if not:
deliveryservice=http://<ipv-remote-server>:<configured-port>/datadeliveryservice
For instance:
deliveryservice=http://158.37.63.219:443/datadeliveryservice
Start the remote Datadelivery service
$ systemctl start arc-datadelivery-service
That is it. Now you have a remote data delivery server set up, and the ARC-CE only takes care of the high level logic of the transfers, and not the actual transfers themselves.
For more details on the A-REX data transfer framework, see here, and for the remote data deliver service, see here.
Go back to the Overview