ARC tutorial

In this tutorial we will take an existing HPC cluster, install and configure ARC to become a production grid site, and test that it is working by submitting some test-jobs from a remote ARC client server.

Note

This is a tutorial with the not yet released ARC 7 and with using tokens. ARC still works as before with x509 user certificates, the differences will be pointed out.

Prequisites

There are many ways to set up ARC for production depending on the infrastructure you are installing ARC in front of. Sites will need different configuration options as a result. For a detailed overview of getting a production ready site the ARC installation and configuration guide is a good source of reference, in addition to the ARC configuration reference document.

In this tutorial we will set up a site with a set of recommended and commonly used configuration options.

To follow this tutorial you should have a functional cluster with the following required components:

Requirements

  • Already installed and working batch system (in this tutorial we will focus on SLURM or HTCondor, but other supported batch systems are also ok)

  • One server for ARC installation - this server could also host the batch system (be the SLURM master or HTCondor CM)

    • The ARC server must be able to submit jobs to the batch system

    • The ARC server must have an x509 host certificate

  • Minimally 1 compute node

  • Shared filesystem between the ARC server and the compute nodes

  • Optional: a separate server for the SLURM master/HTCondor CM

Requirements for tutorial part 3

  • Minimally 1 server for remote datadelivery service

Tutorial overview

All commands in an bash file

All commands that are executed in this tutorial are aggregated in the two scripts belows. You can download these on the ARC-CE and execute them as root.

Warning: this will only work for the servers prepared in the context of an actual hands-on workshop, as configuration values for other sites will be different.