ATLAS Data Challenge 2 on NorduGrid

Links ::  Instructions for sites ::  SE configuration ::  ATLAS software ::  ATLAS kit for RH7.3

ATLAS resources Jobs summary Today's failures Today's finished jobs

 Links

 Check list for participating sites

Coordinator of the ATLAS DC2 in the NorduGrid space is currently Mattias Ellert, . For all the sites participating ATLAS DC2 via the NorduGrid infrastructure, the following steps have to be performed:

Grid-wise preparation

  1. Install latest CERN CA keys, either from their official page or from the NorduGrid's download area
  2. Consider upgrading to NorduGrid ARC 0.4.4. This is the best release so far. It is strongly recommended to upgrade your Globus installation to version 2.4.3-14ng (released on October 29, 2004), as it includes very important bug fixes.
  3. Register your site to the ATLAS GIIS. To do this, add the following block in globus.conf: [mds/gris/registration/GrisToAtlas]
    regname="Atlas"
    reghn=atlasgiis.nbi.dk
    regperiod=30
    servicename=nordugrid-cluster-name
    When done, send an e-mail to Anders Wäänänen . .
  4. Consider joining the dedicated mailing list by sending the message with the body text "subscribe atlas-ng-arc" to .

ATLAS runtime environment

  1. Install ATLAS software release 8.0.7. You can either use the official CERN release or the NorduGrid produced one. At the moment, ATLAS-8.0.7 versions for RH7.3, RHEL3 and FC1 are available. If you have a different system, you can either try the RH7.3 one (should work), or request to get your system supported from Jakob Nielsen, . ftp://ftp.nordugrid.org/applications/hep/atlas/8.0.7 To install the packages for your Linux distribution, fetch a corresponding script, define a set of necessary environment variables and execute it: wget ftp://ftp.nordugrid.org/applications/hep/atlas/8.0.7/installatlasrpms-<dist>.sh
    export ATLAS_ROOT=<where_you_want_to_have_atlas>
    export G4INSTALL=<where_you_want_to_have_geant4>
    export ROOTSYS=<where_you_want_to_have_root>
    export CERN=<where_you_want_to_have_cernlib>
    chmod u+x installatlasrpms-<dist>.sh
    ./installatlasrpms-<dist>.sh
    Here Geant4, ROOT and CERNLIB are 3d party tools and libraries distributed along with ATLAS. $CERN variable has to be set by hand only on RH7.3, RH9 and RHEL3.
    • Do NOT use AFS, GPFS or similar file systems for private RPM database location
    • Some installation scripts need ncftp client. Replace ncftpget calls with wget in the script if you prefer the latter.
  2. You can validate the ATLAS software installation using the standard ATLAS KitValidation tool. Please note that this will not check the entire functionality, but will only test whether the installation was successful. If you installed ATLAS s/w from the CERN Kit, follow instructions in the Kit description.
  3. Copy the script TEST-ATLAS-8.0.7 produced by the installation procedure into your Grid runtimeenvironment directory.
  4. Validate the ATLAS installation Grid-wise. Validation scripts for the NorduGrid installation are available from ftp://ftp.nordugrid.org/applications/hep/atlas/8.0.7/validation Fetch the vaildation job definition: kitval.xrsl and submit the validation job to your cluster: ngsub -f kitval.xrsl -c mycluster.boo.org It takes about two hours to test everything. Once the job has finished, please retrieve the results with ngget and check that there are no errors in the logfile KitValidation.log. If so you can change RE to ATLAS-8.0.7 and your cluster is ready.
  5. If the validation is passed, rename the script TEST-ATLAS-8.0.7 to ATLAS-8.0.7.
  6. Authorize the NorduGrid production managers (Mattias Ellert and Alex Read): /O=Grid/O=NorduGrid/OU=tsl.uu.se/CN=Mattias Ellert
    /O=Grid/O=NorduGrid/OU=uio.no/CN=Alex Read
    and make sure you have the public keys of the NorduGrid CA installed.

Setting up a Storage Element

The important thing when committing a Storage Element (SE) for ATLAS usage is to authorise on the SE level the ATLAS Virtual Organisation members, which implies accepting their respecitve Certificate Authorities (CA). Below are the instructions on how to achieve it, and other related information.

Instructions for authorizing ATLAS physicists to SE's

  1. Add the following line to /etc/grid-security/nordugridmap.conf (this line is the ATLAS VO server contact string): group "ldap://grid-vo.nikhef.nl/ou=lcg1,o=atlas,dc=eu-datagrid,dc=org" The next time the nordugridmap utility is run, the grid-mapfile /etc/grid-security/grid-mapfile is filled with the DN's of the members of the ATLAS VO. nordugridmap by default makes use of the /etc/grid-security/nordugridmap.conf file which can be overwritten on the command line with the -c switch. The name and location of the generated mapfile (default is /etc/grid-security/grid-mapfile) can be modified in the configuraton file, which might be useful for generating different ATLAS grid-mapfiles (see below).
  2. Install all the necessary CA-public-certificates. Those include all those accredited by the European Policy Management Authority for Grid Authentication in e-Science (also called EUGridPMA), http://eugridpma.org.
    The certificates can be downloaded from http://eugridpma.org/distribution/current/accredited/ All these CA's are recognized by LCG as well, see http://lcg-registrar.cern.ch/pki_certificates.html
  3. Configure the fileplugin SE to contain a read-only location through which data can be downloaded. Note: for the stable release-series 0.4.x, it is not possible to configure the SE so that some people have read- and other people write-access to the SE unless one uses the low-level configuration-file gridftpd.conf. In fact, the people that have read-access through the read-only location defined below will also write-access through the ordinary write-location that are used by the NorduGrid DC2 production managers. It is nevertheless recommended to make a read-only location to prevent accidents. The above restriction is removed in the development series 0.5.
    Below two examples are given: configuring the SE using nordugrid.conf, and configuring the SE using gridftpd.conf.
    • To configure a read-only location in the SE using nordugrid.conf, add the block [gridftpd/dc2_read] to nordugrid.conf with the following content: plugin=fileplugin.so
      path=/dc2_read
      mount="<your physical filedir with dc2 files>"
      dir="/ nouser read cd dirlist"
      This gives read-access to people through the path: gsiftp://<clustername>/dc2_read
    • Using the low-level gridftpd.conf configuration file (usually placed in /opt/nordugrid/etc) for defining a read-only path in the SE is also easy and gives a real opportunity to distinguish between people having read- and people having write-access. There is a small problem though: the gridftpd.conf configuration file is overwritten with the information from nordugrid.conf if one uses the standard method of starting the gridftp-server "service gridftpd start".
      Instead one should start the gridftpd using the command: /opt/nordugrid/sbin/gridftpd -c /opt/nordugrid/etc/gridftpd.conf This may need adding /opt/voms/lib to LD_LIBRARY_PATH first.
      With this in mind, the following is a standard gridftpd.conf configuration file:
      pidfile /var/run/gridftpd.pid
      logfile /var/log/gridftpd.log
      port 2811
      pluginpath /opt/nordugrid/lib
      encryption yes
      allowunknown no
      
      group atlas
        file /etc/grid-security/atlas-mapfile
      end
      
      group atlas_read
        file /etc/grid-security/atlasreaders-mapfile
      end
      
      groupcfg atlas
      plugin /dc2 fileplugin.so
        mount /
        dir / nouser read cd dirlist delete create *:* 664:664 mkdir *:* 775:775
      end
      
      groupcfg atlas_read
      plugin /dc2_read fileplugin.so
        mount /
        dir / nouser read cd dirlist
      end
      
      In this case, the people in /etc/grid-security/atlasreaders-mapfile will have read-access to the files and the people in /etc/grid-security/atlas-mapfile will have write-access. The file /etc/grid-security/atlas-mapfile should be filled with (at least) the DN's of the ATLAS DC production managers while the file /etc/grid-security/atlasreaders-mapfile should be filled with the DN's of the ATLAS VO people.

SE maintenance notes

The list of SE's used in the production at the moment can always be gotten by the query: globus-rls-cli query lrc lfn __storage_service__ rls://atlasrls.nordugrid.org:39282

If your SE goes down and/or is down for maintenace for a short while, please let the coordinator know beforehand, so that this list can be adjusted. This is also to make sure that enough space is available at all times.

If you plan to pemanently shut down a SE, please notify the coordinator and take the necessary steps to rescue the stored data, by replicating them to another SE and eventually creating backups.

It is generally a good practice to have regular backups of the stored data, whenever possible.