NORDUGRID-MEMO-7

NorduGrid Replica Catalog Cheatsheet

Introduction

Below is a list of basic examples showing how to manipulate the records in the NorduGrid Replica Catalog (RC). Only authorised RC managers can perform operations which require write access. For details, contact

NorduGrid makes use of the Globus RC. It is an hierarchical database: the catalogue consists of collections, each collection consists of logical file names which correspond to one or more physical files at pre-defined locations. To register an arbitrary file in a RC, the following steps should be performed:

  1. Ensure the collection exists
  2. Ensure the physical location of the file is registered as one of the locations for the collection
  3. Enter the file record into the RC, specifying the location and the file name RELATIVE TO THE LOCATION

The limitation of the Globus RC is that it is impossible to assign arbitrary logical file names: a logical file name must be the actual file name, eventually prepend with the path relative to the registered location.

Examples

Define who you are

This is the strongly recommended first step, simplifying authentication at the NorduGrid Replica Catalog

export GLOBUS_REPLICA_CATALOG_MANAGER="/O=Grid/O=NorduGrid/OU=nordugrid.org/CN=Jane Doe"

Or, for C shells:

setenv GLOBUS_REPLICA_CATALOG_MANAGER "/O=Grid/O=NorduGrid/OU=nordugrid.org/CN=Jane Doe"

Emphasized string is your identity, as specified in the Subject of your Grid certificate.

Create a collection

globus-replica-management -collection ldap://grid.uio.no/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org -create

Here, grid.uio.no is the RC server address (default port 389), and A_Collection is your new collection name. Since the RC is an LDAP database, collection URL must contain the LDAP-complying Distinguished Name, which includes the full NorduGrid RC name space.

Add a location to a collection

globus-replica-management -collection ldap://grid.uio.no/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org -location CASTOR-ATLAS-DC1 -create gsiftp://wacdr002d.cern.ch/castor/cern.ch/atlas/transfer/dc1/

The location name CASTOR-ATLAS-DC1 should later be used as a mount point, instead of the physical URL gsiftp://...

Upload and register a file to a collection

You must have a valid Grid proxy to perform this operation

ngcopy gsiftp://grid.quark.lu.se/ATLAS/mydata.zebra rc://grid.uio.no/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org/yourdata.zebra

In the example above, the first available registered location will be used to store the file. To specify an explicit destination, do:

ngcopy gsiftp://grid.quark.lu.se/ATLAS/mydata.zebra rc://CASTOR-ATLAS-DC1@grid.uio.no/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org/yourdata.zebra

Register existing at a location file to a collection

File registration includes three steps:

  1. Create of a logical file entry in the collection:
  2. globus-replica-catalog -host ldap://grid.uio.no/lc=A_collection,rc=NorduGrid,dc=nordugrid,dc=org -logicalfile path/file1.zebra -create size 0

  3. Verify that file location (mount point) is registered in the collection, otherwise it has to be added
  4. Register the physical location(s):
  5. globus-replica-management -collection ldap://grid.uio.no/lc=A_collection,rc=NorduGrid,dc=nordugrid,dc=org -location CASTOR-ATLAS-DC1 -files filelist -register

If file size is unknown, specify "0" as shown above. Logical entries can only be added one by one, while physical location(s) must be added via a filelist, one line per file:

path/file1.zebra
path/file2.zebra

Note the prepended path, relative to the registered location: it should be possible to construct a correct URL as ${LOCATION}${FILENAME}.

Remove and unregister a file

ngremove rc://grid.uio.no/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org/yourdata.zebra

Remove a location from a collection

globus-replica-management -collection ldap://grid.uio.no/lc=A_collection,rc=NorduGrid,dc=nordugrid,dc=org -location CASTOR-ATLAS-DC1 -delete

Find locations (URL prefixes) for a collection

ldapsearch -h grid.uio.no -p 389 -b "lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org" "(&(objectclass="GlobusReplicaLocation")(uc=*))" uc

or, using the Globus interface to LDAP:

globus-replica-catalog -host ldap://grid.uio.no:389/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org -collection -list-locations uc

Find a URL prefix for a location known by name

globus-replica-catalog -host ldap://grid.uio.no:389/lc=A_Collection,rc=NorduGrid,dc=nordugrid,dc=org -location CASTOR-ATLAS-DC1 -list-attributes uc

NorduGrid homepage