INTERNAL interface - ARC 6

NB! WIP [TODO] Fix references, now in tex style ref to arc-ce sys admin guide

The INTERNAL submission interface is aimed for restrictive HPC sites. When ARC runs in the internal mode, it should do so alongside a local instance of the ARC Control Tower (aCT). aCT pulls jobs from the central job server (like PaNDA for ATLAS), and and feeds ARC with new jobs internally. Since there is no external access, there is no need for a web-service, gridftp server or ldap, as the purpose of these components are to facilitate external access. There is neither a need for a host-certificate on such a machine. The only service running on ARC is A-REX. Therefore a stripped-down version of aCT and ARC-CE can be used which is beneficial for installation, configuration and maintenance. Furthermore, in the INTERNAL mode ARC should be installed as a normal user. No user mapping (See Section myref{sub:access_control}) is needed in this case, as the default behaviour in ARC 6 is to map the user submitting the job to the A-REX user.

Implementation overview

The INTERNAL submission plugin which is part of the ARC client, interacts with the parent plugin classes using the same API as the other plugins such as gridftp plugin or the emi-es plugin. However, the INTERNAL plugin interacts directly with the A-REX memory and methods, and therefore is integrated as part of the A-REX service which belongs to the ARC-CE code-base. Therefore, both the ARC client and the ARC CE must be installed and on the same machine for the INTERNAL submission plugin interface to function.

All interaction between the client and A-REX happens directly via files in the controldir or via A-REX memory.

Actions

Retrieving Service Information

As a site running in the INTERNAL mode is not accessible from the outside, any service retrieval information can only be done from within the site. You may inspect the service information as usual by issuing the arcinfo command. The information is extracted by direct access to the info.xml file in the controldir. The sstat method of the INTERNALClient reads the info.xml and outputs information in xml-format to the client. An example of the output of arcinfo for localhost

[root ~]# arcinfo -c localhost
Computing service:
  Information endpoint: file://localhost
  Submission endpoint: file://localhost (status: ok, interface: org.nordugrid.internal)

When arcinfo is called, the INTERNAL submission interface extracts the site information by directly accessing the info.xml file in the controldir. The INTERNAL plugin reads the info.xml and outputs information in xml-format to the client, which in turn displays it to the user.

Job submission

When a job is submitted via the INTERNAL submission interface the plugin creates an A-REX job object, which in turn takes care of creating all necessary files (like for instance the ARC job description) and folders (sessiondir) for the job, in addition to creating a job ID. The INTERNAL plugin then places any input files local to the client in the newly created sessiondir. Remaining remote input files are downloaded by the DTR (See Section myref{sub:datastaging}). Once these files are present in the controldir A-REX adds the job to its joblist, and takes over the handling of the job from there.

Accessing Information About Job

Job information evoked by calling arcstat is extracted from a combination of information stored in A-REX memory (job state) and the job.ID.local file in the controldir (session, stagein and stageout directories).

Controlling Execution Of Job

Killing, cleaning and resubmitting jobs is initiated by direct call the existing ARexJob methods: Kill(), Clean(), Resume(). These methods all place files in the controldir that the grid-manager acts upon, such as job.jobid.clean mark or job.jobid.cancel mark.

Delegation Interface [TODO]