Release Notes for NorduGrid ARC 15.03 update 7

May 25, 2016

This is a minor release, introducing new features and addressing bugs discovered since release 15.03u6.

NorduGrid ARC 15.03 has received an update to:

core, clients, CE, Infosys and gridftp - from version from version 5.0.5 to 5.1.1
documents - from 2.0.6 to 2.0.7

Nagios plugins, CAnL C++ and metapackages are unchanged.

New features highlights

An option to arc.conf to gather various performance metrics related to A-REX, as specified in Wiki document.

Can be enabled by setting enable_perflog_reporting="yes" in common block.
Default location for performance logs is /var/log/arc/perfdata. Can be changed with perflogdir option in common block.
A helper script to upload performance data is partially implemented. Can be invoked by setting helper=". /usr/share/arc/perferator" in [grid-manager] (path to perferator may be different if ARC is installed from other sources than rpm or deb packages).

The URL protocol is s3://
The environment variables S3_ACCESS_KEY and S3_SECRET_KEY should be set to the S3 keys.

New URL option "failureallowed" can be set to "yes" and failure to transfer the file will not cause job failure.
New configuration option "cachedump" in [acix/cacheserver] for creating periodic cache dumps
Various harmonizations to logging.
Various improvements to the ARC SDK.
Numerous bugfixes, enhancements and improvements.

The following issues were fixed in the ARC core:

Added performance metrics for A-REX and data components.
Improved handling of VOMS and VOs.
Added gm-jobs option to obtain delegation for a specified job.
Added delegation handling to WS interface.
New optional technology preview to use sqlite rather than bdb as a delegation DB. Can be enabled with delegationdb=sqlite in [grid-manager]. Please note that jobs should be drained before switching between DBs.

The following issues were fixed in the information system:

Added performance metrics for infosys based on the PERL Devel::NYTProf profiler available from EPEL, Debian/Ubuntu repositories and CPAN.
Changed log format to be consistent with A-REX logging.

The following issues were fixed in the ARC Clients:

Enhanced job description parsing.
Made it possible to change UserConfig object for SubmitterPlugin and JobControllerPlugin classes.
Created structure for mapping policy and retrieve information from GLUE2.
Enhanced job description parsing and assembling error reporting.

No changes

Since ARC 15.03 update 6, the following bugs were fixed:

3414 Request for job logging
3477 sessiondir = path [drain] can't prevent new jobs entering sessiondir
3479 Description of voms-based authorisation is wrong
3486 External helper log file location is hardcoded to controldir/job.helper.errors
3492 certificate generated with arcproxy is not usable with arcls/arccp + dCache SRM SE - one with voms-proxy-init is
3496 arcplugin occasionally segfaults
3520 emies maps to wrong uid
3522 Renewing proxy through EMI-ES doesn't work with dynamic output files
3524 Encoded characters in location break HTTP redirection
3525 Wrong ARC version number in /usr/include/arc/ArcVersion.h
3529 arc-lcas and arc-lcmaps helpers segfault when validating voms credentials of VOs whose VOMS certificates are signed by "/DC=com/DC=DigiCert-Grid/O=DigiCert Grid/CN=DigiCert Grid CA-1"
3535 URL option for opportunistic output file upload
3537 Unable to submit jobs with EMI ES interface only
3539 Submission to PBS backend with memory limit does not work well with threads
3542 %R substitution in auth plugin uses first session directory
3543 Invalid input file URL can make A-REX crash
3552 Restarting A-REX causes FINISHING jobs with dynamic outputs to fail

There is a memory leak when using Java API for multiple job submission with files to BES interface.
The CPU time is not measured correctly for jobs that kill the parent process, such as some agent-based/pilot (e.g., ALICE)
JURA will not publish records to the APEL on a standard Debian/Ubuntu system, because the python-dirq package is not available for them. The workaround is to build this package from source.
When using ARC client tools to submit jobs to CREAM, only JSDL can be used to describe jobs, and the broker type must be set to Null in the client.conf file
ARC GUI (arcjobtool) is not available yet, pending implementation of client library changes.
Standalone client tar-balls for Linux are not yet available.
Bug 2905 is solved using workaround. Source of problem is not yet identified.
A-REX can under some circumstances lose connection with CEinfo.pl and go into an infinite loop. The only current workaround is to restart the a-rex service.
twistd, the underlying engine for ACIX, sometimes logs into rotated ACIX log files. While all log messages are correctly logged in the main log file, some rotated log files may receive new log messages.
submit-*-job do not have permission to write performance metrics to log.

ARC release 15.03u7 consists of the following source packages:

NorduGrid ARC, version 5.1.1 (main components)
NorduGrid ARC Documents version 2.0.7
metapackages for client tools, computing element and information index, version 1.0.7
Nagios probes for ARC CE, version 1.8.4
gangliarc - ARC Computing Element monitoring in ganglia, version 1.0.0
Common authentication library caNl++, version 1.0.1

Source for metapackages is available from:
http://svn.nordugrid.org/repos/packaging/{fedora,debian}/nordugrid-arc-meta/tags/1.0.7

Source for the common authentication library caNl++ is available from:
http://svn.nordugrid.org/repos/workarea/caNl++/tags/1.0.1

See detailed description at NorduGrid downloads

These repositories provide binary packages for:

Debian: 5.0, 6.0, 7.0 and 8.0 (i386 and amd64)
Fedora: from 3 to 23 (i386 and x86_64)
RedHat: EL4, EL5, EL6 and EL7 (i386 and x86_64)
Ubuntu: 8.04, 8.10, 9.04, 9.10, 10.04, 10.10, 11.04, 11.10, 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 15.10 and 16.04 (i386 and amd64)

Scientific Linux and RedHat are implicitly supported through corresponding CentOS repositories.