Release Notes for NorduGrid ARC 15.03 update 10

October 26, 2016

This is a minor release, introducing new features and addressing bugs discovered since release 15.03u9. The release includes all recent developments up until now excluding only those that would result in backward incompatabilities. For these a major release is expected in the beginning of next year.

Note that from next major release we will not continue support for EL5 and Ubuntu 10.04 LTS. This is a consequence of significant code changes when implementing OpenSSL 1.1.0 in ARC needed for continued support of Fedora, which as of Fedora 25 moves to OpenSSL 1.1.0. In general, support of OpenSSL versions lower than 1.0.0 will be dropped. In addition legacy proxies will not be supported starting with the next major release.

NorduGrid ARC 15.03 has received an update to:

• core, clients, CE, Infosys and gridftp - from version 5.1.3 to 5.2.0
• documents - from 2.0.9 to 2.0.11

Nagios plugins, CAnL C++ and metapackages are unchanged.

New features highlights

• Added method in JobSupervisor to select jobs using custom selector class.
• Added possibility to configure host and port for ACIX cacheserver.
• Added support for default user assignement to VO.
• Added offline script to generate performance data in inforproviders.
• Added support for VO in GLUE2 schema.
• Added support for using Rucio as a proxy to read/write to objectstores.
• Added several options to HTTP URL's.

Detailed notes

ARC Server and core components

The following issues were fixed in the ARC core:

• Add method in JobSupervisor to select jobs using custom selector class (see bug 3484).
• Allow configurable host and port for ACIX cacheserver (see bug 3572).
• Better credential checks and error messages for data commands(see bug 3597).
• Handling of Globus GSSAPI version which changed behavior in 12.2 (see bug 3604).
• Fixed a bug that resulted in truncation of long certificate subject (see bug 3610).
• Added support for default user assignement to VO.
• Added support for using Rucio as a proxy to read/write to objectstores.
• Unifying credentials delegation and handling among interfaces.
• Code improvements related to configuration classes for unification.
• Added support for Ganglia in A-REX.
• Followed up on SSL/TLS usage trends (for VOMS servers).
  • Adding more SSL/TLS handshake options.
  • Adjusting code to newer OpenSSL API.
  • Switching VOMS code to TLS handshake as modern VOMS servers seems to accept TLS and Ubuntu disables SSLv3 handshake.
• Fix support for platforms where date(1) does not support @ in dates specified as seconds since the epoch.
• Added several options to HTTP URL's.
  • Disallowed metadata options in http URLs since they interfere with query string.
  • Added new option relativeURI to force HTTP operations to use relative paths
• Protection against undefined buffer which occasionally makes A-REX crash.
• Fixed build-time conflict between GNU/Hurd and MacOS X.
• Code cleaning and various small fixes.

Accounting

The following issues were fixed in the accounting system:

• Corrected cpucores to 1 to avoid over-reporting.

Information system

The following issues were fixed in the information system:

• Added support for VO in the GLUE2 schema.
• Every authorized vo will generate a different ComputingShare with statistics about the defined VO. For minimum backward compatibility there is a Share without any VO that presents information the same way ARC did before this change, but doesn't list all the VOs as in the previous version. This breaks backward compatibility with EMIES clients.
• Added PerfData.pl, an offline script to generate ARC performance data collected from a NYTProf database file generated by infoproviders.
• VO support fix for compatibility with Perl 5.14 (see bug 3464).

ARC Clients

The following issues were fixed in the ARC Clients:

• Fix submission scripts to handle special cases with spaces and quoting.

Nagios plugins

• No changes

Common authentication library CaNL++

No changes

Fixed bugs:

Since ARC 15.03 update 9, the following bugs were fixed:

• 3433 Publish authorised VOs per queue
• 3464 ComputingShare and MappingPolicy should be created on a VO basis for WLCG\/CERN use case and job data calculated per VO
• 3484 Time-based cleaning of jobs using ARC api
• 3509 GLUE2PolicyID must be unique if policy entries are different
• 3564 Set a default VO that applies to the whole cluster or to a queue
• 3572 configurable interface and port in cacheserver
• 3590 Months in Durations
• 3591 arcproxy on Ubuntu 16.04 can not contact VOMS servers
• 3597 Useless and meaningless error message from arc client tools
• 3601 Errors in log if jobreport_publisher not specified
• 3604 Job submission breaks when updating globus-gssapi-gsi from 11.22-1 to 12.5-2
• 3610 Client certificate subject truncated (buffer size 100)

Known issues

• There is a memory leak when using Java API for multiple job submission with files to BES interface.
• The CPU time is not measured correctly for jobs that kill the parent process, such as some agent-based/pilot (e.g., ALICE)
• JURA will not publish records to the APEL on a standard Debian/Ubuntu system, because the python-dirq package is not available for them. The workaround is to build this package from source.
• When using ARC client tools to submit jobs to CREAM, only JSDL can be used to describe jobs, and the broker type must be set to Null in the client.conf file
• ARC GUI (arcjobtool) is not available yet, pending implementation of client library changes.
• Standalone client tar-balls for Linux are not yet available.
• Bug 2905 is solved using workaround. Source of problem is not yet identified.
• A-REX can under some circumstances lose connection with CEinfo.pl and go into an infinite loop. The only current workaround is to restart the a-rex service.
• twistd, the underlying engine for ACIX, sometimes logs into rotated ACIX log files. While all log messages are correctly logged in the main log file, some rotated log files may receive new log messages.
• submit-*-job do not have permission to write performance metrics to log.
• authorizedvo=<voname> will no longer create a list of VOs under each Share. As a consequence, EMIES WS clients can no longer find a queue by VO name the same way as in previous versions of ARC due to changes in the GLUE2 schema rendering.

Availability

Source

ARC release 15.03u10 consists of the following source packages:

• NorduGrid ARC, version 5.2.0 (main components)
• NorduGrid ARC Documents version 2.0.11
• metapackages for client tools, computing element and information index, version 1.0.7
• Nagios probes for ARC CE, version 1.8.4
• gangliarc - ARC Computing Element monitoring in ganglia, version 1.0.0
• Common authentication library caNl++, version 1.0.1

Source code for main components is available from:
http://svn.nordugrid.org/repos/nordugrid/arc1/tags/5.2.0

Documentation source (mostly LaTeX) is available from:
http://svn.nordugrid.org/repos/nordugrid/doc/tags/2.0.11

Source for metapackages is available from:
http://svn.nordugrid.org/repos/packaging/{fedora,debian}/nordugrid-arc-meta/tags/1.0.7

Source for Nagios probes is available from:
http://svn.nordugrid.org/repos/nordugrid/nagios/tags/release-1.8.4

Source for the common authentication library caNl++ is available from:
http://svn.nordugrid.org/repos/workarea/caNl++/tags/1.0.1

Source for gangliarc is available from:
http://svn.nordugrid.org/repos/nordugrid/contrib/gangliarc/tags/1.0.1

Repositories

See detailed description at NorduGrid downloads

These repositories provide binary packages for:

• Debian: 5.0, 6.0, 7.0 and 8.0 (i386 and amd64)
• Fedora: from 3 to 24 (i386 and x86_64)
• RedHat: EL4, EL5, EL6 (i386 and x86_64) and EL7 (x86_64)
• Ubuntu: 8.04, 8.10, 9.04, 9.10, 10.04, 10.10, 11.04, 11.10, 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 15.10 and 16.04 (i386 and amd64)

Scientific Linux and RedHat are implicitly supported through corresponding CentOS repositories.

Previous releases

Details of previous releases can be found at the ARC Releases page