NORDUGRID-MEMO-13

Bug procedures

NorduGrid uses a Bugzilla system for bug report and request tracking. For general description of the system, please read Bugzilla Guide.

ARC software is sub-divided logically into components, each of which has a responsible person («owner»), assigned by the technical coordination group. Reporters are asked to specify the relevant component, such that Bugzilla automatically assigns new reports to the respective owners. The following procedures and policies are designed to ensure speedy and quality processing of the requests.

Bug reporting

First-time user - registration

In order to report bugs, you should register as a user in Bugzilla. This is done to avoid bogus reports and to keep you updated about the bug status. The procedure is fully authomatic and only requires a valid e-mail address.

Once registered, you can use your Grid certificate (installed in your browser) to get authorised by Bugzilla over a secure channel. In case of problems, contact Bugzilla maintainer as indicated in the rregistration page.

Report ARC bugs, request features

ARC bugs and feature requests are reported in the section NorduGrid ARC

Problems related to infrastructure services (Bugzilla itself, SVN etc) or to contributed software (e.g. PHP VOMS-Admin, ACIX) can be reported via Bugzilla as well.

Please always use the template suggetsed by Bugzilla for problem description. For feature requests, use free text, and select Severity level feature request.

Bugzilla fields

Click Show Advanced Fields in order to access such fields as Priority or CC:

Most Bugzilla report fields are self-explanatory; some cases that are known to cause controversy are clarified below.

Severity vs Priority

Severity field is to be used by bug submitters to indicate impact of the bug/issue. In rough order of decreasing impact, Severity values are:

feature request - request for new functionality
enhancement - request to improve existing functionality
blocker - severe defect that prevents system from working, no workarounds available
critical - severe defect that needs complex workarounds, frequently manifested
major - severe defect with simple workaround or rarely manifested
normal - defect that is nice to be fixed, but is not an obstacle
minor - minor defect, such as unclear documentation
trivial - trivial defect, such as a spelling error

This severity/impact is also used by release managers in order to assess release urgency, version number increment etc.

Priority field indicates urgency with which the issue must be addressed. Default priority is P3, and does not need to be changed by the bug reporter, unless necessary. See more details below.

Bug fixing and handling

A bug's life cycle goes through the following states: NEW, ASISGNED, RESOLVED (with several resolution options), VERIFIED and CLOSED. Fixed bugs (in states RESOLVED, VERIFIED or CLOSED) can be resurrected and get into the state REOPENED. Newly reported bugs are automatically assigned NEW state.

Priority

Bugs are addressed in order of their Priority. Priority is assigned by the technical coordination group, default being P3. P1 is the highest (immediate) priority, while P5 is the lowest.

A feature request with P1 must be treated as the top priority as well as a blocker bug.

If a developer accumulates several bug/feature reports of same priority and can not decide how to prioritize between them, they may consult the group leader or the technical coordinator.

Status

Bug status proceeds from NEW to CLOSED in a straight sequence, unless the bug gets re-opened, in which case the life cycle of the bug re-starts.

Different states are set by different players:

NEW

Set by Bugzilla

NEW bugs should be acted upon within 2 working days. Possible actions are:

Reminders will be sent to the prospective bug owners by Bugzilla automatically. Every week the technical coordination group will check whether there are unattended NEW bugs and decide what action is to be taken (re-assign, close, contact the owner etc). In case a bug is re-assigned from any state, it acquires NEW status again, and from this moment the new 2 days deadline applies.

ASSIGNED

Set by the assigned developer

Bugs are placed into ASSIGNED state when the owner "accepts" the report. "Accept" action indicates that the owner accepts the report as valid or agrees to check whether it is valid. If the owner believes the report belongs to someone else, he/she must re-assign it within the same limit of 2 days.

There is no general limit on how long the bugs can remain in the ASSIGNED state.

High priority bugs (P1, P2) are expected to be acted upon immediately.

If a person accumulates unreasonably high amount of assigned bugs, or otherwise has no possibility to take care of the assigned bugs, an effort must be made to re-assign the bugs. This effort should be primarily undertaken by the bug owner him/herself, optionally with the help from the technical coordination group. Re-assigning stalled bugs is strongly encouraged.

RESOLVED

Set by the assigned developer (owner)

As soon as developers commit fixes to the code or finds other resolution, bug status must be set by them to RESOLVED. This will indicate that testing can start.

Resolution can be of several kinds: FIXED, INVALID, WONTFIX, LATER, REMIND, WORKSFORME. Meanings of the terms are self-explanatory, for example, if a bug is found to be in other software (not ARC), it should be resolved as INVALID.

Whoever claims the bug is resolved must elaborate on resolution in the "Additional comments" box. The common practice for the FIXED resolution is to write "Fixed in SVN, expected to be available in tag NNN". If the bug is resolved with any other resolution, it must be commented as well (e.g., explain why it is invalid or why it won't be fixed, or how much later it will be fixed etc).

When it is known which SVN revision number corresponds to the solution, this number (or a list, comma-separated) must be entered in "Fixed in" field.

REOPENED

Set by a requester

Bugs in states RESOLVED, VERIFIED or CLOSED can be REOPENED by the original requester or anybody who believes the problem is not gone. The bug life cycle then restarts from the NEW state. REOPENED state is completely equivalent to NEW, implying the same 2 days deadline.

VERIFIED

Set by a tester

Every resolution must be verified. This can be done by any tester - preferably an independent person and not the bug owner - who has evidence that the resolution is adequate. The evidence must be reported in the comment field.

If the bug is declared FIXED, the original reporter is expected to check whether the problem is indeed solved, at the first opportunity.

Any other person can provide an evidence and mark the bug VERIFIED as well, in which case the evidence must be reported in the "Additional comments" field.

Other resolutions (INVALID, WONTFIX etc) must be verified by the technical coordination group.

If the resolution is WONTFIX, the technical coordination group may decide to re-assign the bug.

CLOSED

Set by the bug keeper

The bug is declared closed by the technical coordination group. This happens in the following cases:

  1. the bug is fixed, verified and the fix is shipped with an official release, or
  2. the bug is confirmed to be INVALID or definitely WONTFIX

The technical coordination group inspects and closes VERIFIED bugs within few days after a release.

Quality assurance

Quality assurance for bugs reported to all ARC versions is done by the technical coordination group during the regular meetings.

Every weekly meeting has a standing agenda item "New bugs" dedicated to identifying unattended bugs and decide on actions to be taken.

Every other month, a dedicated meeting is held to address bug management. The procedure outline is such:

  1. Check NEW and REOPENED bugs
  2. Check bugs in states RESOLVED LATER and RESOLVED REMIND
  3. Check and close VERIFIED bugs
  4. Check ASSIGNED bugs, prioritized by severity. Minor bugs can be skipped if time does not permit. Bug priorities can be changed: for example, priority for a long-waiting bug can be upped.
  5. Check RESOLVED bugs other than FIXED
  6. Check FIXED bugs, if time permits

The technical coordination group may decide to invite an external expert to discuss specific outstanding bugs.

Summary reports on bug status are circulated by the technical coordination group to the nordugrid-discuss mailing list.

NorduGrid homepage