Contributing to Documentation¶
NorduGrid ARC6 documentation is mainly written in reStructuredText and build with Sphinx to HTML pages, LaTeX (for printable PDF versions) and ePub.
Recent source tree (
master branch) build is available instantly via Coderefinery GitLab Pages and deployed to the nordugrid.org on nightly basis.
Everyone is welcome to contribute. For trivial fixes just click on the “Edit in GitLab” button and commmit the changes.
You need to have an account in the Coderefinery GitLab and be a member of the project to be allowed to edit. Contact us if you want to contribute, but are not a member yet.
Commiting and reviewing changes¶
Contribution is possible by direct push to the repo, no need for merge requests if you are fixing typos or other content that does not requires review!
In the renderend HTML version there is a link at the bottom of each page labeled Edit page source on GitLab. Using this link you can edit the page source in your web-browser without the need to checkout source tree
- In case you want to add something new that requires the review process:
- create the new branch (in the same repository, no need to fork)
- open a merge request to
- in-line and general discussion of the particular contribution is inside merge request.
As reflected in the main index page, the documentation logically divided into the following groups:
Admins documentation is the most developed part for ARC6. We aimed to have dedicated documents for each particular topic and then bound them in the following way:
- The introductory Try ARC6: towards distributed computing in a few minutes that contains a zero-configuration case hands-on instruction to help new users and admins getting started with ARC CE.
- Main ARC Computing Element Installation and Configuration Guide that contains general installation and configuration flow for production Computing Element deployment. The guide itself should be brief enough and holds only main examples. All detailed instruction for each particular subsystem configuration should be written in the dedicated document that linked to this guide.
- ARC Configuration Reference Document automatically rendered from text version in the code source tree. Targeted for web-search. The ultimate configuration options description we have.
- All other dedicated documents goes to Operating ARC CE Subsystems section in case someone wants to access them directly instead of following installation guides links.
Source tree directory structure¶
Sources of documents in rST format are placed into the
source directory that contains the following structure:
_static- logo and CSS files 
_templates- Sphinx HTML theme layout tuning 
_extensions- Custom sphinx extensions developed for ARC 
common- directory used to store common for all ARC customers documentation, including admins, users and developes. This documents are hidden in the main TOC tree, instead they are linked from other various documents when needed.
repos- this directory holds repository configuration instructions for release, nightlies, etc. Main index page have repos pointers as well as Try ARC6 and Install Guide.
changelog- documents that describe changes between ARC versions.
admins- directory used to store Documentation for Infrastructure Admins. The main documents that represent the entry points to ARC Computing Element installation and configuration resides directly in this directory.
details- this subdirectory designed to hold detailed configuration instructions for ARC CE subsystems, like RunTime Environments in ARC6 that describe all aspects of RTEs in-depth. This documents intentionally moved deeper in the main TOC tree to prevent first levels flooding. If you can find a chapter in main install guide that cat point to the in-depth document in question - this is a right place to store it.
commands- place for automatically generated command line option reference for ARC tools. At the time of writing the Python tools represented there.
archery- for the documents that not related to Computing Element another subdirectory should be created. At the time of writing only ARCHERY documentation of that kind is available.
developers- directory used to store Documentation for Developers. As long as we don’t have many of this kind of documents everything that contains implementation details (how stuff coded, which variable used, etc) should go there.
tech- directory used to store Technical Documents Describing ARC Components. Now it holds only index document that contains references to the available PDFs that is verified to be applicable to ARC6 release. To add more, use template inside the index file.
users- directory used to store Documentation for Infrastructure Users. If document is user-oriented - place it here.
sdk- place for Doxygen SDK documentation integration to Sphinx build 
testing- directory used to store the documents for ARC 6 Testing Area.
wip- “work in progress” area hidden in the TOC tree. You can use it for incomplete documents that should not be publicly advertised yet, but still will be built and available via
/wip/URL in produced HTML files.
|||(1, 2, 3, 4) Do not touch unless you are modifying the Sphinx build itself. Nothing there affects documentation writting process.|
Each directory in the source tree contains
index.rst file that is used to link other documents in the same TOC tree. Upper-level
index.rst file contains references to
index.rst files in the subdirectories, that in turns contains pointers to the other documents.
When you add a new document, add a reference to the
index.rst in the same directory
Just follow the
index.rst chain (look for
toctree keyword) starting from the
source directory to get familiar with TOC linking structure.
When you need to add images to your document you should upload the image file itself and refer to it from the
ARC6 documentation structure implies that image files are stored inside the
images subdirectory in the document location. Then referencing is done by relative path, e.g:
.. figure:: images/shared_sessiondir_yes.svg
Building the docs¶
The top directory in the source tree contains
build.sh script that:
- checkout the ARC source code tree and configures it with packaging-like paths
- builds Doxygen SDK documentation
- copies documentation parts from ARC source code tree (for developers section)
- prepare automatically generated documentation for CLI commands
- builds HTML
- builds PDF (with LaTeX)
- builds ePub
On commit the GitLab CI configured to automatically invoke the
build.sh to produce rendered documentation. Documentation archive is avaiable as CI job artifacts and for
master branch is deployed to GitLab Pages.
To build the docs on your local machine you should have at least Sphinx installed. Additionally you should be able to configure ARC source tree and ARC Python command dependencies for auto-generated parts.
Complete and up-to-date list of dependencies defined for CI build and can be found in
PDF builds with LaTeX is the most heavy part from the both time and needed additional packages perspective. For HTML rendering local debugging it is recommended to use the
html script argument the skips PDF and ePUB:
[user@localhost doc]$ ./build.sh html
Point your browser to
file:///path/to/doc/build/html/index.html to view the HTML rendering locally.
Writing Documentation in reStructuredText¶
reStructuredText (reST) is the default plaintext markup language used by Sphinx. There is possible to render other markups but for consistency and better cross-referencing NorduGrid ARC6 documentation written solely in reStructuredText.
reStructuredText markup specification is well documented in the several sources and was designed to be a simple and readable in plain-text.
Common text editors (including
emacs) recognize reST markup and provide syntax highlighting out of the box.
Start with reStructuredText Primer on Sphinx docs.
Complete reST Markup Syntax can be found on Docutils starting with Quick reStructuredText document.
Sphinx also uses interpreted text roles to insert semantic markup (cross-referencing, etc) into docuemnts. To get familiar read this document.
Just study the markup following the documentation and readinig the already written documents just here.
There are different options for representing the code, starting with the simple literal blocks identified with
It is advised to use
code-block:: directive to enable syntax highlighted rendering:
.. code-block:: ini [gridftpd/jobs] allowaccess = staticdn dnfromfile
In this example, the
ini tag represents the syntax highlighting lexer identfier for Pygments. Look for available lexers to serve your needs. The most common for our docs are:
Referencing another parts of documentation is necessary to achive usability. The typical referencing cases are:
Using custom label¶
Create label just above any of the paragraph headers with. The following markup creates
.. _my_label: My Heading ==========
Refering to a lable is possible from any other doument with:
read the :ref:`my_label`
In this case paragraph heading will be used for hyperlink text. If you want some custom text for hyperlink text use the following syntax:
read this :ref:`text <my_label>`
Autogenerater reST rendering of
arc.conf.reference already contains labels for all configuration options and blocks that can be used.
- The label name has the following structure:
reference_<block name>[_<sub_block>...]_<option name>represents configration option inside block
reference_<block name>[_<sub_block>...]represents block itself
In the ``arc.conf`` there is a dedicated :ref:`[lrms] <reference_lrms>` block that defines the type of your LRMS. *Job session directory* is configured with :ref:`reference_arex_sessiondir` configurration option.
Custom Sphinx plugin for ARC documentation introduces new reST roles that can be used to mention bugs in NorduGrid Bugzilla or GitLab Issues:
More details can be found in :bugz:`3802` and :issue:`57`.
Referencing the the whole document is similar to using labels, but instead of label name the document name (filename without extension) is used with
Example 1: Refer to the
try_arc6.rst in the same source tree directory (relative path). Use the document header as hyperlink text:
Example 2: Refer to the
repository.rst by absolute path (starting from sources top directory). Use the custom hyperlink text:
:doc:`NorduGrid Repositories </common/repos/repository>`
To highlight statement visually use notes and warnings:
.. note:: Zero configured A-REX comes with EMI-ES and REST interfaces enabled. .. warning:: This information is valid for releases of ARC 6 starting from 6.0.
In the ARC6 documentation images should be included with the caption. It is accomplished with the figure keyword.
After storing image inside
images subdirectory, include it in reST document as:
.. figure:: images/shared_sessiondir_yes.svg :align: center :alt: Sessiondir is shared between ARC CE and WNs Sessiondir is shared between ARC CE and WNs. No local scratchdir defined.
Sphinx build had been configured with image format autocoversion feature. So you can use any image format, including vector graphics.
If you want to illustrate some process or structure that can be shown as graph, consider using built-in Graphviz functionality.
Simple usage examples can be found in
admins/details/rtes.rst document within ARC documentation. Give it a try, it is easy.
Converting from other sources¶
Converting the old documentation parts from LaTeX or Mediawiki markdown to reST is not a magical process that do everything automatically unfortunalely.
However you can get a good start with a pandoc tool that do the conversion.
E.g. to convert LaTeX source simply run:
[user ~]$ pandoc -f latex -t rst acix.tex > acix.rst
The ammount of efforts to edit the resulted
.rst file and fix formatting issues is completely depends on the source itself (e.g. how the console ouput was formatted in the origin document). Images and references in most cases should be fixed separately after the pandoc.