Validation tool: ngtest

A complete job that tests a number of different things on a cluster including downloading of input files from different places. This job downloads from the dedicated repository the source code for a prime number finding program, compiles it and runs it for exactly two minutes. All prime numbers that are found are written to a file that can be downloaded by the user as a part of the job output, with the help of the ngget utility of the User Interface.

This test job is the default one and is executed if user does not specify any arguments to ngtest

A very basic "Hello Grid" job. This test basically does nothing but running the Unix/Linux echo utility with the string "Hello, Grid" as an argument. This is the most basic test one can imagine. To see the output file after the job has finished, run ngget for the corresponding jobID and look in the file stdout in the created output directory.

This tests assumes /bin/echo exists at the remote compute resource

In this test, the User Interface uploads an executable echo.sh and runs it with the string argument "Hello, Grid". This is a simple test of whether the User Interface can upload an executable. To submit this job, it is required that one has the file echo.sh in the current directory.

Don't forget to download echo.sh or compose a similar file by that name prior to running test #2.

Tests that one can specify the executable through a variable set as a part of a run-time environment. A run-time environment is defined as a set of environment variables, pathes, procedures and such, that are set at the execution node when the user requests that run-time environment to be present. Typically such environment variables point to an installation of some software package on the cluster. The test requires that the run-time environment ECHOENV is enabled on a cluster.

Install the ECHOENV script in your cluster's run-time environment directory and modify it accordingly, if you want to run test #3 against your own resource

This test tests the xRSL attribute arguments. This attribute contains the list of the arguments passed to the executable during execution. In this case, three arguments are passed to the executable. As usual, one can download the output using ngget and view the result in the file stdout. One should see exactly three numbers.

Like test #4, but here 10 arguments are passed to the executable.

Tests that the arguments xRSL attribute works in combination with a variable that is defined through a run-time environment.

This test assumes that the run-time environment ECHOENV has already been installed on a resource (see also test #3).

As test #6 but with two run-time environments instead.

This test requires that the run-time environments ECHOENV and MD5SUMENV have already been installed on a resource.

Tests the xRSL attribute executables, that specfies that certain input files should get executable permissions before the job is run. To see whether the test has run correctly, download the output using ngget and look in the stdout file: it must contain a directory listing with three files (stdout, stderr, echo.sh).

In this test, the Grid Manager downloads one input file from the NorduGrid HTTP server before running. This tests whether the Grid Manager – the software that controls the job submission on the resources – on the tested resource is actually able to download from a HTTP server.

The same as test #9 but with two input files.

The same as test #9 but with nine input files. In this case, these files are three files, each of them transferred 3 times. The first file is a minute file, 19 bytes, the second file is a medium file, about 1 megabyte and the last file is a huge one, about 400 megabytes.

In this test, the Grid Manager downloads one input file from a GridFTP server. A GridFTP server is basically a FTP server but with Grid-style access control. This test checks whether the Grid Manager on the resource is able to download from such a GridFTP server.

As test #12 but with two input files.

As test #12 but with nine input files. These files are exactly the same as in test #11 but the files are residing in a different place (GridFTP server instead of the HTTP one).

In this test, the Grid Manager downloads one input file from an ordinary FTP server. This tests whether the Grid Manager on the resource is able to download from an ordinary FTP server.

As test #15 but with two input files.

As test #15 with nine input files. The files are exactly the same as in test #11 and test #14.

In this test, the Grid Manager query a so-called Replica Catalog for some logical filenames. A Replica Catalog is a simple database which can translate logical filenames to actual physical files residing somewhere on a GridFTP or other server. After this translation, the Grid Manager proceeds to download the matched files.

As test #18 but with two input files.

As test #18 but with nine input files. The files are exactly the same as in tests #11, #14 and 17.

A combination of tests #10, #13, #16 and #19. The Grid Manager downloads two files from an HTTP server, two files from a GridFTP server, two files from an ordinary FTP server and two files via the Replica Catalog.

As the tests #9 to #21 above, but in these we explicitly tell the Grid Manager not to download the input files into the cache on the resource but directly into the working directory of the job. The cache is a directory on the remote resource where files are placed and can be re-used by several jobs.

As the tests #22 to #34 above, but in these we instruct the Grid Manager to use 10 GridFTP threads when downloading the input files. This case is very similar to the tests above, but the download should in principle go a bit faster

As the tests #35 to #47 above but here we tell the Grid Manager to use 10 GridFTP threads when downloading, and download the files into the working directory, not the cache.

In this test, the User Interface is requested to upload one input file directly from your submission machine. This tests whether the User Interface is capable of doing so. The test requires the file echo.sh to be available in the current directory.

Don't forget to download echo.sh or compose a similar file by that name prior to running test #61.

As test #61 but with two input files. This test requires that the files echo.sh and somefile are available locally. somefile is a file of size of about 1 megabyte.

Don't forget to download echo.sh and somefile or create similar files by those names prior to running test #62.

As test #61 but with nine input files (three files uploaded three times each). This test requires that the files echo.sh, somefile and somehugefile are available locally. somehugefile is a file of size of about 400 megabytes.

Don't forget to download echo.sh, somefile and somehugefile or create similar files by those names prior to running test #63.

As the tests #61, #62 and #63 above, but the uploader program is told to use 10 GridFTP threads when uploading to the remote directory.

Don't forget to download echo.sh, somefile and somehugefile or create similar files by those names prior to running tests #64, #65 and #66.

This test is a combination of test #21 and test #62. In total, eight files are downloaded by the remote resource and two files are uploaded by the User Interface. In this test, the two files mentioned previously, echo.sh and somefile are needed locally for the upload.

Don't forget to download echo.sh and somefile or create similar files by those names prior to running test #67.

As test #67 but with 10 GridFTP threads both in uploading and downloading.

Don't forget to download echo.sh and somefile or create similar files by those names prior to running test #68.

In this test, the xRSL cluster attribute is tested. This test job should in all cases go the cluster lscf.nbi.dk.

Tests whether one can specify several versions in the xRSL cluster attribute using the OR operator "|". This test job should go either to the cluster grid.tsl.uu.se or the cluster quark.hep.lu.se.

Tests whether the xRSL cluster attribute works in combination with NON-EQUAL operator "!=". This test job should under no circumstances go to the cluster lscf.nbi.dk

Tests whether the cluster and queue attributes work together. This test job should go to the cluster grid.nbi.dk into the short queue. You can check whether this is correct by looking at the NorduGrid Grid Monitor while the job is running.

This test submits a job requiring the same version of the NorduGrid middleware on the remote resource as the version that is used to submit the job. You can check whether this is correct by checking your version first:

ngsub -v

and comparing it with the version of the remote resource. Here you can use the NorduGrid Grid Monitor to find the version of that resource. This test specifies that the test job should only go to a resource with the same version of the NorduGrid middleware as that of the client. This tests whether the xRSL attribute middleware works as designed.

As test #73 but the job can go to a resource with the middleware version bigger or equal to that of the client.

As test #73 but the job can go to a resource with the middleware version less or equal to that of the client.

Tests that one can get extra environment variables set when the job is running by using the xRSL attribute environment. If the job finishes without errors, this works as intented.

This test tests that rsl_substitution attribute works together with the executable attribute.

Tests that rsl_substitution works with the arguments attribute

This test tests that brokering works when the user specifies the xRSL attribute CPUTime. In this example, 2 hours of CPU time is requested.

This test tries to make job to upload one output file to a GridFTP server. The output of the job shows the location of the output files. You can get the output by using ngget and looking in the file stdout.

If the job finishes successfully, the user should erase the file at the GridFTP server again using the command given in the output of the job.

As test #80 but with two output files.

The user should remember to erase the corresponding output files afterwards.

As test #80 but with nine output files. The files used come from different FTP and GridFTP servers connected to NorduGrid.

The user should remember to erase the corresponding output files afterwards.

As test #80 but here the test uploads to a physical location specified by the Replica Catalog. The Grid Manager contacts the Replica Catalog which resolves the logical address to a physical one. After that the Grid Manager uploads the file to that location.

The user should remember to erase the corresponding output files afterwards.

As test #83 but with two output files.

The user should remember to erase the corresponding output files afterwards.

As test #83 but with nine output files.

The user should remember to erase the corresponding output files afterwards.