[Chapter 7] 7.2.66 Test::Harness - Run Perl Standard Test Scripts with Statistics

7.2.66 Test::Harness - Run Perl Standard Test Scripts with Statistics

use Test::Harness;
runtests(@tests);

This module is used by MakeMaker. If you're building a Perl extension and if you have test scripts with filenames matching t/*.t in the extension's subdirectory, then you can run those tests by executing the shell command, make test.

runtests(@tests) runs all test scripts named as arguments and checks standard output for the expected "ok n" strings. (Standard Perl test scripts print "ok n" for each single test, where n is an integer incremented by one each time around.) After all tests have been performed, runtests() prints some performance statistics that are computed by the Benchmark module.

runtests() is exported by Test::Harness by default.

7.2.66.1 The test script output

The first line output by a standard test script should be 1..m with m being the number of tests that the test script attempts to run. Any output from the test script to standard error is ignored and bypassed, and thus will be seen by the user. Lines written to standard output that look like Perl comments (starting with /^\s*\#/) are discarded. Lines containing /^(not\s+)?ok\b/ are interpreted as feedback for runtests().

The global variable $Test::Harness::verbose is exportable and can be used to let runtests() display the standard output of the script without altering the behavior otherwise.

It is tolerated if the script omits test numbers after ok. In this case Test::Harness maintains its own counter. So the following script output:

1..6
not ok
ok
not ok
ok
ok
not ok

will generate:

FAILED tests 1, 3, 6
Failed 3/6 tests, 50.00% okay

7.2.66.2 Diagnostics

All tests successful.\nFiles=%d, Tests=%d, %s: If all tests are successful, some statistics about the performance are printed.
FAILED tests %s\n\tFailed %d/%d tests, %.2f%% okay.: For any single script that has failing subtests, these statistics are printed.
Test returned status %d (wstat %d): Scripts that return a non-zero exit status, both $?>>8 and $?, are printed in a message similar to the above.
Failed 1 test, %.2f%% okay.
Failed %d/%d tests, %.2f%% okay.: If not all tests were successful, the script dies with one of the above messages.

7.2.66.3 Notes

Test::Harness uses $^X to determine which Perl binary to run the tests with. Test scripts running via the shebang (#!) line may not be portable because $^X is not consistent for shebang scripts across platforms. This is no problem when Test::Harness is run with an absolute path to the Perl binary or when $^X can be found in the path.