00e13b1d67
understood by Perl's Test::Harness module and prove(1) commands. Update README to describe the new protocol. The work's broken down into two main sets of changes. First, update the existing test programs (shell scripts and C programs) to produce output in the ok/not ok format, and to, where possible, also produce a header describing the number of tests that are expected to be run. Second, provide the .t files that actually run the tests. In some cases these are copies of, or very similar too, scripts that already existed. I've kept the old scripts around so that it's possible to verify that behaviour under this new system (in terms of whether or not a test fails) is identical to the behaviour under the old system. Add a TODO file.
60 lines
2.1 KiB
Plaintext
60 lines
2.1 KiB
Plaintext
# $FreeBSD$
|
|
|
|
dnl A library of routines for doing regression tests for userland utilities.
|
|
|
|
dnl Start up. We initialise the exit status to 0 (no failure) and change
|
|
dnl into the directory specified by our first argument, which is the
|
|
dnl directory to run the tests inside.
|
|
define(`REGRESSION_START',
|
|
TESTDIR=$1
|
|
if [ -z "$TESTDIR" ]; then
|
|
TESTDIR=.
|
|
fi
|
|
cd $TESTDIR
|
|
|
|
STATUS=0)
|
|
|
|
dnl Check $? to see if we passed or failed. The first parameter is the test
|
|
dnl which passed or failed. It may be nil.
|
|
define(`REGRESSION_PASSFAIL',
|
|
if [ $? -eq 0 ]; then
|
|
echo "ok - $1 # Test detected no regression. (in $TESTDIR)"
|
|
else
|
|
STATUS=$?
|
|
echo "not ok - $1 # Test failed: regression detected. See above. (in $TESTDIR)"
|
|
fi)
|
|
|
|
dnl An actual test. The first parameter is the test name. The second is the
|
|
dnl command/commands to execute for the actual test. Their exit status is
|
|
dnl checked. It is assumed that the test will output to stdout, and that the
|
|
dnl output to be used to check for regression will be in regress.TESTNAME.out.
|
|
define(`REGRESSION_TEST',
|
|
$2 | diff -u regress.$1.out -
|
|
REGRESSION_PASSFAIL($1))
|
|
|
|
dnl A freeform regression test. Only exit status is checked.
|
|
define(`REGRESSION_TEST_FREEFORM',
|
|
$2
|
|
REGRESSION_PASSFAIL($1))
|
|
|
|
dnl A regression test like REGRESSION_TEST, except only regress.out is used
|
|
dnl for checking output differences. The first argument is the command, the
|
|
dnl second argument (which may be empty) is the test name.
|
|
define(`REGRESSION_TEST_ONE',
|
|
$1 | diff -u regress.out -
|
|
REGRESSION_PASSFAIL($2))
|
|
|
|
dnl A fatal error. This will exit with the given status (first argument) and
|
|
dnl print the message (second argument) prefixed with the string "FATAL :" to
|
|
dnl the error stream.
|
|
define(`REGRESSION_FATAL',
|
|
echo "Bail out! $2 (in $TESTDIR)" > /dev/stderr
|
|
exit $1)
|
|
|
|
dnl Cleanup. Exit with the status code of the last failure. Should probably
|
|
dnl be the number of failed tests, but hey presto, this is what it does. This
|
|
dnl could also clean up potential droppings, if some forms of regression tests
|
|
dnl end up using mktemp(1) or such.
|
|
define(`REGRESSION_END',
|
|
exit $STATUS)
|