using my own script to handle it. I wrote my own partially because
of all the quoting-issues involved with testing what I wanted to test,
and partially because this lets me commit one script and one data file,
instead of one-file-per-regression-test.
This suite was good enough for my initial testing (and it did help me
find a few bugs that would have otherwise been missed). I'm not sure
how well it will work in general use, but I figured I might as well
commit it. It won't *hurt* to have it available. At the worst, people
can just ignore it.
Approved by: re (blanket `env')
so that make(1) will run in an almost clean environment and enhance the
description of the test infrastructure.
Add the ability to have multiple tests carried out per test script.
Give some tests more meaningful names.
Fix the usage message from the test scripts.
Make it possible to pass several commands to the test scripts like:
'sh test.t setup run compare clean'.
shell meta characters it is not passed to the shell, but the command
is executed directly (given that the line is not a shell builtin) and
that the line with a meta character is passed to the shell.
and adjust the path in the Makefile for the upgrade_checks target.
These checks are really feature upgrade checks that should be fast
and just find out whether we need to build a new make before
proceeding with other targets like buildworld. This makes the
place free for a real regression test suite in the old place.
by forcing the creation of an object directory for the make regression
tests. Let make handle the tracking of the dependency and installation
of test_shell script.
Submitted by: ru
that you create one of the object directories make knows (see make(1)).
This uses the -C flag, so add a test that checks that make actually accepts
-C. Also fix the test that selects csh via the .SHELL target to work for
tcsh users too.
This commit renames shell_test to shell_test.sh. There is no history
to preserve so go without a repo-copy.
Reviewed by: ru
understood by Perl's Test::Harness module and prove(1) commands.
Update README to describe the new protocol. The work's broken down into
two main sets of changes.
First, update the existing test programs (shell scripts and C programs)
to produce output in the ok/not ok format, and to, where possible, also
produce a header describing the number of tests that are expected to be
run.
Second, provide the .t files that actually run the tests. In some cases
these are copies of, or very similar too, scripts that already existed.
I've kept the old scripts around so that it's possible to verify that
behaviour under this new system (in terms of whether or not a test fails)
is identical to the behaviour under the old system.
Add a TODO file.
regular expression as the first argument to a substitute command. If
used to test a sed which (erroneously) evaluates this at translation
time rather than at execution time, the bugged sed is put into an
infinite loop. This mode of failure seems excessive. Such a failing
sed is the Free Software Foundation's sed 3.02.
The specific test was also not being executed for the BSD sed.
Both problems are now fixed.
PR: misc/25585
Submitted by: Walter Briscoe <w.briscoe@ponl.com>
Approved by: schweikh (mentor)
MFC after: 2 weeks
warning: duplicate script for target "double" ignored
The regression-tests do try to hide that message, but the message does
still appear when using -j (eg: 'make -j5 buildworld'). This changes the
regression-test so the expected warning message will not be seen even
when -j is specified.
Reviewed by: jmallett ru
behavior. Add the bcb regression test which checks for failures due
to a backslash ('\') coinciding with the very last character of the
command buffer. The regression test is cf. this PR (which I did not
know about) and has a different fix for the bug.
PR: bin/22351
Submitted by: Stefan Duerholt <stefan.duerholt@t-online.de>
handle printing of the PASS/FAIL messages. Suffix PASS/FAIL/FATAL with the
string (in $directory) where $directory is ${.CURDIR} from make(1), to
make it easier to use grep(1) and a bit of sed/awk to do statistics of
failure for some utilities over time, etc.
make(1): Does not work like the other tests. Its Makefile is
self-testing.
m4(1): It uses complex voodo to test GNU m4(1) features.
To the new framework. I had worried about passing the binary data that
uudecode(1)'s test passes to diff(1) might give a user something nasty,
but this is unlikely to happen as even with an unmodified old nasty
diff(1) which doesn't recognise many binary files, these binary files
are recognised. Using $DIFF instead of `diff' in the library and making
it possible to override this with `cmp -s' might be nice some day, but
as of this second, there's no immediate need.
to handle the ones which output to stdout and have output in regress.$test.out,
etc. More freeform macros should and will be written, but these are the most
prominent and most straightforward sort of tests we have around, so it makes
sense to try to accomodate them.
expansion of embedded variables in the left-hand-side of an assignment
expression, using the simplest case - hiding recursion using nil-expanded
variables.
uuencode(1), and set a umask, so that the mode in the header is predictable.
If it varies, then the test is right to fail.
Remove the note about this test falsely failing, with that in mind.
and for proper behaviour of some sed functions given a nil pattern space,
as fixed in PR 34813.
The test for G was based on the test in the PR. The nil pattern space test
is slightly different as we need to get *some* output, as the core dump will
also produce no output (old behaviour) and turn up falsely that the utility
is working fine.
