freebsd-dev/tools/regression
Jilles Tjoelker 70d41b5846 wordexp(): Remove wrong IFS usage.
Words in shell script are separated by spaces or tabs independent of the
value of IFS. The value of IFS is only relevant for the result of
substitutions. Therefore, there should be a space between 'wordexp' and the
words to be expanded, not an IFS character.

Paranoia might dictate that the shell ignore IFS from the environment (even
though our sh currently uses it), so do not depend on it in the new test
case.
2013-04-01 20:50:07 +00:00
..
acct tests: sort & fix includes, remove -include from CFLAGS of acct tests 2011-01-06 20:05:24 +00:00
acltools Add test for acl_is_trivial_np(3). 2011-03-22 17:26:56 +00:00
aio - Fix QA issues 2011-02-22 05:13:26 +00:00
atm Update README to reflect removal of netatm/harp test parts some time ago. 2008-08-31 11:41:31 +00:00
audit/audit_pipe_ioctl
bin sh: Recognize "--" and explicitly reject options in wait builtin. 2013-03-15 20:29:31 +00:00
bpf Adjust a test case and make it more jump optimization neutral for JIT case. 2010-04-23 22:42:49 +00:00
capsicum/syscalls If all ioctls are allowed, cap_ioctls_get(2) will return CAP_IOCTLS_ALL. 2013-03-02 23:40:42 +00:00
ccd/layout
doat Fix some memory errors in *at() regression tests. 2011-10-18 22:51:40 +00:00
environ Do not gratuitously fail *env(3) operations due to corrupt ('='-less) 2009-12-01 05:04:31 +00:00
ethernet/ethermulti
execve
fifo Check fchmod()/fchown() in fifo_misc test. 2012-02-26 15:32:02 +00:00
file Spelling fixes for tools/ 2011-12-30 00:04:11 +00:00
filemon Test both active and non-active cases. 2012-10-26 20:14:40 +00:00
fsx getopt(3) returns -1, not EOF. 2008-02-19 07:09:19 +00:00
gaithrstress Fixed missing or broken library dependencies. 2010-02-25 14:42:26 +00:00
geom (S)LIST_HEAD_INITIALIZER takes a (S)LIST_HEAD as an argument. 2009-12-28 22:56:30 +00:00
geom_concat
geom_eli Fix resize regressions tests - we need to destroy BSDlabel before we can 2011-10-25 13:41:12 +00:00
geom_gate
geom_gpt
geom_mirror
geom_nop
geom_raid3
geom_shsec
geom_stripe
geom_uzip
ia64
include/tgmath Switch over to gnu99 compilation on default for userland. 2009-03-14 17:55:16 +00:00
ipfw/fwd Add support for IPv6 to ipfw fwd: 2011-08-20 17:05:11 +00:00
ipsec
kgssapi Fixed missing or broken library dependencies. 2010-02-25 14:42:26 +00:00
kqueue Clarify the meaning of a test. 2011-07-08 12:16:30 +00:00
kthread/kld Revert previous commit... 2012-10-10 08:36:38 +00:00
lib wordexp(): Remove wrong IFS usage. 2013-04-01 20:50:07 +00:00
mac Oops. Correct comment in the LICENSE file. 2009-03-14 21:59:12 +00:00
mlock
mmap Replace the name of the sysctl to security.bsd.map_at_zero and to be 2009-10-02 17:53:48 +00:00
mqueue Add missing headers. While there, arrange headers alphabetically. 2010-03-16 20:42:54 +00:00
msdosfs Convert files over to UTF-8. 2012-01-05 21:36:53 +00:00
net/if_tap Add regression test for kern/172075. 2012-09-26 00:25:09 +00:00
net80211 Revert previous commit... 2012-10-10 08:36:38 +00:00
netatalk/simple_send
netinet Remove unused variable. 2012-11-20 01:42:18 +00:00
netinet6 Remove extra comment that snuck in in last commit. 2008-07-29 18:38:37 +00:00
netipx Use '%zu' and '%zd' as appropriate for size_t / ssize_t. 2012-11-19 23:07:38 +00:00
nfsmmap Spelling fixes for tools/ 2011-12-30 00:04:11 +00:00
p1003_1b
pipe Add a test for r228510. 2011-12-14 23:26:48 +00:00
pjdfstest Update regression tests after adding chflagsat(2). 2013-03-21 23:07:04 +00:00
poll Test the poll(2) return value. 2010-08-28 17:38:40 +00:00
posixsem Make it compile on LP64 arches. 2010-03-14 13:04:09 +00:00
posixsem2 Remove dot at the end of errx() message. 2010-03-14 13:29:09 +00:00
posixshm Fix compile on 64-bit platforms. 2008-06-26 21:26:34 +00:00
priv Spelling fixes for tools/ 2011-12-30 00:04:11 +00:00
pthread Reencode files from latin1 to UTF-8. 2011-12-29 12:33:27 +00:00
redzone9
rpcsec_gss Fixed missing or broken library dependencies. 2010-02-25 14:42:26 +00:00
sbin More mdconfig(8) tests. 2012-01-26 18:16:16 +00:00
security Update regression tests after adding chflagsat(2). 2013-03-21 23:07:04 +00:00
sigqueue Fixed missing or broken library dependencies. 2010-02-25 14:42:26 +00:00
sockets Major update for unix_cmsg from Andrey Simonenko. 2013-02-11 12:56:23 +00:00
sysvmsg According to shared memory man pages, advice including <sys/types.h> 2012-05-21 07:52:46 +00:00
sysvsem According to shared memory man pages, advice including <sys/types.h> 2012-05-21 07:52:46 +00:00
sysvshm According to shared memory man pages, advice including <sys/types.h> 2012-05-21 07:52:46 +00:00
tls Fixed missing or broken library dependencies. 2010-02-25 14:42:26 +00:00
tmpfs The NetBSD Foundation has granted permission to remove clause 3 and 4 from 2010-03-02 22:25:50 +00:00
ufs/uprintf Use '%zu' and '%zd' as appropriate for size_t / ssize_t. 2012-11-19 23:07:38 +00:00
usb Add regression test for USB enumeration. 2012-03-01 20:45:17 +00:00
usr.bin Add an additional regression tests for other cases to ensure these do not get fixed by accident. 2012-12-18 21:42:45 +00:00
usr.sbin The etcupdate utility is a tool for managing updates to files that are 2012-07-13 13:23:48 +00:00
vfs Increment the counter outside the subshell. 2009-05-27 15:15:58 +00:00
zfs Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes. 2008-11-17 20:49:29 +00:00
geom_subr.sh
README Spelling fixes for tools/ 2011-12-30 00:04:11 +00:00
TODO

$FreeBSD$

This directory is for regression test programs.

A regression test program is one that will exercise a particular bit of the
system to check that we have not reintroduced an old bug.

Tests should be implemented in files with a .t extension.  Each .t file
can contain more than one test, and can be implemented in any scripting
language -- /bin/sh, Perl...

The test protocol is quite simple.  At its most basic, each .t file should, 
when run, print a line in this format:

   1..m

where m is the number of tests that will be run.

Each test should produce a single line of output.  This line should start
with one of

   ok n
   not ok n

to indicate whether or not the test succeeded.  'n' is the test's number.
Anything after this on the line (up to the first '#' if present) is 
considered to be the name of the test.  Naming tests is optional, but 
encouraged.

A test may be written which is conditional, and may need to be skipped.
For example, the netatalk tests require 'options NETATALK' in the kernel.
A test may be skipped by printing '# skip Reason for skipping' after the
test name.  For example,

    ok 1 - netatalk # skip 'options NETATALK' not compiled in

A test may be flagged as 'todo'.  This indicates that you expect the test
to fail (perhaps because the necessary functionality hasn't been written
yet).  'todo' tests are expected to fail, so when they start working the
test framework can alert you to this happy occurrence.  Flag these tests 
with a '# TODO' comment after the test name

    not ok 1 - infiniteloop # TODO write test for an infinite loop

This is modelled on the protocol followed by the Test::Harness Perl
module (and therefore much of the automated testing carried out by the 
Perl community).  More documentation can be found at:

    http://search.cpan.org/~petdance/Test-Harness-2.42/lib/Test/Harness.pm

To run the tests and parse their output install the devel/p5-Test-Harness
port.  This includes the prove(1) command which is used to run the tests
and collate the output.

    prove geom_concat		# run all the tests in geom_concat
    prove -r lib		# run all tests in lib/, and subdirectories
    prove -r -v lib		# as above, with verbose output
    prove -r			# run *all* the tests

Tests that are for parts of the base system should go into a directory here
which is the same as their path relative to src/, for example the uuencode(1)
utility resides in src/usr.bin/uuencode so its regression test resides in
src/tools/regression/usr.bin/uuencode.

To avoid the pre-commit check program complaining about the lack of
CVS keywords in test data files, use a .in suffix for input files and
a .out suffix for output files.

To execute individual regression tests for binaries that you are
developing, add their directory in the path before running the tests.
Example:
cd /usr/src/tools/regression/usr.bin
(PATH=/home/user/src/experimental/jot:$PATH ; make SUBDIR=jot)

Please make a subdir per other regression test, and add a brief description to
this file.

acct		Exercise the integer to float conversion used in acct(5)
geom		Some tests and an out-of-kernel simulator for the GEOM code
ia64		ia64 specific regression tests
nfsmmap		Some tests to exercise some tricky cases in NFS and mmap
p1003_1b	Exercise 1003.1B scheduler
pipe		Pipe code regression test
fsx		General filesystem exerciser
sysvmsg 	SysV IPC Message Queue Regression Utility
sysvsem 	SysV IPC Semaphore Regression Utility
sysvshm 	SysV IPC Shared Memory Regression Utility
gaithrstress	General threaded getaddrinfo(3) exerciser
date		Date(1) + format string regression test