freebsd-dev/tools/regression
John Baldwin 6bc1e9cd84 Rework the lifetime management of the kernel implementation of POSIX
semaphores.  Specifically, semaphores are now represented as new file
descriptor type that is set to close on exec.  This removes the need for
all of the manual process reference counting (and fork, exec, and exit
event handlers) as the normal file descriptor operations handle all of
that for us nicely.  It is also suggested as one possible implementation
in the spec and at least one other OS (OS X) uses this approach.

Some bugs that were fixed as a result include:
- References to a named semaphore whose name is removed still work after
  the sem_unlink() operation.  Prior to this patch, if a semaphore's name
  was removed, valid handles from sem_open() would get EINVAL errors from
  sem_getvalue(), sem_post(), etc.  This fixes that.
- Unnamed semaphores created with sem_init() were not cleaned up when a
  process exited or exec'd.  They were only cleaned up if the process
  did an explicit sem_destroy().  This could result in a leak of semaphore
  objects that could never be cleaned up.
- On the other hand, if another process guessed the id (kernel pointer to
  'struct ksem' of an unnamed semaphore (created via sem_init)) and had
  write access to the semaphore based on UID/GID checks, then that other
  process could manipulate the semaphore via sem_destroy(), sem_post(),
  sem_wait(), etc.
- As part of the permission check (UID/GID), the umask of the proces
  creating the semaphore was not honored.  Thus if your umask denied group
  read/write access but the explicit mode in the sem_init() call allowed
  it, the semaphore would be readable/writable by other users in the
  same group, for example.  This includes access via the previous bug.
- If the module refused to unload because there were active semaphores,
  then it might have deregistered one or more of the semaphore system
  calls before it noticed that there was a problem.  I'm not sure if
  this actually happened as the order that modules are discovered by the
  kernel linker depends on how the actual .ko file is linked.  One can
  make the order deterministic by using a single module with a mod_event
  handler that explicitly registers syscalls (and deregisters during
  unload after any checks).  This also fixes a race where even if the
  sem_module unloaded first it would have destroyed locks that the
  syscalls might be trying to access if they are still executing when
  they are unloaded.

  XXX: By the way, deregistering system calls doesn't do any blocking
  to drain any threads from the calls.
- Some minor fixes to errno values on error.  For example, sem_init()
  isn't documented to return ENFILE or EMFILE if we run out of semaphores
  the way that sem_open() can.  Instead, it should return ENOSPC in that
  case.

Other changes:
- Kernel semaphores now use a hash table to manage the namespace of
  named semaphores nearly in a similar fashion to the POSIX shared memory
  object file descriptors.  Kernel semaphores can now also have names
  longer than 14 chars (up to MAXPATHLEN) and can include subdirectories
  in their pathname.
- The UID/GID permission checks for access to a named semaphore are now
  done via vaccess() rather than a home-rolled set of checks.
- Now that kernel semaphores have an associated file object, the various
  MAC checks for POSIX semaphores accept both a file credential and an
  active credential.  There is also a new posixsem_check_stat() since it
  is possible to fstat() a semaphore file descriptor.
- A small set of regression tests (using the ksem API directly) is present
  in src/tools/regression/posixsem.

Reported by:	kris (1)
Tested by:	kris
Reviewed by:	rwatson (lightly)
MFC after:	1 month
2008-06-27 05:39:04 +00:00
..
acct Test the integer-arithmetic float conversions encode_timeval and encode_long 2007-05-22 05:52:04 +00:00
aio Add the ability to do all read, all write, or random read/write. 2008-05-15 16:10:55 +00:00
atm Remove two netatm-specific test parts. 2008-05-26 22:41:40 +00:00
audit/audit_pipe_ioctl Merge Perforce change 93572 from TrustedBSD audit3 branch: 2006-03-19 15:47:53 +00:00
bin Test the expansion of $LINENO. 2008-05-15 19:58:44 +00:00
ccd/layout
doat Simple testing program for *at family of syscalls. 2008-05-18 08:34:04 +00:00
environ Differentiate in the tests against getenv() between an empty string and a 2008-04-23 00:45:48 +00:00
ethernet/ethermulti Add a regression test for ethernet link-layer multicast memberships. 2007-02-25 01:58:02 +00:00
execve o Grammar. 2006-10-09 06:47:08 +00:00
fifo Teach fifo_create regression test to also try to use mknod(2) to create 2008-06-22 21:03:26 +00:00
file Re-implement the client side of rpc.lockd in the kernel. This implementation 2008-06-26 10:21:54 +00:00
fstest Make fstest work out-of-the-box on Solaris: 2007-07-18 18:07:15 +00:00
fsx getopt(3) returns -1, not EOF. 2008-02-19 07:09:19 +00:00
gaithrstress Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
geom Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
geom_concat Finish conversion of gconcat(8) regression tests to the new framework. 2005-12-07 01:24:25 +00:00
geom_eli Hide geli warnings about sector size beeing bigger than the page size when 2007-10-26 08:31:11 +00:00
geom_gate Switch over to a different, more flexible test output protocol that's 2004-11-11 19:47:55 +00:00
geom_gpt MFp4: 2006-04-10 04:13:44 +00:00
geom_mirror - Make use of geom_subr.sh to improve regression tests. 2005-12-07 01:27:23 +00:00
geom_nop - Make use of geom_subr.sh to improve regression tests. 2005-12-07 01:27:23 +00:00
geom_raid3 Convert graid3(8) regression tests to the new framework. 2005-12-07 01:28:59 +00:00
geom_shsec Convert gshsec(8) regression tests to the new framework. 2005-12-07 01:29:50 +00:00
geom_stripe Convert gstripe(8) regression tests to the new framework. 2005-12-07 01:30:44 +00:00
geom_uzip Use mkuzip(8) instead of create_compressed_fs. 2005-09-29 22:41:20 +00:00
ia64 Add regression tests of the emulation of the long branch conditional form. 2006-07-17 00:27:12 +00:00
include/tgmath Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
ipsec Adding tests for Camellia which will be added to the tree soon. 2007-03-26 07:40:51 +00:00
lib Regression test for a recently fixed strtod bug. 2008-06-21 19:28:26 +00:00
mac/mac_bsdextended Add some new options to mac_bsdestended. We can now match on: 2006-04-23 17:06:18 +00:00
mlock Style: NO_MAN doesn't need any value. 2006-03-15 10:46:38 +00:00
mqueue Fix compiling. 2007-01-06 11:32:15 +00:00
msdosfs Adding another multi-byte characters related regression test for recent 2006-01-01 01:28:31 +00:00
net80211 Catching up with the latest crypto API changes in net80211. 2005-05-05 13:16:30 +00:00
netatalk/simple_send Change s_net to s_node when printing the destination address. 2006-03-17 20:14:25 +00:00
netinet This is a regression test to verify the proper behavior of IP ID generation 2008-02-06 15:48:43 +00:00
netinet6 Regression tests for recent changes to inet6_rth_* family of functions 2007-04-21 11:23:33 +00:00
netipx o style.Makefile(5): WARNS= -> WARNS?=. 2006-07-09 11:51:14 +00:00
nfsmmap Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
p1003_1b Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
pipe Add two more pipe regression tests. Not yet hooked into the test framework. 2005-01-21 06:46:18 +00:00
posixsem Rework the lifetime management of the kernel implementation of POSIX 2008-06-27 05:39:04 +00:00
posixshm Fix compile on 64-bit platforms. 2008-06-26 21:26:34 +00:00
priv Remove empty setup and cleanup functions for the pfkey test. 2007-11-16 21:24:45 +00:00
pthread mutex_islocked_np -> mutex_isowned_np 2008-02-06 19:41:05 +00:00
redzone9 Add simple tests which verify that redzone(9) works properly. 2006-01-31 11:20:13 +00:00
security Add open_to_operation, a security regression test that opens files with 2008-02-14 20:57:38 +00:00
sigqueue Add code to test queued SIGCHLD. 2005-11-08 09:19:44 +00:00
sockets Add unix_sorflush, a regression test for the following scenario: 2008-01-30 13:23:34 +00:00
sysvmsg Fix message size tests for msgsnd(3) and msgrcv(3). msgsz is the length of 2006-11-16 19:51:10 +00:00
sysvsem Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
sysvshm Start the dreaded NOFOO -> NO_FOO conversion. 2004-12-21 08:47:35 +00:00
tls NO_MAN is not needed here. 2006-03-16 15:18:17 +00:00
tmpfs New regression test updates for rename, etc. 2007-08-10 10:08:43 +00:00
ufs/uprintf o style.Makefile(5): WARNS= -> WARNS?=. 2006-07-09 11:51:14 +00:00
usr.bin Attach the tr(1) regression tests to the build. 2008-01-13 08:33:58 +00:00
usr.sbin Add sparc64 regression test files. 2007-05-24 07:00:53 +00:00
geom_subr.sh Add file with helper functions for geom tools regression tests. 2005-12-07 01:20:18 +00:00
README Document the .in .out file naming conventions adopted in commit_prep.pl's 2007-05-22 06:11:44 +00:00
TODO Switch over to a different, more flexible test output protocol that's 2004-11-11 19:47:55 +00:00

$FreeBSD$

This directory is for regression test programs.

A regression test program is one that will exercise a particular bit of the
system to check that we have not reintroduced an old bug.

Tests should be implemented in files with a .t extension.  Each .t file
can contain more than one test, and can be implemented in any scripting
language -- /bin/sh, Perl...

The test protocol is quite simple.  At its most basic, each .t file should, 
when run, print a line in this format:

   1..m

where m is the number of tests that will be run.

Each test should produce a single line of output.  This line should start
with one of

   ok n
   not ok n

to indicate whether or not the test succeeded.  'n' is the test's number.
Anything after this on the line (up to the first '#' if present) is 
considered to be the name of the test.  Naming tests is optional, but 
encouraged.

A test may be written which is conditional, and may need to be skipped.
For example, the netatalk tests require 'options NETATALK' in the kernel.
A test may be skipped by printing '# skip Reason for skipping' after the
test name.  For example,

    ok 1 - netatalk # skip 'options NETATALK' not compiled in

A test may be flagged as 'todo'.  This indicates that you expect the test
to fail (perhaps because the necessary functionality hasn't been written
yet).  'todo' tests are expected to fail, so when they start working the
test framework can alert you to this happy occurence.  Flag these tests 
with a '# TODO' comment after the test name

    not ok 1 - infiniteloop # TODO write test for an infinite loop

This is modelled on the protocol followed by the Test::Harness Perl
module (and therefore much of the automated testing carried out by the 
Perl community).  More documentation can be found at:

    http://search.cpan.org/~petdance/Test-Harness-2.42/lib/Test/Harness.pm

To run the tests and parse their output install the devel/p5-Test-Harness
port.  This includes the prove(1) command which is used to run the tests
and collate the output.

    prove geom_concat		# run all the tests in geom_concat
    prove -r lib		# run all tests in lib/, and subdirectories
    prove -r -v lib		# as above, with verbose output
    prove -r			# run *all* the tests

Tests that are for parts of the base system should go into a directory here
which is the same as their path relative to src/, for example the uuencode(1)
utility resides in src/usr.bin/uuencode so its regression test resides in
src/tools/regression/usr.bin/uuencode.

To avoid the pre-commit check program complaining about the lack of
CVS keywords in test data files, use a .in suffix for input files and
a .out suffix for output files.

To execute individual regression tests for binaries that you are
developing, add their directory in the path before running the tests.
Example:
cd /usr/src/tools/regression/usr.bin
(PATH=/home/user/src/experimental/jot:$PATH ; make SUBDIR=jot)

Please make a subdir per other regression test, and add a brief description to
this file.

acct		Exercise the integer to float conversion used in acct(5)
geom		Some tests and an out-of-kernel simulator for the GEOM code
ia64		ia64 specific regression tests
nfsmmap		Some tests to exercise some tricky cases in NFS and mmap
p1003_1b	Exercise 1003.1B scheduler
pipe		Pipe code regression test
fsx		General filesystem exerciser
sysvmsg 	SysV IPC Message Queue Regression Utility
sysvsem 	SysV IPC Semaphore Regression Utility
sysvshm 	SysV IPC Shared Memory Regression Utility
gaithrstress	General threaded getaddrinfo(3) exerciser