a3859aa386
Detailed description and instructions are in the README file. This work had been basically done during GSoC 2006. Approved by: brooks (mentor)
204 lines
6.9 KiB
Plaintext
204 lines
6.9 KiB
Plaintext
$FreeBSD$
|
|
|
|
A brief how-to
|
|
--------------
|
|
|
|
Each nsswitch regression test does 2 kinds of actions:
|
|
1. It runs a series of queries and tests the correctness of results.
|
|
There are 2 basic criterias which are used for that:
|
|
- numbers must be in the correct range
|
|
- certain pointers should not be NULL
|
|
|
|
2. It makes a snapshot of the results of all queries that were made.
|
|
The idea of snapshots is to test that nsswitch-related function
|
|
calls behave equally (i.e. return same results for the same queries)
|
|
between system upgrades. When the test is executed and the snapshot is
|
|
already created, the test will compare the result of each query with
|
|
the appropriate result from the snapshot and will signal if they
|
|
differ.
|
|
|
|
In order for nsswitch tests to be as useful as possible you should use
|
|
them in the following way:
|
|
|
|
Step 1 (before upgrading your system).
|
|
Build the tests with "make" command and execute them with "prove -v"
|
|
command. If there are errors during the execution, then appropriate
|
|
nsswitch functions should be checked. Note, that errors on this state
|
|
can happen only if the particular function return incorrect data.
|
|
|
|
After the stage 1 a number of "snapshot_[test name]" files will appear
|
|
in your test folder.
|
|
|
|
Step 2 (after upgrading you system).
|
|
Rebuild the tests with "make clean; make" command and execute them
|
|
again with "prove -v" (check that "snapshot_[test name]" files
|
|
are in the same folder with tests). On this stage regression tests
|
|
will catch not only the correctness errors, but will also determine
|
|
the changes in nsswitch functions behaviour.
|
|
|
|
In case of the test failure you will get the following message:
|
|
|
|
To get more details about the error you should do the following:
|
|
Step 1. Run the test alone with debug output enabled.
|
|
Step 2. Mail the snapshot file and the debug test output to the
|
|
freebsd-current@ mailing list.
|
|
|
|
Example testing session for getpwXXX() family of functions
|
|
----------------------------------------------------------
|
|
1. make
|
|
|
|
2. prove -v ./test-getpw.t
|
|
|
|
test-getpw....1..8
|
|
ok 1 - getpwnam()
|
|
ok 2 - getpwuid()
|
|
ok 3 - getpwent()
|
|
ok 4 - getpwent() 2-pass
|
|
ok 5 - building snapshot, if needed
|
|
ok 6 - getpwnam() snapshot
|
|
ok 7 - getpwuid() snapshot
|
|
ok 8 - getpwent() snapshot
|
|
ok
|
|
All tests successful.
|
|
Files=1, Tests=8, 1 wallclock secs ( 0.00 cusr + 0.20 csys = 0.20 CPU)
|
|
|
|
|
|
3. Upgrading the system.
|
|
|
|
4. make clean; make
|
|
|
|
5. prove -v ./test-getpw.t (suppose that something has gone wrong)
|
|
|
|
test-getpw....1..8
|
|
ok 1 - getpwnam()
|
|
ok 2 - getpwuid()
|
|
ok 3 - getpwent()
|
|
ok 4 - getpwent() 2-pass
|
|
ok 5 - building snapshot, if needed
|
|
not ok 6 - getpwnam() snapshot
|
|
ok 7 - getpwuid() snapshot
|
|
not ok 8 - getpwent() snapshot
|
|
FAILED tests 6, 8
|
|
Failed 2/8 tests, 75.00% okay
|
|
Failed 1/1 test scripts, 0.00% okay. 2/8 subtests failed, 75.00% okay.
|
|
|
|
6. We see that test number 6 failed. According to get-getpw.t, this test
|
|
is executed like this:
|
|
do_test 6 'getpwnam() snapshot' '-n -s snapshot_pwd'
|
|
|
|
To determine why the test has failed, we need to run it in debug mode -
|
|
it means adding "-d" to the options list.
|
|
|
|
7. ./test-getpw -dn -s snapshot_pwd
|
|
...
|
|
testing getpwnam() with the following data:
|
|
toor:*:0:0:0::ne-again Superuser:/root::0:4831
|
|
testing correctness with the following data:
|
|
toor:*:0:0:0::Bourne-again Superuser:/root::0:4831
|
|
correct
|
|
not ok
|
|
|
|
8. Here we can see that the data from snapshot (first "toor" line) and
|
|
the data received from the getpwnam() call (second "toor" line) are
|
|
different. It is the reason why the test has failed. If you can't
|
|
(or don't want) to investigate the problem by yourself, mail
|
|
the test debug output and the snapshot file to the developers list.
|
|
|
|
Notes on using standalone nsswitch tests
|
|
----------------------------------------
|
|
|
|
All nsswitch tests have [-d] optional command line argument which enables
|
|
debug output. The debug output can be extremely helpful to determine the
|
|
cause of test failure.
|
|
|
|
In all nsswitch tests -s <file> command line argument specifies the
|
|
snapshot file. If this file doesn't exist, it would be built during
|
|
test execution. If it already exists then it will be used to check
|
|
the results of particular function calls. This argument is mostly
|
|
optional, but some tests (test-getaddr and test-getusershell) force
|
|
it to be specified.
|
|
|
|
test-gethostby and test-getaddr require the list of hostnames, that should
|
|
be queried during the test. This list must be specified via -f <file>
|
|
command line argument. Each hostname should occupy exactly one line
|
|
in the file.
|
|
|
|
Detailed tests description
|
|
--------------------------
|
|
|
|
./test-getaddr - tests the getaddrinfo() function.
|
|
Usage: test-getaddr [-d] [-46] [-s <file>] -f <file>
|
|
-d - enable debug output
|
|
-4 - force IPv4 usage
|
|
-6 - force IPv6 usage
|
|
-s - build/use specified snapshot file
|
|
-f - use specified hostnames list for testing
|
|
|
|
./test-getgr
|
|
Usage: test-getgr -nge2 [-d] [-s <file>]
|
|
-d - enable debug output
|
|
-n - test getgrnam(3)
|
|
-g - test getgrgid(3)
|
|
-e - test getgrent(3)
|
|
-2 - test getgrent(3) in 2-pass mode
|
|
-s - build/use specified snapshot file
|
|
|
|
./test-gethostby
|
|
Usage: test-gethostby -na2i [-o] [-d] [-m46] [-s <file>] -f <file>
|
|
-n - test gethostbyname2(3)
|
|
-a - test gethostbyaddr(3)
|
|
-2 - test gethostbyname2(3) results to be equal with getaddrinfo(3)
|
|
results for the similar query
|
|
-i - test gethostbyaddr(3) results to be equal with getnameinfo(3)
|
|
results for the similar query
|
|
-o - use getipnodebyname(3)/getipnodebyaddr(3) for testing instead of
|
|
gethostbyname2(3)/gethostbyaddr(3)
|
|
-d - enable debug output
|
|
-m - force IPv4-to-IPv6 mapping
|
|
-4 - force IPv4 usage
|
|
-6 - force IPv6 usage
|
|
-s - build/use specified snapshot file
|
|
-f - use specified hostnames list for testing
|
|
|
|
./test-getproto
|
|
Usage: test-getproto -nve2 [-d] [-s <file>]
|
|
-d - enable debug output
|
|
-n - test getprotobyname(3)
|
|
-v - test getprotobynumber(3)
|
|
-e - test getprotoent(3)
|
|
-2 - test getprotoent(3) in 2-pass mode
|
|
-s - build/use specified snapshot file
|
|
|
|
./test-getpw
|
|
Usage: test-getpw -nue2 [-d] [-s <file>]
|
|
-d - enable debug output
|
|
-n - test getpwnam(3)
|
|
-u - test getpwuid(3)
|
|
-e - test getpwent(3)
|
|
-2 - test getpwent(3) in 2-pass mode
|
|
-s - build/use snapshot file
|
|
|
|
./test-getrpc
|
|
Usage: test-getrpc -nve2 [-d] [-s <file>]
|
|
-d - enable debug output
|
|
-n - test getrpcbyname(3)
|
|
-v - test getrpcbynumber(3)
|
|
-e - test getrpcent(3)
|
|
-2 - test getrpcent(3) in 2-pass mode
|
|
-s - build/use specified snapshot file
|
|
|
|
./test-getserv
|
|
Usage: test-getserv -npe2 [-d] [-s <file>]
|
|
-d - enable debug output
|
|
-n - test getservbyname(3)
|
|
-p - test getservbyport(3)
|
|
-e - test getservent(3)
|
|
-2 - test getservent(3) in 2-pass mode
|
|
-s - build/use specified snapshot file
|
|
|
|
./test-getusershell
|
|
Usage: test-getusershell [-d] -s <file>
|
|
-d - enable debug output
|
|
-s - build/use specified snapshot file
|
|
|