freebsd-dev

History

Enji Cooper 8ac5aef8f3 Integrate capsicum-test into the FreeBSD test suite This change takes capsicum-test from upstream and applies some local changes to make the tests work on FreeBSD when executed via Kyua. The local modifications are as follows: 1. Make `OpenatTest.WithFlag` pass with the new dot-dot lookup behavior in FreeBSD 12.x+. 2. capsicum-test references a set of helper binaries: `mini-me`, `mini-me.noexec`, and `mini-me.setuid`, as part of the execve/fexecve tests, via execve, fexecve, and open. It achieves this upstream by assuming `mini-me` is in the current directory, however, in order for Kyua to execute `capsicum-test`, it needs to provide a full path to `mini-me`. In order to achieve this, I made `capsicum-test` cache the executable's path from argv[0] in main(..) and use the cached value to compute the path to `mini-me*` as part of the execve/fexecve testcases. 3. The capsicum-test test suite assumes that it's always being run on CAPABILITIES enabled kernels. However, there's a chance that the test will be run on a host without a CAPABILITIES enabled kernel, so we must check for the support before running the tests. The way to achieve this is to add the relevant `feature_present("security_capabilities")` check to SetupEnvironment::SetUp() and skip the tests when the support is not available. While here, add a check for `kern.trap_enotcap` being enabled. As noted by markj@ in https://github.com/google/capsicum-test/issues/23, this sysctl being enabled can trigger non-deterministic failures. Therefore, the tests should be skipped if this sysctl is enabled. All local changes have been submitted to the capsicum-test project (https://github.com/google/capsicum-test) and are in various stages of review. Please see the following pull requests for more details: 1. https://github.com/google/capsicum-test/pull/35 2. https://github.com/google/capsicum-test/pull/41 3. https://github.com/google/capsicum-test/pull/42 Reviewed by: asomers Discussed with: emaste, markj Approved by: emaste (mentor) MFC after: 2 months Differential Revision: https://reviews.freebsd.org/D19758		2019-04-01 21:24:50 +00:00
..
etc	Add supporting changes for `Add limited sandbox capability to "make check"`	2017-08-14 19:21:37 +00:00
freebsd_test_suite	Fix sys/opencrypto/blake2_test when kern.cryptodevallowsoft=0	2018-08-16 23:49:56 +00:00
sys	Integrate capsicum-test into the FreeBSD test suite	2019-04-01 21:24:50 +00:00
Kyuafile
Makefile	Use MK_CHECK_USE_SANDBOX in tests/..., to deal with the fact that	2017-08-02 22:24:08 +00:00
Makefile.depend	DIRDEPS_BUILD: Connect MK_TESTS.	2016-03-09 22:46:01 +00:00
Makefile.inc0	Use bsd.opts.mk, not src.opts.mk	2017-08-03 00:35:35 +00:00
README	Copy README into /usr/tests	2015-11-20 03:24:04 +00:00

README

src/tests: The FreeBSD test suite
=================================

To run the FreeBSD test suite:
(1)  Make sure that kyua is installed:
       pkg install kyua
(2)  To run the tests:
       kyua test -k /usr/tests/Kyuafile
(3)  To see the test results:
       kyua report  

For further information on using the test suite, read tests(7):
       man tests

Description of FreeBSD test suite
=================================
The build of the test suite is organized in the following manner:

* The build of all test artifacts is protected by the MK_TESTS knob.
  The user can disable these with the WITHOUT_TESTS setting in
  src.conf(5).

* The goal for /usr/tests/ (the installed test programs) is to follow
  the same hierarchy as /usr/src/ wherever possible, which in turn drives
  several of the design decisions described below.  This simplifies the
  discoverability of tests.  We want a mapping such as:

    /usr/src/bin/cp/      -> /usr/tests/bin/cp/
    /usr/src/lib/libc/    -> /usr/tests/lib/libc/
    /usr/src/usr.bin/cut/ -> /usr/tests/usr.bin/cut/
    ... and many more ...

* Test programs for specific utilities and libraries are located next
  to the source code of such programs.  For example, the tests for the
  src/lib/libcrypt/ library live in src/lib/libcrypt/tests/.  The tests/
  subdirectory is optional and should, in general, be avoided.

* The src/tests/ hierarchy (this directory) provides generic test
  infrastructure and glue code to join all test programs together into
  a single test suite definition.

* The src/tests/ hierarchy also includes cross-functional test programs:
  i.e. test programs that cover more than a single utility or library
  and thus don't fit anywhere else in the tree.  Consider this to follow
  the same rationale as src/share/man/: this directory contains generic
  manual pages while the manual pages that are specific to individual
  tools or libraries live next to the source code.

In order to keep the src/tests/ hierarchy decoupled from the actual test
programs being installed --which is a worthy goal because it simplifies
the addition of new test programs and simplifies the maintenance of the
tree-- the top-level Kyuafile does not know which subdirectories may
exist upfront.  Instead, such Kyuafile automatically detects, at
run-time, which */Kyuafile files exist and uses those directly.

Similarly, every directory in src/ that wants to install a Kyuafile to
just recurse into other subdirectories reuses this Kyuafile with
auto-discovery features.  As an example, take a look at src/lib/tests/
whose sole purpose is to install a Kyuafile into /usr/tests/lib/.
The goal in this specific case is for /usr/tests/lib/ to be generated
entirely from src/lib/.

-- 
$FreeBSD$