freebsd-nq

History

Conrad Meyer 179f62805c random(4): Fortuna: allow increased concurrency Add experimental feature to increase concurrency in Fortuna. As this diverges slightly from canonical Fortuna, and due to the security sensitivity of random(4), it is off by default. To enable it, set the tunable kern.random.fortuna.concurrent_read="1". The rest of this commit message describes the behavior when enabled. Readers continue to update shared Fortuna state under global mutex, as they do in the status quo implementation of the algorithm, but shift the actual PRF generation out from under the global lock. This massively reduces the CPU time readers spend holding the global lock, allowing for increased concurrency on SMP systems and less bullying of the harvestq kthread. It is somewhat of a deviation from FS&K. I think the primary difference is that the specific sequence of AES keys will differ if READ_RANDOM_UIO is accessed concurrently (as the 2nd thread to take the mutex will no longer receive a key derived from rekeying the first thread). However, I believe the goals of rekeying AES are maintained: trivially, we continue to rekey every 1MB for the statistical property; and each consumer gets a forward-secret, independent AES key for their PRF. Since Chacha doesn't need to rekey for sequences of any length, this change makes no difference to the sequence of Chacha keys and PRF generated when Chacha is used in place of AES. On a GENERIC 4-thread VM (so, INVARIANTS/WITNESS, numbers not necessarily representative), 3x concurrent AES performance jumped from ~55 MiB/s per thread to ~197 MB/s per thread. Concurrent Chacha20 at 3 threads went from roughly ~113 MB/s per thread to ~430 MB/s per thread. Prior to this change, the system was extremely unresponsive with 3-4 concurrent random readers; each thread had high variance in latency and throughput, depending on who got lucky and won the lock. "rand_harvestq" thread CPU use was high (double digits), seemingly due to spinning on the global lock. After the change, concurrent random readers and the system in general are much more responsive, and rand_harvestq CPU use dropped to basically zero. Tests are added to the devrandom suite to ensure the uint128_add64 primitive utilized by unlocked read functions to specification. Reviewed by: markm Approved by: secteam(delphij) Relnotes: yes Differential Revision: https://reviews.freebsd.org/D20313		2019-06-17 20:29:13 +00:00
..
etc	Add supporting changes for `Add limited sandbox capability to "make check"`	2017-08-14 19:21:37 +00:00
freebsd_test_suite	Fix sys/opencrypto/blake2_test when kern.cryptodevallowsoft=0	2018-08-16 23:49:56 +00:00
sys	random(4): Fortuna: allow increased concurrency	2019-06-17 20:29:13 +00:00
Kyuafile
Makefile	Use MK_CHECK_USE_SANDBOX in tests/..., to deal with the fact that	2017-08-02 22:24:08 +00:00
Makefile.depend	DIRDEPS_BUILD: Connect MK_TESTS.	2016-03-09 22:46:01 +00:00
Makefile.inc0	Use bsd.opts.mk, not src.opts.mk	2017-08-03 00:35:35 +00:00
README	Copy README into /usr/tests	2015-11-20 03:24:04 +00:00

README

src/tests: The FreeBSD test suite
=================================

To run the FreeBSD test suite:
(1)  Make sure that kyua is installed:
       pkg install kyua
(2)  To run the tests:
       kyua test -k /usr/tests/Kyuafile
(3)  To see the test results:
       kyua report  

For further information on using the test suite, read tests(7):
       man tests

Description of FreeBSD test suite
=================================
The build of the test suite is organized in the following manner:

* The build of all test artifacts is protected by the MK_TESTS knob.
  The user can disable these with the WITHOUT_TESTS setting in
  src.conf(5).

* The goal for /usr/tests/ (the installed test programs) is to follow
  the same hierarchy as /usr/src/ wherever possible, which in turn drives
  several of the design decisions described below.  This simplifies the
  discoverability of tests.  We want a mapping such as:

    /usr/src/bin/cp/      -> /usr/tests/bin/cp/
    /usr/src/lib/libc/    -> /usr/tests/lib/libc/
    /usr/src/usr.bin/cut/ -> /usr/tests/usr.bin/cut/
    ... and many more ...

* Test programs for specific utilities and libraries are located next
  to the source code of such programs.  For example, the tests for the
  src/lib/libcrypt/ library live in src/lib/libcrypt/tests/.  The tests/
  subdirectory is optional and should, in general, be avoided.

* The src/tests/ hierarchy (this directory) provides generic test
  infrastructure and glue code to join all test programs together into
  a single test suite definition.

* The src/tests/ hierarchy also includes cross-functional test programs:
  i.e. test programs that cover more than a single utility or library
  and thus don't fit anywhere else in the tree.  Consider this to follow
  the same rationale as src/share/man/: this directory contains generic
  manual pages while the manual pages that are specific to individual
  tools or libraries live next to the source code.

In order to keep the src/tests/ hierarchy decoupled from the actual test
programs being installed --which is a worthy goal because it simplifies
the addition of new test programs and simplifies the maintenance of the
tree-- the top-level Kyuafile does not know which subdirectories may
exist upfront.  Instead, such Kyuafile automatically detects, at
run-time, which */Kyuafile files exist and uses those directly.

Similarly, every directory in src/ that wants to install a Kyuafile to
just recurse into other subdirectories reuses this Kyuafile with
auto-discovery features.  As an example, take a look at src/lib/tests/
whose sole purpose is to install a Kyuafile into /usr/tests/lib/.
The goal in this specific case is for /usr/tests/lib/ to be generated
entirely from src/lib/.

-- 
$FreeBSD$