This change includes the framework for testing the auditability of various
syscalls, and includes changes for the first 12. The tests will start
auditd(8) if needed, though they'll be much faster if it's already running.
The syscalls tested in this commit include mkdir(2), mkdirat(2), mknod(2),
mknodat(2), mkfifo(2), mkfifoat(2), link(2), linkat(2), symlink(2),
symlinkat(2), rename(2), and renameat(2).
Submitted by: aniketp
MFC after: 2 weeks
Sponsored by: Google, Inc (GSoC 2018)
Differential Revision: https://reviews.freebsd.org/D15286
Previously it was possible to connect a socket (which had the
CAP_CONNECT right) by calling "connectat(AT_FDCWD, ...)" even in
capabilties mode. This combination should be treated the same as a call
to connect (i.e. forbidden in capabilities mode). Similarly for bindat.
Disable connectat/bindat with AT_FDCWD in capabilities mode, fix up the
documentation and add tests.
PR: 222632
Submitted by: Jan Kokemüller <jan.kokemueller@gmail.com>
Reviewed by: Domagoj Stolfa
MFC after: 1 week
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D15221
A pipe was was left over from a development version of pdeathsig.c and
is not needed.
Process C waits for a signal that'll be generated when process B
exists. Process B waits for process D to send it a byte via pipe_db
before it exits. Process D sends the byte after it has started
ptrace()ing process C. The point of the test is to show that process C
receives the signal because process B exited, even though C has been
reparented to process D. The pipe pipe_cd isn't doing anything useful
(though in an earlier version of the patch it did). Clean that up by
removing the useless pipe.
Submitted by: Thomas Munro
MFC after: 6 days
Differential revision: https://reviews.freebsd.org/D15214
- ptrace__breakpoint_siginfo tests that a SIGTRAP for a software breakpoint
in userland triggers a SIGTRAP with a signal code of TRAP_BRKPT.
- ptrace__step_siginfo tests that a SIGTRAP reported for a step after
stepping via PT_STEP or PT_SETSTEP has a signal code of TRAP_TRACE.
- Use a single list of platforms to define HAVE_BREAKPOINT for platforms
that expose a functional breakpoint() inline to userland. Replace
existing lists of platform tests with HAVE_BREAKPOINT instead.
- Add support for advancing PC past a breakpoint inserted via breakpoint()
to support the existing ptrace__PT_CONTINUE_different_thread test on
non-x86 platforms (x86 advances the PC past the breakpoint instruction,
but other platforms do not). This is implemented by defining a new
SKIP_BREAK macro which accepts a pointer to a 'struct reg' as its sole
argument and modifies the contents to advance the PC. The intention is
to use it in between PT_GETREGS and PT_SETREGS.
Tested on: amd64, i386, mips (after adding a breakpoint() to mips)
MFC after: 1 month
-> PROC_PDEATHSIG_STATUS for consistency with other procctl(2)
operations names.
Requested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 13 days
Allow processes to request the delivery of a signal upon death of
their parent process. Supposed consumer of the feature is PostgreSQL.
Submitted by: Thomas Munro
Reviewed by: jilles, mjg
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D15106
fget_cap() tries to do a cheaper snapshot of a file descriptor without
holding the file descriptor lock. This snapshot does not do a deep
copy of the ioctls capability array, but instead uses a different
return value to inform the caller to retry the copy with the lock
held. However, filecaps_copy() was returning 1 to indicate that a
retry was required, and fget_cap() was checking for 0 (actually
'!filecaps_copy()'). As a result, fget_cap() did not do a deep copy
of the ioctls array and just reused the original pointer. This cause
multiple file descriptor entries to think they owned the same pointer
and eventually resulted in duplicate frees.
The only code path that I'm aware of that triggers this is to create a
listen socket that has a restricted list of ioctls and then call
accept() which calls fget_cap() with a valid filecaps structure from
getsock_cap().
To fix, change the return value of filecaps_copy() to return true if
it succeeds in copying the caps and false if it fails because the lock
is required. I find this more intuitive than fixing the caller in
this case. While here, change the return type from 'int' to 'bool'.
Finally, make filecaps_copy() more robust in the failure case by not
copying any of the source filecaps structure over. This avoids the
possibility of leaking a pointer into a structure if a similar future
caller doesn't properly handle the return value from filecaps_copy()
at the expense of one more branch.
I also added a test case that panics before this change and now passes.
Reviewed by: kib
Discussed with: mjg (not a fan of the extra branch)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15047
This behavior is already documented by the man page, and suggested by POSIX.
Reviewed by: jhb
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D15099
Do not build or install pf tests if WITHOUT_PF is set. This fixes the build
failure with WITHOUT_PF=yes.
Reported by: Vladimir Zakharov <zakharov.vv@gmail.com>
There was a memory leak in the DIOCRADDTABLES ioctl() code which could
be triggered by trying to add tables with the same name.
Try to provoke this memory leak. It was fixed in r331225.
MFC after: 1 week
Validate the DIOCRGETTABLES, DIOCRGETTSTATS, DIOCRCLRTSTATS and
DIOCRSETTFLAGS ioctls with invalid values. These may succeed (because
the kernel uses the minimally required size, not the specified size),
but should not trigger kernel panics.
MFC after: 1 week
The upstream repository is on github BLAKE2/libb2. Files landed in
sys/contrib/libb2 are the unmodified upstream files, except for one
difference: secure_zero_memory's contents have been replaced with
explicit_bzero() only because the previous implementation broke powerpc
link. Preferential use of explicit_bzero() is in progress upstream, so
it is anticipated we will be able to drop this diff in the future.
sys/crypto/blake2 contains the source files needed to port libb2 to our
build system, a wrapped (limited) variant of the algorithm to match the API
of our auth_transform softcrypto abstraction, incorporation into the Open
Crypto Framework (OCF) cryptosoft(4) driver, as well as an x86 SSE/AVX
accelerated OCF driver, blake2(4).
Optimized variants of blake2 are compiled for a number of x86 machines
(anything from SSE2 to AVX + XOP). On those machines, FPU context will need
to be explicitly saved before using blake2(4)-provided algorithms directly.
Use via cryptodev / OCF saves FPU state automatically, and use via the
auth_transform softcrypto abstraction does not use FPU.
The intent of the OCF driver is mostly to enable testing in userspace via
/dev/crypto. ATF tests are added with published KAT test vectors to
validate correctness.
Reviewed by: jhb, markj
Obtained from: github BLAKE2/libb2
Differential Revision: https://reviews.freebsd.org/D14662
It mistakenly believes the 'static' keyword must come first. Fix PPC,
Sparc64, and maybe MIPS world. Fallout from r331279.
Reported by: tinderbox (results come slowly)
The general idea here is to provide userspace programs with well-defined
sources of entropy, in a fashion that doesn't require opening a new file
descriptor (ulimits) or accessing paths (/dev/urandom may be restricted
by chroot or capsicum).
getrandom(2) is the more general API, and comes from the Linux world.
Since our urandom and random devices are identical, the GRND_RANDOM flag
is ignored.
getentropy(3) is added as a compatibility shim for the OpenBSD API.
truss(1) support is included.
Tests for both system calls are provided. Coverage is believed to be at
least as comprehensive as LTP getrandom(2) test coverage. Additionally,
instructions for running the LTP tests directly against FreeBSD are provided
in the "Test Plan" section of the Differential revision linked below. (They
pass, of course.)
PR: 194204
Reported by: David CARLIER <david.carlier AT hardenedbsd.org>
Discussed with: cperciva, delphij, jhb, markj
Relnotes: maybe
Differential Revision: https://reviews.freebsd.org/D14500
The 'physio' fast-path used by AIO requests on md(4) devices, is not
gated on the unsafe_aio knob. Prior to r327755, some AIO requests could
fail the fast-path and fall back to the slow-path (requests for devices
not supporting unmapped I/O and requests which failed with EFAULT during
the fast-path). However, those cases now return a suitable error rather
than using the slow-path.
PR: 217261
Reviewed by: asomers
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D14742
The code assumed that disks (devices) used for testing are always named
like /dev/foo, but there is no reason for that restriction and we can
easily support paths like /dev/stripe/bar.
The change is to account for a different order in which the recursive
destroy may be attempted. If we first try a dataset that can be destroyed
then it will be destroyed, but if we first try a dataset that cannot be
destroyed then we will not attempt to destroy the other dataset.
It was originally written by Sun as part of the STF (Solaris test framework).
They open sourced it in OpenSolaris, then HighCloud partially ported it to
FreeBSD, and Spectra Logic finished the port. We also added many testcases,
fixed many broken ones, and converted them all to the ATF framework. We've had
help along the way from avg, araujo, smh, and brd.
By default most of the tests are disabled. Set the disks Kyua variable to
enable them.
Submitted by: asomers, will, justing, ken, brd, avg, araujo, smh
Sponsored by: Spectra Logic Corp, HighCloud
If the underlying provider's physical path is null, then the gpart device's
physical path will be, too. Otherwise, it will append the partition name,
such as "/p1" or "/s1/a". This will make gpart work better with zfsd(8).
PR: 224965
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D14010
If the underlying provider's physical path is null, then the geli device's
physical path will be, too. Otherwise, it will append "/eli". This will make
geli work better with zfsd(8).
PR: 224962
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D13979
Don't declare some types that FreeBSD incorrectly declares.
Fix an incorrect call to open() (missing mode).
ANSIfy prototypes.
Enable SysV message queue, semaphore, and shared memory tests.
With exception of the workaround for union semun, these fixes have been
committed to NetBSD.
Reviewed by: asomers
Approved by: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13471
* Registers TRNG source for random(4)
* Finds available queues, LSBs; allocates static objects
* Allocates a shared MSI-X for all queues. The hardware does not have
separate interrupts per queue. Working interrupt mode driver.
* Computes SHA hashes, HMAC. Passes cryptotest.py, cryptocheck tests.
* Does AES-CBC, CTR mode, and XTS. cryptotest.py and cryptocheck pass.
* Support for "authenc" (AES + HMAC). (SHA1 seems to result in
"unaligned" cleartext inputs from cryptocheck -- which the engine
cannot handle. SHA2 seems to work fine.)
* GCM passes for block-multiple AAD, input lengths
Largely based on ccr(4), part of cxgbe(4).
Rough performance averages on AMD Ryzen 1950X (4kB buffer):
aesni: SHA1: ~8300 Mb/s SHA256: ~8000 Mb/s
ccp: ~630 Mb/s SHA256: ~660 Mb/s SHA512: ~700 Mb/s
cryptosoft: ~1800 Mb/s SHA256: ~1800 Mb/s SHA512: ~2700 Mb/s
As you can see, performance is poor in comparison to aesni(4) and even
cryptosoft (due to high setup cost). At a larger buffer size (128kB),
throughput is a little better (but still worse than aesni(4)):
aesni: SHA1:~10400 Mb/s SHA256: ~9950 Mb/s
ccp: ~2200 Mb/s SHA256: ~2600 Mb/s SHA512: ~3800 Mb/s
cryptosoft: ~1750 Mb/s SHA256: ~1800 Mb/s SHA512: ~2700 Mb/s
AES performance has a similar story:
aesni: 4kB: ~11250 Mb/s 128kB: ~11250 Mb/s
ccp: ~350 Mb/s 128kB: ~4600 Mb/s
cryptosoft: ~1750 Mb/s 128kB: ~1700 Mb/s
This driver is EXPERIMENTAL. You should verify cryptographic results on
typical and corner case inputs from your application against a known- good
implementation.
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12723
Scapy requires the Raw payload to be a string, which was not the case. This
caused the pft_ping.py script to fail, which in turn caused the test to fail.
Reduce the geli tests' runtime by about a third:
* In integrity_test:copy, use a file-backed md(4) device instead of a
malloc'd one. That way we can corrupt the underlying storage without
needing to detach and reattach the geli device.
* In integrity_test:{copy, hmac, data} and onetime_test:{onetime,
onetime_a}, move reads of /dev/random out of the loop.
MFC after: 2 weeks
The trick is not to destroy an md(4) device during a test. That can create
a "double-free" situation, because we also destroy md devices during test
cleanup.
MFC after: 2 weeks
I'm leaving readonly_test and nokey_test alone for now. In a future commit
they should be broken up into several smaller test cases and distributed
between multiple files.
Reviewed by: ngie
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D13717
The resize test used bsdlabel(8), which is not available on all
architectures. Change it to use gpart(8) instead, which should be available
everywhere.
PR: 221763
Reported by: andrew
MFC after: 2 weeks
This change mostly reverts r293436, which introduced the bug due to a belief
that geli(8) would allocate md(4) devices by itself. However, that belief is
incorrect. Instead of using linear probing to find available md(4) numbers,
it's best to use the existing attach_md function.
Reviewed by: ngie
MFC after: 2 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13666
Some of the ptrace tests need to wait for a child process to become a
zombie before preceding. The parent process polls the child process
via the kern.proc.pid sysctl to wait for it to become a zombie.
Previously the code polled until the sysctl failed with ESRCH. Now it
will poll until either the sysctl fails with ESRCH (for compatiblity
with older kernels) or returns a kinfo_proc structure with the ki_stat
field set to SZOMB.
Reported by: Jenkins
Tested by: markj
Discussed with: mjg
MFC after: 1 week
We can't kldunload in the test head as Kyua interprets any output from
them. This would lead to syntax errors and skipping the entire file.
Move the kld commands into the test case bodies.
Pointed out by: asomers@
Some IPSec in tunnel mode allowing to test multiple IPSec
configurations. These tests are reusing the jail/vnet scripts from pf
tests for generating complex network.
Submitted by: olivier@
Differential Revision: https://reviews.freebsd.org/D13017
Previously, symlinks in FreeBSD were artificially limited to PATH_MAX-2.
Add a short test case to verify the change.
Submitted by: Gaurav Gangalwar <ggangalwar AT isilon.com>
Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12589
tests/sys/aio/aio_kqueue_test.c
Instead of using a hard-coded queue depth, use
vfs.aio.max_aio_queue_per_proc
tests/sys/aio/lio_kqueue_test.c
The old, small limit on lio_listio's operation count was lifted by
change 324941. Raise the operation count as high as possible without
exceeding the process's operation limit.
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
When multiple threads wish to report a tracing event to a debugger,
both threads call ptracestop() and one thread will win the race to be
the reporting thread (p->p_xthread). The debugger uses PT_LWPINFO
with the process ID to determine which thread / LWP is reporting an
event and the details of that event. This event is cleared as a side
effect of the subsequent ptrace event that resumed the process
(PT_CONTINUE, PT_STEP, etc.). However, ptrace() was clearing the
event identified by the LWP ID passed to the resume request even if
that wasn't the 'p_xthread'. This could result in clearing an event
that had not yet been observed by the debugger and leaving the
existing event for 'p_thread' pending so that it was reported a second
time.
Specifically, if the debugger stopped due to a software breakpoint in
one thread, but then switched to another thread that was used to
resume (e.g. if the user switched to a different thread and issued a
step), the resume request (PT_STEP) cleared a pending event (if any)
for the thread being stepped. However, the process immediately
stopped and the first thread reported it's breakpoint event a second
time. The debugger decremented the PC for "both" breakpoint events
which resulted in the PC now pointing into the middle of an
instruction (on x86) and a SIGILL fault when the process was resumed a
second time.
To fix, always clear the pending event for 'p_xthread' when resuming a
process. ptrace() still honors the requested LWP ID when enabling
single-stepping (PT_STEP) or setting a different PC (PT_CONTINUE).
Reported by: GDB testsuite (gdb.threads/continue-pending-status.exp)
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D12794
Test fragmentation handling (i.e. scrub fragment reassemble) code for
IPv6.
Two simple tests: Ping a host (jail) and test forwarding of fragmented
packets.
When cleaning up we must destroy the jails before we destroy the interfaces.
Otherwise we might try to destroy interfaces that belong to a jail, which won't
work and fail to completely clean up.
Pass/block packets in the forwarding path with pf.
Introduce the pft_set_rules() helper function, because we need to
remember to flush states between individual tests. If not we can get
packets passing despite rules blocking them because they match states
created in a previous test.
Extend pft_ping.py to be able to send IPv6 echo requests.
^/head@r323923 changed when MODIFIED is printed at exit. It's better to follow the
documented way of determining whether or not a filesystem is clean per fsck_ffs, i.e.,
ensure that the exit code is either 0 or 7.
The pass/fail determination is brittle prior to this commit, and ^/head@r323923 made
the issue apparent -- thus this needs to be fixed independent of ^/head@r323923.
PR: 222780
MFC after: 1 week
MFC with: r323923
Reported by: Jenkins
If VIMAGE is present we can start jails with their own pf instance. This
makes it fairly easy to run tests.
For example, this basic test verifies that drop/pass and icmp
classification works. It's a basic sanity test for pf, and hopefully an
example on how to write more pf tests.
The tests are skipped if VIMAGE is not enabled.
This work is inspired by the GSoC work of Panagiotes Mousikides.
Differential Revision: https://reviews.freebsd.org/D12580
Some x86 class CPUs have accelerated intrinsics for SHA1 and SHA256.
Provide this functionality on CPUs that support it.
This implements CRYPTO_SHA1, CRYPTO_SHA1_HMAC, and CRYPTO_SHA2_256_HMAC.
Correctness: The cryptotest.py suite in tests/sys/opencrypto has been
enhanced to verify SHA1 and SHA256 HMAC using standard NIST test vectors.
The test passes on this driver. Additionally, jhb's cryptocheck tool has
been used to compare various random inputs against OpenSSL. This test also
passes.
Rough performance averages on AMD Ryzen 1950X (4kB buffer):
aesni: SHA1: ~8300 Mb/s SHA256: ~8000 Mb/s
cryptosoft: ~1800 Mb/s SHA256: ~1800 Mb/s
So ~4.4-4.6x speedup depending on algorithm choice. This is consistent with
the results the Linux folks saw for 4kB buffers.
The driver borrows SHA update code from sys/crypto sha1 and sha256. The
intrinsic step function comes from Intel under a 3-clause BSDL.[0] The
intel_sha_extensions_sha<foo>_intrinsic.c files were renamed and lightly
modified (added const, resolved a warning or two; included the sha_sse
header to declare the functions).
[0]: https://software.intel.com/en-us/articles/intel-sha-extensions-implementations
Reviewed by: jhb
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12452
When crypto_newsession() is given a request for an unsupported capability,
raise a more specific error than EINVAL.
This allows cryptotest.py to skip some HMAC tests that a driver does not
support.
Reviewed by: jhb, rlibby
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12451
- Import print_function from __future__ and use print(..) instead of `print ..`.
- Use repr instead of backticks when the object needs to be dumped, unless
print(..) can do it lazily. Use str instead of backticks as appropriate
for simplification reasons.
This doesn't fully convert these modules over py3k. It just gets over some of
the trivial compatibility hurdles.
I accidentally introduced different whitespace style in r323878. I'm not
used to using tabs for indentation in Python scripts.
Whitespace only; no functional change.
Sponsored by: Dell EMC Isilon
Previously, this test was entirely a no-op as no vector in the NIST-KAT file
has a precisely 20-byte key.
Additionally, not every vector in the file is SHA1. The length field
determines the hash under test, and is now decoded correctly.
Finally, due to a limitation I didn't feel like fixing in cryptodev.py, MACs
are truncated to 16 bytes in this test.
With this change and the uncommitted D12437 (to allow key sizes other than
those used in IPSec), the SHA tests in cryptotest.py actually test something
and e.g. at least cryptosoft passes the test.
Sponsored by: Dell EMC Isilon
For some reason, we only skipped AES-XTS tests if a driver was not in the
aesmodules list. Skip other AES modes as well to prevent spurious failures
in non-AES drivers.
Sponsored by: Dell EMC Isilon
Non-tests/... changes:
- Add HAS_TESTS= to Makefiles with libraries and programs to enable iteration
and propagate the appropriate environment down to *.test.mk.
tests/... changes:
- Add appropriate support Makefile.inc's to set HAS_TESTS in a minimal manner,
since tests/... is a special subdirectory tree compared to the others.
MFC after: 2 months
MFC with: r322511
Reviewed by: arch (silence), testing (silence)
Differential Revision: D12014
Executable bits should be set at install time instead of in the repo.
Setting executable bits on files triggers false positives with Phabricator.
MFC after: 2 months
The layout of st_rdev has changed after this commit, and assumptions made
in the NetBSD tests are no longer valid. Change the hardcoded assumed
values to account for the fact that major/minor are now represented by
64 bits as opposed to the less precise legacy precision of 16 bits.
PR: 221048
Relnotes: st_rdev layout changed; warning about impact of r321920 to
downstream consumers
tests/... is a special snowflake directory and using HAS_TESTS would
result in a nasty layering violation between bsd.tests.mk and
bsd.prog.mk.
Add reachover Makefile.inc's which get the default value from
Makefile.inc0 (inspired by gnu/usr.bin/binutils/Makefile.inc0).
The test code prior to r311893 loaded geom_gate at test start if necessary and
skipped the tests if it couldn't be loaded.
The ATF-ifcation of this test done in r311893 unfortunately dropped this
functionality.
This change restores the geom_gate module load and skips the test(s) if unavailable
in an ATF-like way.
MFC after: 1 month
PR: 220164
Reported by: gjb
Our man pages have always indicated that this was supported, but in fact the
feature was never implemented for lio_listio(2).
Reviewed by: jhb, kib (earlier version)
MFC after: 20 days
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D11680
- Use "atf_check -x 'cmd1 | cmd2'" instead of "cmd1 | atf_check cmd2". The
two forms are idiomatically similar, but subtly different in the sense of
what program invokes the other, and there could be unwanted side effects
of the latter idiom dealing with forking, pipes, etc.
- Remove chmod and instead source coredump_phnum_restore_state.sh directly.
This avoids the need to check the result of the chmod call.
- Fix indentation in an if-block (4 column space indentation -> hard tab).
The testcase no longer fails on ^/head because readelf has established parity
with binutils' copy of readelf.
This issue is not seen on Jenkins because
`test_suites.FreeBSD.allow_sysctl_side_effects` isn't set in kyua.conf on
the CI host, i.e., the test is skipped.
PR: 215019
Tested with: binutils (amd64-binutils-2.28,1); elftoolchain (r3561M)
Remove aio_test's legacy timeout handling and cleanup routines. Instead,
use ATF's builtin capabilities. ATF automatically cleans up newly created
files, too, so we don't have to explicitly unlink them. The only tests than
need a cleanup routine are the md(4) tests, which must destroy their md
device.
Reviewed by: jhb
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D11468
The summary of changes is as follows..
Generic changes::
- Added configure support [2].
- Check for lchmod filesystem support with create_file(..); for
testcases that require lchmod, skip the testcase -- otherwise
use chmod directly [1].
- Added Travis CI integration [2].
- Added utimensat testcases [1].
Linux support::
- Fixed Linux support to pass on later supported versions of
Fedora/Ubuntu [2].
- Conditionally enable posix_fallocate(2) support [2].
OSX support::
- Fixed compilation on OSX [2].
- Added partial OSX support (the test run isn't fully green yet)
[2].
MFC after: 2 months
Obtained from: https://github.com/pjd/pjdfstest/tree/0.1
Relnotes: yes
Submitted by: asomers [1], ngie [2]
Tested with: UFS, ZFS
This change implements NOTE_ABSTIME flag for EVFILT_TIMER, which
specifies that the data field contains absolute time to fire the
event.
To make this useful, data member of the struct kevent must be extended
to 64bit. Using the opportunity, I also added ext members. This
changes struct kevent almost to Apple struct kevent64, except I did
not changed type of ident and udata, the later would cause serious API
incompatibilities.
The type of ident was kept uintptr_t since EVFILT_AIO returns a
pointer in this field, and e.g. CHERI is sensitive to the type
(discussed with brooks, jhb).
Unlike Apple kevent64, symbol versioning allows us to claim ABI
compatibility and still name the new syscall kevent(2). Compat shims
are provided for both host native and compat32.
Requested by: bapt
Reviewed by: bapt, brooks, ngie (previous version)
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D11025
- Refactor kld loading/unloading logic:
-- Use a loop instead of an unrolled one.
-- Check for the module being loaded before trying to load it, to reduce
noise when loading modules that are already loaded.
-- Don't mute stderr from kldload -- it could be potentially useful to
the tester.
-- In the event that the test script was terminated early, it would leave
the modules still attached to the system (which is undesirable).
Always unload the modules at test end with EXIT/SIGINT/SIGTERM so the
system is returned to its former operating state as best possible.
Unload the modules in reverse order, in part for consistency and/or
dependency reasons.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
- `TEST_METADATA.foo` should be `TEST_METADATA.run_tests`: this will unbreak
trying to run the tests on a system without python installed in $PATH.
- The tests require root because they load aesni(4) and/or cryptodev(4) if
not already loaded.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Otherwise, the kernel is free to choose an aribtrary crypto device
rather than the requested device subverting tests that force the use
of a specific device.
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D10762
It was sending only a long's worth (4 or 8 bytes) of data previously
(instead of the entire buffer) via send(2).
MFC after: 1 week
Reported by: Coverity
CID: 1229966, 1229967, 1230004, 1230005
Sponsored by: Dell EMC Isilon
temporary file outside the kyua sandbox
This helps ensure that the file is removed at test exit, and as
a side effect, cures a warning about umasks with Coverity.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
of -1 with err*(3).
An exit code of -1 is implementation defined -- it's best to stick
with something well-defined (1).
MFC after: 3 days
Sponsored by: Dell EMC Isilon
The previous logic was flawed in the sense that it assumed that /dev/md3
was always available. This was a caveat I noted in r306038, that I hadn't
gotten around to solving before now.
Cache the device for the mountpoint after executing mdmfs, then use the
cached value in basic_cleanup(..) when unmounting/disconnecting the md(4)
device.
Apply sed expressions to use reuse logic in the NetBSD code that could
also be applied to FreeBSD, just with different tools.
Differential Revision: D10766
MFC after: 1 week
Reviewed by: bdrewery
Sponsored by: Dell EMC Isilon
These tests have been flapping (failing<->passing) on Jenkins for months.
It passes reliably for me if unsafe AIO is permitted, but it doesn't
pass on Jenkins reliably if unsafe AIO is disabled (the current default).
Mark the tests as requiring unsafe AIO to mitigate the intermittent
failures when unsafe AIO isn't permitted. If the kernel code is changed
to reliably function with md(4) devices using unsafe AIO, this commit can
be reverted.
MFC after: 2 months
PR: 217261
Reported by: Jenkins
Sponsored by: Dell EMC Isilon
These command take an int. The tests work by accident on little-endian,
64-bit systems.
PR: 218919
Tested with: qemu-cheri and CheriBSD built for mips64
Reviewed by: asomers, ngie
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10518
available and if that is true make use of them.
Thank you very much to Andrew Turner for providing help and review the patch!
Reviewed by: andrew
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D10499
This test was failing if run twice because rtadvd takes too long to die.
The rtadvd process from the first run was still running when the
second run created its interfaces. The solution is to use SIGKILL during
the cleanup instead of SIGTERM so rtadvd will die faster.
While I'm here, randomize the addresses used for the test, which makes bugs
like this easier to spot, and fix the cleanup order to be the opposite of
the setup order
PR: 217871
MFC after: 18 days
X-MFC-With: 315458
Sponsored by: Spectra Logic Corp
A couple of the ptrace tests make assumptions about which thread in a
multithreaded process will run after a halt. This makes the tests less
portable across branches, and susceptible to future breakage. Instead,
twiddle thread scheduling and priorities to match the tests'
expectation.
X-MFC with: r313992
Sponsored by: Dell EMC
sys/netinet6/icmp6.c
Use the interface's FIB for source address selection in ICMPv6 error
responses.
sys/netinet6/in6.c
In in6_newaddrmsg, announce arrival of local addresses on the
interface's FIB only. In in6_lltable_rtcheck, use a per-fib ND6
cache instead of a single cache.
sys/netinet6/in6_src.c
In in6_selectsrc, use the caller's fib instead of the default fib.
In in6_selectsrc_socket, remove a superfluous check.
sys/netinet6/nd6.c
In nd6_lle_event, use the interface's fib for routing socket
messages. In nd6_is_new_addr_neighbor, check all FIBs when trying
to determine whether an address is a neighbor. Also, simplify the
code for point to point interfaces.
sys/netinet6/nd6.h
sys/netinet6/nd6.c
sys/netinet6/nd6_rtr.c
Make defrouter_select fib-aware, and make all of its callers pass in
the interface fib.
sys/netinet6/nd6_nbr.c
When inputting a Neighbor Solicitation packet, consider the
interface fib instead of the default fib for DAD. Output NS and
Neighbor Advertisement packets on the correct fib.
sys/netinet6/nd6_rtr.c
Allow installing the same host route on different interfaces in
different FIBs. If rt_add_addr_allfibs=0, only install or delete
the prefix route on the interface fib.
tests/sys/netinet/fibs_test.sh
Clear some expected failures, but add a skip for the newly revealed
BUG217871.
PR: 196361
Submitted by: Erick Turnquist <jhujhiti@adjectivism.org>
Reported by: Jason Healy <jhealy@logn.net>
Reviewed by: asomers
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9451
The ptrace() user has the option of discarding the signal. In such a
case, p_ptevents should not be modified. If the ptrace() user decides to
send a SIGKILL, ptevents will be cleared in ptracestop(). procfs events
do not have the capability to discard the signal, so continue to clear
the mask in that case.
Reviewed by: jhb (initial revision)
MFC after: 1 week
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D9939
This change moves the tests added in r313962 to an existing directory
structure used by the geli TAP tests. It also, renames the test from
pbkdf2 to pbkdf2_test .
The changes to ObsoleteFiles.inc are being committed separately as they
aren't needed for the MFC to ^/stable/11, etc, if the MFC for the tests
is done all in one commit.
MFC after: 2 weeks
X-MFC with: r313962, r313972-r313973
Reviewed by: allanjude
Sponsored by: Dell EMC Isilon
Differential Revision: D9985
Suppose a traced process is stopped in ptracestop() due to receipt of a
SIGSTOP signal, and is awaiting orders from the tracing process on how
to handle the signal. Before sending any such orders, the tracing
process exits. This should kill the traced process. But suppose a second
thread handles the SIGKILL and proceeds to exit1(), calling
thread_single(). The first thread will now awaken and will have a chance
to check once more if it should go to sleep due to the SIGSTOP. It must
not sleep after P_SINGLE_EXIT has been set; this would prevent the
SIGKILL from taking effect, leaving a stopped orphan behind after the
tracing process dies.
Also add new tests for this condition.
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D9890
Tests that an interface can get a SLAAC address and that it inserts its
routes into the correct fib. Does not test anything to do with NDP.
PR: 196361
Reviewed by: Erick Turnquist <jhujhiti@adjectivism.org>
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9776
In the Kyua era, it's no longer necessary to set PJDFSTEST_TEST_PATH. Just
use TMPDIR instead.
Reviewed by: ngie
MFC after: 3 weeks
Relnotes: yes
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9340
When a thread is stopped in ptracestop(), the ptrace(2) user may request
a signal be delivered upon resumption of the thread. Heretofore, those signals
were discarded unless ptracestop()'s caller was issignal(). Fix this by
modifying ptracestop() to queue up signals requested by the ptrace user that
will be delivered when possible. Take special care when the signal is SIGKILL
(usually generated from a PT_KILL request); no new stop events should be
triggered after a PT_KILL.
Add a number of tests for the new functionality. Several tests were authored
by jhb.
PR: 212607
Reviewed by: kib
Approved by: kib (mentor)
MFC after: 2 weeks
Sponsored by: Dell EMC
In collaboration with: jhb
Differential Revision: https://reviews.freebsd.org/D9260
committed in r313972
The code committed in r313962 implicitly relies on python 2.x to generate
testvect.h . There are a handful of issues with this approach:
- python is not an explicit build dependency for FreeBSD
- python 2.x is deprecated and will be removed sometime in the future
(potentially before 11.x's EOL), and the script does not function with
python 3.5 (it uses deprecated idioms and incompatible function calls).
- python(1) (by default) lives in /usr/local/bin (${LOCALBASE}/bin) and
gentestvect.py is a dependency of testvect.h (prior to r313972) which
means that if the mtime of the generator script was newer than the
mtime of the test vector, it could cause a spurious build failure in
build time or at install time.
A better solution using C/C++ should be devised.
Discussed with: allanjude
MFC after: 2 weeks
X-MFC with: r313962, r313972
Sponsored by: Dell EMC Isilon
Using relative paths imply working directory (in this case .OBJDIR), whereas the
sources live in the .CURDIR-relative path.
MFC after: 2 weeks
X-MFC with: r313962
Pointyhat to: allanjude
Sponsored by: Dell EMC Isilon
The PBKDF2 in sys/geom/eli/pkcs5v2.c is around half the speed it could be
GELI's PBKDF2 uses a simple benchmark to determine a number of iterations
that will takes approximately 2 seconds. The security provided is actually
half what is expected, because an attacker could use the optimized
algorithm to brute force the key in half the expected time.
With this change, all newly generated GELI keys will be approximately 2x
as strong. Previously generated keys will talk half as long to calculate,
resulting in faster mounting of encrypted volumes. Users may choose to
rekey, to generate a new key with the larger default number of iterations
using the geli(8) setkey command.
Security of existing data is not compromised, as ~1 second per brute force
attempt is still a very high threshold.
PR: 202365
Original Research: https://jbp.io/2015/08/11/pbkdf2-performance-matters/
Submitted by: Joe Pixton <jpixton@gmail.com> (Original Version), jmg (Later Version)
Reviewed by: ed, pjd, delphij
Approved by: secteam, pjd (maintainer)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D8236
Some recent changes to vm related to mmap(2) have broken the prot checks that
would result with an EINVAL with this case
I suspect r313352 is the root-cause the issue
PR: 216976
Sponsored by: Dell EMC Isilon
results of commands
As noted in r313008, the underlying issue was that geom_gate device
creation wasn't created at ggatel command completion, but some short
time after. ggatec(8) employs similar logic when creating geom_gate(4)
devices.
Switch from retry loops (after the ggatec/dd write calls) to
wait_for_ggate_device function calls after calling ggatec(8) instead
to detect the presence of the /dev/ggate* device, as this function is
sufficient for determining whether or not the character device is ready
for testing
While here, use atf_check consistently with all dd calls to ensure that
data output is as expected.
MFC after: 1 week
Reviewed by: asomers
Differential Revision: D9409
Sponsored by: Dell EMC Isilon
The test assumed that `ggatel create` created a device on completion, but that's
incorrect. This squashes the race by waiting for the device to appear, as
`ggatel create` daemonizes before issuing an ioctl to geom_gate(4) if not called
with `-v`.
Discussed with: asomers
MFC after: 1 week
PR: 204616
Sponsored by: Dell EMC Isilon
Derived from an implementation by Mark Adler.
The fast loop performs three simultaneous CRCs over subsets of the data
before composing them. This takes advantage of certain properties of
the CRC32 implementation in Intel hardware. (The CRC instruction takes 1
cycle but has 2-3 cycles of latency.)
The CRC32 instruction does not manipulate FPU state.
i386 does not have the crc32q instruction, so avoid it there. Otherwise
the implementation is identical to amd64.
Add basic userland tests to verify correctness on a variety of inputs.
PR: 216467
Reported by: Ben RUBSON <ben.rubson at gmail.com>
Reviewed by: kib@, markj@ (earlier version)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9342
* Add tests for aio_suspend(2).
* Add tests for polled completion notification.
* Test the full matrix of file descriptor types and completion notification
mechanisms.
* Don't bother with mkstemp, because ATF runs every test in its own temp dir.
* Fix some typos.
* Remove extraneous ATF_REQUIRE_KERNEL_MODULE calls.
Reviewed by: jhb
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D9045
between exp(3) and `exp` var.
The approach taken previously was not ideal for multiple
functional and stylistic reasons.
Add to existing sed call in Makefile to replace `exp` with
`exponent` instead.
MFC after: 13 days
Requested by: bde
The primary goal for doing this is to leverage the work done in r312114
for enabling WARNS to address trivial code quality issues with new tests
MFC after: 6 days
Tested with: make tinderbox
Sponsored by: Dell EMC Isilon
The primary goal for doing this is to leverage the work done in r312114
for enabling WARNS to address trivial code quality issues with new tests
MFC after: 6 days
Tested with: make tinderbox
Sponsored by: Dell EMC Isilon
Use %zu for printing out results from nitems, as it's size_t based
MFC after: 1 week
X-MFC with: r312120
Reported by: gcc (mips:mipsel tinderbox)
Sponsored by: Dell EMC Isilon
State that execve failed instead of just printing out the program name
and strerror(errno) via err(3).
MFC after: 3 days
X-MFC with: r312118
Sponsored by: Dell EMC Isilon
Don't implement some terrible hack on a test by test basis. The
framework fix is straightforward and can be chased up in the original
bug.
Reviewed by: ngie ("be my guest")
This ensures their cleanup routines will be run even if they should timeout.
tests/sys/geom/class/gate/ggate_test.sh
tests/sys/geom/class/gate/Makefile
Add an ATF test with three testcases, one for each TAP test. Use
ATF-style cleanup functions, and convert sleeps to polling loops.
ObsoleteFiles.inc
tests/sys/geom/class/gate/conf.sh
tests/sys/geom/class/gate/1_test.sh
tests/sys/geom/class/gate/2_test.sh
tests/sys/geom/class/gate/3_test.sh
Delete TAP test files
Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D8891
The tests don't work when reading/writing to file descriptors in the
sandbox after entering capability mode (and wouldn't have, regardless
of the framework), so adjust the tests so they function within the
framework.
For tests that enter capability mode over the course of the test, the
following is now done:
1. Fork child process for capability mode test.
2. In child...
i. Enter capability mode.
ii. Test invariants.
iii. Exit after calling test function.
3. Collect status for child and determine whether or not it completed
successfully.
In order to test the invariants in the child process, they now use assert(3)
instead of ATF_REQUIRE*, as the atf-c-api functions right to results files
in the directories in order to determine where and how tests fail.
While in the area, fix several -Wshadow and -Wunused warnings found when I
bumped WARNS up to 6, and fix some minor style(9) issues with indentation
and type alignment.
PR: 215690
Also, add test cases for the -p, -P, and -s options to gnop create
Reviewed by: ngie
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D8892
There is no reduction in test coverage. On my system runtime is reduced
from 38m32s to 6m24s.
tests/sys/geom/class/eli/conf.sh
tests/sys/geom/class/eli/init_a_test.sh
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/integrity_copy_test.sh
tests/sys/geom/class/eli/integrity_data_test.sh
tests/sys/geom/class/eli/integrity_hmac_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
tests/sys/geom/class/eli/onetime_test.sh
Move the looping code into common functions in conf.sh, and remove
alias ciphers from the list.
tests/sys/geom/class/eli/init_a_test.sh
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/integrity_copy_test.sh
tests/sys/geom/class/eli/integrity_data_test.sh
tests/sys/geom/class/eli/integrity_hmac_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
Move a few commands that don't need to be in the inner loop out.
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
Reduce the sector count
tests/sys/geom/class/eli/Makefile
tests/sys/geom/class/eli/init_alias_test.sh
Add a test for initializing a GELI device using one of the cipher
aliases, and check that the alias is correctly interpreted.
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D8814
Note: this test still requires binutils readelf (from ports) to pass, until a
few issues are resolved in elftoolchain.
PR: 215019
Reported by: ngie@
Sponsored by: Dell EMC Isilon
Changes were made to ZFS in the past year with respect to how ACLs
are handled, causing failures in this test. Mark it TODO so (hopefully)
someone more knowledgeable (like mav or trasz) will fix the code or the
test.
PR: 212323
The requirement is set via `atf_set "require.config" "allow_sysctl_side_effects"',
not `atf_set "require.config" "allow_sysctl_side_effects"'
X-MFC with: r308177
Pointyhat to: cmeyer
have both big and little-endian images in tree).
Also we don't known the endianness of the platform the
image was generated on.
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
file descriptor for the given posix mqueue. Export the
timer_oshandle_np() symbol to get ktimer id for the given posix timer.
Requested by: Lewis Donzis <lew@perftech.com>
Reviewed by: jilles
Discussed with: kan
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This is a safety belt to ensure that the /tmp sticky bit stuff doesn't
get whacked by accident if someone runs the script outright
MFC after: 1 week
X-MFC with: r305916
Sponsored by: Dell EMC Isilon
This is required to ensure that the temporary script can be executed,
as the default mode is apparently too restrictive
MFC after: 3 days
Sponsored by: Dell EMC Isilon
If wait4() or wait6() return 0 because of WNOHANG, the status, rusage and
wrusage information should not be returned.
PR: 212048
Reported by: Casey Lucas
MFC after: 2 weeks
This is being done to clearly distinguish the libkqueue tests
from the (soon to be imported) NetBSD tests.
MFC after: 58 days
Sponsored by: EMC / Isilon Storage Division
This will make the exit info more meaningful if/when zpool create fails,
and establishes parity with the other 2 zfs acl testcases (01, 03).
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
Change the vnode tests to use the current directory when creating temporary
files, which we can assume is a volatile work directory, and then make the
kqueue_test.sh driver _not_ abandon the directory created by Kyua.
This makes the various kqueue tests independent of each other, and ensures
the temporary file is cleaned up on failure.
Problem spotted by asomers@ when reviewing D4254.
- Use correct lock in aio_cancel_sync when dequeueing job.
- Add _locked variants of aio_set/clear_cancel_function and use those
to avoid lock recursion when adding and removing fsync jobs to the
per-process sync queue.
- While here, add a basic test for aio_fsync().
PR: 211390
Reported by: Randy Westlund <rwestlun@gmail.com>
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7339
First, PL_FLAG_FORKED events now also set a PL_FLAG_VFORKED flag when
the new child was created via vfork() rather than fork(). Second, a
new PL_FLAG_VFORK_DONE event can now be enabled via the PTRACE_VFORK
event mask. This new stop is reported after the vfork parent resumes
due to the child calling exit or exec. Debuggers can use this stop to
reinsert breakpoints in the vfork parent process before it resumes.
Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D7045
ptrace() now stores a mask of optional events in p_ptevents. Currently
this mask is a single integer, but it can be expanded into an array of
integers in the future.
Two new ptrace requests can be used to manipulate the event mask:
PT_GET_EVENT_MASK fetches the current event mask and PT_SET_EVENT_MASK
sets the current event mask.
The current set of events include:
- PTRACE_EXEC: trace calls to execve().
- PTRACE_SCE: trace system call entries.
- PTRACE_SCX: trace syscam call exits.
- PTRACE_FORK: trace forks and auto-attach to new child processes.
- PTRACE_LWP: trace LWP events.
The S_PT_SCX and S_PT_SCE events in the procfs p_stops flags have
been replaced by PTRACE_SCE and PTRACE_SCX. PTRACE_FORK replaces
P_FOLLOW_FORK and PTRACE_LWP replaces P2_LWP_EVENTS.
The PT_FOLLOW_FORK and PT_LWP_EVENTS ptrace requests remain for
compatibility but now simply toggle corresponding flags in the
event mask.
While here, document that PT_SYSCALL, PT_TO_SCE, and PT_TO_SCX both
modify the event mask and continue the traced process.
Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D7044
setting a 32 bit value on each socket. This can be used by applications
and DTrace as a rendezvous point so that an applicaton's data can
more easily be captured at run time. Expose the user cookie via
DTrace by updating the translator in tcp.d and add a quick test
program, a TCP server, that sets the cookie on each connection
accepted.
Reviewed by: hiren
MFC after: 1 week
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D7152
sys/sys/bitstring.h
Fix a rounding calculation that could undersize a bitstring on
32-bit platforms.
tests/sys/sys/bitstring_test.h
Add a test for bitstr_size
PR: 210260
Reported by: Mark Millard
Reviewed by: gibbs
Approved by: re (marius)
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6848
File and disk-backed I/O requests store counts of read/written disk
blocks in each AIO job so that they can be charged to the thread that
completes an AIO request via aio_return() or aio_waitcomplete(). This
change extends AIO jobs to store counts of received/sent messages and
updates socket backends to set these counts accordingly. Note that
the socket backends are careful to only charge a single messages for
each AIO request even though a single request on a blocking socket might
invoke sosend or soreceive multiple times. This is to mimic the
resource accounting of synchronous read/write.
Adjust the UNIX socketpair AIO test to verify that the message resource
usage counts update accordingly for aio_read and aio_write.
Approved by: re (hrs)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D6911
tests/sys/acl/Makefile
add perl to the required_programs for all tests in this directory
Reviewed by: ngie
Approved by: re (gjb)
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6870
After the previous changes to fix requests on blocking sockets to complete
across multiple operations, an edge case exists where a request can be
cancelled after it has partially completed. POSIX doesn't appear to
dictate exactly how to handle this case, but in general I feel that
aio_cancel() should arrange to cancel any request it can, but that any
partially completed requests should return a partial completion rather
than ECANCELED. To that end, fix the socket AIO cancellation routine to
return a short read/write if a partially completed request is cancelled
rather than ECANCELED.
Sponsored by: Chelsio Communications
Always requeue an AIO job at the head of the socket buffer's queue if
sosend() or soreceive() returns EWOULDBLOCK on a blocking socket.
Previously, requests were only requeued if they returned EWOULDBLOCK
and completed no data. Now after a partial completion on a blocking
socket the request is queued and the remaining request is retried when
the socket is ready. This allows writes larger than the currently
available space on a blocking socket to fully complete. Reads on a
blocking socket that satifsy the low watermark can still return a short
read (same as read()).
In order to track previously completed data, the internal 'status'
field of the AIO job is used to store the amount of previously
computed data.
Non-blocking sockets continue to return short completions for both
reads and writes.
Add a test for a "large" AIO write on a blocking socket that writes
twice the socket buffer size to a UNIX domain socket.
Sponsored by: Chelsio Communications
Add a bit_count function, which efficiently counts the number of bits set in
a bitstring.
sys/sys/bitstring.h
tests/sys/sys/bitstring_test.c
share/man/man3/bitstring.3
Add bit_alloc
sys/kern/subr_unit.c
Use bit_count instead of a naive counting loop in check_unrhdr, used
when INVARIANTS are enabled. The userland test runs about 6x faster
in a generic build, or 8.5x faster when built for Nehalem, which has
the POPCNT instruction.
sys/sys/param.h
Bump __FreeBSD_version due to the addition of bit_alloc
UPDATING
Add a note about the ABI incompatibility of the bitstring(3)
changes, as suggested by lidl.
Suggested by: gibbs
Reviewed by: gibbs, ngie
MFC after: 9 days
X-MFC-With: 299090, 300538
Relnotes: yes
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6255
tests/sys/kern/Makefile
Reenable a disabled compiler warning, the need for which was
eliminated by r299090.
Reviewed by: ngie
MFC after: 4 weeks
X-MFC-With: 299090
Sponsored by: Spectra Logic Corp
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
Two new functions are provided, bit_ffs_at() and bit_ffc_at(), which allow
for efficient searching of set or cleared bits starting from any bit offset
within the bit string.
Performance is improved by operating on longs instead of bytes and using
ffsl() for searches within a long. ffsl() is a compiler builtin in both
clang and gcc for most architectures, converting what was a brute force
while loop search into a couple of instructions.
All of the bitstring(3) API continues to be contained in the header file.
Some of the functions are large enough that perhaps they should be uninlined
and moved to a library, but that is beyond the scope of this commit.
sys/sys/bitstring.h:
Convert the majority of the existing bit string implementation from
macros to inline functions.
Properly protect the implementation from inadvertant macro expansion
when included in a user's program by prefixing all private
macros/functions and local variables with '_'.
Add bit_ffs_at() and bit_ffc_at(). Implement bit_ffs() and
bit_ffc() in terms of their "at" counterparts.
Provide a kernel implementation of bit_alloc(), making the full API
usable in the kernel.
Improve code documenation.
share/man/man3/bitstring.3:
Add pre-exisiting API bit_ffc() to the synopsis.
Document new APIs.
Document the initialization state of the bit strings
allocated/declared by bit_alloc() and bit_decl().
Correct documentation for bitstr_size(). The original code comments
indicate the size is in bytes, not "elements of bitstr_t". The new
implementation follows this lead. Only hastd assumed "elements"
rather than bytes and it has been corrected.
etc/mtree/BSD.tests.dist:
tests/sys/Makefile:
tests/sys/sys/Makefile:
tests/sys/sys/bitstring.c:
Add tests for all existing and new functionality.
include/bitstring.h
Include all headers needed by sys/bitstring.h
lib/libbluetooth/bluetooth.h:
usr.sbin/bluetooth/hccontrol/le.c:
Include bitstring.h instead of sys/bitstring.h.
sbin/hastd/activemap.c:
Correct usage of bitstr_size().
sys/dev/xen/blkback/blkback.c
Use new bit_alloc.
sys/kern/subr_unit.c:
Remove hard-coded assumption that sizeof(bitstr_t) is 1. Get rid of
unrb.busy, which caches the number of bits set in unrb.map. When
INVARIANTS are disabled, nothing needs to know that information.
callapse_unr can be adapted to use bit_ffs and bit_ffc instead.
Eliminating unrb.busy saves memory, simplifies the code, and
provides a slight speedup when INVARIANTS are disabled.
sys/net/flowtable.c:
Use the new kernel implementation of bit-alloc, instead of hacking
the old libc-dependent macro.
sys/sys/param.h
Update __FreeBSD_version to indicate availability of new API
Submitted by: gibbs, asomers
Reviewed by: gibbs, ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6004
Build and install the subr_unit test program originally written by phk, and
run it with the other ATF tests.
tests/sys/kern/Makefile
* Build and install the subr_unit test as a plain test
sys/kern/subr_unit.c
* Reduce the default number of repetitions from 100 to 1, and add a
command-line parser to override it.
* Don't be so noisy by default
* Fix an include problem for the test build
Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6038
- Always munmap memory regions after mmap'ing them.
- Make sure getpagesize() returns a value greater than 0 and use a
cached value instead of always calling getpagesize(3).
- Remove intermediate variable for assigning from $TMPDIR if set in the
environment to eliminate warnings about pointer conversions with "/tmp",
and to mute an invalid buffer overflow concern from Coverity
(snprintf and tacking on a NUL terminator was alleviating that concern
before).
- Remove useless self-test of psize before it's initialized.
- Check the return values of getrlimit/setrlimit.
Cosmetic changes:
- Replace a `(void*)0` with NULL.
- Do some minor whitespace clean up.
- Remove an unnecessary cast to mmap.
- Make all munmap calls use ATF_REQUIRE_MSG instead of using the:
> if (munmap(..) == -1)
> atf_tc_fail(..)
idiom. Employ the new idiom consistently when calling munmap.
CID: 1331351, 1331382-1331386, 1331513, 1331514, 1331565, 1331583, 1331694
Differential Revision: https://reviews.freebsd.org/D6012
MFC after: 2 weeks
Reported by: Coverity
Reviewed by: markj
Sponsored by: EMC / Isilon Storage Division
- close file descriptors after use.
- Always munmap memory regions after mmap'ing them.
- Make sure getpagesize() returns a value greater than 0 and use a
cached value instead of always calling getpagesize(3).
CID: 1331374-1331377, 1331653-1331662
Differential Revision: https://reviews.freebsd.org/D6011
MFC after: 2 weeks
Reported by: Coverity
Reviewed by: cem
Sponsored by: EMC / Isilon Storage Division
The older AIO code awakened all pending AIO requests on a socket
when any data arrived. This could result in AIO daemons blocking on
an empty socket buffer. These requests could not be cancelled
which led to a deadlock during process exit. This test reproduces
this case. The newer AIO code is able to cancel the pending AIO
request correctly.
Reviewed by: ngie (-ish)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D4363
