Add bit_ffs_area_at and bit_ffc_area_at functions for searching a bit
string for a sequence of contiguous set or unset bits of at least the
specified size.
The bit_ffc_area function will be used by the Intel ice driver for
implementing resource assignment logic using a bitstring to represent
whether or not a given index has been assigned or is currently free.
The bit_ffs_area, bit_ffc_area_at and bit_ffs_area_at functions are
implemented for completeness.
I'd like to add further test cases for the new functions, but I'm not
really sure how to add them easily. The new functions depend on specific
sequences of bits being set, while the bitstring tests appear to run for
varying bit sizes.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Submitted by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed by: asomers@, erj@
MFC after: 1 week
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D22400
bit_ffs_at and bit_ffc_at both take _start parameters which indicate to
start searching from _start onwards.
If the given _start index is past the size of the bit string, these
functions will calculate an address of the current bitstring which is
after the expected size. The function will also dereference the memory,
resulting in a read buffer overflow.
The output of the function remains correct, because the tests ensure to
stop the loop if the current bitstring chunk passes the stop bitstring
chunk, and because of a check to ensure the reported _value is never
past _nbits.
However, if <sys/bitstring.h> is ever used in code which is checked by
-fsanitize=undefined, or similar static analysis, it can produce
warnings about reading past the buffer size.
Because of the above mentioned checks, these buffer overflows do not
occur as long as _start is less than _nbits. Additionally, by definition
bit_ffs_at and bif_ffc_at should set _result to -1 in any case where the
_start is after the _nbits.
Check for this case at the start of the function and exit early if so,
preventing the buffer read overflow, and reducing the amount of
computation that occurs.
Note that it may seem odd to ever have code that could call bit_ffc_at
or bit_ffs_at with a _start value greater than _nbits. However, consider
a for-loop that used bit_ffs and bit_ffs_at to loop over a bit string
and perform some operation on each bit that was set. If the last bit of
the bit string was set, the simplest loop implementation would call
bit_ffs_at with a start of _nbits, and expect that to return -1. While
it does infact perform correctly, this is what ultimately triggers the
unexpected buffer read overflow.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Submitted by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed by: asomers@, erj@
MFC after: 1 week
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D22398
After r354748 mld_input() can change the mbuf. The new pointer
is never returned to icmp6_input() and when passed to
icmp6_rip6_input() the mbuf may no longer valid leading to
a panic.
Pass a pointer to the mbuf to mld_input() so we can return an
updated version in the non-error case.
Add a test sending an MLD packet case which will trigger this bug.
Pointyhat to: bz
Reported by: gallatin, thj
MFC After: 2 weeks
X-MFC with: r354748
Sponsored by: Netflix
Co-mingling two things here:
* Addressing some feedback from Konstantin and Kyle re: jail,
capability mode, and a few other things
* Adding audit support as promised.
The audit support change includes a partial refresh of OpenBSM from
upstream, where the change to add shm_rename has already been
accepted. Matthew doesn't plan to work on refreshing anything else to
support audit for those new event types.
Submitted by: Matthew Bryan <matthew.bryan@isilon.com>
Reviewed by: kib
Relnotes: Yes
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D22083
RFC 8200 says:
"If the fragment is a whole datagram (that is, both the Fragment
Offset field and the M flag are zero), then it does not need
any further reassembly and should be processed as a fully
reassembled packet (i.e., updating Next Header, adjust Payload
Length, removing the Fragment header, etc.). .."
That means we should remove the fragment header and make all the adjustments
rather than just skipping over the fragment header. The difference should
be noticeable in that a properly handled atomic fragment triggering an ICMPv6
message at an upper layer (e.g. dest unreach, unreachable port) will not
include the fragment header.
Update the test cases to also test for an unfragmentable part. That is
needed so that the next header is properly updated (not just lengths).
MFC after: 3 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D22155
Remove mentions of fragmentation tests from extension header test.
Remove setting an MTU > IF_MAXMTU from the test cases to avoid warnings;
this was only possible in a local research tree.
MFC after: 2 weeks
Sponsored by: Netflix
Add a simple test case which can exercise some of the IPv6 extension
header code paths. At the moment only a small set of extension headers
is implemented and no options to the ones which take them.
Also implements a "bad" case to make sure that error handling works.
The tests were used to test m_pullup() changes to the code paths while
removing the KAME PULLDOWN_TEST cases and related macros.
MFC after: 3 weeks
Sponsored by: Netflix
There are times when we have to wait for reply packets. There are
either an ICMPv6 (error) reply or the expiration timeout.
In these cases synchonous ICMPv6 replies should arrive, always,
unless the packet is lost. Due to errors experienced with the
test software sending an invlaid request on at least i386 (*) these
packets are not generated. That means we are waiting for a long time
for the replies or even timeout the test case.
Manually set the "End" flag on these test cases as well, so they do
fail rather than timeout as the sniffer timeout happens. This improves
debugging options, reflects the error properly, and saves time on each
test suit run.
(*) The real cause for that is still to be found (see the referenced PRs)
PR: 241493, 239380
MFC after: 2 weeks
Sponsored by: Netflix
stderr:
Traceback (most recent call last):
File "/usr/tests/sys/netpfil/common/pft_ping.py", line 135, in <module>
main()
File "/usr/tests/sys/netpfil/common/pft_ping.py", line 124, in main
ping(args.sendif[0], args.to[0], args)
File "/usr/tests/sys/netpfil/common/pft_ping.py", line 74, in ping
raw = sp.raw(str(PAYLOAD_MAGIC))
File "/usr/local/lib/python3.6/site-packages/scapy/compat.py", line 52, in raw
return bytes(x)
TypeError: string argument without an encoding
MFC with: r354121
Sponsored by: The FreeBSD Foundation
In order to move python2 out of the test framework to avoid py2 vs. py3
confusions upgrade the remaining test cases using scapy to work with py3.
That means only one version of scapy needs to be installed in the CI system.
It also gives a path forward for testing i386 issues observed in the CI
system with some of these tests.
Fixes are:
- Use default python from environment (which is 3.x these days).
- properly ident some lines as common for the rest of the file to avoid
errors.
- cast the calculated offset to an int as the division result is considered
a float which is not accepted input.
- when comparing payload to a magic number make sure we always add the
payload properly to the packet and do not try to compare string in
the result but convert the data payload back into an integer.
- fix print formating.
Discussed with: lwhsu, kp (taking it off his todo :)
MFC after: 2 weeks
The change to conform to RFC 8200 for overlapping fragments now frees
the entire reassembly queue if the overlapping fragments are not an
exact match.
As a result we do see one less packet in the timeout statistics from
expiry. No other statistics change as the event is not counted.
It can be argued that we should improve the statistics counters in
that case.
This test case update should have been committed alongside the original
commit.
Pointyhat to: bz
MFC after: 3 weeks
X-MFC with: r354046
Sponsored by: Netflix
When we receive the packet with the first fragmented part (fragoff=0)
we remember the length of the unfragmentable part and the next header
(and should probably also remember ECN) as meta-data on the reassembly
queue.
Someone replying this packet so far could change these 2 (3) values.
While changing the next header seems more severe, for a full size
fragmented UDP packet, for example, adding an extension header to the
unfragmentable part would go unnoticed (as the framented part would be
considered an exact duplicate) but make reassembly fail.
So do not allow updating the meta-data after we have seen the first
fragmented part anymore.
The frag6_20 test case is added which failed before triggering an
ICMPv6 "param prob" due to the check for each queued fragment for
a max-size violation if a fragoff=0 packet was received.
MFC after: 3 weeks
Sponsored by: Netflix
When done with tests check that both the per-VNET and the global-fragmented-
packets-in-system counters are zero to make sure we do not leak counters or
queue entries.
This implies that for all test cases we either have to check for the ICMPv6
packet sent in case of TLL=0 expiry (if it is sent) or sleep at least long
enough for the TTL to expire for all packets (e.g., fragments where we do not
have the off=0 packet).
This also means that statistics are now updated to include all the expired
packets.
There are cases when we do not check for counters to be zero and this is
when testing VNET teardown to behave properly and not panic, when we are
intentionally leaving fragments in the system.
MFC after: 3 weeks
Sponsored by: Netflix
In order to ensure that changing the frag6 code does not change behaviour
or break code a set of test cases were implemented.
Like some other test cases these use Scapy to generate packets and possibly
wait for expected answers. In most cases we do check the global and
per interface (netstat) statistics output using the libxo output and grep
to validate fields and numbers. This is a bit hackish but we currently have
no better way to match a selected number of stats only (we have to ignore
some of the ND6 variables; otherwise we could use the entire list).
Test cases include atomic fragments, single fragments, multi-fragments,
and try to cover most error cases in the code currently.
In addition vnet teardown is tested to not panic.
A separate set (not in-tree currently) of probes were used in order to
make sure that the test cases actually test what they should.
The "sniffer" code was copied and adjusted from the netpfil version
as we sometimes will not get packets or have longer timeouts to deal with.
Sponsored by: Netflix
Set up two jails connected by an epair. Create VLAN interfaces in both
jails and check connectivity.
This is a very basic test, but exposed panics during the network stack
epoch work, so this is worth testing.
* Don't partition a disk if too few are available. Just rely on Kyua to
ensure that the tests aren't run with insufficient disks.
* Remove redundant cleanup steps
* In zpool_add_003_pos, store the temporary file in $PWD so Kyua will
automatically clean it up.
* Update zpool_add_005_pos to use dumpon instead of dumpadm. This test had
never been ported to FreeBSD.
* In zpool_add_005_pos, don't format the dump disk with UFS. That was
pointless.
MFC after: 2 weeks
Sponsored by: Axcient
The test is necessarily racy, because it depends on being able to complete a
"zpool add" before a previous resilver finishes. But it was racier than it
needed to be. Move the first "zpool add" to before the resilver starts.
MFC after: 2 weeks
Sponsored by: Axcient
The ZFS test suite was overriding the common $PWD variable with the path to
the pwd command, even though no test wanted to use it that way. Most tests
didn't notice, because ksh93 eventually restored it to its proper meaning.
MFC after: 2 weeks
Sponsored by: Axcient
* Don't create a UFS mountpoint just to store some temporary files. The
tests should always be executed with a sufficiently large TMPDIR.
Creating the UFS mountpoint is not only unneccessary, but it slowed
zpool_import_missing_002_pos greatly, because that test moves large files
between TMPDIR and the UFS mountpoint. This change also allows many of
the tests to be executed with just a single test disk, instead of two.
* Move zpool_import_missing_002_pos's backup device dir from / to $PWD to
prevent cross-device moves. On my system, these two changes improved that
test's speed by 39x. It should also prevent ENOSPC errors seen in CI.
* If insufficient disks are available, don't try to partition one of them.
Just rely on Kyua to skip the test. Users who care will configure Kyua
with sufficient disks.
MFC after: 2 weeks
Sponsored by: Axcient
It was trying to destroy the pool while zfsd was detaching the spare, and
"zpool destroy" failed. Fix by waiting until the spare has fully detached.
MFC after: 2 weeks
Sponsored by: Axcient
The test declared that it only needed 5 disks, but actually tried to use 6.
Fix it to use just 5, which is all it really needs.
MFC after: 2 weeks
Sponsored by: Axcient
ATF functions such as ATF_REQUIRE do not work correctly in child processes.
Use plain C functions to report errors instead.
In the parent, check for the untimely demise of children. Without this,
the test hung until the framework's timeout.
Raise the resource limit on the number of open files. If this was too low,
the test hit the two problems above.
Restore the kern.maxfiles sysctl OID in the cleanup function.
The body prematurely removed the symlink in which the old value was saved.
Make the test more robust by opening more files. In fact, due to the
integer division by 4, this was necessary to make the test valid with
some initial values of maxfiles. Thanks, asomers@.
wait() for children instead of sleeping.
Clean up a temporary file created by the test ("afile").
Reviewed by: asomers
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D21900
* Fix force_sync_path, which ensures that a file is fully flushed to disk.
Apparently "zpool history"'s performance has improved, but exporting and
importing the pool still works.
* Fix file_dva by using undocumented zdb syntax to clarify that we're
interested in the pool's root file system, not the pool itself. This
should also fix the zpool_clear_001_pos test.
* Remove a redundant cleanup step
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D21901
These tests have never worked correctly
* Replace runwattr with sudo
* Fix a scoping bug with the "dtst" variable
* Cleanup user properties created during tests
* Eliminate the checks for refreservation and send support. They will always
be supported.
* Fix verify_fs_snapshot. It seemed to assume that permissions would not yet
be delegated, but that's not how it's actually used.
* Combine verify_fs_promote with verify_vol_promote
* Remove some useless sleeps
* Fix backwards condition in verify_vol_volsize
* Remove some redundant cleanup steps in the tests. cleanup.ksh will handle
everything.
* Disable some parts of the tests that FreeBSD doesn't support:
* Creating snapshots with mkdir
* devices
* shareisci
* sharenfs
* xattr
* zoned
The sharenfs parts could probably be reenabled with more work to remove the
Solarisms.
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D21898
ZFS has grown some additional properties that hadn't been added to the
config file yet. While I'm here, improve the error message, and remove a
superfluous command.
MFC after: 2 weeks
Sponsored by: Axcient
This test attempts to corrupt a file-backed vdev by deleting it and then
recreating it with truncate. But that doesn't work, because the pool
already has the vdev open, and it happily hangs on to the open-but-deleted
file. Fix by truncating the file without deleting it.
MFC after: 2 weeks
Sponsored by: Axcient
* Adapt zvol_misc_001_neg to use dumpon instead of Solaris's dumpadm
* Disable zvol_misc_003_neg, zvol_misc_005_neg, and zvol_misc_006_pos,
because they involve using a zvol as a dump device, which FreeBSD does not
yet support.
MFC after: 2 weeks
Sponsored by: Axcient
* Remove zpool_create_013_neg. FreeBSD doesn't have an equivalent of
Solaris's metadevices. GEOM would be the equivalent, but since all geoms
are the same from ZFS's perspective, this test would be redundant with
zpool_create_012_neg
* Remove zpool_create_014_neg. FreeBSD does not support swapping to regular
files.
* Remove zpool_create_016_pos. This test is redundant with literally every
other test that creates a disk-backed pool.
* s:/etc/vfstab:/etc/fstab in zpool_create_011_neg
* Delete the VTOC-related portion of zpool_create_008_pos. FreeBSD doesn't
use VTOC.
* Replace dumpadm with dumpon and swap with swapon in multiple tests.
* In zpool_create_015_neg, don't require "zpool create -n" to fail. It's
reasonable for that variant to succeed, because it doesn't actually open
the zvol.
* Greatly simplify zpool_create_012_neg. Make it safer, too, but not
interfering with the system's regular swap devices.
* Expect zpool_create_011_neg to fail (PR 241070)
* Delete some redundant cleanup steps in various tests
* Remove some unneeeded ATF timeout specifications. The default is fine.
PR: 241070
MFC after: 2 weeks
Sponsored by: Axcient
kern_shm_open2(), since conception, completely fails to pass the mode along
to kern_shm_open(). This breaks most uses of it.
Add tests alongside this that actually check the mode of the returned
files.
PR: 240934 [pulseaudio breakage]
Reported by: ler, Andrew Gierth [postgres breakage]
Diagnosed by: Andrew Gierth (great catch)
Tested by: ler, tmunro
Pointy hat to: kevans
Add an atomic shm rename operation, similar in spirit to a file
rename. Atomically unlink an shm from a source path and link it to a
destination path. If an existing shm is linked at the destination
path, unlink it as part of the same atomic operation. The caller needs
the same permissions as shm_unlink to the shm being renamed, and the
same permissions for the shm at the destination which is being
unlinked, if it exists. If those fail, EACCES is returned, as with the
other shm_* syscalls.
truss support is included; audit support will come later.
This commit includes only the implementation; the sysent-generated
bits will come in a follow-on commit.
Submitted by: Matthew Bryan <matthew.bryan@isilon.com>
Reviewed by: jilles (earlier revision)
Reviewed by: brueffer (manpages, earlier revision)
Relnotes: yes
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D21423
memfd_create is effectively a SHM_ANON shm_open(2) mapping with optional
CLOEXEC and file sealing support. This is used by some mesa parts, some
linux libs, and qemu can also take advantage of it and uses the sealing to
prevent resizing the region.
This reimplements shm_open in terms of shm_open2(2) at the same time.
shm_open(2) will be moved to COMPAT12 shortly.
Reviewed by: markj, kib
Differential Revision: https://reviews.freebsd.org/D21393
It is useful to have some tests for page fault signals.
More tests would be useful but creating the conditions (such as various
kinds of running out of memory and I/O errors) is more complicated.
The tests page_fault_signal__bus_objerr_1 and
page_fault_signal__bus_objerr_2 depend on https://reviews.freebsd.org/D21566
before they can pass.
PR: 211924
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D21624
