Note that support for TLS 1.3 receive offload in OpenSSL is still an
open pull request in active development. However, potential changes
to that pull request should not affect the kernel interface.
Reviewed by: hselasky
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D33007
Log information from the running jails (routing, interfaces and pf) as
well as interfaces on the host.
This information is expected to be useful in debugging test failures.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
also fix test sys/audit/administrative.c.
Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33343
FUSE_COPY_FILE_RANGE instructs the server to write data to a file.
fusefs must invalidate any cached data within the written range.
PR: 260242
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33280
Correctly handle the situation where a FUSE server unlinks a file, then
creates a new file of a different type but with the same inode number.
Previously fuse_vnop_lookup in this situation would return EAGAIN. But
since it didn't call vgone(), the vnode couldn't be reused right away.
Fix this by immediately calling vgone() and reallocating a new vnode.
This problem can occur in three code paths, during VOP_LOOKUP,
VOP_SETATTR, or following FUSE_GETATTR, which usually happens during
VOP_GETATTR but can occur during other vops, too. Note that the correct
response actually doesn't depend on whether the entry cache has expired.
In fact, during VOP_LOOKUP, we can't even tell. Either it has expired
already, or else the vnode got reclaimed by vnlru.
Also, correct the error code during the VOP_SETATTR path.
PR: 258022
Reported by: chogata@moosefs.pro
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33283
Disable the dummynet tests when running the ci tests. This avoids
running into the panic described in https://reviews.freebsd.org/D33064
(where an interface is removed but a dummynet queued packet still has a
pointer to it).
These tests can be re-enabled when the work in
https://reviews.freebsd.org/D33267 lands.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Add a somewhat more extensive pfsync defer mode test. Ensure that pfsync
actually delays the state creating packet until after it has sent the
pfsync update and given the peer time to create the state.
Ideally the test should validate the pfsync state update and generate an
ack message, but to keep the test simple we rely on the timeout of the
deferred packet instead.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D33245
Add two underscore characters "__" to names of BIT_* and BITSET_*
macros to move them to the implementation name space and to prevent
a name space pollution due to BIT_* macros in 3rd party programs with
conflicting parameter signatures.
These prefixed macro names are used in kernel header files to define
macros in e.g. sched.h, sys/cpuset.h and sys/domainset.h.
If C programs are built with either -D_KERNEL (automatically passed
when building a kernel or kernel modules) or -D_WANT_FREENBSD_BITSET
(or this macros is defined in the source code before including the
bitset macros), then all macros are made visible with their previous
names, too. E.g., both __BIT_SET() and BIT_SET() are visible with
either of _KERNEL or _WANT_FREEBSD_BITSET defined.
The main reason for this change is that some 3rd party sources
including sched.h have been found to contain conflicting BIT_*
macros.
As a work-around, parts of shed.h have been made conditional and
depend on _WITH_CPU_SET_T being set when sched.h is included.
Ports that expect the full functionality provided by sched.h need
to be built with -D_WITH_CPU_SET_T. But this leads to conflicts if
BIT_* macros are defined in that program, too.
This patch set makes all of sched.h visible again without this
parameter being passed and without any name space pollution due
to BIT_* macros becoming visible when sched.h is included.
This patch set will be backported to the STABLE branches, but ports
will need to use -D_WITH_CPU_SET_T as long as there are supported
releases that do not contain these patches.
Reviewed by: kib, markj
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D33235
When using cached attributes, whether or not the data cache is enabled,
fusefs must update a file's atime whenever it reads from it, so long as
it wasn't mounted with -o noatime. Update it in-kernel, and flush it to
the server on close or during the next setattr operation.
The downside is that close() will now frequently trigger a FUSE_SETATTR
upcall. But if you care about performance, you should be using
-o noatime anyway.
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33145
When copy_file_range extends a file, it must update the cached file
size.
MFC after: 2 weeks
Reviewed by: rmacklem, pfg
Differential Revision: https://reviews.freebsd.org/D33151
Basic signal tests that tests can we deliver a signal via raise() and
can we deliver one via SIGALARM asynchronously.
In addition, tests whether or not on ARM T32 (Thumb) code can interrupt
A32 (normal) and vice versa.
While this test is aimed at ensuring basic qemu signals are working,
it's good to have in the base.
Sponsored by: Netflix
Discussed with: kevans, cognet
Differential Revision: https://reviews.freebsd.org/D33078
coredump_phnum intends to generate a core file with many PT_LOAD
segments. Previously it called mmap() in a loop with alternating
protections, relying on each mapping following the previous, to produce
a core file with many page-sized PT_LOAD segments. With ASLR on we no
longer have this property of each mmap() following the previous.
Instead, perform a single allocation, and then use mprotect() to set
alternating pages to PROT_READ.
PR: 259970
Reported by: lwhsu, mw
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33070
This test needs to have the loopback interface enabled, or route lookups
for our own IP addresses will fail.
MFC after: 3 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D33041
The TLS header length field is set by the kernel, so if it is
incorrect that is an indication of a kernel bug, not an internal error
in the tests.
Prompted by: markj (comment in an earlier review)
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D33003
Similar to the simple transmit tests added in
a10482ea74, these tests test the kernel
TLS functionality directly by manually encrypting TLS records using
randomly generated keys and writing them to a socket to be processed
by the kernel.
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D32980
For each AES-CBC MTE cipher suite, test sending records with 1 to 16
bytes of payload. This ensures that all of the potential padding
values are covered.
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D32840
We didn't populate dyncnt/tblcnt, so `pfctl -sr -vv` might not have the
table element count.
PR: 259689
MFC after: 3 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D32893
ktls_test requires libcrypto to build, and fails if it is not available
(which is the case when building WITHOUT_OPENSSL).
Reported by: Michael Dexter, Build Option Survey
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32895
AES-CBC OpenSSL assembly is used underneath.
The glue layer(ossl_aes.c) is based on CHACHA20 implementation.
Contrary to the SHA and CHACHA20, AES OpenSSL assembly logic
does not have a fallback implementation in case CPU doesn't
support required instructions.
Because of that CPU caps are checked during initialization and AES
support is advertised only if available.
The feature is available on all architectures that ossl supports:
i386, amd64, arm64.
The biggest advantage of this patch over existing solutions
(aesni(4) and armv8crypto(4)) is that it supports SHA,
allowing for ETA operations.
Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: jhb (previous version)
Differential revision: https://reviews.freebsd.org/D32099
AES-CBC OpenSSL assembly is used underneath.
The glue layer(ossl_aes.c) is based on CHACHA20 implementation.
Contrary to the SHA and CHACHA20, AES OpenSSL assembly logic
does not have a fallback implementation in case CPU doesn't
support required instructions.
Because of that CPU caps are checked during initialization and AES
support is advertised only if available.
The feature is available on all architectures that ossl supports:
i386, amd64, arm64.
The biggest advantage of this patch over existing solutions
(aesni(4) and armv8crypto(4)) is that it supports SHA,
allowing for ETA operations.
Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D32099
Note that these tests test the kernel TLS functionality directly.
Rather than using OpenSSL to perform negotiation and generate keys,
these tests generate random keys send data over a pair of TCP sockets
manually decrypting the TLS records generated by the kernel.
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D32652
Ensure that NAT still works as expected when combined with dummynet.
MFC after: 3 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D32666
Ensure that the ICMP error is returned with the correct
source and destination addresses.
MFC after: 3 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D32572
Eliminate the nested loops and re-implement following a suggestion from
rlibby.
Add some simple regression tests.
Reviewed by: rlibby, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32472
Test the $nr expansion in labels is correct, even if the optimiser
reduces the rule count.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32489
This gives the vfs layer a chance to provide handling for EVFILT_VNODE,
for instance. Change pipe_specops to use the default vop_kqfilter to
accommodate fifoops that don't specify the method (i.e. all in-tree).
Based on a patch by Jan Kokemüller.
PR: 225934
Reviewed by: kib, markj (both pre-KASSERT)
Differential Revision: https://reviews.freebsd.org/D32271
There are two issues with the checks against VM_MAXUSER_ADDRESS. First,
the comparison should consider the values as unsigned, otherwise
addresses with the high bit set will fail to branch. Second, the value
of VM_MAXUSER_ADDRESS is, by convention, one larger than the maximum
mappable user address and invalid itself. Thus, use the bgeu instruction
for these comparisons.
Add a regression test case for copyin(9).
PR: 257193
Reported by: Robert Morris <rtm@lcs.mit.edu>
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D31209
Previously, only test vectors which used the default nonce and tag
sizes (12 and 16, respectively) were tested. This now tests all of
the vectors. This exposed some additional issues around requests with
an empty payload (which wasn't supported) and an empty AAD (which
falls back to CIOCCRYPT instead of CIOCCRYPTAEAD).
- Make use of the 'ivlen' and 'maclen' fields for CIOGSESSION2 to
test AES-CCM vectors with non-default nonce and tag lengths.
- Permit requests with an empty payload.
- Permit an input MAC for requests without AAD.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32121
The DevFusePoll::access/select test would occasionally segfault. The
cause was a file descriptor that was shared between two threads. The
first thread would kill the second and close the file descriptor. But
it was possible that the second would read the file descriptor before it
shut down. That did not cause problems for kqueue, poll, or blocking
operation, but it triggered segfaults in select's macros.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32142
If the FUSE server tells the kernel that a file's size has changed, then
the kernel must invalidate any portion of that file in cache. But the
kernel can't do that during VOP_STRATEGY, because the file's buffers are
already locked. Instead, proceed with the write.
PR: 256937
Reported by: Agata <chogata@moosefs.pro>
Tested by: Agata <chogata@moosefs.pro>
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D32332
fuse_vnop_bmap needs to know the file's size in order to calculate the
optimum amount of readahead. If the file's size is unknown, it must ask
the FUSE server. But if the file's data was previously cached and the
server reports that its size has shrunk, fusefs must invalidate the
cached data. That's not possible during VOP_BMAP because the buffer
object is already locked.
Fix the panic by not querying the FUSE server for the file's size during
VOP_BMAP if we don't need it. That's also a a slight performance
optimization.
PR: 256937
Reported by: Agata <chogata@moosefs.pro>
Tested by: Agata <chogata@moosefs.pro>
MFC after: 2 weeks
NOTE_ABSTIME may also have a zero timeout, which indicates that we
should still fire immediately as an absolute time in the past. A test
has been added for this one as well.
Fixes: 9c999a259f ("kqueue: don't arbitrarily restrict long-past...")
Point hat: kevans
Reported by: syzbot+1c8d1154f560b3930042@syzkaller.appspotmail.com
NOTE_ABSTIME values are converted to values relative to boottime in
filt_timervalidate(), and negative values are currently rejected. We
don't reject times in the past in general, so clamp this up to 0 as
needed such that the timer fires immediately rather than imposing what
looks like an arbitrary restriction.
Another possible scenario is that the system clock had to be adjusted
by ~minutes or ~hours and we have less than that in terms of uptime,
making a reasonable short-timeout suddenly invalid. Firing it is still
a valid choice in this scenario so that applications can at least
expect a consistent behavior.
Reviewed by: kib, markj
Discussed with: allanjude
Differential Revision: https://reviews.freebsd.org/D32230
This test case uses `dtrace -c` but it has some issues at the moment
While here, add a checker for dtrace executes successfully or not to provide
a more informative error message.
PR: 258763
Sponsored by: The FreeBSD Foundation
For file systems that allow it, fusefs will skip FUSE_OPEN,
FUSE_RELEASE, FUSE_OPENDIR, and FUSE_RELEASEDIR operations, a minor
optimization.
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D32141
Now that pf can also use dummynet we should extend the existing dummynet
tests to also test it when used with pf.
Reviewed by: donner
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31905
When this flag is set, operations that update an existing kevent will
not change the udata field. This can be used to NOTE_TRIGGER or
EV_{EN,DIS}ABLE events without overwriting the stashed pointer.
Reviewed by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
Obtained from: CheriBSD
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D30286
There sig_atomic_t is shorter than void *.
As result, it cannot keep pointer.
Assigning to void * is actually safe for us in a signal handler.
Reviewed by: asomers
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Fixes: 4f917847c9
Differential revision: https://reviews.freebsd.org/D32064
During VOP_GETPAGES, fusefs needs to determine the file's length, which
could require a FUSE_GETATTR operation. If that fails, it's better to
SIGBUS than panic.
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: markj, kib
Differential Revision: https://reviews.freebsd.org/D31994
As with FIFOs, a path descriptor for a unix socket cannot be used with
kevent().
In principle connectat(2) and bindat(2) could be modified to support an
AT_EMPTY_PATH-like mode which operates on the socket referenced by an
O_PATH fd referencing a unix socket. That would eliminate the path
length limit imposed by sockaddr_un.
Update O_PATH tests.
Reviewed by: kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31970
- Remove unused format string arguments.
- Remove a set but unused variable.
Reviewed by: khng, kib
Differential Revision: https://reviews.freebsd.org/D31946
soo_aio_queue() did not handle the possibility that the provided socket
is a listening socket. Up until recently, to fix this one would have to
acquire the socket lock first and check, since the socket buffer locks
were destroyed by listen(2).
Now that the socket buffer locks belong to the socket, simply check
SOLISTENING(so) after acquiring them, and make listen(2) return an error
if any AIO jobs are enqueued on the socket.
Add a couple of simple regression test cases.
Note that this fixes things only for the default AIO implementation;
cxgbe(4)'s TCP offload has a separate pru_aio_queue implementation which
requires its own solution.
Reported by: syzbot+c8aa122fa2c6a4e2a28b@syzkaller.appspotmail.com
Reported by: syzbot+39af117d43d4f0faf512@syzkaller.appspotmail.com
Reported by: syzbot+60cceb9569145a0b993b@syzkaller.appspotmail.com
Reported by: syzbot+2d522c5db87710277ca5@syzkaller.appspotmail.com
Reviewed by: tuexen, gallatin, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31901
Test syn-proxying a connection to the local host.
Sponsored by: Modirum MDPay
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31854
Same as the v4 test, but with IPv6.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31789
Test prioritisation and dummynet queues.
We need to give the pipe sufficient bandwidth for dummynet to work.
Given that we can't rely on the TCP connection failing alltogether, but
we can measure the effect of dummynet by imposing a time limit on a
larger data transfer.
If TCP is prioritised it'll get most of the pipe bandwidth and easily
manage to transfer the data in 3 seconds or less. When not prioritised
this will not succeed.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31788
Check that the bridge module is loaded before running this test.
It likely will be (as a result of running the bridge tests), but if it's
not we'll get spurious failures.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
Follow-up d396c67f26 by also silencing warnings about deprecated
implicit copy constructors in the fusefs tests, which use googletest.
Fixes: d396c67f26
MFC after: 3 days
Tests that ggatec appropriately handles unsupported BIO operations,
rather than overflowing a buffer.
Submitted by: Johannes Bruelltuete <johannes@jo-t.de>
PR: 213479
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D31318
Test that ALTQ can prioritise one type of traffic over another. Do this
by establishing a slow link and saturating it with ICMP echos.
When prioritised TCP connections reliably go through. When not
prioritised TCP connections reliably fail.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
The main purpose of this test is to verify that we can use ALTQ on top
of if_vlan, but while we're here we also exercise the CBQ code. There's
already a basis test for HFSC, so it makes sense to test another
algorithm while we test if_vlan.
Reviewed by: donner
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31649
Add missing wrapper code to librt for these new functions so that
SIGEV_THREAD works. Without machinery to convert it to SIGEV_THREAD_ID,
you got EINVAL.
Reviewed by: asomers
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31618
Allow multiple vector IOs to be started with one system call.
aio_readv() and aio_writev() already used these opcodes under the
covers. This commit makes them available to user space.
Being non-standard extensions, they're only visible if __BSD_VISIBLE is
defined, like the functions.
Reviewed by: asomers, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31627
Extend the existing set-tos keyword to also be able to set traffic class
on IPv6 traffic.
Add tests for this as well.
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D31564
Add a credential to the cdev object in sysctl_vmm_create(), then check
that we have the correct credentials in sysctl_vmm_destroy(). This
prevents a process in one jail from opening or destroying the /dev/vmm
file corresponding to a VM in a sibling jail.
Add regression tests.
Reviewed by: jhb, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31156
This has been known to trigger panics. It currently doesn't, but we may
as well have a test for it.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
The macro bit_foreach() traverses all set bits in the bitstring in the
forward direction, assigning each location in turn to variable.
The macro bit_foreach_at() traverses all set bits in the bitstring in
the forward direction at or after the zero-based bit index, assigning
each location in turn to variable.
The bit_foreach_unset() and bit_foreach_unset_at() macros which
traverses unset bits are implemented for completeness.
Reviewed by: asomers, dougm
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31469
This bug is no longer reproducible in 14.0-CURRENT and 13.0-RELEASE
Do not MFC to stable/12 !
PR: 251828
Reported by: markj
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D31535
This implements fspacectl(2) support on shared memory objects. The
semantic of SPACECTL_DEALLOC is equivalent to clearing the backing
store and free the pages within the affected range. If the call
succeeds, subsequent reads on the affected range return all zero.
tests/sys/posixshm/posixshm_tests.c is expanded to include a
fspacectl(2) functional test.
Sponsored by: The FreeBSD Foundation
Reviewed by: kevans, kib
Differential Revision: https://reviews.freebsd.org/D31490
fspacectl(2) is a system call to provide space management support to
userspace applications. VOP_DEALLOCATE(9) is a VOP call to perform the
deallocation. vn_deallocate(9) is a public KPI for kmods' use.
The purpose of proposing a new system call, a KPI and a VOP call is to
allow bhyve or other hypervisor monitors to emulate the behavior of SCSI
UNMAP/NVMe DEALLOCATE on a plain file.
fspacectl(2) comprises of cmd and flags parameters to specify the
space management operation to be performed. Currently cmd has to be
SPACECTL_DEALLOC, and flags has to be 0.
fo_fspacectl is added to fileops.
VOP_DEALLOCATE(9) is added as a new VOP call. A trivial implementation
of VOP_DEALLOCATE(9) is provided.
Sponsored by: The FreeBSD Foundation
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D28347
Changing the bridge MTU will now also change all of the member interface
MTUs. Test this.
Reviewed by: donner
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31289
Only lists the states relevant to the connection we're killing.
Sometimes there are IPv6 related states (due to the usual IPv6
background traffic of router solicitations, DAD, ...) that causes us to
think we failed to kill the state, which in turn caused the test to fail
intermittently.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
Use dhclient with its 'vlan-pcp' option to set a VLAN PCP value and
verify that it actually gets set.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31276
Fairly basic test case for using gif(4)'s ability to tunnel Ethernet
traffic between bridges.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Add a label to md devices created by this test. The next time this
test leaks md devices, finding the culprit will be much easier.
Thanks to: sobomax, for adding labels in r322969
MFC after: 1 week
Sponsored by: Dell EMC Isilon
readlink does not NUL-terminate the output buffer. This led to spurious
failures to destroy the md device because the unit number was garbage.
NUL-terminate the output buffer.
Reported by: ASLR
MFC after: 1 week
Sponsored by: Dell EMC Isilon
ATF cleanup functions cannot use functions such as ATF_REQUIRE
and atf_tc_fail. These functions assert that a test case is
currently running, which is not true during cleanup, so the
process aborts. Change the cleanup function to simply print
to stderr and return.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Test that with syncookies enabled pf does not create state for
connections before the remote peer has replied to the SYN|ACK message.
MFC after: 1 week
Sponsored by: Modirum MDPay
Differential Revision: https://reviews.freebsd.org/D31142
Test syncookies on a forwarding host. That is, in a setup where the
machine (or vnet) running pf is not the same as the machine (or vnet)
running the server it's protecting.
MFC after: 1 week
Sponsored by: Modirum MDPay
Differential Revision: https://reviews.freebsd.org/D31141
The new match keyword can currently only assign queues, so we can only
test it with ALTQ.
Set up a basic scenario where we use 'match' to assign ICMP traffic to a
slow queue, and confirm that it's really getting slowed down.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31116
One is allowed to use LIO_NOWAIT without specifying a sigevent. In this
case, lj->lioj_signal is left uninitialized, but several code paths
examine liov_signal.sigev_notify to figure out which notification to
post. Unconditionally initialize that field to SIGEV_NONE.
Add a dumb test case which triggers the bug.
Reported by: KMSAN+syzkaller
Reviewed by: asomers
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31197
Create and retrieve 20.000 states. There have been issues with nvlists
causing very slow state retrieval. We don't impose a specific limit on
the time required to retrieve the states, but do log it. In excessive
cases the Kyua timeout will fail this test.
Reviewed by: donner
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D30943
During FUSE_SETLK, the owner field should uniquely identify the calling
process. The fusefs module now sets it to the process's pid.
Previously, it expected the calling process to set it directly, which
was wrong.
libfuse also apparently expects the owner field to be set during
FUSE_GETLK, though I'm not sure why.
PR: 256005
Reported by: Agata <chogata@moosefs.pro>
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D30622
Every FUSE operation has a unique value in its header. As the name
implies, these values are supposed to be unique among all outstanding
operations. And since FUSE_INTERRUPT is asynchronous and racy, it is
desirable that the unique values be unique among all operations that are
"close in time".
Ensure that they are actually unique by incrementing them whenever we
reuse a fuse_dispatcher object, for example during fsync, write, and
listextattr.
PR: 244686
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D30810
/dev/fuse is always ready for writing, so it's kind of dumb to poll it.
But some applications do it anyway. Better to return ready than EINVAL.
MFC after: 2 weeks
Reviewed by: emaste, pfg
Differential Revision: https://reviews.freebsd.org/D30784
Add tests to check for renaming issues reported in PR241954 and solved
in D30110.
MFC: Together with D30629
Differential Revision: https://reviews.freebsd.org/D30713
During D30699 the existing basic tests were missed. Furthermore
debugging output was still in the code, which is removed now.
MFC: together with D30699
Differential Revision: https://reviews.freebsd.org/D30714
Test functionality of ng_vlan_rotate(4):
- Rotate 1 to 9 stagged vlans in any possible direction and length
- Rotate random combinations of ethertypes (8100, 88a8, 9100)
- Automatic reverse rotating for backward data flow
- Test too many and to few vlans
Reviewed by: kp (earlier version)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30670
Test functionality of ng_hub(4):
- replicting traffic to anything but the sending hook
- persistence
- an unrestricted loop
- implementation limits with many hooks.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30633
Factor out the data counter helpers for other tests to use.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30699
Errors raised in the common util functions should raise the location
of their caller to be useful and include the errno description.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30671
Test functionality of ng_bridge(4):
- replicating traffic to anything but the sending hook
- persistence
- detect loops
- unicast to only one link of many
- stretch to implementation limits on broadcast
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30647
Add msg_handler in order to receive messages from netgraph nodes to be
tested.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30657
Provide a framework of functions to test various netgraph modules.
Tests contain:
- creating, renaming, and destroying nodes
- connecting and removing hooks
- sending and receiving data
- sending ASCII messages
- errors can be passed for indiviual inspection or fail the test
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30629
The killstate:match test starts nc as a background process. There was no
guarantee that the nc process would have connected by the time we check
for states, so this test occasionally failed without good reason.
Teach the test to wait for at least some states to turn up before
executing the critical checks.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
Test dummynet pipes (i.e. bandwidth limitation) with ipfw. This is put
in the common tests because we hope to add dummynet support to pf in the
near future.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D30380
Add port forwardings to the performance tests. This will cause random
incoming packets to match the random port forwardings opends beforehand.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30443
gettimeofday(3) is almost as expensive as the calls to libalias.
So the call frequency for this call is reduced by a factor of 1000 in
order to neglect it's influence.
Using NAT entries became more realistic: A communication of a random
length of up to 150 packets (10% outgoing, 90% incoming) is applied
for each entry.
Precision of the execution time is raised to see the trends better.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30405
Extend the test suite for libalias(3) to incoming connections.
Test the various types of redirections.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30408
Rework the tests to check the correct layer in a single test.
Factor out tests for reuse in other modules.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30412
In order to compare upcoming changes for their effectivness, measure
performance by counting opertions and the runtime of each operation
over the time. Accumulate all tests in a single instance, so make it
complicated over the time. If you wait long enough, you will notice
the expiry of old flows.
Reviewed by: kp (earlier version)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30379
Testing LibAliasOut functionality. This concentrates the typical use
case of initiating data transfers from the inside. Provide a
exhaustive test for the data structure in order to check for
performance improvements.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30335
In order to modify libalias for performance, the existing
functionality must not change. Enforce this.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30307
Test the specific case reported in PR 255852. Clearing the skip flag
on groups was broken because pfctl couldn't work out if a kif was a
group or not, because the kernel no longer set the pfik_group pointer.
PR: 255852
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30285
Since 2.4.5 scapy started issuing warnings about a few different
configurations during our tests. These are harmless, but they generate
stderr output, which upsets atf_check.
Configure scapy to only log critical errors (and thus not warnings) to
fix these tests.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Python 3.8 warns about line 112:
'SyntaxWarning: "is" with a literal. Did you mean "=="?'
Use '==' as Python suggests.
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
zfsd uses a device's physical path attribute to automatically replace a
missing ZFS disk when a blank disk is inserted into the same physical
slot. Currently gmultipath passes through its underlying providers'
physical path attribute. That may cause zfsd to replace a missing
gmultipath provider with a newly arrived, single-path disk. That would
be bad.
This commit fixes that problem by simply appending "/mp" to the
underlying providers' physical path, in a manner similar to what geli
already does.
Sponsored by: Axcient
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D29941
There's a problem with pf's reassembly code where it produces incorrect
checksums when reassembling across interfaces with different MTUs.
Test this.
PR: 255432
Reviewed by: donner
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D30013
Teach poll(2) to support Linux-style POLLRDHUP events for sockets, if
requested. Triggered when the remote peer shuts down writing or closes
its end.
Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D29757
Add a test case where the pfctl optimizer will generate a table
automatically. These tables have long names, which we accidentally broke
in the nvlist ADDRULE ioctl.
Reviewed by: melifaro
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29989
Now that we support having multiple labels on a rule ensure that we can
use each rule label to kill states.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29938
Most of the routing tests create per-test VNET, making
it harder to repeat the failure with CLI tools.
Provide an additional route/nexthop data on failure.
Differential Revision: https://reviews.freebsd.org/D29957
Reviewed by: kp
MFC after: 2 weeks
Two of these tests now pass. Looking at Jenkins to find the first commit
where this behaviour changed indicates that
2fe5a79425 is the most likely cause.
Reviewed By: melifaro
Differential Revision: https://reviews.freebsd.org/D28886
This was briefly broken, so ensure that we can read and clear rules
counters.
MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29728
MAP-E (RFC 7597) requires special care for selecting source ports
in NAT operation on the Customer Edge because a part of bits of the port
numbers are used by the Border Relay to distinguish another side of the
IPv4-over-IPv6 tunnel.
PR: 254577
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D29468
Add a regression test for a scenario where a shadow vm object is shared
by multiple mappings. If a page COW occurs through one of the mappings,
then the virtual-to-physical mapping may become invalidated.
This tests the scenario from CVE-2021-29626 which was fixed by
982693bb72.
Reviewed by: markj
Sponsored by: Dell EMC Isilon
This replicates an issue observed on pfSense: https://redmine.pfsense.org/issues/11436
In essence, reply-to is needed to ensure that connections always leave
the WAN interface they came in on, but this confused the state tracking.
MFC after: 2 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
tests/sys/netfil/pf/synproxy fails if inetd has been running
outside of the jail because pidfile_open() fails with EEXIST.
tests/sys/netfil/pf/nat has the same problem but the test succeeds
because whether inetd is running is not so important.
Fix the problem by changing the pidfile path from the default
location.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D29622
They have been failing for 1.5 months and the patch to fix them is stuck
in review so mark them as XFAIL for now to get Jenkins back to green.
To be reverted when https://reviews.freebsd.org/D28886 (or similar) is
commited.
Reviewed By: kp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D29528
This should bring the number of Jenkins failures from 4 down to 3.
Locally kyua now prints `skipped: could not find a valid interface [0.115s]`
when I run it in QEMU without a network device.
Reviewed By: lwhsu
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D29414
The LOR also happens on amd64 and other architectures. Ideally we would
fix this. However, in order to get Jenkins green again to catch real
regressions, we should skip this test for now.
PR: 251726
Reviewed By: lwhsu
Differential Revision: https://reviews.freebsd.org/D29341
I was trying to debug why this test is working locally but failing in CI.
While doing so I made some small changes to allow running it with set -e.
It turns out the problem is that find_iface does not return anything in
Jenkins, so all following tests fail with obscure error messages.
To handle this case exit early if $eth is empty.
Reviewed By: lwhsu
Differential Revision: https://reviews.freebsd.org/D29340
1) F_SETLKW (blocking) operations would be sent to the FUSE server as
F_SETLK (non-blocking).
2) Release operations, F_SETLK with lk_type = F_UNLCK, would simply
return EINVAL.
PR: 253500
Reported by: John Millikin <jmillikin@gmail.com>
MFC after: 2 weeks
MSG_CMSG_CLOEXEC has not been working since 2015 (SVN r284380) because
_finstall expects O_CLOEXEC and not UF_EXCLOSE as the flags argument.
This was probably not noticed because we don't have a test for this flag
so this commit adds one. I found this problem because one of the
libwayland tests was failing.
Fixes: ea31808c3b ("fd: move out actual fp installation to _finstall")
MFC after: 3 days
Reviewed By: mjg, kib
Differential Revision: https://reviews.freebsd.org/D29328
Test that pfsync works as expected with bulk updates. That is, create
some state before setting up the second firewall. Let that firewall
request a bulk update so it can catch up, and check that it got the
state which was created before it enable pfsync.
PR: 254236
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D29272
After length decisions, we've decided that the if_wg(4) driver and
related work is not yet ready to live in the tree. This driver has
larger security implications than many, and thus will be held to
more scrutiny than other drivers.
Please also see the related message sent to the freebsd-hackers@
and freebsd-arch@ lists by Kyle Evans <kevans@FreeBSD.org> on
2021/03/16, with the subject line "Removing WireGuard Support From Base"
for additional context.
This is the culmination of about a week of work from three developers to
fix a number of functional and security issues. This patch consists of
work done by the following folks:
- Jason A. Donenfeld <Jason@zx2c4.com>
- Matt Dunwoodie <ncon@noconroy.net>
- Kyle Evans <kevans@FreeBSD.org>
Notable changes include:
- Packets are now correctly staged for processing once the handshake has
completed, resulting in less packet loss in the interim.
- Various race conditions have been resolved, particularly w.r.t. socket
and packet lifetime (panics)
- Various tests have been added to assure correct functionality and
tooling conformance
- Many security issues have been addressed
- if_wg now maintains jail-friendly semantics: sockets are created in
the interface's home vnet so that it can act as the sole network
connection for a jail
- if_wg no longer fails to remove peer allowed-ips of 0.0.0.0/0
- if_wg now exports via ioctl a format that is future proof and
complete. It is additionally supported by the upstream
wireguard-tools (which we plan to merge in to base soon)
- if_wg now conforms to the WireGuard protocol and is more closely
aligned with security auditing guidelines
Note that the driver has been rebased away from using iflib. iflib
poses a number of challenges for a cloned device trying to operate in a
vnet that are non-trivial to solve and adds complexity to the
implementation for little gain.
The crypto implementation that was previously added to the tree was a
super complex integration of what previously appeared in an old out of
tree Linux module, which has been reduced to crypto.c containing simple
boring reference implementations. This is part of a near-to-mid term
goal to work with FreeBSD kernel crypto folks and take advantage of or
improve accelerated crypto already offered elsewhere.
There's additional test suite effort underway out-of-tree taking
advantage of the aforementioned jail-friendly semantics to test a number
of real-world topologies, based on netns.sh.
Also note that this is still a work in progress; work going further will
be much smaller in nature.
MFC after: 1 month (maybe)
This should allow the test to pass in Jenkins. Testing it locally now
reports "passed" instead of "invalid TAP data".
While touching this file also fix some shellcheck warnings that were
pointed out by my IDE.
Reviewed By: lwhsu, afedorov
Differential Revision: https://reviews.freebsd.org/D29054
The argument has to be a single whitespace-separate value. While touching
all these lines also add ksh93, since `atf_set "require.progs"` overrides
the default value specified in the Kyuafile. This then results in tests
being executed despite ksh93 not being installed.
Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D29066
It seems like GCC's -Wsign-compare is stricter and also warns for
constants. Appease GCC by adding the required casts.
Fixes: 96a9e50e63 ("ptrace_test: Add more debug output on test failures")
Reported by: Jenkins CI
This makes the `kyua report --verbose` output a lot easier to parse when
looking at failed tests. It also fixes the closefrom() test since I
tested my changes with this commit but forgot to push it together with
fa32350347.
Fixes: fa32350347 ("close_range: add audit support")
Instead of running tests one-by-one with the shell wrapper we now run
the full gtest testsuite twice (once as root, once as non root). This
significantly speeds up running tests despite running them twice.
This change also passes the missing -u flag to capsicum-test that caused
test failures (https://bugs.freebsd.org/250178)
Previously, running the testsuite with the wrapper script took ~3s per
test on aarch64 QEMU, i.e. a total of almost 5 minutes.
Now it takes 6 seconds to run all tests twice.
Before:
root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional
94/96 passed (2 failed)
309.97 real 58.46 user 244.31 sys
After:
root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional
functional:test_root -> passed [2.659s]
functional:test_unprivileged -> passed [2.391s]
2/2 passed (0 failed)
5.48 real 1.06 user 2.52 sys
This overhead is caused by kyua + atf-sh spawning lots of additional
processes and can be avoided by just running the googletest test binary.
syscall seconds calls errors
fork 39.810229456 1275 0
sigprocmask 13.546928736 572 0
i.e. 1275 processes spawned to run a single test.
Test Plan: All tests pass with D28907.
PR: 250178
Reviewed By: lwhsu
Differential Revision: https://reviews.freebsd.org/D29014
This includes various fixes that I submitted recently such as updating the
pdkill() tests for the actual implemented behaviour
(https://github.com/google/capsicum-test/pull/53) and lots of changes to
avoid calling sleep() and replacing it with reliable synchronization
(pull requests 49,51,52,53,54). This should make the testsuite more reliable
when running on Jenkins. Additionally, process status is now retrieved using
libprocstat instead of running `ps` and parsing the output
(https://github.com/google/capsicum-test/pull/50). This fixes one previously
failing test and speeds up execution.
Overall, this update reduces the total runtime from ~60s to about 4-5 seconds.
ATF now opens the results file (without CLOEXEC), so the child actually
has a valid file descriptor 3. To fix this simply use a large number that
will definitely not be a valid file descriptor.
Reviewed by: jhb, cem, lwhsu
Differential Revision: https://reviews.freebsd.org/D28889
I've run these tests many times in a loop on multiple architectures and
it works reliably for me, maybe it's time to retire these skips?
This also adds an additional waitpid to one of the tests to avoid
a potential race condition (suggested by markj@).
PR: 239397, 244056, 239425, 240510, 220841, 243605
Reviewed By: markj
Differential Revision: https://reviews.freebsd.org/D28888
Mostly automatic, using
`CHILD_REQUIRE\(([^|&\n]*) ==` -> `CHILD_REQUIRE_EQ_INT($1,`
`ATF_REQUIRE\(([^|&\n]*) ==` -> `REQUIRE_EQ_INT($1,` followed by
git-clang-format -f and then manually checking ones that contain ||/&&.
Test Plan:
Still getting the same failure but now it prints
`psr.sr_error (0) == EBADF (9) not met` instead of just failing
without printing the values.
PR: 243605
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D28887
This also fixes a typo in the dup test that caused the head function to
not be called. On my test system without python3 the tests are now
skipped instead of failing.
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D28903
Ensure that we not only block on some interfaces, but also forward on
some. Without the previous commit we wound up discarding on all ports,
rather than only on the ports needed to break the loop.
MFC after: 1 week
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D28917
All supported platforms support thread-local vars and __thread.
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28796
This is basically the same test as the existing STP test, but now on top
of VLAN interfaces instead of directly using the epair devices.
MFC after: 1 week
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D28861
df093aa946 linked against libprivateauditd.a, but that is currently
(and incorrectly) built as position-dependent. For now just force PIE
off for this test to fix the WITH_PIE build.
Sponsored by: The FreeBSD Foundation
In the CheriBSD CI we reproducibly see the first test in sys/audit
(administrative:acct_failure) fail due to a missing startup message.
It appears this is caused by a race condition when starting auditd:
`service auditd onestart` returns as soon as the initial auditd() parent
exits (after the daemon(3) call).
We can avoid this problem by setting up the auditd infrastructure
in-process: libauditd contains audit_quick_{start,stop}() functions that
look like they are ideally suited to this task.
This patch also avoids forking lots of shell processes for each of the 418
tests by using `auditon(A_SENDTRIGGER, &trigger, sizeof(trigger))` to check
for a running auditd(8) instead of using `service auditd onestatus`.
With these two changes (and D28388 to fix the XFAIL'd test) I can now
boot and run `cd /usr/tests/sys/audit && kyua test` without any failures
in a single-core QEMU instance. Before there would always be at least one
failed test.
Besides making the tests more reliable in CI, a nice side-effect of this
change is that it also significantly speeds up running them by avoiding
lots of fork()/execve() caused by shell scripts:
Running kyua test on an AArch64 QEMU took 315s before and now takes 68s,
so it's roughly 3.5 times faster. This effect is even larger when running
on a CHERI-RISC-V QEMU since emulating CHERI instructions on an x86 host
is noticeably slower than emulating AArch64.
Test Plan: aarch64+amd64 QEMU no longer fail.
Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D28451
If we install the scapy package (which we do list as a dependency) we
don't automatically install python (but we do have python3).
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC (“Netgate”’)
Traditionally routing socket code did almost zero checks on
the input message except for the most basic size checks.
This resulted in the unclear KPI boundary for the routing system code
(`rtrequest*` and now `rib_action()`) w.r.t message validness.
Multiple potential problems and nuances exists:
* Host bits in RTAX_DST sockaddr. Existing applications do send prefixes
with hostbits uncleared. Even `route(8)` does this, as they hope the kernel
would do the job of fixing it. Code inside `rib_action()` needs to handle
it on its own (see `rt_maskedcopy()` ugly hack).
* There are multiple way of adding the host route: it can be DST without
netmask or DST with /32(/128) netmask. Also, RTF_HOST has to be set correspondingly.
Currently, these 2 options create 2 DIFFERENT routes in the kernel.
* no sockaddr length/content checking for the "secondary" fields exists: nothing
stops rtsock application to send sockaddr_in with length of 25 (instead of 16).
Kernel will accept it, install to RIB as is and propagate to all rtsock consumers,
potentially triggering bugs in their code. Same goes for sin_port, sin_zero, etc.
The goal of this change is to make rtsock verify all sockaddr and prefix consistency.
Said differently, `rib_action()` or internals should NOT require to change any of the
sockaddrs supplied by `rt_addrinfo` structure due to incorrectness.
To be more specific, this change implements the following:
* sockaddr cleanup/validation check is added immediately after getting sockaddrs from rtm.
* Per-family dst/netmask checks clears host bits in dst and zeros all dst/netmask "secondary" fields.
* The same netmask checking code converts /32(/128) netmasks to "host" route case
(NULL netmask, RTF_HOST), removing the dualism.
* Instead of allowing ANY "known" sockaddr families (0<..<AF_MAX), allow only actually
supported ones (inet, inet6, link).
* Automatically convert `sockaddr_sdl` (AF_LINK) gateways to
`sockaddr_sdl_short`.
Reported by: Guy Yur <guyyur at gmail.com>
Reviewed By: donner
Differential Revision: https://reviews.freebsd.org/D28668
MFC after: 3 days
This allows d_off to be used with lseek to position the file so that
getdirentries(2) will return the next entry. It is not used by
readdir(3).
PR: 253411
Reported by: John Millikin <jmillikin@gmail.com>
Reviewed by: cem
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28605
I changed the Makefile to use SRCS instead of LDADD, but since there is
still and absolute path to the source the .o file was created inside the
source directory instead of the build directory.
It would be nice if this was an error/warning by default, but for now just
fix this issue by using .PATH and the base name of the file.
Reported by: cy, peterj
to be a true RFC 6598 NAT444 setup, where each network segment (e.g. user,
subnet) can have their own dedicated port aliasing ranges.
Reviewed by: donner, kp
Approved by: 0mp (mentor), donner, kp
Differential Revision: https://reviews.freebsd.org/D23450
In the CheriBSD CI, we run the testsuite with /tmp as tmpfs. This causes
the extattr audit tests to fail since tmpfs does not (yet) support
extattrs. Skip those tests if the target path is on a file system that
does not support extended file attributes.
While touching these two files also convert the ATF_REQUIRE_EQ(-1, ...)
checks to use ATF_REQURIE_ERRNO().
Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D28392
This avoids a SIGILL when running these tests on QEMU (which
defaults to a basic amd64 CPU without SSE4.2).
This commit also tests the table-based implementations in addition to
testing the hw-accelerated crc32 versions.
Reviewed By: cem, kib, markj
Differential Revision: https://reviews.freebsd.org/D28395
This changes the behaviour to a 30s total timeout (needed when running
on slow emulated uniprocessor systems) and timing out after 10s without
any input. This also uses timespecsub() instead of ignoring the
nanoseconds field.
After this change the tests runs more reliably on QEMU and time out less
frequently.
Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D28391
rtinit[1]() is a function used to add or remove interface address prefix routes,
similar to ifa_maintain_loopback_route().
It was intended to be family-agnostic. There is a problem with this approach
in reality.
1) IPv6 code does not use it for the ifa routes. There is a separate layer,
nd6_prelist_(), providing interface for maintaining interface routes. Its part,
responsible for the actual route table interaction, mimics rtenty() code.
2) rtinit tries to combine multiple actions in the same function: constructing
proper route attributes and handling iterations over multiple fibs, for the
non-zero net.add_addr_allfibs use case. It notably increases the code complexity.
3) dstaddr handling. flags parameter re-uses RTF_ flags. As there is no special flag
for p2p connections, host routes and p2p routes are handled in the same way.
Additionally, mapping IFA flags to RTF flags makes the interface pretty messy.
It make rtinit() to clash with ifa_mainain_loopback_route() for IPV4 interface
aliases.
4) rtinit() is the last customer passing non-masked prefixes to rib_action(),
complicating rib_action() implementation.
5) rtinit() coupled ifa announce/withdrawal notifications, producing "false positive"
ifa messages in certain corner cases.
To address all these points, the following has been done:
* rtinit() has been split into multiple functions:
- Route attribute construction were moved to the per-address-family functions,
dealing with (2), (3) and (4).
- funnction providing net.add_addr_allfibs handling and route rtsock notificaions
is the new routing table inteface.
- rtsock ifa notificaion has been moved out as well. resulting set of funcion are only
responsible for the actual route notifications.
Side effects:
* /32 alias does not result in interface routes (/32 route and "host" route)
* RTF_PINNED is now set for IPv6 prefixes corresponding to the interface addresses
Differential revision: https://reviews.freebsd.org/D28186
Previously, we would accept any kind of LIO_* opcode, including ones
that were intended for in-kernel use only like LIO_SYNC (which is not
defined in userland). The situation became more serious with
022ca2fc7f. After that revision, setting
aio_lio_opcode to LIO_WRITEV or LIO_READV would trigger an assertion.
Note that POSIX does not specify what should happen if aio_lio_opcode is
invalid.
MFC-with: 022ca2fc7f
Reviewed by: jhb, tmunro, 0mp
Differential Revision: <https://reviews.freebsd.org/D28078
aio_fsync(O_DSYNC, ...) is the asynchronous version of fdatasync(2).
Reviewed by: kib, asomers, jhb
Differential Review: https://reviews.freebsd.org/D25071
POSIX AIO is great, but it lacks vectored I/O functions. This commit
fixes that shortcoming by adding aio_writev and aio_readv. They aren't
part of the standard, but they're an obvious extension. They work just
like their synchronous equivalents pwritev and preadv.
It isn't yet possible to use vectored aiocbs with lio_listio, but that
could be added in the future.
Reviewed by: jhb, kib, bcr
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D27743
This updates the FUSE protocol to 7.28, though most of the new features
are optional and are not yet implemented.
MFC after: 2 weeks
Relnotes: yes
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27818
An order-of-operations problem caused an expectation intended for
FUSE_READ to instead match FUSE_ACCESS. Surprisingly, only one test
case was affected.
MFC after: 2 weeks
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27818
FUSE_LSEEK reports holes on fuse file systems, and is used for example
by bsdtar.
MFC after: 2 weeks
Relnotes: yes
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27804
maxphys is now a tunable, ever since r368124. The default value is also
larger than it used to be. That broke several fusefs tests that made
assumptions about maxphys.
* WriteCluster.clustering used the MAXPHYS compile-time constant.
* WriteBackAsync.direct_io_partially_overlaps_cached_block implicitly
depended on the default value of maxphys. Fix it by making the
dependency explicit.
* Write.write_large implicitly assumed that maxphys would be no more
than twice maxbcachebuf. Fix it by explicitly setting m_max_write.
* WriteCluster.clustering and several others failed because the MockFS
module did not work for max_write > 128KB (which most tests would set
when maxphys > 256KB). Limit max_write accordingly. This is the same
as fusefs-libs's behavior.
* Bmap's tests were originally written for MAXPHYS=128KB. With larger
values, the simulated file size was too small.
PR: 252096
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D27769
It turns out pf incorrectly updates the TCP checksum if the TCP option
we're modifying is not 2-byte algined with respect to the start of the
packet.
Create a TCP packet with such an option and throw it through a scrub
rule, which will update timestamps and modify the packet.
PR: 240416
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27688
Macfilter to route packets through different hooks based on sender MAC address.
Based on ng_macfilter written by Pekka Nikander
Sponsered by Retina b.v.
Reviewed by: afedorov
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D27268
Enable in-kernel acceleration of SHA1 and SHA2 operations on arm64 by adding
support for the ossl(4) crypto driver. This uses OpenSSL's assembly routines
under the hood, which will detect and use SHA intrinsics if they are
supported by the CPU.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27390
Changing a table from not having counters to having counters (or vice versa)
may trigger panics.
PR: 251414
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27441
During the life of a process, new file descriptor tables may be allocated. When
a new table is allocated, the old table is placed in a free list and held onto
until all processes referencing them exit.
When a new file descriptor table is allocated, the old file descriptor table
can be freed when the current process has a single-thread and the file
descriptor table is not being shared with any other processes.
Reviewed by: kevans
Approved by: kevans (mentor)
Differential Revision: https://reviews.freebsd.org/D18617
sysctl 'kern.cryptodevallowsoft' was renamed to 'kern.crypto.allow_soft' in
r359374 and the prevous one is only available in kernel built with
"options COMPAT_FREEBSD12".
Foundation copyrights, approved by emaste@. It does not include
files which carry other people's copyrights; if you're one
of those people, feel free to make similar change.
Reviewed by: emaste, imp, gbe (manpages)
Differential Revision: https://reviews.freebsd.org/D26980
802.1ad interfaces are created with ifconfig using the "vlanproto" parameter.
Eg., the following creates a 802.1Q VLAN (id #42) over a 802.1ad S-VLAN
(id #5) over a physical Ethernet interface (em0).
ifconfig vlan5 create vlandev em0 vlan 5 vlanproto 802.1ad up
ifconfig vlan42 create vlandev vlan5 vlan 42 inet 10.5.42.1/24
VLAN_MTU, VLAN_HWCSUM and VLAN_TSO capabilities should be properly
supported. VLAN_HWTAGGING is only partially supported, as there is
currently no IFCAP_VLAN_* denoting the possibility to set the VLAN
EtherType to anything else than 0x8100 (802.1ad uses 0x88A8).
Submitted by: Olivier Piras
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D26436
When trapping on a wrote access to a buffer the kernel has mapped as write
only we should only pass the VM_PROT_WRITE flag. Previously the call to
vm_fault_trap as the VM_PROT_READ flag was unexpected.
Reported by: manu
Sponsored by: Innovate UK
There's a know issue where new group members don't get the 'set skip on'
applied until the rules are re-loaded.
Do this by setting rules that block all traffic, but skip members of the
'epair' group. If we can communicate over the epair interface we know the set
skip rule took effect, even if the rule was set before the interface was
created.
MFC after: 2 weeks
sys.capsicum.functional.Capability__NoBypassDAC
sys.capsicum.functional.Pdfork__OtherUserForked
PR: 250178, 250179
Sponsored by: The FreeBSD Foundation
This leaves the main test body untouched and only skip running in the CI env,
makes doing local test easier while developing.
PR: 244165
Sponsored by: The FreeBSD Foundation
sys.capsicum.functional.ForkedOpenatTest_WithFlagInCapabilityMode___
sys.capsicum.functional.OpenatTest__WithFlag
PR: 249960
Sponsored by: The FreeBSD Foundation
Add a wrapping script to use ATF to run tests written with Googletest
one by one. This helps locating and tracking the failing case in CI easier.
This is a temporarily solution while Googletest support in Kyua is developing.
We will revert this once Kyua+Googletest integration is ready.
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25896
This is a workaround for the current continuously failing test case
sys.kern.sonewconn_overflow.sonewconn_overflow_01
The side effect is the dmesg buffer got cleared and may effect other tests
depends on dmesg output running in parallel. The better solution would be
tailing the log file like /var/log/debug.log
Sponsored by: The FreeBSD Foundation
Nexthop lookup was not consireding rt_flags when doing
structure comparison, which lead to an original nexthop
selection when changing flags. Fix the case by adding
rt_flags field into comparison and rearranging nhop_priv
fields to allow for efficient matching.
Fix `route change X/Y flags` case - recent changes
disallowed specifying RTF_GATEWAY flag without actual gateway.
It turns out, route(8) fills in RTF_GATEWAY by default, unless
-interface flag is specified. Fix regression by clearing
RTF_GATEWAY flag instead of failing.
Fix route flag reporting in RTM_CHANGE messages by explicitly
updating rtm_flags after operation competion.
Add IPv4/IPv6 tests for flag-only route changes.
Repeating the default WARNS here makes it slightly more difficult to
experiment with default WARNS changes, e.g. if we did something absolutely
bananas and introduced a WARNS=7 and wanted to try lifting the default to
that.
Drop most of them; there is one in the blake2 kernel module, but I suspect
it should be dropped -- the default WARNS in the rest of the build doesn't
currently apply to kernel modules, and I haven't put too much thought into
whether it makes sense to make it so.
many sockets in TIME_WAIT state at the end of the test.
PR: 249885
Reviewed by: markj
Approved by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D26549
If a FUSE server returns FOPEN_DIRECT_IO in response to FUSE_OPEN, that
instructs the kernel to bypass the page cache for that file. This feature
is also known by libfuse's name: "direct_io".
However, when accessing a file via mmap, there is no possible way to bypass
the cache completely. This change fixes a deadlock that would happen when
an mmap'd write tried to invalidate a portion of the cache, wrongly assuming
that a write couldn't possibly come from cache if direct_io were set.
Arguably, we could instead disable mmap for files with FOPEN_DIRECT_IO set.
But allowing it is less likely to cause user complaints, and is more in
keeping with the spirit of open(2), where O_DIRECT instructs the kernel to
"reduce", not "eliminate" cache effects.
PR: 247276
Reported by: trapexit@spawn.link
Reviewed by: cem
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26485
memfd_create is implemented on top of posixshm, so this is a logically
correct place for them to be. Moreover, this reduces the number of places to
look to run tests when working in this part of the tree.
Discussed with: kib (to some extent, a while ago)
The current default is provided in various Makefile.inc in some top-level
directories and covers a good portion of the tree, but doesn't cover parts
of the build a little deeper (e.g. libcasper).
Provide a default in src.sys.mk and set WARNS to it in bsd.sys.mk if that
variable is defined. This lets us relatively cleanly provide a default WARNS
no matter where you're building in the src tree without breaking things
outside of the tree.
Crunchgen has been updated as a bootstrap tool to work on this change
because it needs r365605 at a minimum to succeed. The cleanup necessary to
successfully walk over this change on WITHOUT_CLEAN builds has been added.
There is a supplemental project to this to list all of the warnings that are
encountered when the environment has WARNS=6 NO_WERROR=yes:
https://warns.kevans.dev -- this project will hopefully eventually go away
in favor of CI doing a much better job than it.
Reviewed by: emaste, brooks, ngie (all earlier version)
Reviewed by: emaste, arichardson (depend-cleanup.sh change)
Differential Revision: https://reviews.freebsd.org/D26455
Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
consumers targeting prototype instruction sets.
The only place we check for MACHINE_ARCH == aarch64 is when building the
device tree blobs. As these are targeting current generation ISAs.
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26370
Main part is that kern_copyin on amd64 after LA57 should query the top
of UVA for correct operations. In fact it should started doing that
after the workaround for AMD bug with IRET in the last user page was
fixed by reducing UVA by a page.
Also since we started calculating top of UVA, fix MIPS according to
the comment.
Reported by: lwhsu
PR: 248933
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D26312
Messing with gnop devices under a zpool fails in this test, causing
the pool to be suspended and eventually the system to deadlock.
Skip the test for now until the issue is resolved.
PR: tests/248910
Discussed with: lwhsu
Sponsored by: iXsystems, Inc.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.
I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.
Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.
Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.
Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872
RTF_HOST indicates whether route is a host route
(netmask is empty or /{32,128}).
Check that if netmask is empty and host route is not specified, kernel
returns an error.
Differential Revision: https://reviews.freebsd.org/D26155
Thanks to r364064, the name cache now returns a hit where previously it
would miss. Adjust the expectations accordingly.
PR: 248583
Reported by: lwhsu
MFC with: r364064
This fixes possible link errors, similar to:
ld: error: undefined symbol: iface_setup_addr
>>> referenced by test_rtsock_l3.c:111 (tests/sys/net/routing/test_rtsock_l3.c:111)
>>> test_rtsock_l3.o:(presetup_ipv4)
>>> referenced by test_rtsock_l3.c:79 (tests/sys/net/routing/test_rtsock_l3.c:79)
>>> test_rtsock_l3.o:(presetup_ipv6)
>>> referenced by test_rtsock_l3.c:512 (tests/sys/net/routing/test_rtsock_l3.c:512)
>>> test_rtsock_l3.o:(atfu_rtm_change_v4_gw_success_body)
>>> referenced 10 more times
In C (not C++), 'naked' inline is almost always a mistake. Either use
static inline (this is appropriate for most cases), or extern inline.
MFC after: 3 days
This avoids injecting errors into the test system's mirrors.
gnop seems like a good solution here but it injects errors at the wrong
place vs where these tests expect and does not support a 'max global count'
like the failpoints do with 'n*' syntax.
Reviewed by: cem, vangyzen
Sponsored by: Dell EMC Isilon
Prior to this change a `SF_IMMUTABLE` chflagsat(2)'ed file (`path`) was left
behind, which sabotaged kyua(1) from being able to clean up the work directory,
This resulted in unnecessary work for folks having to clean up the work
directory on non-disposable systems, which defaults to `/tmp`. Use `UF_OFFLINE`
instead of `SF_IMMUTABLE`, in part because setting `SF_IMMUTABLE` isn't relevant
to the test and `SF_IMMUTABLE` cannot be cleared at all securelevels, as pointed
out by @asomers.
Additional work is required to catch cases like this upfront in the future to
avoid tester headache. See PR # 247765 for more details/followup.
Suggested by: asomers
Reviewed By: asomers, #tests
MFC after: 1 week
PR: 247761
Sponsored by: DellEMC
Differential Revision: https://reviews.freebsd.org/D25561
memfd_create fds will no longer require an ftruncate(2) to set the size;
they'll grow (to the extent that it's possible) upon write(2)-like syscalls.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D25502
geli does all of its crypto operations in a separate thread pool, so
g_eli_start, g_eli_read_done, and g_eli_write_done don't actually do very
much work. Enabling direct dispatch eliminates the g_up/g_down bottlenecks,
doubling IOPs on my system. This change does not affect the thread pool.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D25587
