Many pf tests use identical setup where one jail is a router and
optionally another jail is a server. Add functions to create such jails
for IPv6 and IPv4 and functions to perform tests on such setup.
Add tests using those functions: scrub actions, routing table, tcp
sequence number modulation.
Reviewed by: kp
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38126
Skip the test (rather than loading it) if if_ovpn.ko isn't loaded. Match
the behaviour of other tests.
Sponsored by: Rubicon Communications, LLC ("Netgate")
In CURRENT for some time an overflowed unix/dgram socket would
return EAGAIN if it has O_NONBLOCK set. This proved to be
undesired. See 71e70c25c0 for details. Update tests to match
the "new" behavior, which actually is the historical behavior.
The actual overflow occured in the ReadAhead.readahead test.
Surprisingly it has never segfaulted or resulted in any bad behavior.
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D38718
The previous fix was incorrect: we need to verify that the current node, if it exists, is not a directory, but we were checking the parent node instead. Address this, add more tests, and fix the test cleanup routines.
PR: 269519, 269561
Fixes: ae6cff8973
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D38645
The current code missed interface addition when reallocating
temporary buffer.
Tweak the code to perform the reallocation first and add
interface afterwards unconditionally.
Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
MFC after: 3 days
PR 254212 was resolved to avoid issues with not having IPv6 addresses
present in the jail, so we can safely drop this workaround now...
Reviewed by: jhb, markj
Differential Revision: https://reviews.freebsd.org/D37755
This adds a test for a semantic that we added to mirror a feature of
the Linux implementation w/ netns: if a wg interface is moved into a
jail, we still create the socket in the context of the home vnet. With
this added, one can actually create vnet jails that only have a wg
tunnel to the outside world providing network connectivity without any
epairs in the setup.
Reviewed by: jhb, markj (both earlier version)
Differential Revision: https://reviews.freebsd.org/D37755
* tarfs_alloc_mount(): Remove an unnecessary null check (CID 1504505) and an unused variable.
* tarfs_alloc_one(): Verify that the file size is not negative (CID 1504506). While there, also validate the mode, owner and group.
* tarfs_vget(), tarfs_zio_init(): Explicitly ignore return value from getnewvnode(), which cannot fail (CID 1504508)
* tarfs_lookup_path(): Fix a case where a specially-crafted tarball could trigger a null pointer dereference by first descending into, and then backing out of, a previously unknown directory. (CID 1504515)
* mktar: Construct a tarball that triggers the aforementioned null pointer dereference.
Reported by: Coverity
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D38463
Some operations like interface creation may need to return metadata
- in this case, interface name - back to the caller if the operation
is successful.
This change implements attaching an `NLMSGERR_ATTR_COOKIE` nla to the
operation reply message via `nlmsg_report_cookie()`.
Additionally, on successful interface creation, interface index and
interface name are returned in the `IFLA_NEW_IFINDEX` and `IFLA_IFNAME
TLVs, encapsulated in the `NLMSGERR_ATTR_COOKIE`.
Reviewed By: pauamma
Differential Revision: https://reviews.freebsd.org/D38283
MFC after: 1 week
Currently isolation and resource requirements are handled directly
by the kyua runner, based on the requirements specified by the test.
It works well for simple tests, but may cause discrepancy with tests
doing complex pre-setups. For example, all tests that perform
VNET setups require root access to properly function.
This change adds additional handling of the "require_user" property
within the python testing framework. Specifically, it requests
root access if the test class signals its root requirements and
drops privileges to the desired user after performing the pre-setup.
Differential Revision: https://reviews.freebsd.org/D37923
MFC after: 2 weeks
In e2650af157 was added "_S" macros for compatibility with glibc, but it's still
incompatible as under glibc the macros whose names end with "_S" operate on the
dynamically allocated CPU set(s) whose size is in bytes, not in bits.
While here remove limiting ifdef to non-kernel case.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D38110
MFC after: 1 week
Return different exit code depending on which failure was encountered.
The pfsync test expect a very particular failure, not just any.
MFC after: 1 week
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38123
Multiple improvements to pft_ping.py:
* Automatically use IPv6 when IPv6 addresses are used, --ip6 is not needed.
* Building of ping requests and parsing of ping replies is done layer by
layer. This way most arguments are available both for IPv6 and IPv4,
for ICMP and TCP.
* Use argument groups for improved readability.
* Change ToS and TTL argument name to TC and HL to reflect the modern
IPv6 nomenclature. The argument still set related IPv4 header fields
properly.
* Instead of sniffing for the very specific case of duplicated packets,
allow for sniffing on multiple interfaces.
* Report which sniffer has failed by setting bits of error code.
* Raise meaningful exceptions when irrecoverable errors happen.
* Make IPv4 fragmentation flags configurable.
* Make IPv6 HL / IPv4 TTL configurable.
* Make TCP MSS configurable.
* Make TCP sequence number configurable.
* Make ICMP payload size configurable.
* Add debug output.
* Move command line argument parsing out of network functions.
* Make the code somehow PEP-8 compliant.
MFC after: 1 week
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38122
Multiple improvements to sniffer.py:
* Remove ambiguity of configuring recvif, it must be now explicitly specified.
* Don't catch exceptions around creating the sniffer, let it properly
fail and display the whole stack trace.
* Count correct packets so that duplicates can be found.
MFC after: 1 week
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38120
Some existing applications setup Netlink socket with
SOCK_DGRAM instead of SOCK_RAW. Update the manpage to clarify
that the default way of creating the socket should be with
SOCK_RAW. Update the code to support both SOCK_RAW and SOCK_DGRAM.
Reviewed By: pauamma
Differential Revision: https://reviews.freebsd.org/D38075
Some apps try to provide only the non-zero part of the required message
header instead of the full one. It happens when fetching routes or
interface addresses, where the first header byte is the family.
This behavior is "illegal" under the "strict" Netlink socket option,
however there are many applications out there doing things in the
"old" way.
Support this usecase by copying the provided bytes into the temporary
zero-filled header and running the parser on this header instead.
Reported by: Goran Mekić <meka@tilda.center>
Currently `close(2)` erroneously return `EOPNOTSUPP` for `PF_ROUTE` sockets.
It happened after making rtsock socket implementation self-contained (
36b10ac2cd ). Rtsock code marks socket as connected in `rts_attach()`.
`soclose()` tries to disconnect such socket using `.pr_disconnect` callback.
Rtsock does not implement this callback, resulting in the default method being
substituted. This default method returns `ENOTSUPP`, failing `soclose()` logic.
This diff restores the previous behaviour by adding custom `pr_disconnect()`
returning `ENOTCONN`.
Reviewed by: glebius
Differential Revision: https://reviews.freebsd.org/D38059
The pfsync:pbr tests leaves lot of interfaces when they finish, making
other tests slower due to long time of loading scapy from pft_ping.py
when more interfaces are present. When both sides of epair are assigned
to jails, they are both removed from created_interfaces.lst and thus
won't be removed during cleanup from this file. An interface assigned to
jail is stored in created_jails.lst but if it is renamed, it won't be
cleaned up either. Furthermore this test uses identical names for
multiple interfaces across multiple jails which after destroying those
jails adds to overall confusion.
To address this issue a new function is provided for renaming interfaces
in jails and storing their new names in created_jails.lst for deletion
during cleanup.
MFC after: 1 week
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38024
When a src/dst ip/port tuple is re-used before the pf state fully
expires we clean up the state and create a new one, unless syncookies
are enabled.
Test this, by running two back-to-back nc sessions, with a fixed source
port. Move the interface and IP to a different (vnet) jail, to trick the
network stack into letting us do this.
MFC after: 2 weeks
Event: Aberdeen hackathon 2022
Differential Revision: https://reviews.freebsd.org/D36886
Checks basic forward to local address, also with presence of
a listener that matches original port (see aab8c844b9). Based
on non-ATF test case written by Pavel Polyakov.
Reviewed by: kp, melifaro
Differential revision: https://reviews.freebsd.org/D37960
In order to provide more flexibility for the test writers,
add per-test-method cleanups in addition to the per-class cleanups.
Now the test 'test_one' can perform cleanup by either defining
per-class 'cleanup' method (typically used in VNET classes) and
per-test method 'cleanup_test_one'. The latter has preference.
In order to handle paramatrization, testid is passed as a single
argument to both of the methods.
MFC after: 2 weeks
* Derive jail name from class name and method name, instead of just
method name. This change reduces the chances of different tests
clashing.
Old: 'jail_test_one'. New: 'pytest:TestExampleSimplest:test_one'
* Simplify vnetX_handler() method signature by skipping obj_map (unused)
and pipe. The latter can be accessed as the vnet property.
* Add `send_object()` method as a pair to the `wait_object` inside the
VnetTestTemplate class.
* Add `test_id` property to the BaseTest method. Previously it was
provided only for the VnetTestTemplate class. This change makes
the identifier easily accessible for all users.
MFC after: 2 weeks
For tests that send invalid data to a TLS socket to trigger read
errors the kernel may end up dropping the connection before close is
called at the conclusion of the test resulting in spurious ECONNRESET
errors from close. Ignore any errors from close for these tests.
PR: 268390
Reported by: olivier, Michal Gulbicki <michalx.gulbicki@intel.com>
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37717
For some of the "bad size" tests, the remote end can notice the error
and drop the connection before the test program returns from write to
call shutdown. In that case, shutdown fails with ENOTCONN. Permit
these ENOTCONN errors without failing the test.
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37693
These were originally added in commit
74ae3f3e33 but have been updated to use
wg(8).
Reviewed by: kevans, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37400
- Make use of ATF_REQUIRE_INTEQ to log the values of integers for
unexpected mismatches.
- Use ATF_REQUIRE_MSG in a few more places to log values when a
requirement fails.
- Consistently use ATF_REQUIRE_ERRNO when checking for an expected
errno value.
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37691
* Separate interface creation from interface modification code
* Support setting some interface attributes (ifdescr, mtu, up/down, promisc)
* Improve interaction with the cloners requiring to parse/write custom
interface attributes
* Add bitmask-based way of checking if the attribute is present in the
message
* Don't use multipart RTM_GETLINK replies when searching for the
specific interface names
* Use ENODEV instead of ENOENT in case of failed RTM_GETLINK search
* Add python netlink test helpers
* Add some netlink interface tests
Differential Revision: https://reviews.freebsd.org/D37668
The previous commit fixed a memory leak, where we'd fail to clean up
removed groups (and interfaces).
Check that we now clean those up as expected.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37570
GCC 12's -Wsign-compare complains if the two alternative results of
the ?: operator are differently signed. Cast the small, sub-page
off_t values to size_t to quiet the warning.
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D37539
This avoids leaking a pointer to the on-stack test_nmctx which
triggers a -Wdangling-pointer warning from GCC.
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D37536
We can now tell scrub rules to not reassemble packets. Test that this
affects packets being passed or dropped as expected.
Sponsored by: Rubicon Communications, LLC ("Netgate")
These should all trigger errors when reading from the socket.
Tests include truncated records (socket closed early on the other
side), corrupted records (bits flipped in explicit IVs, ciphertext, or
MAC), invalid header fields, and various invalid record lengths.
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37373
TL;DR: Three ZFS tests created ZFS pools on all unmounted devices listed
in /etc/fstab, corrupting their contents. Stop that.
Imagine my surprise when the ESP on my main dev/test VM would "randomly"
become corrupted, making it unbootable. Three tests collect various devices
from the system and try to add them to a test pool. The test expects this
to fail because it _assumes_ these devices are in use and ZFS will correctly
reject the request.
My /etc/fstab has two entries for devices in /dev:
/dev/gpt/swap0 none swap sw,trimonce,late
/dev/gpt/esp0 /boot/efi msdosfs rw,noauto
Note the `noauto` on the ESP. In a remarkable example of irony, I chose
this because it should keep the ESP more protected from corruption;
in fact, mounting it would have protected it from this case.
The tests added all of these devices to a test pool in a _single command_,
expecting the command to fail. The swap device was in use, so the command
correctly failed, but the ESP was added and therefore corrupted. However,
since the command correctly failed, the test didn't notice the ESP problem.
If each device had been added with its own command, the test _might_ have
noticed that one of them incorrectly succeeded. However, two of these
tests would not have noticed:
hotspare_create_001_neg was incorrectly specified as needing the Solaris
dumpadm command, so it was skipped. _Some_ of the test needs that command,
but it checks for its presence and runs fine without it.
Due to bug 241070, zpool_add_005_pos was marked as an expected failure.
Due to the coarse level of integration with ATF, this test would still
"pass" even if it failed for the wrong reason. I wrote bug 267554 to
reconsider the use of atf_expect_fail in these tests.
Let's further consider the use of various devices found around the system.
In addition to devices in /etc/fstab, the tests also used mounted devices
listed by the `mount` command. If ZFS behaves correctly, it will refuse
to added mounted devices and swap devices to a pool. However, these are
unit tests used by developers to ensure that ZFS still works after they
modify it, so it's reasonable to expect ZFS to do the _wrong_ thing
sometimes. Using random host devices is unsafe.
Fix the root problem by using only the disks provided via the "disks"
variable in kyua.conf. Use one to create a UFS file system and mount it.
Use another as a swap device. Use a third as a dump device, but expect
it to fail due to bug 241070.
While I'm here:
Due to commit 6b6e2954dd, we can simply add a second dump device and
remove it in cleanup. We no longer need to save, replace, and restore the
pre-existing dump device.
The cleanup_devices function used `camcontrol inquiry` to distinguish disks
from other devices, such as partitions. That works fine for SCSI, but not
for ATA or VirtIO block. Use `geom disk list` instead.
PR: 241070
PR: 267554
Reviewed by: asomers
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D37257
fq_codel was broken due to a missing m_rcvif_serialize() in the enqueue
path.
Add a simple test case to ensure it at least passes traffic.
Sponsored by: Rubicon Communications, LLC ("Netgate")
The root cause of the intermittent span test failures has been
identified as a race between sending the packet and starting the bpf
capture.
This is now resolved, so the test can be re-enabled.
PR: 260461
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
The Sniffer class is often used by test tools such as pft_ping to verify
that packets actually get sent where they're expected.
It starts a background thread to capture packets, but this thread needs
some time to start, leading to intermittent test failures when the
capture doesn't start before the relevant packet is sent.
Add a semaphore to ensure the Sniffer constructor doesn't return until
the capture is actually running.
PR: 260461
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Rather than using a Scapy-based Python script only check if the state
still exists. Scapy tends to be slow to start, it appears because it
lists all interfaces and gets their (IPv6) addresses a couple of times
at startup. This can be sufficient for the ICMP state to time out and
the test to fail.
We now only check if the state exists or is removed as expected, which
makes things faster, and should mean the test is more robust on slower
machines (such as CI VMs).
Sponsored by: Rubicon Communications, LLC ("Netgate")
This makes diffs when adding or removing tests easier to read.
While here, sort the list of tests.
Reviewed by: kevans, melifaro, asomers, markj, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36908
When the user specifies SEEK_END, unlike SEEK_CUR, VOP_ADVLOCK must
adjust lock offsets itself.
Sort-of related to bug 266886.
MFC after: 2 weeks
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D37040
Use the management interface to work out if we've timed out the client,
rather than looking for an openvpn process to die (with incorrect
syntax).
Sponsored by: Rubicon Communications, LLC ("Netgate")
When yield a.k.a "generator" iterator is used we need to return all
data using "yield", before returning from the function.
Because of that only encryption tests were run for AES-CBC, other modes
were affected as well.
Add one more loop to the iterator "next" routine to fix that.
This unveiled a problem in the GCM AEAD parser logic, which didn't
correctly handle tests cases with empty plaintext, i.e. AAD only.
Include the fix in this patch as it's a rather trivial one.
Obtained from: Semihalf
Differential Revision: https://reviews.freebsd.org/D36861
When the tunneled (IPv6) traffic had traffic class bits set (but only >=
16) the packet got lost on the receive side.
This happened because the address family check in ovpn_get_af() failed
to mask correctly, so the version check didn't match, causing us to drop
the packet.
While here also extend the existing 6-in-6 test case to trigger this
issue.
PR: 266598
Sponsored by: Rubicon Communications, LLC ("Netgate")
The pft_ether.py script requires both python and scapy to be installed.
Check for this so we properly skip the test when it is unavailable.
Reviewed by: kp
Fixes: 07ffa50ba0 ("pf tests: test short packets")
Differential Revision: https://reviews.freebsd.org/D36561
Test that we can successfuly set syncookie high/low watermarks for very
low or very high state limits.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D36498
With _RB_DIAGNOSTIC defined, provide an RB_RANK method to compute the
rank of a node in an rb-tree, if the subtree rooted at that node is
rank-balanced, and -1 otherwise.
In rb_test, rewrite a bit to avoid malloc/free and nondeterministic
running times because of randomness. Allocate all the nodes on the
stack, and shuffle a set of keys to get randomness for the testing.
Add a rank-balance check for the completed tree.
Reviewed by: markj
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D36484
Python's socketmodule isn't able to construct sockaddr when it doesn't
recognize the address family. Until this is solved in python let the
tests use the compatibility way to create divert(4) socket.
Ensure that a wildcard anchor actually includes any nested anchors (i.e.
foo/* will call into foo/bar).
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D36414
