Now that we've split up the datastructures used by the kernel and
userspace there's essentually no more overlap between the pf_ruleset.c
code used by userspace and kernelspace.
Copy the userspace bits to the pfctl directory and stop using the kernel
file.
Reviewed by: philip
MFC after: 2 weeks
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D27764
Our gcc-6.4 flags require non-empty function declarations.
Fix this to match the rest of the codebase.
Tested:
* compiled on gcc-6.4 for amd64
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D26795
The new C test takes 25 seconds on QEMU-RISC-V, wheras the shell version
takes 332 seconds.
Even with the latest optimizations to atf-sh this test still takes a few
seconds to startup in QEMU. Re-writing it in C reduces the runtime for a
single test from about 2-3 seconds to less than .5 seconds. Since there
are ~80 tests, this adds up to about 3-4 minutes.
This may not seem like a big speedup, but before the recent optimizations
to avoid atf_get_srcdir, each test took almost 100 seconds on QEMU RISC-V
instead of 3. This also significantly reduces the time it takes to list
the available test cases, which speeds up running the tests via kyua:
```
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test_old
...
158/158 passed (0 failed)
332.08 real 42.58 user 286.17 sys
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test
158/158 passed (0 failed)
24.96 real 9.75 user 14.26 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
0.17 real 0.06 user 0.08 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old pf1001
pfctl_test_old: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_old: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id Refs Name
141 1 pf
Executing command [ pfctl -o none -nvf - ]
passed
1.73 real 0.25 user 1.41 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old -l > /dev/null
24.36 real 2.26 user 21.86 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
0.04 real 0.02 user 0.01 sys
```
The speedups are even more noticeable on CHERI-RISC-V (since QEMU runs
slower when emulating CHERI instructions):
```
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new -l > /dev/null
0.51 real 0.49 user 0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
34.20 real 32.69 user 0.16 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id Refs Name
147 1 pf
Executing command [ pfctl -o none -nvf - ]
passed
5.74 real 5.41 user 0.03 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new pf1001
pfctl_test_new: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_new: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
0.68 real 0.66 user 0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl #
```
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D26779
I have been trying to reduce the time that testsuite runs take for CheriBSD
on QEMU (currently about 22 hours). One of the slowest tests is pfctl_test:
Just listing the available test cases currently takes 98 seconds on a
CheriBSD RISC-V system due to all the processes being spawned. This trivial
patch reduces the time to 92 seconds. The better solution would be to
rewrite the test in C/C++ which I may do as a follow-up change.
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D26417
ifa_grouplookup() uses the data loaded in ifa_load() (through is_a_group()), so
we must call ifa_load() before we can rely on any of the data it populates.
Submitted by: Nick Rogers
MFC after: 1 week
Sponsored by: RG Nets
r343287 / D18759 introduced ifa_add_groups_to_map() which is now run by
ifa_load/ifa_lookup/host_if. When loading an anchor or ruleset via pfctl that
does NOT contain ifnames as hosts, host() still ends up iterating all
interfaces twice, grabbing SIOCGIFGROUP ioctl twice for each. This adds an
unnecessary amount of time on systems with thousands or tens of thousands of
interfaces.
Prioritize the IPv4/6 check over the interface name lookup, which skips loading
the iftab and iterating all interfaces when the configuration does not contain
interface names.
Submitted by: Nick Rogers
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24100
Warn users when they try to add/delete/modify more items than the kernel will
allow.
Reviewed by: allanjude (previous version), Lutz Donnerhacke
Differential Revision: https://reviews.freebsd.org/D22733
We cannot just assume that any name which ends with a letter is a group
That's not been true since we allowed renaming of network interfaces. It's also
not true for things like epair0a.
Try to retrieve the group members for the name to check, since we'll get ENOENT
if the group doesn't exist.
MFC after: 1 week
Event: Aberdeen hackathon 2019
The logic added in r343287 to avoid false-positive
sum-of-child-bandwidth check errors for HFSC queues has a bug in it
that causes the upperlimit service curve of an HFSC queue to be pulled
down to its parent's linkshare service curve if it happens to be above
it.
Upon further inspection/reflection, this generic
sum-of-child-bandwidths check does not need to be fixed for HFSC - it
needs to be skipped. For HFSC, the equivalent check is to ensure the
sum of child linkshare service curves are at or below the parent's
linkshare service curve, and this check is already being performed by
eval_pfqueue_hfsc().
This commit reverts the affected parts of r343287 and adds new logic
to skip the generic sum-of-child-bandwidths check for HFSC.
MFC after: 1 day
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D19124
Setting the length of the request got lost in r343287, which means SIOCGIFGMEMB
gives us the required length, but does not copy the names of the group members.
As a result we don't get a correct list of group members, and 'set skip on
<ifgroup>' broke.
This produced all sorts of very unexpected results, because we would end up
applying 'set skip' to unexpected interfaces.
X-MFC-with: r343287
The kernel will reject very large tables to avoid resource exhaustion
attacks. Some users run into this limit with legitimate table
configurations.
The error message in this case was not very clear:
pf.conf:1: cannot define table nets: Invalid argument
pfctl: Syntax error in config file: pf rules not loaded
If a table definition fails we now check the request_maxcount sysctl,
and if we've tried to create more than that point the user at
net.pf.request_maxcount:
pf.conf:1: cannot define table nets: too many elements.
Consider increasing net.pf.request_maxcount.
pfctl: Syntax error in config file: pf rules not loaded
PR: 235076
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18909
The number of syscalls made during parsing of any config that
defines tables is also reduced, and incorrect warnings that HFSC
parent queue bandwidths were smaller than the sum of their child
bandwidths have been fixed.
Reviewed by: kp
MFC after: 1 week
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D18759
When we skip on a group the kernel will automatically skip on the member
interfaces. We still need to update our own cache though, or we risk
overruling the kernel afterwards.
This manifested as 'set skip' working initially, then not working when
the rules were reloaded.
PR: 229241
MFC after: 1 week
When users mark an interface to not use aliases they likely also don't
want to use the link-local v6 address there.
PR: 201695
Submitted by: Russell Yount <Russell.Yount AT gmail.com>
Differential Revision: https://reviews.freebsd.org/D17633
When we set the ifname we have to copy the string, rather than just keep
the pointer.
PR: 231323
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D17507
higher bandwidth interfaces. The new value is used above 2.5 Gbps,
which is the highest standard rate that could be used prior to
r338209, so the default behavior for all existing systems should
remain the same.
The value of 128 chosen is a balance between being big enough to
reduce potential precision/quantization effects stemming from frequent
bucket refills over small time intervals and being small enough to
prevent a greedy driver from burst dequeuing more packets than it has
available hardware ring slots for whenever altq transitions from idle
to backlogged.
Reviewed by: jmallett, kp
MFC after: 2 weeks
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D16852
2^32 bps or greater to be used. Prior to this, bandwidth parameters
would simply wrap at the 2^32 boundary. The computations in the HFSC
scheduler and token bucket regulator have been modified to operate
correctly up to at least 100 Gbps. No other algorithms have been
examined or modified for correct operation above 2^32 bps (some may
have existing computation resolution or overflow issues at rates below
that threshold). pfctl(8) will now limit non-HFSC bandwidth
parameters to 2^32 - 1 before passing them to the kernel.
The extensions to the pf(4) ioctl interface have been made in a
backwards-compatible way by versioning affected data structures,
supporting all versions in the kernel, and implementing macros that
will cause existing code that consumes that interface to use version 0
without source modifications. If version 0 consumers of the interface
are used against a new kernel that has had bandwidth parameters of
2^32 or greater configured by updated tools, such bandwidth parameters
will be reported as 2^32 - 1 bps by those old consumers.
All in-tree consumers of the pf(4) interface have been updated. To
update out-of-tree consumers to the latest version of the interface,
define PFIOC_USE_LATEST ahead of any includes and use the code of
pfctl(8) as a guide for the ioctls of interest.
PR: 211730
Reviewed by: jmallett, kp, loos
MFC after: 2 weeks
Relnotes: yes
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D16782
Rely on the kernel to appropriately mark group members as skipped.
Once a group is skipped we can clear the update flag on all the members.
PR: 229241
Submitted by: Andreas Longwitz <longwitz AT incore.de>
MFC after: 1 week
target.
Also update the pfctl tests Makefile to work with this change.
Approved by: bapt (mentor)
Differential Revision: https://reviews.freebsd.org/D16430
If '-n' is set we don't use the list of skip interfaces, so don't retrieve it.
This fixes issues if 'pfctl -n' is used before the pf module is loaded. This
was broken by r333181.
Reported by: Jakub Chromy <hicks AT cgi.cz>
MFC after: 1 week
be executed in the if() conditional. If its not supposed to be printed
inside the conditional, then the braces should be removed and the extra
tabs on the fprintf() should be removed.
Noted by cross compilation with gcc-mips.
Normally pf rules are expected to do one of two things: pass the traffic or
block it. Blocking can be silent - "drop", or loud - "return", "return-rst",
"return-icmp". Yet there is a 3rd category of traffic passing through pf:
Packets matching a "pass" rule but when applying the rule fails. This happens
when redirection table is empty or when src node or state creation fails. Such
rules always fail silently without notifying the sender.
Allow users to configure this behaviour too, so that pf returns an error packet
in these cases.
PR: 226850
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
MFC after: 1 week
Sponsored by: InnoGames GmbH
In the pf rc.d script the output of `/etc/rc.d/pf status` or `/etc/rc.d/pf
onestatus` always provided an exit status of zero. This made it fiddly to
programmatically determine if pf was running or not.
Return a non-zero status if the pf module is not loaded, extend pfctl to have
an option to return an error status if pf is not enabled.
PR: 228632
Submitted by: James Park-Watt <jimmypw AT gmail.com>
MFC after: 1 week
On reload we used to first flush everything, including the list of skipped
interfaces. This can lead to termination of these connections if they send
packets before the new configuration is applied.
Note that this doesn't currently happen on 12 or 11, because of special EACCES
handling introduced in r315514. This special behaviour in tcp_output() may
change, hence the fix in pfctl.
PR: 214613
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
No functional change intended.
The route_host parsing code set the interface name, but only for the first
node_host in the list. If that one happened to be the inet6 address and the
rule wanted an inet address it'd get removed by remove_invalid_hosts() later
on, and we'd have no interface name.
We must set the interface name for all node_host entries in the list, not just
the first one.
PR: 223208
MFC after: 2 weeks
directories to SUBDIR.${MK_TESTS} idiom
This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .
No functional change intended.
MFC after: 1 weeks
Copy the most important test cases from OpenBSD's corresponding
src/regress/sbin/pfctl, those that run pfctl on a test input file and check
correctness of its output. We have also added some new tests using the same
format.
The tests consist of a collection of input files (pf*.in) and
corresponding output files (pf*.ok). We run pfctl -nv on the input
files and check that the output matches the output files. If any
discrepancy is discovered during future development in the source
tree, we know that a regression bug has been introduced into the tree.
Submitted by: paggas
Sponsored by: Google, Inc (GSoC 2017)
Differential Revision: https://reviews.freebsd.org/D11322
In this specific case the src address can be set to any, which was not
accepted prior to this commit.
pfSense bug report: https://redmine.pfsense.org/issues/6985
Reviewed by: kp
Obtained from: pfSense
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC (Netgate)
