128 Commits

Author SHA1 Message Date
Kristof Provost
8e492101ec pf: add COMPAT_FREEBSD13 for DIOCKEEPCOUNTERS
DIOCKEEPCOUNTERS used to overlap with DIOCGIFSPEEDV0, which has been
fixed in 14, but remains in stable/12 and stable/13.
Support the old, overlapping, call under COMPAT_FREEBSD13.

Reviewed by:	jhb
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D33001
2021-11-17 03:09:20 +01:00
Kristof Provost
047c4e365d pf: renumber DIOCKEEPCOUNTERS
We accidentally had two ioctls use the same base number
(DIOCKEEPCOUNTERS and DIOCGIFSPEEDV{0,1}). We get away with that on most
platforms because the size of the argument structures is different.
This does break CHERI, and is generally a bad idea anyway.
Renumber to avoid this collision.

Reported by:	jhb
2021-11-14 15:36:59 +01:00
Kristof Provost
76c5eecc34 pf: Introduce ridentifier
Allow users to set a number on rules which will be exposed as part of
the pflog header.
The intent behind this is to allow users to correlate rules across
updates (remember that pf rules continue to exist and match existing
states, even if they're removed from the active ruleset) and pflog.

Obtained from:	pfSense
MFC after:	3 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D32750
2021-11-05 09:39:56 +01:00
Mateusz Guzik
8f3d786cb3 pf: remove the flags argument from pf_unlink_state
All consumers call it with PF_ENTER_LOCKED.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-11-01 20:59:14 +01:00
Kristof Provost
76c2e71c4c pf: remove unused field from pf_kanchor
The 'match' field is only used in the userspace version of the struct
(pf_anchor).

MFC after:	2 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-10-07 19:50:22 +02:00
Kristof Provost
5062afff9d pfctl: userspace adaptive syncookies configration
Hook up the userspace bits to configure syncookies in adaptive mode.

MFC after:	1 week
Sponsored by:	Modirum MDPay
Differential Revision:	https://reviews.freebsd.org/D32136
2021-09-29 15:11:54 +02:00
Kristof Provost
bf8637181a pf: implement adaptive mode
Use atomic counters to ensure that we correctly track the number of half
open states and syncookie responses in-flight.
This determines if we activate or deactivate syncookies in adaptive
mode.

MFC after:	1 week
Sponsored by:	Modirum MDPay
Differential Revision:	https://reviews.freebsd.org/D32134
2021-09-29 15:11:54 +02:00
Kristof Provost
63b3c1c770 pf: support dummynet
Allow pf to use dummynet pipes and queues.

We re-use the currently unused IPFW_IS_DUMMYNET flag to allow dummynet
to tell us that a packet is being re-injected after being delayed. This
is needed to avoid endlessly looping the packet between pf and dummynet.

MFC after:	2 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31904
2021-09-24 11:41:25 +02:00
Kristof Provost
b64f7ce98f pf: qid and pqid can be uint16_t
tag2name() returns a uint16_t, so we don't need to use uint32_t for the
qid (or pqid). This reduces the size of struct pf_kstate slightly. That
in turn buys us space to add extra fields for dummynet later.

Happily these fields are not exposed to user space (there are user space
versions of them, but they can just stay uint32_t), so there's no ABI
breakage in modifying this.

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31873
2021-09-10 17:07:57 +02:00
Kristof Provost
bb25e36e13 pf: remove unused function prototype
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-09-07 16:38:49 +02:00
Kristof Provost
4cab80a8df pf: Add counters for syncookies
Count when we send a syncookie, receive a valid syncookie or detect a
synflood.

Reviewed by:	kbowling
MFC after:	1 week
Sponsored by:	Modirum MDPay
Differential Revision:	https://reviews.freebsd.org/D31713
2021-09-01 12:02:19 +02:00
Kristof Provost
2b10cf85f8 pf: Introduce nvlist variant of DIOCGETSTATUS
Make it possible to extend the GETSTATUS call (e.g. when we want to add
new counters, such as for syncookie support) by introducing an
nvlist-based alternative.

MFC after:	1 week
Sponsored by:   Modirum MDPay
Differential Revision:	https://reviews.freebsd.org/D31694
2021-08-29 14:59:04 +02:00
Kristof Provost
b69019c14c pf: remove DIOCGETSTATESNV
While nvlists are very useful in maximising flexibility for future
extensions their performance is simply unacceptably bad for the
getstates feature, where we can easily want to export a million states
or more.

The DIOCGETSTATESNV call has been MFCd, but has not hit a release on any
branch, so we can still remove it everywhere.

Reviewed by:	mjg
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31099
2021-07-30 11:45:28 +02:00
Mateusz Guzik
02cf67ccf6 pf: switch rule counters to pf_counter_u64
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-25 10:22:17 +02:00
Mateusz Guzik
d40d4b3ed7 pf: switch kif counters to pf_counter_u64
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-25 10:22:17 +02:00
Mateusz Guzik
fc4c42ce0b pf: switch pf_status.fcounters to pf_counter_u64
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-25 10:22:16 +02:00
Mateusz Guzik
defdcdd564 pf: add hybrid 32- an 64- bit counters
Numerous counters got migrated from straight uint64_t to the counter(9)
API. Unfortunately the implementation comes with a significiant
performance hit on some platforms and cannot be easily fixed.

Work around the problem by implementing a pf-specific variant.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-25 10:22:16 +02:00
Mateusz Guzik
d9cc6ea270 pf: hide struct pf_kstatus behind ifdef _KERNEL
Reviewed by:    kp
Sponsored by:   Rubicon Communications, LLC ("Netgate")
2021-07-23 17:34:43 +00:00
Kristof Provost
32271c4d38 pf: clean up syncookie callout on vnet shutdown
Ensure that we cancel any outstanding callouts for syncookies when we
terminate the vnet.

MFC after:	1 week
Sponsored by:	Modirum MDPay
2021-07-20 21:13:25 +02:00
Mateusz Guzik
907257d696 pf: embed a pointer to the lock in struct pf_kstate
This shaves calculation which in particular helps on arm.

Note using the & hack instead would still be more work.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-20 16:11:31 +00:00
Kristof Provost
231e83d342 pf: syncookie ioctl interface
Kernel side implementation to allow switching between on and off modes,
and allow this configuration to be retrieved.

MFC after:	1 week
Sponsored by:	Modirum MDPay
Differential Revision:	https://reviews.freebsd.org/D31139
2021-07-20 10:36:13 +02:00
Kristof Provost
8e1864ed07 pf: syncookie support
Import OpenBSD's syncookie support for pf. This feature help pf resist
TCP SYN floods by only creating states once the remote host completes
the TCP handshake rather than when the initial SYN packet is received.

This is accomplished by using the initial sequence numbers to encode a
cookie (hence the name) in the SYN+ACK response and verifying this on
receipt of the client ACK.

Reviewed by:	kbowling
Obtained from:	OpenBSD
MFC after:	1 week
Sponsored by:	Modirum MDPay
Differential Revision:	https://reviews.freebsd.org/D31138
2021-07-20 10:36:13 +02:00
Mateusz Guzik
9009d36afd pf: shrink struct pf_kstate
Makes room for a pointer.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-19 14:54:49 +02:00
Mateusz Guzik
f9aa757d8d pf: add a comment to pf_kstate concerning compat with pf_state_cmp
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-19 14:54:49 +02:00
Kristof Provost
ef950daa35 pf: match keyword support
Support the 'match' keyword.
Note that support is limited to adding queuing information, so without
ALTQ support in the kernel setting match rules is pointless.

For the avoidance of doubt: this is NOT full support for the match
keyword as found in OpenBSD's pf. That could potentially be built on top
of this, but this commit is NOT that.

MFC after:	2 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31115
2021-07-17 12:01:08 +02:00
Kristof Provost
c6bf20a2a4 pf: add DIOCGETSTATESV2
Add a new version of the DIOCGETSTATES call, which extends the struct to
include the original interface information.

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31097
2021-07-09 10:29:53 +02:00
Mateusz Guzik
19d6e29b87 pf: add pf_find_state_all_exists
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-08 14:00:55 +00:00
Kristof Provost
211cddf9e3 pf: rename pf_state to pf_kstate
Indicate that this is a kernel-only structure, and make it easier to
distinguish from others used to communicate with userspace.

Reviewed by:	mjg
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31096
2021-07-08 10:31:43 +02:00
Mateusz Guzik
f649cff587 pf: padalign global locks found in pf.c
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-05 09:56:54 +00:00
Mateusz Guzik
dc1ab04e4c pf: allow table stats clearing and reading with ruleset rlock
Instead serialize against these operations with a dedicated lock.

Prior to the change, When pushing 17 mln pps of traffic, calling
DIOCRGETTSTATS in a loop would restrict throughput to about 7 mln.  With
the change there is no slowdown.

Reviewed by:	kp (previous version)
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-05 10:42:01 +02:00
Mateusz Guzik
f92c21a28c pf: depessimize table handling
Creating tables and zeroing their counters induces excessive IPIs (14
per table), which in turns kills single- and multi-threaded performance.

Work around the problem by extending per-CPU counters with a general
counter populated on "zeroing" requests -- it stores the currently found
sum. Then requests to report the current value are the sum of per-CPU
counters subtracted by the saved value.

Sample timings when loading a config with 100k tables on a 104-way box:

stock:

pfctl -f tables100000.conf  0.39s user 69.37s system 99% cpu 1:09.76 total
pfctl -f tables100000.conf  0.40s user 68.14s system 99% cpu 1:08.54 total

patched:

pfctl -f tables100000.conf  0.35s user 6.41s system 99% cpu 6.771 total
pfctl -f tables100000.conf  0.48s user 6.47s system 99% cpu 6.949 total

Reviewed by:	kp (previous version)
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-05 10:42:01 +02:00
Mateusz Guzik
55cc305dfc pf: revert: Use counter(9) for pf_state byte/packet tracking
stats are not shared and consequently per-CPU counters only waste
memory.

No slowdown was measured when passing over 20M pps.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-06-29 07:24:53 +00:00
Mateusz Guzik
803dfe3da0 pf: deduplicate V_pf_state_z handling with pfsync
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-06-29 07:24:53 +00:00
Mateusz Guzik
e6dd0e2e8d pf: assert that sizeof(struct pf_state) <= 312
To prevent accidentally going over a threshold which makes UMA fit only
12 objects per page instead of 13.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-06-28 15:49:20 +00:00
Mateusz Guzik
d09388d013 pf: add pf_release_staten and use it in pf_unlink_state
Saves one atomic op.

Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-06-28 15:49:20 +00:00
Kristof Provost
d38630f619 pf: store L4 headers in pf_pdesc
Rather than pointers to the headers store full copies. This brings us
slightly closer to what OpenBSD does, and also makes more sense than
storing pointers to stack variable copies of the headers.

Reviewed by:	donner, scottl
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30719
2021-06-14 14:22:06 +02:00
Kristof Provost
ec7b47fc81 pf: Move provider declaration to pf.h
This simplifies life a bit, by not requiring us to repease the
declaration for every file where we want static probe points.

It also makes the gcc6 build happy.
2021-06-01 09:02:05 +02:00
Kristof Provost
d0fdf2b28f pf: Track the original kif for floating states
Track (and display) the interface that created a state, even if it's a
floating state (and thus uses virtual interface 'all').

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30245
2021-05-20 12:49:27 +02:00
Kristof Provost
0592a4c83d pf: Add DIOCGETSTATESNV
Add DIOCGETSTATESNV, an nvlist-based alternative to DIOCGETSTATES.

MFC after:      1 week
Sponsored by:   Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30243
2021-05-20 12:49:27 +02:00
Kristof Provost
1732afaa0d pf: Add DIOCGETSTATENV
Add DIOCGETSTATENV, an nvlist-based alternative to DIOCGETSTATE.

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30242
2021-05-20 12:49:26 +02:00
Kristof Provost
93abcf17e6 pf: Support killing 'matching' states
Optionally also kill states that match (i.e. are the NATed state or
opposite direction state entry for) the state we're killing.

See also https://redmine.pfsense.org/issues/8555

Submitted by:	Steven Brown
Reviewed by:	bcr (man page)
Obtained from:	https://github.com/pfsense/FreeBSD-src/pull/11/
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30092
2021-05-07 22:13:31 +02:00
Kristof Provost
abbcba9cf5 pf: Allow states to by killed per 'gateway'
This allows us to kill states created from a rule with route-to/reply-to
set.  This is particularly useful in multi-wan setups, where one of the
WAN links goes down.

Submitted by:	Steven Brown
Obtained from:	https://github.com/pfsense/FreeBSD-src/pull/11/
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30058
2021-05-07 22:13:31 +02:00
Kristof Provost
e989530a09 pf: Introduce DIOCKILLSTATESNV
Introduce an nvlist based alternative to DIOCKILLSTATES.

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30054
2021-05-07 22:13:30 +02:00
Kristof Provost
7606a45dcc pf: Introduce DIOCCLRSTATESNV
Introduce an nvlist variant of DIOCCLRSTATES.

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D30052
2021-05-07 22:13:30 +02:00
Kristof Provost
6fcc8e042a pf: Allow multiple labels to be set on a rule
Allow up to 5 labels to be set on each rule.
This offers more flexibility in using labels. For example, it replaces
the customer 'schedule' keyword used by pfSense to terminate states
according to a schedule.

Reviewed by:	glebius
MFC after:	2 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D29936
2021-04-26 14:14:21 +02:00
Kristof Provost
42ec75f83a pf: Optionally attempt to preserve rule counter values across ruleset updates
Usually rule counters are reset to zero on every update of the ruleset.
With keepcounters set pf will attempt to find matching rules between old
and new rulesets and preserve the rule counters.

MFC after:	4 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D29780
2021-04-19 14:31:47 +02:00
Kristof Provost
4f1f67e888 pf: PFRULE_REFS should not be user-visible
Split the PFRULE_REFS flag from the rule_flag field. PFRULE_REFS is a
kernel-internal flag and should not be exposed to or read from
userspace.

MFC after:	4 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D29778
2021-04-19 14:31:47 +02:00
Kurosawa Takahiro
2aa21096c7 pf: Implement the NAT source port selection of MAP-E Customer Edge
MAP-E (RFC 7597) requires special care for selecting source ports
in NAT operation on the Customer Edge because a part of bits of the port
numbers are used by the Border Relay to distinguish another side of the
IPv4-over-IPv6 tunnel.

PR:		254577
Reviewed by:	kp
Differential Revision:	https://reviews.freebsd.org/D29468
2021-04-13 10:53:18 +02:00
Kristof Provost
d710367d11 pf: Implement nvlist variant of DIOCGETRULE
MFC after:	4 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D29559
2021-04-10 11:16:01 +02:00
Kristof Provost
5c62eded5a pf: Introduce nvlist variant of DIOCADDRULE
This will make future extensions of the API much easier.
The intent is to remove support for DIOCADDRULE in FreeBSD 14.

Reviewed by:	markj (previous version), glebius (previous version)
MFC after:	4 weeks
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D29557
2021-04-10 11:16:00 +02:00