Commit Graph

6706 Commits

Author SHA1 Message Date
Alexander V. Chernikov
a624ca3dff Move net/route/shared.h definitions to net/route/route_var.h.
No functional changes.

net/route/shared.h was created in the inital phases of nexthop conversion.
It was intended to serve the same purpose as route_var.h - share definitions
 of functions and structures between the routing subsystem components. At
 that time route_var.h was included by many files external to the routing
 subsystem, which largerly defeats its purpose.

As currently this is not the case anymore and amount of route_var.h includes
 is roughly the same as shared.h, retire the latter in favour of the former.
2020-08-28 22:50:20 +00:00
Michael Tuexen
404ff76bda Fix a regression with the explicit EOR mode I introduced in r364268.
A short MFC time as discussed with the secteam.

Reported by:		Taylor Brandstetter
MFC after:		1 day
2020-08-28 20:05:18 +00:00
Michael Tuexen
1951fa791e RFC 3465 defines a limit L used in TCP slow start for limiting the number
of acked bytes as described in Section 2.2 of that document.
This patch ensures that this limit is not also applied in congestion
avoidance. Applying this limit also in congestion avoidance can result in
using less bandwidth than allowed.

Reported by:		l.tian.email@gmail.com
Reviewed by:		rrs, rscheff
MFC after:		3 days
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D26120
2020-08-25 09:42:03 +00:00
Warner Losh
773e541e8d Use devctl.h instead of bus.h to reduce newbus pollution.
There's no need for these parts of the kernel to know about newbus,
so narrow what is included to devctl.h for device_notify_*.

Suggested by: kib@
2020-08-21 00:03:24 +00:00
Andrew Gallatin
b99781834f TCP: remove special treatment for hardware (ifnet) TLS
Remove most special treatment for ifnet TLS in the TCP stack, except
for code to avoid mixing handshakes and bulk data.

This code made heroic efforts to send down entire TLS records to
NICs. It was added to improve the PCIe bus efficiency of older TLS
offload NICs which did not keep state per-session, and so would need
to re-DMA the first part(s) of a TLS record if a TLS record was sent
in multiple TCP packets or TSOs. Newer TLS offload NICs do not need
this feature.

At Netflix, we've run extensive QoE tests which show that this feature
reduces client quality metrics, presumably because the effort to send
TLS records atomically causes the server to both wait too long to send
data (leading to buffers running dry), and to send too much data at
once (leading to packet loss).

Reviewed by:	hselasky,  jhb, rrs
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26103
2020-08-19 17:59:06 +00:00
Richard Scheffenegger
ad7a0eb189 TCP Cubic: recalculate cwnd for every ACK.
Since cubic calculates cwnd based on absolute
time, retaining RFC3465 (ABC) once-per-window updates
can lead to dramatic changes of cwnd in the convex
region. Updating cwnd for each incoming ack minimizes
this delta, preventing unintentional line-rate bursts.

Reviewed by:	chengc_netapp.com, tuexen (mentor)
MFC after:	2 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D26060
2020-08-18 19:34:31 +00:00
Michael Tuexen
d7351394da Fix two bugs I introduced in r362563.
Found by running syzkaller.

MFC after:	3 days
2020-08-18 19:25:03 +00:00
Michael Tuexen
d59f3890c3 Remove a line which is needed and was added in
https://svnweb.freebsd.org/changeset/base/364268

MFC after:		3 days
2020-08-16 13:31:14 +00:00
Michael Tuexen
f5d30f7f76 Improve the handling of concurrent send() calls for SCTP sockets,
especially when having the explicit EOR mode enabled.

Reported by:		Megan2013678@protonmail.com
Reported by:		syzbot+bc02585076c3cc977f9b@syzkaller.appspotmail.com
MFC after:		3 days
2020-08-16 11:50:37 +00:00
Michael Tuexen
04996cb74b Enter epoch earlier. This is needed because we are exiting it also
in error cases.

MFC after:	1 week
2020-08-15 11:22:07 +00:00
Alexander V. Chernikov
2f23f45b20 Simplify dom_<rtattach|rtdetach>.
Remove unused arguments from dom_rtattach/dom_rtdetach functions and make
  them return/accept 'struct rib_head' instead of 'void **'.
Declare inet/inet6 implementations in the relevant _var.h headers similar
  to domifattach / domifdetach.
Add rib_subscribe_internal() function to accept subscriptions to the rnh
  directly.

Differential Revision:	https://reviews.freebsd.org/D26053
2020-08-14 21:29:56 +00:00
Richard Scheffenegger
a459638fc4 TCP Cubic: Have Fast Convergence Heuristic work for ECN, and align concave region
The Cubic concave region was not aligned nicely for the very first exit from
slow start, where a 50% cwnd reduction is done instead of the normal 30%.

This addresses an issue, where a short line-rate burst could result from that
sudden jump of cwnd.

In addition, the Fast Convergence Heuristic has been expanded to work also
with ECN induced congestion response.

Submitted by:	chengc_netapp.com
Reported by:	chengc_netapp.com
Reviewed by:	tuexen (mentor), rgrimes (mentor)
Approved by:	tuexen (mentor), rgrimes (mentor)
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D25976
2020-08-13 16:45:55 +00:00
Richard Scheffenegger
2bb6dfabbe TCP Cubic: After leaving slowstart fix unintended cwnd jump.
Initializing K to zero in D23655 introduced a miscalculation,
where cwnd would suddenly jump to cwnd_max instead of gradually
increasing, after leaving slow-start.

Properly calculating K instead of resetting it to zero resolves
this issue. Also making sure, that cwnd is recalculated at the
earliest opportunity once slow-start is over.

Reported by:	chengc_netapp.com
Reviewed by:	chengc_netapp.com, tuexen (mentor), rgrimes (mentor)
Approved by:	tuexen (mentor), rgrimes (mentor)
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D25746
2020-08-13 16:38:51 +00:00
Richard Scheffenegger
f359d6ebbc Improve SACK support code for RFC6675 and PRR
Adding proper accounting of sacked_bytes and (per-ACK)
delivered data to the SACK scoreboard. This will
allow more aspects of RFC6675 to be implemented as well
as Proportional Rate Reduction (RFC6937).

Prior to this change, the pipe calculation controlled with
net.inet.tcp.rfc6675_pipe was also susceptible to incorrect
results when more than 3 (or 4) holes in the sequence space
were present, which can no longer all fit into a single
ACK's SACK option.

Reviewed by:	kbowling, rgrimes (mentor)
Approved by:	rgrimes (mentor, blanket)
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D18624
2020-08-13 16:30:09 +00:00
Hans Petter Selasky
b453d3d239 Use a static initializer for the multicast free tasks.
This makes the SYSINIT() function updated in r364072 superfluous.

Suggested by:	glebius@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-08-11 08:31:40 +00:00
Michael Tuexen
cf8a49ab6e Fix the following issues related to the TCP SYN-cache:
* Let the accepted TCP/IPv4 socket inherit the configured TTL and
  TOS value.
* Let the accepted TCP/IPv6 socket inherit the configured Hop Limit.
* Use the configured Hop Limit and Traffic Class when sending
  IPv6 packets.

Reviewed by:		rrs, lutz_donnerhacke.de
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D25909
2020-08-10 20:24:48 +00:00
Bjoern A. Zeeb
f9461246a2 MC: add a note with reference to the discussion and history as-to why we
are where we are now.  The main thing is to try to get rid of the delayed
freeing to avoid blocking on the taskq when shutting down vnets.

X-Timeout:	if you still see this before 14-RELEASE remove it.
2020-08-10 10:58:43 +00:00
Hans Petter Selasky
3689652c65 Make sure the multicast release tasks are properly drained when
destroying a VNET or a network interface.

Else the inm release tasks, both IPv4 and IPv6 may cause a panic
accessing a freed VNET or network interface.

Reviewed by:		jmg@
Discussed with:		bz@
Differential Revision:	https://reviews.freebsd.org/D24914
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-08-10 10:46:08 +00:00
Hans Petter Selasky
a95ef9d38d Use proper prototype for SYSINIT() functions.
Mark the unused argument using the __unused macro.

Discussed with:		kib@
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-08-10 10:40:19 +00:00
Michael Tuexen
1bea15e601 Improve the ECN negotiation when the TCP SYN-cache is used by making
sure that
* ECN is disabled if the client sends an non-ECN-setup SYN segment.
* ECN is disabled is the ECN-setup SYN-ACK segment is retransmitted more
  than net.inet.tcp.ecn.maxretries times.

Reviewed by:		rscheff
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D26008
2020-08-08 19:39:38 +00:00
Bjoern A. Zeeb
a9839c4aee IPV6_PKTINFO support for v4-mapped IPv6 sockets
When using v4-mapped IPv6 sockets with IPV6_PKTINFO we do not
respect the given v4-mapped src address on the IPv4 socket.
Implement the needed functionality. This allows single-socket
UDP applications (such as OpenVPN) to work better on FreeBSD.

Requested by:	Gert Doering (gert greenie.net), pfsense
Tested by:	Gert Doering (gert greenie.net)
Reviewed by:	melifaro
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D24135
2020-08-07 15:13:53 +00:00
Randall Stewart
8315f1ea26 The recent changes to move the ref count increment
back from the end of the function created an issue.
If one of the routines returns NULL during setup
we have inp's with extra references (which is why
the increment was at the end).

Also the stack switch return code was being ignored
and actually has meaning if the stack cannot take over
it should return NULL.

Fix both of these situation by being sure to test the
return code and of course in any case of return NULL (there
are 3) make sure we properly reduce the ref count.

Sponsored by:	Netflix Inc.
Differential Revision:	https://reviews.freebsd.org/D25903
2020-07-31 10:03:32 +00:00
Michael Tuexen
205f3e1597 Clear the pointer to the socket when closing it also in case of
an ungraceful operation.
This fixes a use-after-free bug found and reported by Taylor
Brandstetter of Google by testing the userland stack.

MFC after:		1 week
2020-07-23 19:43:49 +00:00
Michael Tuexen
91e04f9e7a Detect and handle an invalid reassembly constellation, which results in
a memory leak.

Thanks to Felix Weinrank for finding this issue using fuzz testing the
userland stack.

MFC after:		1 week
2020-07-23 01:35:24 +00:00
Richard Scheffenegger
cce999b38f Fix style and comment around concave/convex regions in TCP cubic.
In cubic, the concave region is when snd_cwnd starts growing slower
towards max_cwnd (cwnd at the time of the congestion event), and
the convex region is when snd_cwnd starts to grow faster and
eventually appearing like slow-start like growth.

PR:		238478
Reviewed by:	tuexen (mentor), rgrimes (mentor)
Approved by:	tuexen (mentor), rgrimes (mentor)
MFC after:	2 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D24657
2020-07-21 16:21:52 +00:00
Richard Scheffenegger
66ba9aafcf Add MODULE_VERSION to TCP loadable congestion control modules.
Without versioning information, using preexisting loader /
linker code is not easily possible when another module may
have dependencies on pre-loaded modules, and also doesn't
allow the automatic loading of dependent modules.

No functional change of the actual modules.

Reviewed by:	tuexen (mentor), rgrimes (mentor)
Approved by:	tuexen (mentor), rgrimes (mentor)
MFC after:	2 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D25744
2020-07-20 23:47:27 +00:00
Michael Tuexen
8745f898c4 Add reference counts for inp/stcb/net when timers are running.
This avoids a use-after-free reported for the userland stack.
Thanks to Taylor Brandstetter for suggesting a patch for
the userland stack.

MFC after:		1 week
2020-07-19 12:34:19 +00:00
Michael Tuexen
05bceec68e Remove code which is not needed.
MFC after:		1 week
2020-07-18 13:10:02 +00:00
Michael Tuexen
7f0ad2274b Improve the locking of address lists by adding some asserts and
rearranging the addition of address such that the lock is not
given up during checking and adding.

MFC after:		1 week
2020-07-17 15:09:49 +00:00
Michael Tuexen
f903a308a1 (Re)-allow 0.0.0.0 to be used as an address in connect() for TCP
In r361752 an error handling was introduced for using 0.0.0.0 or
255.255.255.255 as the address in connect() for TCP, since both
addresses can't be used. However, the stack maps 0.0.0.0 implicitly
to a local address and at least two regressions were reported.
Therefore, re-allow the usage of 0.0.0.0.
While there, change the error indicated when using 255.255.255.255
from EAFNOSUPPORT to EACCES as mentioned in the man-page of connect().

Reviewed by:		rrs
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D25401
2020-07-16 16:46:24 +00:00
Michael Tuexen
504ee6a001 Improve the error handling in generating ASCONF chunks.
In case of errors, the cleanup was not consistent.
Thanks to Felix Weinrank for fuzzing the userland stack and making
me aware of the issue.

MFC after:		1 week
2020-07-14 20:32:50 +00:00
Michael Tuexen
cd7518203d Cleanup, no functional change intended.
This file is only compiled if INET or INET6 is defined. So there
is no need for checking that.

Reviewed by:		markj
Differential Revision:	https://reviews.freebsd.org/D25635
2020-07-12 18:34:09 +00:00
Michael Tuexen
83b8204f61 (Re)activate SCTP system calls when compiling SCTP support into the kernel
r363079 introduced the possibility of loading the SCTP stack as a module in
addition to compiling it into the kernel. As part of this, the registration
of the system calls was removed and put into the loading of the module.
Therefore, the system calls are not registered anymore when compiling the
SCTP into the kernel. This patch addresses that.

Reviewed by:		markj
Differential Revision:	https://reviews.freebsd.org/D25632
2020-07-12 14:50:12 +00:00
Michael Tuexen
4ef7c2f28f Whitespace changes due to upstreaming r363079. 2020-07-10 16:59:06 +00:00
Mark Johnston
052c5ec4d0 Provide support for building SCTP as a loadable module.
With this change, a kernel compiled with "options SCTP_SUPPORT" and
without "options SCTP" supports dynamic loading of the SCTP stack.

Currently sctp.ko cannot be unloaded since some prerequisite teardown
logic is not yet implemented.  Attempts to unload the module will return
EOPNOTSUPP.

Discussed with:	tuexen
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21997
2020-07-10 14:56:05 +00:00
Michael Tuexen
6ddc843832 Fix a use-after-free bug for the userland stack. The kernel
stack is not affected.
Thanks to Mark Wodrich from Google for finding and reporting the
bug.

MFC after:		1 week
2020-07-10 11:15:10 +00:00
Michael Tuexen
b6734d8f4a Optimize flushing of receive queues.
This addresses an issue found and reported for the userland stack in
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21243

MFC after:		1 week
2020-07-09 16:18:42 +00:00
Michael Tuexen
fcbfdc0ab6 Improve consistency.
MFC after:		1 week
2020-07-08 16:23:40 +00:00
Michael Tuexen
ef9095c72a Fix error description.
MFC after:		1 week
2020-07-08 16:04:06 +00:00
Michael Tuexen
c96d7c373e Don't accept FORWARD-TSN chunks when I-FORWARD-TSN was negotiated
and vice versa.

MFC after:		1 week
2020-07-08 15:49:30 +00:00
Michael Tuexen
32df1c9ebb Improve handling of PKTDROP chunks. This includes the input validation
to address two issues found by ossfuzz testing the userland stack:
* https://oss-fuzz.com/testcase-detail/5387560242380800
* https://oss-fuzz.com/testcase-detail/4887954068865024
and adding support for I-DATA chunks in addition to DATA chunks.
2020-07-08 12:25:19 +00:00
Richard Scheffenegger
c201ce0b4a Fix KASSERT during tcp_newtcpcb when low on memory
While testing with system default cc set to cubic, and
running a memory exhaustion validation, FreeBSD panics for a
missing inpcb reference / lock.

Reviewed by:	rgrimes (mentor), tuexen (mentor)
Approved by:	rgrimes (mentor), tuexen (mentor)
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D25583
2020-07-07 12:10:59 +00:00
Alexander V. Chernikov
6ad7446c6f Complete conversions from fib<4|6>_lookup_nh_<basic|ext> to fib<4|6>_lookup().
fib[46]_lookup_nh_ represents pre-epoch generation of fib api, providing less guarantees
 over pointer validness and requiring on-stack data copying.

With no callers remaining, remove fib[46]_lookup_nh_ functions.

Submitted by:	Neel Chauhan <neel AT neelc DOT org>
Differential Revision:	https://reviews.freebsd.org/D25445
2020-07-02 21:04:08 +00:00
Michael Tuexen
e54b7cd007 Fix the cleanup handling in a error path for TCP BBR.
Reported by:		syzbot+df7899c55c4cc52f5447@syzkaller.appspotmail.com
Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D25486
2020-07-01 17:17:06 +00:00
Mark Johnston
d16a2e4784 Fix a possible next-hop refcount leak when handling IPSec traffic.
It may be possible to fix this by deferring the lookup, but let's
keep the initial change simple to make MFCs easier.

PR:		246951
Reviewed by:	melifaro
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25519
2020-07-01 15:42:48 +00:00
Michael Tuexen
7a3f60e7f5 Fix a bug introduced in https://svnweb.freebsd.org/changeset/base/362173
Reported by:		syzbot+f3a6fccfa6ae9d3ded29@syzkaller.appspotmail.com
MFC after:		1 week
2020-06-30 21:50:05 +00:00
Michael Tuexen
e99ce3eac5 Don't send packets containing ERROR chunks in response to unknown
chunks when being in a state where the verification tag to be used
is not known yet.

MFC after:		1 week
2020-06-28 14:11:36 +00:00
Michael Tuexen
f2f66ef6d2 Don't check ch for not being NULL, since that is true.
MFC after:		1 week
2020-06-28 11:12:03 +00:00
John Baldwin
4a711b8d04 Use zfree() instead of explicit_bzero() and free().
In addition to reducing lines of code, this also ensures that the full
allocation is always zeroed avoiding possible bugs with incorrect
lengths passed to explicit_bzero().

Suggested by:	cem
Reviewed by:	cem, delphij
Approved by:	csprng (cem)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25435
2020-06-25 20:17:34 +00:00
Michael Tuexen
132c073866 Fix the acconting for fragmented unordered messages when using
interleaving.
This was reported for the userland stack in
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19321

MFC after:		1 week
2020-06-24 14:47:51 +00:00