Commit Graph

6137 Commits

Author SHA1 Message Date
Michael Tuexen
51e08d53ae Fix INET only builds.
r336940 introduced an "unused variable" warning on platforms which
support INET, but not INET6, like MALTA and MALTA64 as reported
by Mark Millard. Improve the #ifdefs to address this issue.

Sponsored by:		Netflix, Inc.
2018-07-31 06:27:05 +00:00
Michael Tuexen
888973f5ae Allow implicit TCP connection setup for TCP/IPv6.
TCP/IPv4 allows an implicit connection setup using sendto(), which
is used for TTCP and TCP fast open. This patch adds support for
TCP/IPv6.
While there, improve some tests for detecting multicast addresses,
which are mapped.

Reviewed by:		bz@, kbowling@, rrs@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16458
2018-07-30 21:27:26 +00:00
Michael Tuexen
e2662978b8 Send consistent SEG.WIN when using timewait codepath for TCP.
When sending TCP segments from the timewait code path, a stored
value of the last sent window is used. Use the same code for
computing this in the timewait code path as in the main code
path used in tcp_output() to avoiv inconsistencies.

Reviewed by:		rrs@
MFC after:		1 month
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16503
2018-07-30 21:13:42 +00:00
Michael Tuexen
8db239dc6b Fix some TCP fast open issues.
The following issues are fixed:
* Whenever a TCP server with TCP fast open enabled, calls accept(),
  recv(), send(), and close() before the TCP-ACK segment has been received,
  the TCP connection is just dropped and the reception of the TCP-ACK
  segment triggers the sending of a TCP-RST segment.
* Whenever a TCP server with TCP fast open enabled, calls accept(), recv(),
  send(), send(), and close() before the TCP-ACK segment has been received,
  the first byte provided in the second send call is not transferred.
* Whenever a TCP client with TCP fast open enabled calls sendto() followed
  by close() the TCP connection is just dropped.

Reviewed by:		jtl@, kbowling@, rrs@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16485
2018-07-30 20:35:50 +00:00
Michael Tuexen
6138da62a9 Add missing send/recv dtrace probes for TCP.
These missing probe are mostly in the syncache and timewait code.

Reviewed by:		markj@, rrs@
MFC after:		1 month
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16369
2018-07-30 20:13:38 +00:00
Alan Somers
6040822c4e Make timespecadd(3) and friends public
The timespecadd(3) family of macros were imported from NetBSD back in
r35029. However, they were initially guarded by #ifdef _KERNEL. In the
meantime, we have grown at least 28 syscalls that use timespecs in some
way, leading many programs both inside and outside of the base system to
redefine those macros. It's better just to make the definitions public.

Our kernel currently defines two-argument versions of timespecadd and
timespecsub.  NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define
three-argument versions.  Solaris also defines a three-argument version, but
only in its kernel.  This revision changes our definition to match the
common three-argument version.

Bump _FreeBSD_version due to the breaking KPI change.

Discussed with:	cem, jilles, ian, bde
Differential Revision:	https://reviews.freebsd.org/D14725
2018-07-30 15:46:40 +00:00
Randall Stewart
4ad5b7a0ac This fixes a hole where rack could end up
sending an invalid segment into the reassembly
queue. This would happen if you enabled the
data after close option.

Sponsored by:	Netflix
Differential Revision: https://reviews.freebsd.org/D16453
2018-07-30 10:23:29 +00:00
Andrew Turner
1e0582fd55 icmp_quotelen was accidentially changes in r336676, undo this.
Sponsored by:	DARPA, AFRL
2018-07-24 16:45:01 +00:00
Andrew Turner
5f901c92a8 Use the new VNET_DEFINE_STATIC macro when we are defining static VNET
variables.

Reviewed by:	bz
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D16147
2018-07-24 16:35:52 +00:00
Randall Stewart
399973c33d Delete the example tcp stack "fastpath" which
was only put in has an example.

Sponsored by:	Netflix inc.
Differential Revision:	https://reviews.freebsd.org/D16420
2018-07-24 14:55:47 +00:00
Matt Macy
e5e3e746fe Fix a potential use after free in getsockopt() access to inp_options
Discussed with: jhb
Reviewed by:	sbruno, transport
MFC after:	2 weeks
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D14621
2018-07-22 20:02:14 +00:00
Matt Macy
2269988749 NULL out cc_data in pluggable TCP {cc}_cb_destroy
When ABE was added (rS331214) to NewReno and leak fixed (rS333699) , it now has
a destructor (newreno_cb_destroy) for per connection state. Other congestion
controls may allocate and free cc_data on entry and exit, but the field is
never explicitly NULLed if moving back to NewReno which only internally
allocates stateful data (no entry contstructor) resulting in a situation where
newreno_cb_destory might be called on a junk pointer.

 -    NULL out cc_data in the framework after calling {cc}_cb_destroy
 -    free(9) checks for NULL so there is no need to perform not NULL checks
     before calling free.
 -    Improve a comment about NewReno in tcp_ccalgounload

This is the result of a debugging session from Jason Wolfe, Jason Eggleston,
and mmacy@ and very helpful insight from lstewart@.

Submitted by: Kevin Bowling
Reviewed by: lstewart
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D16282
2018-07-22 05:37:58 +00:00
Michael Tuexen
34fc9072ce Set the IPv4 version in the IP header for UDP and UDPLite. 2018-07-21 02:14:13 +00:00
Michael Tuexen
e1526d5a5b Add missing dtrace probes for received UDP packets.
Fire UDP receive probes when a packet is received and there is no
endpoint consuming it. Fire the probe also if the TTL of the
received packet is smaller than the minimum required by the endpoint.

Clarify also in the man page, when the probe fires.

Reviewed by:		dteske@, markj@, rrs@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16046
2018-07-20 15:32:20 +00:00
Michael Tuexen
0053ed28ff Whitespace changes due to changes in ident. 2018-07-19 20:16:33 +00:00
Michael Tuexen
b0471b4b95 Revert https://svnweb.freebsd.org/changeset/base/336503
since I also ran the export script with different parameters.
2018-07-19 20:11:14 +00:00
Michael Tuexen
7679e49dd4 Whitespace changes due to change if ident. 2018-07-19 19:33:42 +00:00
Randall Stewart
8de9ac5eec Bump the ICMP echo limits to match the RFC
Reviewed by:	tuexen
Sponsored by: Netflix Inc.
Differential Revision:		https://reviews.freebsd.org/D16333
2018-07-18 22:49:53 +00:00
Andrey V. Elsukov
acf673edf0 Move invoking of callout_stop(&lle->lle_timer) into llentry_free().
This deduplicates the code a bit, and also implicitly adds missing
callout_stop() to in[6]_lltable_delete_entry() functions.

PR:		209682, 225927
Submitted by:	hselasky (previous version)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D4605
2018-07-17 11:33:23 +00:00
Sean Bruno
c8b1bdc31c There was quite a bit of feedback on r336282 that has led to the
submitter to want to revert it.
2018-07-14 23:53:51 +00:00
Sean Bruno
179a28b098 Fixup memory management for fetching options in ip_ctloutput()
Submitted by:	Jason Eggleston <jason@eggnet.com>
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D14621
2018-07-14 16:19:46 +00:00
Mark Johnston
aaf268f9f6 Remove a duplicate check.
PR:		229663
Submitted by:	David Binderman <dcb314@hotmail.com>
MFC after:	3 days
2018-07-11 14:54:56 +00:00
Brooks Davis
3a20f06a1c Use uintptr_t alone when assigning to kvaddr_t variables.
Suggested by:	jhb
2018-07-10 13:03:06 +00:00
Michael Tuexen
c9da58534d Add support for printing the TCP FO client-side cookie cache via the
sysctl interface. This is similar to the TCP host cache.

Reviewed by:		pkelsey@, kbowling@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D14554
2018-07-10 10:50:43 +00:00
Michael Tuexen
a026a53a76 Use appropriate MSS value when populating the TCP FO client cookie cache
When a client receives a SYN-ACK segment with a TFP fast open cookie,
but without an MSS option, an MSS value from uninitialised stack memory is used.
This patch ensures that in case no MSS option is included in the SYN-ACK,
the appropriate value as given in RFC 7413 is used.

Reviewed by:		kbowling@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16175
2018-07-10 10:42:48 +00:00
Steven Hartland
65c3a353e6 Removed pointless NULL check
Removed pointless NULL check after malloc with M_WAITOK which can never
return NULL.

Sponsored by:	Multiplay
2018-07-10 08:05:32 +00:00
Andrey V. Elsukov
f7c4fdee1a Add "record-state", "set-limit" and "defer-action" rule options to ipfw.
"record-state" is similar to "keep-state", but it doesn't produce implicit
O_PROBE_STATE opcode in a rule. "set-limit" is like "limit", but it has the
same feature as "record-state", it is single opcode without implicit
O_PROBE_STATE opcode. "defer-action" is targeted to be used with dynamic
states. When rule with this opcode is matched, the rule's action will
not be executed, instead dynamic state will be created. And when this
state will be matched by "check-state", then rule action will be executed.
This allows create a more complicated rulesets.

Submitted by:	lev
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D1776
2018-07-09 11:35:18 +00:00
Michael Tuexen
5f1347d7c9 Allow alternate TCP stack to populate the TCP FO client cookie
cache.

Without this patch, TCP FO could be used when using alternate
TCP stack, but only existing entires in the TCP client cookie
cache could be used. This cache was not populated by connections
using alternate TCP stacks.

Sponsored by:		Netflix, Inc.
2018-07-07 12:28:16 +00:00
Michael Tuexen
c556884f8e When initializing the TCP FO client cookie cache, take into account
whether the TCP FO support is enabled or not for the client side.

The code in tcp_fastopen_init() implicitly assumed that the sysctl
variable V_tcp_fastopen_client_enable was initialized to 0. This
was initially true, but was changed in r335610, which unmasked this
bug.

Thanks to Pieter de Goeje for reporting the issue on freebsd-net@
2018-07-07 11:18:26 +00:00
Brooks Davis
5c5e39e3d5 One more 32-bit fix for r335979.
Reported by:	tuexen
2018-07-06 13:34:45 +00:00
Brooks Davis
7524b4c14b Correct breakage on 32-bit platforms from r335979. 2018-07-06 10:03:33 +00:00
Andrew Turner
2bf9501287 Create a new macro for static DPCPU data.
On arm64 (and possible other architectures) we are unable to use static
DPCPU data in kernel modules. This is because the compiler will generate
PC-relative accesses, however the runtime-linker expects to be able to
relocate these.

In preparation to fix this create two macros depending on if the data is
global or static.

Reviewed by:	bz, emaste, markj
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D16140
2018-07-05 17:13:37 +00:00
Brooks Davis
f38b68ae8a Make struct xinpcb and friends word-size independent.
Replace size_t members with ksize_t (uint64_t) and pointer members
(never used as pointers in userspace, but instead as unique
idenitifiers) with kvaddr_t (uint64_t). This makes the structs
identical between 32-bit and 64-bit ABIs.

On 64-bit bit systems, the ABI is maintained. On 32-bit systems,
this is an ABI breaking change. The ABI of most of these structs
was previously broken in r315662.  This also imposes a small API
change on userspace consumers who must handle kernel pointers
becoming virtual addresses.

PR:		228301 (exp-run by antoine)
Reviewed by:	jtl, kib, rwatson (various versions)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15386
2018-07-05 13:13:48 +00:00
Hiroki Sato
5ba05d3d0e - Fix a double unlock in inp_block_unblock_source() and
lock leakage in inp_leave_group() which caused a panic.
- Make order of CTR1() and IN_MULTI_LIST_LOCK() consistent
  around inm_merge().
2018-07-04 06:47:34 +00:00
Matt Macy
6573d7580b epoch(9): allow preemptible epochs to compose
- Add tracker argument to preemptible epochs
- Inline epoch read path in kernel and tied modules
- Change in_epoch to take an epoch as argument
- Simplify tfb_tcp_do_segment to not take a ti_locked argument,
  there's no longer any benefit to dropping the pcbinfo lock
  and trying to do so just adds an error prone branchfest to
  these functions
- Remove cases of same function recursion on the epoch as
  recursing is no longer free.
- Remove the the TAILQ_ENTRY and epoch_section from struct
  thread as the tracker field is now stack or heap allocated
  as appropriate.

Tested by: pho and Limelight Networks
Reviewed by: kbowling at llnw dot com
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D16066
2018-07-04 02:47:16 +00:00
Matt Macy
99208b820f inpcb: don't gratuitously defer frees
Don't defer frees in sysctl handlers. It isn't necessary
and it just confuses things.
revert: r333911, r334104, and r334125

Requested by: jtl
2018-07-02 05:19:44 +00:00
Kristof Provost
0d3d234cd1 carp: Set DSCP value CS7
Update carp to set DSCP value CS7(Network Traffic) in the flowlabel field of
packets by default. Currently carp only sets TOS_LOWDELAY in IPv4 which was
deprecated in 1998. This also implements sysctl that can revert carp back to
it's old behavior if desired.

This will allow implementation of QOS on modern network devices to make sure
carp packets aren't dropped during interface contention.

Submitted by:	Nick Wolff <darkfiberiru AT gmail.com>
Reviewed by:	kp, mav (earlier version)
Differential Revision:	https://reviews.freebsd.org/D14536
2018-07-01 08:37:07 +00:00
Andrey V. Elsukov
6e081509db Add NULL pointer check.
encap_lookup_t method can be invoked by IP encap subsytem even if none
of gif/gre/me interfaces are exist. Hash tables are allocated on demand,
when first interface is created. So, make NULL pointer check before
doing access to hash table.

PR:		229378
2018-06-28 11:39:27 +00:00
Gleb Smirnoff
b8ab659396 Check the inp_flags under inp lock. Looks like the race was hidden
before, the conversion of tcbinfo to CK_LIST have uncovered it.
2018-06-27 22:01:59 +00:00
Sean Bruno
af4da58655 Enable TCP_FASTOPEN by default for FreeBSD 12.
Submitted by:	kbowling
Reviewed by:	tuexen
Differential Revision:	https://reviews.freebsd.org/D15959
2018-06-24 21:46:29 +00:00
Sean Bruno
45fc0718d8 Reap unused variable and assignment that had no effect. Noted by cross
compiling with gcc on mips.

Reviewed by:	mmacy
2018-06-24 21:36:37 +00:00
Gleb Smirnoff
a00f4ac22f Revert r334843, and partially revert r335180.
tcp_outflags[] were defined since 4BSD and are defined nowadays in
all its descendants. Removing them breaks third party application.
2018-06-23 06:53:53 +00:00
Randall Stewart
581a046a8b This adds in an optimization so that we only walk one
time through the mbuf chain during copy and TSO limiting.
It is used by both Rack and now the FreeBSD stack.
Sponsored by:	Netflix Inc
Differential Revision: https://reviews.freebsd.org/D15937
2018-06-21 21:03:58 +00:00
Matt Macy
e93fdbe212 raw_ip: validate inp in both loops
Continuation of r335497. Also move the lock acquisition up to
validate before referencing inp_cred.

Reported by:	pho
2018-06-21 20:18:23 +00:00
Matt Macy
3d348772e7 in_pcblookup_hash: validate inp before return
Post r335356 it is possible to have an inpcb on the hash lists that is
partially torn down. Validate before using. Also as a side effect of this
change the lock ordering issue between hash lock and inpcb no longer exists
allowing some simplification.

Reported by:	pho@
2018-06-21 18:40:15 +00:00
Matt Macy
e5c331cf78 raw_ip: validate inp
Post r335356 it is possible to have an inpcb on the hash lists that is
partially torn down. Validate before using.

Reported by:	pho
2018-06-21 17:24:10 +00:00
Matt Macy
46374cbf54 udp_ctlinput: don't refer to unpcb after we drop the lock
Reported by: pho@
2018-06-21 06:10:52 +00:00
Randall Stewart
c6f76759ca Make sure that the t_peakrate_thr is not compiled in
by default until NF can upstream it.

Reviewed by:	and suggested lstewart
Sponsored by:	Netflix Inc.
2018-06-19 11:20:28 +00:00
Randall Stewart
f923a734b3 Move the tp set back to where it was before
we started playing with the VNET sets. This
way we have verified the INP settings before
we go to the trouble of de-referencing it.

Reviewed by:	and suggested by lstewart
Sponsored by:	Netflix Inc.
2018-06-19 05:28:14 +00:00
Matt Macy
9e58ff6ff9 convert inpcbinfo hash and info rwlocks to epoch + mutex
- Convert inpcbinfo info & hash locks to epoch for read and mutex for write
- Garbage collect code that handled INP_INFO_TRY_RLOCK failures as
  INP_INFO_RLOCK which can no longer fail

When running 64 netperfs sending minimal sized packets on a 2x8x2 reduces
unhalted core cycles samples in rwlock rlock/runlock in udp_send from 51% to
3%.

Overall packet throughput rate limited by CPU affinity and NIC driver design
choices.

On the receiver unhalted core cycles samples in in_pcblookup_hash went from
13% to to 1.6%

Tested by LLNW and pho@

Reviewed by: jtl
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15686
2018-06-19 01:54:00 +00:00
Randall Stewart
f994ead330 Move to using the inp->vnet pointer has suggested by lstewart.
This is far better since the hpts system is using the inp
as its basis anyway. Unfortunately his comments came late.

Sponsored by:	Netflix Inc.
2018-06-18 14:10:12 +00:00
Andrey V. Elsukov
20efcfc602 Switch RIB and RADIX_NODE_HEAD lock from rwlock(9) to rmlock(9).
Using of rwlock with multiqueue NICs for IP forwarding on high pps
produces high lock contention and inefficient. Rmlock fits better for
such workloads.

Reviewed by:	melifaro, olivier
Obtained from:	Yandex LLC
Sponsored by:	Yandex LLC
Differential Revision:	https://reviews.freebsd.org/D15789
2018-06-16 08:26:23 +00:00
Michael Tuexen
43b223f42e When retransmitting TCP SYN-ACK segments with the TCP timestamp option
enabled use an updated timestamp instead of reusing the one used in
the initial TCP SYN-ACK segment.

This patch ensures that an updated timestamp is used when sending the
SYN-ACK from the syncache code. It was already done if the
SYN-ACK was retransmitted from the generic code.

This makes the behaviour consistent and also conformant with
the TCP specification.

Reviewed by:		jtl@, Jason Eggleston
MFC after:		1 month
Sponsored by:		Neflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D15634
2018-06-15 12:28:43 +00:00
Gleb Smirnoff
9293873e83 TCPOUTFLAGS no longer exists since r334843. 2018-06-14 22:25:10 +00:00
Michael Tuexen
33ef123090 Provide the ip6_plen in network byte order when calling ip6_output().
This is not strictly required by ip6_output(), since it overrides it,
but it is needed for upcoming dtrace support.
2018-06-14 21:30:52 +00:00
Michael Tuexen
8d86bd564f Whitespace changes. 2018-06-14 21:22:14 +00:00
Andrey V. Elsukov
eb548a1a5c In m_megapullup() use m_getjcl() to allocate 9k or 16k mbuf when requested.
It is better to try allocate a big mbuf, than just silently drop a big
packet. A better solution could be reworking of libalias modules to be
able use m_copydata()/m_copyback() instead of requiring the single
contiguous buffer.

PR:		229006
MFC after:	1 week
2018-06-14 11:15:39 +00:00
Randall Stewart
4aec110f70 This fixes several bugs that Larry Rosenman helped me find in
Rack with respect to its handling of TCP Fast Open. Several
fixes all related to TFO are included in this commit:
1) Handling of non-TFO retransmissions
2) Building the proper send-map when we are doing TFO
3) Dealing with the ack that comes back that includes the
   SYN and data.

It appears that with this commit TFO now works :-)

Thanks Larry for all your help!!

Sponsored by:	Netflix Inc.
Differential Revision:	https://reviews.freebsd.org/D15758
2018-06-14 03:27:42 +00:00
Matt Macy
feeef8509b Fix PCBGROUPS build post CK conversion of pcbinfo 2018-06-13 23:19:54 +00:00
Andrey V. Elsukov
a5185adeb6 Rework if_gre(4) to use encap_lookup_t method to speedup lookup
of needed interface when many gre interfaces are present.

Remove rmlock from gre_softc, use epoch(9) and CK_LIST instead.
Move more AF-related code into AF-related locations. Use hash table to
speedup lookup of needed softc.
2018-06-13 11:11:33 +00:00
Matt Macy
483305b99c Handle INP_FREED when looking up an inpcb
When hash table lookups are not serialized with in_pcbfree it will be
possible for callers to find an inpcb that has been marked free. We
need to check for this and return NULL.
2018-06-13 04:23:49 +00:00
Randall Stewart
c9b4ac7587 This fixes missing VNET sets in the hpts system. Basically
without this and running vnets with a TCP stack that uses
some of the features is a recipe for panic (without this commit).

Reported by:	Larry Rosenman
Sponsored by:	Netflix Inc.
Differential Revision:	https://reviews.freebsd.org/D15757
2018-06-12 23:54:08 +00:00
Matt Macy
700e893c34 Defer inpcbport free in in_pcbremlists as well 2018-06-12 23:26:25 +00:00
Matt Macy
f09ee4fc01 Defer inpcbport free until after a grace period has elapsed
This is a dependency for inpcbinfo rlock conversion to epoch
2018-06-12 22:18:27 +00:00
Matt Macy
b872626dbe mechanical CK macro conversion of inpcbinfo lists
This is a dependency for converting the inpcbinfo hash and info rlocks
to epoch.
2018-06-12 22:18:20 +00:00
Matt Macy
addf2b2009 Defer inpcb deletion until after a grace period has elapsed
Deferring the actual free of the inpcb until after a grace
period has elapsed will allow us to convert the inpcbinfo
info and hash read locks to epoch.

Reviewed by: gallatin, jtl
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15510
2018-06-12 22:18:15 +00:00
Jonathan T. Looney
cff21e484b Change RACK dependency on TCPHPTS from a build-time dependency to a load-
time dependency.

At present, RACK requires the TCPHPTS option to run. However, because
modules can be moved from machine to machine, this dependency is really
best assessed at load time rather than at build time.

Reviewed by:	rrs
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D15756
2018-06-11 14:27:19 +00:00
Matt Macy
3db28e6656 avoid 'tcp_outflags defined but not used' 2018-06-08 17:37:49 +00:00
Matt Macy
afbd6cfa72 hpts: remove redundant decl breaking gcc build 2018-06-08 17:37:43 +00:00
Randall Stewart
89e560f441 This commit brings in a new refactored TCP stack called Rack.
Rack includes the following features:
 - A different SACK processing scheme (the old sack structures are not used).
 - RACK (Recent acknowledgment) where counting dup-acks is no longer done
        instead time is used to knwo when to retransmit. (see the I-D)
 - TLP (Tail Loss Probe) where we will probe for tail-losses to attempt
        to try not to take a retransmit time-out. (see the I-D)
 - Burst mitigation using TCPHTPS
 - PRR (partial rate reduction) see the RFC.

Once built into your kernel, you can select this stack by either
socket option with the name of the stack is "rack" or by setting
the global sysctl so the default is rack.

Note that any connection that does not support SACK will be kicked
back to the "default" base  FreeBSD stack (currently known as "default").

To build this into your kernel you will need to enable in your
kernel:
   makeoptions WITH_EXTRA_TCP_STACKS=1
   options TCPHPTS

Sponsored by:	Netflix Inc.
Differential Revision:		https://reviews.freebsd.org/D15525
2018-06-07 18:18:13 +00:00
Michael Tuexen
ff34bbe9c2 Improve compliance with RFC 4895 and RFC 6458.
Silently dicard SCTP chunks which have been requested to be
authenticated but are received unauthenticated no matter if support
for SCTP authentication has been negotiated. This improves compliance
with RFC 4895.

When the application uses the SCTP_AUTH_CHUNK socket option to
request a chunk to be received in an authenticated way, enable
the SCTP authentication extension for the end-point. This improves
compliance with RFC 6458.

Discussed with:	Peter Lei
MFC after:	3 days
2018-06-06 19:27:06 +00:00
Sean Bruno
1a43cff92a Load balance sockets with new SO_REUSEPORT_LB option.
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).

This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967.  Thanks to rwatson@
for the substantive feedback that is included in this commit.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Obtained from:	DragonflyBSD
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-06-06 15:45:57 +00:00
Andrey V. Elsukov
590d0a43b6 Make in_delayed_cksum() be similar to IPv6 implementation.
Use m_copyback() function to write checksum when it isn't located
in the first mbuf of the chain. Handmade analog doesn't handle the
case when parts of checksum are located in different mbufs.
Also in case when mbuf is too short, m_copyback() will allocate new
mbuf in the chain instead of making out of bounds write.

Also wrap long line and remove now useless KASSERTs.

X-MFC after:	r334705
2018-06-06 13:01:53 +00:00
Tom Jones
1fdbfb909f Use UDP len when calculating UDP checksums
The length of the IP payload is normally equal to the UDP length, UDP Options
(draft-ietf-tsvwg-udp-options-02) suggests using the difference between IP
length and UDP length to create space for trailing data.

Correct checksum length calculation to use the UDP length rather than the IP
length when not offloading UDP checksums.

Approved by: jtl (mentor)
Differential Revision:	https://reviews.freebsd.org/D15222
2018-06-06 07:04:40 +00:00
Andrey V. Elsukov
b941bc1d6e Rework if_gif(4) to use new encap_lookup_t method to speedup lookup
of needed interface when many gif interfaces are present.

Remove rmlock from gif_softc, use epoch(9) and CK_LIST instead.
Move more AF-related code into AF-related locations.
Use hash table to speedup lookup of needed softc. Interfaces
with GIF_IGNORE_SOURCE flag are stored in plain CK_LIST.
Sysctl net.link.gif.parallel_tunnels is removed. The removal was planed
16 years ago, and actually it could work only for outbound direction.
Each protocol, that can be handled by if_gif(4) interface is registered
by separate encap handler, this helps avoid invoking the handler
for unrelated protocols (GRE, PIM, etc.).

This change allows dramatically improve performance when many gif(4)
interfaces are used.

Sponsored by:	Yandex LLC
2018-06-05 21:24:59 +00:00
Andrey V. Elsukov
6d8fdfa9d5 Rework IP encapsulation handling code.
Currently it has several disadvantages:
- it uses single mutex to protect internal structures. It is used by
  data- and control- path, thus there are no parallelism at all.
- it uses single list to keep encap handlers for both INET and INET6
  families.
- struct encaptab keeps unneeded information (src, dst, masks, protosw),
  that isn't used by code in the source tree.
- matches are prioritized and when many tunneling interfaces are
  registered, encapcheck handler of each interface is invoked for each
  packet. The search takes O(n) for n interfaces. All this work is done
  with exclusive lock held.

What this patch includes:
- the datapath is converted to be lockless using epoch(9) KPI.
- struct encaptab now linked using CK_LIST.
- all unused fields removed from struct encaptab. Several new fields
  addedr: min_length is the minimum packet length, that encapsulation
  handler expects to see; exact_match is maximum number of bits, that
  can return an encapsulation handler, when it wants to consume a packet.
- IPv6 and IPv4 handlers are stored in separate lists;
- added new "encap_lookup_t" method, that will be used later. It is
  targeted to speedup lookup of needed interface, when gif(4)/gre(4) have
  many interfaces.
- the need to use protosw structure is eliminated. The only pr_input
  method was used from this structure, so I don't see the need to keep
  using it.
- encap_input_t method changed to avoid using mbuf tags to store softc
  pointer. Now it is passed directly trough encap_input_t method.
  encap_getarg() funtions is removed.
- all sockaddr structures and code that uses them removed. We don't have
  any code in the tree that uses them. All consumers use encap_attach_func()
  method, that relies on invoking of encapcheck() to determine the needed
  handler.
- introduced struct encap_config, it contains parameters of encap handler
  that is going to be registered by encap_attach() function.
- encap handlers are stored in lists ordered by exact_match value, thus
  handlers that need more bits to match will be checked first, and if
  encapcheck method returns exact_match value, the search will be stopped.
- all current consumers changed to use new KPI.

Reviewed by:	mmacy
Sponsored by:	Yandex LLC
Differential Revision:	https://reviews.freebsd.org/D15617
2018-06-05 20:51:01 +00:00
Mateusz Guzik
34c538c356 malloc: try to use builtins for zeroing at the callsite
Plenty of allocation sites pass M_ZERO and sizes which are small and known
at compilation time. Handling them internally in malloc loses this information
and results in avoidable calls to memset.

Instead, let the compiler take the advantage of it whenever possible.

Discussed with:	jeff
2018-06-02 22:20:09 +00:00
Michael Tuexen
13500cbb61 Don't overflow a buffer if we receive an INIT or INIT-ACK chunk
without a RANDOM parameter but with a CHUNKS or HMAC-ALGO parameter.
Please note that sending this combination violates the specification.

Thnanks to Ronald E. Crane for reporting the issue for the userland
stack.

MFC after:	3 days
2018-06-02 16:28:10 +00:00
Michael Tuexen
c14f9fe5ef Limit the retransmission timer for SYN-ACKs by TCPTV_REXMTMAX.
Use the same logic to handle the SYN-ACK retransmission when sent from
the syn cache code as when sent from the main code.

MFC after:	3 days
Sponsored by:	Netflix, Inc.
2018-06-01 21:24:27 +00:00
Michael Tuexen
badef00d58 Ensure net.inet.tcp.syncache.rexmtlimit is limited by TCP_MAXRXTSHIFT.
If the sysctl variable is set to a value larger than TCP_MAXRXTSHIFT+1,
the array tcp_syn_backoff[] is accessed out of bounds.

Discussed with: jtl@
MFC after:	3 days
Sponsored by:	Netflix, Inc.
2018-06-01 19:58:19 +00:00
Andrey V. Elsukov
12e7376216 Remove empty encap_init() function.
MFC after:	2 weeks
2018-05-29 12:32:08 +00:00
Michael Tuexen
eef8d4a973 Use correct mask.
Introduced in https://svnweb.freebsd.org/changeset/base/333603.
Thanks to Irene Ruengler for testing and reporting the issue.

MFC after:		1 week
X-MFC-with:		333603
2018-05-28 13:31:47 +00:00
Matt Macy
62d733a11d in_pcbladdr: remove debug code that snuck in with ifa epoch conversion r334118 2018-05-27 06:47:09 +00:00
Matt Macy
0f8d79d977 CK: update consumers to use CK macros across the board
r334189 changed the fields to have names distinct from those in queue.h
in order to expose the oversights as compile time errors
2018-05-24 23:21:23 +00:00
Matt Macy
fe524329e4 convert allocations to INVARIANTS M_ZERO 2018-05-24 01:04:56 +00:00
Matt Macy
4f6c66cc9c UDP: further performance improvements on tx
Cumulative throughput while running 64
  netperf -H $DUT -t UDP_STREAM -- -m 1
on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps

Single stream throughput increases from 910kpps to 1.18Mpps

Baseline:
https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg

- Protect read access to global ifnet list with epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg

- Protect short lived ifaddr references with epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg

- Convert if_afdata read lock path to epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg

A fix for the inpcbhash contention is pending sufficient time
on a canary at LLNW.

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
Matt Macy
630ba2c514 udp: assign flowid to udp sockets round-robin
On a 2x8x2 SKL this increases measured throughput with 64
netperf -H $DUT -t UDP_STREAM -- -m 1

from 590kpps to 1.1Mpps
before:
https://people.freebsd.org/~mmacy/2018.05.11/udpsender.svg
after:
https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg
2018-05-23 20:50:09 +00:00
Matt Macy
246a619924 epoch: allow for conditionally asserting that the epoch context fields
are unused by zeroing on INVARIANTS builds
2018-05-23 17:00:05 +00:00
Mark Johnston
9f78e2b83d Initialize the dumper struct before calling set_dumper().
Fields owned by the generic code were being left uninitialized,
causing problems in clear_dumper() if an error occurred.

Coverity CID:	1391200
X-MFC with:	r333283
2018-05-22 16:01:56 +00:00
Matt Macy
f42a83f2a6 inpcb: revert deferred inpcb free pending further review 2018-05-21 16:13:43 +00:00
Michael Tuexen
c692df45fc Only fillin data srucuture when actually stored. 2018-05-21 14:53:22 +00:00
Michael Tuexen
d3132db2b5 Do the appropriate accounting when ip_output() fails. 2018-05-21 14:52:18 +00:00
Michael Tuexen
95844fce7d Make clear why there is an assignment, which is not necessary. 2018-05-21 14:51:20 +00:00
Ed Maste
3b9b6b1704 Pair CURVNET_SET and CURVNET_RESTORE in a block
Per vnet(9), CURVNET_SET and CURVNET_RESTORE cannot be used as a single
statement for a conditional and CURVNET_RESTORE must be in the same
block as CURVNET_SET (or a subblock).

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
2018-05-21 13:08:44 +00:00
Ed Maste
15f8acc53f Revert r333968, it broke all archs but i386 and amd64 2018-05-21 11:56:07 +00:00
Matt Macy
ed6bb714b2 in(6)_mcast: Expand out vnet set / restore macro so that they work in a conditional block
Reported by:	zec at fer.hr
2018-05-21 08:34:10 +00:00
Matt Macy
06b15160e1 ensure that vnet is set when doing in_leavegroup 2018-05-21 07:12:06 +00:00
Matt Macy
1a3d880c26 in(s)_moptions: free before tearing down inpcb 2018-05-20 20:08:21 +00:00
Matt Macy
47d2a58560 inpcb: defer destruction of inpcb until after a grace period has elapsed
in_pcbfree will remove the incpb from the list and release the rtentry
while the vnet is set, but the actual destruction will be deferred
until any threads in a (not yet used) epoch section, no longer potentially
have references.
2018-05-20 04:38:04 +00:00
Matt Macy
056b40e29c inpcb: consolidate possible deletion in pcblist functions in to epoch
deferred context.
2018-05-20 02:27:58 +00:00
Matt Macy
ddece765f3 in_pcb: add helper for deferring inpcb rele calls from list functions 2018-05-20 02:17:30 +00:00
Matt Macy
cb6bb2303e ip(6)_freemoptions: defer imo destruction to epoch callback task
Avoid the ugly unlock / lock of the inpcbinfo where we need to
figure out what kind of lock we hold by simply deferring the
operation to another context. (Also a small dependency for
converting the pcbinfo read lock to epoch)
2018-05-20 00:22:28 +00:00
Matt Macy
f6960e207e netinet silence warnings 2018-05-19 05:56:21 +00:00
Matt Macy
7a3c5b05e4 tcp sysctl fix may be uninitialized 2018-05-19 05:55:31 +00:00
Matt Macy
21f6fe2d7c tcp fastopen: fix may be uninitialized 2018-05-19 05:55:00 +00:00
Matt Macy
d7c5a620e2 ifnet: Replace if_addr_lock rwlock with epoch + mutex
Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
4.98   0.00   4.42   0.00 4235592     33   83.80 4720653 2149771   1235 247.32
4.73   0.00   4.20   0.00 4025260     33   82.99 4724900 2139833   1204 247.32
4.72   0.00   4.20   0.00 4035252     33   82.14 4719162 2132023   1264 247.32
4.71   0.00   4.21   0.00 4073206     33   83.68 4744973 2123317   1347 247.32
4.72   0.00   4.21   0.00 4061118     33   80.82 4713615 2188091   1490 247.32
4.72   0.00   4.21   0.00 4051675     33   85.29 4727399 2109011   1205 247.32
4.73   0.00   4.21   0.00 4039056     33   84.65 4724735 2102603   1053 247.32

After the patch

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
5.43   0.00   4.20   0.00 3313143     33   84.96 5434214 1900162   2656 245.51
5.43   0.00   4.20   0.00 3308527     33   85.24 5439695 1809382   2521 245.51
5.42   0.00   4.19   0.00 3316778     33   87.54 5416028 1805835   2256 245.51
5.42   0.00   4.19   0.00 3317673     33   90.44 5426044 1763056   2332 245.51
5.42   0.00   4.19   0.00 3314839     33   88.11 5435732 1792218   2499 245.52
5.44   0.00   4.19   0.00 3293228     33   91.84 5426301 1668597   2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
Mark Johnston
b35822d9d0 Fix netdump configuration when VIMAGE is enabled.
We need to set the current vnet before iterating over the global
interface list. Because the dump device may only be set from the host,
only proceed with configuration if the thread belongs to the default
vnet. [1]

Also fix a resource leak that occurs if the priv_check() in set_dumper()
fails.

Reported by:	mmacy, sbruno [1]
Reviewed by:	sbruno
X-MFC with:	r333283
Differential Revision:	https://reviews.freebsd.org/D15449
2018-05-17 04:08:57 +00:00
Lawrence Stewart
9891578a40 Plug a memory leak and potential NULL-pointer dereference introduced in r331214.
Each TCP connection that uses the system default cc_newreno(4) congestion
control algorithm module leaks a "struct newreno" (8 bytes of memory) at
connection initialisation time. The NULL-pointer dereference is only germane
when using the ABE feature, which is disabled by default.

While at it:

- Defer the allocation of memory until it is actually needed given that ABE is
  optional and disabled by default.

- Document the ENOMEM errno in getsockopt(2)/setsockopt(2).

- Document ENOMEM and ENOBUFS in tcp(4) as being synonymous given that they are
  used interchangeably throughout the code.

- Fix a few other nits also accidentally omitted from the original patch.

Reported by:	Harsh Jain on freebsd-net@
Tested by:	tjh@
Differential Revision:	https://reviews.freebsd.org/D15358
2018-05-17 02:46:27 +00:00
Brooks Davis
514ef08caf Unwrap a line that no longer requires wrapping. 2018-05-15 20:14:38 +00:00
Brooks Davis
423349f696 Remove stray tabs from in_lltable_dump_entry(). 2018-05-15 20:13:00 +00:00
Stephen Hurd
f2cf90e264 Check that ifma_protospec != NULL in inm_lookup
If ifma_protospec is NULL when inm_lookup() is called, there
is a dereference in a NULL struct pointer. This ensures that struct is
not NULL before comparing the address.

Reported by:	dumbbell
Reviewed by:	sbruno
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15440
2018-05-15 16:54:41 +00:00
Michael Tuexen
482d420926 sctp_get_mbuf_for_msg() should honor the allinone parameter.
When it is not required that the buffer is not a chain, return
a chain. This is based on a patch provided by Irene Ruengeler.
2018-05-14 15:16:51 +00:00
Michael Tuexen
589c42c2c8 Ensure that the MTU's used are multiple of 4.
The length of SCTP packets is always a multiple of 4. Therefore,
ensure that the MTUs used are also a multiple of 4.

Thanks to Irene Ruengeler for providing an earlier version of this
patch.

MFC after:	1 week
2018-05-14 13:50:17 +00:00
Stephen Hurd
b69888c28f Fix LORs in in6?_leave_group()
r333175 updated the join_group functions, but not the leave_group ones.

Reviewed by:	sbruno
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15393
2018-05-11 21:42:27 +00:00
Warner Losh
33123867af Use the full year, for real this time. 2018-05-09 20:26:37 +00:00
Warner Losh
603bbd0631 Minor style nits
Use full copyright year.
Remove 'All Rights Reserved' from new file (rights holder OK'd)
Minor #ifdef motion and #endif tagging
Remove __FBSDID macro from comments

Sponsored by: Netflix
OK'd by: rrs@
2018-05-09 14:11:35 +00:00
Michael Tuexen
45d41de5e6 Fix two typos reported by N. J. Mann, which were introduced in
https://svnweb.freebsd.org/changeset/base/333382 by me.

MFC after:	3 days
2018-05-08 20:39:35 +00:00
Michael Tuexen
9669e724d1 When reporting ERROR or ABORT chunks, don't use more data
that is guaranteed to be contigous.
Thanks to Felix Weinrank for finding and reporting this bug
by fuzzing the usrsctp stack.

MFC after:	3 days
2018-05-08 18:48:51 +00:00
Matt Macy
10d20c84ed Fix spurious retransmit recovery on low latency networks
TCP's smoothed RTT (SRTT) can be much larger than an actual observed RTT. This can be either because of hz restricting the calculable RTT to 10ms in VMs or 1ms using the default 1000hz or simply because SRTT recently incorporated a larger value.

If an ACK arrives before the calculated badrxtwin (now + SRTT):
tp->t_badrxtwin = ticks + (tp->t_srtt >> (TCP_RTT_SHIFT + 1));

We'll erroneously reset snd_una to snd_max. If multiple segments were dropped and this happens repeatedly the transmit rate will be limited to 1MSS per RTO until we've retransmitted all drops.

Reported by:	rstone
Reviewed by:	hiren, transport
Approved by:	sbruno
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D8556
2018-05-08 02:22:34 +00:00
Alexander Motin
167a34407c Keep CARP state as INIT when net.inet.carp.allow=0.
Currently when net.inet.carp.allow=0 CARP state remains as MASTER, which is
not very useful (if there are other masters -- it can lead to split brain,
if there are none -- it makes no sense).  Having it as INIT makes it clear
that carp packets are disabled.

Submitted by:	wg
MFC after:	1 month
Relnotes:	yes
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14477
2018-05-07 14:44:55 +00:00
Matt Macy
b6f6f88018 r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl
to sleep on commands to the NIC when updating multicast filters. More generally this permitted
driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a
a multicast update would still be queued for deletion when ifconfig deleted the interface
thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses
on the interface.

Synchronously remove all external references to a multicast address before enqueueing for delete.

Reported by:	lwhsu
Approved by:	sbruno
2018-05-06 20:34:13 +00:00
Michael Tuexen
67e8b08bbe Ensure we are not dereferencing a NULL pointer.
This was found by Coverity scanning the usrsctp stack (CID 203808).

MFC after:	3 days
2018-05-06 14:19:50 +00:00
Mark Johnston
e505460228 Import the netdump client code.
This is a component of a system which lets the kernel dump core to
a remote host after a panic, rather than to a local storage device.
The server component is available in the ports tree. netdump is
particularly useful on diskless systems.

The netdump(4) man page contains some details describing the protocol.
Support for configuring netdump will be added to dumpon(8) in a future
commit. To use netdump, the kernel must have been compiled with the
NETDUMP option.

The initial revision of netdump was written by Darrell Anderson and
was integrated into Sandvine's OS, from which this version was derived.

Reviewed by:	bdrewery, cem (earlier versions), julian, sbruno
MFC after:	1 month
X-MFC note:	use a spare field in struct ifnet
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D15253
2018-05-06 00:38:29 +00:00
Matt Macy
28c001002a Currently in_pcbfree will unconditionally wunlock the pcbinfo lock
to avoid a LOR on the multicast list lock in the freemoptions routines.
As it turns out, tcp_usr_detach can acquire the tcbinfo lock readonly.
Trying to wunlock the pcbinfo lock in that context has caused a number
of reported crashes.

This change unclutters in_pcbfree and moves the handling of wunlock vs
runlock of pcbinfo to the freemoptions routine.

Reported by:	mjg@, bde@, o.hartmann at walstatt.org
Approved by:	sbruno
2018-05-05 22:40:40 +00:00
Andrey V. Elsukov
5ada542398 Immediately propagate EACCES error code to application from tcp_output.
In r309610 and r315514 the behavior of handling EACCES was changed, and
tcp_output() now returns zero when EACCES happens. The reason of this
change was a hesitation that applications that use TCP-MD5 will be
affected by changes in project/ipsec.

TCP-MD5 code returns EACCES when security assocition for given connection
is not configured. But the same error code can return pfil(9), and this
change has affected connections blocked by pfil(9). E.g. application
doesn't return immediately when SYN segment is blocked, instead it waits
when several tries will be failed.

Actually, for TCP-MD5 application it doesn't matter will it get EACCES
after first SYN, or after several tries. Security associtions must be
configured before initiating TCP connection.

I left the EACCES in the switch() to show that it has special handling.

Reported by:	Andreas Longwitz <longwitz at incore dot de>
MFC after:	10 days
2018-05-04 09:28:12 +00:00
Sean Bruno
68ff29affa cc_cubic:
- Update cubic parameters to draft-ietf-tcpm-cubic-04

Submitted by:	Matt Macy <mmacy@mattmacy.io>
Reviewed by:	lstewart
Differential Revision:	https://reviews.freebsd.org/D10556
2018-05-03 15:01:27 +00:00
Michael Tuexen
4c6a10903f SImplify the call to tcp_drop(), since the handling of soft error
is also done in tcp_drop(). No functional change.

Sponsored by:	Netflix, Inc.
2018-05-02 20:04:31 +00:00
Stephen Hurd
f3e1324b41 Separate list manipulation locking from state change in multicast
Multicast incorrectly calls in to drivers with a mutex held causing drivers
to have to go through all manner of contortions to use a non sleepable lock.
Serialize multicast updates instead.

Submitted by:	mmacy <mmacy@mattmacy.io>
Reviewed by:	shurd, sbruno
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D14969
2018-05-02 19:36:29 +00:00
Randall Stewart
803a230592 This change re-arranges the fields within the tcp-pcb so that
they are more in order of cache line use as one passes
through the tcp_input/output paths (non-errors most likely path). This
helps speed up cache line optimization so that the tcp stack runs
a bit more efficently.

Sponsored by:	Netflix Inc.
Differential Revision:	https://reviews.freebsd.org/D15136
2018-04-26 21:41:16 +00:00
Sean Bruno
7875017ca9 Revert r332894 at the request of the submitter.
Submitted by:	Johannes Lundberg <johalun0_gmail.com>
Sponsored by:	Limelight Networks
2018-04-24 19:55:12 +00:00
Sean Bruno
7b7796eea5 Load balance sockets with new SO_REUSEPORT_LB option
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations
As DragonflyBSD, a load balance group is limited to 256 pcbs
(256 programs or threads sharing the same socket).

Submitted by:	Johannes Lundberg <johanlun0@gmail.com>
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-04-23 19:51:00 +00:00
Randall Stewart
fd389e7cd5 These two modules need the tcp_hpts.h file for
when the option is enabled (not sure how LINT/build-universe
missed this) opps.

Sponsored by:	Netflix Inc
2018-04-19 15:03:48 +00:00
Randall Stewart
3ee9c3c4eb This commit brings in the TCP high precision timer system (tcp_hpts).
It is the forerunner/foundational work of bringing in both Rack and BBR
which use hpts for pacing out packets. The feature is optional and requires
the TCPHPTS option to be enabled before the feature will be active. TCP
modules that use it must assure that the base component is compile in
the kernel in which they are loaded.

MFC after:	Never
Sponsored by:	Netflix Inc.
Differential Revision:	https://reviews.freebsd.org/D15020
2018-04-19 13:37:59 +00:00
Brooks Davis
3a4fc8a8a1 Remove support for the Arcnet protocol.
While Arcnet has some continued deployment in industrial controls, the
lack of drivers for any of the PCI, USB, or PCIe NICs on the market
suggests such users aren't running FreeBSD.

Evidence in the PR database suggests that the cm(4) driver (our sole
Arcnet NIC) was broken in 5.0 and has not worked since.

PR:		182297
Reviewed by:	jhibbits, vangyzen
Relnotes:	yes
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15057
2018-04-13 21:18:04 +00:00
Brooks Davis
0437c8e3b1 Remove support for FDDI networks.
Defines in net/if_media.h remain in case code copied from ifconfig is in
use elsewere (supporting non-existant media type is harmless).

Reviewed by:	kib, jhb
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15017
2018-04-11 17:28:24 +00:00
Jonathan T. Looney
f897951965 Add missing header change from r332381.
Sponsored by:	Netflix, Inc.
Pointy hat:	jtl
2018-04-10 17:00:37 +00:00
Jonathan T. Looney
8b8718b5b0 Modify the net.inet.tcp.function_ids sysctl introduced in r331347.
Export additional information which may be helpful to userspace
consumers and rename the sysctl to net.inet.tcp.function_info.

Sponsored by:	Netflix, Inc.
2018-04-10 16:59:36 +00:00
Jonathan T. Looney
dcaffbd6fb Move the TCP Blackbox Recorder probe in tcp_output.c to be with the
other tracing/debugging code.

Sponsored by:	Netflix, Inc.
2018-04-10 15:54:29 +00:00
Jonathan T. Looney
4889b58ce8 Clean up some debugging code left in tcp_log_buf.c from r331347.
Sponsored by:	Netflix, Inc.
2018-04-10 15:51:37 +00:00
Michael Tuexen
efcf28ef77 Fix a logical inversion bug.
Thanks to Irene Ruengeler for finding and reporting this bug.

MFC after:	3 days
2018-04-08 12:08:20 +00:00
Michael Tuexen
3bcfd10fe1 Small cleanup, no functional change.
MFC after:	3 days
2018-04-08 11:50:06 +00:00
Michael Tuexen
c3115feb7e Fix a signed/unsigned warning showing up for the userland stack
on some platforms.
Thanks to Felix Weinrank for reporting the issue.

MFC after:i	3 days
2018-04-08 11:37:00 +00:00
Brooks Davis
6469bdcdb6 Move most of the contents of opt_compat.h to opt_global.h.
opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c.  A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by:	kib, cem, jhb, jtl
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14941
2018-04-06 17:35:35 +00:00
Jonathan T. Looney
8fa799bd74 If a user closes the socket before we call tcp_usr_abort(), then
tcp_drop() may unlock the INP.  Currently, tcp_usr_abort() does not
check for this case, which results in a panic while trying to unlock
the already-unlocked INP (not to mention, a use-after-free violation).

Make tcp_usr_abort() check the return value of tcp_drop(). In the case
where tcp_drop() returns NULL, tcp_usr_abort() can skip further steps
to abort the connection and simply unlock the INP_INFO lock prior to
returning.

Reviewed by:	glebius
MFC after:	2 weeks
Sponsored by:	Netflix, Inc.
2018-04-06 17:20:37 +00:00
Jonathan T. Looney
45a48d07c3 Check that in_pcbfree() is only called once for each PCB. If that
assumption is violated, "bad things" could follow.

I believe such an assert would have detected some of the problems jch@
was chasing in PR 203175 (see r307551).  We also use it in our internal
TCP development efforts.  And, in case a bug does slip through to
released code, this change silently ignores subsequent calls to
in_pcbfree().

Reviewed by:	rrs
Sponsored by:	Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D14990
2018-04-06 16:48:11 +00:00
Ed Maste
c73b6f4da9 Fix kernel memory disclosure in tcp_ctloutput
strcpy was used to copy a string into a buffer copied to userland, which
left uninitialized data after the terminating 0-byte.  Use the same
approach as in tcp_subr.c: strncpy and explicit '\0'.

admbugs:	765, 822
MFC after:	1 day
Reported by:	Ilja Van Sprundel <ivansprundel@ioactive.com>
Reported by:	Vlad Tsyrklevich
Security:	Kernel memory disclosure
Sponsored by:	The FreeBSD Foundation
2018-04-04 21:12:35 +00:00
Jonathan T. Looney
4b9dc36454 r330675 introduced an extra window check in the LRO code to ensure it
captured and reported the highest window advertisement with the same
SEQ/ACK.  However, the window comparison uses modulo 2**16 math, rather
than directly comparing the absolute values.  Because windows use
absolute values and not modulo 2**16 math (i.e. they don't wrap), we
need to compare the absolute values.

Reviewed by:	gallatin
MFC after:	3 days
Sponsored by:	Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D14937
2018-04-03 13:54:38 +00:00
Navdeep Parhar
a64564109a Add a hook to allow the toedev handling an offloaded connection to
provide accurate TCP_INFO.

Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D14816
2018-04-03 01:08:54 +00:00
Brooks Davis
541d96aaaf Use an accessor function to access ifr_data.
This fixes 32-bit compat (no ioctl command defintions are required
as struct ifreq is the same size).  This is believed to be sufficent to
fully support ifconfig on 32-bit systems.

Reviewed by:	kib
Obtained from:	CheriBSD
MFC after:	1 week
Relnotes:	yes
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14900
2018-03-30 18:50:13 +00:00
Navdeep Parhar
b9a9e8e9bd Fix RSS build (broken in r331309).
Sponsored by:	Chelsio Communications
2018-03-29 19:48:17 +00:00