Commit Graph

5770 Commits

Author SHA1 Message Date
Jonathan T. Looney
cb503ae22d Don't overpromote values when calculating len in tcp_output().
sbavail() returns u_int and sendwin is a uint32_t. Therefore, min() (which
operates on two u_int values) is able to correctly calculate the minimum
of these two arguments.

Reported by:	rrs
MFC after:	1 week
Sponsored by:	Netflix
2017-07-05 16:10:30 +00:00
Michael Tuexen
1698cbd919 Move to open state after plausibility checks.
When doing this too early, the MIB counters go wrong.

MFC after:	1 week
2017-07-04 18:24:50 +00:00
Michael Tuexen
afffa1a9ad Don't hold if refcount on an stcb when it is not needed.
This improves the consistency with other parts of the code.
2017-07-04 18:04:44 +00:00
Sean Bruno
ac952dd274 Add a sysctl to toggle the use of the sockets LOWAT when calculating auto window growth
Submitted by:	j@nitrology.com (Jason Wolfe)
Reviewed by:	gnn hiren
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11016
2017-07-03 19:39:58 +00:00
Michael Tuexen
f4358911bf Handle sctp_get_next_param() in a consistent way.
This addresses an issue found by Felix Weinrank using libfuzz.
While there, use also consistent nameing.

MFC after:	3 days
2017-06-23 21:01:57 +00:00
Michael Tuexen
d44b45df2c Check the length of a COOKIE chunk before accessing fields in it.
Thanks to Felix Weinrank for reporting the issue he found by using
libFuzzer.

MFC after:	3 days
2017-06-23 10:09:49 +00:00
Michael Tuexen
1a7abbb3be Use a longer buffer for messages in ERROR chunks.
This allows them to be sent in a non truncated way and addresses a warning
given by newver versions of gcc.
Thanks to Anselm Jonas Scholl for reporting it and providing a patch.
2017-06-23 09:27:31 +00:00
Michael Tuexen
94f66d603a Honor the backlog field. 2017-06-23 08:35:54 +00:00
Michael Tuexen
3017b21bb6 Improve compilation on platforms different from FreeBSD. 2017-06-23 08:34:01 +00:00
Gleb Smirnoff
779f106aa1 Listening sockets improvements.
o Separate fields of struct socket that belong to listening from
  fields that belong to normal dataflow, and unionize them.  This
  shrinks the structure a bit.
  - Take out selinfo's from the socket buffers into the socket. The
    first reason is to support braindamaged scenario when a socket is
    added to kevent(2) and then listen(2) is cast on it. The second
    reason is that there is future plan to make socket buffers pluggable,
    so that for a dataflow socket a socket buffer can be changed, and
    in this case we also want to keep same selinfos through the lifetime
    of a socket.
  - Remove struct struct so_accf. Since now listening stuff no longer
    affects struct socket size, just move its fields into listening part
    of the union.
  - Provide sol_upcall field and enforce that so_upcall_set() may be called
    only on a dataflow socket, which has buffers, and for listening sockets
    provide solisten_upcall_set().

o Remove ACCEPT_LOCK() global.
  - Add a mutex to socket, to be used instead of socket buffer lock to lock
    fields of struct socket that don't belong to a socket buffer.
  - Allow to acquire two socket locks, but the first one must belong to a
    listening socket.
  - Make soref()/sorele() to use atomic(9).  This allows in some situations
    to do soref() without owning socket lock.  There is place for improvement
    here, it is possible to make sorele() also to lock optionally.
  - Most protocols aren't touched by this change, except UNIX local sockets.
    See below for more information.

o Reduce copy-and-paste in kernel modules that accept connections from
  listening sockets: provide function solisten_dequeue(), and use it in
  the following modules: ctl(4), iscsi(4), ng_btsocket(4), ng_ksocket(4),
  infiniband, rpc.

o UNIX local sockets.
  - Removal of ACCEPT_LOCK() global uncovered several races in the UNIX
    local sockets.  Most races exist around spawning a new socket, when we
    are connecting to a local listening socket.  To cover them, we need to
    hold locks on both PCBs when spawning a third one.  This means holding
    them across sonewconn().  This creates a LOR between pcb locks and
    unp_list_lock.
  - To fix the new LOR, abandon the global unp_list_lock in favor of global
    unp_link_lock.  Indeed, separating these two locks didn't provide us any
    extra parralelism in the UNIX sockets.
  - Now call into uipc_attach() may happen with unp_link_lock hold if, we
    are accepting, or without unp_link_lock in case if we are just creating
    a socket.
  - Another problem in UNIX sockets is that uipc_close() basicly did nothing
    for a listening socket.  The vnode remained opened for connections.  This
    is fixed by removing vnode in uipc_close().  Maybe the right way would be
    to do it for all sockets (not only listening), simply move the vnode
    teardown from uipc_detach() to uipc_close()?

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D9770
2017-06-08 21:30:34 +00:00
Jonathan T. Looney
dc6a41b936 Add the infrastructure to support loading multiple versions of TCP
stack modules.

It adds support for mangling symbols exported by a module by prepending
a string to them. (This avoids overlapping symbols in the kernel linker.)

It allows the use of a macro as the module name in the DECLARE_MACRO()
and MACRO_VERSION() macros.

It allows the code to register stack aliases (e.g. both a generic name
["default"] and version-specific name ["default_10_3p1"]).

With these changes, it is trivial to compile TCP stack modules with
the name defined in the Makefile and to load multiple versions of the
same stack simultaneously. This functionality can be used to enable
side-by-side testing of an old and new version of the same TCP stack.
It also could support upgrading the TCP stack without a reboot.

Reviewed by:	gnn, sjg (makefiles only)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D11086
2017-06-08 20:41:28 +00:00
Gleb Smirnoff
3acfe1e1b0 This code was missing socket unlock and socket buffer lock, but it
worked since right now these two locks are the same.
2017-06-08 06:37:11 +00:00
Gleb Smirnoff
12d8a8e7a3 The desired lock here is socket buffer, not socket.
Right now they match, but won't in future.
2017-06-08 06:34:09 +00:00
Michael Tuexen
8cb5a8e90a Fix the ICMP6 handling for TCP.
The ICMP6 packets might not be contained in a single mbuf. So don't
assume this. Keep the IPv4 and IPv6 code in sync and make explicit
that the syncache code only need the TCP sequence number, not the
complete TCP header.

MFC after:		3 days
Sponsored by:		Netflix, Inc.
2017-06-03 21:53:58 +00:00
Michael Tuexen
98732609d5 Improve comments to describe what the code does.
Reported by:		jtl
Sponsored by:		Netflix, Inc.
2017-06-01 15:11:18 +00:00
Jonathan T. Looney
382a6bbcf1 Enforce the limit on ICMP messages before doing work to formulate the
response.

Delete an unneeded rate limit for UDP under IPv6. Because ICMP6
messages have their own rate limit, it is unnecessary to apply a
second rate limit to UDP messages.

Reviewed by:	glebius
MFC after:	2 weeks
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D10387
2017-05-30 14:32:44 +00:00
Michael Tuexen
5d08768a2b Use the SCTP_PCB_FLAGS_ACCEPTING flags to check for listeners.
While there, use a macro for checking the listen state to allow for
easier changes if required.

This done to help glebius@ with his listen changes.
2017-05-26 16:29:00 +00:00
Gleb Smirnoff
6a6cefac3d o Rearrange struct inpcb fields to optimize the TCP output code path
considering cache line hits and misses.  Put the lock and hash list
  glue into the first cache line, put inp_refcount inp_flags inp_socket
  into the second cache line.
o On allocation zero out entire structure except the lock and list entries,
  including inp_route inp_lle inp_gencnt.  When inp_route and inp_lle were
  introduced, they were added below inp_zero_size, resulting on not being
  cleared after free/alloc.  This definitely was a source of bugs with route
  caching.  Could be that r315956 has just fixed one of them.
  The inp_gencnt is reinitialized on every alloc, so it is safe to clear it.

This has been proved to improve TCP performance at Netflix.

Obtained from:		rrs
Differential Revision:	D10686
2017-05-24 17:47:16 +00:00
Michael Tuexen
5dba6ada91 The connect() system call should return -1 and set errno to EAFNOSUPPORT
if it is called on a TCP socket
 * with an IPv6 address and the socket is bound to an
    IPv4-mapped IPv6 address.
 * with an IPv4-mapped IPv6 address and the socket is bound to an
   IPv6 address.
Thanks to Jonathan T. Leighton for reporting this issue.

Reviewed by:		bz gnn
MFC after:		3 days
Differential Revision:	https://reviews.freebsd.org/D9163
2017-05-22 15:29:10 +00:00
Andrey V. Elsukov
38cc96a887 Set M_BCAST and M_MCAST flags on mbuf sent via divert socket.
r290383 has changed how mbufs sent by divert socket are handled.
Previously they are always handled by slow path processing in ip_input().
Now ip_tryforward() is invoked from ip_input() before in_broadcast() check.
Since diverted packet lost all mbuf flags, it passes the broadcast check
in ip_tryforward() due to missing M_BCAST flag. In the result the broadcast
packet is forwarded to the wire instead of be consumed by network stack.

Add in_broadcast() check to the div_output() function. And restore the
M_BCAST flag if destination address is broadcast for the given network
interface.

PR:		209491
MFC after:	1 week
2017-05-17 09:04:09 +00:00
Ed Maste
3e85b721d6 Remove register keyword from sys/ and ANSIfy prototypes
A long long time ago the register keyword told the compiler to store
the corresponding variable in a CPU register, but it is not relevant
for any compiler used in the FreeBSD world today.

ANSIfy related prototypes while here.

Reviewed by:	cem, jhb
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10193
2017-05-17 00:34:34 +00:00
Gleb Smirnoff
cc487c1697 Reduce in_pcbinfo_init() by two params. No users supply any flags to this
function (they used to say UMA_ZONE_NOFREE), so flag parameter goes away.
The zone_fini parameter also goes away.  Previously no protocols (except
divert) supplied zone_fini function, so inpcb locks were leaked with slabs.
This was okay while zones were allocated with UMA_ZONE_NOFREE flag, but now
this is a leak.  Fix that by suppling inpcb_fini() function as fini method
for all inpcb zones.
2017-05-15 21:58:36 +00:00
Enji Cooper
bd7459366e Add missing braces around MCAST_EXCLUDE check when KTR support is
compiled into the kernel

This ensures that .iss_asm (the number of ASM listeners) isn't incorrectly
decremented for MLD-layer source datagrams when inspecting im*s_st[1]
(the second state in the structure).

MFC after:	2 months
PR:		217509 [1]
Reported by:	Coverity (Isilon)
Reviewed by:	ae ("This patch looks correct to me." [1])
Submitted by:	Miles Ohlrich <miles.ohlrich@isilon.com>
Sponsored by:	Dell EMC Isilon
2017-05-13 18:41:24 +00:00
Gleb Smirnoff
7637c57ee1 There is no good reason for TCP reassembly zone to be UMA_ZONE_NOFREE.
It has strong locking model, doesn't have any timers associated with
entries.  The entries theirselves are referenced only from the tcpcb zone,
which itself is a normal zone, without the UMA_ZONE_NOFREE flag.
2017-05-10 23:32:31 +00:00
Eugene Grosbein
1a356b8b90 ipfw nat and natd support multiple aliasing instances with "nat global" feature
that chooses right alias_address for outgoing packets that already have
corresponding state in one of aliasing instances. This feature works just fine
for ICMP, UDP, TCP and SCTP packes but not for others. For example,
outgoing PPtP/GRE packets always get alias_address of latest configured
instance no matter whether such packets have corresponding state or not.

This change unbreaks translation of transit PPtP/GRE connections
for "nat global" case fixing a bug in static ProtoAliasOut() function
that ignores its "create" argument and performs translation
regardless of its value. This static function is called only
by LibAliasOutLocked() function and only for packers other than
ICMP, UDP, TCP and SCTP. LibAliasOutLocked() passes its "create"
argument unmodified.

We have only two consumers of LibAliasOutLocked() in the source tree
calling it with "create" unequal to 1: "ipfw nat global" code and similar
natd code having same problem. All other consumers of LibAliasOutLocked()
call it with create = 1 and the patch is "no-op" for such cases.

PR:		218968
Approved by:	ae, vsevolod (mentor)
MFC after:	1 week
2017-05-10 19:41:52 +00:00
Michael Tuexen
10e0318afa Allow SCTP to use the hostcache.
This patch allows the MTU stored in the hostcache to be used as an
initial value for SCTP paths. When an ICMP PTB message is received,
store the MTU in the hostcache.

MFC after:	1 week
2017-04-29 19:20:50 +00:00
Michael Tuexen
4f43a14a85 Don't set the DF-bit on timer based retransmissions.
MFC after:	1 week
2017-04-29 09:57:27 +00:00
Michael Tuexen
b6ecf43450 Set the DF bit for responses to out-of-the-blue packets.
MFC after:	1 week
2017-04-28 15:38:34 +00:00
Michael Tuexen
d274bcc661 Fix an issue with MTU calculation if an ICMP messaeg is received
for an SCTP/UDP packet.

MFC after:	1 week
2017-04-26 20:21:05 +00:00
Michael Tuexen
6ebfa5ee14 Use consistently uint32_t for mtu values.
This does not change functionality, but this cleanup is need for further
improvements of ICMP handling.

MFC after:	1 week
2017-04-26 19:26:40 +00:00
Michael Tuexen
ebfd753408 When a SYN-ACK is received in SYN-SENT state, RFC 793 requires the
validation of SEG.ACK as the first step. If the ACK is not acceptable,
a RST segment should be sent and the segment should be dropped.
Up to now, the segment was partially processed.
This patch moves the check for the SEG.ACK validation up to the front
as required.
Reviewed by:		hiren, gnn
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D10424
2017-04-26 06:20:58 +00:00
Navdeep Parhar
f8acc03ef1 Flush the LRO ctrl as soon as lro_mbufs fills up. There is no need to
wait for the next enqueue from the driver.

Reviewed by:	gnn@, hselasky@, gallatin@
MFC after:	1 week
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D10432
2017-04-24 22:35:00 +00:00
Navdeep Parhar
ea9a92f112 Frames that are not considered for LRO should not be counted in LRO statistics.
Reviewed by:	gnn@, hselasky@, gallatin@
MFC after:	1 week
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D10430
2017-04-24 22:31:56 +00:00
Brooks Davis
a7dc31283a Remove the NATM framework including the en(4), fatm(4), hatm(4), and
patm(4) devices.

Maintaining an address family and framework has real costs when we make
infrastructure improvements.  In the case of NATM we support no devices
manufactured in the last 20 years and some will not even work in modern
motherboards (some newer devices that patm(4) could be updated to
support apparently exist, but we do not currently have support).

With this change, support remains for some netgraph modules that don't
require NATM support code. It is unclear if all these should remain,
though ng_atmllc certainly stands alone.

Note well: FreeBSD 11 supports NATM and will continue to do so until at
least September 30, 2021.  Improvements to the code in FreeBSD 11 are
certainly welcome.

Reviewed by:	philip
Approved by:	harti
2017-04-24 21:21:49 +00:00
Michael Tuexen
75e7a91649 Represent "a syncache overflow hasn't happend yet" by using
-(SYNCOOKIE_LIFETIME + 1) instead of INT64_MIN, since it is
good enough and works when time_t is int32 or int64.
This fixes the issue reported by cy@ on i386.

Reported by:	cy
MFC after:	1 week
Sponsored by:	Netflix, Inc.
2017-04-21 06:05:34 +00:00
Michael Tuexen
190d9abce7 Syncoockies can be used in combination with the syncache. If the cache
overflows, syncookies are used.
This patch restricts the usage of syncookies in this case: accept
syncookies only if there was an overflow of the syncache recently.
This mitigates a problem reported in PR217637, where is syncookie was
accepted without any recent drops.
Thanks to glebius@ for suggesting an improvement.

PR:			217637
Reviewed by:		gnn, glebius
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D10272
2017-04-20 19:19:33 +00:00
Navdeep Parhar
b0ca71f0a0 Free lro_hash unconditionally, just like lro_mbuf_data a few lines
later.  Fix whitespace nit while here.
2017-04-19 23:06:07 +00:00
Navdeep Parhar
a3927369fa Do not leak lro_hash on failure to allocate lro_mbuf_data.
MFC after:	1 week
2017-04-19 22:27:26 +00:00
Navdeep Parhar
3d24e03800 Remove redundant assignment. 2017-04-19 22:20:41 +00:00
Andrey V. Elsukov
c33a231337 Rework r316770 to make it protocol independent and general, like we
do for streaming sockets.

And do more cleanup in the sbappendaddr_locked_internal() to prevent
leak information from existing mbuf to the one, that will be possible
created later by netgraph.

Suggested by:	glebius
Tested by:	Irina Liakh <spell at itl ua>
MFC after:	1 week
2017-04-14 09:00:48 +00:00
Andrey V. Elsukov
8428914909 Clear h/w csum flags on mbuf handled by UDP.
When checksums of received IP and UDP header already checked, UDP uses
sbappendaddr_locked() to pass received data to the socket.
sbappendaddr_locked() uses given mbuf as is, and if NIC supports checksum
offloading, mbuf contains csum_data and csum_flags that were calculated
for already stripped headers. Some NICs support only limited checksums
offloading and do not use CSUM_PSEUDO_HDR flag, and csum_data contains
some value that UDP/TCP should use for pseudo header checksum calculation.

When L2TP is used for tunneling with mpd5, ng_ksocket receives mbuf with
filled csum_flags and csum_data, that were calculated for outer headers.
When L2TP header is stripped, a packet that was tunneled goes to the IP
layer and due to presence of csum_flags (without CSUM_PSEUDO_HDR) and
csum_data, the UDP/TCP checksum check fails for this packet.

Reported by:	Irina Liakh <spell at itl ua>
Tested by:	Irina Liakh <spell at itl ua>
MFC after:	1 week
2017-04-13 17:03:57 +00:00
Michael Tuexen
013f4df643 The sysctl variable net.inet.tcp.drop_synfin is not honored in all states,
for example not in SYN-SENT.
This patch adds code to check the sysctl variable in other states than
LISTEN.
Thanks to ae and gnn for providing comments.
Reviewed by:		gnn
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D9894
2017-04-12 20:27:15 +00:00
Andrey V. Elsukov
7faa0d213b Make sysctl identifiers for direct netisr queue unique.
Introduce IPCTL_INTRDQMAXLEN and IPCTL_INTRDQDROPS macros for this purpose.

Reviewed by:	gnn
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D10358
2017-04-11 19:20:20 +00:00
Steven Hartland
e44c1887fd Use estimated RTT for receive buffer auto resizing instead of timestamps
Switched from using timestamps to RTT estimates when performing TCP receive
buffer auto resizing, as not all hosts support / enable TCP timestamps.

Disabled reset of receive buffer auto scaling when not in bulk receive mode,
which gives an extra 20% performance increase.

Also extracted auto resizing to a common method shared between standard and
fastpath modules.

With this AWS S3 downloads at ~17ms latency on a 1Gbps connection jump from
~3MB/s to ~100MB/s using the default settings.

Reviewed by:    lstewart, gnn
MFC after:      2 weeks
Relnotes:       Yes
Sponsored by:   Multiplay
Differential Revision:  https://reviews.freebsd.org/D9668
2017-04-10 08:19:35 +00:00
Ryan Stone
4af540d197 Revert the optimization from r304436
r304436 attempted to optimize the handling of incoming UDP packet by only
making an expensive call to in_broadcast() if the mbuf was marked as an
broadcast packet.  Unfortunately, this cannot work in the case of point-to-
point L2 protocols like PPP, which have no notion of "broadcast".  The
optimization has been disabled for several months now with no progress
towards fixing it, so it needs to go.
2017-04-05 16:57:13 +00:00
Andrey V. Elsukov
11c56650f0 Add O_EXTERNAL_DATA opcode support.
This opcode can be used to attach some data to external action opcode.
And unlike to O_EXTERNAL_INSTANCE opcode, this opcode does not require
creating of named instance to pass configuration arguments to external
action handler. The data is coming just next to O_EXTERNAL_ACTION opcode.

The userlevel part currenly supports formatting for opcode with ipfw_insn
size, by default it expects u16 numeric value in the arg1.

Obtained from:	Yandex LLC
MFC after:	2 weeks
Sponsored by:	Yandex LLC
2017-04-03 02:44:40 +00:00
Steven Hartland
6ebc1b7b7d Allow explicitly assigned IPv4 loopback address to be used in jails
If a jail has an explicitly assigned loopback address then allow it to be
used instead of remapping requests for the loopback adddress to the first
IPv4 address assigned to the jail.

This fixes issues where applications attempt to detect their bound port
where they requested a loopback address, which was available, but instead
the kernel remapped it to the jails first address.

A example of this is binding nginx to 127.0.0.1 and then running "service
nginx upgrade" which before this change would cause nginx to fail.

Also:
* Correct the description of prison_check_ip4_locked to match the code.

MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	Multiplay
2017-03-31 00:41:54 +00:00
Mike Karels
4a5c6c6ab0 Enable route and LLE (ndp) caching in TCP/IPv6
tcp_output.c was using a route on the stack for IPv6, which does not
allow route caching or LLE/ndp caching. Switch to using the route
(v6 flavor) in the in_pcb, which was already present, which caches
both L3 and L2 lookups.

Reviewed by:	gnn hiren
MFC after:	2 weeks
2017-03-27 23:48:36 +00:00
Mike Karels
8c1960d506 Fix reference count leak with L2 caching.
ip_forward, TCP/IPv6, and probably SCTP leaked references to L2 cache
entry because they used their own routes on the stack, not in_pcb routes.
The original model for route caching was callers that provided a route
structure to ip{,6}input() would keep the route, and this model was used
for L2 caching as well. Instead, change L2 caching to be done by default
only when using a route structure in the in_pcb; the pcb deallocation
code frees L2 as well as L3 cacches. A separate change will add route
caching to TCP/IPv6.

Another suggestion was to have the transport protocols indicate willingness
to use L2 caching, but this approach keeps the changes in the network
level

Reviewed by:    ae gnn
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D10059
2017-03-25 15:06:28 +00:00
Gleb Smirnoff
3ae4b0e7e8 Force same alignment on struct xinpgen as we have on struct xinpcb. This
fixes 32-bit builds.
2017-03-21 16:23:44 +00:00