fb264c6326
Allow TLS records to be decrypted in the kernel after being received by a NIC. At a high level this is somewhat similar to software KTLS for the transmit path except in reverse. Protocols enqueue mbufs containing encrypted TLS records (or portions of records) into the tail of a socket buffer and the KTLS layer decrypts those records before returning them to userland applications. However, there is an important difference: - In the transmit case, the socket buffer is always a single "record" holding a chain of mbufs. Not-yet-encrypted mbufs are marked not ready (M_NOTREADY) and released to protocols for transmit by marking mbufs ready once their data is encrypted. - In the receive case, incoming (encrypted) data appended to the socket buffer is still a single stream of data from the protocol, but decrypted TLS records are stored as separate records in the socket buffer and read individually via recvmsg(). Initially I tried to make this work by marking incoming mbufs as M_NOTREADY, but there didn't seemed to be a non-gross way to deal with picking a portion of the mbuf chain and turning it into a new record in the socket buffer after decrypting the TLS record it contained (along with prepending a control message). Also, such mbufs would also need to be "pinned" in some way while they are being decrypted such that a concurrent sbcut() wouldn't free them out from under the thread performing decryption. As such, I settled on the following solution: - Socket buffers now contain an additional chain of mbufs (sb_mtls, sb_mtlstail, and sb_tlscc) containing encrypted mbufs appended by the protocol layer. These mbufs are still marked M_NOTREADY, but soreceive*() generally don't know about them (except that they will block waiting for data to be decrypted for a blocking read). - Each time a new mbuf is appended to this TLS mbuf chain, the socket buffer peeks at the TLS record header at the head of the chain to determine the encrypted record's length. If enough data is queued for the TLS record, the socket is placed on a per-CPU TLS workqueue (reusing the existing KTLS workqueues and worker threads). - The worker thread loops over the TLS mbuf chain decrypting records until it runs out of data. Each record is detached from the TLS mbuf chain while it is being decrypted to keep the mbufs "pinned". However, a new sb_dtlscc field tracks the character count of the detached record and sbcut()/sbdrop() is updated to account for the detached record. After the record is decrypted, the worker thread first checks to see if sbcut() dropped the record. If so, it is freed (can happen when a socket is closed with pending data). Otherwise, the header and trailer are stripped from the original mbufs, a control message is created holding the decrypted TLS header, and the decrypted TLS record is appended to the "normal" socket buffer chain. (Side note: the SBCHECK() infrastucture was very useful as I was able to add assertions there about the TLS chain that caught several bugs during development.) Tested by: rmacklem (various versions) Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24628
811 lines
24 KiB
Groff
811 lines
24 KiB
Groff
.\" Copyright (c) 1983, 1991, 1993
|
|
.\" The Regents of the University of California.
|
|
.\" Copyright (c) 2010-2011 The FreeBSD Foundation
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" Portions of this documentation were written at the Centre for Advanced
|
|
.\" Internet Architectures, Swinburne University of Technology, Melbourne,
|
|
.\" Australia by David Hayes under sponsorship from the FreeBSD Foundation.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\" 3. Neither the name of the University nor the names of its contributors
|
|
.\" may be used to endorse or promote products derived from this software
|
|
.\" without specific prior written permission.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
.\" SUCH DAMAGE.
|
|
.\"
|
|
.\" From: @(#)tcp.4 8.1 (Berkeley) 6/5/93
|
|
.\" $FreeBSD$
|
|
.\"
|
|
.Dd July 23, 2020
|
|
.Dt TCP 4
|
|
.Os
|
|
.Sh NAME
|
|
.Nm tcp
|
|
.Nd Internet Transmission Control Protocol
|
|
.Sh SYNOPSIS
|
|
.In sys/types.h
|
|
.In sys/socket.h
|
|
.In netinet/in.h
|
|
.In netinet/tcp.h
|
|
.Ft int
|
|
.Fn socket AF_INET SOCK_STREAM 0
|
|
.Sh DESCRIPTION
|
|
The
|
|
.Tn TCP
|
|
protocol provides reliable, flow-controlled, two-way
|
|
transmission of data.
|
|
It is a byte-stream protocol used to
|
|
support the
|
|
.Dv SOCK_STREAM
|
|
abstraction.
|
|
.Tn TCP
|
|
uses the standard
|
|
Internet address format and, in addition, provides a per-host
|
|
collection of
|
|
.Dq "port addresses" .
|
|
Thus, each address is composed
|
|
of an Internet address specifying the host and network,
|
|
with a specific
|
|
.Tn TCP
|
|
port on the host identifying the peer entity.
|
|
.Pp
|
|
Sockets utilizing the
|
|
.Tn TCP
|
|
protocol are either
|
|
.Dq active
|
|
or
|
|
.Dq passive .
|
|
Active sockets initiate connections to passive
|
|
sockets.
|
|
By default,
|
|
.Tn TCP
|
|
sockets are created active; to create a
|
|
passive socket, the
|
|
.Xr listen 2
|
|
system call must be used
|
|
after binding the socket with the
|
|
.Xr bind 2
|
|
system call.
|
|
Only passive sockets may use the
|
|
.Xr accept 2
|
|
call to accept incoming connections.
|
|
Only active sockets may use the
|
|
.Xr connect 2
|
|
call to initiate connections.
|
|
.Pp
|
|
Passive sockets may
|
|
.Dq underspecify
|
|
their location to match
|
|
incoming connection requests from multiple networks.
|
|
This technique, termed
|
|
.Dq "wildcard addressing" ,
|
|
allows a single
|
|
server to provide service to clients on multiple networks.
|
|
To create a socket which listens on all networks, the Internet
|
|
address
|
|
.Dv INADDR_ANY
|
|
must be bound.
|
|
The
|
|
.Tn TCP
|
|
port may still be specified
|
|
at this time; if the port is not specified, the system will assign one.
|
|
Once a connection has been established, the socket's address is
|
|
fixed by the peer entity's location.
|
|
The address assigned to the
|
|
socket is the address associated with the network interface
|
|
through which packets are being transmitted and received.
|
|
Normally, this address corresponds to the peer entity's network.
|
|
.Pp
|
|
.Tn TCP
|
|
supports a number of socket options which can be set with
|
|
.Xr setsockopt 2
|
|
and tested with
|
|
.Xr getsockopt 2 :
|
|
.Bl -tag -width ".Dv TCP_FUNCTION_BLK"
|
|
.It Dv TCP_INFO
|
|
Information about a socket's underlying TCP session may be retrieved
|
|
by passing the read-only option
|
|
.Dv TCP_INFO
|
|
to
|
|
.Xr getsockopt 2 .
|
|
It accepts a single argument: a pointer to an instance of
|
|
.Vt "struct tcp_info" .
|
|
.Pp
|
|
This API is subject to change; consult the source to determine
|
|
which fields are currently filled out by this option.
|
|
.Fx
|
|
specific additions include
|
|
send window size,
|
|
receive window size,
|
|
and
|
|
bandwidth-controlled window space.
|
|
.It Dv TCP_CCALGOOPT
|
|
Set or query congestion control algorithm specific parameters.
|
|
See
|
|
.Xr mod_cc 4
|
|
for details.
|
|
.It Dv TCP_CONGESTION
|
|
Select or query the congestion control algorithm that TCP will use for the
|
|
connection.
|
|
See
|
|
.Xr mod_cc 4
|
|
for details.
|
|
.It Dv TCP_FUNCTION_BLK
|
|
Select or query the set of functions that TCP will use for this connection.
|
|
This allows a user to select an alternate TCP stack.
|
|
The alternate TCP stack must already be loaded in the kernel.
|
|
To list the available TCP stacks, see
|
|
.Va functions_available
|
|
in the
|
|
.Sx MIB Variables
|
|
section further down.
|
|
To list the default TCP stack, see
|
|
.Va functions_default
|
|
in the
|
|
.Sx MIB Variables
|
|
section.
|
|
.It Dv TCP_KEEPINIT
|
|
This
|
|
.Xr setsockopt 2
|
|
option accepts a per-socket timeout argument of
|
|
.Vt "u_int"
|
|
in seconds, for new, non-established
|
|
.Tn TCP
|
|
connections.
|
|
For the global default in milliseconds see
|
|
.Va keepinit
|
|
in the
|
|
.Sx MIB Variables
|
|
section further down.
|
|
.It Dv TCP_KEEPIDLE
|
|
This
|
|
.Xr setsockopt 2
|
|
option accepts an argument of
|
|
.Vt "u_int"
|
|
for the amount of time, in seconds, that the connection must be idle
|
|
before keepalive probes (if enabled) are sent for the connection of this
|
|
socket.
|
|
If set on a listening socket, the value is inherited by the newly created
|
|
socket upon
|
|
.Xr accept 2 .
|
|
For the global default in milliseconds see
|
|
.Va keepidle
|
|
in the
|
|
.Sx MIB Variables
|
|
section further down.
|
|
.It Dv TCP_KEEPINTVL
|
|
This
|
|
.Xr setsockopt 2
|
|
option accepts an argument of
|
|
.Vt "u_int"
|
|
to set the per-socket interval, in seconds, between keepalive probes sent
|
|
to a peer.
|
|
If set on a listening socket, the value is inherited by the newly created
|
|
socket upon
|
|
.Xr accept 2 .
|
|
For the global default in milliseconds see
|
|
.Va keepintvl
|
|
in the
|
|
.Sx MIB Variables
|
|
section further down.
|
|
.It Dv TCP_KEEPCNT
|
|
This
|
|
.Xr setsockopt 2
|
|
option accepts an argument of
|
|
.Vt "u_int"
|
|
and allows a per-socket tuning of the number of probes sent, with no response,
|
|
before the connection will be dropped.
|
|
If set on a listening socket, the value is inherited by the newly created
|
|
socket upon
|
|
.Xr accept 2 .
|
|
For the global default see the
|
|
.Va keepcnt
|
|
in the
|
|
.Sx MIB Variables
|
|
section further down.
|
|
.It Dv TCP_NODELAY
|
|
Under most circumstances,
|
|
.Tn TCP
|
|
sends data when it is presented;
|
|
when outstanding data has not yet been acknowledged, it gathers
|
|
small amounts of output to be sent in a single packet once
|
|
an acknowledgement is received.
|
|
For a small number of clients, such as window systems
|
|
that send a stream of mouse events which receive no replies,
|
|
this packetization may cause significant delays.
|
|
The boolean option
|
|
.Dv TCP_NODELAY
|
|
defeats this algorithm.
|
|
.It Dv TCP_MAXSEG
|
|
By default, a sender- and
|
|
.No receiver- Ns Tn TCP
|
|
will negotiate among themselves to determine the maximum segment size
|
|
to be used for each connection.
|
|
The
|
|
.Dv TCP_MAXSEG
|
|
option allows the user to determine the result of this negotiation,
|
|
and to reduce it if desired.
|
|
.It Dv TCP_NOOPT
|
|
.Tn TCP
|
|
usually sends a number of options in each packet, corresponding to
|
|
various
|
|
.Tn TCP
|
|
extensions which are provided in this implementation.
|
|
The boolean option
|
|
.Dv TCP_NOOPT
|
|
is provided to disable
|
|
.Tn TCP
|
|
option use on a per-connection basis.
|
|
.It Dv TCP_NOPUSH
|
|
By convention, the
|
|
.No sender- Ns Tn TCP
|
|
will set the
|
|
.Dq push
|
|
bit, and begin transmission immediately (if permitted) at the end of
|
|
every user call to
|
|
.Xr write 2
|
|
or
|
|
.Xr writev 2 .
|
|
When this option is set to a non-zero value,
|
|
.Tn TCP
|
|
will delay sending any data at all until either the socket is closed,
|
|
or the internal send buffer is filled.
|
|
.It Dv TCP_MD5SIG
|
|
This option enables the use of MD5 digests (also known as TCP-MD5)
|
|
on writes to the specified socket.
|
|
Outgoing traffic is digested;
|
|
digests on incoming traffic are verified.
|
|
When this option is enabled on a socket, all inbound and outgoing
|
|
TCP segments must be signed with MD5 digests.
|
|
.Pp
|
|
One common use for this in a
|
|
.Fx
|
|
router deployment is to enable
|
|
based routers to interwork with Cisco equipment at peering points.
|
|
Support for this feature conforms to RFC 2385.
|
|
.Pp
|
|
In order for this option to function correctly, it is necessary for the
|
|
administrator to add a tcp-md5 key entry to the system's security
|
|
associations database (SADB) using the
|
|
.Xr setkey 8
|
|
utility.
|
|
This entry can only be specified on a per-host basis at this time.
|
|
.Pp
|
|
If an SADB entry cannot be found for the destination,
|
|
the system does not send any outgoing segments and drops any inbound segments.
|
|
.It Dv TCP_STATS
|
|
Manage collection of connection level statistics using the
|
|
.Xr stats 3
|
|
framework.
|
|
.Pp
|
|
Each dropped segment is taken into account in the TCP protocol statistics.
|
|
.It Dv TCP_TXTLS_ENABLE
|
|
Enable in-kernel Transport Layer Security (TLS) for data written to this
|
|
socket.
|
|
The
|
|
.Vt struct tls_so_enable
|
|
argument defines the encryption and authentication algorithms and keys
|
|
used to encrypt the socket data as well as the maximum TLS record
|
|
payload size.
|
|
.Pp
|
|
All data written to this socket will be encapsulated in TLS records
|
|
and subsequently encrypted.
|
|
By default all data written to this socket is treated as application data.
|
|
Individual TLS records with a type other than application data
|
|
(for example, handshake messages),
|
|
may be transmitted by invoking
|
|
.Xr sendmsg 2
|
|
with a custom TLS record type set in a
|
|
.Dv TLS_SET_RECORD_TYPE
|
|
control message.
|
|
The payload of this control message is a single byte holding the desired
|
|
TLS record type.
|
|
.Pp
|
|
At present, only a single transmit key may be set on a socket.
|
|
As such, users of this option must disable rekeying.
|
|
.It Dv TCP_TXTLS_MODE
|
|
The integer argument can be used to get or set the current TLS transmit mode
|
|
of a socket.
|
|
Setting the mode can only used to toggle between software and NIC TLS after
|
|
TLS has been initially enabled via the
|
|
.Dv TCP_TXTLS_ENABLE
|
|
option.
|
|
The available modes are:
|
|
.Bl -tag -width "Dv TCP_TLS_MODE_IFNET"
|
|
.It Dv TCP_TLS_MODE_NONE
|
|
In-kernel TLS framing and encryption is not enabled for this socket.
|
|
.It Dv TCP_TLS_MODE_SW
|
|
TLS records are encrypted by the kernel prior to placing the data in the
|
|
socket buffer.
|
|
Typically this encryption is performed in software.
|
|
.It Dv TCP_TLS_MODE_IFNET
|
|
TLS records are encrypted by the network interface card (NIC).
|
|
.It Dv TCP_TLS_MODE_TOE
|
|
TLS records are encrypted by the NIC using a TCP offload engine (TOE).
|
|
.El
|
|
.It Dv TCP_RXTLS_ENABLE
|
|
Enable in-kernel TLS for data read from this socket.
|
|
The
|
|
.Vt struct tls_so_enable
|
|
argument defines the encryption and authentication algorithms and keys
|
|
used to decrypt the socket data.
|
|
.Pp
|
|
Each received TLS record must be read from the socket using
|
|
.Xr recvmsg 2 .
|
|
Each received TLS record will contain a
|
|
.Dv TLS_GET_RECORD
|
|
control message along with the decrypted payload.
|
|
The control message contains a
|
|
.Vt struct tls_get_record
|
|
which includes fields from the TLS record header.
|
|
If an invalid or corrupted TLS record is received,
|
|
recvmsg 2
|
|
will fail with one of the following errors:
|
|
.Bl -tag -width Er
|
|
.It Bq Er EINVAL
|
|
The version fields in a TLS record's header did not match the version required
|
|
by the
|
|
.Vt struct tls_so_enable
|
|
structure used to enable in-kernel TLS.
|
|
.It Bq Er EMSGSIZE
|
|
A TLS record's length was either too small or too large.
|
|
.It Bq Er EMSGSIZE
|
|
The connection was closed after sending a truncated TLS record.
|
|
.It Bq Er EBADMSG
|
|
The TLS record failed to match the included authentication tag.
|
|
.El
|
|
.Pp
|
|
At present, only a single receive key may be set on a socket.
|
|
As such, users of this option must disable rekeying.
|
|
.It Dv TCP_RXTLS_MODE
|
|
The integer argument can be used to get the current TLS receive mode
|
|
of a socket.
|
|
The available modes are the same as for
|
|
.Dv TCP_TXTLS_MODE .
|
|
.El
|
|
.Pp
|
|
The option level for the
|
|
.Xr setsockopt 2
|
|
call is the protocol number for
|
|
.Tn TCP ,
|
|
available from
|
|
.Xr getprotobyname 3 ,
|
|
or
|
|
.Dv IPPROTO_TCP .
|
|
All options are declared in
|
|
.In netinet/tcp.h .
|
|
.Pp
|
|
Options at the
|
|
.Tn IP
|
|
transport level may be used with
|
|
.Tn TCP ;
|
|
see
|
|
.Xr ip 4 .
|
|
Incoming connection requests that are source-routed are noted,
|
|
and the reverse source route is used in responding.
|
|
.Pp
|
|
The default congestion control algorithm for
|
|
.Tn TCP
|
|
is
|
|
.Xr cc_newreno 4 .
|
|
Other congestion control algorithms can be made available using the
|
|
.Xr mod_cc 4
|
|
framework.
|
|
.Ss MIB Variables
|
|
The
|
|
.Tn TCP
|
|
protocol implements a number of variables in the
|
|
.Va net.inet.tcp
|
|
branch of the
|
|
.Xr sysctl 3
|
|
MIB.
|
|
.Bl -tag -width ".Va TCPCTL_DO_RFC1323"
|
|
.It Dv TCPCTL_DO_RFC1323
|
|
.Pq Va rfc1323
|
|
Implement the window scaling and timestamp options of RFC 1323
|
|
(default is true).
|
|
.It Dv TCPCTL_MSSDFLT
|
|
.Pq Va mssdflt
|
|
The default value used for the maximum segment size
|
|
.Pq Dq MSS
|
|
when no advice to the contrary is received from MSS negotiation.
|
|
.It Dv TCPCTL_SENDSPACE
|
|
.Pq Va sendspace
|
|
Maximum
|
|
.Tn TCP
|
|
send window.
|
|
.It Dv TCPCTL_RECVSPACE
|
|
.Pq Va recvspace
|
|
Maximum
|
|
.Tn TCP
|
|
receive window.
|
|
.It Va log_in_vain
|
|
Log any connection attempts to ports where there is not a socket
|
|
accepting connections.
|
|
The value of 1 limits the logging to
|
|
.Tn SYN
|
|
(connection establishment) packets only.
|
|
That of 2 results in any
|
|
.Tn TCP
|
|
packets to closed ports being logged.
|
|
Any value unlisted above disables the logging
|
|
(default is 0, i.e., the logging is disabled).
|
|
.It Va msl
|
|
The Maximum Segment Lifetime, in milliseconds, for a packet.
|
|
.It Va keepinit
|
|
Timeout, in milliseconds, for new, non-established
|
|
.Tn TCP
|
|
connections.
|
|
The default is 75000 msec.
|
|
.It Va keepidle
|
|
Amount of time, in milliseconds, that the connection must be idle
|
|
before keepalive probes (if enabled) are sent.
|
|
The default is 7200000 msec (2 hours).
|
|
.It Va keepintvl
|
|
The interval, in milliseconds, between keepalive probes sent to remote
|
|
machines, when no response is received on a
|
|
.Va keepidle
|
|
probe.
|
|
The default is 75000 msec.
|
|
.It Va keepcnt
|
|
Number of probes sent, with no response, before a connection
|
|
is dropped.
|
|
The default is 8 packets.
|
|
.It Va always_keepalive
|
|
Assume that
|
|
.Dv SO_KEEPALIVE
|
|
is set on all
|
|
.Tn TCP
|
|
connections, the kernel will
|
|
periodically send a packet to the remote host to verify the connection
|
|
is still up.
|
|
.It Va icmp_may_rst
|
|
Certain
|
|
.Tn ICMP
|
|
unreachable messages may abort connections in
|
|
.Tn SYN-SENT
|
|
state.
|
|
.It Va do_tcpdrain
|
|
Flush packets in the
|
|
.Tn TCP
|
|
reassembly queue if the system is low on mbufs.
|
|
.It Va blackhole
|
|
If enabled, disable sending of RST when a connection is attempted
|
|
to a port where there is not a socket accepting connections.
|
|
See
|
|
.Xr blackhole 4 .
|
|
.It Va delayed_ack
|
|
Delay ACK to try and piggyback it onto a data packet.
|
|
.It Va delacktime
|
|
Maximum amount of time, in milliseconds, before a delayed ACK is sent.
|
|
.It Va path_mtu_discovery
|
|
Enable Path MTU Discovery.
|
|
.It Va tcbhashsize
|
|
Size of the
|
|
.Tn TCP
|
|
control-block hash table
|
|
(read-only).
|
|
This may be tuned using the kernel option
|
|
.Dv TCBHASHSIZE
|
|
or by setting
|
|
.Va net.inet.tcp.tcbhashsize
|
|
in the
|
|
.Xr loader 8 .
|
|
.It Va pcbcount
|
|
Number of active process control blocks
|
|
(read-only).
|
|
.It Va syncookies
|
|
Determines whether or not
|
|
.Tn SYN
|
|
cookies should be generated for outbound
|
|
.Tn SYN-ACK
|
|
packets.
|
|
.Tn SYN
|
|
cookies are a great help during
|
|
.Tn SYN
|
|
flood attacks, and are enabled by default.
|
|
(See
|
|
.Xr syncookies 4 . )
|
|
.It Va isn_reseed_interval
|
|
The interval (in seconds) specifying how often the secret data used in
|
|
RFC 1948 initial sequence number calculations should be reseeded.
|
|
By default, this variable is set to zero, indicating that
|
|
no reseeding will occur.
|
|
Reseeding should not be necessary, and will break
|
|
.Dv TIME_WAIT
|
|
recycling for a few minutes.
|
|
.It Va reass.cursegments
|
|
The current total number of segments present in all reassembly queues.
|
|
.It Va reass.maxsegments
|
|
The maximum limit on the total number of segments across all reassembly
|
|
queues.
|
|
The limit can be adjusted as a tunable.
|
|
.It Va reass.maxqueuelen
|
|
The maximum number of segments allowed in each reassembly queue.
|
|
By default, the system chooses a limit based on each TCP connection's
|
|
receive buffer size and maximum segment size (MSS).
|
|
The actual limit applied to a session's reassembly queue will be the lower of
|
|
the system-calculated automatic limit and the user-specified
|
|
.Va reass.maxqueuelen
|
|
limit.
|
|
.It Va rexmit_initial , rexmit_min , rexmit_slop
|
|
Adjust the retransmit timer calculation for
|
|
.Tn TCP .
|
|
The slop is
|
|
typically added to the raw calculation to take into account
|
|
occasional variances that the
|
|
.Tn SRTT
|
|
(smoothed round-trip time)
|
|
is unable to accommodate, while the minimum specifies an
|
|
absolute minimum.
|
|
While a number of
|
|
.Tn TCP
|
|
RFCs suggest a 1
|
|
second minimum, these RFCs tend to focus on streaming behavior,
|
|
and fail to deal with the fact that a 1 second minimum has severe
|
|
detrimental effects over lossy interactive connections, such
|
|
as a 802.11b wireless link, and over very fast but lossy
|
|
connections for those cases not covered by the fast retransmit
|
|
code.
|
|
For this reason, we use 200ms of slop and a near-0
|
|
minimum, which gives us an effective minimum of 200ms (similar to
|
|
.Tn Linux ) .
|
|
The initial value is used before an RTT measurement has been performed.
|
|
.It Va initcwnd_segments
|
|
Enable the ability to specify initial congestion window in number of segments.
|
|
The default value is 10 as suggested by RFC 6928.
|
|
Changing the value on fly would not affect connections using congestion window
|
|
from the hostcache.
|
|
Caution:
|
|
This regulates the burst of packets allowed to be sent in the first RTT.
|
|
The value should be relative to the link capacity.
|
|
Start with small values for lower-capacity links.
|
|
Large bursts can cause buffer overruns and packet drops if routers have small
|
|
buffers or the link is experiencing congestion.
|
|
.It Va newcwd
|
|
Enable the New Congestion Window Validation mechanism as described in RFC 7661.
|
|
This gently reduces the congestion window during periods, where TCP is
|
|
application limited and the network bandwidth is not utilized completely.
|
|
That prevents self-inflicted packet losses once the application starts to
|
|
transmit data at a higher speed.
|
|
.It Va rfc6675_pipe
|
|
Calculate the bytes in flight using the algorithm described in RFC 6675, and
|
|
is also a prerequisite to enable Proportional Rate Reduction.
|
|
.It Va rfc3042
|
|
Enable the Limited Transmit algorithm as described in RFC 3042.
|
|
It helps avoid timeouts on lossy links and also when the congestion window
|
|
is small, as happens on short transfers.
|
|
.It Va rfc3390
|
|
Enable support for RFC 3390, which allows for a variable-sized
|
|
starting congestion window on new connections, depending on the
|
|
maximum segment size.
|
|
This helps throughput in general, but
|
|
particularly affects short transfers and high-bandwidth large
|
|
propagation-delay connections.
|
|
.It Va sack.enable
|
|
Enable support for RFC 2018, TCP Selective Acknowledgment option,
|
|
which allows the receiver to inform the sender about all successfully
|
|
arrived segments, allowing the sender to retransmit the missing segments
|
|
only.
|
|
.It Va sack.maxholes
|
|
Maximum number of SACK holes per connection.
|
|
Defaults to 128.
|
|
.It Va sack.globalmaxholes
|
|
Maximum number of SACK holes per system, across all connections.
|
|
Defaults to 65536.
|
|
.It Va maxtcptw
|
|
When a TCP connection enters the
|
|
.Dv TIME_WAIT
|
|
state, its associated socket structure is freed, since it is of
|
|
negligible size and use, and a new structure is allocated to contain a
|
|
minimal amount of information necessary for sustaining a connection in
|
|
this state, called the compressed TCP TIME_WAIT state.
|
|
Since this structure is smaller than a socket structure, it can save
|
|
a significant amount of system memory.
|
|
The
|
|
.Va net.inet.tcp.maxtcptw
|
|
MIB variable controls the maximum number of these structures allocated.
|
|
By default, it is initialized to
|
|
.Va kern.ipc.maxsockets
|
|
/ 5.
|
|
.It Va nolocaltimewait
|
|
Suppress creating of compressed TCP TIME_WAIT states for connections in
|
|
which both endpoints are local.
|
|
.It Va fast_finwait2_recycle
|
|
Recycle
|
|
.Tn TCP
|
|
.Dv FIN_WAIT_2
|
|
connections faster when the socket is marked as
|
|
.Dv SBS_CANTRCVMORE
|
|
(no user process has the socket open, data received on
|
|
the socket cannot be read).
|
|
The timeout used here is
|
|
.Va finwait2_timeout .
|
|
.It Va finwait2_timeout
|
|
Timeout to use for fast recycling of
|
|
.Tn TCP
|
|
.Dv FIN_WAIT_2
|
|
connections.
|
|
Defaults to 60 seconds.
|
|
.It Va ecn.enable
|
|
Enable support for TCP Explicit Congestion Notification (ECN).
|
|
ECN allows a TCP sender to reduce the transmission rate in order to
|
|
avoid packet drops.
|
|
Settings:
|
|
.Bl -tag -compact
|
|
.It 0
|
|
Disable ECN.
|
|
.It 1
|
|
Allow incoming connections to request ECN.
|
|
Outgoing connections will request ECN.
|
|
.It 2
|
|
Allow incoming connections to request ECN.
|
|
Outgoing connections will not request ECN.
|
|
.El
|
|
.It Va ecn.maxretries
|
|
Number of retries (SYN or SYN/ACK retransmits) before disabling ECN on a
|
|
specific connection.
|
|
This is needed to help with connection establishment
|
|
when a broken firewall is in the network path.
|
|
.It Va pmtud_blackhole_detection
|
|
Enable automatic path MTU blackhole detection.
|
|
In case of retransmits of MSS sized segments,
|
|
the OS will lower the MSS to check if it's an MTU problem.
|
|
If the current MSS is greater than the configured value to try
|
|
.Po Va net.inet.tcp.pmtud_blackhole_mss
|
|
and
|
|
.Va net.inet.tcp.v6pmtud_blackhole_mss
|
|
.Pc ,
|
|
it will be set to this value, otherwise,
|
|
the MSS will be set to the default values
|
|
.Po Va net.inet.tcp.mssdflt
|
|
and
|
|
.Va net.inet.tcp.v6mssdflt
|
|
.Pc .
|
|
Settings:
|
|
.Bl -tag -compact
|
|
.It 0
|
|
Disable path MTU blackhole detection.
|
|
.It 1
|
|
Enable path MTU blackhole detection for IPv4 and IPv6.
|
|
.It 2
|
|
Enable path MTU blackhole detection only for IPv4.
|
|
.It 3
|
|
Enable path MTU blackhole detection only for IPv6.
|
|
.El
|
|
.It Va pmtud_blackhole_mss
|
|
MSS to try for IPv4 if PMTU blackhole detection is turned on.
|
|
.It Va v6pmtud_blackhole_mss
|
|
MSS to try for IPv6 if PMTU blackhole detection is turned on.
|
|
.It Va functions_available
|
|
List of available TCP function blocks (TCP stacks).
|
|
.It Va functions_default
|
|
The default TCP function block (TCP stack).
|
|
.It Va functions_inherit_listen_socket_stack
|
|
Determines whether to inherit listen socket's tcp stack or use the current
|
|
system default tcp stack, as defined by
|
|
.Va functions_default .
|
|
Default is true.
|
|
.It Va insecure_rst
|
|
Use criteria defined in RFC793 instead of RFC5961 for accepting RST segments.
|
|
Default is false.
|
|
.It Va insecure_syn
|
|
Use criteria defined in RFC793 instead of RFC5961 for accepting SYN segments.
|
|
Default is false.
|
|
.It Va ts_offset_per_conn
|
|
When initializing the TCP timestamps, use a per connection offset instead of a
|
|
per host pair offset.
|
|
Default is to use per connection offsets as recommended in RFC 7323.
|
|
.It Va perconn_stats_enable
|
|
Controls the default collection of statistics for all connections using the
|
|
.Xr stats 3
|
|
framework.
|
|
0 disables, 1 enables, 2 enables random sampling across log id connection
|
|
groups with all connections in a group receiving the same setting.
|
|
.It Va perconn_stats_sample_rates
|
|
A CSV list of template_spec=percent key-value pairs which controls the per
|
|
template sampling rates when
|
|
.Xr stats 3
|
|
sampling is enabled.
|
|
.El
|
|
.Sh ERRORS
|
|
A socket operation may fail with one of the following errors returned:
|
|
.Bl -tag -width Er
|
|
.It Bq Er EISCONN
|
|
when trying to establish a connection on a socket which
|
|
already has one;
|
|
.It Bo Er ENOBUFS Bc or Bo Er ENOMEM Bc
|
|
when the system runs out of memory for
|
|
an internal data structure;
|
|
.It Bq Er ETIMEDOUT
|
|
when a connection was dropped
|
|
due to excessive retransmissions;
|
|
.It Bq Er ECONNRESET
|
|
when the remote peer
|
|
forces the connection to be closed;
|
|
.It Bq Er ECONNREFUSED
|
|
when the remote
|
|
peer actively refuses connection establishment (usually because
|
|
no process is listening to the port);
|
|
.It Bq Er EADDRINUSE
|
|
when an attempt
|
|
is made to create a socket with a port which has already been
|
|
allocated;
|
|
.It Bq Er EADDRNOTAVAIL
|
|
when an attempt is made to create a
|
|
socket with a network address for which no network interface
|
|
exists;
|
|
.It Bq Er EAFNOSUPPORT
|
|
when an attempt is made to bind or connect a socket to a multicast
|
|
address.
|
|
.It Bq Er EINVAL
|
|
when trying to change TCP function blocks at an invalid point in the session;
|
|
.It Bq Er ENOENT
|
|
when trying to use a TCP function block that is not available;
|
|
.El
|
|
.Sh SEE ALSO
|
|
.Xr getsockopt 2 ,
|
|
.Xr socket 2 ,
|
|
.Xr stats 3 ,
|
|
.Xr sysctl 3 ,
|
|
.Xr blackhole 4 ,
|
|
.Xr inet 4 ,
|
|
.Xr intro 4 ,
|
|
.Xr ip 4 ,
|
|
.Xr mod_cc 4 ,
|
|
.Xr siftr 4 ,
|
|
.Xr syncache 4 ,
|
|
.Xr setkey 8 ,
|
|
.Xr tcp_functions 9
|
|
.Rs
|
|
.%A "V. Jacobson"
|
|
.%A "R. Braden"
|
|
.%A "D. Borman"
|
|
.%T "TCP Extensions for High Performance"
|
|
.%O "RFC 1323"
|
|
.Re
|
|
.Rs
|
|
.%A "A. Heffernan"
|
|
.%T "Protection of BGP Sessions via the TCP MD5 Signature Option"
|
|
.%O "RFC 2385"
|
|
.Re
|
|
.Rs
|
|
.%A "K. Ramakrishnan"
|
|
.%A "S. Floyd"
|
|
.%A "D. Black"
|
|
.%T "The Addition of Explicit Congestion Notification (ECN) to IP"
|
|
.%O "RFC 3168"
|
|
.Re
|
|
.Sh HISTORY
|
|
The
|
|
.Tn TCP
|
|
protocol appeared in
|
|
.Bx 4.2 .
|
|
The RFC 1323 extensions for window scaling and timestamps were added
|
|
in
|
|
.Bx 4.4 .
|
|
The
|
|
.Dv TCP_INFO
|
|
option was introduced in
|
|
.Tn Linux 2.6
|
|
and is
|
|
.Em subject to change .
|