2005-01-07 01:45:51 +00:00
|
|
|
/*-
|
1998-03-18 01:40:12 +00:00
|
|
|
* Copyright 1998 Massachusetts Institute of Technology
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
* Copyright 2012 ADARA Networks, Inc.
|
2017-08-15 17:52:37 +00:00
|
|
|
* Copyright 2017 Dell EMC Isilon
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
*
|
|
|
|
* Portions of this software were developed by Robert N. M. Watson under
|
|
|
|
* contract to ADARA Networks, Inc.
|
1998-03-18 01:40:12 +00:00
|
|
|
*
|
|
|
|
* Permission to use, copy, modify, and distribute this software and
|
|
|
|
* its documentation for any purpose and without fee is hereby
|
|
|
|
* granted, provided that both the above copyright notice and this
|
|
|
|
* permission notice appear in all copies, that both the above
|
|
|
|
* copyright notice and this permission notice appear in all
|
|
|
|
* supporting documentation, and that the name of M.I.T. not be used
|
|
|
|
* in advertising or publicity pertaining to distribution of the
|
|
|
|
* software without specific, written prior permission. M.I.T. makes
|
|
|
|
* no representations about the suitability of this software for any
|
|
|
|
* purpose. It is provided "as is" without express or implied
|
|
|
|
* warranty.
|
2020-10-21 21:28:20 +00:00
|
|
|
*
|
1998-03-18 01:40:12 +00:00
|
|
|
* THIS SOFTWARE IS PROVIDED BY M.I.T. ``AS IS''. M.I.T. DISCLAIMS
|
|
|
|
* ALL EXPRESS OR IMPLIED WARRANTIES WITH REGARD TO THIS SOFTWARE,
|
|
|
|
* INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
|
|
|
* MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT
|
|
|
|
* SHALL M.I.T. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
|
|
|
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
|
|
|
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
|
|
|
|
* USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
|
|
|
|
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
|
|
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
|
|
|
|
* OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
* SUCH DAMAGE.
|
|
|
|
*/
|
|
|
|
|
|
|
|
/*
|
|
|
|
* if_vlan.c - pseudo-device driver for IEEE 802.1Q virtual LANs.
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
* This is sort of sneaky in the implementation, since
|
1998-03-18 01:40:12 +00:00
|
|
|
* we need to pretend to be enough of an Ethernet implementation
|
|
|
|
* to make arp work. The way we do this is by telling everyone
|
|
|
|
* that we are an Ethernet, and then catch the packets that
|
2011-11-28 19:35:08 +00:00
|
|
|
* ether_output() sends to us via if_transmit(), rewrite them for
|
|
|
|
* use by the real outgoing interface, and ask it to send them.
|
1998-03-18 01:40:12 +00:00
|
|
|
*/
|
|
|
|
|
2010-02-21 00:07:45 +00:00
|
|
|
#include <sys/cdefs.h>
|
|
|
|
__FBSDID("$FreeBSD$");
|
|
|
|
|
2013-04-18 20:13:33 +00:00
|
|
|
#include "opt_inet.h"
|
2020-01-22 20:36:45 +00:00
|
|
|
#include "opt_inet6.h"
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
#include "opt_kern_tls.h"
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#include "opt_vlan.h"
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
#include "opt_ratelimit.h"
|
1998-03-18 01:40:12 +00:00
|
|
|
|
|
|
|
#include <sys/param.h>
|
2013-10-28 07:29:16 +00:00
|
|
|
#include <sys/eventhandler.h>
|
1998-03-18 01:40:12 +00:00
|
|
|
#include <sys/kernel.h>
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#include <sys/lock.h>
|
1999-03-15 01:17:26 +00:00
|
|
|
#include <sys/malloc.h>
|
1998-03-18 01:40:12 +00:00
|
|
|
#include <sys/mbuf.h>
|
2001-01-31 07:58:58 +00:00
|
|
|
#include <sys/module.h>
|
2014-09-13 18:41:24 +00:00
|
|
|
#include <sys/rmlock.h>
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
#include <sys/priv.h>
|
1999-03-15 01:17:26 +00:00
|
|
|
#include <sys/queue.h>
|
1998-03-18 01:40:12 +00:00
|
|
|
#include <sys/socket.h>
|
|
|
|
#include <sys/sockio.h>
|
|
|
|
#include <sys/sysctl.h>
|
|
|
|
#include <sys/systm.h>
|
2011-03-21 09:40:01 +00:00
|
|
|
#include <sys/sx.h>
|
2017-08-15 17:52:37 +00:00
|
|
|
#include <sys/taskqueue.h>
|
1998-03-18 01:40:12 +00:00
|
|
|
|
|
|
|
#include <net/bpf.h>
|
|
|
|
#include <net/ethernet.h>
|
|
|
|
#include <net/if.h>
|
2013-10-26 17:58:36 +00:00
|
|
|
#include <net/if_var.h>
|
2023-01-13 15:43:51 +00:00
|
|
|
#include <net/if_private.h>
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
#include <net/if_clone.h>
|
1998-03-18 01:40:12 +00:00
|
|
|
#include <net/if_dl.h>
|
|
|
|
#include <net/if_types.h>
|
|
|
|
#include <net/if_vlan_var.h>
|
2020-01-22 20:36:45 +00:00
|
|
|
#include <net/route.h>
|
2008-12-02 21:37:28 +00:00
|
|
|
#include <net/vnet.h>
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2013-04-18 20:13:33 +00:00
|
|
|
#ifdef INET
|
|
|
|
#include <netinet/in.h>
|
|
|
|
#include <netinet/if_ether.h>
|
|
|
|
#endif
|
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#define VLAN_DEF_HWIDTH 4
|
2006-01-31 16:41:05 +00:00
|
|
|
#define VLAN_IFFLAGS (IFF_BROADCAST | IFF_MULTICAST)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
2006-12-30 21:10:25 +00:00
|
|
|
#define UP_AND_RUNNING(ifp) \
|
|
|
|
((ifp)->if_flags & IFF_UP && (ifp)->if_drv_flags & IFF_DRV_RUNNING)
|
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_HEAD(ifvlanhead, ifvlan);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
|
|
|
struct ifvlantrunk {
|
|
|
|
struct ifnet *parent; /* parent interface of this trunk */
|
2018-09-21 01:37:08 +00:00
|
|
|
struct mtx lock;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#ifdef VLAN_ARRAY
|
2006-06-21 07:29:44 +00:00
|
|
|
#define VLAN_ARRAY_SIZE (EVL_VLID_MASK + 1)
|
|
|
|
struct ifvlan *vlans[VLAN_ARRAY_SIZE]; /* static table */
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#else
|
|
|
|
struct ifvlanhead *hash; /* dynamic hash-list table */
|
|
|
|
uint16_t hmask;
|
|
|
|
uint16_t hwidth;
|
|
|
|
#endif
|
|
|
|
int refcnt;
|
|
|
|
};
|
2001-09-05 21:10:28 +00:00
|
|
|
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
#if defined(KERN_TLS) || defined(RATELIMIT)
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
struct vlan_snd_tag {
|
|
|
|
struct m_snd_tag com;
|
|
|
|
struct m_snd_tag *tag;
|
|
|
|
};
|
|
|
|
|
|
|
|
static inline struct vlan_snd_tag *
|
|
|
|
mst_to_vst(struct m_snd_tag *mst)
|
|
|
|
{
|
|
|
|
|
|
|
|
return (__containerof(mst, struct vlan_snd_tag, com));
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
/*
|
|
|
|
* This macro provides a facility to iterate over every vlan on a trunk with
|
|
|
|
* the assumption that none will be added/removed during iteration.
|
|
|
|
*/
|
|
|
|
#ifdef VLAN_ARRAY
|
|
|
|
#define VLAN_FOREACH(_ifv, _trunk) \
|
|
|
|
size_t _i; \
|
|
|
|
for (_i = 0; _i < VLAN_ARRAY_SIZE; _i++) \
|
|
|
|
if (((_ifv) = (_trunk)->vlans[_i]) != NULL)
|
|
|
|
#else /* VLAN_ARRAY */
|
|
|
|
#define VLAN_FOREACH(_ifv, _trunk) \
|
|
|
|
struct ifvlan *_next; \
|
|
|
|
size_t _i; \
|
|
|
|
for (_i = 0; _i < (1 << (_trunk)->hwidth); _i++) \
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_FOREACH_SAFE((_ifv), &(_trunk)->hash[_i], ifv_list, _next)
|
2017-08-15 17:52:37 +00:00
|
|
|
#endif /* VLAN_ARRAY */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* This macro provides a facility to iterate over every vlan on a trunk while
|
|
|
|
* also modifying the number of vlans on the trunk. The iteration continues
|
|
|
|
* until some condition is met or there are no more vlans on the trunk.
|
|
|
|
*/
|
|
|
|
#ifdef VLAN_ARRAY
|
|
|
|
/* The VLAN_ARRAY case is simple -- just a for loop using the condition. */
|
|
|
|
#define VLAN_FOREACH_UNTIL_SAFE(_ifv, _trunk, _cond) \
|
|
|
|
size_t _i; \
|
|
|
|
for (_i = 0; !(_cond) && _i < VLAN_ARRAY_SIZE; _i++) \
|
|
|
|
if (((_ifv) = (_trunk)->vlans[_i]))
|
|
|
|
#else /* VLAN_ARRAY */
|
|
|
|
/*
|
|
|
|
* The hash table case is more complicated. We allow for the hash table to be
|
|
|
|
* modified (i.e. vlans removed) while we are iterating over it. To allow for
|
|
|
|
* this we must restart the iteration every time we "touch" something during
|
|
|
|
* the iteration, since removal will resize the hash table and invalidate our
|
|
|
|
* current position. If acting on the touched element causes the trunk to be
|
|
|
|
* emptied, then iteration also stops.
|
|
|
|
*/
|
|
|
|
#define VLAN_FOREACH_UNTIL_SAFE(_ifv, _trunk, _cond) \
|
|
|
|
size_t _i; \
|
|
|
|
bool _touch = false; \
|
|
|
|
for (_i = 0; \
|
|
|
|
!(_cond) && _i < (1 << (_trunk)->hwidth); \
|
|
|
|
_i = (_touch && ((_trunk) != NULL) ? 0 : _i + 1), _touch = false) \
|
2018-09-21 01:37:08 +00:00
|
|
|
if (((_ifv) = CK_SLIST_FIRST(&(_trunk)->hash[_i])) != NULL && \
|
2017-08-15 17:52:37 +00:00
|
|
|
(_touch = true))
|
|
|
|
#endif /* VLAN_ARRAY */
|
|
|
|
|
2002-11-14 23:43:16 +00:00
|
|
|
struct vlan_mc_entry {
|
2011-03-21 09:40:01 +00:00
|
|
|
struct sockaddr_dl mc_addr;
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_ENTRY(vlan_mc_entry) mc_entries;
|
2018-10-15 10:29:29 +00:00
|
|
|
struct epoch_context mc_epoch_ctx;
|
2002-11-14 23:43:16 +00:00
|
|
|
};
|
|
|
|
|
2019-01-09 16:47:16 +00:00
|
|
|
struct ifvlan {
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
struct ifvlantrunk *ifv_trunk;
|
2005-06-10 16:49:24 +00:00
|
|
|
struct ifnet *ifv_ifp;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#define TRUNK(ifv) ((ifv)->ifv_trunk)
|
2020-10-21 21:28:20 +00:00
|
|
|
#define PARENT(ifv) (TRUNK(ifv)->parent)
|
2014-09-16 21:48:48 +00:00
|
|
|
void *ifv_cookie;
|
2005-10-03 02:24:21 +00:00
|
|
|
int ifv_pflags; /* special flags we have set on parent */
|
2017-04-28 11:00:58 +00:00
|
|
|
int ifv_capenable;
|
2019-01-09 16:47:16 +00:00
|
|
|
int ifv_encaplen; /* encapsulation length */
|
|
|
|
int ifv_mtufudge; /* MTU fudged by this much */
|
|
|
|
int ifv_mintu; /* min transmission unit */
|
2020-10-21 21:28:20 +00:00
|
|
|
struct ether_8021q_tag ifv_qtag;
|
|
|
|
#define ifv_proto ifv_qtag.proto
|
|
|
|
#define ifv_vid ifv_qtag.vid
|
|
|
|
#define ifv_pcp ifv_qtag.pcp
|
2017-08-15 17:52:37 +00:00
|
|
|
struct task lladdr_task;
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_HEAD(, vlan_mc_entry) vlan_mc_listhead;
|
2007-03-12 12:27:30 +00:00
|
|
|
#ifndef VLAN_ARRAY
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_ENTRY(ifvlan) ifv_list;
|
2007-03-12 12:27:30 +00:00
|
|
|
#endif
|
2002-11-14 23:43:16 +00:00
|
|
|
};
|
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
/* Special flags we should propagate to parent. */
|
2005-10-03 02:24:21 +00:00
|
|
|
static struct {
|
|
|
|
int flag;
|
|
|
|
int (*func)(struct ifnet *, int);
|
|
|
|
} vlan_pflags[] = {
|
|
|
|
{IFF_PROMISC, ifpromisc},
|
|
|
|
{IFF_ALLMULTI, if_allmulti},
|
|
|
|
{0, NULL}
|
|
|
|
};
|
2002-11-14 23:43:16 +00:00
|
|
|
|
2022-02-14 19:09:54 +00:00
|
|
|
VNET_DECLARE(int, vlan_mtag_pcp);
|
|
|
|
#define V_vlan_mtag_pcp VNET(vlan_mtag_pcp)
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
|
2012-10-16 13:37:54 +00:00
|
|
|
static const char vlanname[] = "vlan";
|
|
|
|
static MALLOC_DEFINE(M_VLAN, vlanname, "802.1Q Virtual LAN Interface");
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2006-06-21 07:29:44 +00:00
|
|
|
static eventhandler_tag ifdetach_tag;
|
2010-01-18 20:34:00 +00:00
|
|
|
static eventhandler_tag iflladdr_tag;
|
2022-05-19 06:02:27 +00:00
|
|
|
static eventhandler_tag ifevent_tag;
|
2006-06-21 07:29:44 +00:00
|
|
|
|
2003-09-05 20:58:59 +00:00
|
|
|
/*
|
2020-10-21 21:28:20 +00:00
|
|
|
* if_vlan uses two module-level synchronizations primitives to allow concurrent
|
|
|
|
* modification of vlan interfaces and (mostly) allow for vlans to be destroyed
|
|
|
|
* while they are being used for tx/rx. To accomplish this in a way that has
|
2018-09-21 01:37:08 +00:00
|
|
|
* acceptable performance and cooperation with other parts of the network stack
|
|
|
|
* there is a non-sleepable epoch(9) and an sx(9).
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
*
|
2018-09-21 01:37:08 +00:00
|
|
|
* The performance-sensitive paths that warrant using the epoch(9) are
|
2017-08-15 17:52:37 +00:00
|
|
|
* vlan_transmit and vlan_input. Both have to check for the vlan interface's
|
|
|
|
* existence using if_vlantrunk, and being in the network tx/rx paths the use
|
2018-09-21 01:37:08 +00:00
|
|
|
* of an epoch(9) gives a measureable improvement in performance.
|
2017-08-15 17:52:37 +00:00
|
|
|
*
|
|
|
|
* The reason for having an sx(9) is mostly because there are still areas that
|
|
|
|
* must be sleepable and also have safe concurrent access to a vlan interface.
|
|
|
|
* Since the sx(9) exists, it is used by default in most paths unless sleeping
|
|
|
|
* is not permitted, or if it is not clear whether sleeping is permitted.
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
*
|
2017-08-15 17:52:37 +00:00
|
|
|
*/
|
|
|
|
#define _VLAN_SX_ID ifv_sx
|
|
|
|
|
|
|
|
static struct sx _VLAN_SX_ID;
|
|
|
|
|
|
|
|
#define VLAN_LOCKING_INIT() \
|
2020-10-21 21:28:20 +00:00
|
|
|
sx_init_flags(&_VLAN_SX_ID, "vlan_sx", SX_RECURSE)
|
2017-08-15 17:52:37 +00:00
|
|
|
|
|
|
|
#define VLAN_LOCKING_DESTROY() \
|
|
|
|
sx_destroy(&_VLAN_SX_ID)
|
|
|
|
|
|
|
|
#define VLAN_SLOCK() sx_slock(&_VLAN_SX_ID)
|
|
|
|
#define VLAN_SUNLOCK() sx_sunlock(&_VLAN_SX_ID)
|
|
|
|
#define VLAN_XLOCK() sx_xlock(&_VLAN_SX_ID)
|
|
|
|
#define VLAN_XUNLOCK() sx_xunlock(&_VLAN_SX_ID)
|
|
|
|
#define VLAN_SLOCK_ASSERT() sx_assert(&_VLAN_SX_ID, SA_SLOCKED)
|
|
|
|
#define VLAN_XLOCK_ASSERT() sx_assert(&_VLAN_SX_ID, SA_XLOCKED)
|
|
|
|
#define VLAN_SXLOCK_ASSERT() sx_assert(&_VLAN_SX_ID, SA_LOCKED)
|
|
|
|
|
|
|
|
/*
|
2018-09-21 01:37:08 +00:00
|
|
|
* We also have a per-trunk mutex that should be acquired when changing
|
|
|
|
* its state.
|
2017-08-15 17:52:37 +00:00
|
|
|
*/
|
2018-09-21 01:37:08 +00:00
|
|
|
#define TRUNK_LOCK_INIT(trunk) mtx_init(&(trunk)->lock, vlanname, NULL, MTX_DEF)
|
|
|
|
#define TRUNK_LOCK_DESTROY(trunk) mtx_destroy(&(trunk)->lock)
|
|
|
|
#define TRUNK_WLOCK(trunk) mtx_lock(&(trunk)->lock)
|
|
|
|
#define TRUNK_WUNLOCK(trunk) mtx_unlock(&(trunk)->lock)
|
|
|
|
#define TRUNK_WLOCK_ASSERT(trunk) mtx_assert(&(trunk)->lock, MA_OWNED);
|
2017-08-15 17:52:37 +00:00
|
|
|
|
|
|
|
/*
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
* The VLAN_ARRAY substitutes the dynamic hash with a static array
|
2006-08-25 08:25:35 +00:00
|
|
|
* with 4096 entries. In theory this can give a boost in processing,
|
2017-08-15 17:52:37 +00:00
|
|
|
* however in practice it does not. Probably this is because the array
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
* is too big to fit into CPU cache.
|
2003-09-05 20:58:59 +00:00
|
|
|
*/
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#ifndef VLAN_ARRAY
|
|
|
|
static void vlan_inithash(struct ifvlantrunk *trunk);
|
|
|
|
static void vlan_freehash(struct ifvlantrunk *trunk);
|
|
|
|
static int vlan_inshash(struct ifvlantrunk *trunk, struct ifvlan *ifv);
|
|
|
|
static int vlan_remhash(struct ifvlantrunk *trunk, struct ifvlan *ifv);
|
|
|
|
static void vlan_growhash(struct ifvlantrunk *trunk, int howmuch);
|
|
|
|
static __inline struct ifvlan * vlan_gethash(struct ifvlantrunk *trunk,
|
2012-01-12 18:39:37 +00:00
|
|
|
uint16_t vid);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#endif
|
|
|
|
static void trunk_destroy(struct ifvlantrunk *trunk);
|
2003-09-05 20:58:59 +00:00
|
|
|
|
2006-06-29 07:30:39 +00:00
|
|
|
static void vlan_init(void *foo);
|
2002-11-14 23:43:16 +00:00
|
|
|
static void vlan_input(struct ifnet *ifp, struct mbuf *m);
|
1998-08-23 03:07:17 +00:00
|
|
|
static int vlan_ioctl(struct ifnet *ifp, u_long cmd, caddr_t addr);
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
#if defined(KERN_TLS) || defined(RATELIMIT)
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
static int vlan_snd_tag_alloc(struct ifnet *,
|
|
|
|
union if_snd_tag_alloc_params *, struct m_snd_tag **);
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
static int vlan_snd_tag_modify(struct m_snd_tag *,
|
|
|
|
union if_snd_tag_modify_params *);
|
|
|
|
static int vlan_snd_tag_query(struct m_snd_tag *,
|
|
|
|
union if_snd_tag_query_params *);
|
|
|
|
static void vlan_snd_tag_free(struct m_snd_tag *);
|
2021-01-26 16:54:42 +00:00
|
|
|
static struct m_snd_tag *vlan_next_snd_tag(struct m_snd_tag *);
|
|
|
|
static void vlan_ratelimit_query(struct ifnet *,
|
|
|
|
struct if_ratelimit_query_results *);
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
#endif
|
2011-11-28 19:35:08 +00:00
|
|
|
static void vlan_qflush(struct ifnet *ifp);
|
2005-10-03 02:24:21 +00:00
|
|
|
static int vlan_setflag(struct ifnet *ifp, int flag, int status,
|
|
|
|
int (*func)(struct ifnet *, int));
|
|
|
|
static int vlan_setflags(struct ifnet *ifp, int status);
|
1999-03-15 01:17:26 +00:00
|
|
|
static int vlan_setmulti(struct ifnet *ifp);
|
2011-11-28 19:35:08 +00:00
|
|
|
static int vlan_transmit(struct ifnet *ifp, struct mbuf *m);
|
2021-08-21 09:41:03 +00:00
|
|
|
#ifdef ALTQ
|
|
|
|
static void vlan_altq_start(struct ifnet *ifp);
|
|
|
|
static int vlan_altq_transmit(struct ifnet *ifp, struct mbuf *m);
|
|
|
|
#endif
|
2019-08-30 20:19:43 +00:00
|
|
|
static int vlan_output(struct ifnet *ifp, struct mbuf *m,
|
|
|
|
const struct sockaddr *dst, struct route *ro);
|
2010-05-17 19:36:56 +00:00
|
|
|
static void vlan_unconfig(struct ifnet *ifp);
|
2012-08-20 16:00:33 +00:00
|
|
|
static void vlan_unconfig_locked(struct ifnet *ifp, int departing);
|
2020-10-21 21:28:20 +00:00
|
|
|
static int vlan_config(struct ifvlan *ifv, struct ifnet *p, uint16_t tag,
|
|
|
|
uint16_t proto);
|
2009-12-31 20:29:58 +00:00
|
|
|
static void vlan_link_state(struct ifnet *ifp);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
static void vlan_capabilities(struct ifvlan *ifv);
|
|
|
|
static void vlan_trunk_capabilities(struct ifnet *ifp);
|
1999-03-15 01:17:26 +00:00
|
|
|
|
2015-03-20 20:42:58 +00:00
|
|
|
static struct ifnet *vlan_clone_match_ethervid(const char *, int *);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
static int vlan_clone_match(struct if_clone *, const char *);
|
2022-09-22 12:30:09 +00:00
|
|
|
static int vlan_clone_create(struct if_clone *, char *, size_t,
|
|
|
|
struct ifc_data *, struct ifnet **);
|
|
|
|
static int vlan_clone_destroy(struct if_clone *, struct ifnet *, uint32_t);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2006-06-21 07:29:44 +00:00
|
|
|
static void vlan_ifdetach(void *arg, struct ifnet *ifp);
|
2010-01-18 20:34:00 +00:00
|
|
|
static void vlan_iflladdr(void *arg, struct ifnet *ifp);
|
2022-05-19 06:02:27 +00:00
|
|
|
static void vlan_ifevent(void *arg, struct ifnet *ifp, int event);
|
2006-06-21 07:29:44 +00:00
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
static void vlan_lladdr_fn(void *arg, int pending);
|
|
|
|
|
2012-10-16 13:37:54 +00:00
|
|
|
static struct if_clone *vlan_cloner;
|
2001-09-05 21:10:28 +00:00
|
|
|
|
2010-11-22 23:35:29 +00:00
|
|
|
#ifdef VIMAGE
|
2018-07-24 16:35:52 +00:00
|
|
|
VNET_DEFINE_STATIC(struct if_clone *, vlan_cloner);
|
2010-11-22 23:35:29 +00:00
|
|
|
#define V_vlan_cloner VNET(vlan_cloner)
|
|
|
|
#endif
|
|
|
|
|
2021-09-14 18:43:41 +00:00
|
|
|
#ifdef RATELIMIT
|
|
|
|
static const struct if_snd_tag_sw vlan_snd_tag_ul_sw = {
|
|
|
|
.snd_tag_modify = vlan_snd_tag_modify,
|
|
|
|
.snd_tag_query = vlan_snd_tag_query,
|
|
|
|
.snd_tag_free = vlan_snd_tag_free,
|
|
|
|
.next_snd_tag = vlan_next_snd_tag,
|
|
|
|
.type = IF_SND_TAG_TYPE_UNLIMITED
|
|
|
|
};
|
|
|
|
|
|
|
|
static const struct if_snd_tag_sw vlan_snd_tag_rl_sw = {
|
|
|
|
.snd_tag_modify = vlan_snd_tag_modify,
|
|
|
|
.snd_tag_query = vlan_snd_tag_query,
|
|
|
|
.snd_tag_free = vlan_snd_tag_free,
|
|
|
|
.next_snd_tag = vlan_next_snd_tag,
|
|
|
|
.type = IF_SND_TAG_TYPE_RATE_LIMIT
|
|
|
|
};
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#ifdef KERN_TLS
|
|
|
|
static const struct if_snd_tag_sw vlan_snd_tag_tls_sw = {
|
|
|
|
.snd_tag_modify = vlan_snd_tag_modify,
|
|
|
|
.snd_tag_query = vlan_snd_tag_query,
|
|
|
|
.snd_tag_free = vlan_snd_tag_free,
|
|
|
|
.next_snd_tag = vlan_next_snd_tag,
|
|
|
|
.type = IF_SND_TAG_TYPE_TLS
|
|
|
|
};
|
|
|
|
|
|
|
|
#ifdef RATELIMIT
|
|
|
|
static const struct if_snd_tag_sw vlan_snd_tag_tls_rl_sw = {
|
|
|
|
.snd_tag_modify = vlan_snd_tag_modify,
|
|
|
|
.snd_tag_query = vlan_snd_tag_query,
|
|
|
|
.snd_tag_free = vlan_snd_tag_free,
|
|
|
|
.next_snd_tag = vlan_next_snd_tag,
|
|
|
|
.type = IF_SND_TAG_TYPE_TLS_RATE_LIMIT
|
|
|
|
};
|
|
|
|
#endif
|
|
|
|
#endif
|
|
|
|
|
2018-10-15 10:29:29 +00:00
|
|
|
static void
|
|
|
|
vlan_mc_free(struct epoch_context *ctx)
|
|
|
|
{
|
|
|
|
struct vlan_mc_entry *mc = __containerof(ctx, struct vlan_mc_entry, mc_epoch_ctx);
|
|
|
|
free(mc, M_VLAN);
|
|
|
|
}
|
|
|
|
|
2018-11-21 13:34:21 +00:00
|
|
|
#ifndef VLAN_ARRAY
|
|
|
|
#define HASH(n, m) ((((n) >> 8) ^ ((n) >> 4) ^ (n)) & (m))
|
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
static void
|
|
|
|
vlan_inithash(struct ifvlantrunk *trunk)
|
|
|
|
{
|
|
|
|
int i, n;
|
2020-09-01 21:19:14 +00:00
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
/*
|
|
|
|
* The trunk must not be locked here since we call malloc(M_WAITOK).
|
|
|
|
* It is OK in case this function is called before the trunk struct
|
|
|
|
* gets hooked up and becomes visible from other threads.
|
|
|
|
*/
|
|
|
|
|
|
|
|
KASSERT(trunk->hwidth == 0 && trunk->hash == NULL,
|
|
|
|
("%s: hash already initialized", __func__));
|
|
|
|
|
|
|
|
trunk->hwidth = VLAN_DEF_HWIDTH;
|
|
|
|
n = 1 << trunk->hwidth;
|
|
|
|
trunk->hmask = n - 1;
|
|
|
|
trunk->hash = malloc(sizeof(struct ifvlanhead) * n, M_VLAN, M_WAITOK);
|
|
|
|
for (i = 0; i < n; i++)
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_INIT(&trunk->hash[i]);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void
|
|
|
|
vlan_freehash(struct ifvlantrunk *trunk)
|
|
|
|
{
|
|
|
|
#ifdef INVARIANTS
|
|
|
|
int i;
|
|
|
|
|
|
|
|
KASSERT(trunk->hwidth > 0, ("%s: hwidth not positive", __func__));
|
|
|
|
for (i = 0; i < (1 << trunk->hwidth); i++)
|
2018-09-21 01:37:08 +00:00
|
|
|
KASSERT(CK_SLIST_EMPTY(&trunk->hash[i]),
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
("%s: hash table not empty", __func__));
|
|
|
|
#endif
|
|
|
|
free(trunk->hash, M_VLAN);
|
|
|
|
trunk->hash = NULL;
|
|
|
|
trunk->hwidth = trunk->hmask = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
|
|
|
vlan_inshash(struct ifvlantrunk *trunk, struct ifvlan *ifv)
|
|
|
|
{
|
|
|
|
int i, b;
|
|
|
|
struct ifvlan *ifv2;
|
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
VLAN_XLOCK_ASSERT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
KASSERT(trunk->hwidth > 0, ("%s: hwidth not positive", __func__));
|
|
|
|
|
|
|
|
b = 1 << trunk->hwidth;
|
2012-01-12 18:39:37 +00:00
|
|
|
i = HASH(ifv->ifv_vid, trunk->hmask);
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_FOREACH(ifv2, &trunk->hash[i], ifv_list)
|
2012-01-12 18:39:37 +00:00
|
|
|
if (ifv->ifv_vid == ifv2->ifv_vid)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
return (EEXIST);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Grow the hash when the number of vlans exceeds half of the number of
|
|
|
|
* hash buckets squared. This will make the average linked-list length
|
|
|
|
* buckets/2.
|
|
|
|
*/
|
|
|
|
if (trunk->refcnt > (b * b) / 2) {
|
|
|
|
vlan_growhash(trunk, 1);
|
2012-01-12 18:39:37 +00:00
|
|
|
i = HASH(ifv->ifv_vid, trunk->hmask);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
}
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_INSERT_HEAD(&trunk->hash[i], ifv, ifv_list);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
trunk->refcnt++;
|
|
|
|
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
|
|
|
vlan_remhash(struct ifvlantrunk *trunk, struct ifvlan *ifv)
|
|
|
|
{
|
|
|
|
int i, b;
|
|
|
|
struct ifvlan *ifv2;
|
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
VLAN_XLOCK_ASSERT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
KASSERT(trunk->hwidth > 0, ("%s: hwidth not positive", __func__));
|
2020-09-01 21:19:14 +00:00
|
|
|
|
2022-07-22 17:17:04 +00:00
|
|
|
b = 1 << (trunk->hwidth - 1);
|
2012-01-12 18:39:37 +00:00
|
|
|
i = HASH(ifv->ifv_vid, trunk->hmask);
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_FOREACH(ifv2, &trunk->hash[i], ifv_list)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (ifv2 == ifv) {
|
|
|
|
trunk->refcnt--;
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_REMOVE(&trunk->hash[i], ifv2, ifvlan, ifv_list);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (trunk->refcnt < (b * b) / 2)
|
|
|
|
vlan_growhash(trunk, -1);
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
|
|
|
panic("%s: vlan not found\n", __func__);
|
|
|
|
return (ENOENT); /*NOTREACHED*/
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Grow the hash larger or smaller if memory permits.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
vlan_growhash(struct ifvlantrunk *trunk, int howmuch)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifvlanhead *hash2;
|
|
|
|
int hwidth2, i, j, n, n2;
|
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
VLAN_XLOCK_ASSERT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
KASSERT(trunk->hwidth > 0, ("%s: hwidth not positive", __func__));
|
|
|
|
|
|
|
|
if (howmuch == 0) {
|
|
|
|
/* Harmless yet obvious coding error */
|
|
|
|
printf("%s: howmuch is 0\n", __func__);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
hwidth2 = trunk->hwidth + howmuch;
|
|
|
|
n = 1 << trunk->hwidth;
|
|
|
|
n2 = 1 << hwidth2;
|
|
|
|
/* Do not shrink the table below the default */
|
|
|
|
if (hwidth2 < VLAN_DEF_HWIDTH)
|
|
|
|
return;
|
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
hash2 = malloc(sizeof(struct ifvlanhead) * n2, M_VLAN, M_WAITOK);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (hash2 == NULL) {
|
|
|
|
printf("%s: out of memory -- hash size not changed\n",
|
|
|
|
__func__);
|
|
|
|
return; /* We can live with the old hash table */
|
|
|
|
}
|
|
|
|
for (j = 0; j < n2; j++)
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_INIT(&hash2[j]);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
for (i = 0; i < n; i++)
|
2018-09-21 01:37:08 +00:00
|
|
|
while ((ifv = CK_SLIST_FIRST(&trunk->hash[i])) != NULL) {
|
|
|
|
CK_SLIST_REMOVE(&trunk->hash[i], ifv, ifvlan, ifv_list);
|
2012-01-12 18:39:37 +00:00
|
|
|
j = HASH(ifv->ifv_vid, n2 - 1);
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_INSERT_HEAD(&hash2[j], ifv, ifv_list);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
}
|
2018-09-21 01:37:08 +00:00
|
|
|
NET_EPOCH_WAIT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
free(trunk->hash, M_VLAN);
|
|
|
|
trunk->hash = hash2;
|
|
|
|
trunk->hwidth = hwidth2;
|
|
|
|
trunk->hmask = n2 - 1;
|
2007-03-15 14:10:51 +00:00
|
|
|
|
|
|
|
if (bootverbose)
|
|
|
|
if_printf(trunk->parent,
|
|
|
|
"VLAN hash table resized from %d to %d buckets\n", n, n2);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static __inline struct ifvlan *
|
2012-01-12 18:39:37 +00:00
|
|
|
vlan_gethash(struct ifvlantrunk *trunk, uint16_t vid)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
2019-01-09 01:11:19 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_FOREACH(ifv, &trunk->hash[HASH(vid, trunk->hmask)], ifv_list)
|
2012-01-12 18:39:37 +00:00
|
|
|
if (ifv->ifv_vid == vid)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
return (ifv);
|
|
|
|
return (NULL);
|
|
|
|
}
|
|
|
|
|
|
|
|
#if 0
|
|
|
|
/* Debugging code to view the hashtables. */
|
|
|
|
static void
|
|
|
|
vlan_dumphash(struct ifvlantrunk *trunk)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
|
|
|
for (i = 0; i < (1 << trunk->hwidth); i++) {
|
|
|
|
printf("%d: ", i);
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_FOREACH(ifv, &trunk->hash[i], ifv_list)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
printf("%s ", ifv->ifv_ifp->if_xname);
|
|
|
|
printf("\n");
|
|
|
|
}
|
|
|
|
}
|
|
|
|
#endif /* 0 */
|
2011-03-21 09:40:01 +00:00
|
|
|
#else
|
|
|
|
|
|
|
|
static __inline struct ifvlan *
|
2012-01-12 18:39:37 +00:00
|
|
|
vlan_gethash(struct ifvlantrunk *trunk, uint16_t vid)
|
2011-03-21 09:40:01 +00:00
|
|
|
{
|
|
|
|
|
2012-01-12 18:39:37 +00:00
|
|
|
return trunk->vlans[vid];
|
2011-03-21 09:40:01 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static __inline int
|
|
|
|
vlan_inshash(struct ifvlantrunk *trunk, struct ifvlan *ifv)
|
|
|
|
{
|
|
|
|
|
2012-01-12 18:39:37 +00:00
|
|
|
if (trunk->vlans[ifv->ifv_vid] != NULL)
|
2011-03-21 09:40:01 +00:00
|
|
|
return EEXIST;
|
2012-01-12 18:39:37 +00:00
|
|
|
trunk->vlans[ifv->ifv_vid] = ifv;
|
2011-03-21 09:40:01 +00:00
|
|
|
trunk->refcnt++;
|
|
|
|
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static __inline int
|
|
|
|
vlan_remhash(struct ifvlantrunk *trunk, struct ifvlan *ifv)
|
|
|
|
{
|
|
|
|
|
2012-01-12 18:39:37 +00:00
|
|
|
trunk->vlans[ifv->ifv_vid] = NULL;
|
2011-03-21 09:40:01 +00:00
|
|
|
trunk->refcnt--;
|
|
|
|
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static __inline void
|
|
|
|
vlan_freehash(struct ifvlantrunk *trunk)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
static __inline void
|
|
|
|
vlan_inithash(struct ifvlantrunk *trunk)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
#endif /* !VLAN_ARRAY */
|
|
|
|
|
|
|
|
static void
|
|
|
|
trunk_destroy(struct ifvlantrunk *trunk)
|
|
|
|
{
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XLOCK_ASSERT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
|
|
|
vlan_freehash(trunk);
|
|
|
|
trunk->parent->if_vlantrunk = NULL;
|
2006-02-24 17:25:16 +00:00
|
|
|
TRUNK_LOCK_DESTROY(trunk);
|
2017-05-23 00:13:27 +00:00
|
|
|
if_rele(trunk->parent);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
free(trunk, M_VLAN);
|
|
|
|
}
|
|
|
|
|
1999-03-15 01:17:26 +00:00
|
|
|
/*
|
|
|
|
* Program our multicast filter. What we're actually doing is
|
|
|
|
* programming the multicast filter of the parent. This has the
|
|
|
|
* side effect of causing the parent interface to receive multicast
|
|
|
|
* traffic that it doesn't really want, which ends up being discarded
|
|
|
|
* later by the upper protocol layers. Unfortunately, there's no way
|
|
|
|
* to avoid this: there really is only one physical interface.
|
|
|
|
*/
|
2001-01-31 07:58:58 +00:00
|
|
|
static int
|
|
|
|
vlan_setmulti(struct ifnet *ifp)
|
1999-03-15 01:17:26 +00:00
|
|
|
{
|
|
|
|
struct ifnet *ifp_p;
|
2014-08-04 00:58:12 +00:00
|
|
|
struct ifmultiaddr *ifma;
|
1999-03-15 01:17:26 +00:00
|
|
|
struct ifvlan *sc;
|
2007-03-12 12:27:30 +00:00
|
|
|
struct vlan_mc_entry *mc;
|
1999-03-15 01:17:26 +00:00
|
|
|
int error;
|
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
VLAN_XLOCK_ASSERT();
|
2017-08-15 17:52:37 +00:00
|
|
|
|
1999-03-15 01:17:26 +00:00
|
|
|
/* Find the parent. */
|
|
|
|
sc = ifp->if_softc;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
ifp_p = PARENT(sc);
|
2001-07-24 17:14:37 +00:00
|
|
|
|
Step 1.5 of importing the network stack virtualization infrastructure
from the vimage project, as per plan established at devsummit 08/08:
http://wiki.freebsd.org/Image/Notes200808DevSummit
Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator
macros, and CURVNET_SET() context setting macros, all currently
resolving to NOPs.
Prepare for virtualization of selected SYSCTL objects by introducing a
family of SYSCTL_V_*() macros, currently resolving to their global
counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT().
Move selected #defines from sys/sys/vimage.h to newly introduced header
files specific to virtualized subsystems (sys/net/vnet.h,
sys/netinet/vinet.h etc.).
All the changes are verified to have zero functional impact at this
point in time by doing MD5 comparision between pre- and post-change
object files(*).
(*) netipsec/keysock.c did not validate depending on compile time options.
Implemented by: julian, bz, brooks, zec
Reviewed by: julian, bz, brooks, kris, rwatson, ...
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation
2008-10-02 15:37:58 +00:00
|
|
|
CURVNET_SET_QUIET(ifp_p->if_vnet);
|
|
|
|
|
1999-03-15 01:17:26 +00:00
|
|
|
/* First, remove any existing filter entries. */
|
2018-09-21 01:37:08 +00:00
|
|
|
while ((mc = CK_SLIST_FIRST(&sc->vlan_mc_listhead)) != NULL) {
|
|
|
|
CK_SLIST_REMOVE_HEAD(&sc->vlan_mc_listhead, mc_entries);
|
2014-08-04 00:58:12 +00:00
|
|
|
(void)if_delmulti(ifp_p, (struct sockaddr *)&mc->mc_addr);
|
2020-01-15 06:05:20 +00:00
|
|
|
NET_EPOCH_CALL(vlan_mc_free, &mc->mc_epoch_ctx);
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Now program new ones. */
|
2014-08-04 00:58:12 +00:00
|
|
|
IF_ADDR_WLOCK(ifp);
|
2018-05-18 20:13:34 +00:00
|
|
|
CK_STAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
|
1999-03-15 01:17:26 +00:00
|
|
|
if (ifma->ifma_addr->sa_family != AF_LINK)
|
|
|
|
continue;
|
2004-07-04 18:32:54 +00:00
|
|
|
mc = malloc(sizeof(struct vlan_mc_entry), M_VLAN, M_NOWAIT);
|
2014-08-04 00:58:12 +00:00
|
|
|
if (mc == NULL) {
|
|
|
|
IF_ADDR_WUNLOCK(ifp);
|
2021-06-21 14:46:20 +00:00
|
|
|
CURVNET_RESTORE();
|
2004-07-04 18:32:54 +00:00
|
|
|
return (ENOMEM);
|
2014-08-04 00:58:12 +00:00
|
|
|
}
|
2011-03-21 09:40:01 +00:00
|
|
|
bcopy(ifma->ifma_addr, &mc->mc_addr, ifma->ifma_addr->sa_len);
|
|
|
|
mc->mc_addr.sdl_index = ifp_p->if_index;
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_INSERT_HEAD(&sc->vlan_mc_listhead, mc, mc_entries);
|
2014-08-04 00:58:12 +00:00
|
|
|
}
|
|
|
|
IF_ADDR_WUNLOCK(ifp);
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_FOREACH (mc, &sc->vlan_mc_listhead, mc_entries) {
|
2011-03-21 09:40:01 +00:00
|
|
|
error = if_addmulti(ifp_p, (struct sockaddr *)&mc->mc_addr,
|
2014-08-04 00:58:12 +00:00
|
|
|
NULL);
|
2021-06-21 14:46:20 +00:00
|
|
|
if (error) {
|
|
|
|
CURVNET_RESTORE();
|
2004-07-04 16:43:24 +00:00
|
|
|
return (error);
|
2021-06-21 14:46:20 +00:00
|
|
|
}
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
|
|
|
|
Step 1.5 of importing the network stack virtualization infrastructure
from the vimage project, as per plan established at devsummit 08/08:
http://wiki.freebsd.org/Image/Notes200808DevSummit
Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator
macros, and CURVNET_SET() context setting macros, all currently
resolving to NOPs.
Prepare for virtualization of selected SYSCTL objects by introducing a
family of SYSCTL_V_*() macros, currently resolving to their global
counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT().
Move selected #defines from sys/sys/vimage.h to newly introduced header
files specific to virtualized subsystems (sys/net/vnet.h,
sys/netinet/vinet.h etc.).
All the changes are verified to have zero functional impact at this
point in time by doing MD5 comparision between pre- and post-change
object files(*).
(*) netipsec/keysock.c did not validate depending on compile time options.
Implemented by: julian, bz, brooks, zec
Reviewed by: julian, bz, brooks, kris, rwatson, ...
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation
2008-10-02 15:37:58 +00:00
|
|
|
CURVNET_RESTORE();
|
2004-07-04 16:43:24 +00:00
|
|
|
return (0);
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2022-05-19 06:02:27 +00:00
|
|
|
/*
|
|
|
|
* A handler for interface ifnet events.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
vlan_ifevent(void *arg __unused, struct ifnet *ifp, int event)
|
|
|
|
{
|
|
|
|
struct epoch_tracker et;
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifvlantrunk *trunk;
|
|
|
|
|
|
|
|
if (event != IFNET_EVENT_UPDATE_BAUDRATE)
|
|
|
|
return;
|
|
|
|
|
|
|
|
NET_EPOCH_ENTER(et);
|
|
|
|
trunk = ifp->if_vlantrunk;
|
|
|
|
if (trunk == NULL) {
|
|
|
|
NET_EPOCH_EXIT(et);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
TRUNK_WLOCK(trunk);
|
|
|
|
VLAN_FOREACH(ifv, trunk) {
|
|
|
|
ifv->ifv_ifp->if_baudrate = ifp->if_baudrate;
|
|
|
|
}
|
|
|
|
TRUNK_WUNLOCK(trunk);
|
|
|
|
NET_EPOCH_EXIT(et);
|
|
|
|
}
|
|
|
|
|
2010-01-18 20:34:00 +00:00
|
|
|
/*
|
|
|
|
* A handler for parent interface link layer address changes.
|
|
|
|
* If the parent interface link layer address is changed we
|
|
|
|
* should also change it on all children vlans.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
vlan_iflladdr(void *arg __unused, struct ifnet *ifp)
|
|
|
|
{
|
2019-01-09 01:11:19 +00:00
|
|
|
struct epoch_tracker et;
|
2010-01-18 20:34:00 +00:00
|
|
|
struct ifvlan *ifv;
|
2017-08-15 17:52:37 +00:00
|
|
|
struct ifnet *ifv_ifp;
|
|
|
|
struct ifvlantrunk *trunk;
|
|
|
|
struct sockaddr_dl *sdl;
|
2010-01-18 20:34:00 +00:00
|
|
|
|
2019-01-10 00:37:14 +00:00
|
|
|
/* Need the epoch since this is run on taskqueue_swi. */
|
2019-01-09 01:11:19 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
2017-08-15 17:52:37 +00:00
|
|
|
trunk = ifp->if_vlantrunk;
|
|
|
|
if (trunk == NULL) {
|
2019-01-09 01:11:19 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2010-01-18 20:34:00 +00:00
|
|
|
return;
|
2017-08-15 17:52:37 +00:00
|
|
|
}
|
2010-01-18 20:34:00 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* OK, it's a trunk. Loop over and change all vlan's lladdrs on it.
|
2017-08-15 17:52:37 +00:00
|
|
|
* We need an exclusive lock here to prevent concurrent SIOCSIFLLADDR
|
|
|
|
* ioctl calls on the parent garbling the lladdr of the child vlan.
|
2010-01-18 20:34:00 +00:00
|
|
|
*/
|
2017-08-15 17:52:37 +00:00
|
|
|
TRUNK_WLOCK(trunk);
|
|
|
|
VLAN_FOREACH(ifv, trunk) {
|
|
|
|
/*
|
|
|
|
* Copy new new lladdr into the ifv_ifp, enqueue a task
|
|
|
|
* to actually call if_setlladdr. if_setlladdr needs to
|
|
|
|
* be deferred to a taskqueue because it will call into
|
|
|
|
* the if_vlan ioctl path and try to acquire the global
|
|
|
|
* lock.
|
|
|
|
*/
|
|
|
|
ifv_ifp = ifv->ifv_ifp;
|
|
|
|
bcopy(IF_LLADDR(ifp), IF_LLADDR(ifv_ifp),
|
|
|
|
ifp->if_addrlen);
|
|
|
|
sdl = (struct sockaddr_dl *)ifv_ifp->if_addr->ifa_addr;
|
|
|
|
sdl->sdl_alen = ifp->if_addrlen;
|
|
|
|
taskqueue_enqueue(taskqueue_thread, &ifv->lladdr_task);
|
|
|
|
}
|
|
|
|
TRUNK_WUNLOCK(trunk);
|
2019-01-09 01:11:19 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2010-01-18 20:34:00 +00:00
|
|
|
}
|
|
|
|
|
2006-06-21 07:29:44 +00:00
|
|
|
/*
|
|
|
|
* A handler for network interface departure events.
|
|
|
|
* Track departure of trunks here so that we don't access invalid
|
|
|
|
* pointers or whatever if a trunk is ripped from under us, e.g.,
|
2009-12-29 13:35:18 +00:00
|
|
|
* by ejecting its hot-plug card. However, if an ifnet is simply
|
|
|
|
* being renamed, then there's no need to tear down the state.
|
2006-06-21 07:29:44 +00:00
|
|
|
*/
|
|
|
|
static void
|
|
|
|
vlan_ifdetach(void *arg __unused, struct ifnet *ifp)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
2017-08-15 17:52:37 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
2006-06-21 07:29:44 +00:00
|
|
|
|
2009-12-29 13:35:18 +00:00
|
|
|
/* If the ifnet is just being renamed, don't do anything. */
|
|
|
|
if (ifp->if_flags & IFF_RENAMING)
|
|
|
|
return;
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XLOCK();
|
|
|
|
trunk = ifp->if_vlantrunk;
|
|
|
|
if (trunk == NULL) {
|
|
|
|
VLAN_XUNLOCK();
|
|
|
|
return;
|
|
|
|
}
|
2009-12-29 13:35:18 +00:00
|
|
|
|
2006-06-21 07:29:44 +00:00
|
|
|
/*
|
|
|
|
* OK, it's a trunk. Loop over and detach all vlan's on it.
|
|
|
|
* Check trunk pointer after each vlan_unconfig() as it will
|
|
|
|
* free it and set to NULL after the last vlan was detached.
|
|
|
|
*/
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_FOREACH_UNTIL_SAFE(ifv, ifp->if_vlantrunk,
|
|
|
|
ifp->if_vlantrunk == NULL)
|
|
|
|
vlan_unconfig_locked(ifv->ifv_ifp, 1);
|
|
|
|
|
2006-06-21 07:29:44 +00:00
|
|
|
/* Trunk should have been destroyed in vlan_unconfig(). */
|
|
|
|
KASSERT(ifp->if_vlantrunk == NULL, ("%s: purge failed", __func__));
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XUNLOCK();
|
2006-06-21 07:29:44 +00:00
|
|
|
}
|
|
|
|
|
2011-03-21 09:40:01 +00:00
|
|
|
/*
|
|
|
|
* Return the trunk device for a virtual interface.
|
|
|
|
*/
|
|
|
|
static struct ifnet *
|
|
|
|
vlan_trunkdev(struct ifnet *ifp)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
|
|
|
|
2011-03-21 09:40:01 +00:00
|
|
|
if (ifp->if_type != IFT_L2VLAN)
|
|
|
|
return (NULL);
|
2017-08-15 17:52:37 +00:00
|
|
|
|
2011-03-21 09:40:01 +00:00
|
|
|
ifv = ifp->if_softc;
|
|
|
|
ifp = NULL;
|
|
|
|
if (ifv->ifv_trunk)
|
|
|
|
ifp = PARENT(ifv);
|
|
|
|
return (ifp);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2012-01-12 18:39:37 +00:00
|
|
|
* Return the 12-bit VLAN VID for this interface, for use by external
|
|
|
|
* components such as Infiniband.
|
|
|
|
*
|
|
|
|
* XXXRW: Note that the function name here is historical; it should be named
|
|
|
|
* vlan_vid().
|
2011-03-21 09:40:01 +00:00
|
|
|
*/
|
|
|
|
static int
|
2012-01-12 18:39:37 +00:00
|
|
|
vlan_tag(struct ifnet *ifp, uint16_t *vidp)
|
2011-03-21 09:40:01 +00:00
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
|
|
|
if (ifp->if_type != IFT_L2VLAN)
|
|
|
|
return (EINVAL);
|
|
|
|
ifv = ifp->if_softc;
|
2012-01-12 18:39:37 +00:00
|
|
|
*vidp = ifv->ifv_vid;
|
2011-03-21 09:40:01 +00:00
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
2018-08-16 23:46:38 +00:00
|
|
|
static int
|
|
|
|
vlan_pcp(struct ifnet *ifp, uint16_t *pcpp)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
|
|
|
if (ifp->if_type != IFT_L2VLAN)
|
|
|
|
return (EINVAL);
|
|
|
|
ifv = ifp->if_softc;
|
|
|
|
*pcpp = ifv->ifv_pcp;
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
2011-03-21 09:40:01 +00:00
|
|
|
/*
|
|
|
|
* Return a driver specific cookie for this interface. Synchronization
|
2020-10-21 21:28:20 +00:00
|
|
|
* with setcookie must be provided by the driver.
|
2011-03-21 09:40:01 +00:00
|
|
|
*/
|
|
|
|
static void *
|
|
|
|
vlan_cookie(struct ifnet *ifp)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
|
|
|
if (ifp->if_type != IFT_L2VLAN)
|
|
|
|
return (NULL);
|
|
|
|
ifv = ifp->if_softc;
|
|
|
|
return (ifv->ifv_cookie);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Store a cookie in our softc that drivers can use to store driver
|
|
|
|
* private per-instance data in.
|
|
|
|
*/
|
|
|
|
static int
|
|
|
|
vlan_setcookie(struct ifnet *ifp, void *cookie)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
|
|
|
if (ifp->if_type != IFT_L2VLAN)
|
|
|
|
return (EINVAL);
|
|
|
|
ifv = ifp->if_softc;
|
|
|
|
ifv->ifv_cookie = cookie;
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2012-01-12 18:39:37 +00:00
|
|
|
* Return the vlan device present at the specific VID.
|
2011-03-21 09:40:01 +00:00
|
|
|
*/
|
|
|
|
static struct ifnet *
|
2012-01-12 18:39:37 +00:00
|
|
|
vlan_devat(struct ifnet *ifp, uint16_t vid)
|
2011-03-21 09:40:01 +00:00
|
|
|
{
|
|
|
|
struct ifvlantrunk *trunk;
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
|
|
|
|
2011-03-21 09:40:01 +00:00
|
|
|
trunk = ifp->if_vlantrunk;
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
if (trunk == NULL)
|
2011-03-21 09:40:01 +00:00
|
|
|
return (NULL);
|
|
|
|
ifp = NULL;
|
2012-01-12 18:39:37 +00:00
|
|
|
ifv = vlan_gethash(trunk, vid);
|
2011-03-21 09:40:01 +00:00
|
|
|
if (ifv)
|
|
|
|
ifp = ifv->ifv_ifp;
|
|
|
|
return (ifp);
|
|
|
|
}
|
|
|
|
|
2002-11-14 23:43:16 +00:00
|
|
|
/*
|
|
|
|
* VLAN support can be loaded as a module. The only place in the
|
|
|
|
* system that's intimately aware of this is ether_input. We hook
|
|
|
|
* into this code through vlan_input_p which is defined there and
|
2016-05-03 18:05:43 +00:00
|
|
|
* set here. No one else in the system should be aware of this so
|
2002-11-14 23:43:16 +00:00
|
|
|
* we use an explicit reference here.
|
|
|
|
*/
|
|
|
|
extern void (*vlan_input_p)(struct ifnet *, struct mbuf *);
|
|
|
|
|
2005-04-20 12:16:41 +00:00
|
|
|
/* For if_link_state_change() eyes only... */
|
2009-12-31 20:29:58 +00:00
|
|
|
extern void (*vlan_link_state_p)(struct ifnet *);
|
2004-05-03 13:48:35 +00:00
|
|
|
|
2022-09-22 12:30:09 +00:00
|
|
|
static struct if_clone_addreq vlan_addreq = {
|
|
|
|
.match_f = vlan_clone_match,
|
|
|
|
.create_f = vlan_clone_create,
|
|
|
|
.destroy_f = vlan_clone_destroy,
|
|
|
|
};
|
|
|
|
|
2001-01-31 07:58:58 +00:00
|
|
|
static int
|
2004-07-04 16:43:24 +00:00
|
|
|
vlan_modevent(module_t mod, int type, void *data)
|
|
|
|
{
|
2001-09-05 21:10:28 +00:00
|
|
|
|
2004-07-04 16:43:24 +00:00
|
|
|
switch (type) {
|
|
|
|
case MOD_LOAD:
|
2006-06-21 07:29:44 +00:00
|
|
|
ifdetach_tag = EVENTHANDLER_REGISTER(ifnet_departure_event,
|
|
|
|
vlan_ifdetach, NULL, EVENTHANDLER_PRI_ANY);
|
|
|
|
if (ifdetach_tag == NULL)
|
|
|
|
return (ENOMEM);
|
2010-01-18 20:34:00 +00:00
|
|
|
iflladdr_tag = EVENTHANDLER_REGISTER(iflladdr_event,
|
|
|
|
vlan_iflladdr, NULL, EVENTHANDLER_PRI_ANY);
|
|
|
|
if (iflladdr_tag == NULL)
|
|
|
|
return (ENOMEM);
|
2022-05-19 06:02:27 +00:00
|
|
|
ifevent_tag = EVENTHANDLER_REGISTER(ifnet_event,
|
|
|
|
vlan_ifevent, NULL, EVENTHANDLER_PRI_ANY);
|
|
|
|
if (ifevent_tag == NULL)
|
|
|
|
return (ENOMEM);
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_LOCKING_INIT();
|
2001-09-05 21:10:28 +00:00
|
|
|
vlan_input_p = vlan_input;
|
2004-05-03 13:48:35 +00:00
|
|
|
vlan_link_state_p = vlan_link_state;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
vlan_trunk_cap_p = vlan_trunk_capabilities;
|
2011-03-21 09:40:01 +00:00
|
|
|
vlan_trunkdev_p = vlan_trunkdev;
|
|
|
|
vlan_cookie_p = vlan_cookie;
|
|
|
|
vlan_setcookie_p = vlan_setcookie;
|
|
|
|
vlan_tag_p = vlan_tag;
|
2018-08-16 23:46:38 +00:00
|
|
|
vlan_pcp_p = vlan_pcp;
|
2011-03-21 09:40:01 +00:00
|
|
|
vlan_devat_p = vlan_devat;
|
2010-11-22 23:35:29 +00:00
|
|
|
#ifndef VIMAGE
|
2022-09-22 12:30:09 +00:00
|
|
|
vlan_cloner = ifc_attach_cloner(vlanname, &vlan_addreq);
|
2010-11-22 23:35:29 +00:00
|
|
|
#endif
|
2007-03-12 12:42:14 +00:00
|
|
|
if (bootverbose)
|
|
|
|
printf("vlan: initialized, using "
|
|
|
|
#ifdef VLAN_ARRAY
|
|
|
|
"full-size arrays"
|
|
|
|
#else
|
|
|
|
"hash tables with chaining"
|
|
|
|
#endif
|
2020-10-21 21:28:20 +00:00
|
|
|
|
2007-03-12 12:42:14 +00:00
|
|
|
"\n");
|
2004-07-04 16:43:24 +00:00
|
|
|
break;
|
|
|
|
case MOD_UNLOAD:
|
2010-11-22 23:35:29 +00:00
|
|
|
#ifndef VIMAGE
|
2022-09-22 12:30:09 +00:00
|
|
|
ifc_detach_cloner(vlan_cloner);
|
2010-11-22 23:35:29 +00:00
|
|
|
#endif
|
2006-06-21 07:29:44 +00:00
|
|
|
EVENTHANDLER_DEREGISTER(ifnet_departure_event, ifdetach_tag);
|
2010-01-18 20:34:00 +00:00
|
|
|
EVENTHANDLER_DEREGISTER(iflladdr_event, iflladdr_tag);
|
2022-05-19 06:02:27 +00:00
|
|
|
EVENTHANDLER_DEREGISTER(ifnet_event, ifevent_tag);
|
2001-09-05 21:10:28 +00:00
|
|
|
vlan_input_p = NULL;
|
2004-05-03 13:48:35 +00:00
|
|
|
vlan_link_state_p = NULL;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
vlan_trunk_cap_p = NULL;
|
2011-03-21 09:40:01 +00:00
|
|
|
vlan_trunkdev_p = NULL;
|
|
|
|
vlan_tag_p = NULL;
|
2012-06-19 07:34:13 +00:00
|
|
|
vlan_cookie_p = NULL;
|
|
|
|
vlan_setcookie_p = NULL;
|
2011-03-21 09:40:01 +00:00
|
|
|
vlan_devat_p = NULL;
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_LOCKING_DESTROY();
|
2007-03-12 12:42:14 +00:00
|
|
|
if (bootverbose)
|
|
|
|
printf("vlan: unloaded\n");
|
2001-09-05 21:10:28 +00:00
|
|
|
break;
|
2004-07-15 08:26:07 +00:00
|
|
|
default:
|
|
|
|
return (EOPNOTSUPP);
|
2004-07-04 16:43:24 +00:00
|
|
|
}
|
|
|
|
return (0);
|
|
|
|
}
|
2001-01-31 07:58:58 +00:00
|
|
|
|
2004-07-04 16:43:24 +00:00
|
|
|
static moduledata_t vlan_mod = {
|
|
|
|
"if_vlan",
|
|
|
|
vlan_modevent,
|
2012-10-10 08:36:38 +00:00
|
|
|
0
|
2004-07-04 16:43:24 +00:00
|
|
|
};
|
2001-01-31 07:58:58 +00:00
|
|
|
|
|
|
|
DECLARE_MODULE(if_vlan, vlan_mod, SI_SUB_PSEUDO, SI_ORDER_ANY);
|
2006-02-10 18:38:33 +00:00
|
|
|
MODULE_VERSION(if_vlan, 3);
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2010-11-22 23:35:29 +00:00
|
|
|
#ifdef VIMAGE
|
|
|
|
static void
|
|
|
|
vnet_vlan_init(const void *unused __unused)
|
|
|
|
{
|
2022-09-22 12:30:09 +00:00
|
|
|
vlan_cloner = ifc_attach_cloner(vlanname, &vlan_addreq);
|
2010-11-22 23:35:29 +00:00
|
|
|
V_vlan_cloner = vlan_cloner;
|
|
|
|
}
|
|
|
|
VNET_SYSINIT(vnet_vlan_init, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_ANY,
|
|
|
|
vnet_vlan_init, NULL);
|
|
|
|
|
|
|
|
static void
|
|
|
|
vnet_vlan_uninit(const void *unused __unused)
|
|
|
|
{
|
|
|
|
|
2022-09-22 12:30:09 +00:00
|
|
|
ifc_detach_cloner(V_vlan_cloner);
|
2010-11-22 23:35:29 +00:00
|
|
|
}
|
2020-01-31 22:54:44 +00:00
|
|
|
VNET_SYSUNINIT(vnet_vlan_uninit, SI_SUB_INIT_IF, SI_ORDER_ANY,
|
2010-11-22 23:35:29 +00:00
|
|
|
vnet_vlan_uninit, NULL);
|
|
|
|
#endif
|
|
|
|
|
2015-03-20 20:42:58 +00:00
|
|
|
/*
|
2020-10-21 21:28:20 +00:00
|
|
|
* Check for <etherif>.<vlan>[.<vlan> ...] style interface names.
|
2015-03-20 20:42:58 +00:00
|
|
|
*/
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
static struct ifnet *
|
2015-03-20 20:42:58 +00:00
|
|
|
vlan_clone_match_ethervid(const char *name, int *vidp)
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
{
|
2015-03-20 20:42:58 +00:00
|
|
|
char ifname[IFNAMSIZ];
|
|
|
|
char *cp;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
struct ifnet *ifp;
|
2012-01-12 18:39:37 +00:00
|
|
|
int vid;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2015-03-20 20:42:58 +00:00
|
|
|
strlcpy(ifname, name, IFNAMSIZ);
|
2020-10-21 21:28:20 +00:00
|
|
|
if ((cp = strrchr(ifname, '.')) == NULL)
|
2015-03-20 20:42:58 +00:00
|
|
|
return (NULL);
|
|
|
|
*cp = '\0';
|
2017-05-23 00:13:27 +00:00
|
|
|
if ((ifp = ifunit_ref(ifname)) == NULL)
|
2015-03-20 20:42:58 +00:00
|
|
|
return (NULL);
|
|
|
|
/* Parse VID. */
|
2017-05-23 00:13:27 +00:00
|
|
|
if (*++cp == '\0') {
|
|
|
|
if_rele(ifp);
|
2015-03-20 20:42:58 +00:00
|
|
|
return (NULL);
|
2017-05-23 00:13:27 +00:00
|
|
|
}
|
2015-03-20 20:42:58 +00:00
|
|
|
vid = 0;
|
|
|
|
for(; *cp >= '0' && *cp <= '9'; cp++)
|
|
|
|
vid = (vid * 10) + (*cp - '0');
|
2017-05-23 00:13:27 +00:00
|
|
|
if (*cp != '\0') {
|
|
|
|
if_rele(ifp);
|
2015-03-20 20:42:58 +00:00
|
|
|
return (NULL);
|
2017-05-23 00:13:27 +00:00
|
|
|
}
|
2015-03-20 20:42:58 +00:00
|
|
|
if (vidp != NULL)
|
|
|
|
*vidp = vid;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2004-07-04 16:43:24 +00:00
|
|
|
return (ifp);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
}
|
|
|
|
|
2001-09-05 21:10:28 +00:00
|
|
|
static int
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
vlan_clone_match(struct if_clone *ifc, const char *name)
|
2001-09-05 21:10:28 +00:00
|
|
|
{
|
2020-01-29 18:41:35 +00:00
|
|
|
struct ifnet *ifp;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
const char *cp;
|
|
|
|
|
2020-01-29 18:41:35 +00:00
|
|
|
ifp = vlan_clone_match_ethervid(name, NULL);
|
|
|
|
if (ifp != NULL) {
|
|
|
|
if_rele(ifp);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
return (1);
|
2020-01-29 18:41:35 +00:00
|
|
|
}
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2012-10-16 13:37:54 +00:00
|
|
|
if (strncmp(vlanname, name, strlen(vlanname)) != 0)
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
return (0);
|
|
|
|
for (cp = name + 4; *cp != '\0'; cp++) {
|
|
|
|
if (*cp < '0' || *cp > '9')
|
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
|
|
|
return (1);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
2022-09-22 12:30:09 +00:00
|
|
|
vlan_clone_create(struct if_clone *ifc, char *name, size_t len,
|
|
|
|
struct ifc_data *ifd, struct ifnet **ifpp)
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
{
|
|
|
|
char *dp;
|
2021-01-26 07:22:23 +00:00
|
|
|
bool wildcard = false;
|
|
|
|
bool subinterface = false;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
int unit;
|
|
|
|
int error;
|
2021-01-26 07:22:23 +00:00
|
|
|
int vid = 0;
|
|
|
|
uint16_t proto = ETHERTYPE_VLAN;
|
2001-09-05 21:10:28 +00:00
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifnet *ifp;
|
2021-01-26 07:22:23 +00:00
|
|
|
struct ifnet *p = NULL;
|
2010-08-06 15:15:26 +00:00
|
|
|
struct ifaddr *ifa;
|
|
|
|
struct sockaddr_dl *sdl;
|
2006-07-09 06:04:01 +00:00
|
|
|
struct vlanreq vlr;
|
2007-03-19 16:58:07 +00:00
|
|
|
static const u_char eaddr[ETHER_ADDR_LEN]; /* 00:00:00:00:00:00 */
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2020-10-21 21:28:20 +00:00
|
|
|
|
2006-07-09 06:04:01 +00:00
|
|
|
/*
|
2021-01-26 07:22:23 +00:00
|
|
|
* There are three ways to specify the cloned device:
|
2006-07-09 06:04:01 +00:00
|
|
|
* o pass a parameter block with the clone request.
|
2021-01-26 07:22:23 +00:00
|
|
|
* o specify parameters in the text of the clone device name
|
2006-07-09 06:04:01 +00:00
|
|
|
* o specify no parameters and get an unattached device that
|
|
|
|
* must be configured separately.
|
2021-01-26 07:22:23 +00:00
|
|
|
* The first technique is preferred; the latter two are supported
|
2020-10-21 21:28:20 +00:00
|
|
|
* for backwards compatibility.
|
2012-01-12 18:39:37 +00:00
|
|
|
*
|
|
|
|
* XXXRW: Note historic use of the word "tag" here. New ioctls may be
|
|
|
|
* called for.
|
2006-07-09 06:04:01 +00:00
|
|
|
*/
|
2021-01-26 07:22:23 +00:00
|
|
|
|
2022-09-22 12:30:09 +00:00
|
|
|
if (ifd->params != NULL) {
|
|
|
|
error = ifc_copyin(ifd, &vlr, sizeof(vlr));
|
2006-07-09 06:04:01 +00:00
|
|
|
if (error)
|
|
|
|
return error;
|
2021-01-26 07:22:23 +00:00
|
|
|
vid = vlr.vlr_tag;
|
|
|
|
proto = vlr.vlr_proto;
|
|
|
|
|
2021-04-11 16:47:03 +00:00
|
|
|
#ifdef COMPAT_FREEBSD12
|
|
|
|
if (proto == 0)
|
|
|
|
proto = ETHERTYPE_VLAN;
|
|
|
|
#endif
|
2017-05-23 00:13:27 +00:00
|
|
|
p = ifunit_ref(vlr.vlr_parent);
|
2006-07-09 06:04:01 +00:00
|
|
|
if (p == NULL)
|
2015-03-20 21:09:03 +00:00
|
|
|
return (ENXIO);
|
2021-01-26 07:22:23 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if ((error = ifc_name2unit(name, &unit)) == 0) {
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2021-01-26 07:22:23 +00:00
|
|
|
/*
|
|
|
|
* vlanX interface. Set wildcard to true if the unit number
|
|
|
|
* is not fixed (-1)
|
|
|
|
*/
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
wildcard = (unit < 0);
|
2021-01-26 07:22:23 +00:00
|
|
|
} else {
|
|
|
|
struct ifnet *p_tmp = vlan_clone_match_ethervid(name, &vid);
|
|
|
|
if (p_tmp != NULL) {
|
|
|
|
error = 0;
|
|
|
|
subinterface = true;
|
|
|
|
unit = IF_DUNIT_NONE;
|
|
|
|
wildcard = false;
|
|
|
|
if (p != NULL) {
|
|
|
|
if_rele(p_tmp);
|
|
|
|
if (p != p_tmp)
|
|
|
|
error = EINVAL;
|
|
|
|
} else
|
|
|
|
p = p_tmp;
|
|
|
|
} else
|
|
|
|
error = ENXIO;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
}
|
|
|
|
|
2017-05-23 00:13:27 +00:00
|
|
|
if (error != 0) {
|
|
|
|
if (p != NULL)
|
|
|
|
if_rele(p);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
return (error);
|
2017-05-23 00:13:27 +00:00
|
|
|
}
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2021-01-26 07:22:23 +00:00
|
|
|
if (!subinterface) {
|
|
|
|
/* vlanX interface, mark X as busy or allocate new unit # */
|
|
|
|
error = ifc_alloc_unit(ifc, &unit);
|
|
|
|
if (error != 0) {
|
|
|
|
if (p != NULL)
|
|
|
|
if_rele(p);
|
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
/* In the wildcard case, we need to update the name. */
|
|
|
|
if (wildcard) {
|
|
|
|
for (dp = name; *dp != '\0'; dp++);
|
|
|
|
if (snprintf(dp, len - (dp-name), "%d", unit) >
|
|
|
|
len - (dp-name) - 1) {
|
|
|
|
panic("%s: interface name too long", __func__);
|
|
|
|
}
|
|
|
|
}
|
2001-09-05 21:10:28 +00:00
|
|
|
|
2003-02-19 05:47:46 +00:00
|
|
|
ifv = malloc(sizeof(struct ifvlan), M_VLAN, M_WAITOK | M_ZERO);
|
2005-06-10 16:49:24 +00:00
|
|
|
ifp = ifv->ifv_ifp = if_alloc(IFT_ETHER);
|
|
|
|
if (ifp == NULL) {
|
2021-01-26 07:22:23 +00:00
|
|
|
if (!subinterface)
|
|
|
|
ifc_free_unit(ifc, unit);
|
2005-06-10 16:49:24 +00:00
|
|
|
free(ifv, M_VLAN);
|
2017-05-23 00:13:27 +00:00
|
|
|
if (p != NULL)
|
|
|
|
if_rele(p);
|
2005-06-10 16:49:24 +00:00
|
|
|
return (ENOSPC);
|
|
|
|
}
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_INIT(&ifv->vlan_mc_listhead);
|
2001-09-05 21:10:28 +00:00
|
|
|
ifp->if_softc = ifv;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
/*
|
2005-01-24 15:48:00 +00:00
|
|
|
* Set the name manually rather than using if_initname because
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
* we don't conform to the default naming convention for interfaces.
|
|
|
|
*/
|
|
|
|
strlcpy(ifp->if_xname, name, IFNAMSIZ);
|
2012-10-16 13:37:54 +00:00
|
|
|
ifp->if_dname = vlanname;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
ifp->if_dunit = unit;
|
2001-09-05 21:10:28 +00:00
|
|
|
|
2006-06-29 07:30:39 +00:00
|
|
|
ifp->if_init = vlan_init;
|
2021-08-21 09:41:03 +00:00
|
|
|
#ifdef ALTQ
|
|
|
|
ifp->if_start = vlan_altq_start;
|
|
|
|
ifp->if_transmit = vlan_altq_transmit;
|
|
|
|
IFQ_SET_MAXLEN(&ifp->if_snd, ifqmaxlen);
|
|
|
|
ifp->if_snd.ifq_drv_maxlen = 0;
|
|
|
|
IFQ_SET_READY(&ifp->if_snd);
|
|
|
|
#else
|
2011-11-28 19:35:08 +00:00
|
|
|
ifp->if_transmit = vlan_transmit;
|
2021-08-21 09:41:03 +00:00
|
|
|
#endif
|
2011-11-28 19:35:08 +00:00
|
|
|
ifp->if_qflush = vlan_qflush;
|
2001-09-05 21:10:28 +00:00
|
|
|
ifp->if_ioctl = vlan_ioctl;
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
#if defined(KERN_TLS) || defined(RATELIMIT)
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
ifp->if_snd_tag_alloc = vlan_snd_tag_alloc;
|
2021-01-26 16:54:42 +00:00
|
|
|
ifp->if_ratelimit_query = vlan_ratelimit_query;
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
#endif
|
2006-01-31 16:41:05 +00:00
|
|
|
ifp->if_flags = VLAN_IFFLAGS;
|
2005-06-10 16:49:24 +00:00
|
|
|
ether_ifattach(ifp, eaddr);
|
2001-09-05 21:10:28 +00:00
|
|
|
/* Now undo some of the damage... */
|
2001-10-15 19:21:01 +00:00
|
|
|
ifp->if_baudrate = 0;
|
2002-11-14 23:43:16 +00:00
|
|
|
ifp->if_type = IFT_L2VLAN;
|
|
|
|
ifp->if_hdrlen = ETHER_VLAN_ENCAP_LEN;
|
2010-08-06 15:15:26 +00:00
|
|
|
ifa = ifp->if_addr;
|
|
|
|
sdl = (struct sockaddr_dl *)ifa->ifa_addr;
|
|
|
|
sdl->sdl_type = IFT_L2VLAN;
|
2001-09-05 21:10:28 +00:00
|
|
|
|
2017-05-23 00:13:27 +00:00
|
|
|
if (p != NULL) {
|
2020-10-21 21:28:20 +00:00
|
|
|
error = vlan_config(ifv, p, vid, proto);
|
2017-05-23 00:13:27 +00:00
|
|
|
if_rele(p);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
if (error != 0) {
|
|
|
|
/*
|
2012-08-20 16:00:33 +00:00
|
|
|
* Since we've partially failed, we need to back
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
* out all the way, otherwise userland could get
|
|
|
|
* confused. Thus, we destroy the interface.
|
|
|
|
*/
|
|
|
|
ether_ifdetach(ifp);
|
2006-06-29 07:52:30 +00:00
|
|
|
vlan_unconfig(ifp);
|
2011-11-11 22:57:52 +00:00
|
|
|
if_free(ifp);
|
2021-01-26 07:22:23 +00:00
|
|
|
if (!subinterface)
|
|
|
|
ifc_free_unit(ifc, unit);
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
free(ifv, M_VLAN);
|
|
|
|
|
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
}
|
2022-09-22 12:30:09 +00:00
|
|
|
*ifpp = ifp;
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
|
2001-09-05 21:10:28 +00:00
|
|
|
return (0);
|
|
|
|
}
|
|
|
|
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
static int
|
2022-09-22 12:30:09 +00:00
|
|
|
vlan_clone_destroy(struct if_clone *ifc, struct ifnet *ifp, uint32_t flags)
|
2001-09-05 21:10:28 +00:00
|
|
|
{
|
|
|
|
struct ifvlan *ifv = ifp->if_softc;
|
2021-01-26 07:22:23 +00:00
|
|
|
int unit = ifp->if_dunit;
|
2020-10-21 21:28:20 +00:00
|
|
|
|
|
|
|
if (ifp->if_vlantrunk)
|
|
|
|
return (EBUSY);
|
2004-07-22 22:50:15 +00:00
|
|
|
|
2021-08-21 09:41:03 +00:00
|
|
|
#ifdef ALTQ
|
|
|
|
IFQ_PURGE(&ifp->if_snd);
|
|
|
|
#endif
|
2006-06-29 07:52:30 +00:00
|
|
|
ether_ifdetach(ifp); /* first, remove it from system-wide lists */
|
|
|
|
vlan_unconfig(ifp); /* now it can be unconfigured and freed */
|
2017-08-15 17:52:37 +00:00
|
|
|
/*
|
|
|
|
* We should have the only reference to the ifv now, so we can now
|
|
|
|
* drain any remaining lladdr task before freeing the ifnet and the
|
|
|
|
* ifvlan.
|
|
|
|
*/
|
|
|
|
taskqueue_drain(taskqueue_thread, &ifv->lladdr_task);
|
2018-09-21 01:37:08 +00:00
|
|
|
NET_EPOCH_WAIT();
|
2011-11-11 22:57:52 +00:00
|
|
|
if_free(ifp);
|
2001-09-05 21:10:28 +00:00
|
|
|
free(ifv, M_VLAN);
|
2021-01-26 07:22:23 +00:00
|
|
|
if (unit != IF_DUNIT_NONE)
|
|
|
|
ifc_free_unit(ifc, unit);
|
2004-07-22 22:50:15 +00:00
|
|
|
|
Major overhaul of pseudo-interface cloning. Highlights include:
- Split the code out into if_clone.[ch].
- Locked struct if_clone. [1]
- Add a per-cloner match function rather then simply matching names of
the form <name><unit> and <name>.
- Use the match function to allow creation of <interface>.<tag>
vlan interfaces. The old way is preserved unchanged!
- Also the match function to allow creation of stf(4) interfaces named
stf0, stf, or 6to4. This is the only major user visible change in
that "ifconfig stf" creates the interface stf rather then stf0 and
does not print "stf0" to stdout.
- Allow destroy functions to fail so they can refuse to delete
interfaces. Currently, we forbid the deletion of interfaces which
were created in the init function, particularly lo0, pflog0, and
pfsync0. In the case of lo0 this was a panic implementation so it
does not count as a user visiable change. :-)
- Since most interfaces do not need the new functionality, an family of
wrapper functions, ifc_simple_*(), were created to wrap old style
cloner functions.
- The IF_CLONE_INITIALIZER macro is replaced with a new incompatible
IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE
instead.
Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1]
Reviewed by: andre, mlaier
Discussed on: net
2004-06-22 20:13:25 +00:00
|
|
|
return (0);
|
2001-09-05 21:10:28 +00:00
|
|
|
}
|
|
|
|
|
2004-07-04 16:43:24 +00:00
|
|
|
/*
|
|
|
|
* The ifp->if_init entry point for vlan(4) is a no-op.
|
|
|
|
*/
|
1998-03-18 01:40:12 +00:00
|
|
|
static void
|
2006-06-29 07:30:39 +00:00
|
|
|
vlan_init(void *foo __unused)
|
1998-03-18 01:40:12 +00:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2005-11-06 19:43:04 +00:00
|
|
|
/*
|
2011-11-28 19:35:08 +00:00
|
|
|
* The if_transmit method for vlan(4) interface.
|
2005-11-06 19:43:04 +00:00
|
|
|
*/
|
2011-11-28 19:35:08 +00:00
|
|
|
static int
|
|
|
|
vlan_transmit(struct ifnet *ifp, struct mbuf *m)
|
1998-03-18 01:40:12 +00:00
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifnet *p;
|
2011-12-29 18:40:58 +00:00
|
|
|
int error, len, mcast;
|
1998-03-18 01:40:12 +00:00
|
|
|
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
ifv = ifp->if_softc;
|
2017-08-15 17:52:37 +00:00
|
|
|
if (TRUNK(ifv) == NULL) {
|
|
|
|
if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
|
|
|
|
m_freem(m);
|
|
|
|
return (ENETDOWN);
|
|
|
|
}
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
p = PARENT(ifv);
|
2011-12-29 18:40:58 +00:00
|
|
|
len = m->m_pkthdr.len;
|
|
|
|
mcast = (m->m_flags & (M_MCAST | M_BCAST)) ? 1 : 0;
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2011-11-28 19:35:08 +00:00
|
|
|
BPF_MTAP(ifp, m);
|
1998-03-18 01:40:12 +00:00
|
|
|
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
#if defined(KERN_TLS) || defined(RATELIMIT)
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
if (m->m_pkthdr.csum_flags & CSUM_SND_TAG) {
|
|
|
|
struct vlan_snd_tag *vst;
|
|
|
|
struct m_snd_tag *mst;
|
|
|
|
|
|
|
|
MPASS(m->m_pkthdr.snd_tag->ifp == ifp);
|
|
|
|
mst = m->m_pkthdr.snd_tag;
|
|
|
|
vst = mst_to_vst(mst);
|
|
|
|
if (vst->tag->ifp != p) {
|
|
|
|
if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
|
|
|
|
m_freem(m);
|
|
|
|
return (EAGAIN);
|
|
|
|
}
|
|
|
|
|
|
|
|
m->m_pkthdr.snd_tag = m_snd_tag_ref(vst->tag);
|
|
|
|
m_snd_tag_rele(mst);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2011-11-28 19:35:08 +00:00
|
|
|
/*
|
|
|
|
* Do not run parent's if_transmit() if the parent is not up,
|
|
|
|
* or parent's driver will cause a system crash.
|
|
|
|
*/
|
|
|
|
if (!UP_AND_RUNNING(p)) {
|
2015-03-20 14:05:17 +00:00
|
|
|
if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
|
2017-08-15 17:52:37 +00:00
|
|
|
m_freem(m);
|
2013-06-16 04:40:02 +00:00
|
|
|
return (ENETDOWN);
|
2011-11-28 19:35:08 +00:00
|
|
|
}
|
2001-03-28 15:52:12 +00:00
|
|
|
|
2020-10-21 21:28:20 +00:00
|
|
|
if (!ether_8021q_frame(&m, ifp, p, &ifv->ifv_qtag)) {
|
2018-03-27 15:29:32 +00:00
|
|
|
if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
|
|
|
|
return (0);
|
1998-03-18 01:40:12 +00:00
|
|
|
}
|
2011-11-28 19:35:08 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Send it, precisely as ether_output() would have.
|
|
|
|
*/
|
|
|
|
error = (p->if_transmit)(p, m);
|
2014-09-13 18:13:08 +00:00
|
|
|
if (error == 0) {
|
2015-03-20 14:05:17 +00:00
|
|
|
if_inc_counter(ifp, IFCOUNTER_OPACKETS, 1);
|
|
|
|
if_inc_counter(ifp, IFCOUNTER_OBYTES, len);
|
|
|
|
if_inc_counter(ifp, IFCOUNTER_OMCASTS, mcast);
|
2011-12-29 18:40:58 +00:00
|
|
|
} else
|
2015-03-20 14:05:17 +00:00
|
|
|
if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
|
2011-11-28 19:35:08 +00:00
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
|
2019-08-30 20:19:43 +00:00
|
|
|
static int
|
|
|
|
vlan_output(struct ifnet *ifp, struct mbuf *m, const struct sockaddr *dst,
|
|
|
|
struct route *ro)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifnet *p;
|
|
|
|
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
|
|
|
|
2020-10-21 21:28:20 +00:00
|
|
|
/*
|
|
|
|
* Find the first non-VLAN parent interface.
|
|
|
|
*/
|
2019-08-30 20:19:43 +00:00
|
|
|
ifv = ifp->if_softc;
|
2020-10-21 21:28:20 +00:00
|
|
|
do {
|
|
|
|
if (TRUNK(ifv) == NULL) {
|
|
|
|
m_freem(m);
|
|
|
|
return (ENETDOWN);
|
|
|
|
}
|
|
|
|
p = PARENT(ifv);
|
|
|
|
ifv = p->if_softc;
|
|
|
|
} while (p->if_type == IFT_L2VLAN);
|
|
|
|
|
2019-08-30 20:19:43 +00:00
|
|
|
return p->if_output(ifp, m, dst, ro);
|
|
|
|
}
|
|
|
|
|
2021-08-21 09:41:03 +00:00
|
|
|
#ifdef ALTQ
|
|
|
|
static void
|
|
|
|
vlan_altq_start(if_t ifp)
|
|
|
|
{
|
|
|
|
struct ifaltq *ifq = &ifp->if_snd;
|
|
|
|
struct mbuf *m;
|
|
|
|
|
|
|
|
IFQ_LOCK(ifq);
|
|
|
|
IFQ_DEQUEUE_NOLOCK(ifq, m);
|
|
|
|
while (m != NULL) {
|
|
|
|
vlan_transmit(ifp, m);
|
|
|
|
IFQ_DEQUEUE_NOLOCK(ifq, m);
|
|
|
|
}
|
|
|
|
IFQ_UNLOCK(ifq);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
|
|
|
vlan_altq_transmit(if_t ifp, struct mbuf *m)
|
|
|
|
{
|
|
|
|
int err;
|
|
|
|
|
|
|
|
if (ALTQ_IS_ENABLED(&ifp->if_snd)) {
|
|
|
|
IFQ_ENQUEUE(&ifp->if_snd, m, err);
|
|
|
|
if (err == 0)
|
|
|
|
vlan_altq_start(ifp);
|
|
|
|
} else
|
|
|
|
err = vlan_transmit(ifp, m);
|
|
|
|
|
|
|
|
return (err);
|
|
|
|
}
|
|
|
|
#endif /* ALTQ */
|
|
|
|
|
2011-11-28 19:35:08 +00:00
|
|
|
/*
|
|
|
|
* The ifp->if_qflush entry point for vlan(4) is a no-op.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
vlan_qflush(struct ifnet *ifp __unused)
|
|
|
|
{
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
|
|
|
|
2002-11-14 23:43:16 +00:00
|
|
|
static void
|
|
|
|
vlan_input(struct ifnet *ifp, struct mbuf *m)
|
1999-03-15 01:17:26 +00:00
|
|
|
{
|
2017-08-15 17:52:37 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
1999-03-15 01:17:26 +00:00
|
|
|
struct ifvlan *ifv;
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
struct m_tag *mtag;
|
|
|
|
uint16_t vid, tag;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
trunk = ifp->if_vlantrunk;
|
|
|
|
if (trunk == NULL) {
|
|
|
|
m_freem(m);
|
|
|
|
return;
|
|
|
|
}
|
1999-03-15 01:17:26 +00:00
|
|
|
|
2005-09-16 11:44:43 +00:00
|
|
|
if (m->m_flags & M_VLANTAG) {
|
2002-11-14 23:43:16 +00:00
|
|
|
/*
|
2005-09-16 11:58:58 +00:00
|
|
|
* Packet is tagged, but m contains a normal
|
2002-11-14 23:43:16 +00:00
|
|
|
* Ethernet frame; the tag is stored out-of-band.
|
|
|
|
*/
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
tag = m->m_pkthdr.ether_vtag;
|
2005-02-18 22:31:19 +00:00
|
|
|
m->m_flags &= ~M_VLANTAG;
|
2002-11-14 23:43:16 +00:00
|
|
|
} else {
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
struct ether_vlan_header *evl;
|
|
|
|
|
2005-09-16 11:58:58 +00:00
|
|
|
/*
|
|
|
|
* Packet is tagged in-band as specified by 802.1q.
|
|
|
|
*/
|
2002-11-14 23:43:16 +00:00
|
|
|
switch (ifp->if_type) {
|
|
|
|
case IFT_ETHER:
|
2004-07-04 16:43:24 +00:00
|
|
|
if (m->m_len < sizeof(*evl) &&
|
|
|
|
(m = m_pullup(m, sizeof(*evl))) == NULL) {
|
2002-11-14 23:43:16 +00:00
|
|
|
if_printf(ifp, "cannot pullup VLAN header\n");
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
evl = mtod(m, struct ether_vlan_header *);
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
tag = ntohs(evl->evl_tag);
|
2006-08-03 09:50:15 +00:00
|
|
|
|
|
|
|
/*
|
2006-12-30 21:10:25 +00:00
|
|
|
* Remove the 802.1q header by copying the Ethernet
|
|
|
|
* addresses over it and adjusting the beginning of
|
|
|
|
* the data in the mbuf. The encapsulated Ethernet
|
|
|
|
* type field is already in place.
|
2006-08-03 09:50:15 +00:00
|
|
|
*/
|
2006-12-30 21:10:25 +00:00
|
|
|
bcopy((char *)evl, (char *)evl + ETHER_VLAN_ENCAP_LEN,
|
|
|
|
ETHER_HDR_LEN - ETHER_TYPE_LEN);
|
|
|
|
m_adj(m, ETHER_VLAN_ENCAP_LEN);
|
2002-11-14 23:43:16 +00:00
|
|
|
break;
|
2006-12-30 21:10:25 +00:00
|
|
|
|
2002-11-14 23:43:16 +00:00
|
|
|
default:
|
2006-08-03 09:50:15 +00:00
|
|
|
#ifdef INVARIANTS
|
2006-08-03 09:59:08 +00:00
|
|
|
panic("%s: %s has unsupported if_type %u",
|
|
|
|
__func__, ifp->if_xname, ifp->if_type);
|
2002-11-14 23:43:16 +00:00
|
|
|
#endif
|
2014-09-19 10:39:58 +00:00
|
|
|
if_inc_counter(ifp, IFCOUNTER_NOPROTO, 1);
|
2017-08-15 17:52:37 +00:00
|
|
|
m_freem(m);
|
2006-08-03 09:59:08 +00:00
|
|
|
return;
|
2002-11-14 23:43:16 +00:00
|
|
|
}
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
|
|
|
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
vid = EVL_VLANOFTAG(tag);
|
|
|
|
|
2012-01-12 18:39:37 +00:00
|
|
|
ifv = vlan_gethash(trunk, vid);
|
2006-12-30 21:10:25 +00:00
|
|
|
if (ifv == NULL || !UP_AND_RUNNING(ifv->ifv_ifp)) {
|
2018-09-21 01:37:08 +00:00
|
|
|
if_inc_counter(ifp, IFCOUNTER_NOPROTO, 1);
|
2017-08-15 17:52:37 +00:00
|
|
|
m_freem(m);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
return;
|
|
|
|
}
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2022-02-14 19:09:54 +00:00
|
|
|
if (V_vlan_mtag_pcp) {
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
/*
|
|
|
|
* While uncommon, it is possible that we will find a 802.1q
|
|
|
|
* packet encapsulated inside another packet that also had an
|
|
|
|
* 802.1q header. For example, ethernet tunneled over IPSEC
|
|
|
|
* arriving over ethernet. In that case, we replace the
|
|
|
|
* existing 802.1q PCP m_tag value.
|
|
|
|
*/
|
|
|
|
mtag = m_tag_locate(m, MTAG_8021Q, MTAG_8021Q_PCP_IN, NULL);
|
|
|
|
if (mtag == NULL) {
|
|
|
|
mtag = m_tag_alloc(MTAG_8021Q, MTAG_8021Q_PCP_IN,
|
|
|
|
sizeof(uint8_t), M_NOWAIT);
|
|
|
|
if (mtag == NULL) {
|
|
|
|
if_inc_counter(ifp, IFCOUNTER_IERRORS, 1);
|
2017-08-15 17:52:37 +00:00
|
|
|
m_freem(m);
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
m_tag_prepend(m, mtag);
|
|
|
|
}
|
|
|
|
*(uint8_t *)(mtag + 1) = EVL_PRIOFTAG(tag);
|
|
|
|
}
|
|
|
|
|
2005-06-10 16:49:24 +00:00
|
|
|
m->m_pkthdr.rcvif = ifv->ifv_ifp;
|
2015-03-25 16:01:46 +00:00
|
|
|
if_inc_counter(ifv->ifv_ifp, IFCOUNTER_IPACKETS, 1);
|
2002-11-14 23:43:16 +00:00
|
|
|
|
|
|
|
/* Pass it back through the parent's input routine. */
|
2017-09-13 00:25:09 +00:00
|
|
|
(*ifv->ifv_ifp->if_input)(ifv->ifv_ifp, m);
|
1998-03-18 01:40:12 +00:00
|
|
|
}
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
static void
|
|
|
|
vlan_lladdr_fn(void *arg, int pending __unused)
|
|
|
|
{
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifnet *ifp;
|
|
|
|
|
|
|
|
ifv = (struct ifvlan *)arg;
|
|
|
|
ifp = ifv->ifv_ifp;
|
2018-10-21 16:51:35 +00:00
|
|
|
|
|
|
|
CURVNET_SET(ifp->if_vnet);
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
/* The ifv_ifp already has the lladdr copied in. */
|
|
|
|
if_setlladdr(ifp, IF_LLADDR(ifp), ifp->if_addrlen);
|
2018-10-21 16:51:35 +00:00
|
|
|
|
|
|
|
CURVNET_RESTORE();
|
2017-08-15 17:52:37 +00:00
|
|
|
}
|
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
static int
|
2020-10-21 21:28:20 +00:00
|
|
|
vlan_config(struct ifvlan *ifv, struct ifnet *p, uint16_t vid,
|
|
|
|
uint16_t proto)
|
1998-03-18 01:40:12 +00:00
|
|
|
{
|
2019-10-13 15:15:09 +00:00
|
|
|
struct epoch_tracker et;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
2005-10-03 02:24:21 +00:00
|
|
|
struct ifnet *ifp;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
int error = 0;
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2015-03-20 21:09:03 +00:00
|
|
|
/*
|
|
|
|
* We can handle non-ethernet hardware types as long as
|
|
|
|
* they handle the tagging and headers themselves.
|
|
|
|
*/
|
2011-03-21 09:40:01 +00:00
|
|
|
if (p->if_type != IFT_ETHER &&
|
2020-10-21 21:28:20 +00:00
|
|
|
p->if_type != IFT_L2VLAN &&
|
2011-03-21 09:40:01 +00:00
|
|
|
(p->if_capenable & IFCAP_VLAN_HWTAGGING) == 0)
|
2004-07-04 16:43:24 +00:00
|
|
|
return (EPROTONOSUPPORT);
|
2006-01-31 16:41:05 +00:00
|
|
|
if ((p->if_flags & VLAN_IFFLAGS) != VLAN_IFFLAGS)
|
|
|
|
return (EPROTONOSUPPORT);
|
2015-03-20 21:09:03 +00:00
|
|
|
/*
|
|
|
|
* Don't let the caller set up a VLAN VID with
|
|
|
|
* anything except VLID bits.
|
|
|
|
* VID numbers 0x0 and 0xFFF are reserved.
|
|
|
|
*/
|
|
|
|
if (vid == 0 || vid == 0xFFF || (vid & ~EVL_VLID_MASK))
|
|
|
|
return (EINVAL);
|
2022-07-18 22:23:50 +00:00
|
|
|
if (ifv->ifv_trunk) {
|
|
|
|
trunk = ifv->ifv_trunk;
|
|
|
|
if (trunk->parent != p)
|
|
|
|
return (EBUSY);
|
|
|
|
|
|
|
|
VLAN_XLOCK();
|
|
|
|
|
|
|
|
ifv->ifv_proto = proto;
|
|
|
|
|
|
|
|
if (ifv->ifv_vid != vid) {
|
|
|
|
/* Re-hash */
|
|
|
|
vlan_remhash(trunk, ifv);
|
|
|
|
ifv->ifv_vid = vid;
|
|
|
|
error = vlan_inshash(trunk, ifv);
|
|
|
|
}
|
|
|
|
/* Will unlock */
|
|
|
|
goto done;
|
|
|
|
}
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XLOCK();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (p->if_vlantrunk == NULL) {
|
|
|
|
trunk = malloc(sizeof(struct ifvlantrunk),
|
|
|
|
M_VLAN, M_WAITOK | M_ZERO);
|
2006-02-02 22:11:38 +00:00
|
|
|
vlan_inithash(trunk);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
TRUNK_LOCK_INIT(trunk);
|
2017-08-15 17:52:37 +00:00
|
|
|
TRUNK_WLOCK(trunk);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
p->if_vlantrunk = trunk;
|
|
|
|
trunk->parent = p;
|
2017-05-23 00:13:27 +00:00
|
|
|
if_ref(trunk->parent);
|
2018-09-21 01:37:08 +00:00
|
|
|
TRUNK_WUNLOCK(trunk);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
} else {
|
|
|
|
trunk = p->if_vlantrunk;
|
|
|
|
}
|
|
|
|
|
2012-01-12 18:39:37 +00:00
|
|
|
ifv->ifv_vid = vid; /* must set this before vlan_inshash() */
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
ifv->ifv_pcp = 0; /* Default: best effort delivery. */
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
error = vlan_inshash(trunk, ifv);
|
|
|
|
if (error)
|
|
|
|
goto done;
|
2020-10-21 21:28:20 +00:00
|
|
|
ifv->ifv_proto = proto;
|
2002-11-14 23:43:16 +00:00
|
|
|
ifv->ifv_encaplen = ETHER_VLAN_ENCAP_LEN;
|
|
|
|
ifv->ifv_mintu = ETHERMIN;
|
2005-10-03 02:24:21 +00:00
|
|
|
ifv->ifv_pflags = 0;
|
2017-04-28 11:00:58 +00:00
|
|
|
ifv->ifv_capenable = -1;
|
2002-11-14 23:43:16 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If the parent supports the VLAN_MTU capability,
|
|
|
|
* i.e. can Tx/Rx larger than ETHER_MAX_LEN frames,
|
2004-05-25 14:30:12 +00:00
|
|
|
* use it.
|
2002-11-14 23:43:16 +00:00
|
|
|
*/
|
2004-05-25 14:30:12 +00:00
|
|
|
if (p->if_capenable & IFCAP_VLAN_MTU) {
|
2002-11-14 23:43:16 +00:00
|
|
|
/*
|
2004-05-25 14:30:12 +00:00
|
|
|
* No need to fudge the MTU since the parent can
|
|
|
|
* handle extended frames.
|
2002-11-14 23:43:16 +00:00
|
|
|
*/
|
|
|
|
ifv->ifv_mtufudge = 0;
|
2004-05-25 14:30:12 +00:00
|
|
|
} else {
|
2002-11-14 23:43:16 +00:00
|
|
|
/*
|
|
|
|
* Fudge the MTU by the encapsulation size. This
|
|
|
|
* makes us incompatible with strictly compliant
|
|
|
|
* 802.1Q implementations, but allows us to use
|
|
|
|
* the feature with other NetBSD implementations,
|
|
|
|
* which might still be useful.
|
|
|
|
*/
|
|
|
|
ifv->ifv_mtufudge = ifv->ifv_encaplen;
|
|
|
|
}
|
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
ifv->ifv_trunk = trunk;
|
2005-10-03 02:24:21 +00:00
|
|
|
ifp = ifv->ifv_ifp;
|
2011-03-21 09:40:01 +00:00
|
|
|
/*
|
|
|
|
* Initialize fields from our parent. This duplicates some
|
|
|
|
* work with ether_ifattach() but allows for non-ethernet
|
|
|
|
* interfaces to also work.
|
|
|
|
*/
|
2005-10-03 02:24:21 +00:00
|
|
|
ifp->if_mtu = p->if_mtu - ifv->ifv_mtufudge;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
ifp->if_baudrate = p->if_baudrate;
|
2011-03-21 09:40:01 +00:00
|
|
|
ifp->if_input = p->if_input;
|
|
|
|
ifp->if_resolvemulti = p->if_resolvemulti;
|
|
|
|
ifp->if_addrlen = p->if_addrlen;
|
|
|
|
ifp->if_broadcastaddr = p->if_broadcastaddr;
|
2018-08-17 01:03:23 +00:00
|
|
|
ifp->if_pcp = ifv->ifv_pcp;
|
2011-03-21 09:40:01 +00:00
|
|
|
|
2019-08-30 20:19:43 +00:00
|
|
|
/*
|
|
|
|
* We wrap the parent's if_output using vlan_output to ensure that it
|
|
|
|
* can't become stale.
|
|
|
|
*/
|
|
|
|
ifp->if_output = vlan_output;
|
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
/*
|
2001-03-28 15:52:12 +00:00
|
|
|
* Copy only a selected subset of flags from the parent.
|
|
|
|
* Other flags are none of our business.
|
1998-03-18 01:40:12 +00:00
|
|
|
*/
|
2006-01-31 16:41:05 +00:00
|
|
|
#define VLAN_COPY_FLAGS (IFF_SIMPLEX)
|
2005-10-03 02:24:21 +00:00
|
|
|
ifp->if_flags &= ~VLAN_COPY_FLAGS;
|
|
|
|
ifp->if_flags |= p->if_flags & VLAN_COPY_FLAGS;
|
|
|
|
#undef VLAN_COPY_FLAGS
|
|
|
|
|
|
|
|
ifp->if_link_state = p->if_link_state;
|
1998-03-18 01:40:12 +00:00
|
|
|
|
2019-10-13 15:15:09 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
vlan_capabilities(ifv);
|
2019-10-13 15:15:09 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2002-11-14 23:43:16 +00:00
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
/*
|
2011-03-21 09:40:01 +00:00
|
|
|
* Set up our interface address to reflect the underlying
|
1998-03-18 01:40:12 +00:00
|
|
|
* physical interface's.
|
|
|
|
*/
|
2019-11-07 15:00:37 +00:00
|
|
|
TASK_INIT(&ifv->lladdr_task, 0, vlan_lladdr_fn, ifv);
|
2011-03-21 09:40:01 +00:00
|
|
|
((struct sockaddr_dl *)ifp->if_addr->ifa_addr)->sdl_alen =
|
|
|
|
p->if_addrlen;
|
2001-07-24 17:14:37 +00:00
|
|
|
|
2019-11-07 15:00:37 +00:00
|
|
|
/*
|
|
|
|
* Do not schedule link address update if it was the same
|
|
|
|
* as previous parent's. This helps avoid updating for each
|
|
|
|
* associated llentry.
|
|
|
|
*/
|
|
|
|
if (memcmp(IF_LLADDR(p), IF_LLADDR(ifp), p->if_addrlen) != 0) {
|
|
|
|
bcopy(IF_LLADDR(p), IF_LLADDR(ifp), p->if_addrlen);
|
|
|
|
taskqueue_enqueue(taskqueue_thread, &ifv->lladdr_task);
|
|
|
|
}
|
2006-08-15 16:40:27 +00:00
|
|
|
|
|
|
|
/* We are ready for operation now. */
|
|
|
|
ifp->if_drv_flags |= IFF_DRV_RUNNING;
|
2017-08-15 17:52:37 +00:00
|
|
|
|
|
|
|
/* Update flags on the parent, if necessary. */
|
|
|
|
vlan_setflags(ifp, 1);
|
2018-09-21 01:37:08 +00:00
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
/*
|
2018-09-21 01:37:08 +00:00
|
|
|
* Configure multicast addresses that may already be
|
|
|
|
* joined on the vlan device.
|
2017-08-15 17:52:37 +00:00
|
|
|
*/
|
2018-09-21 01:37:08 +00:00
|
|
|
(void)vlan_setmulti(ifp);
|
|
|
|
|
|
|
|
done:
|
2008-07-14 18:40:21 +00:00
|
|
|
if (error == 0)
|
2012-01-12 18:39:37 +00:00
|
|
|
EVENTHANDLER_INVOKE(vlan_config, p, ifv->ifv_vid);
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XUNLOCK();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
|
|
|
return (error);
|
1998-03-18 01:40:12 +00:00
|
|
|
}
|
|
|
|
|
2010-05-17 19:36:56 +00:00
|
|
|
static void
|
1999-03-15 01:17:26 +00:00
|
|
|
vlan_unconfig(struct ifnet *ifp)
|
2006-06-21 07:29:44 +00:00
|
|
|
{
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XLOCK();
|
2012-08-20 16:00:33 +00:00
|
|
|
vlan_unconfig_locked(ifp, 0);
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XUNLOCK();
|
2006-06-21 07:29:44 +00:00
|
|
|
}
|
|
|
|
|
2010-05-17 19:36:56 +00:00
|
|
|
static void
|
2012-08-20 16:00:33 +00:00
|
|
|
vlan_unconfig_locked(struct ifnet *ifp, int departing)
|
1999-03-15 01:17:26 +00:00
|
|
|
{
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
1999-03-15 01:17:26 +00:00
|
|
|
struct vlan_mc_entry *mc;
|
|
|
|
struct ifvlan *ifv;
|
2008-07-14 18:40:21 +00:00
|
|
|
struct ifnet *parent;
|
2012-08-20 16:00:33 +00:00
|
|
|
int error;
|
1999-03-15 01:17:26 +00:00
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XLOCK_ASSERT();
|
2003-09-05 20:58:59 +00:00
|
|
|
|
1999-03-15 01:17:26 +00:00
|
|
|
ifv = ifp->if_softc;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
trunk = ifv->ifv_trunk;
|
2008-08-28 22:05:19 +00:00
|
|
|
parent = NULL;
|
1999-03-15 01:17:26 +00:00
|
|
|
|
2008-08-28 22:05:19 +00:00
|
|
|
if (trunk != NULL) {
|
|
|
|
parent = trunk->parent;
|
1999-03-15 01:17:26 +00:00
|
|
|
|
2001-07-24 17:14:37 +00:00
|
|
|
/*
|
|
|
|
* Since the interface is being unconfigured, we need to
|
|
|
|
* empty the list of multicast groups that we may have joined
|
|
|
|
* while we were alive from the parent's list.
|
|
|
|
*/
|
2018-09-21 01:37:08 +00:00
|
|
|
while ((mc = CK_SLIST_FIRST(&ifv->vlan_mc_listhead)) != NULL) {
|
2010-05-17 19:36:56 +00:00
|
|
|
/*
|
2012-08-20 16:00:33 +00:00
|
|
|
* If the parent interface is being detached,
|
2012-08-21 19:07:28 +00:00
|
|
|
* all its multicast addresses have already
|
2012-08-20 16:00:33 +00:00
|
|
|
* been removed. Warn about errors if
|
|
|
|
* if_delmulti() does fail, but don't abort as
|
|
|
|
* all callers expect vlan destruction to
|
|
|
|
* succeed.
|
2010-05-17 19:36:56 +00:00
|
|
|
*/
|
2012-08-20 16:00:33 +00:00
|
|
|
if (!departing) {
|
|
|
|
error = if_delmulti(parent,
|
|
|
|
(struct sockaddr *)&mc->mc_addr);
|
|
|
|
if (error)
|
|
|
|
if_printf(ifp,
|
|
|
|
"Failed to delete multicast address from parent: %d\n",
|
|
|
|
error);
|
|
|
|
}
|
2018-09-21 01:37:08 +00:00
|
|
|
CK_SLIST_REMOVE_HEAD(&ifv->vlan_mc_listhead, mc_entries);
|
2020-01-15 06:05:20 +00:00
|
|
|
NET_EPOCH_CALL(vlan_mc_free, &mc->mc_epoch_ctx);
|
2001-07-24 17:14:37 +00:00
|
|
|
}
|
2002-11-14 23:43:16 +00:00
|
|
|
|
2005-10-03 02:24:21 +00:00
|
|
|
vlan_setflags(ifp, 0); /* clear special flags on parent */
|
2017-08-15 17:52:37 +00:00
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
vlan_remhash(trunk, ifv);
|
|
|
|
ifv->ifv_trunk = NULL;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Check if we were the last.
|
|
|
|
*/
|
|
|
|
if (trunk->refcnt == 0) {
|
2014-08-04 00:58:12 +00:00
|
|
|
parent->if_vlantrunk = NULL;
|
2018-09-21 01:37:08 +00:00
|
|
|
NET_EPOCH_WAIT();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
trunk_destroy(trunk);
|
2017-08-15 17:52:37 +00:00
|
|
|
}
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Disconnect from parent. */
|
2005-10-03 02:24:21 +00:00
|
|
|
if (ifv->ifv_pflags)
|
|
|
|
if_printf(ifp, "%s: ifv_pflags unclean\n", __func__);
|
2006-06-21 07:29:44 +00:00
|
|
|
ifp->if_mtu = ETHERMTU;
|
|
|
|
ifp->if_link_state = LINK_STATE_UNKNOWN;
|
|
|
|
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
|
1999-03-15 01:17:26 +00:00
|
|
|
|
2008-08-28 22:05:19 +00:00
|
|
|
/*
|
|
|
|
* Only dispatch an event if vlan was
|
|
|
|
* attached, otherwise there is nothing
|
|
|
|
* to cleanup anyway.
|
|
|
|
*/
|
|
|
|
if (parent != NULL)
|
2012-01-12 18:39:37 +00:00
|
|
|
EVENTHANDLER_INVOKE(vlan_unconfig, parent, ifv->ifv_vid);
|
1999-03-15 01:17:26 +00:00
|
|
|
}
|
|
|
|
|
2005-10-03 02:24:21 +00:00
|
|
|
/* Handle a reference counted flag that should be set on the parent as well */
|
2002-11-14 23:43:16 +00:00
|
|
|
static int
|
2005-10-03 02:24:21 +00:00
|
|
|
vlan_setflag(struct ifnet *ifp, int flag, int status,
|
|
|
|
int (*func)(struct ifnet *, int))
|
2002-11-14 23:43:16 +00:00
|
|
|
{
|
2005-10-03 02:24:21 +00:00
|
|
|
struct ifvlan *ifv;
|
|
|
|
int error;
|
2002-11-14 23:43:16 +00:00
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SXLOCK_ASSERT();
|
2005-10-03 02:24:21 +00:00
|
|
|
|
|
|
|
ifv = ifp->if_softc;
|
|
|
|
status = status ? (ifp->if_flags & flag) : 0;
|
|
|
|
/* Now "status" contains the flag value or 0 */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* See if recorded parent's status is different from what
|
|
|
|
* we want it to be. If it is, flip it. We record parent's
|
|
|
|
* status in ifv_pflags so that we won't clear parent's flag
|
|
|
|
* we haven't set. In fact, we don't clear or set parent's
|
|
|
|
* flags directly, but get or release references to them.
|
|
|
|
* That's why we can be sure that recorded flags still are
|
|
|
|
* in accord with actual parent's flags.
|
|
|
|
*/
|
|
|
|
if (status != (ifv->ifv_pflags & flag)) {
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
error = (*func)(PARENT(ifv), status);
|
2005-10-03 02:24:21 +00:00
|
|
|
if (error)
|
|
|
|
return (error);
|
|
|
|
ifv->ifv_pflags &= ~flag;
|
|
|
|
ifv->ifv_pflags |= status;
|
2002-11-14 23:43:16 +00:00
|
|
|
}
|
2005-10-03 02:24:21 +00:00
|
|
|
return (0);
|
|
|
|
}
|
2002-11-14 23:43:16 +00:00
|
|
|
|
2005-10-03 02:24:21 +00:00
|
|
|
/*
|
|
|
|
* Handle IFF_* flags that require certain changes on the parent:
|
|
|
|
* if "status" is true, update parent's flags respective to our if_flags;
|
|
|
|
* if "status" is false, forcedly clear the flags set on parent.
|
|
|
|
*/
|
|
|
|
static int
|
|
|
|
vlan_setflags(struct ifnet *ifp, int status)
|
|
|
|
{
|
|
|
|
int error, i;
|
2020-09-01 21:19:14 +00:00
|
|
|
|
2005-10-03 02:24:21 +00:00
|
|
|
for (i = 0; vlan_pflags[i].flag; i++) {
|
|
|
|
error = vlan_setflag(ifp, vlan_pflags[i].flag,
|
|
|
|
status, vlan_pflags[i].func);
|
|
|
|
if (error)
|
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
return (0);
|
2002-11-14 23:43:16 +00:00
|
|
|
}
|
|
|
|
|
2004-05-03 13:48:35 +00:00
|
|
|
/* Inform all vlans that their parent has changed link state */
|
|
|
|
static void
|
2009-12-31 20:29:58 +00:00
|
|
|
vlan_link_state(struct ifnet *ifp)
|
2004-05-03 13:48:35 +00:00
|
|
|
{
|
2019-10-17 20:18:07 +00:00
|
|
|
struct epoch_tracker et;
|
2017-08-15 17:52:37 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
2004-05-03 13:48:35 +00:00
|
|
|
struct ifvlan *ifv;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
2019-10-17 20:18:07 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
2017-08-15 17:52:37 +00:00
|
|
|
trunk = ifp->if_vlantrunk;
|
2019-10-17 20:18:07 +00:00
|
|
|
if (trunk == NULL) {
|
|
|
|
NET_EPOCH_EXIT(et);
|
2017-08-15 17:52:37 +00:00
|
|
|
return;
|
2019-10-17 20:18:07 +00:00
|
|
|
}
|
2017-08-15 17:52:37 +00:00
|
|
|
|
|
|
|
TRUNK_WLOCK(trunk);
|
|
|
|
VLAN_FOREACH(ifv, trunk) {
|
|
|
|
ifv->ifv_ifp->if_baudrate = trunk->parent->if_baudrate;
|
|
|
|
if_link_state_change(ifv->ifv_ifp,
|
|
|
|
trunk->parent->if_link_state);
|
|
|
|
}
|
|
|
|
TRUNK_WUNLOCK(trunk);
|
2019-10-17 20:18:07 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void
|
|
|
|
vlan_capabilities(struct ifvlan *ifv)
|
|
|
|
{
|
2017-08-15 17:52:37 +00:00
|
|
|
struct ifnet *p;
|
|
|
|
struct ifnet *ifp;
|
2014-09-22 08:27:27 +00:00
|
|
|
struct ifnet_hw_tsomax hw_tsomax;
|
2017-04-28 11:00:58 +00:00
|
|
|
int cap = 0, ena = 0, mena;
|
|
|
|
u_long hwa = 0;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
2019-01-09 01:11:19 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
VLAN_SXLOCK_ASSERT();
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
p = PARENT(ifv);
|
|
|
|
ifp = ifv->ifv_ifp;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
2017-04-28 11:00:58 +00:00
|
|
|
/* Mask parent interface enabled capabilities disabled by user. */
|
|
|
|
mena = p->if_capenable & ifv->ifv_capenable;
|
|
|
|
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface can do checksum offloading
|
|
|
|
* on VLANs, then propagate its hardware-assisted
|
|
|
|
* checksumming flags. Also assert that checksum
|
|
|
|
* offloading requires hardware VLAN tagging.
|
|
|
|
*/
|
|
|
|
if (p->if_capabilities & IFCAP_VLAN_HWCSUM)
|
2017-04-28 11:00:58 +00:00
|
|
|
cap |= p->if_capabilities & (IFCAP_HWCSUM | IFCAP_HWCSUM_IPV6);
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (p->if_capenable & IFCAP_VLAN_HWCSUM &&
|
|
|
|
p->if_capenable & IFCAP_VLAN_HWTAGGING) {
|
2017-04-28 11:00:58 +00:00
|
|
|
ena |= mena & (IFCAP_HWCSUM | IFCAP_HWCSUM_IPV6);
|
|
|
|
if (ena & IFCAP_TXCSUM)
|
|
|
|
hwa |= p->if_hwassist & (CSUM_IP | CSUM_TCP |
|
|
|
|
CSUM_UDP | CSUM_SCTP);
|
|
|
|
if (ena & IFCAP_TXCSUM_IPV6)
|
|
|
|
hwa |= p->if_hwassist & (CSUM_TCP_IPV6 |
|
|
|
|
CSUM_UDP_IPV6 | CSUM_SCTP_IPV6);
|
2004-05-03 13:48:35 +00:00
|
|
|
}
|
2017-04-28 11:00:58 +00:00
|
|
|
|
2010-02-20 22:47:20 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface can do TSO on VLANs then
|
|
|
|
* propagate the hardware-assisted flag. TSO on VLANs
|
|
|
|
* does not necessarily require hardware VLAN tagging.
|
|
|
|
*/
|
2014-09-22 08:27:27 +00:00
|
|
|
memset(&hw_tsomax, 0, sizeof(hw_tsomax));
|
|
|
|
if_hw_tsomax_common(p, &hw_tsomax);
|
|
|
|
if_hw_tsomax_update(ifp, &hw_tsomax);
|
2010-02-20 22:47:20 +00:00
|
|
|
if (p->if_capabilities & IFCAP_VLAN_HWTSO)
|
2017-04-28 11:00:58 +00:00
|
|
|
cap |= p->if_capabilities & IFCAP_TSO;
|
2010-02-20 22:47:20 +00:00
|
|
|
if (p->if_capenable & IFCAP_VLAN_HWTSO) {
|
2017-04-28 11:00:58 +00:00
|
|
|
ena |= mena & IFCAP_TSO;
|
|
|
|
if (ena & IFCAP_TSO)
|
|
|
|
hwa |= p->if_hwassist & CSUM_TSO;
|
2010-02-20 22:47:20 +00:00
|
|
|
}
|
2012-06-19 07:34:13 +00:00
|
|
|
|
2017-04-29 08:28:59 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface can do LRO and checksum offloading on
|
|
|
|
* VLANs, then guess it may do LRO on VLANs. False positive here
|
|
|
|
* cost nothing, while false negative may lead to some confusions.
|
|
|
|
*/
|
|
|
|
if (p->if_capabilities & IFCAP_VLAN_HWCSUM)
|
|
|
|
cap |= p->if_capabilities & IFCAP_LRO;
|
|
|
|
if (p->if_capenable & IFCAP_VLAN_HWCSUM)
|
|
|
|
ena |= p->if_capenable & IFCAP_LRO;
|
|
|
|
|
2012-06-19 07:34:13 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface can offload TCP connections over VLANs then
|
|
|
|
* propagate its TOE capability to the VLAN interface.
|
|
|
|
*
|
|
|
|
* All TOE drivers in the tree today can deal with VLANs. If this
|
|
|
|
* changes then IFCAP_VLAN_TOE should be promoted to a full capability
|
|
|
|
* with its own bit.
|
|
|
|
*/
|
|
|
|
#define IFCAP_VLAN_TOE IFCAP_TOE
|
|
|
|
if (p->if_capabilities & IFCAP_VLAN_TOE)
|
2017-04-28 11:00:58 +00:00
|
|
|
cap |= p->if_capabilities & IFCAP_TOE;
|
2012-06-19 07:34:13 +00:00
|
|
|
if (p->if_capenable & IFCAP_VLAN_TOE) {
|
2023-01-23 15:11:02 +00:00
|
|
|
SETTOEDEV(ifp, TOEDEV(p));
|
2017-04-28 11:00:58 +00:00
|
|
|
ena |= mena & IFCAP_TOE;
|
2012-06-19 07:34:13 +00:00
|
|
|
}
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
|
2017-04-28 11:00:58 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface supports dynamic link state, so does the
|
|
|
|
* VLAN interface.
|
|
|
|
*/
|
|
|
|
cap |= (p->if_capabilities & IFCAP_LINKSTATE);
|
|
|
|
ena |= (mena & IFCAP_LINKSTATE);
|
|
|
|
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
#ifdef RATELIMIT
|
|
|
|
/*
|
|
|
|
* If the parent interface supports ratelimiting, so does the
|
|
|
|
* VLAN interface.
|
|
|
|
*/
|
2017-04-28 11:00:58 +00:00
|
|
|
cap |= (p->if_capabilities & IFCAP_TXRTLMT);
|
|
|
|
ena |= (mena & IFCAP_TXRTLMT);
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
#endif
|
2017-04-28 11:00:58 +00:00
|
|
|
|
2019-06-29 00:51:38 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface supports unmapped mbufs, so does
|
|
|
|
* the VLAN interface. Note that this should be fine even for
|
|
|
|
* interfaces that don't support hardware tagging as headers
|
|
|
|
* are prepended in normal mbufs to unmapped mbufs holding
|
|
|
|
* payload data.
|
|
|
|
*/
|
2021-01-28 21:08:48 +00:00
|
|
|
cap |= (p->if_capabilities & IFCAP_MEXTPG);
|
|
|
|
ena |= (mena & IFCAP_MEXTPG);
|
2019-06-29 00:51:38 +00:00
|
|
|
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
/*
|
|
|
|
* If the parent interface can offload encryption and segmentation
|
|
|
|
* of TLS records over TCP, propagate it's capability to the VLAN
|
|
|
|
* interface.
|
|
|
|
*
|
|
|
|
* All TLS drivers in the tree today can deal with VLANs. If
|
|
|
|
* this ever changes, then a new IFCAP_VLAN_TXTLS can be
|
|
|
|
* defined.
|
|
|
|
*/
|
2020-10-29 00:23:16 +00:00
|
|
|
if (p->if_capabilities & (IFCAP_TXTLS | IFCAP_TXTLS_RTLMT))
|
|
|
|
cap |= p->if_capabilities & (IFCAP_TXTLS | IFCAP_TXTLS_RTLMT);
|
|
|
|
if (p->if_capenable & (IFCAP_TXTLS | IFCAP_TXTLS_RTLMT))
|
|
|
|
ena |= mena & (IFCAP_TXTLS | IFCAP_TXTLS_RTLMT);
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
|
2017-04-28 11:00:58 +00:00
|
|
|
ifp->if_capabilities = cap;
|
|
|
|
ifp->if_capenable = ena;
|
|
|
|
ifp->if_hwassist = hwa;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void
|
|
|
|
vlan_trunk_capabilities(struct ifnet *ifp)
|
|
|
|
{
|
2019-10-17 20:18:07 +00:00
|
|
|
struct epoch_tracker et;
|
2017-08-15 17:52:37 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
struct ifvlan *ifv;
|
|
|
|
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SLOCK();
|
|
|
|
trunk = ifp->if_vlantrunk;
|
|
|
|
if (trunk == NULL) {
|
|
|
|
VLAN_SUNLOCK();
|
|
|
|
return;
|
|
|
|
}
|
2019-10-17 20:18:07 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
|
|
|
VLAN_FOREACH(ifv, trunk)
|
2017-08-15 17:52:37 +00:00
|
|
|
vlan_capabilities(ifv);
|
2019-10-17 20:18:07 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SUNLOCK();
|
2004-05-03 13:48:35 +00:00
|
|
|
}
|
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
static int
|
1998-08-23 03:07:17 +00:00
|
|
|
vlan_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data)
|
1998-03-18 01:40:12 +00:00
|
|
|
{
|
|
|
|
struct ifnet *p;
|
|
|
|
struct ifreq *ifr;
|
2022-04-13 23:08:21 +00:00
|
|
|
#ifdef INET
|
2011-03-21 09:40:01 +00:00
|
|
|
struct ifaddr *ifa;
|
2022-04-13 23:08:21 +00:00
|
|
|
#endif
|
1998-03-18 01:40:12 +00:00
|
|
|
struct ifvlan *ifv;
|
2014-08-04 00:58:12 +00:00
|
|
|
struct ifvlantrunk *trunk;
|
1998-03-18 01:40:12 +00:00
|
|
|
struct vlanreq vlr;
|
2020-01-22 20:36:45 +00:00
|
|
|
int error = 0, oldmtu;
|
1998-03-18 01:40:12 +00:00
|
|
|
|
|
|
|
ifr = (struct ifreq *)data;
|
2022-04-13 23:08:21 +00:00
|
|
|
#ifdef INET
|
2011-03-21 09:40:01 +00:00
|
|
|
ifa = (struct ifaddr *) data;
|
2022-04-13 23:08:21 +00:00
|
|
|
#endif
|
1998-03-18 01:40:12 +00:00
|
|
|
ifv = ifp->if_softc;
|
|
|
|
|
|
|
|
switch (cmd) {
|
2011-03-21 09:40:01 +00:00
|
|
|
case SIOCSIFADDR:
|
|
|
|
ifp->if_flags |= IFF_UP;
|
|
|
|
#ifdef INET
|
|
|
|
if (ifa->ifa_addr->sa_family == AF_INET)
|
|
|
|
arp_ifinit(ifp, ifa);
|
|
|
|
#endif
|
|
|
|
break;
|
|
|
|
case SIOCGIFADDR:
|
2018-03-27 20:51:49 +00:00
|
|
|
bcopy(IF_LLADDR(ifp), &ifr->ifr_addr.sa_data[0],
|
|
|
|
ifp->if_addrlen);
|
2011-03-21 09:40:01 +00:00
|
|
|
break;
|
2003-01-22 23:30:26 +00:00
|
|
|
case SIOCGIFMEDIA:
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SLOCK();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (TRUNK(ifv) != NULL) {
|
2010-03-21 15:00:33 +00:00
|
|
|
p = PARENT(ifv);
|
2017-05-23 00:13:27 +00:00
|
|
|
if_ref(p);
|
2010-03-21 15:00:33 +00:00
|
|
|
error = (*p->if_ioctl)(p, SIOCGIFMEDIA, data);
|
2017-05-23 00:13:27 +00:00
|
|
|
if_rele(p);
|
2003-01-22 23:30:26 +00:00
|
|
|
/* Limit the result to the parent's current config. */
|
|
|
|
if (error == 0) {
|
|
|
|
struct ifmediareq *ifmr;
|
|
|
|
|
2004-07-04 16:43:24 +00:00
|
|
|
ifmr = (struct ifmediareq *)data;
|
2003-01-22 23:30:26 +00:00
|
|
|
if (ifmr->ifm_count >= 1 && ifmr->ifm_ulist) {
|
|
|
|
ifmr->ifm_count = 1;
|
|
|
|
error = copyout(&ifmr->ifm_current,
|
2004-07-04 16:43:24 +00:00
|
|
|
ifmr->ifm_ulist,
|
2003-01-22 23:30:26 +00:00
|
|
|
sizeof(int));
|
|
|
|
}
|
|
|
|
}
|
2003-09-05 20:58:59 +00:00
|
|
|
} else {
|
2003-01-22 23:30:26 +00:00
|
|
|
error = EINVAL;
|
2003-09-05 20:58:59 +00:00
|
|
|
}
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SUNLOCK();
|
2003-01-22 23:30:26 +00:00
|
|
|
break;
|
|
|
|
|
|
|
|
case SIOCSIFMEDIA:
|
|
|
|
error = EINVAL;
|
|
|
|
break;
|
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
case SIOCSIFMTU:
|
|
|
|
/*
|
|
|
|
* Set the interface MTU.
|
|
|
|
*/
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SLOCK();
|
|
|
|
trunk = TRUNK(ifv);
|
|
|
|
if (trunk != NULL) {
|
|
|
|
TRUNK_WLOCK(trunk);
|
2002-11-14 23:43:16 +00:00
|
|
|
if (ifr->ifr_mtu >
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
(PARENT(ifv)->if_mtu - ifv->ifv_mtufudge) ||
|
2002-11-14 23:43:16 +00:00
|
|
|
ifr->ifr_mtu <
|
|
|
|
(ifv->ifv_mintu - ifv->ifv_mtufudge))
|
|
|
|
error = EINVAL;
|
|
|
|
else
|
|
|
|
ifp->if_mtu = ifr->ifr_mtu;
|
2017-08-15 17:52:37 +00:00
|
|
|
TRUNK_WUNLOCK(trunk);
|
2002-11-14 23:43:16 +00:00
|
|
|
} else
|
1998-03-18 01:40:12 +00:00
|
|
|
error = EINVAL;
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SUNLOCK();
|
1998-03-18 01:40:12 +00:00
|
|
|
break;
|
|
|
|
|
|
|
|
case SIOCSETVLAN:
|
2010-11-22 23:35:29 +00:00
|
|
|
#ifdef VIMAGE
|
2012-01-05 11:24:22 +00:00
|
|
|
/*
|
|
|
|
* XXXRW/XXXBZ: The goal in these checks is to allow a VLAN
|
|
|
|
* interface to be delegated to a jail without allowing the
|
|
|
|
* jail to change what underlying interface/VID it is
|
|
|
|
* associated with. We are not entirely convinced that this
|
2012-01-05 11:42:34 +00:00
|
|
|
* is the right way to accomplish that policy goal.
|
2012-01-05 11:24:22 +00:00
|
|
|
*/
|
2010-11-22 23:35:29 +00:00
|
|
|
if (ifp->if_vnet != ifp->if_home_vnet) {
|
|
|
|
error = EPERM;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
2018-03-30 18:50:13 +00:00
|
|
|
error = copyin(ifr_data_get_ptr(ifr), &vlr, sizeof(vlr));
|
1998-03-18 01:40:12 +00:00
|
|
|
if (error)
|
|
|
|
break;
|
|
|
|
if (vlr.vlr_parent[0] == '\0') {
|
1999-03-15 01:17:26 +00:00
|
|
|
vlan_unconfig(ifp);
|
1998-03-18 01:40:12 +00:00
|
|
|
break;
|
|
|
|
}
|
2017-05-23 00:13:27 +00:00
|
|
|
p = ifunit_ref(vlr.vlr_parent);
|
2009-09-09 03:36:43 +00:00
|
|
|
if (p == NULL) {
|
1998-03-18 01:40:12 +00:00
|
|
|
error = ENOENT;
|
|
|
|
break;
|
|
|
|
}
|
2021-04-11 16:47:03 +00:00
|
|
|
#ifdef COMPAT_FREEBSD12
|
|
|
|
if (vlr.vlr_proto == 0)
|
|
|
|
vlr.vlr_proto = ETHERTYPE_VLAN;
|
|
|
|
#endif
|
2020-01-22 20:36:45 +00:00
|
|
|
oldmtu = ifp->if_mtu;
|
2020-10-21 21:28:20 +00:00
|
|
|
error = vlan_config(ifv, p, vlr.vlr_tag, vlr.vlr_proto);
|
2017-05-23 00:13:27 +00:00
|
|
|
if_rele(p);
|
2020-01-22 20:36:45 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* VLAN MTU may change during addition of the vlandev.
|
|
|
|
* If it did, do network layer specific procedure.
|
|
|
|
*/
|
2023-03-03 16:30:15 +00:00
|
|
|
if (ifp->if_mtu != oldmtu)
|
|
|
|
if_notifymtu(ifp);
|
1998-03-18 01:40:12 +00:00
|
|
|
break;
|
2004-07-04 16:43:24 +00:00
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
case SIOCGETVLAN:
|
2010-11-22 23:35:29 +00:00
|
|
|
#ifdef VIMAGE
|
|
|
|
if (ifp->if_vnet != ifp->if_home_vnet) {
|
|
|
|
error = EPERM;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
2004-07-04 16:43:24 +00:00
|
|
|
bzero(&vlr, sizeof(vlr));
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SLOCK();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (TRUNK(ifv) != NULL) {
|
|
|
|
strlcpy(vlr.vlr_parent, PARENT(ifv)->if_xname,
|
2003-10-31 18:32:15 +00:00
|
|
|
sizeof(vlr.vlr_parent));
|
2012-01-12 18:39:37 +00:00
|
|
|
vlr.vlr_tag = ifv->ifv_vid;
|
2020-10-21 21:28:20 +00:00
|
|
|
vlr.vlr_proto = ifv->ifv_proto;
|
1998-03-18 01:40:12 +00:00
|
|
|
}
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SUNLOCK();
|
2018-03-30 18:50:13 +00:00
|
|
|
error = copyout(&vlr, ifr_data_get_ptr(ifr), sizeof(vlr));
|
1998-03-18 01:40:12 +00:00
|
|
|
break;
|
2020-10-21 21:28:20 +00:00
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
case SIOCSIFFLAGS:
|
|
|
|
/*
|
2005-10-03 02:24:21 +00:00
|
|
|
* We should propagate selected flags to the parent,
|
|
|
|
* e.g., promiscuous mode.
|
1998-03-18 01:40:12 +00:00
|
|
|
*/
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XLOCK();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
if (TRUNK(ifv) != NULL)
|
2005-10-03 02:24:21 +00:00
|
|
|
error = vlan_setflags(ifp, 1);
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_XUNLOCK();
|
1998-03-18 01:40:12 +00:00
|
|
|
break;
|
2002-11-14 23:43:16 +00:00
|
|
|
|
1999-03-15 01:17:26 +00:00
|
|
|
case SIOCADDMULTI:
|
|
|
|
case SIOCDELMULTI:
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
/*
|
|
|
|
* If we don't have a parent, just remember the membership for
|
|
|
|
* when we do.
|
2017-08-15 17:52:37 +00:00
|
|
|
*
|
|
|
|
* XXX We need the rmlock here to avoid sleeping while
|
|
|
|
* holding in6_multi_mtx.
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
*/
|
2018-09-21 01:37:08 +00:00
|
|
|
VLAN_XLOCK();
|
2014-08-04 00:58:12 +00:00
|
|
|
trunk = TRUNK(ifv);
|
2018-09-21 01:37:08 +00:00
|
|
|
if (trunk != NULL)
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
error = vlan_setmulti(ifp);
|
2018-09-21 01:37:08 +00:00
|
|
|
VLAN_XUNLOCK();
|
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective
work by yar, thompsa and myself. The checksum offloading part also involves
work done by Mihail Balikov.
The most important changes:
o Instead of global linked list of all vlan softc use a per-trunk
hash. The size of hash is dynamically adjusted, depending on
number of entries. This changes struct ifnet, replacing counter
of vlans with a pointer to trunk structure. This change is an
improvement for setups with big number of VLANs, several interfaces
and several CPUs. It is a small regression for a setup with a single
VLAN interface.
An alternative to dynamic hash is a per-trunk static array with
4096 entries, which is a compile time option - VLAN_ARRAY. In my
experiments the array is not an improvement, probably because such
a big trunk structure doesn't fit into CPU cache.
o Introduce an UMA zone for VLAN tags. Since drivers depend on it,
the zone is declared in kern_mbuf.c, not in optional vlan(4) driver.
This change is a big improvement for any setup utilizing vlan(4).
o Use rwlock(9) instead of mutex(9) for locking. We are the first
ones to do this! :)
o Some drivers can do hardware VLAN tagging + hardware checksum
offloading. Add an infrastructure for this. Whenever vlan(4) is
attached to a parent or parent configuration is changed, the flags
on vlan(4) interface are updated.
In collaboration with: yar, thompsa
In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
|
|
|
|
2018-09-21 01:37:08 +00:00
|
|
|
break;
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
case SIOCGVLANPCP:
|
|
|
|
#ifdef VIMAGE
|
|
|
|
if (ifp->if_vnet != ifp->if_home_vnet) {
|
|
|
|
error = EPERM;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
ifr->ifr_vlan_pcp = ifv->ifv_pcp;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case SIOCSVLANPCP:
|
|
|
|
#ifdef VIMAGE
|
|
|
|
if (ifp->if_vnet != ifp->if_home_vnet) {
|
|
|
|
error = EPERM;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
error = priv_check(curthread, PRIV_NET_SETVLANPCP);
|
|
|
|
if (error)
|
|
|
|
break;
|
2021-07-22 08:35:08 +00:00
|
|
|
if (ifr->ifr_vlan_pcp > VLAN_PCP_MAX) {
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
error = EINVAL;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
ifv->ifv_pcp = ifr->ifr_vlan_pcp;
|
2018-08-17 01:03:23 +00:00
|
|
|
ifp->if_pcp = ifv->ifv_pcp;
|
2018-04-26 08:58:27 +00:00
|
|
|
/* broadcast event about PCP change */
|
|
|
|
EVENTHANDLER_INVOKE(ifnet_event, ifp, IFNET_EVENT_PCP);
|
Add support to priority code point (PCP) that is an 3-bit field
which refers to IEEE 802.1p class of service and maps to the frame
priority level.
Values in order of priority are: 1 (Background (lowest)),
0 (Best effort (default)), 2 (Excellent effort),
3 (Critical applications), 4 (Video, < 100ms latency),
5 (Video, < 10ms latency), 6 (Internetwork control) and
7 (Network control (highest)).
Example of usage:
root# ifconfig em0.1 create
root# ifconfig em0.1 vlanpcp 3
Note:
The review D801 includes the pf(4) part, but as discussed with kristof,
we won't commit the pf(4) bits for now.
The credits of the original code is from rwatson.
Differential Revision: https://reviews.freebsd.org/D801
Reviewed by: gnn, adrian, loos
Discussed with: rwatson, glebius, kristof
Tested by: many including Matthew Grooms <mgrooms__shrew.net>
Obtained from: pfSense
Relnotes: Yes
2016-06-06 09:51:58 +00:00
|
|
|
break;
|
|
|
|
|
2017-04-28 11:00:58 +00:00
|
|
|
case SIOCSIFCAP:
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SLOCK();
|
2017-04-28 11:00:58 +00:00
|
|
|
ifv->ifv_capenable = ifr->ifr_reqcap;
|
|
|
|
trunk = TRUNK(ifv);
|
2019-10-13 15:15:09 +00:00
|
|
|
if (trunk != NULL) {
|
|
|
|
struct epoch_tracker et;
|
|
|
|
|
|
|
|
NET_EPOCH_ENTER(et);
|
2017-04-28 11:00:58 +00:00
|
|
|
vlan_capabilities(ifv);
|
2019-10-13 15:15:09 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
|
|
|
}
|
2017-08-15 17:52:37 +00:00
|
|
|
VLAN_SUNLOCK();
|
2017-04-28 11:00:58 +00:00
|
|
|
break;
|
|
|
|
|
1998-03-18 01:40:12 +00:00
|
|
|
default:
|
2011-03-21 09:40:01 +00:00
|
|
|
error = EINVAL;
|
|
|
|
break;
|
1998-03-18 01:40:12 +00:00
|
|
|
}
|
2004-07-04 16:43:24 +00:00
|
|
|
|
|
|
|
return (error);
|
1998-03-18 01:40:12 +00:00
|
|
|
}
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
|
Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
|
|
|
#if defined(KERN_TLS) || defined(RATELIMIT)
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
static int
|
|
|
|
vlan_snd_tag_alloc(struct ifnet *ifp,
|
|
|
|
union if_snd_tag_alloc_params *params,
|
|
|
|
struct m_snd_tag **ppmt)
|
|
|
|
{
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
struct epoch_tracker et;
|
2021-09-14 18:43:41 +00:00
|
|
|
const struct if_snd_tag_sw *sw;
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
struct vlan_snd_tag *vst;
|
|
|
|
struct ifvlan *ifv;
|
|
|
|
struct ifnet *parent;
|
2022-05-25 10:39:56 +00:00
|
|
|
struct m_snd_tag *mst;
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
int error;
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
|
2022-05-25 10:39:56 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
|
|
|
ifv = ifp->if_softc;
|
|
|
|
|
2021-09-14 18:43:41 +00:00
|
|
|
switch (params->hdr.type) {
|
|
|
|
#ifdef RATELIMIT
|
|
|
|
case IF_SND_TAG_TYPE_UNLIMITED:
|
|
|
|
sw = &vlan_snd_tag_ul_sw;
|
|
|
|
break;
|
|
|
|
case IF_SND_TAG_TYPE_RATE_LIMIT:
|
|
|
|
sw = &vlan_snd_tag_rl_sw;
|
|
|
|
break;
|
|
|
|
#endif
|
|
|
|
#ifdef KERN_TLS
|
|
|
|
case IF_SND_TAG_TYPE_TLS:
|
|
|
|
sw = &vlan_snd_tag_tls_sw;
|
|
|
|
break;
|
2022-05-25 10:39:56 +00:00
|
|
|
case IF_SND_TAG_TYPE_TLS_RX:
|
|
|
|
sw = NULL;
|
|
|
|
if (params->tls_rx.vlan_id != 0)
|
|
|
|
goto failure;
|
|
|
|
params->tls_rx.vlan_id = ifv->ifv_vid;
|
|
|
|
break;
|
2021-09-14 18:43:41 +00:00
|
|
|
#ifdef RATELIMIT
|
|
|
|
case IF_SND_TAG_TYPE_TLS_RATE_LIMIT:
|
|
|
|
sw = &vlan_snd_tag_tls_rl_sw;
|
|
|
|
break;
|
|
|
|
#endif
|
|
|
|
#endif
|
|
|
|
default:
|
2022-05-25 10:39:56 +00:00
|
|
|
goto failure;
|
2021-09-14 18:43:41 +00:00
|
|
|
}
|
|
|
|
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
if (ifv->ifv_trunk != NULL)
|
|
|
|
parent = PARENT(ifv);
|
|
|
|
else
|
|
|
|
parent = NULL;
|
2022-05-25 10:39:56 +00:00
|
|
|
if (parent == NULL)
|
|
|
|
goto failure;
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
if_ref(parent);
|
|
|
|
NET_EPOCH_EXIT(et);
|
|
|
|
|
2022-05-25 10:39:56 +00:00
|
|
|
if (sw != NULL) {
|
|
|
|
vst = malloc(sizeof(*vst), M_VLAN, M_NOWAIT);
|
|
|
|
if (vst == NULL) {
|
|
|
|
if_rele(parent);
|
|
|
|
return (ENOMEM);
|
|
|
|
}
|
|
|
|
} else
|
|
|
|
vst = NULL;
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
|
2022-05-25 10:39:56 +00:00
|
|
|
error = m_snd_tag_alloc(parent, params, &mst);
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
if_rele(parent);
|
|
|
|
if (error) {
|
|
|
|
free(vst, M_VLAN);
|
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
|
2022-05-25 10:39:56 +00:00
|
|
|
if (sw != NULL) {
|
|
|
|
m_snd_tag_init(&vst->com, ifp, sw);
|
|
|
|
vst->tag = mst;
|
|
|
|
|
|
|
|
*ppmt = &vst->com;
|
|
|
|
} else
|
|
|
|
*ppmt = mst;
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
|
|
|
|
return (0);
|
2022-05-25 10:39:56 +00:00
|
|
|
failure:
|
|
|
|
NET_EPOCH_EXIT(et);
|
|
|
|
return (EOPNOTSUPP);
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
}
|
|
|
|
|
2021-01-26 16:54:42 +00:00
|
|
|
static struct m_snd_tag *
|
|
|
|
vlan_next_snd_tag(struct m_snd_tag *mst)
|
|
|
|
{
|
|
|
|
struct vlan_snd_tag *vst;
|
|
|
|
|
|
|
|
vst = mst_to_vst(mst);
|
|
|
|
return (vst->tag);
|
|
|
|
}
|
|
|
|
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
static int
|
|
|
|
vlan_snd_tag_modify(struct m_snd_tag *mst,
|
|
|
|
union if_snd_tag_modify_params *params)
|
|
|
|
{
|
|
|
|
struct vlan_snd_tag *vst;
|
|
|
|
|
|
|
|
vst = mst_to_vst(mst);
|
2021-09-14 18:43:41 +00:00
|
|
|
return (vst->tag->sw->snd_tag_modify(vst->tag, params));
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
|
|
|
vlan_snd_tag_query(struct m_snd_tag *mst,
|
|
|
|
union if_snd_tag_query_params *params)
|
|
|
|
{
|
|
|
|
struct vlan_snd_tag *vst;
|
|
|
|
|
|
|
|
vst = mst_to_vst(mst);
|
2021-09-14 18:43:41 +00:00
|
|
|
return (vst->tag->sw->snd_tag_query(vst->tag, params));
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
}
|
2019-02-13 14:57:59 +00:00
|
|
|
|
|
|
|
static void
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
vlan_snd_tag_free(struct m_snd_tag *mst)
|
2019-02-13 14:57:59 +00:00
|
|
|
{
|
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
|
|
|
struct vlan_snd_tag *vst;
|
|
|
|
|
|
|
|
vst = mst_to_vst(mst);
|
|
|
|
m_snd_tag_rele(vst->tag);
|
|
|
|
free(vst, M_VLAN);
|
2019-02-13 14:57:59 +00:00
|
|
|
}
|
2021-01-26 16:54:42 +00:00
|
|
|
|
|
|
|
static void
|
|
|
|
vlan_ratelimit_query(struct ifnet *ifp __unused, struct if_ratelimit_query_results *q)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* For vlan, we have an indirect
|
|
|
|
* interface. The caller needs to
|
|
|
|
* get a ratelimit tag on the actual
|
|
|
|
* interface the flow will go on.
|
|
|
|
*/
|
|
|
|
q->rate_table = NULL;
|
|
|
|
q->flags = RT_IS_INDIRECT;
|
|
|
|
q->max_flows = 0;
|
|
|
|
q->number_of_rates = 0;
|
|
|
|
}
|
|
|
|
|
Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.
- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.
- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().
- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.
- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.
- How rate limiting works:
1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.
2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.
3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.
4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.
Reviewed by: wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision: https://reviews.freebsd.org/D3687
Sponsored by: Mellanox Technologies
MFC after: 3 months
2017-01-18 13:31:17 +00:00
|
|
|
#endif
|