d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
cec06a3edc
freebsd-dev/sys/net
History
John Baldwin 82334850ea Add an external mbuf buffer type that holds multiple unmapped pages. 
Unmapped mbufs allow sendfile to carry multiple pages of data in a
single mbuf, without mapping those pages.  It is a requirement for
Netflix's in-kernel TLS, and provides a 5-10% CPU savings on heavy web
serving workloads when used by sendfile, due to effectively
compressing socket buffers by an order of magnitude, and hence
reducing cache misses.

For this new external mbuf buffer type (EXT_PGS), the ext_buf pointer
now points to a struct mbuf_ext_pgs structure instead of a data
buffer.  This structure contains an array of physical addresses (this
reduces cache misses compared to an earlier version that stored an
array of vm_page_t pointers).  It also stores additional fields needed
for in-kernel TLS such as the TLS header and trailer data that are
currently unused.  To more easily detect these mbufs, the M_NOMAP flag
is set in m_flags in addition to M_EXT.

Various functions like m_copydata() have been updated to safely access
packet contents (using uiomove_fromphys()), to make things like BPF
safe.

NIC drivers advertise support for unmapped mbufs on transmit via a new
IFCAP_NOMAP capability.  This capability can be toggled via the new
'nomap' and '-nomap' ifconfig(8) commands.  For NIC drivers that only
transmit packet contents via DMA and use bus_dma, adding the
capability to if_capabilities and if_capenable should be all that is
required.

If a NIC does not support unmapped mbufs, they are converted to a
chain of mapped mbufs (using sf_bufs to provide the mapping) in
ip_output or ip6_output.  If an unmapped mbuf requires software
checksums, it is also converted to a chain of mapped mbufs before
computing the checksum.

Submitted by:	gallatin (earlier version)
Reviewed by:	gallatin, hselasky, rrs
Discussed with:	ae, kp (firewalls)
Relnotes:	yes
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20616
2019-06-29 00:48:33 +00:00
..
altq Reduce the time it takes the kernel to install a new PF config containing a large number of queues 2019-02-11 05:17:31 +00:00
bpf_buffer.c Add an external mbuf buffer type that holds multiple unmapped pages. 2019-06-29 00:48:33 +00:00
bpf_buffer.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
bpf_filter.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
bpf_jitter.c Make UMA and malloc(9) return non-executable memory in most cases. 2018-06-13 17:04:41 +00:00
bpf_jitter.h Make UMA and malloc(9) return non-executable memory in most cases. 2018-06-13 17:04:41 +00:00
bpf_zerocopy.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
bpf_zerocopy.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
bpf.c Add an external mbuf buffer type that holds multiple unmapped pages. 2019-06-29 00:48:33 +00:00
bpf.h Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
bpfdesc.h Rework locking in BPF code to remove rwlock from fast path. 2019-05-13 13:45:28 +00:00
bridgestp.c bridge: Fix panic if the STP root is removed 2019-03-15 11:21:20 +00:00
bridgestp.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
dlt.h Re-apply r190640. 2018-05-31 09:11:21 +00:00
ethernet.h Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
firewire.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
ieee8023ad_lacp.c Select lacp egress ports based on NUMA domain 2019-05-03 14:43:21 +00:00
ieee8023ad_lacp.h Select lacp egress ports based on NUMA domain 2019-05-03 14:43:21 +00:00
ieee_oui.h net: adjust randomized address bits 2019-04-17 17:18:43 +00:00
if_arp.h Improve ARP logging. 2019-03-09 01:12:59 +00:00
if_bridge.c if_bridge(4): Complete bpf auditing of local traffic over the bridge 2019-05-29 01:08:30 +00:00
if_bridgevar.h Allow different bridge types to coexist 2018-05-11 05:00:40 +00:00
if_clone.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
if_clone.h Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
if_dead.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
if_debug.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
if_disc.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
if_dl.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
if_edsc.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
if_enc.c New pfil(9) KPI together with newborn pfil API and control utility. 2019-01-31 23:01:03 +00:00
if_enc.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
if_epair.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
if_ethersubr.c Restructure mbuf send tags to provide stronger guarantees. 2019-05-24 22:30:40 +00:00
if_fwsubr.c Improve copy-and-pasted versions of SIOCGIFADDR. 2018-03-27 20:51:49 +00:00
if_gif.c Add handling for appearing/disappearing of ingress addresses to if_gif(4). 2018-10-21 18:06:15 +00:00
if_gif.h Add handling for appearing/disappearing of ingress addresses to if_gif(4). 2018-10-21 18:06:15 +00:00
if_gre.c Add GRE-in-UDP encapsulation support as defined in RFC8086. 2019-04-24 09:05:45 +00:00
if_gre.h Add GRE-in-UDP encapsulation support as defined in RFC8086. 2019-04-24 09:05:45 +00:00
if_ipsec.c Allow configuration of several ipsec interfaces with the same tunnel 2018-11-16 14:21:57 +00:00
if_ipsec.h
if_lagg.c Restore the comment removed in r348745. 2019-06-06 17:20:35 +00:00
if_lagg.h Select lacp egress ports based on NUMA domain 2019-05-03 14:43:21 +00:00
if_llatbl.c Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
if_llatbl.h Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
if_llc.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
if_loop.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
if_me.c Add the check that current VNET is ready and access to srchash is allowed. 2018-10-23 13:11:45 +00:00
if_media.c Finish removing FDDI and tokenring media support. 2018-04-23 21:10:33 +00:00
if_media.h if_media: Add new 2.5G/5G/25G/40G/50G/100G/200G/400G media types 2018-08-22 18:19:56 +00:00
if_mib.c
if_mib.h
if_pflog.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
if_pfsync.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
if_sppp.h
if_spppfr.c
if_spppsubr.c Replace read_random(9) with more appropriate arc4rand(9) KPIs 2019-04-04 01:02:50 +00:00
if_stf.c Do not perform DAD on stf(4) interfaces. 2019-03-30 18:00:44 +00:00
if_tap.h tun/tap: merge and rename to tuntap 2019-05-08 02:32:11 +00:00
if_tun.h
if_tuntap.c Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
if_types.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
if_var.h Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
if_vlan_var.h Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
if_vlan.c Restructure mbuf send tags to provide stronger guarantees. 2019-05-24 22:30:40 +00:00
if_vxlan.c Convert all IPv4 and IPv6 multicast memberships into using a STAILQ 2019-06-25 11:54:41 +00:00
if_vxlan.h Add support for IPv6 scoped addresses to vxlan 2017-12-30 04:03:53 +00:00
if.c Need to wait for epoch callbacks to complete before detaching a 2019-06-28 10:49:04 +00:00
if.h Add an external mbuf buffer type that holds multiple unmapped pages. 2019-06-29 00:48:33 +00:00
ifdi_if.m iflib(9): Add support for cloning pseudo interfaces 2018-05-11 20:08:28 +00:00
iflib_clone.c - Remove the unused ifc_link_irq and ifc_mtx_name members of struct iflib_ctx. 2019-05-06 20:56:41 +00:00
iflib_private.h - Replace unused and only ever written to members of public iflib(9) 2019-06-15 11:07:41 +00:00
iflib.c o In iflib_txq_drain(): 2019-06-26 15:28:21 +00:00
iflib.h - Replace unused and only ever written to members of public iflib(9) 2019-06-15 11:07:41 +00:00
ifq.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
mp_ring.c - Merge r338254 from cxgbe(4): 2019-05-09 11:34:46 +00:00
mp_ring.h mp_ring: avoid items offset difference between iflib and mp_ring 2019-01-03 23:06:05 +00:00
mppc.h
mppcc.c
mppcd.c
netisr_internal.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
netisr.c Restructure mbuf send tags to provide stronger guarantees. 2019-05-24 22:30:40 +00:00
netisr.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
netmap_legacy.h netmap: add support for multiple host rings 2019-03-18 12:22:23 +00:00
netmap_user.h netmap: add support for multiple host rings 2019-03-18 12:22:23 +00:00
netmap_virt.h netmap: align codebase to the current upstream (760279cfb2730a585) 2018-12-05 11:57:16 +00:00
netmap.h netmap: add support for multiple host rings 2019-03-18 12:22:23 +00:00
paravirt.h
pfil.c Most Ethernet drivers that potentially can run a pfil(9) hook with 2019-03-10 17:20:09 +00:00
pfil.h Most Ethernet drivers that potentially can run a pfil(9) hook with 2019-03-10 17:20:09 +00:00
pfkeyv2.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
pfvar.h pf :Use counter(9) in pf tables. 2019-03-15 11:08:44 +00:00
ppp_defs.h
radix_mpath.c Switch RIB and RADIX_NODE_HEAD lock from rwlock(9) to rmlock(9). 2018-06-16 08:26:23 +00:00
radix_mpath.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
radix.c Switch RIB and RADIX_NODE_HEAD lock from rwlock(9) to rmlock(9). 2018-06-16 08:26:23 +00:00
radix.h Fix typo. 2018-06-16 19:21:09 +00:00
raw_cb.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
raw_cb.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
raw_usrreq.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
rndis.h
route_var.h Existense of PCB route caching doesn't allow us to use new fast route 2019-05-08 23:39:24 +00:00
route.c Fix gateway setup for the interface routes. 2019-05-22 21:20:15 +00:00
route.h Existense of PCB route caching doesn't allow us to use new fast route 2019-05-08 23:39:24 +00:00
rss_config.c
rss_config.h
rtsock.c When sending a routing message, don't allow the user to set the 2019-04-14 10:18:14 +00:00
sff8436.h
sff8472.h ifconfig(8): Display extended compliance code string for SFP transceivers 2017-12-05 18:42:07 +00:00
slcompress.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
slcompress.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
toeplitz.c
toeplitz.h
vnet.c With more excessive use of modules, more kernel parts working with 2018-10-30 20:45:15 +00:00
vnet.h [PowerPC64] Don't mark module data as static 2019-06-25 17:15:44 +00:00