d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
520aafe3ec
freebsd-skq/sbin
History
jhb 520aafe3ec Add an external mbuf buffer type that holds multiple unmapped pages. 
Unmapped mbufs allow sendfile to carry multiple pages of data in a
single mbuf, without mapping those pages.  It is a requirement for
Netflix's in-kernel TLS, and provides a 5-10% CPU savings on heavy web
serving workloads when used by sendfile, due to effectively
compressing socket buffers by an order of magnitude, and hence
reducing cache misses.

For this new external mbuf buffer type (EXT_PGS), the ext_buf pointer
now points to a struct mbuf_ext_pgs structure instead of a data
buffer.  This structure contains an array of physical addresses (this
reduces cache misses compared to an earlier version that stored an
array of vm_page_t pointers).  It also stores additional fields needed
for in-kernel TLS such as the TLS header and trailer data that are
currently unused.  To more easily detect these mbufs, the M_NOMAP flag
is set in m_flags in addition to M_EXT.

Various functions like m_copydata() have been updated to safely access
packet contents (using uiomove_fromphys()), to make things like BPF
safe.

NIC drivers advertise support for unmapped mbufs on transmit via a new
IFCAP_NOMAP capability.  This capability can be toggled via the new
'nomap' and '-nomap' ifconfig(8) commands.  For NIC drivers that only
transmit packet contents via DMA and use bus_dma, adding the
capability to if_capabilities and if_capenable should be all that is
required.

If a NIC does not support unmapped mbufs, they are converted to a
chain of mapped mbufs (using sf_bufs to provide the mapping) in
ip_output or ip6_output.  If an unmapped mbuf requires software
checksums, it is also converted to a chain of mapped mbufs before
computing the checksum.

Submitted by:	gallatin (earlier version)
Reviewed by:	gallatin, hselasky, rrs
Discussed with:	ae, kp (firewalls)
Relnotes:	yes
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20616
2019-06-29 00:48:33 +00:00
..
adjkerntz various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
bectl bectl(8): create non-recursive boot environments 2019-06-27 14:03:32 +00:00
bsdlabel Move disktab to sbin/bsdlabel/ 2018-09-18 20:52:24 +00:00
camcontrol Replay r349341 by imp accidentally reverted by r349352 2019-06-25 06:14:26 +00:00
ccdconfig ccdconfig: Move VCS tags to be more consistent with our style. 2017-12-30 00:26:42 +00:00
clri In preparation for adding inode check-hashes, clean up and 2018-11-13 21:40:56 +00:00
comcontrol various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
conscontrol various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
ddb Move ddb.conf to sbin/ddb/ and switch to CONFS. 2018-08-11 13:25:39 +00:00
decryptcore EKCD: Add Chacha20 encryption mode 2019-05-23 20:12:24 +00:00
devd acpi_dock(4): Notify devd(8) on dock status change. 2019-05-31 15:44:33 +00:00
devfs Move all devfs related files to sbin/devfs/ 2018-08-22 15:55:23 +00:00
devmatch Regularize the Netflix copyright 2019-02-04 21:28:25 +00:00
dhclient Free DHCP options with length zero. 2019-06-26 20:19:48 +00:00
dmesg General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
dump Clarify that the -r and -R options can be used only to create level 0 dumps. 2019-06-14 06:10:15 +00:00
dumpfs The goal of this change is to prevent accidental foot shooting by 2018-02-08 23:06:58 +00:00
dumpon EKCD: Add Chacha20 encryption mode 2019-05-23 20:12:24 +00:00
etherswitchcfg Add the 'drop tagged' flag support for ethernet switch ports. 2019-06-28 22:12:43 +00:00
fdisk Add DragonFly's partition number to fdisk(8) and diskmbr.h 2019-05-18 00:22:28 +00:00
ffsinfo In preparation for adding inode check-hashes, clean up and 2018-11-13 21:40:56 +00:00
fsck explain ``fsck -f'' more in detail 2019-03-08 10:03:16 +00:00
fsck_ffs This revision began as a simple change to eliminate an uninitialized warning 2019-05-21 22:24:38 +00:00
fsck_msdosfs In ask(): override default option if any of alwaysyes/alwaysno/rdonly is 2019-06-15 07:23:06 +00:00
fsdb After a crash, a file that extends into indirect blocks may end up 2019-02-25 21:58:19 +00:00
fsirand Continuing efforts to provide hardening of FFS. This change adds a 2018-12-11 22:14:37 +00:00
gbde gbde(8) - simplify randomisation with arc4random_buf 2019-02-11 00:11:02 +00:00
geom Add the "-t" option to geom(8) utility, to display geoms hierarchy. 2018-09-14 15:29:45 +00:00
ggate ggated: do not expose stack data in sendfail() 2018-12-04 15:25:15 +00:00
growfs Normally when an attempt is made to mount a UFS/FFS filesystem whose 2018-12-06 00:09:39 +00:00
gvinum gvinum: revert WARNS change in Makefile 2018-06-17 01:39:22 +00:00
hastctl various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
hastd Drop "All rights reserved" from my copyright statements. 2019-03-06 22:11:45 +00:00
ifconfig Add an external mbuf buffer type that holds multiple unmapped pages. 2019-06-29 00:48:33 +00:00
init init: pedantic correction to "can't exec" script warning 2019-03-26 19:38:25 +00:00
ipf Create a link to the ipmon.conf.5 man page as documented in ipmon.5. 2019-06-27 12:37:44 +00:00
ipfw Restore ipfw(8)'s compact output support broken after r331668. 2019-06-25 09:08:24 +00:00
iscontrol various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
kldconfig various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
kldload various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
kldstat Allow three digits of module id without breaking table alignment. 2018-07-02 09:14:00 +00:00
kldunload various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
ldconfig Make ldconfig(8) atomic, by removing an unneccessary call to unlink(2) 2018-08-09 11:46:12 +00:00
md5 Close filedescriptors when done with them. 2019-04-19 06:49:46 +00:00
mdconfig Use VOP_ADVISE() with POSIX_FADV_DONTNEED instead of IO_DIRECT to 2018-12-21 08:15:31 +00:00
mdmfs mdmfs(8): use -o reserve with malloc-backed md(4) 2019-02-16 23:57:38 +00:00
mknod General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
mksnap_ffs various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
mount Exercise some care before sending SIGHUP to mountd. 2019-04-08 18:16:49 +00:00
mount_cd9660 Advise reader to also see mdconfig(8) in mount_cd9660(8). 2018-08-11 08:34:24 +00:00
mount_fusefs Rename fuse(4) to fusefs(4) 2019-03-20 21:48:43 +00:00
mount_msdosfs mount_msdosfs: do not fail mounts requiring locale name conversion table 2018-10-27 16:41:34 +00:00
mount_nfs General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
mount_nullfs General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
mount_udf General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
mount_unionfs General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
natd DIRDEPS_BUILD: Update dependencies. 2017-10-31 00:07:04 +00:00
newfs Allow dashes as a valid character in UFS labels. 2019-01-29 10:21:41 +00:00
newfs_msdos Added option to cluster-align the start of the root directory. 2018-06-15 06:03:40 +00:00
nfsiod General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
nos-tun various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
nvmecontrol Better formatting for the logpage section 2019-05-17 18:15:47 +00:00
pfctl pfctl: Fix ifgroup check 2019-04-19 10:52:54 +00:00
pfilctl Hopefully fix compilation by other compilers. 2019-02-01 00:34:18 +00:00
pflogd DIRDEPS_BUILD: Update dependencies. 2017-10-31 00:07:04 +00:00
ping Use caph_enter_casper() in ping(8). 2018-12-18 16:47:03 +00:00
ping6 General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
quotacheck Normally when an attempt is made to mount a UFS/FFS filesystem whose 2018-12-06 00:09:39 +00:00
rcorder rcorder(8): add support for /etc/rc.resume, so it calls "rcorder -k resume" 2018-10-27 17:21:13 +00:00
reboot Update the manpage text to show the output generated by the first-stage 2019-04-30 17:42:05 +00:00
recoverdisk recoverdisk(1): fclose() file supplied via '-r readlist' parameter when 2019-02-07 14:40:28 +00:00
resolvconf sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
restore Re-enable reading byte swapped NFS_MAGIC dumps. 2018-08-11 16:12:23 +00:00
route route(8): clarify -prefixlen description 2019-01-10 00:10:12 +00:00
routed When bind fails, make sure we closed the socket we tried to bind the 2017-12-28 05:34:24 +00:00
rtsol Capsicumize rtsol(8) and rtsold(8). 2019-01-05 16:05:39 +00:00
savecore cap_fileargs: chase r346315, update fileargs_init in consumers 2019-04-17 16:18:14 +00:00
sconfig DIRDEPS_BUILD: Update dependencies. 2017-10-31 00:07:04 +00:00
setkey General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
shutdown shutdown: Fix r327476 by adding init 2018-01-02 09:02:42 +00:00
spppcontrol various: general adoption of SPDX licensing ID tags. 2017-11-27 15:37:16 +00:00
sunlabel General further adoption of SPDX licensing ID tags. 2017-11-20 19:49:47 +00:00
swapon You can't use block special nodes for swap, so don't let that happen. 2019-06-22 19:27:09 +00:00
sysctl evdev: export event device properties through sysctl interface 2019-02-24 18:47:04 +00:00
tests Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed 2016-05-04 23:20:53 +00:00
tunefs The size of the UFS soft-updates journal must be a multiple of the 2019-03-02 21:22:56 +00:00
umount umount: remove sync(2) call when used with -f 2018-09-13 13:57:42 +00:00
veriexec Add verifying manifest loader for mac_veriexec 2019-02-26 06:17:23 +00:00
zfsbootcfg DIRDEPS_BUILD: Update dependencies. 2017-10-31 00:07:04 +00:00
Makefile Remove NAND and NANDFS support 2019-06-25 04:50:09 +00:00
Makefile.amd64 NVME support is only for x86 and powerpc64. 2018-06-14 01:15:19 +00:00
Makefile.arm
Makefile.i386 NVME support is only for x86 and powerpc64. 2018-06-14 01:15:19 +00:00
Makefile.inc
Makefile.mips
Makefile.powerpc64 NVME support is only for x86 and powerpc64. 2018-06-14 01:15:19 +00:00
Makefile.sparc64