freebsd-skq/sys/kern
Andrew Gallatin 52cd25eb1a mbuf: enable ext_pgs ("unmapped") mbufs by default
Ext_pg mbufs allow carrying multiple pages per mbuf. This
reduces mbuf linked list traversals, especially in socket
buffers, thereby reducing cache misses and CPU use for
applications using sendfile.  Note that ext_pages use
unmapped pages, eliminating KVA mapping costs on 32-bit
platforms.

Ext_pg mbufs are also required for ktls (KERN_TLS), and having
them disabled by default is a stumbling block for those
wishing to enable ktls.

Reviewed-by:	jhb, glebius
Sponsored by:	Netfix
2021-01-08 13:43:30 -05:00
..
bus_if.m Remove trailing white space. 2020-02-26 16:22:28 +00:00
capabilities.conf Add aio_writev and aio_readv 2021-01-02 19:57:58 -07:00
clock_if.m
cpufreq_if.m
device_if.m
firmw.S Use a template assembly file for firmware object files. 2020-12-17 20:31:17 +00:00
genassym.sh
genoffset.c Merge td_epochnest with td_no_sleeping. 2019-10-29 17:28:25 +00:00
genoffset.sh
imgact_aout.c Get rid of sv_errtbl and SV_ABI_ERRNO(). 2020-09-17 11:39:33 +00:00
imgact_binmisc.c imgact_binmisc: limit the extent of match on incoming entries 2020-11-08 04:24:29 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c Add ELF flag to disable ASLR stack gap. 2020-12-18 23:14:39 +00:00
imgact_shell.c
init_main.c dtrace: stop using eventhandlers for the part compiled into the kernel 2020-11-23 18:27:21 +00:00
init_sysent.c Regenerate syscall files after reallocation of aio_writev/aio_readv 2021-01-07 19:50:32 -07:00
kern_acct.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_alq.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_clock.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_clocksource.c
kern_condvar.c
kern_conf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_cons.c Remove sysctl_kern_consmute() 2020-10-05 15:54:19 +00:00
kern_context.c
kern_cpu.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_cpuset.c kern: cpuset: allow jails to modify child jails' roots 2020-12-19 03:30:06 +00:00
kern_ctf.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
kern_descrip.c fd: use seqc_read_notmodify when translating fds 2021-01-07 23:30:04 +00:00
kern_dtrace.c dtrace: stop using eventhandlers for the part compiled into the kernel 2020-11-23 18:27:21 +00:00
kern_dump.c Always use 64-bit physical addresses for dump_avail[] in minidumps 2020-12-03 17:12:31 +00:00
kern_environment.c Move kernel env global variables, etc to sys/kenv.h 2020-10-07 06:16:37 +00:00
kern_et.c Remove NO_EVENTTIMERS support 2020-11-19 02:50:48 +00:00
kern_event.c kqueue: save space by using only one func pointer for assertions 2020-11-09 00:04:35 +00:00
kern_exec.c Add ELF flag to disable ASLR stack gap. 2020-12-18 23:14:39 +00:00
kern_exit.c ddb ps: Use the pidhash to enumerate processes not in allproc. 2020-12-31 16:00:05 -08:00
kern_fail.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_ffclock.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
kern_fork.c linux(4) clone(2): Correctly handle CLONE_FS and CLONE_FILES 2020-11-17 21:20:11 +00:00
kern_hhook.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_idle.c
kern_intr.c Allow swi_sched() to be called from NMI context. 2020-07-25 15:19:38 +00:00
kern_jail.c jail: revert the attachment part of b4e87a6329 2020-12-31 19:55:49 -08:00
kern_kcov.c Mark COVERAGE and KCOV as part of KCSAN 2020-08-19 14:11:25 +00:00
kern_khelp.c
kern_kthread.c Provide ABI modules hooks for process exec/exit and thread exit. 2020-11-23 17:29:25 +00:00
kern_ktr.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_ktrace.c Move KTRUSERRET() from userret() to ast(). It's a really long 2020-10-03 12:03:08 +00:00
kern_linker.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_lock.c locks: push lock_delay_arg_init calls down 2020-11-24 03:49:37 +00:00
kern_lockf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_lockstat.c Add lockmgr(9) probes to the lockstat DTrace provider. 2019-08-21 23:43:58 +00:00
kern_loginclass.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
kern_malloc.c malloc: make malloc_large closer to standalone 2020-11-16 17:56:58 +00:00
kern_mbuf.c mbuf: enable ext_pgs ("unmapped") mbufs by default 2021-01-08 13:43:30 -05:00
kern_mib.c jail: Consistently handle the pr_allow bitmask 2020-12-26 20:25:02 -08:00
kern_module.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_mtxpool.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_mutex.c locks: push lock_delay_arg_init calls down 2020-11-24 03:49:37 +00:00
kern_ntptime.c Add kern_ntp_adjtime(9). 2020-12-04 18:56:44 +00:00
kern_osd.c
kern_physio.c Remove alignment requirements for KVA buffer mapping. 2020-11-29 01:30:17 +00:00
kern_pmc.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_poll.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
kern_priv.c jail: Consistently handle the pr_allow bitmask 2020-12-26 20:25:02 -08:00
kern_proc.c Add a proc_off_p_hash helper variable. 2020-12-31 16:00:33 -08:00
kern_procctl.c correct procctl(PROC_PROTMAX_STATUS _NOFORCE return 2020-05-01 14:30:59 +00:00
kern_prot.c jail: Consistently handle the pr_allow bitmask 2020-12-26 20:25:02 -08:00
kern_racct.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
kern_rangelock.c rangelock: add rangelock_cookie_assert 2019-09-15 02:59:53 +00:00
kern_rctl.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
kern_resource.c thread: batch resource limit free calls 2020-11-14 19:21:46 +00:00
kern_rmlock.c rms: several cleanups + debug read lockers handling 2020-11-07 16:57:53 +00:00
kern_rwlock.c locks: push lock_delay_arg_init calls down 2020-11-24 03:49:37 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c sendfile: Clear page pointers when handling a pager error 2021-01-03 11:50:31 -05:00
kern_sharedpage.c random(4) FenestrasX: Push root seed version to arc4random(3) 2020-10-10 21:52:00 +00:00
kern_shutdown.c Add an option for entering KDB on recursive panics 2020-11-19 18:03:40 +00:00
kern_sig.c sig_intr(9): return early if AST is not scheduled. 2020-10-08 22:34:34 +00:00
kern_switch.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
kern_sx.c locks: push lock_delay_arg_init calls down 2020-11-24 03:49:37 +00:00
kern_synch.c Handle PCATCH in blockcount_sleep() so it can be interrupted. 2020-04-21 17:13:06 +00:00
kern_syscalls.c Move syscall_thread_{enter,exit}() into the slow path. This is only 2020-11-08 15:54:59 +00:00
kern_sysctl.c Move inner loop logic out of sysctl_sysctl_next_ls(). 2020-11-30 21:59:52 +00:00
kern_tc.c Remove double-calls to tc_get_timecount() to warm timecounters. 2020-06-10 22:30:32 +00:00
kern_thr.c Provide ABI modules hooks for process exec/exit and thread exit. 2020-11-23 17:29:25 +00:00
kern_thread.c thread: staticize thread_reap and move td_allocdomain 2020-11-26 06:59:27 +00:00
kern_time.c Stop using eventhandlers for itimers subsystem exec and exit hooks. 2020-11-21 21:43:36 +00:00
kern_timeout.c callout(9): Remove some leftover APM BIOS support 2020-11-27 20:46:02 +00:00
kern_tslog.c
kern_ubsan.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_umtx.c freebsd32: take the _umtx_op struct definitions back 2020-11-23 00:58:14 +00:00
kern_uuid.c validate_uuid: absorb the rest of parse_uuid with a flags arg 2020-04-15 18:39:12 +00:00
kern_xxx.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
ksched.c
link_elf_obj.c link_elf_obj: Colour VM objects 2020-10-19 16:57:59 +00:00
link_elf.c malloc: delegate M_EXEC handling to dedicacted routines 2020-10-30 20:02:32 +00:00
linker_if.m
Make.tags.inc Remove a couple of harmless stray references to nandfs. 2019-06-25 16:39:25 +00:00
Makefile sys/kern sysent: re-add dependency on capabilities.conf 2020-02-12 19:06:34 +00:00
makesyscalls.sh makesyscalls.sh: improve the 'this is going away' message 2020-07-28 01:05:40 +00:00
md4c.c
md5c.c
msi_if.m o Add iommu de-initialization method for MSI interface. 2020-10-24 20:09:27 +00:00
p1003_1b.c
pic_if.m
posix4_mib.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sched_4bsd.c sched: fix an incorrect comparison in sched_lend_user_prio_cond 2020-11-15 01:54:44 +00:00
sched_ule.c sched: fix an incorrect comparison in sched_lend_user_prio_cond 2020-11-15 01:54:44 +00:00
serdev_if.m
stack_protector.c
subr_acl_nfs4.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
subr_acl_posix1e.c vfs: remove the obsolete privused argument from vaccess 2020-08-05 09:27:03 +00:00
subr_atomic64.c emulated atomic64: disable interrupts as the lock mechanism on !SMP 2020-01-03 18:29:20 +00:00
subr_autoconf.c
subr_blist.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_boot.c
subr_bufring.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_bus_dma.c Move the iommu stubs to a generic place, so they are available on all the 2020-10-23 21:27:48 +00:00
subr_bus.c newbus: use ssize_t to match sb's len and size, fix ordering of space check 2020-10-12 22:07:44 +00:00
subr_busdma_bufalloc.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_capability.c Remove unused SYSINIT macros for capability rights. 2020-03-26 15:02:37 +00:00
subr_clock.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_compressor.c clamp kernel dump compression level when using gzip 2020-02-20 23:53:48 +00:00
subr_counter.c Rationalize per-cpu zones. 2020-11-05 15:08:56 +00:00
subr_coverage.c Mark COVERAGE and KCOV as part of KCSAN 2020-08-19 14:11:25 +00:00
subr_csan.c Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions. 2020-09-19 11:06:41 +00:00
subr_devmap.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_devstat.c Fix asymmetry in devstat(9) calls by GEOM. 2020-10-24 21:07:10 +00:00
subr_disk.c Enable bioq 'car limit' added at r335066 at 128 bios. 2020-10-26 04:04:06 +00:00
subr_dummy_vdso_tc.c
subr_early.c
subr_epoch.c epoch: support non-preemptible epochs checking in_epoch() 2020-11-07 03:29:04 +00:00
subr_eventhandler.c
subr_fattime.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_filter.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_firmware.c Fix a panic when unloading firmware 2020-06-29 21:35:50 +00:00
subr_gtaskqueue.c Import kernel WireGuard support 2020-11-29 19:38:03 +00:00
subr_hash.c
subr_hints.c Move kernel env global variables, etc to sys/kenv.h 2020-10-07 06:16:37 +00:00
subr_intr.c Also pass interrupt binding request to non-root interrupt controllers. 2020-11-20 09:05:36 +00:00
subr_kdb.c Don't check P_INMEM in kdb_thr_*(). 2020-12-31 16:01:12 -08:00
subr_kobj.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_lock.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_log.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_mchain.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_module.c sysctl debug.dump_modinfo should recognize font module 2021-01-08 09:24:49 +02:00
subr_msgbuf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_param.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
subr_pcpu.c Add more per-cpu zones. 2020-11-09 00:34:23 +00:00
subr_pctrie.c Use SMR to provide safe unlocked lookup for pctries from SMR zones 2020-07-24 17:32:10 +00:00
subr_physmem.c Add support for hw.physmem tunable for ARM/ARM64/RISC-V platforms 2020-12-03 05:39:27 +00:00
subr_pidctrl.c
subr_power.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_prf.c remove %n support from printf(9) 2020-05-09 15:56:02 +00:00
subr_prng.c Add prng(9) API 2020-08-13 20:48:14 +00:00
subr_prof.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_rangeset.c
subr_rman.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_rtc.c Address whitespace nits in subr_rtc.c 2020-09-28 17:19:57 +00:00
subr_sbuf.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_scanf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_sfbuf.c
subr_sglist.c Step 4.2: start divorce of M_EXT and M_EXTPG 2020-05-03 00:37:16 +00:00
subr_sleepqueue.c Fix TDP_WAKEUP/thr_wake(curthread->td_tid) after r366428. 2020-12-13 19:45:42 +00:00
subr_smp.c smp: fix smp_rendezvous_cpus_retry usage before smp starts 2020-11-19 04:27:51 +00:00
subr_smr.c Use COUNTER_U64_DEFINE_EARLY() in places where it simplifies things. 2020-03-06 19:10:00 +00:00
subr_stack.c kern.tty_info_kstacks: add a compact format 2020-07-06 16:33:28 +00:00
subr_stats.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_syscall.c Allow some VOPs to return ERELOOKUP to indicate VFS operation restart at top level. 2020-11-13 09:42:32 +00:00
subr_taskqueue.c Add flag to struct task to mark the task as requiring network epoch. 2020-02-11 18:48:07 +00:00
subr_terminal.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_trap.c Move KTRUSERRET() from userret() to ast(). It's a really long 2020-10-03 12:03:08 +00:00
subr_turnstile.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_uio.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_unit.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_vmem.c vmem: Revert r364744 2020-12-01 16:06:31 +00:00
subr_witness.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sys_capability.c Enter a write sequence when updating rights. 2020-03-19 15:39:45 +00:00
sys_eventfd.c Expose eventfd in the native API/ABI using a new __specialfd syscall 2020-12-27 12:57:26 +02:00
sys_generic.c Expose eventfd in the native API/ABI using a new __specialfd syscall 2020-12-27 12:57:26 +02:00
sys_getrandom.c Fix a typo in r356667 comment 2020-01-12 23:52:16 +00:00
sys_pipe.c pipe: follow up cleanup to previous 2020-11-25 22:53:21 +00:00
sys_procdesc.c procdesc: convert the zone to a malloc type 2020-11-09 00:05:21 +00:00
sys_process.c Get rid of sa->narg. It serves no purpose; use sa->callp->sy_narg instead. 2020-09-27 18:47:06 +00:00
sys_socket.c Add aio_writev and aio_readv 2021-01-02 19:57:58 -07:00
syscalls.c Regenerate syscall files after reallocation of aio_writev/aio_readv 2021-01-07 19:50:32 -07:00
syscalls.master Reallocate syscall numbers for aio_writev and aio_readv 2021-01-07 19:49:27 -07:00
systrace_args.c Regenerate syscall files after reallocation of aio_writev/aio_readv 2021-01-07 19:50:32 -07:00
sysv_ipc.c
sysv_msg.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sysv_sem.c sysv_sem: semusz depends on semume. 2020-10-22 09:28:11 +00:00
sysv_shm.c sysvshm: pass relevant uap members as arguments 2020-11-03 19:14:03 +00:00
tty_compat.c
tty_info.c kern.tty_info_kstacks: set compact format as default 2020-07-06 16:34:15 +00:00
tty_inq.c
tty_outq.c
tty_pts.c tty_pts: don't rely on tty header pollution for sys/mutex.h 2019-11-29 03:56:01 +00:00
tty_tty.c
tty_ttydisc.c Implement FLUSHO 2020-08-27 05:11:15 +00:00
tty.c Fix a race in tty_signal_sessleader() with unlocked read of s_leader. 2020-12-17 19:51:39 +00:00
uipc_accf.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
uipc_debug.c
uipc_domain.c - Move global network epoch definition to epoch.h, as more different 2020-01-15 03:34:21 +00:00
uipc_ktls.c Optionally bind ktls threads to NUMA domains 2020-12-19 21:46:09 +00:00
uipc_mbuf2.c m_pulldown(): Change an if () panic() into a KASSERT(). 2019-11-06 22:40:19 +00:00
uipc_mbuf.c Permit sending empty fragments for TLS 1.0. 2020-10-13 17:30:34 +00:00
uipc_mbufhash.c Implement mbuf hashing routines for IP over infiniband, IPoIB. 2020-10-22 09:17:56 +00:00
uipc_mqueue.c Ensure that dirent's d_off field is initialized 2021-01-03 11:50:31 -05:00
uipc_sem.c Split out cwd/root/jail, cmask state from filedesc table 2020-11-17 21:14:13 +00:00
uipc_shm.c Split out cwd/root/jail, cmask state from filedesc table 2020-11-17 21:14:13 +00:00
uipc_sockbuf.c Permit sending empty fragments for TLS 1.0. 2020-10-13 17:30:34 +00:00
uipc_socket.c kern: soclose: don't sleep on SO_LINGER w/ timeout=0 2020-12-04 04:39:48 +00:00
uipc_syscalls.c Only enable COMPAT_43 changes for syscalls ABI for a.out processes. 2019-08-11 19:16:07 +00:00
uipc_usrreq.c uipc: disable prediction in unp_pcb_lock_peer 2020-12-13 21:32:19 +00:00
vfs_acl.c vfs: fix trivial whitespace issues which don't interefere with blame 2020-07-10 09:01:36 +00:00
vfs_aio.c aio_fsync(2): Support O_DSYNC. 2021-01-08 13:15:56 +13:00
vfs_bio.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vfs_cache.c cache: just assign ni_resflags = NIRES_ABS 2021-01-08 13:57:10 +00:00
vfs_cluster.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vfs_default.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vfs_export.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
vfs_extattr.c vfs_extattr: Allow extattr names up to the full max 2020-05-14 03:01:23 +00:00
vfs_hash.c vfs: avoid exposing partially constructed vnodes 2020-09-05 00:26:03 +00:00
vfs_init.c vfs: fix trivial whitespace issues which don't interefere with blame 2020-07-10 09:01:36 +00:00
vfs_lookup.c vfs: add FAILIFEXISTS flag 2020-12-28 01:53:27 +00:00
vfs_mount.c vfs: denote vnode being a mount point with VIRF_MOUNTPOINT 2021-01-03 06:50:06 +00:00
vfs_mountroot.c vfs: denote vnode being a mount point with VIRF_MOUNTPOINT 2021-01-03 06:50:06 +00:00
vfs_subr.c vfs: keep seqc unchanged as long as the vnode is accessible via SMR 2021-01-03 21:22:16 +00:00
vfs_syscalls.c vfs: stop open-coding setting WILLBEDIR flag 2020-12-28 01:53:27 +00:00
vfs_vnops.c open(2): Add O_DSYNC flag. 2021-01-08 13:15:56 +13:00
vnode_if.src vfs: prevent avoidable evictions on mkdir of existing directories 2020-10-22 19:28:12 +00:00