freebsd-dev/sys/vm
Doug Moore 8df38859d0 radix_trie: replace node count with popmap
Replace the 'count' field in a trie node with a bitmap that
identifies non-NULL children. Drop the 'last' field, and use the
last bit set in the bitmap instead.  In lookup_le, lookup_ge,
remove, and reclaim_all, use the bitmap to find the
previous/next/only/every non-null child in constant time by
examining the bitmask instead of looping across array elements
and null-checking them one-by-one.

A buildworld test suggests that this reduces the cycle count on
those functions that eliminate some null-checks by 4.9%, 1.5%,
0.0% and 13.3%.
Reviewed by:	alc
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D40775
2023-07-07 11:09:36 -05:00
..
_vm_phys.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
_vm_radix.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
device_pager.c vm_object_kvme_type(): reimplement by embedding kvme_type into pagerops 2021-05-13 20:10:35 +03:00
memguard.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
memguard.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
phys_pager.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
pmap.h Add pmap_enter(9) PMAP_ENTER_LARGEPAGE flag and implement it on amd64. 2020-09-09 21:50:24 +00:00
redzone.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
redzone.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
sg_pager.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
swap_pager.c vm: fix a number of functions to match the expected prototypes 2023-04-25 19:58:18 +02:00
swap_pager.h Make swap_pager_freespace() global 2022-12-09 14:15:37 +02:00
uma_core.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
uma_dbg.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
uma_dbg.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
uma_int.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
uma.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
vm_domainset.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
vm_domainset.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
vm_dumpset.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
vm_extern.h kmem_malloc/free: Use void * instead of vm_offset_t for kernel pointers. 2022-09-22 15:09:19 -07:00
vm_fault.c vm_fault: move FAULT_* return codes out of range for Mach errors 2023-06-28 00:03:14 +03:00
vm_glue.c vm: ansify 2023-02-13 18:23:21 +00:00
vm_init.c tslog: Annotate parts of SYSINIT cpu 2023-06-04 10:16:35 -07:00
vm_kern.c tslog: Annotate parts of SYSINIT cpu 2023-06-04 10:16:35 -07:00
vm_kern.h
vm_map.c vm: Fix anonymous memory clustering under ASLR 2023-06-26 23:42:48 -05:00
vm_map.h vm: Fix anonymous memory clustering under ASLR 2023-06-26 23:42:48 -05:00
vm_meter.c vmmeter(): Fix detection of the named swap objects 2022-02-02 11:39:58 +02:00
vm_mmap.c vm: Fix anonymous memory clustering under ASLR 2023-06-26 23:42:48 -05:00
vm_object.c Drop space in "vm object" lock name to improve wchan 2023-02-15 08:31:17 -05:00
vm_object.h vm: add VM_OBJECT_UNLOCK 2023-03-11 11:08:21 +00:00
vm_page.c vm_phys: add binary segment search 2023-06-16 01:43:45 -05:00
vm_page.h vm_phys: add binary segment search 2023-06-16 01:43:45 -05:00
vm_pageout.c vm: reduce lock contention when processing vm batchqueues 2022-12-14 14:34:07 -05:00
vm_pageout.h protosw: retire pr_drain and use EVENTHANDLER(9) directly 2022-08-17 11:50:31 -07:00
vm_pagequeue.h vm: centralize VM_BATCHQUEUE_SIZE definition 2023-01-21 14:30:00 -05:00
vm_pager.c vm: fix a number of functions to match the expected prototypes 2023-04-25 19:58:18 +02:00
vm_pager.h vm_pager: add method to veto page allocation 2022-12-09 14:15:37 +02:00
vm_param.h
vm_phys.c vm_phys: Fix typo in 9e81742892 2023-06-16 03:12:42 -05:00
vm_phys.h vm_phys: add binary segment search 2023-06-16 01:43:45 -05:00
vm_radix.c radix_trie: replace node count with popmap 2023-07-07 11:09:36 -05:00
vm_radix.h vm_radix: drop unused function; use bool. 2023-06-20 23:52:27 -05:00
vm_reserv.c spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
vm_reserv.h spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD 2023-05-12 10:44:03 -06:00
vm_swapout_dummy.c
vm_swapout.c vm: Use __diagused for variables only used in KASSERT(). 2022-04-13 16:08:20 -07:00
vm_unix.c Prepare to handle non-trivial errors from vm_map_delete(). 2020-09-09 21:34:31 +00:00
vm.h vm/vm.h: cleanup 2022-09-17 11:58:52 +03:00
vnode_pager.c vnode_pager_input: return runningbufspace back 2023-03-26 00:55:29 +02:00
vnode_pager.h