freebsd-skq/sys/vm
Jeff Roberson d4665eaa66 Implement a safe memory reclamation feature that is tightly coupled with UMA.
This is in the same family of algorithms as Epoch/QSBR/RCU/PARSEC but is
a unique algorithm.  This has 3x the performance of epoch in a write heavy
workload with less than half of the read side cost.  The memory overhead
is significantly lessened by limiting the free-to-use latency.  A synthetic
test uses 1/20th of the memory vs Epoch.  There is significant further
discussion in the comments and code review.

This code should be considered experimental.  I will write a man page after
it has settled.  After further validation the VM will begin using this
feature to permit lockless page lookups.

Both markj and cperciva tested on arm64 at large core counts to verify
fences on weaker ordering architectures.  I will commit a stress testing
tool in a follow-up.

Reviewed by:	mmacy, markj, rlibby, hselasky
Discussed with:	sbahara
Differential Revision:	https://reviews.freebsd.org/D22586
2020-01-31 00:49:51 +00:00
..
_vm_radix.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
default_pager.c SPDX: wrong license. 2017-11-30 15:45:42 +00:00
device_pager.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
memguard.c vm: add missing CLTFLAG_MPSAFE annotations 2020-01-12 05:08:57 +00:00
memguard.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
phys_pager.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
pmap.h Remove page locking from pmap_mincore(). 2019-10-16 22:03:27 +00:00
redzone.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
redzone.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
sg_pager.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
swap_pager.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
swap_pager.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
uma_core.c Implement a safe memory reclamation feature that is tightly coupled with UMA. 2020-01-31 00:49:51 +00:00
uma_dbg.c Fix compilation failures on different arches that have vm_machdep.c not 2019-01-15 19:33:47 +00:00
uma_dbg.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
uma_int.h Implement a safe memory reclamation feature that is tightly coupled with UMA. 2020-01-31 00:49:51 +00:00
uma.h Implement a safe memory reclamation feature that is tightly coupled with UMA. 2020-01-31 00:49:51 +00:00
vm_domainset.c Add a !NUMA definition for vm_domainset_iter_policy_ref_init(). 2018-10-24 17:09:20 +00:00
vm_domainset.h Refactor domainset iterators for use by malloc(9) and UMA. 2018-10-23 16:35:58 +00:00
vm_extern.h Improve MD page fault handlers. 2019-09-27 18:43:36 +00:00
vm_fault.c Restore OOM logic on page fault after r357026. 2020-01-29 12:02:47 +00:00
vm_glue.c Consistently use busy and vm_page_valid() rather than touching page bits 2020-01-23 04:54:49 +00:00
vm_init.c Simplify VM and UMA startup by eliminating boot pages. Instead use careful 2020-01-16 05:01:21 +00:00
vm_kern.c Consistently use busy and vm_page_valid() rather than touching page bits 2020-01-23 04:54:49 +00:00
vm_kern.h Do not reserve KVA for paging bufs in vm_ksubmap_init(), since now 2019-01-16 20:14:16 +00:00
vm_map.c Most uses of vm_map_clip_start follow a call to vm_map_lookup. Define 2020-01-24 07:48:11 +00:00
vm_map.h Revert r355505. The code that it allowed to compile has been removed. 2019-12-09 05:09:46 +00:00
vm_meter.c Simplify anonymous memory handling with an OBJ_ANON flag. This eliminates 2019-11-19 23:19:43 +00:00
vm_mmap.c kern_mmap: restore character deleted in transit 2020-01-04 23:51:44 +00:00
vm_object.c Handle a race of collapse with a retrying fault. 2020-01-24 19:42:53 +00:00
vm_object.h Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
vm_page.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
vm_page.h Remove some unused functions. 2019-12-28 19:04:29 +00:00
vm_pageout.c Restore loop break in vm_pageout_lowmem(). 2020-01-14 03:27:57 +00:00
vm_pageout.h Fix OOM handling of some corner cases. 2019-08-16 09:43:49 +00:00
vm_pagequeue.h Simplify vm_pageout_init_domain() and add a "big picture" comment. 2019-11-22 16:31:43 +00:00
vm_pager.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
vm_pager.h Add a deferred free mechanism for freeing swap space that does not require 2019-12-15 03:15:06 +00:00
vm_param.h (fault 3/9) Move map relookup into a dedicated function. 2020-01-23 05:07:01 +00:00
vm_phys.c Fix typo using RB_INITIALIZER. 2019-12-22 21:53:05 +00:00
vm_phys.h Encapsulate phys_avail manipulation in a set of simple routines. Add a 2019-08-18 07:06:31 +00:00
vm_radix.c vm: stop passing M_ZERO when allocating radix nodes 2018-06-24 13:08:05 +00:00
vm_radix.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
vm_reserv.c vm: add missing CLTFLAG_MPSAFE annotations 2020-01-12 05:08:57 +00:00
vm_reserv.h Fix locking in vm_reserv_reclaim_contig(). 2019-11-22 16:28:52 +00:00
vm_swapout_dummy.c SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vm_swapout.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
vm_unix.c Provide separate accounting for user-wired pages. 2019-05-13 16:38:48 +00:00
vm.h Move vm_ndomains to vm.h where it can be used with a single header include 2018-03-27 03:27:02 +00:00
vnode_pager.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
vnode_pager.h vm pager: writemapping accounting for OBJT_SWAP 2019-09-03 20:31:48 +00:00