freebsd-skq/sys/vm
Jeff Roberson 98087a066f Make collapse synchronization more explicit and allow it to complete during
paging.

Shadow objects are marked with a COLLAPSING flag while they are collapsing with
their backing object.  This gives us an explicit test rather than overloading
paging-in-progress.  While split is on-going we mark an object with SPLIT.
These two operations will modify the swap tree so they must be serialized
and swap_pager_getpages() can now directly detect these conditions and page
more conservatively.

Callers to vm_object_collapse() now will reliably wait for a collapse to finish
so that the backing chain is as short as possible before other decisions are
made that may inflate the object chain.  For example, split, coalesce, etc.
It is now safe to run fault concurrently with collapse.  It is safe to increase
or decrease paging in progress with no lock so long as there is another valid
ref on increase.

This change makes collapse more reliable as a secondary benefit.  The primary
benefit is making it safe to drop the object lock much earlier in fault or
never acquire it at all.

This was tested with a new shadow chain test script that uncovered long
standing bugs and will be integrated with stress2.

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D22908
2020-01-19 18:30:23 +00:00
..
_vm_radix.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
default_pager.c SPDX: wrong license. 2017-11-30 15:45:42 +00:00
device_pager.c Make page busy state deterministic on free. Pages must be xbusy when 2019-12-22 06:56:44 +00:00
memguard.c vm: add missing CLTFLAG_MPSAFE annotations 2020-01-12 05:08:57 +00:00
memguard.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
phys_pager.c (4/6) Protect page valid with the busy lock. 2019-10-15 03:45:41 +00:00
pmap.h Remove page locking from pmap_mincore(). 2019-10-16 22:03:27 +00:00
redzone.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
redzone.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
sg_pager.c Make page busy state deterministic on free. Pages must be xbusy when 2019-12-22 06:56:44 +00:00
swap_pager.c Make collapse synchronization more explicit and allow it to complete during 2020-01-19 18:30:23 +00:00
swap_pager.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
uma_core.c pcpu_page_alloc: guard against empty NUMA domains 2020-01-18 18:25:37 +00:00
uma_dbg.c Fix compilation failures on different arches that have vm_machdep.c not 2019-01-15 19:33:47 +00:00
uma_dbg.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
uma_int.h uma: split slabzone into two sizes 2020-01-14 02:14:15 +00:00
uma.h uma: split slabzone into two sizes 2020-01-14 02:14:15 +00:00
vm_domainset.c Add a !NUMA definition for vm_domainset_iter_policy_ref_init(). 2018-10-24 17:09:20 +00:00
vm_domainset.h Refactor domainset iterators for use by malloc(9) and UMA. 2018-10-23 16:35:58 +00:00
vm_extern.h Improve MD page fault handlers. 2019-09-27 18:43:36 +00:00
vm_fault.c Fix a long standing bug that was made worse in r355765. When we are cowing a 2020-01-17 03:44:04 +00:00
vm_glue.c vm: add missing CLTFLAG_MPSAFE annotations 2020-01-12 05:08:57 +00:00
vm_init.c Simplify VM and UMA startup by eliminating boot pages. Instead use careful 2020-01-16 05:01:21 +00:00
vm_kern.c Simplify VM and UMA startup by eliminating boot pages. Instead use careful 2020-01-16 05:01:21 +00:00
vm_kern.h Do not reserve KVA for paging bufs in vm_ksubmap_init(), since now 2019-01-16 20:14:16 +00:00
vm_map.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
vm_map.h Revert r355505. The code that it allowed to compile has been removed. 2019-12-09 05:09:46 +00:00
vm_meter.c Simplify anonymous memory handling with an OBJ_ANON flag. This eliminates 2019-11-19 23:19:43 +00:00
vm_mmap.c kern_mmap: restore character deleted in transit 2020-01-04 23:51:44 +00:00
vm_object.c Make collapse synchronization more explicit and allow it to complete during 2020-01-19 18:30:23 +00:00
vm_object.h Make collapse synchronization more explicit and allow it to complete during 2020-01-19 18:30:23 +00:00
vm_page.c Simplify VM and UMA startup by eliminating boot pages. Instead use careful 2020-01-16 05:01:21 +00:00
vm_page.h Remove some unused functions. 2019-12-28 19:04:29 +00:00
vm_pageout.c Restore loop break in vm_pageout_lowmem(). 2020-01-14 03:27:57 +00:00
vm_pageout.h Fix OOM handling of some corner cases. 2019-08-16 09:43:49 +00:00
vm_pagequeue.h Simplify vm_pageout_init_domain() and add a "big picture" comment. 2019-11-22 16:31:43 +00:00
vm_pager.c Add couple more assertions to vm_pager_assert_in(). The bogus page is 2019-10-25 16:59:54 +00:00
vm_pager.h Add a deferred free mechanism for freeing swap space that does not require 2019-12-15 03:15:06 +00:00
vm_param.h Improve MD page fault handlers. 2019-09-27 18:43:36 +00:00
vm_phys.c Fix typo using RB_INITIALIZER. 2019-12-22 21:53:05 +00:00
vm_phys.h Encapsulate phys_avail manipulation in a set of simple routines. Add a 2019-08-18 07:06:31 +00:00
vm_radix.c vm: stop passing M_ZERO when allocating radix nodes 2018-06-24 13:08:05 +00:00
vm_radix.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
vm_reserv.c vm: add missing CLTFLAG_MPSAFE annotations 2020-01-12 05:08:57 +00:00
vm_reserv.h Fix locking in vm_reserv_reclaim_contig(). 2019-11-22 16:28:52 +00:00
vm_swapout_dummy.c SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vm_swapout.c Remove page locking for queue operations. 2019-12-28 19:04:00 +00:00
vm_unix.c Provide separate accounting for user-wired pages. 2019-05-13 16:38:48 +00:00
vm.h Move vm_ndomains to vm.h where it can be used with a single header include 2018-03-27 03:27:02 +00:00
vnode_pager.c vm: add missing CLTFLAG_MPSAFE annotations 2020-01-12 05:08:57 +00:00
vnode_pager.h vm pager: writemapping accounting for OBJT_SWAP 2019-09-03 20:31:48 +00:00