freebsd-skq/sys/vm
Mark Johnston c6a70eaea8 Avoid dequeuing the fault page during a soft fault.
Such pages are re-enqueued at the end of the fault handler, preserving
LRU. Rather than performing two separate operations per fault, simply
requeue the page at the end of the fault (or bump its activation count
if it resides in PQ_ACTIVE, avoiding the page queue lock entirely).
This elides some page lock and page queue lock operations in common
cases, e.g., CoW faults.

Note that we must still dequeue the source page for "optimized" CoW
faults since the page may not remain enqueued while it is moved to
another object.

Reviewed by:	alc, kib
Tested by:	pho
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D14625
2018-03-18 16:49:30 +00:00
..
_vm_radix.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
default_pager.c SPDX: wrong license. 2017-11-30 15:45:42 +00:00
device_pager.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
memguard.c Eliminate kmem_arena and kmem_object in preparation for further NUMA commits. 2017-11-28 23:40:54 +00:00
memguard.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
phys_pager.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
pmap.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
redzone.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
redzone.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
sg_pager.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
swap_pager.c Use the conventional name for an array of pages. 2018-02-16 15:38:22 +00:00
swap_pager.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
uma_core.c Fix boot_pages exhaustion on machines with many domains and cores, where 2018-02-09 04:45:39 +00:00
uma_dbg.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
uma_dbg.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
uma_int.h Fix three miscalculations in amount of boot pages: 2018-02-07 18:32:51 +00:00
uma.h Followup on r302393 by cperciva, improving calculation of boot pages required 2018-02-06 04:16:00 +00:00
vm_domainset.c Move VM_NUMA_ALLOC and DEVICE_NUMA under the single global config option NUMA. 2018-01-14 03:36:03 +00:00
vm_domainset.h Add files for r327895 2018-01-12 22:57:57 +00:00
vm_extern.h Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_fault.c Avoid dequeuing the fault page during a soft fault. 2018-03-18 16:49:30 +00:00
vm_glue.c Make v_wire_count a per-cpu counter(9) counter. This eliminates a 2018-02-12 22:53:00 +00:00
vm_init.c Fix boot_pages exhaustion on machines with many domains and cores, where 2018-02-09 04:45:39 +00:00
vm_kern.c Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_kern.h SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vm_map.c Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_map.h Assign map->header values to avoid boundary checks. 2018-01-20 12:19:02 +00:00
vm_meter.c Add a generic Proportional Integral Derivative (PID) controller algorithm and 2018-02-23 22:51:51 +00:00
vm_mmap.c Make v_wire_count a per-cpu counter(9) counter. This eliminates a 2018-02-12 22:53:00 +00:00
vm_object.c Dequeue wired pages lazily. 2018-02-07 16:57:10 +00:00
vm_object.h Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_page.c Have vm_page_{deactivate,launder}() requeue already-queued pages. 2018-03-18 16:40:56 +00:00
vm_page.h Unify bulk free operations in several pmaps. 2018-03-04 20:53:20 +00:00
vm_pageout.c Eliminate pageout wakeup races. Take another step towards lockless 2018-03-15 19:23:07 +00:00
vm_pageout.h Fix GCC build: Remove redundant pagedaemon_wakeup declaration 2018-03-16 07:05:09 +00:00
vm_pagequeue.h Eliminate pageout wakeup races. Take another step towards lockless 2018-03-15 19:23:07 +00:00
vm_pager.c SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vm_pager.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
vm_param.h SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vm_phys.c Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_phys.h Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_radix.c Fix boot_pages calculation for machines that don't have UMA_MD_SMALL_ALLOC. 2018-02-06 22:06:59 +00:00
vm_radix.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
vm_reserv.c Eliminate pageout wakeup races. Take another step towards lockless 2018-03-15 19:23:07 +00:00
vm_reserv.h Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vm_swapout_dummy.c SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vm_swapout.c Dequeue wired pages lazily. 2018-02-07 16:57:10 +00:00
vm_unix.c ANSIfy function definitions in sys/vm/ 2018-01-12 03:50:44 +00:00
vm.h SPDX: Consider code from Carnegie-Mellon University. 2017-11-30 15:48:35 +00:00
vnode_pager.c Use per-domain locks for vm page queue free. Move paging control from 2018-02-06 22:10:07 +00:00
vnode_pager.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00