freebsd-skq/sys/vm
Mark Johnston 991f23ef20 uma: Avoid allocating buckets with the cross-domain lock held
Allocation of a bucket can trigger a cross-domain free in the bucket
zone, e.g., if the per-CPU alloc bucket is empty, we free it and get
migrated to a remote domain.  This can lead to deadlocks since a bucket
zone may allocate buckets from itself or a pair of bucket zones could be
allocating from each other.

Fix the problem by dropping the cross-domain lock before allocating a
new bucket and handling refill races.  Use a list of empty buckets to
ensure that we can make forward progress.

Reported by:	imp, mjg (witness(9) warnings)
Discussed with:	jeff
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D27341
2020-11-30 16:18:33 +00:00
..
_vm_phys.h vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
_vm_radix.h
default_pager.c vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
device_pager.c
memguard.c vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
memguard.h LinuxKPI: Implement ksize() function. 2020-08-29 19:26:31 +00:00
phys_pager.c Allow consumer to customize physical pager. 2020-09-09 00:00:43 +00:00
pmap.h Add pmap_enter(9) PMAP_ENTER_LARGEPAGE flag and implement it on amd64. 2020-09-09 21:50:24 +00:00
redzone.c vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
redzone.h
sg_pager.c vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
swap_pager.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
swap_pager.h
uma_core.c uma: Avoid allocating buckets with the cross-domain lock held 2020-11-30 16:18:33 +00:00
uma_dbg.c
uma_dbg.h
uma_int.h vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
uma.h Add more per-cpu zones. 2020-11-09 00:34:23 +00:00
vm_domainset.c Add interruptible variant of vm_wait(9), vm_wait_intr(9). 2020-09-08 23:28:09 +00:00
vm_domainset.h Add interruptible variant of vm_wait(9), vm_wait_intr(9). 2020-09-08 23:28:09 +00:00
vm_dumpset.h Avoid dump_avail[] redefinition. 2020-10-14 22:51:40 +00:00
vm_extern.h Remove the VM map zone. 2020-08-17 13:02:01 +00:00
vm_fault.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vm_glue.c vmspace: Convert to refcount(9) 2020-11-04 16:30:56 +00:00
vm_init.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vm_kern.c vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
vm_kern.h Remove the VM map zone. 2020-08-17 13:02:01 +00:00
vm_map.c vm_map: Handle kernel map entry allocator recursion 2020-11-11 17:16:39 +00:00
vm_map.h Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vm_meter.c vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
vm_mmap.c Support for userspace non-transparent superpages (largepages). 2020-09-09 22:12:51 +00:00
vm_object.c Allow consumer to customize physical pager. 2020-09-09 00:00:43 +00:00
vm_object.h Allow consumer to customize physical pager. 2020-09-09 00:00:43 +00:00
vm_page.c Wrap a long line in vm_pqbatch_process_page() 2020-11-19 15:41:42 +00:00
vm_page.h vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
vm_pageout.c vm_pageout: Avoid rounding down the inactive scan target 2020-10-02 19:16:06 +00:00
vm_pageout.h Add interruptible variant of vm_wait(9), vm_wait_intr(9). 2020-09-08 23:28:09 +00:00
vm_pagequeue.h vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
vm_pager.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vm_pager.h Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vm_param.h
vm_phys.c vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
vm_phys.h vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
vm_radix.c vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
vm_radix.h
vm_reserv.c vm_phys: Try to clean up NUMA KPIs 2020-11-19 03:59:21 +00:00
vm_reserv.h
vm_swapout_dummy.c
vm_swapout.c Use a single VM object for kernel stacks. 2020-04-26 20:08:57 +00:00
vm_unix.c Prepare to handle non-trivial errors from vm_map_delete(). 2020-09-09 21:34:31 +00:00
vm.h vm: clean up empty lines in .c and .h files 2020-09-01 21:20:45 +00:00
vnode_pager.c Make MAXPHYS tunable. Bump MAXPHYS to 1M. 2020-11-28 12:12:51 +00:00
vnode_pager.h