d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
d71896a76a
freebsd-dev/sys/kern
History
Konstantin Belousov 2a339d9e3d Add implementation of robust mutexes, hopefully close enough to the 
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.

A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held.  The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.

The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths.  Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.

The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive).  Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.

Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot.  When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.

The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.

Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
   pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
   the lifetime of the shared mutex associated with a vnode' page.

Reviewed by:	jilles (previous version, supposedly the objection was fixed)
Discussed with:	brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
..
bus_if.m Add a new bus method to fetch device-specific CPU sets. 2016-05-09 20:50:21 +00:00
capabilities.conf Add futimens and utimensat system calls. 2015-01-23 21:07:08 +00:00
clock_if.m
cpufreq_if.m
device_if.m Change the default method for device_quiesce() to return 0 instead of 2015-01-08 21:46:28 +00:00
genassym.sh genassym.sh: call nm(1) with NMFLAGS. 2015-08-14 22:57:13 +00:00
imgact_aout.c Implement vsyscall hack. Prior to 2.13 glibc uses vsyscall 2016-01-09 20:18:53 +00:00
imgact_binmisc.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c sys: use our roundup2/rounddown2() macros when param.h is available. 2016-04-21 19:57:40 +00:00
imgact_gzip.c Implement lockless resource limits. 2015-06-10 10:48:12 +00:00
imgact_shell.c
inflate.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
init_main.c sys: use our roundup2/rounddown2() macros when param.h is available. 2016-04-21 19:57:40 +00:00
init_sysent.c Regen. 2016-03-21 21:38:35 +00:00
kern_acct.c
kern_alq.c Use SI_SUB_LAST instead of SI_SUB_SMP as the "catch-all" subsystem. 2016-03-11 23:18:06 +00:00
kern_clock.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_clocksource.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_condvar.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_conf.c Provide yet another KPI for cdev creation, make_dev_s(9). 2016-01-07 20:08:02 +00:00
kern_cons.c CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten 2015-05-22 17:05:21 +00:00
kern_context.c
kern_cpu.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_cpuset.c Add more fine-grained kernel options for NUMA support. 2016-04-09 13:58:04 +00:00
kern_ctf.c Don't specify a resid parameter if we're just going to ignore it. Instead, 2015-02-20 20:49:00 +00:00
kern_descrip.c fd: assert dropped filedesc lock in fdcloseexec 2016-05-08 03:26:12 +00:00
kern_dtrace.c
kern_dump.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_environment.c Create wrappers for uint64_t and int64_t for the tunables. While not 2016-04-15 03:09:55 +00:00
kern_et.c Trivial change / forced-commit to document prior change that slipped in 2015-03-16 19:29:19 +00:00
kern_event.c Provide high precision conversion from ns,us,ms -> sbintime in kevent 2016-03-12 23:02:53 +00:00
kern_exec.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_exit.c Add implementation of robust mutexes, hopefully close enough to the 2016-05-17 09:56:22 +00:00
kern_fail.c fail(9): Only gather/print stacks if STACK is enabled 2016-03-17 01:05:53 +00:00
kern_ffclock.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_fork.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_gzio.c Move zlib.c from net to libkern. 2015-04-22 14:38:58 +00:00
kern_hhook.c Check that hhk_helper pointer isn't NULL before access. 2015-11-25 07:14:58 +00:00
kern_idle.c
kern_intr.c The part of r285680 which removed release semantic for two stores to 2015-07-21 14:39:34 +00:00
kern_jail.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_khelp.c
kern_kthread.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
kern_ktr.c Fix the logic in the ddb command 'show ktr /a'. Prior to r118269 it would 2016-01-31 17:32:20 +00:00
kern_ktrace.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_linker.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_lock.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_lockf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_lockstat.c Consistently use a reader/writer flag for lockstat probes in rwlock(9) and 2015-07-19 22:24:33 +00:00
kern_loginclass.c Speed up rctl operation with large rulesets, by holding the lock 2015-11-15 12:10:51 +00:00
kern_malloc.c Include sys/_task.h into uma_int.h, so that taskqueue.h isn't a 2016-02-09 20:22:35 +00:00
kern_mbuf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_mib.c Fix build after r295934. 2016-02-23 23:37:10 +00:00
kern_module.c Provide better debug message on kernel module name clash. 2015-10-10 09:21:55 +00:00
kern_mtxpool.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_mutex.c Don't modify curthread->td_locks unless INVARIANTS is enabled. 2015-08-02 00:03:08 +00:00
kern_ntptime.c Use the monotonic (uptime) counter rather than time-of-day to measure elapsed 2015-07-12 18:38:17 +00:00
kern_numa.c Add an initial NUMA affinity/policy configuration for threads and processes. 2015-07-11 15:21:37 +00:00
kern_osd.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
kern_physio.c Add four new RCTL resources - readbps, readiops, writebps and writeiops, 2016-04-07 04:23:25 +00:00
kern_pmc.c
kern_poll.c When a kernel has DEVICE_POLLING turned on but no drivers have 2015-04-14 14:22:34 +00:00
kern_priv.c
kern_proc.c Use a loop instead of a goto in sysctl_kern_proc_kstack(). 2016-04-17 23:22:32 +00:00
kern_procctl.c If process becomes reaper (procctl(PROC_REAP_ACQUIRE)) while already 2015-08-20 22:44:26 +00:00
kern_prot.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_racct.c Get rid of rctl_lock; use racct_lock where appropriate. The fast paths 2016-04-21 16:22:52 +00:00
kern_rangelock.c
kern_rctl.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_resource.c Implement process-shared locks support for libthr.so.3, without 2016-02-28 17:52:33 +00:00
kern_rmlock.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_rwlock.c Don't modify curthread->td_locks unless INVARIANTS is enabled. 2015-08-02 00:03:08 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c The sendfile(2) allows to send extra data from userspace before the file 2016-03-29 19:57:11 +00:00
kern_sharedpage.c Split kerne timekeep ABI structure vdso_sv_tk out of the struct 2015-11-23 07:09:35 +00:00
kern_shutdown.c Add 4Kn kernel dump support 2016-04-15 17:45:12 +00:00
kern_sig.c Add ptrace(2) reporting for LWP events. 2015-12-29 23:25:26 +00:00
kern_switch.c Revert for r277213: 2015-01-22 11:12:42 +00:00
kern_sx.c Don't modify curthread->td_locks unless INVARIANTS is enabled. 2015-08-02 00:03:08 +00:00
kern_synch.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
kern_syscalls.c Implement lockless resource limits. 2015-06-10 10:48:12 +00:00
kern_sysctl.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_tc.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_thr.c Add implementation of robust mutexes, hopefully close enough to the 2016-05-17 09:56:22 +00:00
kern_thread.c Add implementation of robust mutexes, hopefully close enough to the 2016-05-17 09:56:22 +00:00
kern_time.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
kern_timeout.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_umtx.c Add implementation of robust mutexes, hopefully close enough to the 2016-05-17 09:56:22 +00:00
kern_uuid.c
kern_xxx.c
ksched.c Use P1B_PRIO_MAX to designate max posix priority for the RR/FIFO 2015-08-30 18:02:57 +00:00
link_elf_obj.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
link_elf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
linker_if.m sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
Make.tags.inc Bring the tags and links entries for amd64 up to date. 2015-10-27 22:59:24 +00:00
Makefile
makesyscalls.sh Do not include system call wrappers in libc for old FreeBSD system calls. 2016-03-12 22:53:46 +00:00
md4c.c
md5c.c
msi_if.m Introduce MSI and MSI-X support to intrng. This adds a new msi device 2016-05-16 09:11:40 +00:00
p1003_1b.c In preparation for switching linuxulator to the use the native 1:1 2015-05-24 14:44:06 +00:00
pic_if.m Remove FDT specific parts from INTRNG. Change its interface to make it 2016-04-04 09:15:25 +00:00
posix4_mib.c posix4_mib: Don't overrun facility_initialized array 2016-04-27 00:10:32 +00:00
sched_4bsd.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sched_ule.c The struct thread td_estcpu member is only used by the 4BSD scheduler. 2016-04-17 11:04:27 +00:00
serdev_if.m
stack_protector.c Use nitems() macro instead of __arraycount() 2015-06-16 20:19:00 +00:00
subr_acl_nfs4.c Expose an interface to determine if an ACE is inherited. 2015-09-04 00:14:20 +00:00
subr_acl_posix1e.c
subr_autoconf.c
subr_blist.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_bufring.c
subr_bus_dma.c Fix a bug introduced in r291716: 2016-01-11 20:38:39 +00:00
subr_bus.c Add a new bus method to fetch device-specific CPU sets. 2016-05-09 20:50:21 +00:00
subr_busdma_bufalloc.c Fix printf format to allow for bus_size_t not being u_long on all platforms. 2015-10-20 03:25:17 +00:00
subr_capability.c capsicum: plug spurious memset in __cap_rights_init 2015-12-01 02:48:42 +00:00
subr_clock.c
subr_counter.c Free the temporary buffer in sysctl_handle_counter_u64_array(). 2016-03-15 00:21:32 +00:00
subr_devmap.c Add support for RISC-V. 2016-04-26 12:29:47 +00:00
subr_devstat.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_disk.c
subr_dummy_vdso_tc.c Update the vdso timehands only via tc_windup(). 2015-01-20 03:54:30 +00:00
subr_eventhandler.c
subr_fattime.c Where appropriate, use the modern terms for the one true time base 2014-12-21 05:07:11 +00:00
subr_firmware.c Create a dedicated function for ensuring that cdir and rdir are populated. 2015-07-11 16:22:48 +00:00
subr_hash.c kern: Factor out function to convert hash flags to malloc(9) flags 2016-05-04 03:07:52 +00:00
subr_hints.c Add a new device control utility for new-bus devices called devctl. This 2015-02-06 16:09:01 +00:00
subr_intr.c Introduce MSI and MSI-X support to intrng. This adds a new msi device 2016-05-16 09:11:40 +00:00
subr_kdb.c
subr_kobj.c
subr_lock.c
subr_log.c
subr_mbpool.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_mchain.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_module.c preload_search_info: make sure mod is set 2015-08-21 15:57:57 +00:00
subr_msgbuf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_param.c Ensure that maxproc does not exceed pid_max, at the time of boot. 2015-09-21 15:02:59 +00:00
subr_pcpu.c
subr_pctrie.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
subr_power.c
subr_prf.c Add vlog(9). 2015-11-19 05:50:22 +00:00
subr_prof.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_rman.c s/struct device */device_t/g 2016-05-04 23:31:52 +00:00
subr_rtc.c rtc: fix inverted resolution check 2016-05-04 13:48:59 +00:00
subr_sbuf.c Fail the sbuf if vsnprintf(3) fails. 2015-10-02 09:23:14 +00:00
subr_scanf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_sfbuf.c
subr_sglist.c Fix a couple of panics when detaching from a cxgbe/cxl interface that was 2015-01-26 16:26:28 +00:00
subr_sleepqueue.c Rework handling of thread sleeps before timers are working. 2016-03-31 18:10:29 +00:00
subr_smp.c topo_set_pu_id: turn a check into an assertion 2016-04-08 11:59:11 +00:00
subr_stack.c Add support for a configurable output channel to witness(4). 2015-11-19 05:56:59 +00:00
subr_syscall.c Support an arbitrary number of arguments to DTrace syscall probes. 2015-12-17 00:00:27 +00:00
subr_taskqueue.c Remove taskqueue_enqueue_fast(). 2016-03-01 17:47:32 +00:00
subr_terminal.c
subr_trap.c Add four new RCTL resources - readbps, readiops, writebps and writeiops, 2016-04-07 04:23:25 +00:00
subr_turnstile.c Finish r173600. There is no need to test a condition if both cases 2016-02-10 21:16:37 +00:00
subr_uio.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_unit.c Add explicit cast to fix mips and powerpc build after r299090 2016-05-05 15:21:33 +00:00
subr_vmem.c subr_vmem: Fix double-free in error case of vmem_create 2016-05-11 23:16:11 +00:00
subr_witness.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sys_capability.c Introduce kern_cap_rights_limit(). 2015-08-11 08:43:50 +00:00
sys_generic.c Cap IOSIZE_MAX to INT_MAX for 32-bit processes. 2016-04-01 18:29:38 +00:00
sys_pipe.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
sys_procdesc.c The si_status field of the siginfo_t, provided by the waitid(2) and 2015-07-18 09:02:50 +00:00
sys_process.c Add ptrace(2) reporting for LWP events. 2015-12-29 23:25:26 +00:00
sys_socket.c Expose soaio_enqueue(). 2016-04-29 20:12:45 +00:00
syscalls.c Regen. 2016-03-21 21:38:35 +00:00
syscalls.master Fully handle size_t lengths in AIO requests. 2016-03-21 21:37:33 +00:00
systrace_args.c Regen. 2016-03-21 21:38:35 +00:00
sysv_ipc.c
sysv_msg.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
sysv_sem.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sysv_shm.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
tty_compat.c
tty_info.c
tty_inq.c
tty_outq.c
tty_pts.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
tty_tty.c tty: replace several curthread->td_proc with stored curproc 2015-07-06 18:53:56 +00:00
tty_ttydisc.c Don't clear the software flow control flag before draining for last 2016-01-26 14:46:39 +00:00
tty.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_accf.c
uipc_debug.c Refactor the AIO subsystem to permit file-type-specific handling and 2016-03-01 18:12:14 +00:00
uipc_domain.c Introduce a new protocol hook pru_aio_queue. 2016-04-29 20:11:09 +00:00
uipc_mbuf2.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_mbuf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_mbufhash.c Reduce header pollution. 2015-03-17 14:16:50 +00:00
uipc_mqueue.c Use the new PR_METHOD_REMOVE to clean up jail handling in POSIX 2016-04-25 04:36:54 +00:00
uipc_sem.c Clean up some style(9) violations. 2016-04-14 17:07:26 +00:00
uipc_shm.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
uipc_sockbuf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_socket.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_syscalls.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_usrreq.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
vfs_acl.c Replace struct filedesc argument in getvnode with struct thread 2015-06-16 13:09:18 +00:00
vfs_aio.c Tidy up the unmapped I/O code in qphysio. 2016-03-31 17:27:30 +00:00
vfs_bio.c sys/kern: spelling fixes. 2016-04-29 21:54:28 +00:00
vfs_cache.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_cluster.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_default.c Simplify AIO initialization now that it is standard. 2016-03-09 19:05:11 +00:00
vfs_export.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
vfs_extattr.c Replace struct filedesc argument in getvnode with struct thread 2015-06-16 13:09:18 +00:00
vfs_hash.c Add vfs_hash_ref(9) function, which finds a vnode by the hash value 2016-05-11 06:32:22 +00:00
vfs_init.c sysctl: switch sysctllock to a sleepable rmlock, take 2 2015-09-15 23:06:56 +00:00
vfs_lookup.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_mount.c dounmount: do not call mountcheckdirs() for mounts with MNT_IGNORE 2016-05-16 07:23:24 +00:00
vfs_mountroot.c Stop hiding errors that result in failure to mount /dev. Otherwise, 2016-05-12 07:38:10 +00:00
vfs_subr.c vfs_read_dirent: increment ncookies after adding a cookie 2016-05-16 07:31:11 +00:00
vfs_syscalls.c Simplify AIO initialization now that it is standard. 2016-03-09 19:05:11 +00:00
vfs_vnops.c sys/kern: spelling fixes. 2016-04-29 21:54:28 +00:00
vnode_if.src Add EVFILT_VNODE open, read and close notifications. 2016-05-03 15:17:43 +00:00