d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
315048f2ad
freebsd-dev/sys/kern
History
Mateusz Guzik 1ada904147 Implement trivial backoff for locking primitives. 
All current spinning loops retry an atomic op the first chance they get,
which leads to performance degradation under load.

One classic solution to the problem consists of delaying the test to an
extent. This implementation has a trivial linear increment and a random
factor for each attempt.

For simplicity, this first thouch implementation only modifies spinning
loops where the lock owner is running. spin mutexes and thread lock were
not modified.

Current parameters are autotuned on boot based on mp_cpus.

Autotune factors are very conservative and are subject to change later.

Reviewed by:	kib, jhb
Tested by:	pho
MFC after:	1 week
2016-08-01 21:48:37 +00:00
..
bus_if.m (1) Add a new bus method to get a mapping data for an interrupt. 2016-06-05 16:07:57 +00:00
capabilities.conf
clock_if.m
cpufreq_if.m
device_if.m Import the 'iflib' API library for network drivers. From the author: 2016-05-18 04:35:58 +00:00
genassym.sh genassym.sh: call nm(1) with NMFLAGS. 2015-08-14 22:57:13 +00:00
imgact_aout.c Implement vsyscall hack. Prior to 2.13 glibc uses vsyscall 2016-01-09 20:18:53 +00:00
imgact_binmisc.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c imgact_elf: Rename the segment iterator to match reality 2016-07-20 22:51:33 +00:00
imgact_gzip.c Implement lockless resource limits. 2015-06-10 10:48:12 +00:00
imgact_shell.c
inflate.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
init_main.c Remove assumptions in MI code that the BSP is CPU 0. 2016-07-11 21:25:28 +00:00
init_sysent.c Regenerate system call table for r303435. 2016-07-28 12:22:34 +00:00
kern_acct.c Hide the boottime and bootimebin globals, provide the getboottime(9) 2016-07-27 11:08:59 +00:00
kern_alq.c Use SI_SUB_LAST instead of SI_SUB_SMP as the "catch-all" subsystem. 2016-03-11 23:18:06 +00:00
kern_clock.c Prevent parallel tc_windup() calls, both parallel top-level calls from 2016-07-27 11:49:41 +00:00
kern_clocksource.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_condvar.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_conf.c Provide yet another KPI for cdev creation, make_dev_s(9). 2016-01-07 20:08:02 +00:00
kern_cons.c CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten 2015-05-22 17:05:21 +00:00
kern_context.c
kern_cpu.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_cpuset.c Add more fine-grained kernel options for NUMA support. 2016-04-09 13:58:04 +00:00
kern_ctf.c
kern_descrip.c Audit file-descriptor arguments to I/O system calls such as 2016-07-10 08:04:02 +00:00
kern_dtrace.c
kern_dump.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_environment.c Create wrappers for uint64_t and int64_t for the tunables. While not 2016-04-15 03:09:55 +00:00
kern_et.c
kern_event.c Another issue reported on http://seclists.org/oss-sec/2016/q3/68 is 2016-07-16 13:24:58 +00:00
kern_exec.c The assertion re-added in r302614 was triggered when stopping signal 2016-07-18 10:53:47 +00:00
kern_exit.c When a debugger attaches to the process, SIGSTOP is sent to the 2016-07-28 08:41:13 +00:00
kern_fail.c Fix some cosmetic issues in kern_fail.c omitted from r296927. 2016-06-09 13:17:08 +00:00
kern_ffclock.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_fork.c When a debugger attaches to the process, SIGSTOP is sent to the 2016-07-28 08:41:13 +00:00
kern_gzio.c
kern_hhook.c Get closer to a VIMAGE network stack teardown from top to bottom rather 2016-06-21 13:48:49 +00:00
kern_idle.c
kern_intr.c The part of r285680 which removed release semantic for two stores to 2015-07-21 14:39:34 +00:00
kern_jail.c Fix a vnode leak when giving a child jail a too-long path when 2016-06-09 21:59:11 +00:00
kern_khelp.c
kern_kthread.c Update comments for the MD functions managing contexts for new 2016-06-16 12:05:44 +00:00
kern_ktr.c Fix the logic in the ddb command 'show ktr /a'. Prior to r118269 it would 2016-01-31 17:32:20 +00:00
kern_ktrace.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_linker.c Add DDB command "kldstat" 2016-06-09 18:27:41 +00:00
kern_lock.c Microoptimize locking primitives by avoiding unnecessary atomic ops. 2016-06-01 18:32:20 +00:00
kern_lockf.c When sleeping waiting for either local or remote advisory lock, 2016-06-26 20:08:42 +00:00
kern_lockstat.c Consistently use a reader/writer flag for lockstat probes in rwlock(9) and 2015-07-19 22:24:33 +00:00
kern_loginclass.c Speed up rctl operation with large rulesets, by holding the lock 2015-11-15 12:10:51 +00:00
kern_malloc.c Move IPv6 malloc tag definitions into the IPv6 code. 2016-05-20 04:45:08 +00:00
kern_mbuf.c Import the 'iflib' API library for network drivers. From the author: 2016-05-18 04:35:58 +00:00
kern_mib.c Fix build after r295934. 2016-02-23 23:37:10 +00:00
kern_module.c Provide better debug message on kernel module name clash. 2015-10-10 09:21:55 +00:00
kern_mtxpool.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_mutex.c Implement trivial backoff for locking primitives. 2016-08-01 21:48:37 +00:00
kern_ntptime.c Fix a bug in r302252. 2016-07-27 11:40:06 +00:00
kern_numa.c Add an initial NUMA affinity/policy configuration for threads and processes. 2015-07-11 15:21:37 +00:00
kern_osd.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
kern_physio.c Add four new RCTL resources - readbps, readiops, writebps and writeiops, 2016-04-07 04:23:25 +00:00
kern_pmc.c
kern_poll.c
kern_priv.c
kern_proc.c ANSIfy kern_proc.c and delete register keyword 2016-07-27 14:27:08 +00:00
kern_procctl.c If process becomes reaper (procctl(PROC_REAP_ACQUIRE)) while already 2015-08-20 22:44:26 +00:00
kern_prot.c Prepare for network stack as a module 2016-07-27 20:34:09 +00:00
kern_racct.c Get rid of rctl_lock; use racct_lock where appropriate. The fast paths 2016-04-21 16:22:52 +00:00
kern_rangelock.c
kern_rctl.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_resource.c Implement process-shared locks support for libthr.so.3, without 2016-02-28 17:52:33 +00:00
kern_rmlock.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_rwlock.c Implement trivial backoff for locking primitives. 2016-08-01 21:48:37 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c The sendfile(2) allows to send extra data from userspace before the file 2016-03-29 19:57:11 +00:00
kern_sharedpage.c Split kerne timekeep ABI structure vdso_sv_tk out of the struct 2015-11-23 07:09:35 +00:00
kern_shutdown.c Compile in the kassert_panic() function with INVARIANT_SUPPORT 2016-07-05 18:34:34 +00:00
kern_sig.c When a debugger attaches to the process, SIGSTOP is sent to the 2016-07-28 08:41:13 +00:00
kern_switch.c
kern_sx.c Implement trivial backoff for locking primitives. 2016-08-01 21:48:37 +00:00
kern_synch.c Get rid of struct proc p_sched and struct thread td_sched pointers. 2016-06-05 17:04:03 +00:00
kern_syscalls.c Implement lockless resource limits. 2015-06-10 10:48:12 +00:00
kern_sysctl.c Add support for boolean sysctl's. 2016-05-26 08:41:55 +00:00
kern_tc.c Cache getbintime(9) answer in timehands, similarly to getnanotime(9) 2016-07-30 09:25:57 +00:00
kern_thr.c Add a mask of optional ptrace() events. 2016-07-15 15:32:09 +00:00
kern_thread.c Rewrite subr_sleepqueue.c use of callouts to not depend on the 2016-07-28 09:09:55 +00:00
kern_time.c Remove Giant from settime(), tc_setclock_mtx guards tc_windup() calls, 2016-07-27 11:54:24 +00:00
kern_timeout.c Extract the calculation of the callout fire time into the new function 2016-07-28 08:57:01 +00:00
kern_umtx.c Add implementation of robust mutexes, hopefully close enough to the 2016-05-17 09:56:22 +00:00
kern_uuid.c
kern_xxx.c
ksched.c Use P1B_PRIO_MAX to designate max posix priority for the RR/FIFO 2015-08-30 18:02:57 +00:00
link_elf_obj.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
link_elf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
linker_if.m sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
Make.tags.inc Bring the tags and links entries for amd64 up to date. 2015-10-27 22:59:24 +00:00
Makefile Don't create pointless backups of generated files in "make sysent". 2016-07-28 21:29:04 +00:00
makesyscalls.sh Add support for COMPAT10 keywords in syscalls.master. 2016-06-22 21:12:53 +00:00
md4c.c crypto routines: Hint minimum buffer sizes to the compiler 2016-05-26 19:29:29 +00:00
md5c.c crypto routines: Hint minimum buffer sizes to the compiler 2016-05-26 19:29:29 +00:00
msi_if.m Introduce MSI and MSI-X support to intrng. This adds a new msi device 2016-05-16 09:11:40 +00:00
p1003_1b.c In preparation for switching linuxulator to the use the native 1:1 2015-05-24 14:44:06 +00:00
pic_if.m Remove FDT specific parts from INTRNG. Change its interface to make it 2016-04-04 09:15:25 +00:00
posix4_mib.c posix4_mib: Don't overrun facility_initialized array 2016-04-27 00:10:32 +00:00
sched_4bsd.c Don't treat NOCPU as a valid CPU to CPU_ISSET. 2016-07-29 20:19:14 +00:00
sched_ule.c Get rid of struct proc p_sched and struct thread td_sched pointers. 2016-06-05 17:04:03 +00:00
serdev_if.m
stack_protector.c Use nitems() macro instead of __arraycount() 2015-06-16 20:19:00 +00:00
subr_acl_nfs4.c Expose an interface to determine if an ACE is inherited. 2015-09-04 00:14:20 +00:00
subr_acl_posix1e.c
subr_autoconf.c
subr_blist.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_bufring.c
subr_bus_dma.c Fix a bug introduced in r291716: 2016-01-11 20:38:39 +00:00
subr_bus.c (1) Add a new bus method to get a mapping data for an interrupt. 2016-06-05 16:07:57 +00:00
subr_busdma_bufalloc.c Fix printf format to allow for bus_size_t not being u_long on all platforms. 2015-10-20 03:25:17 +00:00
subr_capability.c capsicum: plug spurious memset in __cap_rights_init 2015-12-01 02:48:42 +00:00
subr_clock.c Do not use Giant to prevent parallel calls to CLOCK_SETTIME(). Use 2016-06-28 16:42:40 +00:00
subr_counter.c Free the temporary buffer in sysctl_handle_counter_u64_array(). 2016-03-15 00:21:32 +00:00
subr_devmap.c Include machine/acle-compat.h in cdefs.h on arm if the compiler doesn't 2016-05-25 19:44:26 +00:00
subr_devstat.c Add support for managing Shingled Magnetic Recording (SMR) drives. 2016-05-19 14:08:36 +00:00
subr_disk.c
subr_dummy_vdso_tc.c
subr_eventhandler.c
subr_fattime.c
subr_firmware.c Create a dedicated function for ensuring that cdir and rdir are populated. 2015-07-11 16:22:48 +00:00
subr_hash.c kern: Factor out function to convert hash flags to malloc(9) flags 2016-05-04 03:07:52 +00:00
subr_hints.c
subr_intr.c Remove temporary solution for storing interrupt mapping data as 2016-06-07 09:03:27 +00:00
subr_kdb.c
subr_kobj.c
subr_lock.c Implement trivial backoff for locking primitives. 2016-08-01 21:48:37 +00:00
subr_log.c
subr_mbpool.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_mchain.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_module.c preload_search_info: make sure mod is set 2015-08-21 15:57:57 +00:00
subr_msgbuf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_param.c Add explicit detection of KVM hypervisor 2016-07-13 19:19:18 +00:00
subr_pcpu.c Replace a number of conflations of mp_ncpus and mp_maxid with either 2016-07-06 14:09:49 +00:00
subr_pctrie.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
subr_power.c
subr_prf.c Hide counted_warning(9) under #ifdef _KERNEL braces, to allow building 2016-07-21 17:59:30 +00:00
subr_prof.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_rman.c Add new bus methods for mapping resources. 2016-05-20 17:57:47 +00:00
subr_rtc.c Reduce the resettodr_lock scope to only CLOCK_SETTIME() call. 2016-07-27 11:34:25 +00:00
subr_sbuf.c Fail the sbuf if vsnprintf(3) fails. 2015-10-02 09:23:14 +00:00
subr_scanf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_sfbuf.c
subr_sglist.c Add sglist functions for working with arrays of VM pages. 2016-05-20 23:28:43 +00:00
subr_sleepqueue.c Rewrite subr_sleepqueue.c use of callouts to not depend on the 2016-07-28 09:09:55 +00:00
subr_smp.c topo_set_pu_id: turn a check into an assertion 2016-04-08 11:59:11 +00:00
subr_stack.c Add support for a configurable output channel to witness(4). 2015-11-19 05:56:59 +00:00
subr_syscall.c Add PTRACE_VFORK to trace vfork events. 2016-07-18 14:53:55 +00:00
subr_taskqueue.c Replace a number of conflations of mp_ncpus and mp_maxid with either 2016-07-06 14:09:49 +00:00
subr_terminal.c
subr_trap.c The assertion re-added in r302614 was triggered when stopping signal 2016-07-18 10:53:47 +00:00
subr_turnstile.c Finish r173600. There is no need to test a condition if both cases 2016-02-10 21:16:37 +00:00
subr_uio.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_unit.c Fix build of kern/subr_unit.c, broken by r300539 2016-05-24 00:14:58 +00:00
subr_vmem.c subr_vmem: Fix double-free in error case of vmem_create 2016-05-11 23:16:11 +00:00
subr_witness.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sys_capability.c Introduce kern_cap_rights_limit(). 2015-08-11 08:43:50 +00:00
sys_generic.c Audit file-descriptor arguments to I/O system calls such as 2016-07-10 08:04:02 +00:00
sys_pipe.c Generate syscall tables and update pipe() implementation after r302094. 2016-06-22 21:18:19 +00:00
sys_procdesc.c Hide the boottime and bootimebin globals, provide the getboottime(9) 2016-07-27 11:08:59 +00:00
sys_process.c When a debugger attaches to the process, SIGSTOP is sent to the 2016-07-28 08:41:13 +00:00
sys_socket.c Account for AIO socket operations in thread/process resource usage. 2016-06-21 22:19:06 +00:00
syscalls.c Regenerate system call table for r303435. 2016-07-28 12:22:34 +00:00
syscalls.master Change the return type of msgrcv() to ssize_t as required by POSIX. 2016-07-28 12:22:01 +00:00
systrace_args.c Regenerate system call table for r303435. 2016-07-28 12:22:34 +00:00
sysv_ipc.c
sysv_msg.c Remove a comment that was part of copied code, and is misleading in 2016-06-09 15:34:33 +00:00
sysv_sem.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sysv_shm.c Add shmatt_t. 2016-07-26 17:23:49 +00:00
tty_compat.c
tty_info.c
tty_inq.c
tty_outq.c
tty_pts.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
tty_tty.c tty: replace several curthread->td_proc with stored curproc 2015-07-06 18:53:56 +00:00
tty_ttydisc.c Don't clear the software flow control flag before draining for last 2016-01-26 14:46:39 +00:00
tty.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_accf.c
uipc_debug.c Refactor the AIO subsystem to permit file-type-specific handling and 2016-03-01 18:12:14 +00:00
uipc_domain.c The pr_destroy field does not allow us to run the teardown code in a 2016-06-01 10:14:04 +00:00
uipc_mbuf2.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_mbuf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_mbufhash.c
uipc_mqueue.c Use the new PR_METHOD_REMOVE to clean up jail handling in POSIX 2016-04-25 04:36:54 +00:00
uipc_sem.c Clean up some style(9) violations. 2016-04-14 17:07:26 +00:00
uipc_shm.c posixshm: Fix lock leak when mac_posixshm_check_read rejects read. 2016-06-23 20:59:13 +00:00
uipc_sockbuf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_socket.c Fix typo introduced by me (not the submitter) when fixing typos 2016-05-22 13:10:48 +00:00
uipc_syscalls.c The SA-16:19 wouldn't have happened if the sockargs() had properly typed 2016-05-18 22:05:50 +00:00
uipc_usrreq.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
vfs_acl.c Replace struct filedesc argument in getvnode with struct thread 2015-06-16 13:09:18 +00:00
vfs_aio.c Fix locking issues with aio_fsync(). 2016-07-29 18:26:15 +00:00
vfs_bio.c Let DDB's buf printer handle NULL pointers in the buf page array. 2016-07-14 18:49:05 +00:00
vfs_cache.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_cluster.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_default.c Simplify AIO initialization now that it is standard. 2016-03-09 19:05:11 +00:00
vfs_export.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
vfs_extattr.c Replace struct filedesc argument in getvnode with struct thread 2015-06-16 13:09:18 +00:00
vfs_hash.c Add vfs_hash_ref(9) function, which finds a vnode by the hash value 2016-05-11 06:32:22 +00:00
vfs_init.c sysctl: switch sysctllock to a sleepable rmlock, take 2 2015-09-15 23:06:56 +00:00
vfs_lookup.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_mount.c Add new unmount(2) flag, MNT_NONBUSY, to check whether there are 2016-07-07 09:03:57 +00:00
vfs_mountroot.c Stop hiding errors that result in failure to mount /dev. Otherwise, 2016-05-12 07:38:10 +00:00
vfs_subr.c Fix grammar. 2016-07-11 17:04:22 +00:00
vfs_syscalls.c Do not allow creation of char or block special nodes with VNOVAL dev_t. 2016-07-15 09:23:18 +00:00
vfs_vnops.c devfs: Move most ioctl logic down to vnode layer 2016-07-25 16:28:02 +00:00
vnode_if.src Add EVFILT_VNODE open, read and close notifications. 2016-05-03 15:17:43 +00:00