jeff 4402204d47 Parallelize the buffer cache and rewrite getnewbuf(). This results in a ...

8x performance improvement in a micro benchmark on a 4 socket machine.

 - Get buffer headers from a per-cpu uma cache that sits in from of the
   free queue.
 - Use a per-cpu quantum cache in vmem to eliminate contention for kva.
 - Use multiple clean queues according to buffer cache size to eliminate
   clean queue lock contention.
 - Introduce a bufspace daemon that attempts to prevent getnewbuf() callers
   from blocking or doing direct recycling.
 - Close some bufspace allocation races that could lead to endless
   recycling.
 - Further the transition to a more modern style of small functions grouped
   by prefix in order to improve growing complexity.

Sponsored by:	EMC / Isilon
Reviewed by:	kib
Tested by:	pho

2015-10-14 02:10:07 +00:00

..

bus_if.m

…

capabilities.conf

Add futimens and utimensat system calls.

2015-01-23 21:07:08 +00:00

clock_if.m

…

cpufreq_if.m

…

device_if.m

Change the default method for device_quiesce() to return 0 instead of

2015-01-08 21:46:28 +00:00

genassym.sh

genassym.sh: call nm(1) with NMFLAGS.

2015-08-14 22:57:13 +00:00

imgact_aout.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

imgact_binmisc.c

At the suggestion of jhb, replace atomic_set/clear calls with use of

2015-06-24 15:52:26 +00:00

imgact_elf32.c

…

imgact_elf64.c

…

imgact_elf.c

Fix core corruption caused by race in note_procstat_vmmap

2015-10-06 18:07:00 +00:00

imgact_gzip.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

imgact_shell.c

…

inflate.c

…

init_main.c

Remove now obsolete comment.

2015-08-28 20:06:58 +00:00

init_sysent.c

Add an initial NUMA affinity/policy configuration for threads and processes.

2015-07-11 15:21:37 +00:00

kern_acct.c

…

kern_alq.c

…

kern_clock.c

Initialize ticks so that it wraps 10 minutes after boot to increase the

2015-02-05 01:43:21 +00:00

kern_clocksource.c

Fix typo in comment.

2015-07-20 09:37:42 +00:00

kern_condvar.c

Revert r282971. It depends on condvar consumers not destroying condvars

2015-05-21 16:43:26 +00:00

kern_conf.c

Fix for out of order device destruction notifications when using the

2015-03-22 13:11:56 +00:00

kern_cons.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

kern_context.c

…

kern_cpu.c

…

kern_cpuset.c

Un-static cpuset_which() - it's useful in other contexts, such as some

2015-06-26 04:14:05 +00:00

kern_ctf.c

Don't specify a resid parameter if we're just going to ignore it. Instead,

2015-02-20 20:49:00 +00:00

kern_descrip.c

fd: make rights a mandatory argument to fgetvp_rights

2015-09-07 20:05:56 +00:00

kern_dtrace.c

…

kern_dump.c

Factor out duplicated code from dumpsys() on each architecture into generic

2015-01-07 01:01:39 +00:00

kern_environment.c

…

kern_et.c

Trivial change / forced-commit to document prior change that slipped in

2015-03-16 19:29:19 +00:00

kern_event.c

kqueue: simplify kern_kqueue by not refing/unrefing creds too early

2015-09-23 12:45:08 +00:00

kern_exec.c

Fix core corruption caused by race in note_procstat_vmmap

2015-10-06 18:07:00 +00:00

kern_exit.c

Enforce the maxproc limitation before allocating struct proc, initial

2015-10-08 11:07:09 +00:00

kern_fail.c

Use a regular sbuf + SYSCTL_OUT() rather than sbuf_new_for_sysctl() with

2015-03-16 19:18:45 +00:00

kern_ffclock.c

…

kern_fork.c

Enforce the maxproc limitation before allocating struct proc, initial

2015-10-08 11:07:09 +00:00

kern_gzio.c

Move zlib.c from net to libkern.

2015-04-22 14:38:58 +00:00

kern_hhook.c

…

kern_idle.c

…

kern_intr.c

The part of r285680 which removed release semantic for two stores to

2015-07-21 14:39:34 +00:00

kern_jail.c

Add support to the jail framework to be able to mount linsysfs(5) and

2015-07-19 08:52:35 +00:00

kern_khelp.c

…

kern_kthread.c

Limit rights on process descriptors.

2015-07-31 10:21:58 +00:00

kern_ktr.c

Expand ktr_mask to be a 64-bit unsigned integer.

2015-05-22 11:09:41 +00:00

kern_ktrace.c

…

kern_linker.c

sysctl: switch sysctllock to a sleepable rmlock, take 2

2015-09-15 23:06:56 +00:00

kern_lock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_lockf.c

Improve style and fix a possible use-after-free case introduced in r268384

2015-01-10 06:48:35 +00:00

kern_lockstat.c

Consistently use a reader/writer flag for lockstat probes in rwlock(9) and

2015-07-19 22:24:33 +00:00

kern_loginclass.c

cred: add proc_set_cred helper

2015-03-16 00:10:03 +00:00

kern_malloc.c

The vmem callback to reclaim kmem arena address space on low or

2015-05-09 20:08:36 +00:00

kern_mbuf.c

Fix integer truncation bug in malloc(9)

2015-04-01 12:42:26 +00:00

kern_mib.c

Huge cleanup of random(4) code.

2015-06-30 17:00:45 +00:00

kern_module.c

Provide better debug message on kernel module name clash.

2015-10-10 09:21:55 +00:00

kern_mtxpool.c

…

kern_mutex.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_ntptime.c

Use the monotonic (uptime) counter rather than time-of-day to measure elapsed

2015-07-12 18:38:17 +00:00

kern_numa.c

Add an initial NUMA affinity/policy configuration for threads and processes.

2015-07-11 15:21:37 +00:00

kern_osd.c

…

kern_physio.c

Rewrite physio() to not allocate pbufs for unmapped I/O.

2015-04-21 10:55:53 +00:00

kern_pmc.c

…

kern_poll.c

When a kernel has DEVICE_POLLING turned on but no drivers have

2015-04-14 14:22:34 +00:00

kern_priv.c

…

kern_proc.c

Fix core corruption caused by race in note_procstat_vmmap

2015-10-06 18:07:00 +00:00

kern_procctl.c

If process becomes reaper (procctl(PROC_REAP_ACQUIRE)) while already

2015-08-20 22:44:26 +00:00

kern_prot.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_racct.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

kern_rangelock.c

…

kern_rctl.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

kern_resource.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_rmlock.c

Threads holding a read lock of a sleepable rm lock are not permitted

2015-09-15 22:16:21 +00:00

kern_rwlock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_sdt.c

…

kern_sema.c

…

kern_sharedpage.c

Copy the fencing of the algorithm to do lock-less update and reading

2015-08-04 12:33:51 +00:00

kern_shutdown.c

Disable suspend when we're shutting down. This solves the "tell FreeBSD

2015-10-01 10:52:26 +00:00

kern_sig.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

kern_switch.c

Revert for r277213:

2015-01-22 11:12:42 +00:00

kern_sx.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_synch.c

Remove several write-only variables, all reported by the gcc 4.9

2015-05-29 13:24:17 +00:00

kern_syscalls.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

kern_sysctl.c

sysctl: switch sysctllock to a sleepable rmlock, take 2

2015-09-15 23:06:56 +00:00

kern_tc.c

Use per-cpu values for base and last in tc_cpu_ticks(). The values

2015-09-25 13:03:57 +00:00

kern_thr.c

Add an API for easily creating userspace threads in kernelspace.

2015-07-20 10:20:04 +00:00

kern_thread.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_time.c

Fix an off by one in ppsratecheck(). If you asked for N=1 you'd get one,

2015-01-11 20:48:29 +00:00

kern_timeout.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

kern_umtx.c

Fix bad arithmetic in umtx_key_get() to compute object offset.

2015-08-04 06:01:13 +00:00

kern_uuid.c

…

kern_xxx.c

…

ksched.c

Use P1B_PRIO_MAX to designate max posix priority for the RR/FIFO

2015-08-30 18:02:57 +00:00

link_elf_obj.c

Add support for weak symbols to the kernel linkers. It means that

2015-09-20 01:27:59 +00:00

link_elf.c

Add support for weak symbols to the kernel linkers. It means that

2015-09-20 01:27:59 +00:00

linker_if.m

…

Make.tags.inc

…

Makefile

…

makesyscalls.sh

Import the CloudABI datatypes and create a system call table.

2015-07-09 07:20:15 +00:00

md4c.c

…

md5c.c

…

p1003_1b.c

In preparation for switching linuxulator to the use the native 1:1

2015-05-24 14:44:06 +00:00

posix4_mib.c

…

sched_4bsd.c

kgdb uses td_oncpu to determine if a thread is running and should use

2015-08-03 20:43:36 +00:00

sched_ule.c

Summary: Add the interactivity equations to the header comment for our

2015-08-26 16:36:41 +00:00

serdev_if.m

…

stack_protector.c

Use nitems() macro instead of __arraycount()

2015-06-16 20:19:00 +00:00

subr_acl_nfs4.c

Expose an interface to determine if an ACE is inherited.

2015-09-04 00:14:20 +00:00

subr_acl_posix1e.c

…

subr_autoconf.c

…

subr_blist.c

…

subr_bufring.c

…

subr_bus_dma.c

…

subr_bus.c

Huge cleanup of random(4) code.

2015-06-30 17:00:45 +00:00

subr_busdma_bufalloc.c

Fix integer truncation bug in malloc(9)

2015-04-01 12:42:26 +00:00

subr_capability.c

…

subr_clock.c

…

subr_counter.c

…

subr_devstat.c

…

subr_disk.c

…

subr_dummy_vdso_tc.c

Update the vdso timehands only via tc_windup().

2015-01-20 03:54:30 +00:00

subr_eventhandler.c

…

subr_fattime.c

…

subr_firmware.c

Create a dedicated function for ensuring that cdir and rdir are populated.

2015-07-11 16:22:48 +00:00

subr_hash.c

…

subr_hints.c

Add a new device control utility for new-bus devices called devctl. This

2015-02-06 16:09:01 +00:00

subr_kdb.c

…

subr_kobj.c

…

subr_lock.c

…

subr_log.c

…

subr_mbpool.c

…

subr_mchain.c

…

subr_module.c

preload_search_info: make sure mod is set

2015-08-21 15:57:57 +00:00

subr_msgbuf.c

…

subr_param.c

Ensure that maxproc does not exceed pid_max, at the time of boot.

2015-09-21 15:02:59 +00:00

subr_pcpu.c

…

subr_pctrie.c

…

subr_power.c

…

subr_prf.c

Add support for reading MAM attributes to camcontrol(8) and libcam(3).

2015-06-09 21:39:38 +00:00

subr_prof.c

…

subr_rman.c

…

subr_rtc.c

…

subr_sbuf.c

Fail the sbuf if vsnprintf(3) fails.

2015-10-02 09:23:14 +00:00

subr_scanf.c

…

subr_sfbuf.c

…

subr_sglist.c

Fix a couple of panics when detaching from a cxgbe/cxl interface that was

2015-01-26 16:26:28 +00:00

subr_sleepqueue.c

Revert for r277213:

2015-01-22 11:12:42 +00:00

subr_smp.c

The smp_rendezvous_cpus() function should ensure that all accesses

2015-07-21 22:56:46 +00:00

subr_stack.c

…

subr_syscall.c

Fix various edge cases related to system call tracing.

2015-10-06 19:29:05 +00:00

subr_taskqueue.c

MFuser/delphij/zfs-arc-rebase@r281754:

2015-05-26 01:40:33 +00:00

subr_terminal.c

…

subr_trap.c

racct: perform a lockless check for p_throttled

2015-07-13 22:52:11 +00:00

subr_turnstile.c

ddb: finish converting boolean values.

2015-05-21 15:16:18 +00:00

subr_uio.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

subr_unit.c

…

subr_vmem.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

subr_witness.c

o Revert the other functional half of r239864, i. e. the merge of r134227

2015-07-24 15:13:21 +00:00

sys_capability.c

Introduce kern_cap_rights_limit().

2015-08-11 08:43:50 +00:00

sys_generic.c

Cover a race between doselwakeup() and selfdfree(). If doselwakeup()

2015-07-09 09:22:21 +00:00

sys_pipe.c

Make pipes in CloudABI work.

2015-07-29 17:18:27 +00:00

sys_procdesc.c

The si_status field of the siginfo_t, provided by the waitid(2) and

2015-07-18 09:02:50 +00:00

sys_process.c

Include additional info in ptrace(2) KTR traces:

2015-10-05 21:36:53 +00:00

sys_socket.c

…

syscalls.c

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

syscalls.master

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

systrace_args.c

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

sysv_ipc.c

…

sysv_msg.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

sysv_sem.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

sysv_shm.c

Change the default setting of kern.ipc.shm_allow_removed from 0 to 1.

2015-10-10 09:29:47 +00:00

tty_compat.c

…

tty_info.c

…

tty_inq.c

…

tty_outq.c

…

tty_pts.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

tty_tty.c

tty: replace several curthread->td_proc with stored curproc

2015-07-06 18:53:56 +00:00

tty_ttydisc.c

…

tty.c

filedesc: simplify fget_unlocked & friends

2015-02-17 23:54:06 +00:00

uipc_accf.c

…

uipc_debug.c

Fix sb_state constant names as used e.g. to display in DDB ``show sockbuf''.

2015-07-21 09:57:13 +00:00

uipc_domain.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

uipc_mbuf2.c

Add const-qualifiers for source mbuf argument in m_dup(), m_copym(),

2015-08-08 15:50:46 +00:00

uipc_mbuf.c

There are times when it would be really nice to have a record of the last few

2015-10-14 00:35:37 +00:00

uipc_mbufhash.c

Reduce header pollution.

2015-03-17 14:16:50 +00:00

uipc_mqueue.c

fd: remove filedesc argument from fdclose

2015-04-11 15:40:28 +00:00

uipc_sem.c

Call ksem_get() with initialized 'rights'.

2015-07-23 23:18:03 +00:00

uipc_shm.c

Add kern_shm_open().

2015-08-01 07:21:14 +00:00

uipc_sockbuf.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

uipc_socket.c

Make shutdown() return ENOTCONN as required by POSIX, part deux.

2015-07-27 13:17:57 +00:00

uipc_syscalls.c

Make shutdown() return ENOTCONN as required by POSIX, part deux.

2015-07-27 13:17:57 +00:00

uipc_usrreq.c

fd: make the common case in filecaps_copy work lockless

2015-09-07 20:02:56 +00:00

vfs_acl.c

Replace struct filedesc argument in getvnode with struct thread

2015-06-16 13:09:18 +00:00

vfs_aio.c

Mutex memory is not zeroed, add MTX_NEW.

2015-07-06 14:09:00 +00:00

vfs_bio.c

Parallelize the buffer cache and rewrite getnewbuf(). This results in a

2015-10-14 02:10:07 +00:00

vfs_cache.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

vfs_cluster.c

Refactor unmapped buffer address handling.

2015-07-23 19:13:41 +00:00

vfs_default.c

Ensure that vop_stdadvise() does not call getblk() on vnodes that have an

2015-10-01 16:34:53 +00:00

vfs_export.c

…

vfs_extattr.c

Replace struct filedesc argument in getvnode with struct thread

2015-06-16 13:09:18 +00:00

vfs_hash.c

Track changes to kern.maxvnodes and appropriately increase or decrease

2015-09-06 05:50:51 +00:00

vfs_init.c

sysctl: switch sysctllock to a sleepable rmlock, take 2

2015-09-15 23:06:56 +00:00

vfs_lookup.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

vfs_mount.c

Make vfs_unmountall() unmount /dev after /, not before. The only

2015-08-24 13:18:13 +00:00

vfs_mountroot.c

Remove root_mount_wait(). It's not used anywhere.

2015-10-09 12:11:37 +00:00

vfs_subr.c

Remove a check for a condition that is always false by a preceding KASSERT

2015-09-26 22:26:55 +00:00

vfs_syscalls.c

Revert r288628 and instead fix a discrepancy between the posix_fadvise(2)

2015-10-03 22:27:14 +00:00

vfs_vnops.c

Fix r283998 that broke mapin events for hwpmc.

2015-10-08 09:54:33 +00:00

vnode_if.src

kevent(2): Note DOOMED vnodes with NOTE_REVOKE

2015-09-15 20:22:30 +00:00