Kirk McKusick 41d4f10391 As the kernel allocates and frees vnodes, it fully initializes them ...

on every allocation and fully releases them on every free.  These
are not trivial costs: it starts by zeroing a large structure then
initializes a mutex, a lock manager lock, an rw lock, four lists,
and six pointers. And looking at vfs.vnodes_created, these operations
are being done millions of times an hour on a busy machine.

As a performance optimization, this code update uses the uma_init
and uma_fini routines to do these initializations and cleanups only
as the vnodes enter and leave the vnode_zone. With this change the
initializations are only done kern.maxvnodes times at system startup
and then only rarely again. The frees are done only if the vnode_zone
shrinks which never happens in practice. For those curious about the
avoided work, look at the vnode_init() and vnode_fini() functions in
kern/vfs_subr.c to see the code that has been removed from the main
vnode allocation/free path.

Reviewed by: kib
Tested by:   Peter Holm

2015-11-29 21:42:26 +00:00

..

bus_if.m

Add a note to the effect that BUS_ADD_CHILD calls

2015-10-28 18:53:18 +00:00

capabilities.conf

Add futimens and utimensat system calls.

2015-01-23 21:07:08 +00:00

clock_if.m

…

cpufreq_if.m

…

device_if.m

Change the default method for device_quiesce() to return 0 instead of

2015-01-08 21:46:28 +00:00

genassym.sh

genassym.sh: call nm(1) with NMFLAGS.

2015-08-14 22:57:13 +00:00

imgact_aout.c

Remove sv_prepsyscall, sv_sigsize and sv_sigtbl members of the struct

2015-11-28 08:49:07 +00:00

imgact_binmisc.c

At the suggestion of jhb, replace atomic_set/clear calls with use of

2015-06-24 15:52:26 +00:00

imgact_elf32.c

…

imgact_elf64.c

…

imgact_elf.c

Extend r270123 to run the brand info's header_supported() routine for

2015-11-18 17:03:22 +00:00

imgact_gzip.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

imgact_shell.c

Allow multiple image activators to run on the same execution by changing

2014-09-04 21:31:25 +00:00

inflate.c

…

init_main.c

Remove sv_prepsyscall, sv_sigsize and sv_sigtbl members of the struct

2015-11-28 08:49:07 +00:00

init_sysent.c

Missing regen after last change to sys/kern/syscalls.master.

2015-10-22 21:30:39 +00:00

kern_acct.c

…

kern_alq.c

…

kern_clock.c

Initialize ticks so that it wraps 10 minutes after boot to increase the

2015-02-05 01:43:21 +00:00

kern_clocksource.c

Fix typo in comment.

2015-07-20 09:37:42 +00:00

kern_condvar.c

Revert r282971. It depends on condvar consumers not destroying condvars

2015-05-21 16:43:26 +00:00

kern_conf.c

Fix for out of order device destruction notifications when using the

2015-03-22 13:11:56 +00:00

kern_cons.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

kern_context.c

…

kern_cpu.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_cpuset.c

Un-static cpuset_which() - it's useful in other contexts, such as some

2015-06-26 04:14:05 +00:00

kern_ctf.c

Don't specify a resid parameter if we're just going to ignore it. Instead,

2015-02-20 20:49:00 +00:00

kern_descrip.c

fd: implement kern.proc.nfds sysctl

2015-11-07 00:18:14 +00:00

kern_dtrace.c

Commit the rest of the changes that were intended to be part of r266826.

2014-05-29 01:42:22 +00:00

kern_dump.c

- Unbreak dumpsys(9) on sparc64 after r276772

2015-11-16 23:02:33 +00:00

kern_environment.c

Test if 'env' is NULL before doing memset() and strlen(),

2014-10-23 18:23:50 +00:00

kern_et.c

Trivial change / forced-commit to document prior change that slipped in

2015-03-16 19:29:19 +00:00

kern_event.c

kqueue: simplify kern_kqueue by not refing/unrefing creds too early

2015-09-23 12:45:08 +00:00

kern_exec.c

Fix core corruption caused by race in note_procstat_vmmap

2015-10-06 18:07:00 +00:00

kern_exit.c

Enforce the maxproc limitation before allocating struct proc, initial

2015-10-08 11:07:09 +00:00

kern_fail.c

Use a regular sbuf + SYSCTL_OUT() rather than sbuf_new_for_sysctl() with

2015-03-16 19:18:45 +00:00

kern_ffclock.c

The SYSCTL data pointers can come from userspace and must not be

2014-10-28 12:00:39 +00:00

kern_fork.c

Add a way to distinguish between forking and thread creation in schedtail.

2015-10-22 09:33:34 +00:00

kern_gzio.c

Move zlib.c from net to libkern.

2015-04-22 14:38:58 +00:00

kern_hhook.c

Check that hhk_helper pointer isn't NULL before access.

2015-11-25 07:14:58 +00:00

kern_idle.c

…

kern_intr.c

The part of r285680 which removed release semantic for two stores to

2015-07-21 14:39:34 +00:00

kern_jail.c

Speed up rctl operation with large rulesets, by holding the lock

2015-11-15 12:10:51 +00:00

kern_khelp.c

…

kern_kthread.c

Limit rights on process descriptors.

2015-07-31 10:21:58 +00:00

kern_ktr.c

Expand ktr_mask to be a 64-bit unsigned integer.

2015-05-22 11:09:41 +00:00

kern_ktrace.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_linker.c

Export various helper variables describing the layout and size of

2015-11-12 22:00:59 +00:00

kern_lock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_lockf.c

Improve style and fix a possible use-after-free case introduced in r268384

2015-01-10 06:48:35 +00:00

kern_lockstat.c

Consistently use a reader/writer flag for lockstat probes in rwlock(9) and

2015-07-19 22:24:33 +00:00

kern_loginclass.c

Speed up rctl operation with large rulesets, by holding the lock

2015-11-15 12:10:51 +00:00

kern_malloc.c

Consistently enforce the restriction against calling malloc/free when in a

2015-11-19 14:04:53 +00:00

kern_mbuf.c

Fix integer truncation bug in malloc(9)

2015-04-01 12:42:26 +00:00

kern_mib.c

Export various helper variables describing the layout and size of

2015-11-12 22:00:59 +00:00

kern_module.c

Provide better debug message on kernel module name clash.

2015-10-10 09:21:55 +00:00

kern_mtxpool.c

…

kern_mutex.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_ntptime.c

Use the monotonic (uptime) counter rather than time-of-day to measure elapsed

2015-07-12 18:38:17 +00:00

kern_numa.c

Add an initial NUMA affinity/policy configuration for threads and processes.

2015-07-11 15:21:37 +00:00

kern_osd.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_physio.c

Add missing NULL check in physio().

2015-10-29 13:53:37 +00:00

kern_pmc.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_poll.c

When a kernel has DEVICE_POLLING turned on but no drivers have

2015-04-14 14:22:34 +00:00

kern_priv.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_proc.c

Export various helper variables describing the layout and size of

2015-11-12 22:00:59 +00:00

kern_procctl.c

If process becomes reaper (procctl(PROC_REAP_ACQUIRE)) while already

2015-08-20 22:44:26 +00:00

kern_prot.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_racct.c

Speed up rctl operation with large rulesets, by holding the lock

2015-11-15 12:10:51 +00:00

kern_rangelock.c

…

kern_rctl.c

Speed up rctl operation with large rulesets, by holding the lock

2015-11-15 12:10:51 +00:00

kern_resource.c

Speed up rctl operation with large rulesets, by holding the lock

2015-11-15 12:10:51 +00:00

kern_rmlock.c

Threads holding a read lock of a sleepable rm lock are not permitted

2015-09-15 22:16:21 +00:00

kern_rwlock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_sdt.c

…

kern_sema.c

…

kern_sharedpage.c

Split kerne timekeep ABI structure vdso_sv_tk out of the struct

2015-11-23 07:09:35 +00:00

kern_shutdown.c

Disable suspend when we're shutting down. This solves the "tell FreeBSD

2015-10-01 10:52:26 +00:00

kern_sig.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

kern_switch.c

Revert for r277213:

2015-01-22 11:12:42 +00:00

kern_sx.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_synch.c

Remove several write-only variables, all reported by the gcc 4.9

2015-05-29 13:24:17 +00:00

kern_syscalls.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

kern_sysctl.c

Flesh out sysctl types further (follow-up of r290475)

2015-11-07 18:26:32 +00:00

kern_tc.c

Define fhard in pps_event(..) only when PPS_SYNC is defined to mute

2015-11-02 03:14:37 +00:00

kern_thr.c

Add an API for easily creating userspace threads in kernelspace.

2015-07-20 10:20:04 +00:00

kern_thread.c

Mark struct thread zone as type-stable.

2015-10-20 20:29:21 +00:00

kern_time.c

Fix an off by one in ppsratecheck(). If you asked for N=1 you'd get one,

2015-01-11 20:48:29 +00:00

kern_timeout.c

Add new async_drain to the callout system. This is so-far not used but

2015-11-10 14:49:32 +00:00

kern_umtx.c

Minor (and incomplete) style cleanup.

2015-10-30 20:47:42 +00:00

kern_uuid.c

…

kern_xxx.c

…

ksched.c

Use P1B_PRIO_MAX to designate max posix priority for the RR/FIFO

2015-08-30 18:02:57 +00:00

link_elf_obj.c

Add support for weak symbols to the kernel linkers. It means that

2015-09-20 01:27:59 +00:00

link_elf.c

Have elf_lookup() return an error if the specified non-weak symbol could

2015-11-03 03:29:35 +00:00

linker_if.m

…

Make.tags.inc

Bring the tags and links entries for amd64 up to date.

2015-10-27 22:59:24 +00:00

Makefile

…

makesyscalls.sh

Rename remaining linux32 symbols such as linux_sysent[] and

2015-10-22 21:28:20 +00:00

md4c.c

…

md5c.c

…

p1003_1b.c

In preparation for switching linuxulator to the use the native 1:1

2015-05-24 14:44:06 +00:00

posix4_mib.c

…

sched_4bsd.c

kgdb uses td_oncpu to determine if a thread is running and should use

2015-08-03 20:43:36 +00:00

sched_ule.c

Summary: Add the interactivity equations to the header comment for our

2015-08-26 16:36:41 +00:00

serdev_if.m

…

stack_protector.c

Use nitems() macro instead of __arraycount()

2015-06-16 20:19:00 +00:00

subr_acl_nfs4.c

Expose an interface to determine if an ACE is inherited.

2015-09-04 00:14:20 +00:00

subr_acl_posix1e.c

…

subr_autoconf.c

…

subr_blist.c

…

subr_bufring.c

…

subr_bus_dma.c

…

subr_bus.c

Huge cleanup of random(4) code.

2015-06-30 17:00:45 +00:00

subr_busdma_bufalloc.c

Fix printf format to allow for bus_size_t not being u_long on all platforms.

2015-10-20 03:25:17 +00:00

subr_capability.c

Remove duplicated includes.

2014-06-26 13:57:44 +00:00

subr_clock.c

For architectures where time_t is wide enough, in particular, 64bit

2014-12-12 09:37:18 +00:00

subr_counter.c

…

subr_devstat.c

Fix multiple incorrect SYSCTL arguments in the kernel:

2014-10-21 07:31:21 +00:00

subr_disk.c

…

subr_dummy_vdso_tc.c

Update the vdso timehands only via tc_windup().

2015-01-20 03:54:30 +00:00

subr_eventhandler.c

…

subr_fattime.c

Where appropriate, use the modern terms for the one true time base

2014-12-21 05:07:11 +00:00

subr_firmware.c

Create a dedicated function for ensuring that cdir and rdir are populated.

2015-07-11 16:22:48 +00:00

subr_hash.c

…

subr_hints.c

Add a new device control utility for new-bus devices called devctl. This

2015-02-06 16:09:01 +00:00

subr_kdb.c

Fix multiple incorrect SYSCTL arguments in the kernel:

2014-10-21 07:31:21 +00:00

subr_kobj.c

…

subr_lock.c

Add _NEW flag to mtx(9), sx(9), rmlock(9) and rwlock(9).

2014-12-13 21:00:10 +00:00

subr_log.c

…

subr_mbpool.c

All mbuf external free functions never fail, so let them be void.

2014-07-11 13:58:48 +00:00

subr_mchain.c

…

subr_module.c

preload_search_info: make sure mod is set

2015-08-21 15:57:57 +00:00

subr_msgbuf.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

subr_param.c

Ensure that maxproc does not exceed pid_max, at the time of boot.

2015-09-21 15:02:59 +00:00

subr_pcpu.c

…

subr_pctrie.c

…

subr_power.c

…

subr_prf.c

Add vlog(9).

2015-11-19 05:50:22 +00:00

subr_prof.c

The process spin lock currently has the following distinct uses:

2014-11-26 14:10:00 +00:00

subr_rman.c

When dumping an rman in DDB, include the RID of each resource.

2015-11-05 23:12:23 +00:00

subr_rtc.c

…

subr_sbuf.c

Fail the sbuf if vsnprintf(3) fails.

2015-10-02 09:23:14 +00:00

subr_scanf.c

…

subr_sfbuf.c

Move KASSERT into locked region.

2014-08-11 15:06:07 +00:00

subr_sglist.c

Fix a couple of panics when detaching from a cxgbe/cxl interface that was

2015-01-26 16:26:28 +00:00

subr_sleepqueue.c

Revert for r277213:

2015-01-22 11:12:42 +00:00

subr_smp.c

Since r289279 bufinit() uses mp_ncpus, but some architectures set this

2015-11-08 14:26:50 +00:00

subr_stack.c

Add support for a configurable output channel to witness(4).

2015-11-19 05:56:59 +00:00

subr_syscall.c

Add a way to distinguish between forking and thread creation in schedtail.

2015-10-22 09:33:34 +00:00

subr_taskqueue.c

This fixes several places where callout_stops return is examined. The

2015-11-13 22:51:35 +00:00

subr_terminal.c

vt(4): Adjust the cursor position after changing the window size

2014-11-01 17:05:15 +00:00

subr_trap.c

racct: perform a lockless check for p_throttled

2015-07-13 22:52:11 +00:00

subr_turnstile.c

ddb: finish converting boolean values.

2015-05-21 15:16:18 +00:00

subr_uio.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

subr_unit.c

…

subr_vmem.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

subr_witness.c

The buffer passed to an sbuf drain callback is not necessarily

2015-11-23 18:45:35 +00:00

sys_capability.c

Introduce kern_cap_rights_limit().

2015-08-11 08:43:50 +00:00

sys_generic.c

Cover a race between doselwakeup() and selfdfree(). If doselwakeup()

2015-07-09 09:22:21 +00:00

sys_pipe.c

Make pipes in CloudABI work.

2015-07-29 17:18:27 +00:00

sys_procdesc.c

The si_status field of the siginfo_t, provided by the waitid(2) and

2015-07-18 09:02:50 +00:00

sys_process.c

- Consistently use PROC_ASSERT_HELD() to verify that a process' hold count

2015-11-08 01:38:56 +00:00

sys_socket.c

In preparation of merging projects/sendfile, transform bare access to

2014-11-12 09:57:15 +00:00

syscalls.c

Missing regen after last change to sys/kern/syscalls.master.

2015-10-22 21:30:39 +00:00

syscalls.master

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

systrace_args.c

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

sysv_ipc.c

…

sysv_msg.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

sysv_sem.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

sysv_shm.c

Change the default setting of kern.ipc.shm_allow_removed from 0 to 1.

2015-10-10 09:29:47 +00:00

tty_compat.c

…

tty_info.c

…

tty_inq.c

…

tty_outq.c

…

tty_pts.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

tty_tty.c

tty: replace several curthread->td_proc with stored curproc

2015-07-06 18:53:56 +00:00

tty_ttydisc.c

…

tty.c

filedesc: simplify fget_unlocked & friends

2015-02-17 23:54:06 +00:00

uipc_accf.c

The accept filter code is not specific to the FreeBSD IPv4 network stack,

2014-07-26 19:27:34 +00:00

uipc_debug.c

Fix sb_state constant names as used e.g. to display in DDB ``show sockbuf''.

2015-07-21 09:57:13 +00:00

uipc_domain.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

uipc_mbuf2.c

Add const-qualifiers for source mbuf argument in m_dup(), m_copym(),

2015-08-08 15:50:46 +00:00

uipc_mbuf.c

There are times when it would be really nice to have a record of the last few

2015-10-14 00:35:37 +00:00

uipc_mbufhash.c

Reduce header pollution.

2015-03-17 14:16:50 +00:00

uipc_mqueue.c

fd: remove filedesc argument from fdclose

2015-04-11 15:40:28 +00:00

uipc_sem.c

Call ksem_get() with initialized 'rights'.

2015-07-23 23:18:03 +00:00

uipc_shm.c

Add kern_shm_open().

2015-08-01 07:21:14 +00:00

uipc_sockbuf.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

uipc_socket.c

Make shutdown() return ENOTCONN as required by POSIX, part deux.

2015-07-27 13:17:57 +00:00

uipc_syscalls.c

Make shutdown() return ENOTCONN as required by POSIX, part deux.

2015-07-27 13:17:57 +00:00

uipc_usrreq.c

fd: make the common case in filecaps_copy work lockless

2015-09-07 20:02:56 +00:00

vfs_acl.c

Replace struct filedesc argument in getvnode with struct thread

2015-06-16 13:09:18 +00:00

vfs_aio.c

The aio_waitcomplete(2) syscall should not sleep when the given timeout

2015-10-25 18:48:09 +00:00

vfs_bio.c

Add a sched_yield() to work around low memory conditions in the current code.

2015-11-07 04:04:00 +00:00

vfs_cache.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

vfs_cluster.c

Refactor unmapped buffer address handling.

2015-07-23 19:13:41 +00:00

vfs_default.c

Ensure that vop_stdadvise() does not call getblk() on vnodes that have an

2015-10-01 16:34:53 +00:00

vfs_export.c

After the changes in r274118 make NOIP kernels compile by hiding an

2014-11-06 12:19:39 +00:00

vfs_extattr.c

Replace struct filedesc argument in getvnode with struct thread

2015-06-16 13:09:18 +00:00

vfs_hash.c

Track changes to kern.maxvnodes and appropriately increase or decrease

2015-09-06 05:50:51 +00:00

vfs_init.c

sysctl: switch sysctllock to a sleepable rmlock, take 2

2015-09-15 23:06:56 +00:00

vfs_lookup.c

save some bytes by using more concise SDT_PROBE<n> instead of SDT_PROBE

2015-09-28 12:14:16 +00:00

vfs_mount.c

Make vfs_unmountall() unmount /dev after /, not before. The only

2015-08-24 13:18:13 +00:00

vfs_mountroot.c

After r290196, the kernel won't wait for stuff like gmirror nodes

2015-10-30 15:52:10 +00:00

vfs_subr.c

As the kernel allocates and frees vnodes, it fully initializes them

2015-11-29 21:42:26 +00:00

vfs_syscalls.c

The freebsd4_getfsstat() was broken in r281551 to always return 0 on success.

2015-11-20 14:08:12 +00:00

vfs_vnops.c

Fix r283998 that broke mapin events for hwpmc.

2015-10-08 09:54:33 +00:00

vnode_if.src

kevent(2): Note DOOMED vnodes with NOTE_REVOKE

2015-09-15 20:22:30 +00:00