Alexander Motin 1cf78c85c5 Create own free list for each of the first 32 possible allocation sizes. ...

In case of 4K allocation quantum that means for allocations up to 128K.

With growth of memory fragmentation these lists may grow to quite a large
sizes (tenths and hundreds of thousands items).  Having in one list items
of different sizes in worst case may require full linear list traversal,
that may be very expensive.  Having lists for items of single size means
that unless user specify some alignment or border requirements (that are
very rare cases) first item found on the list should satisfy the request.

While running SPEC NFS benchmark on top of ZFS on 24-core machine with
84GB RAM this change reduces CPU time spent in vmem_xalloc() from 8%
and lock congestion spinning around it from 20% to invisible levels.
And that all is by the cost of just 26 more pointers per vmem instance.

If at some point our kernel will start to actively use KVA allocations
with odd sizes above 128K, something may need to be done to bigger lists
also.

2013-12-11 21:48:04 +00:00

..

bus_if.m

Add a BUS_CHILD_DELETED() method that a bus can hook to allow it to cleanup

2012-08-21 18:13:09 +00:00

capabilities.conf

- Remove mac_get_fd/mac_set_fd - those are not syscalls. The __mac_get_fd() and

2013-11-06 07:46:10 +00:00

clock_if.m

…

cpufreq_if.m

…

device_if.m

Revert r239178 and implement two new functions, namely

2012-08-15 15:42:57 +00:00

genassym.sh

…

imgact_aout.c

Cosmetics: define FREEBSD32_MINUSER and AOUT32_MINUSER for struct

2012-07-22 13:41:45 +00:00

imgact_elf32.c

…

imgact_elf64.c

…

imgact_elf.c

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use

2013-09-09 18:11:59 +00:00

imgact_gzip.c

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use

2013-09-09 18:11:59 +00:00

imgact_shell.c

…

inflate.c

…

init_main.c

Debugging. My attempt at EVENTHANDLER(multiuser) was a failure; use EVENTHANDLER(mountroot) instead.

2013-10-08 06:54:52 +00:00

init_sysent.c

Regen.

2013-09-19 18:56:00 +00:00

kern_acct.c

acct: create a special plimit object and set it for exiting processes

2013-06-30 19:08:06 +00:00

kern_alq.c

The fix committed in r250951 replaced the reported panic with a deadlock... gold

2013-06-17 09:49:07 +00:00

kern_clock.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

kern_clocksource.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_condvar.c

Fix lc_lock/lc_unlock() support for rmlocks held in shared mode. With

2013-09-20 23:06:21 +00:00

kern_conf.c

Remove global device lock acquisition from dev_relthread(), replacing it

2013-10-22 10:40:26 +00:00

kern_cons.c

cngetc: use cpu_spinwait to ease the cncheckc loop a tiny bit

2012-10-06 19:50:23 +00:00

kern_context.c

…

kern_cpu.c

Revert r175376 and tune cpufreq(4) frequency comparison logic instead.

2012-03-10 18:56:16 +00:00

kern_cpuset.c

Several improvements to rmlock(9). Many of these are based on patches

2013-06-25 18:44:15 +00:00

kern_ctf.c

Remove the support for using non-mpsafe filesystem modules.

2012-10-22 17:50:54 +00:00

kern_descrip.c

Migrate the sendfile_sync structure into a public(ish) API in preparation

2013-12-01 03:53:21 +00:00

kern_dtrace.c

…

kern_environment.c

Make getenv_*() functions and respectively TUNABLE_*_FETCH() macros not

2013-11-01 10:32:33 +00:00

kern_et.c

Fix incorrect assertion that caused panic when periodic-only timers used.

2013-03-13 06:42:01 +00:00

kern_event.c

Replace CAP_POLL_EVENT and CAP_POST_EVENT capability rights (which I had

2013-11-15 19:55:35 +00:00

kern_exec.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

kern_exit.c

Make process descriptors standard part of the kernel. rwhod(8) already

2013-11-30 15:08:35 +00:00

kern_fail.c

…

kern_ffclock.c

…

kern_fork.c

Make process descriptors standard part of the kernel. rwhod(8) already

2013-11-30 15:08:35 +00:00

kern_gzio.c

Remove the support for using non-mpsafe filesystem modules.

2012-10-22 17:50:54 +00:00

kern_hhook.c

Move hhook's per-vnet initialisation to an earlier SYSINIT SI_SUB stage to

2013-06-15 10:08:34 +00:00

kern_idle.c

…

kern_intr.c

Snapshot. This passes the build test, but has not yet been finished or debugged.

2013-10-04 06:55:06 +00:00

kern_jail.c

jail_v0.ip_number was always in host byte order. This was handled

2013-11-28 19:40:33 +00:00

kern_khelp.c

Cleanup and simplification in khelp_{register|deregister}_helper(). No

2013-06-15 06:45:17 +00:00

kern_kthread.c

Do not use potentially stale thread in kthread_add()

2013-08-17 17:02:43 +00:00

kern_ktr.c

ktr: correctly handle possible wrap-around in the boot buffer

2013-02-08 07:29:07 +00:00

kern_ktrace.c

Fix panic in ktrcapfail() when no capability rights are passed.

2013-09-18 19:26:08 +00:00

kern_linker.c

Rename the kld_unload event handler to kld_unload_try, and add a new

2013-08-24 21:13:38 +00:00

kern_lock.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_lockf.c

…

kern_lockstat.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_loginclass.c

…

kern_malloc.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_mbuf.c

Ignore attempts to set the nmbcluster sysctls to their current value

2013-10-10 16:11:34 +00:00

kern_mib.c

Rename sysctl kern.supported_abis to kern.supported_archs, since it gives

2013-12-04 16:38:40 +00:00

kern_module.c

Fix a typo.

2012-08-22 20:01:57 +00:00

kern_mtxpool.c

…

kern_mutex.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_ntptime.c

rename scheduler->swapper and SI_SUB_RUN_SCHEDULER->SI_SUB_LAST

2013-07-24 09:45:31 +00:00

kern_osd.c

…

kern_physio.c

Fix some issues in change 254760 pointed out by Bruce Evans:

2013-08-29 16:41:40 +00:00

kern_pmc.c

Add software PMC support.

2012-03-28 20:58:30 +00:00

kern_poll.c

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare

2013-10-26 17:58:36 +00:00

kern_priv.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

kern_proc.c

Add an kinfo sysctl to retrieve signal trampoline location for the

2013-11-26 19:47:09 +00:00

kern_prot.c

Style fix

2012-11-14 10:33:12 +00:00

kern_racct.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

kern_rangelock.c

Change the queue of locks in kern_rangelock.c from holding lock requests in

2013-08-15 20:19:17 +00:00

kern_rctl.c

Add CPU percentage limit enforcement to RCTL. The resouce name is "pcpu".

2012-10-26 16:01:08 +00:00

kern_resource.c

Add a resource limit for the total number of kqueues available to the

2013-10-21 16:44:53 +00:00

kern_rmlock.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_rwlock.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_sdt.c

sdt: add support for solaris/illumos style DTRACE_PROBE macros

2013-11-26 08:49:53 +00:00

kern_sema.c

…

kern_sharedpage.c

Remove the deprecated VM_ALLOC_RETRY flag for the vm_page_grab(9).

2013-08-22 07:39:53 +00:00

kern_shutdown.c

Make panic_reboot_wait_time static.

2013-12-05 03:01:41 +00:00

kern_sig.c

Make process descriptors standard part of the kernel. rwhod(8) already

2013-11-30 15:08:35 +00:00

kern_switch.c

Add a comment on why inlining critical_enter() may not be a good idea

2012-12-09 04:54:22 +00:00

kern_sx.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

kern_synch.c

use saner calculations in should_yield

2013-11-26 14:00:50 +00:00

kern_syscalls.c

…

kern_sysctl.c

Add a helpful message that can help point to why a sysctl tree removal failed

2013-08-09 01:04:44 +00:00

kern_tc.c

- Make callout(9) tickless, relying on eventtimers(4) as backend for

2013-03-04 11:09:56 +00:00

kern_thr.c

Stop treating td_sigmask specially for the purposes of new thread

2012-05-26 20:03:47 +00:00

kern_thread.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

kern_time.c

Implement compat32 wrappers for the ktimer_* syscalls.

2013-07-21 19:43:52 +00:00

kern_timeout.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

kern_umtx.c

Fix two issues with the spin loops in the umtx(2) implementation.

2013-06-13 09:33:22 +00:00

kern_uuid.c

Further restrict the MAC addresses that we use for UUID generation

2013-07-24 18:13:43 +00:00

kern_xxx.c

…

ksched.c

sched_rr_interval() seems always returned period in hz ticks, but same

2012-08-10 18:19:57 +00:00

link_elf_obj.c

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use

2013-09-09 18:11:59 +00:00

link_elf.c

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use

2013-09-09 18:11:59 +00:00

linker_if.m

…

Make.tags.inc

- Trim an unused and bogus Makefile for mount_smbfs.

2013-06-28 21:00:08 +00:00

Makefile

…

makesyscalls.sh

Break the loop once we know we have the SYF_CAPENABLED flag.

2013-12-04 00:10:37 +00:00

md4c.c

…

md5c.c

…

p1003_1b.c

…

posix4_mib.c

…

sched_4bsd.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

sched_ule.c

Fix an off-by-one error in r228960. The maximum priority delta provided

2013-12-03 14:50:12 +00:00

serdev_if.m

…

stack_protector.c

…

subr_acl_nfs4.c

Fix bug where NFSv4 ACL enforcement code wouldn't unconditionally

2012-04-17 14:54:00 +00:00

subr_acl_posix1e.c

Add module load/unload stubs.

2012-03-13 20:27:48 +00:00

subr_autoconf.c

…

subr_blist.c

Remove reference to the rlist code from comments, and fix a typo visible

2013-02-05 20:08:33 +00:00

subr_bufring.c

…

subr_bus_dma.c

Add bus_dmamap_load_ma() function to load map with the array of

2013-10-27 21:39:16 +00:00

subr_bus.c

Add YARROW_RNG and FORTUNA_RNG to sys/conf/options.

2013-10-08 11:05:26 +00:00

subr_busdma_bufalloc.c

Replace kernel virtual address space allocation with vmem. This provides

2013-08-07 06:21:20 +00:00

subr_capability.c

Change cap_rights_merge(3) and cap_rights_remove(3) to return pointer

2013-11-14 22:59:20 +00:00

subr_clock.c

…

subr_counter.c

Revert r249590 and in case if mp_ncpus isn't initialized use MAXCPU. This

2013-07-23 11:16:40 +00:00

subr_devstat.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

subr_disk.c

…

subr_dummy_vdso_tc.c

Implement mechanism to export some kernel timekeeping data to

2012-06-22 07:06:40 +00:00

subr_eventhandler.c

…

subr_fattime.c

…

subr_firmware.c

Correct sizeof usage

2012-06-25 05:41:16 +00:00

subr_hash.c

…

subr_hints.c

Style fixes.

2012-09-04 23:16:55 +00:00

subr_kdb.c

When reentering kdb, typically due to a bug causing trap or assert in

2013-10-27 16:20:52 +00:00

subr_kobj.c

…

subr_lock.c

Several improvements to rmlock(9). Many of these are based on patches

2013-06-25 18:44:15 +00:00

subr_log.c

MFcalloutng (r244255 by mav, with minor changes):

2013-03-04 16:07:55 +00:00

subr_mbpool.c

Give (*ext_free) an int return value allowing for very sophisticated

2013-08-25 10:57:09 +00:00

subr_mchain.c

Mechanically substitute flags from historic mbuf allocator with

2012-12-05 08:04:20 +00:00

subr_module.c

…

subr_msgbuf.c

- Clean up timestamps in msgbuf code. The timestamps should now be

2012-03-19 00:36:32 +00:00

subr_param.c

Add VM_LAST, a special last element in enum VM_GUEST and use it in CTASSERT

2013-11-12 20:13:10 +00:00

subr_pcpu.c

…

subr_pctrie.c

- Add a new general purpose path-compressed radix trie which can be used

2013-05-12 04:05:01 +00:00

subr_power.c

…

subr_prf.c

Reduce the scope of the proctree_lock. If several processes cause

2013-09-13 06:39:10 +00:00

subr_prof.c

…

subr_rman.c

Unlock in the error path to prevent a lock leak.

2012-05-31 17:27:05 +00:00

subr_rtc.c

…

subr_sbuf.c

Always request zeroed memory, in case we're dumb enough to leak it later.

2013-09-22 23:47:56 +00:00

subr_scanf.c

…

subr_sglist.c

…

subr_sleepqueue.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

subr_smp.c

Fix ia64 and mips kernel builds due to XENHVM=>GENERIC integration in

2013-09-22 02:46:13 +00:00

subr_stack.c

…

subr_syscall.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

subr_taskqueue.c

add taskqueue_drain_all

2013-11-28 18:56:34 +00:00

subr_terminal.c

Merge VT(9) project (a.k.a. newcons).

2013-12-05 22:38:53 +00:00

subr_trap.c

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging

2013-11-25 07:38:45 +00:00

subr_turnstile.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

subr_uio.c

Remove zero-copy sockets code. It only worked for anonymous memory,

2013-09-16 06:25:54 +00:00

subr_unit.c

Move the definition of the struct unrhdr into a separate header file,

2013-08-30 07:37:45 +00:00

subr_vmem.c

Create own free list for each of the first 32 possible allocation sizes.

2013-12-11 21:48:04 +00:00

subr_witness.c

Trim a couple of panic messages.

2013-09-04 11:52:28 +00:00

sys_capability.c

This looks like a typo that breaks the build. Yell at me if this isn't the

2013-09-05 03:36:57 +00:00

sys_generic.c

Replace CAP_POLL_EVENT and CAP_POST_EVENT capability rights (which I had

2013-11-15 19:55:35 +00:00

sys_pipe.c

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use

2013-09-09 18:11:59 +00:00

sys_procdesc.c

Make process descriptors standard part of the kernel. rwhod(8) already

2013-11-30 15:08:35 +00:00

sys_process.c

Extend the support for exempting processes from being killed when swap is

2013-09-19 18:53:42 +00:00

sys_socket.c

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare

2013-10-26 17:58:36 +00:00

syscalls.c

Regen.

2013-09-19 18:56:00 +00:00

syscalls.master

Extend the support for exempting processes from being killed when swap is

2013-09-19 18:53:42 +00:00

systrace_args.c

Regenerate syscall argument strings after r255777.

2013-09-21 23:06:36 +00:00

sysv_ipc.c

…

sysv_msg.c

…

sysv_sem.c

…

sysv_shm.c

Eliminate the gratuitous use of mmap(2) flags from the implementation

2013-11-12 17:46:11 +00:00

tty_compat.c

…

tty_info.c

Fix whitespace inconsistencies in TTY code.

2012-02-06 18:15:46 +00:00

tty_inq.c

…

tty_outq.c

…

tty_pts.c

Make sendfile() a method in the struct fileops. Currently only

2013-08-15 07:54:31 +00:00

tty_tty.c

…

tty_ttydisc.c

Correct SIGTTIN handling.

2012-10-25 09:05:21 +00:00

tty.c

Change the cap_rights_t type from uint64_t to a structure that we can extend

2013-09-05 00:09:56 +00:00

uipc_accf.c

…

uipc_debug.c

Fix socket buffer timeouts precision using the new sbintime_t KPI instead

2013-09-01 23:34:53 +00:00

uipc_domain.c

- Implement two new system calls:

2013-03-02 21:11:30 +00:00

uipc_mbuf2.c

Mechanically substitute flags from historic mbuf allocator with

2012-12-05 08:04:20 +00:00

uipc_mbuf.c

Fix a very bad typo from r248887.

2013-11-14 09:45:33 +00:00

uipc_mqueue.c

Replace CAP_POLL_EVENT and CAP_POST_EVENT capability rights (which I had

2013-11-15 19:55:35 +00:00

uipc_sem.c

Change the cap_rights_t type from uint64_t to a structure that we can extend

2013-09-05 00:09:56 +00:00

uipc_shm.c

Implement sendfile(2) for the posix shared memory segment file descriptor,

2013-09-11 06:41:15 +00:00

uipc_sockbuf.c

- Substitute sbdrop_internal() with sbcut_internal(). The latter doesn't free

2013-10-09 11:57:53 +00:00

uipc_socket.c

Fix typo in a comment.

2013-11-08 20:11:15 +00:00

uipc_syscalls.c

Remove unused variable.

2013-12-01 20:03:00 +00:00

uipc_usrreq.c

Provide pr_ctloutput method for AF_LOCAL/SOCK_SEQPACKET sockets.

2013-09-11 18:22:30 +00:00

vfs_acl.c

Change the cap_rights_t type from uint64_t to a structure that we can extend

2013-09-05 00:09:56 +00:00

vfs_aio.c

The fget() function now takes pointer to cap_rights_t, so change 0 to NULL.

2013-09-05 11:59:23 +00:00

vfs_bio.c

Fix detection of EOF in kern_physio(). If bio_length was clipped by

2013-12-10 21:15:18 +00:00

vfs_cache.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

vfs_cluster.c

When allocating a pbuf for the cluster write, do not sleep waiting

2013-08-27 01:31:12 +00:00

vfs_default.c

- Convert the bufobj lock to rwlock.

2013-05-31 00:43:41 +00:00

vfs_export.c

Further refine the handling of stop signals in the NFS client. The

2013-02-21 19:02:50 +00:00

vfs_extattr.c

Change the cap_rights_t type from uint64_t to a structure that we can extend

2013-09-05 00:09:56 +00:00

vfs_hash.c

Add exported vfs_hash_index() function, which calculates the canonical

2013-01-14 05:41:40 +00:00

vfs_init.c

Revert accidental commit.

2013-06-29 05:05:57 +00:00

vfs_lookup.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

vfs_mount.c

Change len checks for fstypelen and fspathlen to be against absolute len

2013-10-03 22:52:03 +00:00

vfs_mountroot.c

In r243868, the error message buffer errmsg have been changed from

2013-09-09 05:01:18 +00:00

vfs_subr.c

Do not flush buffers when the v_object of the passed vnode does not

2013-10-09 18:43:29 +00:00

vfs_syscalls.c

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

2013-11-26 08:46:27 +00:00

vfs_vnops.c

Revert back to use int for the page counts. In vn_io_fault(), the i/o

2013-11-20 08:45:26 +00:00

vnode_if.src

If filesystem declares that it supports shared locking for writes, use

2013-11-09 20:36:29 +00:00