freebsd-dev/sys/kern
Mateusz Guzik 35bb59edc5 threads: reimplement tid allocation on top of a bitmap
There are workloads with very bursty tid allocation and since unr tries very
hard to have small-sized bitmaps it keeps reallocating memory. Just doing
buildkernel gives almost 150k calls to free coming from unr.

This also gets rid of the hack which tried to postpone TID reuse.

Reviewed by:	kib, markj
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D27101
2020-11-09 23:05:28 +00:00
..
bus_if.m Remove trailing white space. 2020-02-26 16:22:28 +00:00
capabilities.conf Permit cpuset_(get|set)domain() in capability mode. 2020-07-06 16:34:29 +00:00
clock_if.m
cpufreq_if.m
device_if.m
genassym.sh
genoffset.c
genoffset.sh
imgact_aout.c Get rid of sv_errtbl and SV_ABI_ERRNO(). 2020-09-17 11:39:33 +00:00
imgact_binmisc.c imgact_binmisc: limit the extent of match on incoming entries 2020-11-08 04:24:29 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c random(4) FenestrasX: Push root seed version to arc4random(3) 2020-10-10 21:52:00 +00:00
imgact_shell.c
init_main.c vmspace: Convert to refcount(9) 2020-11-04 16:30:56 +00:00
init_sysent.c Regen after r366145. 2020-09-25 10:05:38 +00:00
kern_acct.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_alq.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_clock.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_clocksource.c
kern_condvar.c
kern_conf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_cons.c Remove sysctl_kern_consmute() 2020-10-05 15:54:19 +00:00
kern_context.c
kern_cpu.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_cpuset.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_ctf.c
kern_descrip.c fd: make all f_count uses go through refcount_* 2020-11-05 02:12:33 +00:00
kern_dtrace.c
kern_dump.c Avoid dump_avail[] redefinition. 2020-10-14 22:51:40 +00:00
kern_environment.c Move kernel env global variables, etc to sys/kenv.h 2020-10-07 06:16:37 +00:00
kern_et.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_event.c kqueue: save space by using only one func pointer for assertions 2020-11-09 00:04:35 +00:00
kern_exec.c vmspace: Convert to refcount(9) 2020-11-04 16:30:56 +00:00
kern_exit.c proc_realparent: if p_oppid does not match pid of the current parent 2020-09-16 21:38:24 +00:00
kern_fail.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_ffclock.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
kern_fork.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_hhook.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_idle.c
kern_intr.c Allow swi_sched() to be called from NMI context. 2020-07-25 15:19:38 +00:00
kern_jail.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_kcov.c Mark COVERAGE and KCOV as part of KCSAN 2020-08-19 14:11:25 +00:00
kern_khelp.c
kern_kthread.c Do a lockless check in kthread_suspend_check 2020-07-25 07:14:33 +00:00
kern_ktr.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_ktrace.c Move KTRUSERRET() from userret() to ast(). It's a really long 2020-10-03 12:03:08 +00:00
kern_linker.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_lock.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_lockf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_lockstat.c
kern_loginclass.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
kern_malloc.c malloc: provide 384 byte zone 2020-11-09 22:59:41 +00:00
kern_mbuf.c Add m_snd_tag_alloc() as a wrapper around if_snd_tag_alloc(). 2020-10-29 23:28:39 +00:00
kern_mib.c Make sysctl user.local a tunable that can be written at run-time 2020-10-31 23:48:41 +00:00
kern_module.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_mtxpool.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_mutex.c mtx: add mtx_wait_unlocked 2020-08-04 23:00:00 +00:00
kern_ntptime.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_osd.c
kern_physio.c physio: Don't store user addresses in bio_data 2020-10-15 17:05:21 +00:00
kern_pmc.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_poll.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
kern_priv.c vfs: respect PRIV_VFS_LOOKUP in vaccess_smr 2020-08-25 14:18:50 +00:00
kern_proc.c Put calls to check_pgrp_jobc() in fixjobc_kill() under INVARIANTS. 2020-09-17 00:07:15 +00:00
kern_procctl.c correct procctl(PROC_PROTMAX_STATUS _NOFORCE return 2020-05-01 14:30:59 +00:00
kern_prot.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_racct.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
kern_rangelock.c
kern_rctl.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
kern_resource.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_rmlock.c rms: several cleanups + debug read lockers handling 2020-11-07 16:57:53 +00:00
kern_rwlock.c locks: fix a long standing bug for primitives with kdtrace but without spinning 2020-07-23 17:26:53 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_sharedpage.c random(4) FenestrasX: Push root seed version to arc4random(3) 2020-10-10 21:52:00 +00:00
kern_shutdown.c Remove unused function cpu_boot() 2020-10-06 23:16:56 +00:00
kern_sig.c sig_intr(9): return early if AST is not scheduled. 2020-10-08 22:34:34 +00:00
kern_switch.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
kern_sx.c locks: fix a long standing bug for primitives with kdtrace but without spinning 2020-07-23 17:26:53 +00:00
kern_synch.c Handle PCATCH in blockcount_sleep() so it can be interrupted. 2020-04-21 17:13:06 +00:00
kern_syscalls.c Move syscall_thread_{enter,exit}() into the slow path. This is only 2020-11-08 15:54:59 +00:00
kern_sysctl.c sysctl+kern_sysctl: Honor SKIP for descendant nodes 2020-10-24 16:17:07 +00:00
kern_tc.c Remove double-calls to tc_get_timecount() to warm timecounters. 2020-06-10 22:30:32 +00:00
kern_thr.c threads: introduce a limit for total number 2020-11-09 23:04:30 +00:00
kern_thread.c threads: reimplement tid allocation on top of a bitmap 2020-11-09 23:05:28 +00:00
kern_time.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
kern_timeout.c Assert that cc_exec_drain(cc, direct) is NULL before assigning a new value. 2020-09-02 10:00:30 +00:00
kern_tslog.c
kern_ubsan.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_umtx.c Support for userspace non-transparent superpages (largepages). 2020-09-09 22:12:51 +00:00
kern_uuid.c validate_uuid: absorb the rest of parse_uuid with a flags arg 2020-04-15 18:39:12 +00:00
kern_xxx.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
ksched.c
link_elf_obj.c link_elf_obj: Colour VM objects 2020-10-19 16:57:59 +00:00
link_elf.c malloc: delegate M_EXEC handling to dedicacted routines 2020-10-30 20:02:32 +00:00
linker_if.m
Make.tags.inc
Makefile sys/kern sysent: re-add dependency on capabilities.conf 2020-02-12 19:06:34 +00:00
makesyscalls.sh makesyscalls.sh: improve the 'this is going away' message 2020-07-28 01:05:40 +00:00
md4c.c
md5c.c
msi_if.m o Add iommu de-initialization method for MSI interface. 2020-10-24 20:09:27 +00:00
p1003_1b.c
pic_if.m
posix4_mib.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sched_4bsd.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sched_ule.c sys/: Document few more sysctls. 2020-03-02 15:30:52 +00:00
serdev_if.m
stack_protector.c
subr_acl_nfs4.c Drop "All rights reserved" from all my stuff. This includes 2020-10-28 13:46:11 +00:00
subr_acl_posix1e.c vfs: remove the obsolete privused argument from vaccess 2020-08-05 09:27:03 +00:00
subr_atomic64.c
subr_autoconf.c
subr_blist.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_boot.c
subr_bufring.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_bus_dma.c Move the iommu stubs to a generic place, so they are available on all the 2020-10-23 21:27:48 +00:00
subr_bus.c newbus: use ssize_t to match sb's len and size, fix ordering of space check 2020-10-12 22:07:44 +00:00
subr_busdma_bufalloc.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_capability.c Remove unused SYSINIT macros for capability rights. 2020-03-26 15:02:37 +00:00
subr_clock.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_compressor.c clamp kernel dump compression level when using gzip 2020-02-20 23:53:48 +00:00
subr_counter.c Rationalize per-cpu zones. 2020-11-05 15:08:56 +00:00
subr_coverage.c Mark COVERAGE and KCOV as part of KCSAN 2020-08-19 14:11:25 +00:00
subr_csan.c Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions. 2020-09-19 11:06:41 +00:00
subr_devmap.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_devstat.c Fix asymmetry in devstat(9) calls by GEOM. 2020-10-24 21:07:10 +00:00
subr_disk.c Enable bioq 'car limit' added at r335066 at 128 bios. 2020-10-26 04:04:06 +00:00
subr_dummy_vdso_tc.c
subr_early.c
subr_epoch.c epoch: support non-preemptible epochs checking in_epoch() 2020-11-07 03:29:04 +00:00
subr_eventhandler.c
subr_fattime.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_filter.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_firmware.c Fix a panic when unloading firmware 2020-06-29 21:35:50 +00:00
subr_gtaskqueue.c Remove the "config" taskqgroup and its KPIs. 2020-03-30 14:24:03 +00:00
subr_hash.c
subr_hints.c Move kernel env global variables, etc to sys/kenv.h 2020-10-07 06:16:37 +00:00
subr_intr.c Add a method to determine whether given interrupt is per CPU or not. 2020-11-07 14:58:01 +00:00
subr_kdb.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_kobj.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_lock.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_log.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_mchain.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_module.c Fix a loop condition 2020-10-08 18:29:17 +00:00
subr_msgbuf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_param.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_pcpu.c Add more per-cpu zones. 2020-11-09 00:34:23 +00:00
subr_pctrie.c Use SMR to provide safe unlocked lookup for pctries from SMR zones 2020-07-24 17:32:10 +00:00
subr_physmem.c Avoid dump_avail[] redefinition. 2020-10-14 22:51:40 +00:00
subr_pidctrl.c
subr_power.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_prf.c remove %n support from printf(9) 2020-05-09 15:56:02 +00:00
subr_prng.c Add prng(9) API 2020-08-13 20:48:14 +00:00
subr_prof.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_rangeset.c
subr_rman.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_rtc.c Address whitespace nits in subr_rtc.c 2020-09-28 17:19:57 +00:00
subr_sbuf.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_scanf.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_sfbuf.c
subr_sglist.c Step 4.2: start divorce of M_EXT and M_EXTPG 2020-05-03 00:37:16 +00:00
subr_sleepqueue.c Refactor sleepq_catch_signals(). 2020-10-04 16:30:05 +00:00
subr_smp.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_smr.c Use COUNTER_U64_DEFINE_EARLY() in places where it simplifies things. 2020-03-06 19:10:00 +00:00
subr_stack.c kern.tty_info_kstacks: add a compact format 2020-07-06 16:33:28 +00:00
subr_stats.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
subr_syscall.c Move syscall_thread_{enter,exit}() into the slow path. This is only 2020-11-08 15:54:59 +00:00
subr_taskqueue.c Add flag to struct task to mark the task as requiring network epoch. 2020-02-11 18:48:07 +00:00
subr_terminal.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_trap.c Move KTRUSERRET() from userret() to ast(). It's a really long 2020-10-03 12:03:08 +00:00
subr_turnstile.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_uio.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_unit.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
subr_vmem.c vmem: Allocate btags before looping in vmem_xalloc() 2020-10-19 16:54:06 +00:00
subr_witness.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sys_capability.c Enter a write sequence when updating rights. 2020-03-19 15:39:45 +00:00
sys_generic.c poll/select: change selfd_zone into a malloc type 2020-11-05 12:24:37 +00:00
sys_getrandom.c
sys_pipe.c pipe: whitespace nit in previous 2020-11-04 23:17:41 +00:00
sys_procdesc.c procdesc: convert the zone to a malloc type 2020-11-09 00:05:21 +00:00
sys_process.c Get rid of sa->narg. It serves no purpose; use sa->callp->sy_narg instead. 2020-09-27 18:47:06 +00:00
sys_socket.c The ioctl() calls using FIONREAD, FIONWRITE, FIONSPACE, and SIOCATMARK 2020-11-07 21:17:49 +00:00
syscalls.c Update the files created from the new syscalls.master from r361599. 2020-05-28 21:23:02 +00:00
syscalls.master Add a syscall for the nfs-over-tls daemons to use. 2020-05-28 21:06:10 +00:00
systrace_args.c Update the files created from the new syscalls.master from r361599. 2020-05-28 21:23:02 +00:00
sysv_ipc.c
sysv_msg.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
sysv_sem.c sysv_sem: semusz depends on semume. 2020-10-22 09:28:11 +00:00
sysv_shm.c sysvshm: pass relevant uap members as arguments 2020-11-03 19:14:03 +00:00
tty_compat.c
tty_info.c kern.tty_info_kstacks: set compact format as default 2020-07-06 16:34:15 +00:00
tty_inq.c
tty_outq.c
tty_pts.c
tty_tty.c
tty_ttydisc.c Implement FLUSHO 2020-08-27 05:11:15 +00:00
tty.c Check for process group change in tty_wait_background(). 2020-10-28 22:12:47 +00:00
uipc_accf.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
uipc_debug.c
uipc_domain.c
uipc_ktls.c Add m_snd_tag_alloc() as a wrapper around if_snd_tag_alloc(). 2020-10-29 23:28:39 +00:00
uipc_mbuf2.c
uipc_mbuf.c Permit sending empty fragments for TLS 1.0. 2020-10-13 17:30:34 +00:00
uipc_mbufhash.c Implement mbuf hashing routines for IP over infiniband, IPoIB. 2020-10-22 09:17:56 +00:00
uipc_mqueue.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
uipc_sem.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
uipc_shm.c Add a vmparam.h constant indicating pmap support for large pages. 2020-09-23 19:34:21 +00:00
uipc_sockbuf.c Permit sending empty fragments for TLS 1.0. 2020-10-13 17:30:34 +00:00
uipc_socket.c kqueue: save space by using only one func pointer for assertions 2020-11-09 00:04:35 +00:00
uipc_syscalls.c
uipc_usrreq.c fd: make all f_count uses go through refcount_* 2020-11-05 02:12:33 +00:00
vfs_acl.c vfs: fix trivial whitespace issues which don't interefere with blame 2020-07-10 09:01:36 +00:00
vfs_aio.c vmspace: Convert to refcount(9) 2020-11-04 16:30:56 +00:00
vfs_bio.c vmapbuf: don't smuggle address or length in buf 2020-10-21 16:00:15 +00:00
vfs_cache.c vfs: group mount per-cpu vars into one struct 2020-11-09 23:02:13 +00:00
vfs_cluster.c vfs: fix trivial whitespace issues which don't interefere with blame 2020-07-10 09:01:36 +00:00
vfs_default.c vfs: group mount per-cpu vars into one struct 2020-11-09 23:02:13 +00:00
vfs_export.c kern: clean up empty lines in .c and .h files 2020-09-01 22:12:32 +00:00
vfs_extattr.c vfs_extattr: Allow extattr names up to the full max 2020-05-14 03:01:23 +00:00
vfs_hash.c vfs: avoid exposing partially constructed vnodes 2020-09-05 00:26:03 +00:00
vfs_init.c vfs: fix trivial whitespace issues which don't interefere with blame 2020-07-10 09:01:36 +00:00
vfs_lookup.c vfs: change nt_zone into a malloc type 2020-11-05 12:06:50 +00:00
vfs_mount.c vfs: group mount per-cpu vars into one struct 2020-11-09 23:02:13 +00:00
vfs_mountroot.c cache: drop the force flag from purgevfs 2020-09-23 10:46:07 +00:00
vfs_subr.c vfs: group mount per-cpu vars into one struct 2020-11-09 23:02:13 +00:00
vfs_syscalls.c vfs: prevent avoidable evictions on mkdir of existing directories 2020-10-22 19:28:12 +00:00
vfs_vnops.c vfs: group mount per-cpu vars into one struct 2020-11-09 23:02:13 +00:00
vnode_if.src vfs: prevent avoidable evictions on mkdir of existing directories 2020-10-22 19:28:12 +00:00