d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
45606ae1d5
freebsd-dev/sys/kern
History
John Baldwin fdce57a042 Add an EARLY_AP_STARTUP option to start APs earlier during boot. 
Currently, Application Processors (non-boot CPUs) are started by
MD code at SI_SUB_CPU, but they are kept waiting in a "pen" until
SI_SUB_SMP at which point they are released to run kernel threads.
SI_SUB_SMP is one of the last SYSINIT levels, so APs don't enter
the scheduler and start running threads until fairly late in the
boot.

This change moves SI_SUB_SMP up to just before software interrupt
threads are created allowing the APs to start executing kernel
threads much sooner (before any devices are probed).  This allows
several initialization routines that need to perform initialization
on all CPUs to now perform that initialization in one step rather
than having to defer the AP initialization to a second SYSINIT run
at SI_SUB_SMP.  It also permits all CPUs to be available for
handling interrupts before any devices are probed.

This last feature fixes a problem on with interrupt vector exhaustion.
Specifically, in the old model all device interrupts were routed
onto the boot CPU during boot.  Later after the APs were released at
SI_SUB_SMP, interrupts were redistributed across all CPUs.

However, several drivers for multiqueue hardware allocate N interrupts
per CPU in the system.  In a system with many CPUs, just a few drivers
doing this could exhaust the available pool of interrupt vectors on
the boot CPU as each driver was allocating N * mp_ncpu vectors on the
boot CPU.  Now, drivers will allocate interrupts on their desired CPUs
during boot meaning that only N interrupts are allocated from the boot
CPU instead of N * mp_ncpu.

Some other bits of code can also be simplified as smp_started is
now true much earlier and will now always be true for these bits of
code.  This removes the need to treat the single-CPU boot environment
as a special case.

As a transition aid, the new behavior is available under a new kernel
option (EARLY_AP_STARTUP).  This will allow the option to be turned off
if need be during initial testing.  I plan to enable this on x86 by
default in a followup commit in the next few days and to have all
platforms moved over before 11.0.  Once the transition is complete,
the option will be removed along with the !EARLY_AP_STARTUP code.

These changes have only been tested on x86.  Other platform maintainers
are encouraged to port their architectures over as well.  The main
things to check for are any uses of smp_started in MD code that can be
simplified and SI_SUB_SMP SYSINITs in MD code that can be removed in
the EARLY_AP_STARTUP case (e.g. the interrupt shuffling).

PR:		kern/199321
Reviewed by:	markj, gnn, kib
Sponsored by:	Netflix
2016-05-14 18:22:52 +00:00
..
bus_if.m Add a new bus method to fetch device-specific CPU sets. 2016-05-09 20:50:21 +00:00
capabilities.conf Add futimens and utimensat system calls. 2015-01-23 21:07:08 +00:00
clock_if.m
cpufreq_if.m
device_if.m Change the default method for device_quiesce() to return 0 instead of 2015-01-08 21:46:28 +00:00
genassym.sh genassym.sh: call nm(1) with NMFLAGS. 2015-08-14 22:57:13 +00:00
imgact_aout.c Implement vsyscall hack. Prior to 2.13 glibc uses vsyscall 2016-01-09 20:18:53 +00:00
imgact_binmisc.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c sys: use our roundup2/rounddown2() macros when param.h is available. 2016-04-21 19:57:40 +00:00
imgact_gzip.c Implement lockless resource limits. 2015-06-10 10:48:12 +00:00
imgact_shell.c
inflate.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
init_main.c sys: use our roundup2/rounddown2() macros when param.h is available. 2016-04-21 19:57:40 +00:00
init_sysent.c Regen. 2016-03-21 21:38:35 +00:00
kern_acct.c
kern_alq.c Use SI_SUB_LAST instead of SI_SUB_SMP as the "catch-all" subsystem. 2016-03-11 23:18:06 +00:00
kern_clock.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_clocksource.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_condvar.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_conf.c Provide yet another KPI for cdev creation, make_dev_s(9). 2016-01-07 20:08:02 +00:00
kern_cons.c CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten 2015-05-22 17:05:21 +00:00
kern_context.c
kern_cpu.c Add an EARLY_AP_STARTUP option to start APs earlier during boot. 2016-05-14 18:22:52 +00:00
kern_cpuset.c Add more fine-grained kernel options for NUMA support. 2016-04-09 13:58:04 +00:00
kern_ctf.c Don't specify a resid parameter if we're just going to ignore it. Instead, 2015-02-20 20:49:00 +00:00
kern_descrip.c fd: assert dropped filedesc lock in fdcloseexec 2016-05-08 03:26:12 +00:00
kern_dtrace.c
kern_dump.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_environment.c Create wrappers for uint64_t and int64_t for the tunables. While not 2016-04-15 03:09:55 +00:00
kern_et.c Trivial change / forced-commit to document prior change that slipped in 2015-03-16 19:29:19 +00:00
kern_event.c Provide high precision conversion from ns,us,ms -> sbintime in kevent 2016-03-12 23:02:53 +00:00
kern_exec.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_exit.c session: avoid proctree lock on proc exit when possible 2016-01-20 23:33:58 +00:00
kern_fail.c fail(9): Only gather/print stacks if STACK is enabled 2016-03-17 01:05:53 +00:00
kern_ffclock.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_fork.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_gzio.c Move zlib.c from net to libkern. 2015-04-22 14:38:58 +00:00
kern_hhook.c Check that hhk_helper pointer isn't NULL before access. 2015-11-25 07:14:58 +00:00
kern_idle.c
kern_intr.c The part of r285680 which removed release semantic for two stores to 2015-07-21 14:39:34 +00:00
kern_jail.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_khelp.c
kern_kthread.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
kern_ktr.c Fix the logic in the ddb command 'show ktr /a'. Prior to r118269 it would 2016-01-31 17:32:20 +00:00
kern_ktrace.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
kern_linker.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_lock.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_lockf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_lockstat.c Consistently use a reader/writer flag for lockstat probes in rwlock(9) and 2015-07-19 22:24:33 +00:00
kern_loginclass.c Speed up rctl operation with large rulesets, by holding the lock 2015-11-15 12:10:51 +00:00
kern_malloc.c Include sys/_task.h into uma_int.h, so that taskqueue.h isn't a 2016-02-09 20:22:35 +00:00
kern_mbuf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_mib.c Fix build after r295934. 2016-02-23 23:37:10 +00:00
kern_module.c Provide better debug message on kernel module name clash. 2015-10-10 09:21:55 +00:00
kern_mtxpool.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_mutex.c Don't modify curthread->td_locks unless INVARIANTS is enabled. 2015-08-02 00:03:08 +00:00
kern_ntptime.c Use the monotonic (uptime) counter rather than time-of-day to measure elapsed 2015-07-12 18:38:17 +00:00
kern_numa.c Add an initial NUMA affinity/policy configuration for threads and processes. 2015-07-11 15:21:37 +00:00
kern_osd.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
kern_physio.c Add four new RCTL resources - readbps, readiops, writebps and writeiops, 2016-04-07 04:23:25 +00:00
kern_pmc.c
kern_poll.c When a kernel has DEVICE_POLLING turned on but no drivers have 2015-04-14 14:22:34 +00:00
kern_priv.c
kern_proc.c Use a loop instead of a goto in sysctl_kern_proc_kstack(). 2016-04-17 23:22:32 +00:00
kern_procctl.c If process becomes reaper (procctl(PROC_REAP_ACQUIRE)) while already 2015-08-20 22:44:26 +00:00
kern_prot.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_racct.c Get rid of rctl_lock; use racct_lock where appropriate. The fast paths 2016-04-21 16:22:52 +00:00
kern_rangelock.c
kern_rctl.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_resource.c Implement process-shared locks support for libthr.so.3, without 2016-02-28 17:52:33 +00:00
kern_rmlock.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_rwlock.c Don't modify curthread->td_locks unless INVARIANTS is enabled. 2015-08-02 00:03:08 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c The sendfile(2) allows to send extra data from userspace before the file 2016-03-29 19:57:11 +00:00
kern_sharedpage.c Split kerne timekeep ABI structure vdso_sv_tk out of the struct 2015-11-23 07:09:35 +00:00
kern_shutdown.c Add 4Kn kernel dump support 2016-04-15 17:45:12 +00:00
kern_sig.c Add ptrace(2) reporting for LWP events. 2015-12-29 23:25:26 +00:00
kern_switch.c Revert for r277213: 2015-01-22 11:12:42 +00:00
kern_sx.c Don't modify curthread->td_locks unless INVARIANTS is enabled. 2015-08-02 00:03:08 +00:00
kern_synch.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
kern_syscalls.c Implement lockless resource limits. 2015-06-10 10:48:12 +00:00
kern_sysctl.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_tc.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_thr.c Add ptrace(2) reporting for LWP events. 2015-12-29 23:25:26 +00:00
kern_thread.c Call kern_thr_exit() instead of duplicating it. 2015-12-29 23:16:20 +00:00
kern_time.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
kern_timeout.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
kern_umtx.c Fix umtx lock/trylock for compat32. 2016-04-19 11:37:43 +00:00
kern_uuid.c
kern_xxx.c
ksched.c Use P1B_PRIO_MAX to designate max posix priority for the RR/FIFO 2015-08-30 18:02:57 +00:00
link_elf_obj.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
link_elf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
linker_if.m sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
Make.tags.inc Bring the tags and links entries for amd64 up to date. 2015-10-27 22:59:24 +00:00
Makefile
makesyscalls.sh Do not include system call wrappers in libc for old FreeBSD system calls. 2016-03-12 22:53:46 +00:00
md4c.c
md5c.c
p1003_1b.c In preparation for switching linuxulator to the use the native 1:1 2015-05-24 14:44:06 +00:00
pic_if.m Remove FDT specific parts from INTRNG. Change its interface to make it 2016-04-04 09:15:25 +00:00
posix4_mib.c posix4_mib: Don't overrun facility_initialized array 2016-04-27 00:10:32 +00:00
sched_4bsd.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sched_ule.c The struct thread td_estcpu member is only used by the 4BSD scheduler. 2016-04-17 11:04:27 +00:00
serdev_if.m
stack_protector.c Use nitems() macro instead of __arraycount() 2015-06-16 20:19:00 +00:00
subr_acl_nfs4.c Expose an interface to determine if an ACE is inherited. 2015-09-04 00:14:20 +00:00
subr_acl_posix1e.c
subr_autoconf.c
subr_blist.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_bufring.c
subr_bus_dma.c Fix a bug introduced in r291716: 2016-01-11 20:38:39 +00:00
subr_bus.c Add a new bus method to fetch device-specific CPU sets. 2016-05-09 20:50:21 +00:00
subr_busdma_bufalloc.c Fix printf format to allow for bus_size_t not being u_long on all platforms. 2015-10-20 03:25:17 +00:00
subr_capability.c capsicum: plug spurious memset in __cap_rights_init 2015-12-01 02:48:42 +00:00
subr_clock.c For architectures where time_t is wide enough, in particular, 64bit 2014-12-12 09:37:18 +00:00
subr_counter.c Free the temporary buffer in sysctl_handle_counter_u64_array(). 2016-03-15 00:21:32 +00:00
subr_devmap.c Add support for RISC-V. 2016-04-26 12:29:47 +00:00
subr_devstat.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_disk.c
subr_dummy_vdso_tc.c Update the vdso timehands only via tc_windup(). 2015-01-20 03:54:30 +00:00
subr_eventhandler.c
subr_fattime.c Where appropriate, use the modern terms for the one true time base 2014-12-21 05:07:11 +00:00
subr_firmware.c Create a dedicated function for ensuring that cdir and rdir are populated. 2015-07-11 16:22:48 +00:00
subr_hash.c kern: Factor out function to convert hash flags to malloc(9) flags 2016-05-04 03:07:52 +00:00
subr_hints.c Add a new device control utility for new-bus devices called devctl. This 2015-02-06 16:09:01 +00:00
subr_intr.c Check malloc succeeded in pic_create, with M_NOWAIT it may return NULL. 2016-05-09 12:24:39 +00:00
subr_kdb.c Fix multiple incorrect SYSCTL arguments in the kernel: 2014-10-21 07:31:21 +00:00
subr_kobj.c
subr_lock.c Add _NEW flag to mtx(9), sx(9), rmlock(9) and rwlock(9). 2014-12-13 21:00:10 +00:00
subr_log.c
subr_mbpool.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_mchain.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_module.c preload_search_info: make sure mod is set 2015-08-21 15:57:57 +00:00
subr_msgbuf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_param.c Ensure that maxproc does not exceed pid_max, at the time of boot. 2015-09-21 15:02:59 +00:00
subr_pcpu.c
subr_pctrie.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
subr_power.c
subr_prf.c Add vlog(9). 2015-11-19 05:50:22 +00:00
subr_prof.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_rman.c s/struct device */device_t/g 2016-05-04 23:31:52 +00:00
subr_rtc.c rtc: fix inverted resolution check 2016-05-04 13:48:59 +00:00
subr_sbuf.c Fail the sbuf if vsnprintf(3) fails. 2015-10-02 09:23:14 +00:00
subr_scanf.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_sfbuf.c
subr_sglist.c Fix a couple of panics when detaching from a cxgbe/cxl interface that was 2015-01-26 16:26:28 +00:00
subr_sleepqueue.c Rework handling of thread sleeps before timers are working. 2016-03-31 18:10:29 +00:00
subr_smp.c topo_set_pu_id: turn a check into an assertion 2016-04-08 11:59:11 +00:00
subr_stack.c Add support for a configurable output channel to witness(4). 2015-11-19 05:56:59 +00:00
subr_syscall.c Support an arbitrary number of arguments to DTrace syscall probes. 2015-12-17 00:00:27 +00:00
subr_taskqueue.c Remove taskqueue_enqueue_fast(). 2016-03-01 17:47:32 +00:00
subr_terminal.c vt(4): Adjust the cursor position after changing the window size 2014-11-01 17:05:15 +00:00
subr_trap.c Add four new RCTL resources - readbps, readiops, writebps and writeiops, 2016-04-07 04:23:25 +00:00
subr_turnstile.c Finish r173600. There is no need to test a condition if both cases 2016-02-10 21:16:37 +00:00
subr_uio.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
subr_unit.c Add explicit cast to fix mips and powerpc build after r299090 2016-05-05 15:21:33 +00:00
subr_vmem.c subr_vmem: Fix double-free in error case of vmem_create 2016-05-11 23:16:11 +00:00
subr_witness.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sys_capability.c Introduce kern_cap_rights_limit(). 2015-08-11 08:43:50 +00:00
sys_generic.c Cap IOSIZE_MAX to INT_MAX for 32-bit processes. 2016-04-01 18:29:38 +00:00
sys_pipe.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
sys_procdesc.c The si_status field of the siginfo_t, provided by the waitid(2) and 2015-07-18 09:02:50 +00:00
sys_process.c Add ptrace(2) reporting for LWP events. 2015-12-29 23:25:26 +00:00
sys_socket.c Expose soaio_enqueue(). 2016-04-29 20:12:45 +00:00
syscalls.c Regen. 2016-03-21 21:38:35 +00:00
syscalls.master Fully handle size_t lengths in AIO requests. 2016-03-21 21:37:33 +00:00
systrace_args.c Regen. 2016-03-21 21:38:35 +00:00
sysv_ipc.c
sysv_msg.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
sysv_sem.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
sysv_shm.c osd(9): Change array pointer to array pointer type from void* 2016-04-26 19:57:35 +00:00
tty_compat.c
tty_info.c
tty_inq.c
tty_outq.c
tty_pts.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
tty_tty.c tty: replace several curthread->td_proc with stored curproc 2015-07-06 18:53:56 +00:00
tty_ttydisc.c Don't clear the software flow control flag before draining for last 2016-01-26 14:46:39 +00:00
tty.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_accf.c
uipc_debug.c Refactor the AIO subsystem to permit file-type-specific handling and 2016-03-01 18:12:14 +00:00
uipc_domain.c Introduce a new protocol hook pru_aio_queue. 2016-04-29 20:11:09 +00:00
uipc_mbuf2.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_mbuf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_mbufhash.c Reduce header pollution. 2015-03-17 14:16:50 +00:00
uipc_mqueue.c Use the new PR_METHOD_REMOVE to clean up jail handling in POSIX 2016-04-25 04:36:54 +00:00
uipc_sem.c Clean up some style(9) violations. 2016-04-14 17:07:26 +00:00
uipc_shm.c sys: extend use of the howmany() macro when available. 2016-04-26 15:38:17 +00:00
uipc_sockbuf.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_socket.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
uipc_syscalls.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
uipc_usrreq.c kernel: use our nitems() macro when it is available through param.h. 2016-04-19 23:48:27 +00:00
vfs_acl.c Replace struct filedesc argument in getvnode with struct thread 2015-06-16 13:09:18 +00:00
vfs_aio.c Tidy up the unmapped I/O code in qphysio. 2016-03-31 17:27:30 +00:00
vfs_bio.c sys/kern: spelling fixes. 2016-04-29 21:54:28 +00:00
vfs_cache.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_cluster.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_default.c Simplify AIO initialization now that it is standard. 2016-03-09 19:05:11 +00:00
vfs_export.c kern: for pointers replace 0 with NULL. 2016-04-15 16:10:11 +00:00
vfs_extattr.c Replace struct filedesc argument in getvnode with struct thread 2015-06-16 13:09:18 +00:00
vfs_hash.c Add vfs_hash_ref(9) function, which finds a vnode by the hash value 2016-05-11 06:32:22 +00:00
vfs_init.c sysctl: switch sysctllock to a sleepable rmlock, take 2 2015-09-15 23:06:56 +00:00
vfs_lookup.c sys/kern: spelling fixes in comments. 2016-04-29 22:15:33 +00:00
vfs_mount.c Do not copy by field when converting struct oexport_args to struct 2016-02-04 16:32:21 +00:00
vfs_mountroot.c Stop hiding errors that result in failure to mount /dev. Otherwise, 2016-05-12 07:38:10 +00:00
vfs_subr.c Add EVFILT_VNODE open, read and close notifications. 2016-05-03 15:17:43 +00:00
vfs_syscalls.c Simplify AIO initialization now that it is standard. 2016-03-09 19:05:11 +00:00
vfs_vnops.c sys/kern: spelling fixes. 2016-04-29 21:54:28 +00:00
vnode_if.src Add EVFILT_VNODE open, read and close notifications. 2016-05-03 15:17:43 +00:00