d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
c2705ceaeb
freebsd-dev/sys/kern
History
Colin Percival c2705ceaeb x86: Speed up clock calibration 
Prior to this commit, the TSC and local APIC frequencies were calibrated
at boot time by measuring the clocks before and after a one-second sleep.
This was simple and effective, but had the disadvantage of *requiring a
one-second sleep*.

Rather than making two clock measurements (before and after sleeping) we
now perform many measurements; and rather than simply subtracting the
starting count from the ending count, we calculate a best-fit regression
between the target clock and the reference clock (for which the current
best available timecounter is used). While we do this, we keep track
of an estimate of the uncertainty in the regression slope (aka. the ratio
of clock speeds), and stop measuring when we believe the uncertainty is
less than 1 PPM.

In order to avoid the risk of aliasing resulting from the data-gathering
loop synchronizing with (a multiple of) the frequency of the reference
clock, we add some additional spinning depending upon the iteration number.

For numerical stability and simplicity of implementation, we make use of
floating-point arithmetic for the statistical calculations.

On the author's Dell laptop, this reduces the time spent in calibration
from 2000 ms to 29 ms; on an EC2 c5.xlarge instance, it is reduced from
2000 ms to 2.5 ms.

Reviewed by:	bde (previous version), kib
MFC after:	1 month
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D33802
2022-01-12 12:34:07 -08:00
..
bus_if.m bus: Make BUS_TRANSLATE_RESOURCE behave more like other bus methods 2021-11-15 13:01:30 -05:00
clock_if.m
cpufreq_if.m
device_if.m
firmw.S
genassym.sh genassym.sh: Fix two minor issues found by shellcheck 2021-07-28 13:49:16 -06:00
genoffset.c
genoffset.sh genoffset.sh: Use 10 X's instead of 5 for pick mkdtemp implementations 2021-09-07 10:08:51 -06:00
imgact_aout.c vdso for ia32 on amd64 2021-12-06 20:46:49 +02:00
imgact_binmisc.c
imgact_elf32.c
imgact_elf64.c
imgact_elf.c imgact_elf: Disable the stack gap for now 2021-12-14 16:19:40 -05:00
imgact_shell.c
init_main.c start_init: use 'p' 2021-11-15 02:33:01 +02:00
init_sysent.c Regen 2021-12-09 02:49:10 +02:00
kern_acct.c kern: Remove CTLFLAG_NEEDGIANT from some sysctls. 2021-12-26 12:03:33 -05:00
kern_alq.c vfs: remove the unused thread argument from NDINIT* 2021-11-25 22:50:42 +00:00
kern_clock.c clock: Group the "clocks" SYSINIT with the function definition 2021-11-15 16:13:24 -05:00
kern_clocksource.c Reduce minimum idle hardclock rate from 2Hz to 1Hz. 2022-01-09 19:25:56 -05:00
kern_condvar.c Use sleepq_signal(SLEEPQ_DROP) in cv_signal(). 2021-07-09 20:57:58 -04:00
kern_conf.c Fix handling of D_GIANTOK 2021-06-07 16:45:50 -04:00
kern_cons.c sysbeep: Adjust interface to take a duration as a sbt 2021-11-03 16:03:51 -06:00
kern_context.c
kern_cpu.c Fix "set but not used" in kern_cpu.c 2021-12-05 15:33:04 -07:00
kern_cpuset.c Make CPU_SET macros compliant with other implementations 2021-12-30 12:20:32 +01:00
kern_ctf.c vfs: remove the unused thread argument from NDINIT* 2021-11-25 22:50:42 +00:00
kern_descrip.c fd: Avoid truncating output buffers for KERN_PROC_{CWD,FILEDESC} 2021-12-17 13:10:23 -05:00
kern_dtrace.c
kern_dump.c Allow minidumps to be performed on the live system 2021-11-19 15:05:53 -04:00
kern_environment.c kenv: allow listing of static kernel environments 2021-07-18 23:06:19 -05:00
kern_et.c
kern_event.c struct kevent_freebsd11 -> struct freebsd11_kevent 2021-11-15 18:34:27 +00:00
kern_exec.c exec: Simplify sv_copyout_strings implementations a bit 2021-12-31 12:50:15 -05:00
kern_exit.c syscalls: normalize exit 2021-11-29 22:04:43 +00:00
kern_fail.c
kern_ffclock.c
kern_fork.c Move fork_rfppwait() check into ast() 2021-12-26 17:22:21 +00:00
kern_hhook.c
kern_idle.c
kern_intr.c Simplify swi for bus_dma. 2021-12-28 13:51:25 -08:00
kern_jail.c jail: network epoch protection for IP address lists 2021-12-26 10:45:50 -08:00
kern_kcov.c kcov: Disable address and memory sanitizers in get_kinfo() 2021-09-17 14:19:05 -04:00
kern_khelp.c
kern_kthread.c umtx: Split umtx.h on two counterparts. 2021-07-29 12:41:29 +03:00
kern_ktr.c
kern_ktrace.c vfs: remove the unused thread argument from NDINIT* 2021-11-25 22:50:42 +00:00
kern_linker.c kernel linker: do not read debug symbol tables for non-debug symbols 2021-12-08 23:32:29 +02:00
kern_lock.c lockmgr: fix lock profiling of face adaptive spinning 2021-09-18 10:16:58 +00:00
kern_lockf.c
kern_lockstat.c
kern_loginclass.c
kern_malloc.c malloc: Unmark KASAN redzones if the full allocation size was requested 2021-10-06 16:09:41 -04:00
kern_mbuf.c kern: Remove CTLFLAG_NEEDGIANT from some more sysctls. 2021-12-26 23:07:33 -05:00
kern_mib.c
kern_module.c modules: increase MAXMODNAME and provide backward compat 2021-12-09 18:09:53 +00:00
kern_mtxpool.c
kern_mutex.c callout(9): Allow spin locks use with callout_init_mtx(). 2021-09-02 21:16:46 -04:00
kern_ntptime.c
kern_osd.c osd: Fix racy assertions 2021-09-09 10:11:02 -04:00
kern_physio.c Minor style tidy: if( -> if ( 2021-04-18 11:19:15 -06:00
kern_pmc.c
kern_poll.c
kern_priv.c
kern_proc.c amd64: wrap 64bit sigtramp into vdso 2021-12-06 20:46:49 +02:00
kern_procctl.c procctl: stop using SA_*LOCKED, define local enum 2021-10-20 00:25:19 +03:00
kern_prot.c kern: fail getgroup and setgroup with negative int 2021-06-02 13:22:57 -06:00
kern_racct.c umtx: Split umtx.h on two counterparts. 2021-07-29 12:41:29 +03:00
kern_rangelock.c
kern_rctl.c rctl(4): support throttling resource usage to 0 2021-01-11 15:36:57 -08:00
kern_resource.c Add idle priority scheduling privilege group to MAC/priority 2021-12-10 04:54:48 +02:00
kern_rmlock.c Restore variable aliasing in the context of cpu set operations 2022-01-01 11:58:40 +01:00
kern_rwlock.c lockprof: pass lock type as an argument instead of reading the spin flag 2021-05-23 17:55:27 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c kern: Remove CTLFLAG_NEEDGIANT from some sysctls. 2021-12-26 12:03:33 -05:00
kern_sharedpage.c Fix assert check for SV_DSO_SIG in exec_sysvec_init_secondary() 2021-12-08 22:54:07 -06:00
kern_shutdown.c boot(9): update to match reality 2021-09-28 11:36:09 -03:00
kern_sig.c Ignore debugger-injected signals left after detaching 2022-01-12 07:33:30 +02:00
kern_switch.c kern: Remove CTLFLAG_NEEDGIANT from some more sysctls. 2021-12-26 23:07:33 -05:00
kern_sx.c lockprof: pass lock type as an argument instead of reading the spin flag 2021-05-23 17:55:27 +00:00
kern_synch.c Add sched_getcpu() 2021-11-10 21:18:54 +02:00
kern_syscalls.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
kern_sysctl.c sys___sysctl: regularize argument struct 2021-11-22 22:36:54 +00:00
kern_tc.c kern_tc: unify timecounter to bintime delta conversion 2021-11-30 15:23:23 +02:00
kern_thr.c umtx: Split umtx.h on two counterparts. 2021-07-29 12:41:29 +03:00
kern_thread.c i386: fix struct proc layout asserts after 351d5f7fc5 2021-10-28 21:56:21 +03:00
kern_time.c itimers: strip unused bits from struct itimer and struct itimers 2021-12-28 03:02:53 +02:00
kern_timeout.c Reduce minimum idle hardclock rate from 2Hz to 1Hz. 2022-01-09 19:25:56 -05:00
kern_tslog.c TSLOG: Report final execname, not first 2021-10-17 13:36:38 -07:00
kern_ubsan.c ubsan: Fix a typo in an error message 2021-09-25 11:47:24 +02:00
kern_umtx.c freebsd32: add _'s to _umtx_(un)lock 2021-11-22 22:36:55 +00:00
kern_uuid.c
kern_xxx.c
ksched.c
link_elf_obj.c Kernel linkers: add emergency sysctl to restore old behavior 2021-12-08 23:32:30 +02:00
link_elf.c Kernel linkers: add emergency sysctl to restore old behavior 2021-12-08 23:32:30 +02:00
linker_if.m kernel linker: do not read debug symbol tables for non-debug symbols 2021-12-08 23:32:29 +02:00
Make.tags.inc Bring the tags and links entries for amd64 up to date. 2021-09-27 20:04:51 -07:00
Makefile syscalls.master: switch to CAPENABLED flags 2021-09-01 21:58:16 +01:00
md4c.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
md5c.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
msi_if.m Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights 2021-08-08 10:42:24 -04:00
p1003_1b.c
pic_if.m
posix4_mib.c
sched_4bsd.c sys/kern/sched_4bsd.c: fix typo introduced in previous commit 2022-01-01 15:33:38 +01:00
sched_ule.c sched_ule(4): Fix two typo in source code comments 2021-11-19 19:13:28 +01:00
serdev_if.m
stack_protector.c
subr_acl_nfs4.c
subr_acl_posix1e.c
subr_asan.c amd64: Deduplicate routines for expanding KASAN/KMSAN shadow maps 2021-11-03 12:36:02 -04:00
subr_atomic64.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
subr_autoconf.c Add run_interrupt_driven_config_hooks to TSLOG 2021-09-05 12:45:29 -07:00
subr_blist.c blist: Correct the node count computed in blist_create() 2021-07-13 17:47:27 -04:00
subr_boot.c
subr_bufring.c
subr_bus_dma.c bus_dma: Deduplicate locking helper functions. 2022-01-05 13:50:40 -08:00
subr_bus.c kern: Remove CTLFLAG_NEEDGIANT from some more sysctls. 2021-12-26 23:07:33 -05:00
subr_busdma_bounce.c Deduplicate bus_dma bounce code. 2022-01-05 13:50:40 -08:00
subr_busdma_bufalloc.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
subr_capability.c Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights 2021-08-08 10:42:24 -04:00
subr_clock.c
subr_clockcalib.c x86: Speed up clock calibration 2022-01-12 12:34:07 -08:00
subr_compressor.c
subr_counter.c
subr_coverage.c kcov: Integrate with KMSAN 2021-09-14 14:29:27 -04:00
subr_csan.c Simplify kernel sanitizer interceptors 2021-07-29 21:13:32 -04:00
subr_devmap.c riscv: Implement pmap_mapdev_attr 2021-10-17 15:31:35 +01:00
subr_devstat.c Speed up geom_stats_resync in the presence of many devices 2021-03-02 18:33:45 -07:00
subr_disk.c
subr_dummy_vdso_tc.c
subr_early.c
subr_epoch.c Revert "wpa: Import wpa_supplicant/hostapd commit 14ab4a816" 2021-12-02 14:45:04 -08:00
subr_eventhandler.c
subr_fattime.c
subr_filter.c
subr_firmware.c subr_firmware.c::unloadentry(): remote write-only variable 2021-10-21 21:40:46 +03:00
subr_gtaskqueue.c
subr_hash.c
subr_hints.c Optimize res_find(). 2021-08-08 21:54:49 -04:00
subr_intr.c intrng: Use less confusing return value for intr_pic_add_handler 2022-01-03 17:08:44 +00:00
subr_kdb.c Restore variable aliasing in the context of cpu set operations 2022-01-01 11:58:40 +01:00
subr_kobj.c kobj: plug set-but-not-used vars 2021-12-14 14:52:25 +00:00
subr_lock.c lockprof: move panic check after inspecting the state 2021-05-23 17:55:27 +00:00
subr_log.c Document kern.log_wakeups_per_second. 2021-08-04 11:50:34 -07:00
subr_mchain.c
subr_module.c
subr_msan.c amd64: Deduplicate routines for expanding KASAN/KMSAN shadow maps 2021-11-03 12:36:02 -04:00
subr_msgbuf.c Make msgbuf_peekbytes() not return leading zeroes. 2021-12-10 23:35:53 -05:00
subr_param.c Move mips and arm to 1000Hz by default. 2021-06-16 20:00:14 -06:00
subr_pcpu.c
subr_pctrie.c
subr_physmem.c kern: physmem: improve region coalescing logic 2021-11-03 02:32:46 -05:00
subr_pidctrl.c
subr_power.c
subr_prf.c Unify console output. 2021-09-03 23:13:42 -04:00
subr_prng.c
subr_prof.c x86: remove gcov kernel support 2021-04-02 15:41:51 +03:00
subr_rangeset.c Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights 2021-08-08 10:42:24 -04:00
subr_rman.c rman: fix overflow in rman_reserve_resource_bound() 2021-09-27 14:38:26 -03:00
subr_rtc.c
subr_sbuf.c sbuf(9): Microoptimize sbuf_put_byte() 2021-10-05 14:47:38 -04:00
subr_scanf.c
subr_sfbuf.c
subr_sglist.c sglist: Add sglist_append_single_mbuf(). 2021-05-25 16:59:18 -07:00
subr_sleepqueue.c When queuing ignored signal, only abort target thread' sleep if it is inside sigwait() 2021-10-06 17:05:22 +03:00
subr_smp.c Make CPU_SET macros compliant with other implementations 2021-12-30 12:20:32 +01:00
subr_smr.c
subr_stack.c
subr_stats.c stats(3): Improve t-digest merging of samples which result in mu adjustment underflow. 2021-04-02 13:17:53 +11:00
subr_syscall.c Move fork_rfppwait() check into ast() 2021-12-26 17:22:21 +00:00
subr_taskqueue.c callout(9): Allow spin locks use with callout_init_mtx(). 2021-09-02 21:16:46 -04:00
subr_terminal.c Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights 2021-08-08 10:42:24 -04:00
subr_trap.c Move fork_rfppwait() check into ast() 2021-12-26 17:22:21 +00:00
subr_turnstile.c Use atomic loads/stores when updating td->td_state 2021-02-18 14:02:48 +00:00
subr_uio.c
subr_unit.c
subr_vmem.c vmem: disable debug.vmem_check by default 2021-09-02 18:28:45 +00:00
subr_witness.c Revert "wpa: Import wpa_supplicant/hostapd commit 14ab4a816" 2021-12-02 14:45:04 -08:00
sys_capability.c
sys_eventfd.c vfs: remove thread argument from VOP_STAT 2021-10-11 13:22:32 +00:00
sys_generic.c selsocket: handle sopoll() errors correctly 2021-10-14 00:43:48 +01:00
sys_getrandom.c
sys_pipe.c fifos: delegate unhandled kqueue filters to underlying filesystem 2021-10-12 02:43:07 -05:00
sys_procdesc.c vfs: remove thread argument from VOP_STAT 2021-10-11 13:22:32 +00:00
sys_process.c Remove PT_GET_SC_ARGS_ALL 2021-09-16 20:11:27 +03:00
sys_socket.c socket: plug set-but-not-used vars 2021-12-14 17:12:53 +00:00
syscalls.c Regen 2021-12-09 02:49:10 +02:00
syscalls.master swapoff: add one more variant of the syscall 2021-12-09 02:48:46 +02:00
systrace_args.c Regen 2021-12-09 02:49:10 +02:00
sysv_ipc.c freebsd32: rename old SysV IPC types 2021-11-17 20:12:23 +00:00
sysv_msg.c freebsd32: rename old SysV IPC types 2021-11-17 20:12:23 +00:00
sysv_sem.c freebsd32: include __ in semctl names 2021-11-17 20:12:24 +00:00
sysv_shm.c freebsd32: rename old SysV IPC types 2021-11-17 20:12:23 +00:00
tty_compat.c
tty_info.c
tty_inq.c
tty_outq.c Minor style cleanup 2021-04-18 11:14:17 -06:00
tty_pts.c vfs: remove thread argument from VOP_STAT 2021-10-11 13:22:32 +00:00
tty_tty.c
tty_ttydisc.c
tty.c Re-implement virtual console (constty). 2021-09-03 22:18:51 -04:00
uipc_accf.c Consistently use the SOLISTENING() macro 2021-06-14 17:32:27 -04:00
uipc_debug.c domains: init with standard SYSINIT(9) or VNET_SYSINIT() 2022-01-03 10:15:22 -08:00
uipc_domain.c domains: make domain_init() initialize only global state 2022-01-03 10:15:22 -08:00
uipc_ktls.c ktls: Support for TLS 1.3 receive offload. 2021-12-14 11:01:05 -08:00
uipc_mbuf2.c mbuf_tags: use explicitly sized type for 'type' parameter 2021-12-29 09:23:52 +01:00
uipc_mbuf.c kern: Remove CTLFLAG_NEEDGIANT from some more sysctls. 2021-12-26 23:07:33 -05:00
uipc_mbufhash.c
uipc_mqueue.c Change VOP_READDIR's cookies argument to a **uint64_t 2021-12-15 20:54:57 -07:00
uipc_sem.c freebsd32: semid_t -> int32_t 2021-11-17 20:12:26 +00:00
uipc_shm.c uipc_shm: silent warnings about write-only variables in largepage code 2021-10-21 21:40:46 +03:00
uipc_sockbuf.c kern: Remove CTLFLAG_NEEDGIANT from some sysctls. 2021-12-26 12:03:33 -05:00
uipc_socket.c kern: Remove CTLFLAG_NEEDGIANT from some sysctls. 2021-12-26 12:03:33 -05:00
uipc_syscalls.c syscalls: normalize orecvfrom and ogetsockname 2021-11-29 22:04:42 +00:00
uipc_usrreq.c domains: init with standard SYSINIT(9) or VNET_SYSINIT() 2022-01-03 10:15:22 -08:00
vfs_acl.c vfs: remove the unused thread argument from NDINIT* 2021-11-25 22:50:42 +00:00
vfs_aio.c aio: whack "set but not used" warnings 2021-11-14 16:59:53 +00:00
vfs_bio.c Expand comment explaining reasons for automatic swapoff on shutdown 2021-12-03 10:42:21 +02:00
vfs_cache.c vfs: remove the unused thread argument from NDINIT* 2021-11-25 22:50:42 +00:00
vfs_cluster.c cluster_write(): do not access buffer after it is released 2021-09-02 21:36:33 +03:00
vfs_default.c vfs: fully lockless v_writecount adjustment 2021-11-27 23:07:26 +00:00
vfs_export.c
vfs_extattr.c vfs: remove the unused thread argument from NDINIT* 2021-11-25 22:50:42 +00:00
vfs_hash.c vfs_hash_rehash(): require the vnode to be exclusively locked 2021-08-27 18:39:45 +03:00
vfs_init.c VFS_QUOTACTL(9): allow implementation to indicate busy state changes 2021-05-30 14:53:47 -07:00
vfs_lookup.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
vfs_mount.c Add vfs_remount_ro() 2022-01-08 05:41:44 +02:00
vfs_mountroot.c vfs_mountroot: Check for root dev before waiting 2021-12-29 12:41:09 -08:00
vfs_subr.c Change VOP_READDIR's cookies argument to a **uint64_t 2021-12-15 20:54:57 -07:00
vfs_syscalls.c sys/kern: Use C99 fixed-width integer types. 2021-12-28 09:41:08 -08:00
vfs_vnops.c vm: Add a mode to vm_object_page_remove() which skips invalid pages 2021-11-15 13:01:30 -05:00
vnode_if.src Change VOP_READDIR's cookies argument to a **uint64_t 2021-12-15 20:54:57 -07:00