freebsd-dev/sys/kern
Konstantin Belousov 6cf2362e2c Consolidate read code for timecounters and fix possible overflow in
bintime()/binuptime().

The algorithm to read the consistent snapshot of current timehand is
repeated in each accessor, including the details proper rollup
detection and synchronization with the writer.  In fact there are only
two different kind of readers: one for bintime()/binuptime() which has
to do the in-place calculation, and another kind which fetches some
member from struct timehand.

Extract the logic into type-checked macros, GETTHBINTIME() for bintime
calculation, and GETTHMEMBER() for safe read of a structure' member.
This way, the synchronization is only written in bintime_off() and
getthmember().

In bintime_off(), use overflow-safe calculation of th_scale *
delta(timecounter).  In tc_windup, pre-calculate the min delta value
which overflows and require slow algorithm, into the new timehands
th_large_delta member.

This part with overflow fix was written by Bruce Evans.

Reported by:	Mark Millard <marklmi@yahoo.com> (the overflow issue)
Tested by:	pho
Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation (kib)
MFC after:	3 weeks
2020-02-14 23:27:45 +00:00
..
bus_if.m Add necessary bits for Linux KPI to work correctly on powerpc 2019-08-04 19:28:10 +00:00
capabilities.conf Allow getloginclass in capability mode 2020-02-12 18:59:00 +00:00
clock_if.m
cpufreq_if.m
device_if.m
genassym.sh
genoffset.c Merge td_epochnest with td_no_sleeping. 2019-10-29 17:28:25 +00:00
genoffset.sh expose thread_lite definition to tied modules 2018-07-03 02:50:07 +00:00
imgact_aout.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
imgact_binmisc.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c Add AT_BSDFLAGS auxv entry. 2020-02-09 12:10:37 +00:00
imgact_shell.c Add helper functions to copy strings into struct image_args. 2018-11-29 21:00:56 +00:00
init_main.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
init_sysent.c regen sysent after r357831, r357838 2020-02-12 19:05:10 +00:00
kern_acct.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
kern_alq.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_clock.c sleep(9), sleepqueue(9): const'ify wchan pointers 2019-12-24 16:19:33 +00:00
kern_clocksource.c Rename hardclock_cnt() to hardclock() and remove the old implementation. 2018-09-06 02:10:59 +00:00
kern_condvar.c
kern_conf.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_cons.c kern_cons: add a stub kbdinit for configs with no keyboard/console drivers 2019-12-26 15:47:19 +00:00
kern_context.c Remove superfluous bzero in getcontext/swapcontext/sendsig 2018-11-26 20:56:05 +00:00
kern_cpu.c hwpstate_intel(4): Silence/fix Coverity reports 2020-01-29 03:15:34 +00:00
kern_cpuset.c bitset: rename confusing macro NAND to ANDNOT 2019-12-13 09:32:16 +00:00
kern_ctf.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
kern_descrip.c fd: remove no longer needed atomic_load_ptr casts 2020-02-14 23:18:22 +00:00
kern_dtrace.c
kern_dump.c [PPC64] Initial kernel minidump implementation 2019-10-14 13:04:04 +00:00
kern_environment.c Follow up r352244: kenv: tighten up assertions 2019-09-12 14:34:46 +00:00
kern_et.c
kern_event.c Avoid relying on header pollution from sys/refcount.h. 2019-07-29 20:26:01 +00:00
kern_exec.c Add a way to manage thread signal mask using shared word, instead of syscall. 2020-02-09 11:53:12 +00:00
kern_exit.c Remove sparc64 kernel support 2020-02-03 17:35:11 +00:00
kern_fail.c Use callout(9) instead of deprecated timeout(9) for fail points. 2019-12-13 19:26:04 +00:00
kern_ffclock.c
kern_fork.c Add a way to manage thread signal mask using shared word, instead of syscall. 2020-02-09 11:53:12 +00:00
kern_hhook.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_idle.c
kern_intr.c Fix for unbalanced EPOCH(9) usage in the generic kernel interrupt 2020-02-10 20:23:08 +00:00
kern_jail.c Partially decompose priv_check by adding priv_check_cred_vfs_generation 2020-02-13 22:22:15 +00:00
kern_kcov.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_khelp.c
kern_kthread.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_ktr.c Add KERNEL_PANICKED macro for use in place of direct panicstr tests 2020-01-12 06:07:54 +00:00
kern_ktrace.c ktrace: provide ktrstat_error 2020-02-03 22:26:00 +00:00
kern_linker.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_lock.c lockmgr: rename lock_fast_path to lock_flags 2020-02-14 11:21:28 +00:00
kern_lockf.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_lockstat.c Add lockmgr(9) probes to the lockstat DTrace provider. 2019-08-21 23:43:58 +00:00
kern_loginclass.c Replace hand-rolled unrefs if > 1 with refcount_release_if_not_last 2018-12-07 16:11:45 +00:00
kern_malloc.c Remove sparc64 kernel support 2020-02-03 17:35:11 +00:00
kern_mbuf.c uma: convert mbuf_jumbo_alloc to UMA_ZONE_CONTIG & tag others 2020-02-04 22:40:23 +00:00
kern_mib.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_module.c
kern_mtxpool.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_mutex.c Add KERNEL_PANICKED macro for use in place of direct panicstr tests 2020-01-12 06:07:54 +00:00
kern_ntptime.c Clear pad bytes in the struct exported by kern.ntp_pll.gettime. 2018-11-20 20:32:10 +00:00
kern_osd.c
kern_physio.c Allocate pager bufs from UMA instead of 80-ish mutex protected linked list. 2019-01-15 01:02:16 +00:00
kern_pmc.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_poll.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_priv.c Partially decompose priv_check by adding priv_check_cred_vfs_generation 2020-02-13 22:22:15 +00:00
kern_proc.c Add sysctl kern.proc.sigfastblk for reporting sigfastblock word address. 2020-02-09 12:29:51 +00:00
kern_procctl.c Add procctl(PROC_STACKGAP_CTL) 2019-09-03 18:56:25 +00:00
kern_prot.c malloc: remove assumptions about MINALLOCSIZE 2020-01-14 02:14:02 +00:00
kern_racct.c proc: eliminate the zombproc list 2019-08-28 16:18:23 +00:00
kern_rangelock.c rangelock: add rangelock_cookie_assert 2019-09-15 02:59:53 +00:00
kern_rctl.c
kern_resource.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_rmlock.c rms: use smp_rendezvous_cpus_retry instead of a hand-rolled variant 2020-02-12 11:17:18 +00:00
kern_rwlock.c locks: add default delay struct 2020-01-05 12:48:19 +00:00
kern_sdt.c
kern_sema.c
kern_sendfile.c Avoid releasing object PIP in vn_sendfile() if no pages were grabbed. 2020-02-05 16:09:21 +00:00
kern_sharedpage.c Fix the last few cases that grab without busy or valid. The grab functions must 2019-12-02 22:38:25 +00:00
kern_shutdown.c Add "panicked" boolean which can be tested instead of panicstr 2020-01-12 06:09:10 +00:00
kern_sig.c Add a way to manage thread signal mask using shared word, instead of syscall. 2020-02-09 11:53:12 +00:00
kern_switch.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_sx.c locks: add default delay struct 2020-01-05 12:48:19 +00:00
kern_synch.c Provide an API for interlocked refcount sleeps. 2020-01-19 18:18:17 +00:00
kern_syscalls.c add a sanity check to the system call registration code 2019-12-11 15:52:29 +00:00
kern_sysctl.c Restore the behavior of allowing empty string in a string sysctl 2020-02-10 20:53:59 +00:00
kern_tc.c Consolidate read code for timecounters and fix possible overflow in 2020-02-14 23:27:45 +00:00
kern_thr.c Fix a recursion on the thread lock by acquiring it after call rtp_to_pri(). 2020-02-04 02:42:54 +00:00
kern_thread.c Add a way to manage thread signal mask using shared word, instead of syscall. 2020-02-09 11:53:12 +00:00
kern_time.c Remove unneeded assert for curproc. Simplify. 2020-02-04 21:02:08 +00:00
kern_timeout.c Remove the deprecated timeout(9) interface. 2019-12-13 21:03:12 +00:00
kern_tslog.c
kern_ubsan.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_umtx.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
kern_uuid.c Use sbuf_cat() in GEOM confxml generation. 2019-06-19 15:36:02 +00:00
kern_xxx.c Normalize COMPAT_43 syscall declarations. 2018-12-04 16:48:47 +00:00
ksched.c
link_elf_obj.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
link_elf.c Add SYSCTL to get KERNBASE and relocated KERNBASE 2020-02-05 11:34:10 +00:00
linker_if.m
Make.tags.inc Remove a couple of harmless stray references to nandfs. 2019-06-25 16:39:25 +00:00
Makefile sys/kern sysent: re-add dependency on capabilities.conf 2020-02-12 19:06:34 +00:00
makesyscalls.sh Restore the ability to set capenabled directly in syscalls.conf. 2019-09-30 20:58:29 +00:00
md4c.c
md5c.c
msi_if.m
p1003_1b.c
pic_if.m
posix4_mib.c
sched_4bsd.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
sched_ule.c Fix the !SMP case in sched_add() after r355779. 2020-02-03 22:49:05 +00:00
serdev_if.m
stack_protector.c Revert r346292 (permit_nonrandom_stackcookies) 2019-05-13 23:37:44 +00:00
subr_acl_nfs4.c Remove unused argument to priv_check_cred. 2018-12-11 19:32:16 +00:00
subr_acl_posix1e.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_atomic64.c emulated atomic64: disable interrupts as the lock mechanism on !SMP 2020-01-03 18:29:20 +00:00
subr_autoconf.c
subr_blist.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_boot.c When parsing command line stuff, treat tabs and spaces the same. 2019-04-18 22:52:12 +00:00
subr_bufring.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_bus_dma.c Add an external mbuf buffer type that holds multiple unmapped pages. 2019-06-29 00:48:33 +00:00
subr_bus.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_busdma_bufalloc.c Add malloc_domainset(9) and _domainset variants to other allocator KPIs. 2018-10-30 18:26:34 +00:00
subr_capability.c capsicum: restore the cap_rights_contains symbol 2020-02-11 18:13:53 +00:00
subr_clock.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_compressor.c GZIO: Update to use zlib 1.2.11. 2019-08-25 07:50:44 +00:00
subr_counter.c Fix pre-SI_SUB_CPU initialization of per-CPU counters. 2018-07-10 00:18:12 +00:00
subr_coverage.c kcov: remove no longer needed atomic_load_ptr casts 2020-02-14 23:18:03 +00:00
subr_csan.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_devmap.c Print upper 32 bits in devmap table entries 2019-12-20 03:40:53 +00:00
subr_devstat.c Use atomic for start_count in devstat_start_transaction(). 2019-12-30 03:13:38 +00:00
subr_disk.c Fix botched merge with 355066 2019-03-12 05:10:41 +00:00
subr_dummy_vdso_tc.c
subr_early.c Add a file missed in r339321 2018-10-12 00:32:45 +00:00
subr_epoch.c epoch: convert zpcpu_get_cpua(.., curcpu) to zpcpu_get 2020-02-12 11:10:10 +00:00
subr_eventhandler.c Include ktr.h in more compilation units 2019-05-21 20:38:48 +00:00
subr_fattime.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_filter.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_firmware.c Factor out check for mounted root file system. 2019-11-28 08:47:36 +00:00
subr_gtaskqueue.c Add flag to struct task to mark the task as requiring network epoch. 2020-02-11 18:48:07 +00:00
subr_hash.c
subr_hints.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_intr.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_kdb.c Reimplement stack capture of running threads on i386 and amd64. 2020-01-31 15:43:33 +00:00
subr_kobj.c Prevent some kobj memory allocation failures from panicking the system. 2019-01-31 22:27:39 +00:00
subr_lock.c locks: add default delay struct 2020-01-05 12:48:19 +00:00
subr_log.c
subr_mchain.c
subr_module.c Have preload_delete_name() free pages backing preloaded data. 2018-07-19 20:00:28 +00:00
subr_msgbuf.c msgbuf: Light detailing (const'ify and bool'itize) 2018-08-09 17:42:27 +00:00
subr_param.c Remove sparc64 kernel support 2020-02-03 17:35:11 +00:00
subr_pcpu.c Store offset into zpcpu allocations in the per-cpu area. 2020-02-12 11:11:22 +00:00
subr_pctrie.c kern/subr_pctrie: Fix mismatched signedness in assertion comparison 2019-04-06 21:56:24 +00:00
subr_pidctrl.c When pidctrl_daemon() is called multiple times within an interval, it 2018-06-07 07:48:50 +00:00
subr_power.c Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
subr_prf.c Add KERNEL_PANICKED macro for use in place of direct panicstr tests 2020-01-12 06:07:54 +00:00
subr_prof.c ANSIfy sys/kern 2018-06-01 13:26:45 +00:00
subr_rangeset.c Implement rangesets. 2019-02-20 09:38:19 +00:00
subr_rman.c add support for marking interrupt handlers as suspended 2018-12-17 17:11:00 +00:00
subr_rtc.c Improve error messages from clock_if.m method failures. 2018-09-02 20:17:51 +00:00
subr_sbuf.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_scanf.c Add support for 'j', 't' and 'z' flags to kernel sscanf(). 2019-08-16 19:46:22 +00:00
subr_sfbuf.c
subr_sglist.c Add an external mbuf buffer type that holds multiple unmapped pages. 2019-06-29 00:48:33 +00:00
subr_sleepqueue.c Reimplement stack capture of running threads on i386 and amd64. 2020-01-31 15:43:33 +00:00
subr_smp.c Add smp_rendezvous_cpus_retry 2020-02-12 11:16:55 +00:00
subr_smr.c Since r357804 pcpu zones are required to use zalloc_pcpu(). Prior to this 2020-02-13 21:10:17 +00:00
subr_stack.c Add debugging facility EPOCH_TRACE that checks that epochs entered are 2019-09-25 18:26:31 +00:00
subr_stats.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_syscall.c Merge audit and systrace checks 2020-02-14 13:09:41 +00:00
subr_taskqueue.c Add flag to struct task to mark the task as requiring network epoch. 2020-02-11 18:48:07 +00:00
subr_terminal.c kernel terminal should initialize fg and bg variables before calling TUNABLE_INT_FETCH 2019-09-26 07:19:26 +00:00
subr_trap.c Annotate branches in the syscall path 2020-02-14 13:08:46 +00:00
subr_turnstile.c schedlock 4/4 2019-12-15 21:26:50 +00:00
subr_uio.c Remove sparc64 kernel support 2020-02-03 17:35:11 +00:00
subr_unit.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_vmem.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
subr_witness.c Remove sparc64 kernel support 2020-02-03 17:35:11 +00:00
sys_capability.c u_char -> vm_prot_t in a couple of places, NFC 2020-02-14 02:22:08 +00:00
sys_generic.c fd: remove the seq argument from fget_unlocked 2020-02-03 22:27:55 +00:00
sys_getrandom.c Fix a typo in r356667 comment 2020-01-12 23:52:16 +00:00
sys_pipe.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
sys_procdesc.c procdesc: allow to collect status through wait(1) if process is traced 2019-11-25 18:33:21 +00:00
sys_process.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
sys_socket.c
syscalls.c Regen. 2020-02-09 11:53:37 +00:00
syscalls.master Add a way to manage thread signal mask using shared word, instead of syscall. 2020-02-09 11:53:12 +00:00
systrace_args.c Regen. 2020-02-09 11:53:37 +00:00
sysv_ipc.c sysv: get rid of fork/exit hooks if the code is compiled in 2019-05-04 19:05:30 +00:00
sysv_msg.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
sysv_sem.c Correct the malloc tag used when freeing the temporary semop(2) buffer. 2020-02-04 20:00:45 +00:00
sysv_shm.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
tty_compat.c
tty_info.c Reimplement stack capture of running threads on i386 and amd64. 2020-01-31 15:43:33 +00:00
tty_inq.c
tty_outq.c
tty_pts.c tty_pts: don't rely on tty header pollution for sys/mutex.h 2019-11-29 03:56:01 +00:00
tty_tty.c Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
tty_ttydisc.c Replace ttyprintf with sbuf_printf and tty drain routine 2018-10-20 18:31:36 +00:00
tty.c fd: remove the seq argument from fget_unlocked 2020-02-03 22:27:55 +00:00
uipc_accf.c
uipc_debug.c Load balance sockets with new SO_REUSEPORT_LB option. 2018-06-06 15:45:57 +00:00
uipc_domain.c - Move global network epoch definition to epoch.h, as more different 2020-01-15 03:34:21 +00:00
uipc_ktls.c Make sure the VNET is properly set when calling tcp_drop() from 2020-01-21 11:43:25 +00:00
uipc_mbuf2.c m_pulldown(): Change an if () panic() into a KASSERT(). 2019-11-06 22:40:19 +00:00
uipc_mbuf.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
uipc_mbufhash.c
uipc_mqueue.c Make mqueue objects work across a fork again. 2020-01-27 22:36:54 +00:00
uipc_sem.c Remove unused argument to priv_check_cred. 2018-12-11 19:32:16 +00:00
uipc_shm.c Don't hold the object lock while calling getpages. 2020-01-19 23:47:32 +00:00
uipc_sockbuf.c sysctl: hide 2.x era compat node 2020-01-02 01:23:43 +00:00
uipc_socket.c Cleanup unneeded includes that crept in with r353292. 2019-10-09 16:59:42 +00:00
uipc_syscalls.c Only enable COMPAT_43 changes for syscalls ABI for a.out processes. 2019-08-11 19:16:07 +00:00
uipc_usrreq.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
vfs_acl.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
vfs_aio.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
vfs_bio.c Fix up various vnode-related asserts which did not dump the used vnode 2020-02-03 14:25:32 +00:00
vfs_cache.c Provide O_SEARCH 2020-02-02 16:34:57 +00:00
vfs_cluster.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
vfs_default.c lockmgr: add a change missed in r357907 2020-02-14 11:56:50 +00:00
vfs_export.c Ensure that directory entry padding bytes are zeroed. 2018-11-23 22:24:59 +00:00
vfs_extattr.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
vfs_hash.c vfs: incomplete pass at converting more ints to u_long 2020-01-11 22:56:20 +00:00
vfs_init.c Remove duplicated empty lines from kern/*.c 2020-01-30 20:05:05 +00:00
vfs_lookup.c vfs: use mac fastpath for lookup, open, read, write, mmap 2020-02-13 22:22:55 +00:00
vfs_mount.c vfs: drop remaining zpcpu casts 2020-02-12 11:18:12 +00:00
vfs_mountroot.c vfs: drop the mostly unused flags argument from VOP_UNLOCK 2020-01-03 22:29:58 +00:00
vfs_subr.c vfs: remove no longer needed atomic_load_ptr casts 2020-02-14 23:18:32 +00:00
vfs_syscalls.c Partially decompose priv_check by adding priv_check_cred_vfs_generation 2020-02-13 22:22:15 +00:00
vfs_vnops.c Partially decompose priv_check by adding priv_check_cred_vfs_generation 2020-02-13 22:22:15 +00:00
vnode_if.src vfs: remove the now empty vop_unlock_post 2020-02-02 09:36:32 +00:00