The timespecadd(3) family of macros were imported from NetBSD back in
r35029. However, they were initially guarded by #ifdef _KERNEL. In the
meantime, we have grown at least 28 syscalls that use timespecs in some
way, leading many programs both inside and outside of the base system to
redefine those macros. It's better just to make the definitions public.
Our kernel currently defines two-argument versions of timespecadd and
timespecsub. NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define
three-argument versions. Solaris also defines a three-argument version, but
only in its kernel. This revision changes our definition to match the
common three-argument version.
Bump _FreeBSD_version due to the breaking KPI change.
Discussed with: cem, jilles, ian, bde
Differential Revision: https://reviews.freebsd.org/D14725
the AMD document 55449 'Revision Guide for AMD Family 17h Models
00h-0Fh Processors' rev 1.12.
The errata numbers are mentioned near each action.
It seems that newer BIOSes already include required chicken bits
settings, so the magic MSR updates are only needed when BIOS cannot be
updated. On the other hand, MWAIT avoidance seems to be important.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
In order to setup an initial environment and jump into the generic
hammer_time initialization function. Some of the code is shared with
PVHv1, while other code is PVHv2 specific.
This allows booting FreeBSD as a PVHv2 DomU and Dom0.
Sponsored by: Citrix Systems R&D
Allow the hypercall page to be initialized very early, even before
vtophys is functional. Also make the function global so it can be
called by other files.
This will be needed in order to perform the early bringup on PVHv2
guests.
Sponsored by: Citrix Systems R&D
HYPERVISOR_start_info is only available to PV and PVHv1 guests, HVM
and PVHv2 guests get this data from HVM parameters that are fetched
using a hypercall.
Instead provide a set of helper functions that should be used to fetch
this data. The helper functions have different implementations
depending on whether FreeBSD is running as PVHv1 or HVM/PVHv2 guest
type.
This helps to cleanup generic Xen code by removing quite a lot of
xen_pv_domain and xen_hvm_domain macro usages.
Sponsored by: Citrix Systems R&D
boot_parse_arg to parse a single arg
boot_parse_cmdline to parse a command line string
boot_parse_args to parse all the args in a vector
boot_howto_to_env Convert howto bits to env vars
boot_env_to_howto Return howto mask mased on what's set in the environment.
All these routines return an int that's the bitmask of the args
translated to RB_* flags. As a special case, the 'S' flag sets the
comconsole_speed env var. Any arg that looks like a=b will set the env
key 'a' to value 'b'. If =b is omitted, 'a' is set to '1'. This
should help us reduce the number of redundant copies of these routines
in the tree. It should also give a more uniform experience between
platforms.
Also, invent a new flag RB_PROBE that's set when 'P' is parsed. On
x86 + BIOS, this means 'probe for the keyboard, and if it's not there
set both RB_MULTIPLE and RB_SERIAL (which means show the output on
both video and serial consoles, but make serial primary). Others it
may be some similar concept of probing, but it's loader dependent
what, exactly, it means.
These routines are suitable for /boot/loader and/or the kernel,
though they may not be suitable for the tightly hand-rolled-for-space
environments like boot2.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D16205
- Change pcpu zone consumers to use a stride size of PAGE_SIZE.
(defined as UMA_PCPU_ALLOC_SIZE to make future identification easier)
- Allocate page from the correct domain for a given cpu.
- Don't initialize pc_domain to non-zero value if NUMA is not defined
There are some misconceptions surrounding this field. It is the
_VM_ NUMA domain and should only ever correspond to valid domain
values as understood by the VM.
The former slab size of sizeof(struct pcpu) was somewhat arbitrary.
The new value is PAGE_SIZE because that's the smallest granularity
which the VM can allocate a slab for a given domain. If you have
fewer than PAGE_SIZE/8 counters on your system there will be some
memory wasted, but this is obviously something where you want the
cache line to be coming from the correct domain.
Reviewed by: jeff
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15933
On arm64 (and possible other architectures) we are unable to use static
DPCPU data in kernel modules. This is because the compiler will generate
PC-relative accesses, however the runtime-linker expects to be able to
relocate these.
In preparation to fix this create two macros depending on if the data is
global or static.
Reviewed by: bz, emaste, markj
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D16140
The interface already guarantees that the number of hypercall pages is
always going to be 1, see the comment in interface/arch-x86/cpuid.h
Sponsored by: Citrix Systems R&D
devices present.
On at least one machine where it would matter since the ISA timer is
power gated when booted in the UEFI mode, BIOS still reports that the
legacy devices are present. That is, user still have to manually
disable TSC calibration on such machines. Hopefully it will be more
useful in the future.
Discussed with: Ben Widawsky <benjamin.widawsky@intel.com>
Reviewed by: royger
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D16004
MFC after: 1 week
and also on apic in common and i386 files (except for xen it is optional
only on xenhvm), but it was not ifdefed except on apic in common and i386
files.
This is all that is left from an attempt to build a (sub-)minimal kernel
without any devices. The isa "option" is still used without ifdefs in many
standard files even on amd64. ISAPNP is not optional on at least i386.
ATPIC is not optional on i386 (it is used mainly for Xspuriousint). But
pci is now supposed to be optional on x86.
Expected NMI-s are those than are either generated by the software (such
as a CPU sending NMI to other CPU) or generated by the hardware after
the software configured it to do so (such as NMI-s on PMC events).
Some unexpected NMI-s can be caused by hardware failures and it is
possible to inquire the hardware about them (somewhat like MCA but much
more primitive) using an EISA mechanism. In some cases the origin of
the NMI can remain truly unknown.
This commit should not change any functionality. It just reorganizes
the code, so that it is easier to extend with new checks for the origin
of the NMI. Also, it frees the code that has nothing to do with ISA
from DEV_ISA.
MFC after: 3 weeks
This change adds a new optional console method cn_resume and a kernel
console interface cnresume. Consoles that may need to re-initialize
their hardware after suspend (e.g., because firmware does not care to do
it) will implement cn_resume. Note that it is called in rather early
environment not unlike early boot, so the same restrictions apply.
Platform specific code, for platforms that support hardware suspend,
should call cnresume early after resume, before any console output is
expected.
This change fixes a problem with a system of mine failing to resume when
a serial console is used. I found that the serial port was in a strange
configuration and an attempt to write to it likely resulted in an
infinite loop.
To avoid adding cn_resume method to every console driver, CONSOLE_DRIVER
macro has been extended to support optional methods.
Reviewed by: imp, mav
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D15552
The TSC-s are checked and synchronized only if they were good
originally. That is, invariant, synchronized, etc.
This is necessary on an AMD-based system where after a wakeup from STR I
see that BSP clock differs from AP clocks by a count that roughly
corresponds to one second. The APs are in sync with each other. Not
sure if this is a hardware quirk or a firmware bug.
This is what I see after a resume with this change:
SMP: passed TSC synchronization test after adjustment
acpi_timer0: restoring timecounter, ACPI-fast -> TSC-low
Reviewed by: kib
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D15551
- Add constants for fields in DR6 and the reserved fields in DR7. Use
these constants instead of magic numbers in most places that use DR6
and DR7.
- Refer to T_TRCTRAP as "debug exception" rather than a "trace trap"
as it is not just for trace exceptions.
- Always read DR6 for debug exceptions and only clear TF in the flags
register for user exceptions where DR6.BS is set.
- Clear DR6 before returning from a debug exception handler as
recommended by the SDM dating all the way back to the 386. This
allows debuggers to determine the cause of each exception. For
kernel traps, clear DR6 in the T_TRCTRAP case and pass DR6 by value
to other parts of the handler (namely, user_dbreg_trap()). For user
traps, wait until after trapsignal to clear DR6 so that userland
debuggers can read DR6 via PT_GETDBREGS while the thread is stopped
in trapsignal().
Reviewed by: kib, rgrimes
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D15189
Speculative Store Bypass (SSB) is a speculative execution side channel
vulnerability identified by Jann Horn of Google Project Zero (GPZ) and
Ken Johnson of the Microsoft Security Response Center (MSRC)
https://bugs.chromium.org/p/project-zero/issues/detail?id=1528.
Updated Intel microcode introduces a MSR bit to disable SSB as a
mitigation for the vulnerability.
Introduce a sysctl hw.spec_store_bypass_disable to provide global
control over the SSBD bit, akin to the existing sysctl that controls
IBRS. The sysctl can be set to one of three values:
0: off
1: on
2: auto
Future work will enable applications to control SSBD on a per-process
basis (when it is not enabled globally).
SSBD bit detection and control was verified with prerelease microcode.
Security: CVE-2018-3639
Tested by: emaste (previous version, without updated microcode)
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Install appropriate pti-aware shootdown IPI handlers, otherwise user
page tables do not get enough invalidations. The non-pti handlers
were used so far.
Reported and tested by: cperciva
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
The intent was to disable IBPB and IBRS around MWAIT, and re-enable on
the sleep end.
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
This change reverts a "while here" part of r333321 that moved clearing
of suspended_cpus to an earlier place.
Apparently, there can be a problem when modifying (shared) memory before
restoring proper cache attributes. So, to be safe, move the clearing to
the old place.
Many thanks to Johannes Lundberg for bisecting the changes to that
particular commit and then bisecting the commit to the particular
change.
Reported by: many
Debugged by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
X-MFC with: r333321
The idea is to calibrate the LAPIC timer just once and only on boot,
given that [at present] the timer constants are global and shared
between all processors.
My primary motivation is to fix a panic that can happen when dynamically
switching to lapic timer. The panic is caused by a recursion on
et_hw_mtx when printing the calibration results to console. See the
review for the details of the panic.
Also, the code should become slightly simpler and easier to read. The
previous code was racy too. Multiple processors could start calibrating
the global constants concurrently, although that seems to have been
benign.
Reviewed by: kib, mav, jhb
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D15422
Without a subsequent wbinvd the changes to suspended_cpus (and
resuming_cpus) can be lost at least on AMD systems that use MOESI cache
coherency protocol. That can happen because one of APs ends up as an
Owner of the corresponding cache line(s) and the changes may never reach
the main memory before the AP is reset.
While here, move clearing of suspended_cpus a little bit earlier as the
fact of returning from savectx (with zero return value) means that the
CPU has fully restored it execution context.
Also, rework the comment that describes the need for resuming_cpus.
This change fixed suspend to RAM a previously broken AMD-based system.
Reviewed by: kib
Discussed with: bde
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D15295
ifuncs on x86.
Also keep helpers to define 'pseudo-ifuncs' which are emulated by the
indirect jmp.
Reviewed by: jhb (previous version, as part of the larger patch)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D13838
Resume starts CPU from the init state, which clears any loaded
microcode updates. As result, IBRS MSRs are no longer available,
until the microcode is reloaded.
I have to forcibly clear cpu_stdext_feature3, which assumes that CPUID
leaf 7 reg %ebx does not report anything except Meltdown/Spectre bugs
bits. If future CPUs add new bits there, hw_ibrs_recalculate() and
identify_cpu1()/identify_cpu2() need to be adjusted for that.
Submitted and tested by: Michael Danilov <mike.d.ft402@gmail.com>
PR: 227866
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D15236
The APL31 NDA errata is APL30 public errata. Add the reference and
provide the description [2].
Noted by: emaste [2], rpokala [1]
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
If the workaround is activated, always send IPI for wake up, not rely
on the write to the monitor line. This fixes Appolo Lake machines
early hang in sched_bind(), without requiring user to manually select
idle method.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Use designated initializers for the idlt_tlb elements.
Remove strstr() use, add flag field to detect supported MWAIT.
Use nitems() instead of the terminating NULL entry for idle_tlb.
Move several functions into cpu_idle_* namespace.
Based on the discussion with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
disabled.
Intel finally added this information, which allows us to not parse CPU
identification string looking for the nominal frequency. The leaf is
present e.g. on Appolo Lake Atom CPUs. It is only used if the TSC
calibration is disabled by user.
Also, report the TSC frequency in bootverbose mode always, regardless
of the way it was obtained.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
It is applied before it is possible for idle threads to execute on any
CPU, allowing to work around against some bugs.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Otherwise, under bootverbose, the lapic_enable_cmc() banner 'lapicX:
CMCI unmasked' is printed by several CPUs in parallel, causing garbled
output for the LAPIC dumps.
Reported by: royger
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D15157
machine check banks must be only monitored by single CPU.
Noted and reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D15157
We must ensure that accesses occur, they do not have any other
compiler-visible effects. Bruce found some situations where
optimization could remove an access, and provided a patch to use
volatile qualifier for the state variables. Since volatile behaviour
there is the compiler-specific interpretation of the keyword, use
relaxed atomics instead, which gives exactly the desired semantic.
Noted by and discussed with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This sysctl allows a deeper dive into the sleep abyss comparing to
debug.acpi.suspend_bounce. When the new sysctl is set the system will
execute the suspend sequence up to the call to AcpiEnterSleepState().
That includes saving processor contexts and parking APs. Then, instead
of actually entering the sleep state, the BSP will call resumectx() to
emulate the wakeup. The APs should get restarted by the sequence of
Init and Startup IPIs that BSP sends to them.
MFC after: 8 days
x86 enforces an (arbitray) limit on the number of available MSI and
MSI-X interrupts to simplify code (in particular, interrupt_source[]
is statically sized). This means that an attempt to allocate an MSI
vector needs to fail if it would go beyond the limit, but the checks
for exceeding the limit had an off-by-one error. In the case of MSI-X
which allocates interrupts one at a time this meant that IRQ 768 kept
getting handed out multiple times for msix_alloc() instead of failing
because all MSI IRQs were in use.
Tested by: lidl
MFC after: 1 week
