Commit Graph

958 Commits

Author SHA1 Message Date
Konstantin Belousov
b7b6b7a9c5 PR: 239143
Reported and tested by:	Wes Maag <jwmaag@gmail.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-07-14 21:08:54 +00:00
Konstantin Belousov
e2e0470dfa Ensure that mds_handler always points to a valid method.
Depending on system configuration, version, and architecture,
mds_handler might be dereferenced from doreti before
hw_mds_recalculate_boot() initialized it.  Statically assign void
method to cover all cases.

Reported by:	"Schuendehuette, Matthias (LDA IT PLM)" <matthias.schuendehuette@siemens.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2019-07-11 16:22:49 +00:00
Jonathan T. Looney
ca8929d2a3 Currently, MCA entries remain on an every-growing linked list. This means
that it becomes increasingly expensive to process a steady stream of
correctable errors. Additionally, the memory used by the MCA entries can
grow without bound.

Change the code to maintain two separate lists: a list of entries which
still need to be logged, and a list of entries which have already been
logged. Additionally, allow a user-configurable limit on the number of
entries which will be saved after they are logged. (The limit defaults
to -1 [unlimited], which is the current behavior.)

Reviewed by:	imp, jhb
MFC after:	2 weeks
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20482
2019-06-08 18:26:48 +00:00
Tycho Nightingale
56db4ebd34 another occurrence where a very large dma mapping can cause integer overflow
Submitted by:	rlibby
Sponsored by:	Dell EMC Isilon
2019-06-05 13:08:21 +00:00
Tycho Nightingale
88e9fbe568 very large dma mappings can cause integer overflow
Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20505
2019-06-03 19:19:35 +00:00
John Baldwin
bebcdc0073 Add a constant for the LS config MSR on AMD CPUs.
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D19506
2019-05-23 23:37:11 +00:00
Conrad Meyer
c63f1e21da Decode and name additional x86 feature bits
These are all enumerated in Intel's ISA extension reference, 37th ed.

Sponsored by:	Dell EMC Isilon
2019-05-22 23:22:36 +00:00
Andrew Gallatin
18f9bb6fe0 x86 MCA: introduce MCA hooks for different vendor implementations
This is needed for AMD SMCA processors, as SMCA uses different
MSR address for access MCA banks.

Use IA32 specific msr_ops as defualt, and use SMCA-specific msr_ops
when on an SMCA-enabled processor

Submitted by:	chandu from amd dot com
Reviewed by:	cem
Differential Revision:	https://reviews.freebsd.org/D18055
2019-05-22 13:44:15 +00:00
Konstantin Belousov
48ec6d3bc9 Do not call hw_mds_recalculate() from initializecpu().
If MDS mitigation is enabled by the tunable but MDS microcode is not
early-loaded, software mitigation is selected.  This causes
initializecpu() to try to allocate memory which makes boot process
very unhappy.

Create SYSINIT that runs sufficiently late to succeed.

Reported by:	naddy
PR:	237968
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-05-21 22:56:21 +00:00
Stephen J. Kiernan
1177d38ce1 The older detection methods (smbios.bios.vendor and smbios.system.product)
are able to determine some virtual machines, but the vm_guest variable was
still only being set to VM_GUEST_VM.

Since we do know what some of them specifically are, we can set vm_guest
appropriately.

Also, if we see the CPUID has the HV flag, but we were unable to find a
definitive vendor in the Hypervisor CPUID Information Leaf, fall back to
the older detection methods, as they may be able to determine a specific
HV type.

Add VM_GUEST_PARALLELS value to VM_GUEST for Parallels.

Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D20305
2019-05-21 13:29:53 +00:00
Conrad Meyer
e2e050c8ef Extract eventfilter declarations to sys/_eventfilter.h
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions.  The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended).  Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed.  __FreeBSD_version has been bumped.
2019-05-20 00:38:23 +00:00
Stephen J. Kiernan
c5c8916278 Add missing setting of hv_base to the leaf that we used.
Correct setting hv_high to use regs[0], not leaf.
2019-05-19 15:07:14 +00:00
Stephen J. Kiernan
949f834a61 Instead of individual conditional statements to look for each hypervisor
type, use a table to make it easier to add more in the future, if needed.

Add VirtualBox detection to the table ("VBoxVBoxVBox" is the hypervisor
vendor string to look for.) Also add VM_GUEST_VBOX to the VM_GUEST
enumeration to indicate VirtualBox.

Save the CPUID base for the hypervisor entry that we detected. Driver code
may need to know about it in order to obtain additional CPUID features.

Approved by:	bryanv, jhb
Differential Revision:	https://reviews.freebsd.org/D16305
2019-05-17 17:21:32 +00:00
Konstantin Belousov
8f7f38457f Free microcode memory later.
With lockless DI, pmap_remove() requires operational thread lock,
which is initialized at SI_SUB_RUN_QUEUE for thread0.  Move it even
later where APs are started, the moment after which other boot memory
like trampoline stacks is already being freed.

Reported by:	gtetlow
Sponsored by:	The FreeBSD Foundation
MFC after:	30 days
2019-05-17 17:11:01 +00:00
Konstantin Belousov
7c5a46a1bc Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.
In all practical situations, the resolver visibility is static.

Requested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	so (emaste)
Differential revision:	https://reviews.freebsd.org/D20281
2019-05-16 22:20:54 +00:00
Tycho Nightingale
b961c0f244 Allow loading the same DMA address multiple times without any prior
unload for the LinuxKPI.

Reviewed by:	kib, zeising
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20181
2019-05-16 17:41:16 +00:00
Ryan Libby
244081120e iommu static analysis cleanup
A static analyzer complained about a couple instances of checking a
variable against NULL after already having dereferenced it.
 - dmar_gas_alloc_region: remove the tautological NULL checks
 - dmar_release_resources / dmar_fini_fault_log: don't deref unit->regs
   unless initialized.

And while here, fix an inverted initialization check in dmar_fini_qi.

Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential revision:	https://reviews.freebsd.org/D20263
2019-05-16 04:24:08 +00:00
Conrad Meyer
e7e3d5223f x86: Correctly identify bhyve hypervisor
Spotted after a similar report by Olivier Cochard-Labbé.

Sponsored by:	Dell EMC Isilon
2019-05-16 01:32:54 +00:00
Konstantin Belousov
b55d4ebe5f Properly announce MD_CLEAR.
Submitted by:	Petr Lampa <lampa@fit.vutbr.cz>
MFC after:	3 days
2019-05-15 17:55:41 +00:00
Konstantin Belousov
7355a02bdd Mitigations for Microarchitectural Data Sampling.
Microarchitectural buffers on some Intel processors utilizing
speculative execution may allow a local process to obtain a memory
disclosure.  An attacker may be able to read secret data from the
kernel or from a process when executing untrusted code (for example,
in a web browser).

Reference: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00233.html
Security:	CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, CVE-2019-11091
Security:	FreeBSD-SA-19:07.mds
Reviewed by:	jhb
Tested by:	emaste, lwhsu
Approved by:	so (gtetlow)
2019-05-14 17:02:20 +00:00
Mateusz Guzik
a8c2fcb287 x86: store pending bitmapped IPIs in per-cpu areas
This gets rid of the global cpu_ipi_pending array.

While replace cmpset with fcmpset in the delivery code and opportunistically
check if given IPI is already pending.

Sponsored by:	The FreeBSD Foundation
2019-05-12 06:36:54 +00:00
Konstantin Belousov
078116a662 amd64: fix BUS_SPACE_MAXSIZE to 64bit max value.
Reviewed by:	jhb, tychon (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D20154
2019-05-07 01:18:57 +00:00
Tycho Nightingale
8d2a55ca67 zero inputs to vm_page_initfake() for predictable results
Reviewed by:	kib
Submitted by:	Anton Rang <rang at acm.org>
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20162
2019-05-06 00:57:05 +00:00
Conrad Meyer
665919aaaf x86: Implement MWAIT support for stopping a CPU
IPI_STOP is used after panic or when ddb is entered manually.  MONITOR/
MWAIT allows CPUs that support the feature to sleep in a low power way
instead of spinning.  Something similar is already used at idle.

It is perhaps especially useful in oversubscribed VM environments, and is
safe to use even if the panic/ddb thread is not the BSP.  (Except in the
presence of MWAIT errata, which are detected automatically on platforms with
known wakeup problems.)

It can be tuned/sysctled with "machdep.stop_mwait," which defaults to 0
(off).  This commit also introduces the tunable
"machdep.mwait_cpustop_broken," which defaults to 0, unless the CPU has
known errata, but may be set to "1" in loader.conf to signal that mwait
wakeup is broken on CPUs FreeBSD does not yet know about.

Unfortunately, Bhyve doesn't yet support MONITOR extensions, so this doesn't
help bhyve hypervisors running FreeBSD guests.

Submitted by:   Anton Rang <rang AT acm.org> (earlier version)
Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20135
2019-05-04 20:34:26 +00:00
Conrad Meyer
83dc49beaf x86: Define pc_monitorbuf as a logical structure
Rather than just accessing it via pointer cast.

No functional change intended.

Discussed with:	kib (earlier version)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20135
2019-05-04 17:35:13 +00:00
Konstantin Belousov
9891fa5592 Remove witness warning, same as r346351 for busdma_dmar.
bounce_bus_dmamap_create() does not sleep either.

Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-28 18:45:44 +00:00
Conrad Meyer
f1498d7aa3 x86: Halt non-BSP CPUs on panic IPI_STOP
We may need the BSP to reboot, but we don't need any AP CPU that isn't the
panic thread.  Any CPU landing in this routine during panic isn't the panic
thread, so we can just detect !BSP && panic and shut down the logical core.

The savings can be demonstrated in a bhyve guest with multiple cores; before
this change, N guest threads would spin at 100% CPU.  After this change,
only one or two threads spin (depending on if the panicing CPU was the BSP
or not).

Konstantin points out that this may break any future patches which allow
switching ddb(4) CPUs after panic and examining CPU-local state that cannot
be inspected remotely.  In the event that such a mechanism is incorporated,
this behavior could be made configurable by tunable/sysctl.

Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20019
2019-04-24 18:24:22 +00:00
Tycho Nightingale
96ca24dc32 remove the 4GB boundary requirement on PCI DMA segments
Reviewed by:	kib
Discussed with:	jhb
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19867
2019-04-19 13:43:33 +00:00
Konstantin Belousov
2d8bfbdcb2 Use correct type name.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 15:31:03 +00:00
Konstantin Belousov
f9feb09189 Correct handling of RMRR during early enumeration stages.
On some machines, DMAR contexts must be created before all devices
under the scope of the corresponding DMAR unit are enumerated.
Current code has two problems with that:
- scope lookup returns NULL device_t, which causes to skip creating a
  context with RMRR, which is fatal for the affected device.
- calculation of the final pci dbsf address fails if any bridge in the
  scope is not yet enumerated, because code relies on pcib_get_bus().

Make creation of contexts work either with device_t, or with DMAR PCI
scope paths.  Scope provides enough information to infer context
address, and it is directly matched against DMAR tables scopes.

When calculating bus addresses for the scope or device, use direct
pci_cfgregread(PCIR_SECBUS_1) to get the secondary bus number, instead
of pcib_get_bus().

The issue was observed on HP Gen servers, where iLO PCI devices are
located behind south bridge switch.  Turning on translation without
satisfying RMRR requests caused iLO to mostly hang, up to the level of
being unusable to control the server.

While there, remove hw.dmar.dmar_match_verbose tunable, and make the
normal logging under bootverbose useful and sufficient to diagnose
DRHD and RMRR parsing and matching.

Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 14:18:06 +00:00
Konstantin Belousov
c07640c430 Remove witness warning. dmar_bus_dmamap_create() does not sleep.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 14:03:59 +00:00
Konstantin Belousov
1ad4a0314f Reduce verbosity, do not announce details of irte programming by default.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 14:02:33 +00:00
Konstantin Belousov
2a508645b4 pci_cfgreg.c: Use io port config access for early boot time.
Some early PCIe chipsets are explicitly listed in the white-list to
enable use of the MMIO config space accesses, perhaps because ACPI
tables were not reliable source of the base MCFG address at that time.
For that chipsets, MCFG base was read from the known chipset MCFGbase
config register.

During very early stage of boot, when access to the PCI config space
is performed (see e.g. pci_early_quirks.c), we cannot map 255MB of
registers because the method used with pre-boot pmap overflows initial
kernel page tables.

Move fallback to read MCFGbase to the attachment method of the
x86/legacy device, which removes code duplication, and results in the
use of io accesses until MCFG is parsed or legacy attach called.

For amd64, pre-initialize cfgmech with CFGMECH_1, right now we
dynamically assign CFGMECH_1 to it anyway, and remove checks for
CFGMECH_NONE.

There is a mention in the Intel documentation for corresponding
chipsets that OS must use either io port or MMIO access method, but we
already break this rule by reading MCFGbase register, so one more
access seems to be innocent.

Reported by:	longwitz@incore.de
PR:	236838
Reviewed by:	avg (other version), jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19833
2019-04-09 18:07:17 +00:00
Tycho Nightingale
9708c3a2b8 DMAR driver assumes all physical addresses are backed by a fully
initialized struct vm_page.

Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19753
2019-04-02 18:50:49 +00:00
Tycho Nightingale
cec2287b6a Use the BUS_DMA_NOWRITE flag to expose and create the read-only VT-d
IOMMU mappings.

Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19729
2019-03-27 20:15:51 +00:00
Konstantin Belousov
fd8d844f76 amd64 KPTI: add control from procctl(2).
Add the infrastructure to allow MD procctl(2) commands, and use it to
introduce amd64 PTI control and reporting.  PTI mode cannot be
modified for existing pmap, the knob controls PTI of the new vmspace
created on exec.

Requested by:	jhb
Reviewed by:	jhb, markj (previous version)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:44:33 +00:00
Konstantin Belousov
7e0a345bc5 Add symbolic name for TSC_AUX MSR address.
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2019-03-15 16:39:05 +00:00
Konstantin Belousov
3dcf329ee5 Add register number, CPUID bits, and print identification for TSX
force abort errata.

Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2019-03-12 18:59:01 +00:00
Vladimir Kondratyev
76cefcd810 Fix amd64/i386 LINT build after r344982
Submitted by:	jkim
Reported by:	rpokala
MFC with:	r344982
2019-03-11 19:46:15 +00:00
Vladimir Kondratyev
2b4ee39838 atrtc(4): install ACPI RTC/CMOS operation region handler
FreeBSD base system does not provide an ACPI handler for the PC/AT RTC/CMOS
device with PnP ID PNP0B00; on some HP laptops, the absence of this handler
causes suspend/resume and poweroff(8) to hang or fail [1], [2]. On these
laptops EC _REG method queries the RTC date/time registers via ACPI
before suspending/powering off. The handler should be registered before
acpi_ec driver is loaded.

This change adds handler to access CMOS RTC operation region described in
section 9.15 of ACPI-6.2 specification [3]. It is installed only for ACPI
version of atrtc(4) so it should not affect old ACPI-less i386 systems.

It is possible to disable the handler with loader tunable:
debug.acpi.disabled=atrtc

Informational debugging printf can be enabled by setting hw.acpi.verbose=1
in loader.conf

[1] https://wiki.freebsd.org/Laptops/HP_Envy_6Z-1100
[2] https://wiki.freebsd.org/Laptops/HP_Notebook_15-af104ur
[3] https://uefi.org/sites/default/files/resources/ACPI_6_2.pdf

PR:		207419, 213039
Submitted by:	Anthony Jenkins <Scoobi_doo@yahoo.com>
Reviewed by:	ian
Discussed on:	acpi@, 2013-2015, several threads
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D19314
2019-03-10 20:19:43 +00:00
John Baldwin
2e43efd0bb Drop "All rights reserved" from my copyright statements.
Reviewed by:	rgrimes
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D19485
2019-03-06 22:11:45 +00:00
Konstantin Belousov
a2d95495ee Add usermode helpers for for Intel userspace protection keys feature.
Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D18893
2019-02-20 09:56:23 +00:00
Konstantin Belousov
e7a9df16e6 Add kernel support for Intel userspace protection keys feature on
Skylake Xeons.

See SDM rev. 68 Vol 3 4.6.2 Protection Keys and the description of the
RDPKRU and WRPKRU instructions.

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D18893
2019-02-20 09:51:13 +00:00
Bruce Evans
27c56cf357 Fix hangs in r341810 waiting for AP startup.
idle_td is dereferenced without thread-locking it to make its contents is
invariant, and was accessed without telling the compiler that its contents
is invariant.  Some compilers optimized accesses to the supposedly invariant
contents by moving the critical checks for changes outside of the loop that
waits for changes.  Fix this using atomic ops.

This bug only showed up for the following configuration: a Turion2
system, amd64 kernels, compiled by gcc, and SCHED_4BSD.  clang fails
to do the optimization with all CFLAGS that I tried, because it doesn't
fully optimize the '__asm __volatile' for cpu_spinwait() although this
asm has no memory clobber.  gcc only does the optimization with most
CFLAGS.  I mostly used -Os with all compilers.  i386 works because gcc
-m32 -Os only moves 1 or the 2 accesses outside of the loop.
Non-Turion2 systems and SCHED_ULE worked due to different timing (when
all APs start before the BP checks them outside of the loop).

Reviewed by:	kib
2019-02-20 02:40:38 +00:00
Konstantin Belousov
5671e0d62e Add definition for %cr4 PKRU enable bit.
Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
Differential revision:	https://reviews.freebsd.org/D18893
2019-02-19 19:13:48 +00:00
Konstantin Belousov
eb785fab3b Port sysctl kern.elf32.read_exec from amd64 to i386.
Make it more comprehensive on i386, by not setting nx bit for any
mapping, not just adding PF_X to all kernel-loaded ELF segments.  This
is needed for the compatibility with older i386 programs that assume
that read access implies exec, e.g. old X servers with hand-rolled
module loader.

Reported and tested by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-07 02:17:34 +00:00
Konstantin Belousov
f76b5ab6cc Fix resume on i386 PAE.
It was broken before PAE/no-PAE merge, but since now PAE is the
default, resume is apparently becomes for all machines.

The corrected issues:
- the trampoline page is not mapped executable, so machine faults when
  paging is on;
- MSR.EFER and %cr4 both should be loaded before paging is enabled,
  otherwise paging structures are invalid (cr4.PAE and EFER.NX).
- MSR.EFER and %cr4 should be only loaded if present.  I attempt to handle
  this by not touching the registers if the value is zero.

There are some more bits still not quite correct, e.g. unconditional
access to %cr4 in resumectx.

Reported and debugging help by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-07 02:09:34 +00:00
Konstantin Belousov
ccc2d07e77 Update CPUID bits definitions and CPU identification based on changes
in SDM rev. 069.

Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2019-02-04 23:57:59 +00:00
Konstantin Belousov
c3f5a36651 x86: correctly limit max memory resource address..
CPU and buses can manage up to the limit reported by cpu_maxphyaddr,
so set mem_rman to the value returned by cpu_getmaxphyaddr().  For the
PAE mode, it was missed both when rman_res_t was increased to
uintmax_t, and from the PAE merge commit.

When importing smaps or dump_avail chunks into memory rman, do not
blindly ignore resources which ends above the limit, chomp them
instead if start is below the limit.  The same change was already done
to i386 add_physmap_entry().

Based on the submission by:	bde
MFC after:	2 months
2019-02-01 20:46:47 +00:00
Roger Pau Monné
27c36a12f1 xen: introduce a new way to setup event channel upcall
The main differences with the currently implemented method are:

 - Requires a local APIC EOI, since it doesn't bypass the local APIC
   as the previous method used to do.
 - Can be set to use different IDT vectors on each vCPU. Note that
   FreeBSD doesn't make use of this feature since the event channel
   IDT vector is reserved system wide.

Note that the old method of setting the event channel upcall is
not removed, and will be used as a fallback if this newly introduced
method is not available.

MFC after:	1 month
Sponsored by:	Citrix Systems R&D
2019-01-30 11:34:52 +00:00