Commit Graph

747 Commits

Author SHA1 Message Date
Pedro F. Giffuni
74641f0bc6 x86: make some use of mallocarray(9).
Focus on code where we are doing multiplications within malloc(9). None of
these ire likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.

X-Differential revision: https://reviews.freebsd.org/D13837
2018-01-15 21:08:22 +00:00
Ian Lepore
7c63e50188 Convert the x86 RTC driver to use new validated BCD<->timespec conversions.
New common routines were added to kern/subr_clock.c for converting between
calendrical time expressed in BCD and struct timespec. The new functions
return EINVAL on error, as expected when the clock hardware does not provide
valid time.

PR:		224813
Differential Revision:	https://reviews.freebsd.org/D13731 (no reviewers)
2018-01-15 16:40:43 +00:00
Konstantin Belousov
e8c770a66e Enumerate and print Intel CPU features for Speculative Execution Side
Channel Mitigations.

The definitions are taken from the document 336996-001.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-01-14 12:36:23 +00:00
Jeff Roberson
b6715dab8f Move VM_NUMA_ALLOC and DEVICE_NUMA under the single global config option NUMA.
Sponsored by:	Netflix, Dell/EMC Isilon
Discussed with:	jhb
2018-01-14 03:36:03 +00:00
Conrad Meyer
233933cb00 amd64: Add a 48-bit MAXADDR constant
Some devices (e.g., ccp(4) -- to be committed) can only access the low 48
bits of physical memory.

Reviewed by:	markj
Sponsored by:	Dell EMC Isilon
2018-01-13 17:55:22 +00:00
Jeff Roberson
6f4acaf4c9 Add support for NUMA domains to bus dma tags. This causes all memory
allocated with a tag to come from the specified domain if it meets the
other constraints provided by the tag.  Automatically create a tag at
the root of each bus specifying the domain local to that bus if
available.

Reviewed by:	jhb, kib
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13545
2018-01-12 23:34:16 +00:00
Jeff Roberson
3f289c3fcf Implement 'domainset', a cpuset based NUMA policy mechanism. This allows
userspace to control NUMA policy administratively and programmatically.

Implement domainset based iterators in the page layer.

Remove the now legacy numa_* syscalls.

Cleanup some header polution created by having seq.h in proc.h.

Reviewed by:	markj, kib
Discussed with:	alc
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13403
2018-01-12 22:48:23 +00:00
Konstantin Belousov
0530a9360f Make it possible to re-evaluate cpu_features.
Add cpuctl(4) ioctl CPUCTL_EVAL_CPU_FEATURES which forces re-read of
cpu_features, cpu_features2, cpu_stdext_features, and
std_stdext_features2.

The intent is to allow the kernel to see the changes in the CPU
features after micocode update.  Of course, the update is not atomic
across variables and not synchronized with readers.  See the man page
warning as well.

Reviewed by:	imp (previous version), jilles
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D13770
2018-01-05 21:06:19 +00:00
Konstantin Belousov
af317aa4e5 Use the new SDM-approved way to serialize x2APIC MSR writes.
SDM editions 64 and below stated that it is enough to use MFENCe or
LFENCE to serialize x2APIC register writes.  New edition 65 requires
either full serialization instruction or MFENCE;LFENCE sequence.  Use
the later, FreeBSD needs serialization to ensure that writes done
before IPI request are visible to the target IPI CPU.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-01-03 11:23:47 +00:00
Konstantin Belousov
da457ed9d6 Add CR4.SMAP control bit.
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2018-01-01 19:34:19 +00:00
Colin Percival
d5d7606c0c Use the TSLOG framework to record entry/exit timestamps for DELAY and
_vprintf; these functions are called in many places and can contribute
meaningfully to the total time spent booting.
2017-12-31 09:24:41 +00:00
Marius Strobl
15f0034553 With the advent of interrupt remapping, Intel has repurposed bit 11
(now: Interrupt_Index[15]) and assigned the previously reserved bits
55:48 (Interrupt_Index[14:0] goes into 63:49 while Destination Field
used 63:56 and bit 48 now is Interrupt_Format) in the IO redirection
tables (see the VT-d specification, "5.1.5.1 I/OxAPIC Programming").
Thus, when not using interrupt remapping, ensure that all previously
reserved bits in the high part of the RTEs are zero instead of doing
a read-modify-write for their Destination Field bits only.
Otherwise, on machines based on Apollo Lake and its derivatives such
as Denverton, typically some of the previously preserved bits remain
set after boot when not employing interrupt remapping. The result is
that INTx interrupts are not getting delivered.
Note: With an AMD IOMMU, interrupt remapping apparently bypasses the
IO APIC altogether.

Submitted by:	loos (modulo comment)
Reviewed by:	jhb (modulo comment)
2017-12-28 21:46:09 +00:00
Poul-Henning Kamp
8ba749fbe3 Introduce an architecture-agnostic <sys/_stdarg.h> to reduce
platform divergence.

Only architectures which pass arguments in registers (mips)
and platforms which use really weird compilers (any?) would
need to augment the contents of <sys/_stdarg.h>

Convert x86, arm and arm64 architectures to use <sys/_stdarg.h>
2017-12-25 20:54:00 +00:00
Warner Losh
ed98ce5cad Further investigation shows this shouldn't have been added at all.
Remove it.
2017-12-24 17:59:48 +00:00
Warner Losh
d76103580a Comment this out until I have time to get to the bottom of why it's
failing for some people.
2017-12-24 16:36:50 +00:00
Warner Losh
7dcb3b1295 Warn when nonPNP ISA devices are attached in GENERIC that they are
being removed from GENERIC in 12. Always print PNP info for ISA when
it exists: it doesn't depend on ISAPNP. Add PNP ID to orm and vga to
prevent us from warning about them since those devices aren't being
removed from GENERIC. PNP devices will be removed from GENERIC too,
but they will be automatically loaded, so need no warning. We don't
warn for non-GENERIC kernels because people running them are presumed
to know what they are doing.

MFC After: 2 weeks
2017-12-23 22:57:14 +00:00
Konstantin Belousov
6332b14887 Add missed AVX512VL (128 and 256 bit vector length) extension
identification bit.

Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2017-12-23 21:32:50 +00:00
Bruce Evans
da9fba5447 Use resume_cpus() instead of restart_cpus() to resume from ACPI suspension.
restart_cpus() worked well enough by accident.  Before this set of fixes,
resume_cpus() used the same cpuset (started_cpus, meaning CPUs directed to
restart) as restart_cpus().  resume_cpus() waited for the wrong cpuset
(stopped_cpus) to become empty, but since mixtures of stopped and suspended
CPUs are not close to working, stopped_cpus must be empty when resuming so
the wait is null -- restart_cpus just allows the other CPUs to restart and
returns without waiting.

Fix resume_cpus() to wait on a non-wrong cpuset for the ACPI case, and
add further kludges to try to keep it working for the XEN case.  It
was only used for XEN.  It waited on suspended_cpus.  This works for
XEN.  However, for ACPI, resuming is a 2-step process.  ACPI has already
woken up the other CPUs and removed them from suspended_cpus.  This
fix records the move by putting them in a new cpuset resuming_cpus.
Waiting on suspended_cpus would give the same null wait as waiting on
stopped_cpus.  Wait on resuming_cpus instead.

Add a cpuset toresume_cpus to map the CPUs being told to resume to keep
this separate from the cpuset started_cpus for mapping the CPUs being told
to restart.  Mixtures of stopped and suspended/resuming CPUs are still far
from working.  Describe new and some old cpusets in comments.

Add further kludges to cpususpend_handler() to try to avoid breaking it
for XEN.  XEN doesn't use resumectx(), so it doesn't use the second
return path for savectx(), and it goes from the suspended state directly
to the restarted state, while ACPI resume goes through the resuming state.
Enter the resuming state early for all cases so that resume_cpus can test
for being in this state and not have to worry about the intermediate
!suspended state for ACPI only.

Reviewed by:	kib
2017-12-21 09:17:48 +00:00
Bruce Evans
2ba6fe0009 Remove the permanent double mapping of low physical memory and replace
it by a transient double mapping for the one instruction in ACPI wakeup
where it is needed (and for many surrounding instructions in ACPI resume).
Invalidate the TLB as soon as convenient after undoing the transient
mapping.  ACPI resume already has the strict ordering needed for this.

This fixes the non-trapping of null pointers and other garbage pointers
below NBPDR (except transiently).  NBPDR is quite large (4MB, or 2MB for
PAE).

This fixes spurious traps at the first instruction in VM86 bioscalls.
The traps are for transiently missing read permission in the first
VM86 page (physical page 0) which was just written to at KERNBASE in
the kernel.  The mechanism is unknown (it is not simply PG_G).

locore uses a similar but larger transient double mapping and needs
it for 2 instructions instead of 1.  Unmap the first PDE in it after
the 2 instructions to detect most garbage pointers while bootstrapping.
pmap_bootstrap() finishes the unmapping.

Remove the avoidance of the double mapping for a recently fixed special
case.  ACPI resume could use this avoidance (made non-special) to avoid
any problems with the transient double mapping, but no such problems
are known.

Update comments in locore.  Many were for old versions of FreeBSD which
tried to map low memory r/o except for special cases, or might have
allowed access to low memory via physical offsets.  Now all kernel
maps are r/w, and removal of of the double map disallows use of physical
offsets again.
2017-12-18 13:53:22 +00:00
Pedro F. Giffuni
64de3fdd58 SPDX: use the Beerware identifier. 2017-11-30 20:33:45 +00:00
Jung-uk Kim
82f0844956 Properly skip the first CPU. It only accidentally worked because the
CPU_FOREACH() loop always starts from BSP (cpu0) and the if condition
is always false for APs.

Reported by:	cem
2017-11-30 20:21:42 +00:00
Pedro F. Giffuni
8820ecc040 SPDX: Fix some cases wrongly attributed to MIT.
In the cases of BSD-style license variants without clauses, use 0BSD for
the time being in lack of a better description.
2017-11-30 15:10:11 +00:00
Jung-uk Kim
e374a321fe Add a tunable "debug.hwpstate_verify" to check P-state after changing it and
turn it off by default.  It is very inefficient to verify current P-state of
each core, especially for CPUs with many cores.  When multiple commands are
requested to the same power domain before completion of pending transitions,
the last command is executed according to the manual.  Because requests are
serialized by the caller, all cores will receive the same command for each
call.  Do not call sched_bind() and sched_unbind().  It is redundant because
the caller does it anyway.
2017-11-30 01:40:07 +00:00
Jung-uk Kim
72b27e9773 Fix style(9). 2017-11-29 23:52:31 +00:00
Pedro F. Giffuni
ebf5747bdb sys/x86: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:11:47 +00:00
Konstantin Belousov
383f241dce Remove lint support from system headers and MD x86 headers.
Reviewed by:	dim, jhb
Discussed with:	imp
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D13156
2017-11-23 11:40:16 +00:00
Pedro F. Giffuni
51369649b0 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:43:44 +00:00
Pedro F. Giffuni
df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Ruslan Bukin
3b418d1b9a Add Intel Processor Trace registers for:
- CPUID
- Table of Physical Addresses (ToPA).

Sponsored by:	DARPA, AFRL
2017-11-17 17:54:10 +00:00
Konstantin Belousov
4e421792ec Remove i386 XBOX support.
It is for console presented at 2001 and featuring Pentium III
processor.  Even if any of them are still alive and run FreeBSD, we do
not have any sign of life from their users.  While removing another
dozens of #ifdefs from the i386 sources reduces the aversion from
looking at the code and improves the platform vitality.

Reviewed by:	cem, pfg, rink (XBOX support author)
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D13016
2017-11-16 14:27:02 +00:00
Ruslan Bukin
b510dab312 Add Intel Processor Trace (PT) MSRs.
Sponsored by:	DARPA, AFRL
2017-11-12 23:13:04 +00:00
Konstantin Belousov
dc00696a27 Correct operators precedence.
Also keep the calculated vm_page_alloc_contig() flags in the variable
to not re-evaluate it on the loop iteration.

Noted by:	alc
Sponsored by:	The FreeBSD Foundation
2017-11-09 13:09:07 +00:00
Jeff Roberson
8d6fbbb867 Replace manyinstances of VM_WAIT with blocking page allocation flags
similar to the kernel memory allocator.

This simplifies NUMA allocation because the domain will be known at wait
time and races between failure and sleeping are eliminated.  This also
reduces boilerplate code and simplifies callers.

A wait primitive is supplied for uma zones for similar reasons.  This
eliminates some non-specific VM_WAIT calls in favor of more explicit
sleeps that may be satisfied without new pages.

Reviewed by:	alc, kib, markj
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
2017-11-08 02:39:37 +00:00
Michal Meloun
904d8c492f Add AT_HWCAP2 ELF auxiliary vector.
- allocate value for new AT_HWCAP2 auxiliary vector on all platforms.
 - expand 'struct sysentvec' by new 'u_long *sv_hwcap2', in exactly
   same way as for AT_HWCAP.

MFC after:	1 month
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D12699
2017-10-21 12:05:01 +00:00
Conrad Meyer
194446f9b7 x86: Decode AMD "Extended Feature Extensions ID EBX" bits
In particular, this determines CPU support for the CLZERO instruction.

(No, I am not making this name up.)

Sponsored by:	Dell EMC Isilon
2017-09-20 18:30:37 +00:00
Conrad Meyer
c50df68a08 MCA: Expand AMD Thresholding support to cover all banks
When it was added in r314636, AMD Thresholding was hardcoded to only
bank 4 (Northbridge) for some reason.  However, even on family 10h the
MCAx_MISC register Valid/Present bits determine whether thresholding is
supported on that bank.

Expand thresholding support to monitor all monitorable banks.  This
simplifies some of the logic and makes it more consistent with our Intel
CMCI support.

Reviewed by:	markj (earlier version)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12321
2017-09-17 22:58:13 +00:00
John Baldwin
8df419f2df Add AT_EHDRFLAGS and AT_HWCAP on amd64.
x86 has two separate (but identical) list of AT_* constants and the
earlier commit to add AT_HWCAP only updated the i386 list.
2017-09-14 15:34:29 +00:00
John Baldwin
c2f37b9245 Add AT_HWCAP and AT_EHDRFLAGS on all platforms.
A new 'u_long *sv_hwcap' field is added to 'struct sysentvec'.  A
process ABI can set this field to point to a value holding a mask of
architecture-specific CPU feature flags.  If an ABI does not wish to
supply AT_HWCAP to processes the field can be left as NULL.

The support code for AT_EHDRFLAGS was already present on all systems,
just the #define was not present.  This is a step towards unifying the
AT_* constants across platforms.

Reviewed by:	kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D12290
2017-09-14 14:26:55 +00:00
Conrad Meyer
d63edb4dc6 MCA: Rename AMD MISC bits/masks
They apply to all AMD MCAi_MISC0 registers, not just MCA4 (NB).

No functional change.

Sponsored by:	Dell EMC Isilon
2017-09-11 20:42:07 +00:00
Conrad Meyer
f739be66e6 x86 MCA: Extract CMCI support predicate into function
On AMD, the MCG_CAP feature bit is reserved -- not explicitly zero.  Do not
use it to determine CMCI support.

Reviewed by:	avg, markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12320
2017-09-11 20:41:25 +00:00
Konstantin Belousov
809f2d8b8b Fix ioapic acpi id matching on PCI attach and rid calculation.
Sponsored by:	The FreeBSD Foundation
MFC after:	11 days
2017-09-11 18:29:09 +00:00
Conrad Meyer
e8be4e41c6 Decode new AMD SVM feature bits on family 17h
Sponsored by:	Dell EMC Isilon
2017-09-11 18:11:53 +00:00
Konstantin Belousov
3c700e2e4c Enhance qpi.c to make it usable on all Core-microarchitecture Xeons.
Scan all buses for CSR bus, not stopping on the first failed
match. Scan all slots for function 0 on the found bus, for instance on
IvyBridge the slot 0 is not decoded at all. Since the scan is quite
unsafe, and access to the buses is mostly useful for developers,
enable the csr buses scan with the tunable.

Current qpi.c makes too many assumptions about the uncore
configuration buses location and about slots occupied.  Also it
restricts itself only to Nehalem CPUs.  It is needed on all Core-based
Xeons.  On the 2600 v2 (IvyBridge) machine I have access to, the CSR
buses have numbers 31 (BSP socket) and 63 (second socket), and there
is no functions pci0.31.0.0 or pci0.63.0.0.  According to the CPU
datasheet, all devices on the uncore bus occupy slots >= 8.

Practically, the attach to config buses is required for the intel-pcm
pcm-memory.x tool to work, for instance.

Reviewed by:	jhb (previous version)
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D12268
2017-09-08 19:51:03 +00:00
Konstantin Belousov
fd15fee1ed Use IOAPIC PCI rid as the interrupt TLP source id for DMAR interrupt
remapping.

VT-d specification requires use of PCI rid as source id for IOAPICs
enumerated by PCI bus.  The values from the DMAR ACPI table should be
only used when IOAPIC is not on PCI.

Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
Hardware provided by:	Intel
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D12205
2017-09-08 19:45:37 +00:00
Konstantin Belousov
3fd0053a50 Add an ioapic_get_rid() function to obtain PCIe TLP requester-id for
the interrupt messages from given IOAPIC, if the IOAPIC can be
enumerated on PCI bus.

If IOAPIC has PCI binding, match the PCI device against MADT
enumerated IOAPIC.  Match is done first by registers window physical
address, then by IOAPIC ID as read from the APIC ID register.

PCI bsf address of the matched PCI device is the rid.

Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
Hardware provided by:	Intel
MFC after:	2 weeks
X-Differential revision:	https://reviews.freebsd.org/D12205
2017-09-08 19:39:20 +00:00
Konstantin Belousov
1a92c8402d Add a constant specifying the min size of the IOAPIC registers window.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-09-08 19:25:11 +00:00
Konstantin Belousov
6ff9ce94ce Consistently use tabs for indent.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-09-08 10:39:28 +00:00
Conrad Meyer
01a20b9875 mca: Fix printf types from r323289 on i386
Reported by:	Michael Butler <imb AT protected-networks.net>
Sponsored by:	Dell EMC Isilon
2017-09-08 01:06:35 +00:00
Conrad Meyer
092c0e867a x86 MCA: Helpfully, print why ECC thresholding is not enabled on AMD
Sponsored by:	Dell EMC Isilon
2017-09-07 21:33:27 +00:00
Conrad Meyer
d848ecfb7e x86 MCA: Enable AMD thresholding support on 17h
17h supports MCA thresholding in the same way as 16h and earlier.
Supposedly a ScalableMca feature bit in CPUID 8000_0007:EBX must be set, but
that was not true for earlier models, so be careful about relying on it.

While here, document a missing bit in LS MCA MISC0.

Reviewed by:	truckman
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12237
2017-09-07 21:31:07 +00:00