1000 Commits

Author SHA1 Message Date
kib
69f7c0ee63 Add support for Hygon Dhyana Family 18h processor.
As a new x86 CPU vendor, Chengdu Haiguang IC Design Co., Ltd (Hygon)
is a joint venture between AMD and Haiguang Information Technology Co.,
Ltd., aims at providing x86 processors for China server market.

The first generation Hygon processor(Dhyana) shares most architecture
with AMD's family 17h, but with different CPU vendor ID("HygonGenuine")
and PCI vendor ID(0x1d94) and family series number 18h(Hygon negotiated
with AMD to confirm that only Hygon use family 18h).

To enable Hygon Dhyana support in FreeBSD, add new definitions
HYGON_VENDOR_ID("HygonGenuine") and X86_VENDOR_HYGON(0x1d94) to identify
Hygon Dhyana CPU.

Initialize the CPU features(topology, local APIC ext, MSI, TSC, hwpstate,
MCA, DEBUG_CTL, etc) for amd64 and i386 mode by sharing the code path of
AMD family 17h.

The changes have been applied on FreeBSD 13.0-CURRENT and tested
successfully on Hygon Dhyana processor.

References:
[1] Linux kernel patches for Hygon Dhyana, merged in 4.20:

https://git.kernel.org/tip/c9661c1e80b609cd038db7c908e061f0535804ef

[2] MSR and CPUID definition:

https://www.amd.com/system/files/TechDocs/54945_PPR_Family_17h_Models_00h-0Fh.pdf

Submitted by:	Pu Wen <puwen@hygon.cn>
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D23163
2020-01-21 13:22:35 +00:00
kib
85db09767d x86: Wait for curthread to be set up as an indicator that the boot stack
is no longer used.

pc_curthread is set by cpu_switch after it stopped using the old
thread (or boot) stack.  This makes the smp_after_idle_runnable()
function not dependent on the internals of the scheduler operations.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D23276
2020-01-20 17:23:03 +00:00
mjg
a5ea665498 x86: fix assertion in ipi_send_cpu to range check the passed id
Prior to the change for sufficiently bad id (and in particular NOCPU which is -1)
it would access memory outside of the cpu_apic_ids array.
2020-01-19 21:35:51 +00:00
mjg
165ba25434 Add KERNEL_PANICKED macro for use in place of direct panicstr tests 2020-01-12 06:07:54 +00:00
scottl
15e06ff0d9 Introduce the concept of busdma tag templates. A template can be allocated
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created.  See the man page for
details on how to initialize a template.

Templates do not support tag filters.  Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine.  Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.

Reviewed by:	imp, kib
Differential Revision:	https://reviews.freebsd.org/D22906
2019-12-24 14:48:46 +00:00
rlibby
dbf795e374 bitset: rename confusing macro NAND to ANDNOT
s/BIT_NAND/BIT_ANDNOT/, and for CPU and DOMAINSET too.  The actual
implementation is "and not" (or "but not"), i.e. A but not B.
Fortunately this does appear to be what all existing callers want.

Don't supply a NAND (not (A and B)) operation at this time.

Discussed with:	jeff
Reviewed by:	cem
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D22791
2019-12-13 09:32:16 +00:00
scottl
3df8998871 Fix the TAA state machine to do the right thing when the TAA
migitation is available in microcode and the operator has set
the sysctl to automatic mode.

Reported by:	Coverity
CID: 1408334

MFC after:	3 days
Sponsored by:	Intel
2019-12-10 18:57:39 +00:00
kib
8fca58390a x86: Restore the critical section around whole ipi_bitmap_handler() if
hardclock IPI is delivered.

In the current code after r355311, critical section is taken only
around hardclockintr() call, and sched_preempt() is called after the
section is exited. If we reschedule after exit, as we typically would
due to conditions that caused IPI, in ULE the runq tdq_ipipending is
not cleared, which blocks generation of further preempt IPIs.

Since all relatively modern (10 years) hardware has per-cpu event
timers, restoring the critical section conditionally does not affect
it.

Reported and tested by: cy
Diagnosed and reviewed by: jeff (previous version)
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D22716
2019-12-07 00:28:08 +00:00
scottl
ed396a5316 Move the mds, irbs, and ssb mitigation knobs into machdep.mitigations.
They're in both the old and new places in HEAD for the moment for
discussion and transition.  The old locations will be garbage collected
in 4 weeks.  MFCs to 12 an 11 will keep the old and new for transition
purposes.

Reviewed by:	kib
MFC after:	4 weeks
Sponsored by:	Intel
Differential Revision:	https://reviews.freebsd.org/D22590
2019-12-06 02:43:05 +00:00
cem
81483ccf9b x86: Add missed break to TAA status sysctl
Just a typo that Coverity identified.

Coverity also identified an unused store in the same functional area (x86 TAA
stuff), but this commit does not address that issue (CID 1408334).

Reported by:	Coverity
CID:		1408328, 1408332
2019-12-04 02:42:22 +00:00
jeff
412097ad50 Fix a few places that free a page from an object without busy held. This is
tightening constraints on busy as a precursor to lockless page lookup and
should largely be a NOP for these cases.

Reviewed by:	alc, kib, markj
Differential Revision:	https://reviews.freebsd.org/D22611
2019-12-02 22:42:05 +00:00
jeff
f52af54850 Don't run sched_preempt() inside of an extra critical section. This disables
the sched_preempt() switch optimization and causes the sched lock to be dropped
and immediately reacquired.

Reviewed by:	jhb, kib, mav, markj (with changes)
Differential Revision:	https://reviews.freebsd.org/D22623
2019-12-02 22:34:19 +00:00
kib
6c3d8ccf87 bus_dma_dmar_load_ident(9): load identity mapping into the map.
Requested, reviewed and tested by:	mav
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D22559
2019-11-27 19:57:17 +00:00
scottl
c7ed5211be Clean up and clarify meta commentary on TAA. Add a state to denote
that TSX doesn't exist on the CPU.

MFC after:	3 days
Sponsored by:	Intel
2019-11-27 19:12:32 +00:00
kib
9da47925f0 Limit bus_dma_dmar_set_buswide() definition to kernel only.
The header is abused for inclusion into userspace, and on stable
branches neither device_t nor bool types are not defined when used
from userspace.

Sponsored by:	The FreeBSD Foundation
X-MFC after:	now
2019-11-25 14:16:41 +00:00
andrew
6e5970c8f4 Port the NetBSD KCSAN runtime to FreeBSD.
Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in
the FreeBSD kernel. It is a useful tool for finding data races between
threads executing on different CPUs.

This can be enabled by enabling KCSAN in the kernel config, or by using the
GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later
needs a compiler change to allow -fsanitize=thread that KCSAN uses.

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22315
2019-11-21 11:22:08 +00:00
kib
b5677bab60 bus_dma_dmar_set_buswide(9): KPI to indicate that the whole dmar
context should share page tables.

Practically it means that dma requests from any device on the bus are
translated according to the entries loaded for the bus:0:0 device.
KPI requires that the slot and function of the device be 0:0, and that
no tags for other devices on the bus were used.

The intended use are NTBs which pass TLPs from the downstream to the
host with slot:func of the downstream originator.

Reviewed and tested by:	mav
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D22434
2019-11-18 20:56:59 +00:00
kib
4387bf78c9 Add x86 msr tweak KPI.
Use the KPI to tweak MSRs in mitigation code.

Reviewed by:	markj, scottl
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D22431
2019-11-18 20:53:57 +00:00
scottl
3a09fa0468 TSX Asynchronous Abort mitigation for Intel CVE-2019-11135.
This CVE has already been announced in FreeBSD SA-19:26.mcu.

Mitigation for TAA involves either turning off TSX or turning on the
VERW mitigation used for MDS. Some CPUs will also be self-mitigating
for TAA and require no software workaround.

Control knobs are:
machdep.mitigations.taa.enable:
        0 - no software mitigation is enabled
        1 - attempt to disable TSX
        2 - use the VERW mitigation
        3 - automatically select the mitigation based on processor
	    features.

machdep.mitigations.taa.state:
        inactive        - no mitigation is active/enabled
        TSX disable     - TSX is disabled in the bare metal CPU as well as
                        - any virtualized CPUs
        VERW            - VERW instruction clears CPU buffers
	not vulnerable	- The CPU has identified itself as not being
			  vulnerable

Nothing in the base FreeBSD system uses TSX.  However, the instructions
are straight-forward to add to custom applications and require no kernel
support, so the mitigation is provided for users with untrusted
applications and tenants.

Reviewed by:	emaste, imp, kib, scottph
Sponsored by:	Intel
Differential Revision:	22374
2019-11-16 00:26:42 +00:00
scottl
609b5798d2 Revert a patch that accidentally was committed with r354729 2019-11-15 11:54:51 +00:00
scottl
3eb06a2526 Fix a typo in how the AVX512DQ feature bit is checked.
Reviewed by:	kib
Sponsored by:	Intel
2019-11-15 11:53:06 +00:00
scottl
04cfdf0426 Add new bit definitions for TSX, related to the TAA issue. The actual
mitigation will follow in a future commit.

Sponsored by:	Intel
2019-11-12 19:15:16 +00:00
kib
f6d6546684 Workaround for Intel SKL002/SKL012S errata.
Disable the use of executable 2M page mappings in EPT-format page
tables on affected CPUs.  For bhyve virtual machines, this effectively
disables all use of superpage mappings on affected CPUs.  The
vm.pmap.allow_2m_x_ept sysctl can be set to override the default and
enable mappings on affected CPUs.

Alternate approaches have been suggested, but at present we do not
believe the complexity is warranted for typical bhyve's use cases.

Reviewed by:	alc, emaste, markj, scottl
Security:	CVE-2018-12207
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D21884
2019-11-12 18:01:33 +00:00
royger
f106f64dc9 xen: fix dispatching of NMIs
Currently NMIs are sent over event channels, but that defeats the
purpose of NMIs since event channels can be masked. Fix this by
issuing NMIs using a hypercall, which injects a NMI (vector #2) to the
desired vCPU.

Note that NMIs could also be triggered using the emulated local APIC,
but using a hypercall is better from a performance point of view
since it doesn't involve instruction decoding when not using x2APIC
mode.

Reported and Tested by:	avg
Sponsored by:		Citrix Systems R&D
2019-11-12 10:31:28 +00:00
scottl
dfad9d3698 Add the text attribute for MDS_NO in the IA32_ARCH_CAP MSR. 2019-11-11 22:18:05 +00:00
avg
dd5d23274b revert r354482, checking for XENHVM was a wrong way of checking for Xen 2019-11-07 21:43:31 +00:00
avg
cf471e1451 IPI_TRACE is not really supported on xen
x86 stack_save_td_running() can work safely only if IPI_TRACE is a
non-maskable interrupt.  But at the moment FreeBSD/Xen does not provide
support for the NMI delivery mode.  So, mark the functionality as
unsupported similarly to other platforms without NMI.
Maybe there is a way to provide a Xen-specific working
stack_save_td_running(), but I couldn't figure it out.

MFC after:	3 weeks
Sponsored by:	Panzura
2019-11-07 21:14:59 +00:00
gallatin
45d526df8c Add tunable to allow interrupts on hyperthreaded cores
Enabling interrupts on htt cores has benefits to workloads which are primarily
interrupt driven by increasing the logical cores available for interrupt handling.
The tunable is named machdep.hyperthreading_intr_allowed

Reviewed by:	kib, jhb
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D22233
2019-11-04 19:30:19 +00:00
cem
3d40147282 amd64: Fix typo: RDPRU bit is 0x10, not 0x04
Bit 4 != 4, of course.

X-MFC-With:	r354162
2019-10-30 04:00:44 +00:00
cem
4de94d5356 amd64: Define and decode new AMD64 feature bits
These are documented in revisions 3.32 of the public AMD64 Vol. 2 and
revision 3.28 of Vol. 3, published October and September 2019, respectively.
2019-10-30 01:41:14 +00:00
cem
5e4aa6b57b hw.intrbalance: Make sysctl tunable
This allows specifying a boot-time preference in loader.conf.
2019-10-19 16:37:49 +00:00
avg
c9402a53ed remove wmb() call from x86 cpu_reset()
The rationale is pretty much the same as in r353747.
There is no subsequent dependent store.
The store is to the regular (TSO) memory anyway.

MFC after:	23 days
2019-10-19 07:13:15 +00:00
cem
22adb9b140 x86: Remove unused variable from r353712
It was in my git tree (uncommitted) and didn't get carried over to SVN in
r353712.

X-MFC-With:	r353712
2019-10-18 02:25:30 +00:00
cem
6487f370d2 x86: Fetch and save standard CPUID leaf 6 in identcpu
Rather than a few scattered places in the tree.  Organize flag names in a
contiguous region of specialreg.h.

While here, delete deprecated PCOMMIT from leaf 7.

No functional change.
2019-10-18 02:18:17 +00:00
cem
7c575799c3 x86: Use canonical spelling of MOVDIR64B feature/instruction
The former spelling probably confused MOVDIR64B with MOVDIRI64.

MOVDIR_64B is the 64-*byte* direct store instruction; MOVDIR_I64 is the
64-*bit* direct store instruction (underscores added here for clarity; they are
not part of the canonical instruction name).

No functional change.

Sponsored by:	Dell EMC Isilon
2019-10-14 20:55:01 +00:00
mjg
b68ee60bab amd64: plug spurious cld instructions
ABI already guarantees the direction is forward. Note this does not take care
of i386-specific cld's.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21906
2019-10-08 21:14:11 +00:00
vangyzen
744f502c8e Make the hw.intrs sysctl OID read-only
The handler ignores the new value, so make the OID read-only.

I found this while working on r353111.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2019-10-04 21:46:11 +00:00
markj
fbe7e9c7e4 Complete the removal of the "wire_count" field from struct vm_page.
Convert all remaining references to that field to "ref_count" and update
comments accordingly.  No functional change intended.

Reviewed by:	alc, kib
Sponsored by:	Intel, Netflix
Differential Revision:	https://reviews.freebsd.org/D21768
2019-09-25 16:11:35 +00:00
kib
c6f3c4a8cf x86: Fall back to leaf 0x16 if TSC frequency is obtained by CPUID and
leaf 0x15 is not functional.

This should improve automatic TSC frequency determination on
Skylake/Kabylake/... families, where 0x15 exists but does not provide
all necessary information.  SDM contains relatively strong wording
against such uses of 0x16, but Intel does not give us any other way to
obtain the frequency. Linux did the same in the commit
604dc9170f2435d27da5039a3efd757dceadc684.

Based on submission by:	Neel Chauhan <neel@neelc.org>
PR:	240475
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21777
2019-09-25 13:36:56 +00:00
jeff
685a292036 Move phys_avail definition into MI code. It is consumed in the MI layer and
doing so adds more flexibility with less redundant code.

Reviewed by:	jhb, markj, kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D21250
2019-08-16 00:45:14 +00:00
jhb
c423c7a497 Stop listing "on motherboard" as the parent of nexus devices on x86.
Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D21256
2019-08-14 22:13:11 +00:00
emaste
d3067c6e6a sys/{x86,amd64}: remove one of doubled ;s
MFC after:	1 week
2019-08-13 19:39:36 +00:00
imp
75e675195a Floppy driver really only works on x86
Move the floppy driver to the x86 specific notes file.

Reviewed by: jhb, manu, jhibbits, emaste
Differential Revision: https://reviews.freebsd.org/D21208
2019-08-12 22:58:50 +00:00
imp
c0702cb45a Move sc out of the global file
x86 needs sc, as does sparc64. powerpc doesn't use it by default, but some old
powermac notebooks do not work with vt yet for reasons unknonw. Even so, I've
removed it from powerpc LINT. It's not in daily use there, and the intent is to
100% switch to vt now that it works for that platform to limit support burden.

All the other architectures omit some or all of the screen savers from their
lint config. Move them to the x86 NOTES files and remove the exclusions. This
reduces slightly the number of savers sparc64 compiles, but since they are in
GENERIC, the overage is adequate and if someone reaelly wants to sort them out
in sparc64 they can sweat the details and the testing.

Reviewed by: jhb (earlier version), manu (earlier version), jhibbits
Differential Revision: https://reviews.freebsd.org/D21233
2019-08-12 22:58:44 +00:00
imp
20fde27184 Start to split out the really x86 specific NOTES from the global notes file.
Start with COMPAT_43, since it's really only relevant to x86.

Reviewed by: jhb@
Differential Revision: https://reviews.freebsd.org/D21203
2019-08-12 22:58:13 +00:00
kib
4aa27a3b56 PR: 239143
Reported and tested by:	Wes Maag <jwmaag@gmail.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-07-14 21:08:54 +00:00
kib
66a23058ee Ensure that mds_handler always points to a valid method.
Depending on system configuration, version, and architecture,
mds_handler might be dereferenced from doreti before
hw_mds_recalculate_boot() initialized it.  Statically assign void
method to cover all cases.

Reported by:	"Schuendehuette, Matthias (LDA IT PLM)" <matthias.schuendehuette@siemens.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2019-07-11 16:22:49 +00:00
jtl
c4bbb0fa7d Currently, MCA entries remain on an every-growing linked list. This means
that it becomes increasingly expensive to process a steady stream of
correctable errors. Additionally, the memory used by the MCA entries can
grow without bound.

Change the code to maintain two separate lists: a list of entries which
still need to be logged, and a list of entries which have already been
logged. Additionally, allow a user-configurable limit on the number of
entries which will be saved after they are logged. (The limit defaults
to -1 [unlimited], which is the current behavior.)

Reviewed by:	imp, jhb
MFC after:	2 weeks
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20482
2019-06-08 18:26:48 +00:00
tychon
8f4f39f989 another occurrence where a very large dma mapping can cause integer overflow
Submitted by:	rlibby
Sponsored by:	Dell EMC Isilon
2019-06-05 13:08:21 +00:00
tychon
e0b11c4354 very large dma mappings can cause integer overflow
Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20505
2019-06-03 19:19:35 +00:00