Commit Graph

78 Commits

Author SHA1 Message Date
royger
e45f85c53f x86: make the arrays that depend on MAX_APIC_ID dynamic
So that MAX_APIC_ID can be bumped without wasting memory.

Note that the usage of MAX_APIC_ID in the SRAT parsing forces the
parser to allocate memory directly from the phys_avail physical memory
array, which is not the best approach probably, but I haven't found
any other way to allocate memory so early in boot. This memory is not
returned to the system afterwards, but at least it's sized according
to the maximum APIC ID found in the MADT table.

Sponsored by:		Citrix Systems R&D
MFC after:		1 month
Reviewed by:		kib
Differential revision:	https://reviews.freebsd.org/D11912
2017-08-10 09:16:03 +00:00
royger
345fb32684 apic_enumerator: only set mp_ncpus and mp_maxid at probe cpus phase
Populate the lapics arrays and call cpu_add/lapic_create in the setup
phase instead. Also store the max APIC ID found in the newly
introduced max_apic_id global variable.

This is a requirement in order to make the static arrays currently
using MAX_LAPIC_ID dynamic.

Sponsored by:		Citrix Systems R&D
MFC after:		1 month
Reviewed by:		kib
Differential revision:	https://reviews.freebsd.org/D11911
2017-08-10 09:15:18 +00:00
kib
a17a213fbb More accurately handle early EFER restoration on resume.
Do not try to set LMA bit while CPU is still in legacy mode.
Apparently Intel CPUs ignore non-id writes to LMA, while AMD's
(over-)react with #GP.

Reported and tested by:	danfe
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2017-06-11 14:39:08 +00:00
kib
542fd28222 Ensure that resume path on amd64 only accesses page tables for normal
operation after processor is configured to allow all required
features.

In particular, NX must be enabled in EFER, otherwise load of page
table element with nx bit set causes reserved bit page fault.  Since
malloc uses direct mapping for small allocations, in particular for
the suspension pcbs, and DMAP is nx after r316767, this commit tripped
fault on resume path.

Restore complete state of EFER while wakeup code is still executing
with custom page table, before calling resumectx, instead of trying to
guess which features might be needed before resumectx restored EFER on
its own.

Bisected and tested by:	trasz
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2017-05-15 20:52:43 +00:00
glebius
21ead51d79 - Remove 'struct vmmeter' from 'struct pcpu', leaving only global vmmeter
in place.  To do per-cpu stats, convert all fields that previously were
  maintained in the vmmeters that sit in pcpus to counter(9).
- Since some vmmeter stats may be touched at very early stages of boot,
  before we have set up UMA and we can do counter_u64_alloc(), provide an
  early counter mechanism:
  o Leave one spare uint64_t in struct pcpu, named pc_early_dummy_counter.
  o Point counter(9) fields of vmmeter to pcpu[0].pc_early_dummy_counter,
    so that at early stages of boot, before counters are allocated we already
    point to a counter that can be safely written to.
  o For sparc64 that required a whole dummy pcpu[MAXCPU] array.

Further related changes:
- Don't include vmmeter.h into pcpu.h.
- vm.stats.vm.v_swappgsout and vm.stats.vm.v_swappgsin changed to 64-bit,
  to match kernel representation.
- struct vmmeter hidden under _KERNEL, and only vmstat(1) is an exclusion.

This is based on benno@'s 4-year old patch:
https://lists.freebsd.org/pipermail/freebsd-arch/2013-July/014471.html

Reviewed by:	kib, gallatin, marius, lidl
Differential Revision:	https://reviews.freebsd.org/D10156
2017-04-17 17:34:47 +00:00
royger
4c1b412f1d x86/srat: fix parsing of APIC IDs > MAX_APIC_ID
Ignore them like it's done in the MADT parser. This allows booting on a box
with SRAT and APIC IDs > 255.

Reported by:	Wei Liu <wei.liu2@citrix.com>
Tested by:	Wei Liu <wei.liu2@citrix.com>
Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	Citrix Systems R&D
2017-03-16 09:33:36 +00:00
kib
5cb41cd56e For i386, remove config options CPU_DISABLE_CMPXCHG, CPU_DISABLE_SSE
and device npx.

This means that FPU is always initialized and handled when available,
and SSE+ register file and exception are handled when available.  This
makes the kernel FPU code much easier to maintain by the cost of
slight bloat for CPUs older than 25 years.

CPU_DISABLE_CMPXCHG outlived its usefulness, see the removed comment
explaining the original purpose.

Suggested by and discussed with:	bde
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	3 weeks
2017-02-03 12:51:40 +00:00
kib
16f58155c2 Prefix hex memory addresses with 0x in diagnostic messages from the
SRAT parser.

Submitted by:	Oliver Pinter
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D8750
2016-12-11 19:01:27 +00:00
jkim
7706f244fc Merge ACPICA 20160930. 2016-10-04 20:27:15 +00:00
kib
3f4c126b42 Detect x2APIC mode on boot and obey it.
If BIOS performed hand-off to OS with BSP LAPIC in the x2APIC mode,
system usually consumes such configuration without a notice, since
x2APIC is turned on by OS if possible (nop).  But if BIOS
simultaneously requested OS to not use x2APIC, code assumption that
that xAPIC is active breaks.

In my opinion, we cannot safely turn off x2APIC if control is passed
in this mode.  Make madt.c ignore user or BIOS requests to turn x2APIC
off, and do not check the x2APIC black list.  Just trust the config
and try to continue, giving a warning in dmesg.

Reported and tested by:	Slawa Olhovchenkov <slw@zxy.spb.ru> (previous version)
Diagnosed by and discussed with:	avg
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2016-09-19 15:58:45 +00:00
markj
ce1a3c9ce1 Allow ACPI wakeup code and page tables to be stored in non-contiguous pages.
Since these pages are allocated from a narrow range of memory, this makes
the allocation more likely to succeed.

Suggested by:	kib
Reviewed by:	jkim, kib
MFC after:	2 months
Differential Revision:	https://reviews.freebsd.org/D7154
2016-07-14 00:38:04 +00:00
markj
8ac07d0f79 Use M_NOWAIT when allocating memory for the ACPI wakeup handler.
If the allocation attempt fails, we may otherwise VM_WAIT after a failed
attempt to reclaim contiguous memory in the requested range. After r297466,
this results in the thread going to sleep, causing a hang during boot.

Reviewed by:	jkim, kib
Approved by:	re (gjb)
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D6945
2016-06-23 19:24:38 +00:00
vangyzen
9474ad68a7 Work around (ignore) broken SRAT tables
Instead of panicking when parsing an invalid ACPI SRAT table,
just ignore it, effectively disabling NUMA.

https://lists.freebsd.org/pipermail/freebsd-current/2016-May/060984.html

Reported and tested by:	 Bill O'Hanlon (bill.ohanlon at gmail.com)
Reviewed by:	jhb
MFC after:	1 week
Relnotes:	If dmesg shows "SRAT: Duplicate local APIC ID",
                try updating your BIOS to fix NUMA support.
Sponsored by:	Dell Inc.
2016-05-03 20:14:04 +00:00
jhb
c71e075efb Revert bus_get_cpus() for now.
I really thought I had run this through the tinderbox before committing,
but many places need <sys/types.h> -> <sys/param.h> for <sys/bus.h> now.
2016-05-03 01:17:40 +00:00
jhb
2da46e01a0 Add a new bus method to fetch device-specific CPU sets.
bus_get_cpus() returns a specified set of CPUs for a device.  It accepts
an enum for the second parameter that indicates the type of cpuset to
request.  Currently two valus are supported:

 - LOCAL_CPUS (on x86 this returns all the CPUs in the package closest to
   the device when DEVICE_NUMA is enabled)
 - INTR_CPUS (like LOCAL_CPUS but only returns 1 SMT thread for each core)

For systems that do not support NUMA (or if it is not enabled in the kernel
config), LOCAL_CPUS fails with EINVAL.  INTR_CPUS is mapped to 'all_cpus'
by default.  The idea is that INTR_CPUS should always return a valid set.

Device drivers which want to use per-CPU interrupts should start using
INTR_CPUS instead of simply assigning interrupts to all available CPUs.
In the future we may wish to add tunables to control the policy of
INTR_CPUS (e.g. should it be local-only or global, should it ignore
SMT threads or not).

The x86 nexus driver exposes the internal set of interrupt CPUs from the
the x86 interrupt code via INTR_CPUS.

The ACPI bus driver and PCI bridge drivers use _PXM to return a suitable
LOCAL_CPUS set when _PXM exists and DEVICE_NUMA is enabled.  They also and
the global INTR_CPUS set from the nexus driver with the per-domain set from
_PXM to generate a local INTR_CPUS set for child devices.

Reviewed by:	wblock (manpage)
Differential Revision:	https://reviews.freebsd.org/D5519
2016-05-02 18:00:38 +00:00
cem
74adfc723a SRAT: Don't overflow domain_pxm table
If we reached MAXMEMDOM, we would previously try to insert an additional
element and only detect overflow after causing (probably trivial) memory
overflow.  Instead, detect the ndomain > MAXMEMDOM case before we write past
the end.

Reported by:	Coverity
CID:		1354783
Sponsored by:	EMC / Isilon Storage Division
2016-04-20 01:10:07 +00:00
imp
4c3365f0f1 Deprecate using hints.acpi.0.rsdp to communicate the RSDP to the
system. This uses the hints mechnanism. This mostly works today
because when there's no static hints (the default), this value can be
fetched from the hint. When there is a static hints file, the hint
passed from the boot loader to the kernel is ignored, but for the BIOS
case we're able to find it anyway. However, with UEFI, the fallback
doesn't work, so we get a panic instead.

Switch to acpi.rsdp and use TUNABLE_ULONG_FETCH instead. Continue to
generate the old values to allow for transitions. In addition, fall
back to the old method if the new method isn't present.

Add comments about all this.

Differential Revision: https://reviews.freebsd.org/D5866
2016-04-14 04:59:51 +00:00
jhb
6beb82443a Add more fine-grained kernel options for NUMA support.
VM_NUMA_ALLOC is used to enable use of domain-aware memory allocation in
the virtual memory system.  DEVICE_NUMA is used to enable affinity
reporting for devices such as bus_get_domain().

MAXMEMDOM must still be set to a value greater than for any NUMA support
to be effective.  Note that 'cpuset -gd' always works if MAXMEMDOM is
enabled and the system supports NUMA.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D5782
2016-04-09 13:58:04 +00:00
skra
f4b6499ab5 As <machine/pmap.h> is included from <vm/pmap.h>, there is no need to
include it explicitly when <vm/pmap.h> is already included.

Reviewed by:	alc, kib
Differential Revision:	https://reviews.freebsd.org/D5373
2016-02-22 09:02:20 +00:00
kib
559961c499 In the SandyBridge x2APIC workaround detection code, only fetch the
environment variable when SandyBridge CPU is detected.  Reduce code
duplication.

Sponsored by:	The FreeBSD Foundation
2015-12-03 10:59:10 +00:00
adrian
1b75a44eb3 Add ASUS Sandybridge laptops to the similar x2apic disable logic
that was recently added for Lenovo laptops.

This is a prime candidate for conversion into a table and also
checking other fields like "product".

Tested:

* ASUS UX31E
2015-09-16 01:44:11 +00:00
kib
b4e34c7b42 Automatically disable x2APIC mode on SandyBridge Lenovo machines. I
believe that the bug only affects mobile CPUs, at least I did not see
other reports, but it is impossible to detect it in madt_setup_local().

While there, reduce duplication in the information strings printed
when x2APIC is auto-disabled, and do not print the line when user
manually override the setting.

Tested and reviewed by:	  royger (previous version)
Sponsored by:	The FreeBSD Foundation
2015-08-21 15:13:25 +00:00
jkim
834a59ac96 Merge ACPICA 20150619. 2015-06-18 23:14:45 +00:00
jhb
8d7fa71fca Handle X2APIC entries in the MADT for APICs with an ID < 255. At least one
BIOS has been seen to include such entries even though the relevant specs
require that X2APIC entries only be used for CPUs with an APIC ID >= 255.

This was tested on a system with "plain" local APIC entries in the MADT
to ensure no regressions, but it has not yet been tested on a system with
X2APIC entries in the MADT.  Currently such systems do not boot at all,
and with this change they might now boot correctly.

Differential Revision:	https://reviews.freebsd.org/D2521
Reviewed by:	kib
MFC after:	2 weeks
2015-06-09 10:49:40 +00:00
adrian
8ef81eaec3 Update the comments to match what the code ended up becoming.
-1 is now "no locality information available".

Sponsored by:	Norse Corp, Inc.
2015-05-15 21:33:19 +00:00
adrian
2cad336903 Add initial memory locality cost awareness to the VM, and include
a basic ACPI SLIT table parser.

For now this just exports the map via sysctl; it'll eventually be useful
to userland when there's more useful NUMA support in -HEAD.

* Add an optional mem_locality map;
* add a mapping function taking from/to domain and returning the
  relative cost, or -1 if it's not available;
* Add a very basic SLIT parser to x86 ACPI.

Differential Revision:	https://reviews.freebsd.org/D2460
Reviewed by:	rpaulo, stas, jhb
Sponsored by:	Norse Corp, Inc (hardware, coding); Dell (hardware)
2015-05-08 00:56:56 +00:00
jhb
e4683250d1 Reassign copyright statements on several files from Advanced
Computing Technologies LLC to Hudson River Trading LLC.

Approved by:	Hudson River Trading LLC (who owns ACT LLC)
MFC after:	1 week
2015-04-23 14:22:20 +00:00
kib
3278a55c03 Add config option PAE_TABLES for the i386 kernel. It switches pmap to
use PAE format for the page tables, but does not incur other
consequences of the full PAE config.  In particular, vm_paddr_t and
bus_addr_t are left 32bit, and max supported memory is still limited
by 4GB.

The option allows to have nx permissions for memory mappings on i386
kernel, while keeping the usual i386 KBI and avoiding the kernel data
sizing problems typical for the PAE config.

Intel documented that the PAE format for page tables is available
starting with the Pentium Pro, but it is possible that the plain
Pentium CPUs have the required support (Appendix H).  The goal is to
enable the option and non-exec mappings on i386 for the GENERIC
kernel.  Anybody wanting a useful system on 486, have to reconfigure
the modern i386 kernel anyway.

Discussed with:	alc, jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2015-04-13 15:22:45 +00:00
jkim
0b04dab420 Fix build on i386.
Reported by:	bz
2015-04-12 22:40:27 +00:00
kib
df1d9177e5 For now, disable x2APIC mode when Xen is detected, even if CPU
declares support for it.  Newer versions of Xen works fine with x2APIC
code, but e.g. Xen 4.2 delivers GPF on the LAPIC MSR write, despite
x2APIC mode being known to hypervisor.

Discussed with:	royger
Sponsored by:	The FreeBSD Foundation
2015-02-25 16:44:07 +00:00
tijl
1d2f909a1d Fix build on i386 without "device apic"
Reviewed by:	kib
2015-02-20 19:42:26 +00:00
kib
ce948ff36c Fix UP build.
Sponsored by:	The FreeBSD Foundation
MFC after:	2 months
2015-02-18 10:51:48 +00:00
kib
7a89c3df60 Initialize x2APIC mode on the resume path before accessing LAPIC.
Remove unneeded disable of LAPIC in the native_lapic_xapic_mode().  We
attempt to send wakeup IPI on the resume path right after BSP wakeup,
so disabling is wrong.

Reported and tested by:	glebius, "Ranjan1018 ." <214748mv@gmail.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	2 months
2015-02-16 21:56:19 +00:00
kib
45b91c251b Detect whether x2APIC on VMWare is usable without interrupt
redirection support.  Older versions of the hypervisor mis-interpret
the cpuid format in ioapic registers when x2APIC is turned on, but IR
is not used by the guest OS.

Based on:	Linux commit 4cca6ea04d31c22a7d0436949c072b27bde41f86
Tested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	2 months
2015-02-14 09:00:12 +00:00
kib
0754f0eac9 Add x2APIC support. Enable it by default if CPU is capable. The
hw.x2apic_enable tunable allows disabling it from the loader prompt.

To closely repeat effects of the uncached memory ops when accessing
registers in the xAPIC mode, the x2APIC writes to MSRs are preceeded
by mfence, except for the EOI notifications.  This is probably too
strict, only ICR writes to send IPI require serialization to ensure
that other CPUs see the previous actions when IPI is delivered.  This
may be changed later.

In vmm justreturn IPI handler, call doreti_iret instead of doing iretd
inline, to handle corner conditions.

Note that the patch only switches LAPICs into x2APIC mode. It does not
enables FreeBSD to support > 255 CPUs, which requires parsing x2APIC
MADT entries and doing interrupts remapping, but is the required step
on the way.

Reviewed by:	neel
Tested by:	pho (real hardware), neel (on bhyve)
Discussed with:	jhb, grehan
Sponsored by:	The FreeBSD Foundation
MFC after:	2 months
2015-02-09 21:00:56 +00:00
jhb
06e75f0dba Create a cpuset mask for each NUMA domain that is available in the
kernel via the global cpuset_domain[] array. To export these to userland,
add a CPU_WHICH_DOMAIN level that can be used to fetch the mask for a
specific domain. Add a -d flag to cpuset(1) that can be used to fetch
the mask for a given domain.

Differential Revision:	https://reviews.freebsd.org/D1232
Submitted by:	jeff (kernel bits)
Reviewed by:	adrian, jeff
2015-01-08 15:53:13 +00:00
jhb
fdfced8ce8 MFamd64: Add support for extended FPU states on i386. This includes
support for AVX on i386.
- Similar to amd64, move the FPU save area out of the PCB and instead
  store saved FPU state in a variable-sized buffer after the PCB on the
  stack.
- To support the variable PCB location, alter the locore code to only use
  the bottom-most page of proc0stack for init386().  init386() returns
  the correct stack pointer to locore which adjusts the stack for thread0
  before calling mi_startup().
- Don't bother setting cr3 in thread0's pcb in locore before calling
  init386().  It wasn't used (init386() overwrote it at the end) and
  it doesn't work with the variable-sized FPU save area.
- Remove the new-bus attachment from npx.  This was only ever useful for
  external co-processors using IRQ13, but those have not been supported
  for several years.  npxinit() is now called much earlier during boot
  (init386()) similar to amd64.
- Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE.
- npxsave() is now only called from context switch contexts so it can
  use XSAVEOPT.

Differential Revision:	https://reviews.freebsd.org/D1058
Reviewed by:	kib
Tested on:	FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
adrian
678ef3c9f9 Missing from previous commit - keep the VM domain -> PXM mapping
array and use it to map PXM -> VM domain when needed.

Differential Revision:	D906
Reviewed by:	jhb
2014-10-09 05:34:28 +00:00
jhb
3a8cf1a38b Create a separate structure for per-CPU state saved across suspend and
resume that is a superset of a pcb.  Move the FPU state out of the pcb and
into this new structure.  As part of this, move the FPU resume code on
amd64 into a C function.  This allows resumectx() to still operate only on
a pcb and more closely mirrors the i386 code.

Reviewed by:	kib (earlier version)
2014-09-06 15:23:28 +00:00
jhb
59920f0385 Save and restore FPU state across suspend and resume. In earlier revisions
of this patch, resumectx() called npxresume() directly, but that doesn't
work because resumectx() runs with a non-standard %cs selector.  Instead,
all of the FPU suspend/resume handling is done in C.

MFC after:	1 week
2014-08-30 17:48:38 +00:00
royger
afa7324d1a x86/madt: make the interrupt override parser a public function
Split a portion of the code in madt_parse_interrupt_override to a
separate function, that is public and can be used from other code.
This will be needed by the Xen port, since FreeBSD needs to parse the
interrupt overrides and notify Xen about them.

This commit should not introduce any functional change.

Sponsored by: Citrix Systems R&D
Reviewed by: jhb, gibbs

x86/acpica/madt.c:
 - Introduce madt_parse_interrupt_values() that parses the intr
   information from ACPI and returns the triggering and the polarity.
   This is a subset of the functionality that used to be part of
   madt_parse_interrupt_override().
 - Make madt_found_sci_override a global variable that can be used
   from other files.

x86/include/acpica_machdep.h:
 - Prototype of madt_parse_interrupt_values.
 - Extern declaration of madt_found_sci_override.
2014-08-04 08:58:50 +00:00
royger
668dd4b0cb xen: change quality of the MADT ACPI enumerator
Lower the quality of the MADT ACPI enumerator, so on Xen Dom0 we can
force the usage of the Xen mptable enumerator even when ACPI is
detected.

This is needed because Xen might restrict the number of vCPUs
available to Dom0, but the MADT ACPI table parsed in FreeBSD is the
native one (which enumerates all the CPUs available in the system).

Sponsored by: Citrix Systems R&D
Reviewed by: gibbs

x86/acpica/madt.c:
 - Lower MADT enumerator quality to -50.

x86/xen/pvcpu_enum.c:
 - Rise Xen PV enumerator to 0.
2014-08-04 08:56:20 +00:00
akiyama
966a1bc8d6 Add missing newline to output dmesg properly. 2014-07-28 13:47:02 +00:00
takawata
d968f19902 Change default logic to CONFORM because this routine is shared
with SCI polarity setting.

Reviewed by: jhb
2014-03-28 02:38:14 +00:00
takawata
a40940f6b2 Strict value checking will cause problem.
Bay trail DN2820FYKH is supported on Linux but does not work on FreeBSD.
This behaviour is bug-compatible with Linux-3.13.5.

References:
http://d.hatena.ne.jp/syuu1228/20140326
http://lxr.linux.no/linux+v3.13.5/arch/x86/kernel/acpi/boot.c#L1094

Submitted by: syuu
2014-03-27 06:36:38 +00:00
takawata
35538e1725 To check polarity, check ACPI_MADT_POLARITY_CONFORMS, instead of ACPI_MADT_TRIGGER_CONFORMS.
PR:amd64/188010
Submitted by: syuu
2014-03-27 06:08:07 +00:00
jkim
9b4d3b43ca Move fpusave() wrapper for suspend hander to sys/amd64/amd64/fpu.c.
Inspired by:	jhb
2014-03-04 21:35:57 +00:00
jhb
94d685456e Drop the 3rd clause from all 3 clause BSD licenses where I am the sole
holder to convert them to 2 clause BSD licenses.

MFC after:	1 week
2014-02-05 18:13:27 +00:00
jhb
b2533ec507 Move <machine/apicvar.h> to <x86/apicvar.h>. 2014-01-23 20:10:22 +00:00
jhb
0dede48f9c Fix i386 build.
Pointy hat to:	jhb
2013-12-24 14:48:52 +00:00