Devices that use ISA IRQs expect them to be already configured, and don't
call bus_config_intr, which prevents those IRQs from working on Xen. In
order to solve it pre-register all the legacy IRQs with the default values
(edge triggered, low polarity) if no override is found.
While there add a panic if the registration of an interrupt override fails.
Sponsored by: Citrix Systems R&D
Improve and cleanup the Xen PIRQ event channel code:
- Remove the xi_shared field as it is unused.
- Clean the "pending" bit in the EOI handler, this is more similar to how
native interrupts are handled.
- Don't mask edge triggered PIRQs, edge trigger interrupts cannot be
masked.
- Panic if PHYSDEVOP_eoi fails.
- Remove the usage of the PHYSDEVOP_alloc_irq_vector hypercall because
it's just a no-op in the Xen versions that are supported by FreeBSD Dom0.
Sponsored by: Citrix Systems R&D
redirection support. Older versions of the hypervisor mis-interpret
the cpuid format in ioapic registers when x2APIC is turned on, but IR
is not used by the guest OS.
Based on: Linux commit 4cca6ea04d31c22a7d0436949c072b27bde41f86
Tested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 months
VT-d specification. Also add definitions for the interrupt remapping
table and IEC.
Print new capabilities on boot. although there is no hardware which
support it.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
hw.x2apic_enable tunable allows disabling it from the loader prompt.
To closely repeat effects of the uncached memory ops when accessing
registers in the xAPIC mode, the x2APIC writes to MSRs are preceeded
by mfence, except for the EOI notifications. This is probably too
strict, only ICR writes to send IPI require serialization to ensure
that other CPUs see the previous actions when IPI is delivered. This
may be changed later.
In vmm justreturn IPI handler, call doreti_iret instead of doing iretd
inline, to handle corner conditions.
Note that the patch only switches LAPICs into x2APIC mode. It does not
enables FreeBSD to support > 255 CPUs, which requires parsing x2APIC
MADT entries and doing interrupts remapping, but is the required step
on the way.
Reviewed by: neel
Tested by: pho (real hardware), neel (on bhyve)
Discussed with: jhb, grehan
Sponsored by: The FreeBSD Foundation
MFC after: 2 months
Intel Multiprocessor Specification v1.4. The Intel SDM claims that
the INIT IPIs here are invalid, but other systems follow the MP
spec instead.
While here, fix the IPI wait routine to accept a timeout in microseconds
instead of a raw spin count, and don't spin forever during AP startup.
Instead, panic if a STARTUP IPI is not delivered after 20 us.
PR: 196542
Differential Revision: https://reviews.freebsd.org/D1719
MFC after: 2 weeks
This can later use this to determine the TSC frequency like is done with
VMware, instead of using a DELAY loop that is not always accurate in an VM.
MFC after: 1 month
KVM clock shares the same data structures between the guest and the host
as Xen so it makes sense to just have a single copy of this code.
Differential Revision: https://reviews.freebsd.org/D1429
Reviewed by: royger (eariler version)
MFC after: 1 month
P-state but not C-state invariant TSC by changing the default behavior
to leaving the TSC enabled as the timecounter and disabling C2+ instead
of disabling the TSC by default.
Discussed with: jkim
Tested by: Jan Kokemuller <jan.kokemueller@gmail.com>
The data in MODINFOMD_MODULEP is packed by the loader as a 4 byte type, but
the amd64 kernel expects a vm_paddr_t, which is of size 8 bytes. Fix this by
saving it as 8 bytes in the loader and retrieving it using the proper type
in the kernel.
Sponsored by: Citrix Systems R&D
Prior to this change CLOCK_MONOTONIC could go backwards when the timecounter
hardware was changed via 'sysctl kern.timecounter.hardware'. This happened
because the vdso timehands update was missing the special treatment in
tc_windup() when changing timecounters.
Reviewed by: kib
Implement a subset of the multiboot specification in order to boot Xen
and a FreeBSD Dom0 from the FreeBSD bootloader. This multiboot
implementation is tailored to boot Xen and FreeBSD Dom0, and it will
most surely fail to boot any other multiboot compilant kernel.
In order to detect and boot the Xen microkernel, two new file formats
are added to the bootloader, multiboot and multiboot_obj. Multiboot
support must be tested before regular ELF support, since Xen is a
multiboot kernel that also uses ELF. After a multiboot kernel is
detected, all the other loaded kernels/modules are parsed by the
multiboot_obj format.
The layout of the loaded objects in memory is the following; first the
Xen kernel is loaded as a 32bit ELF into memory (Xen will switch to
long mode by itself), after that the FreeBSD kernel is loaded as a RAW
file (Xen will parse and load it using it's internal ELF loader), and
finally the metadata and the modules are loaded using the native
FreeBSD way. After everything is loaded we jump into Xen's entry point
using a small trampoline. The order of the multiboot modules passed to
Xen is the following, the first module is the RAW FreeBSD kernel, and
the second module is the metadata and the FreeBSD modules.
Since Xen will relocate the memory position of the second
multiboot module (the one that contains the metadata and native
FreeBSD modules), we need to stash the original modulep address inside
of the metadata itself in order to recalculate its position once
booted. This also means the metadata must come before the loaded
modules, so after loading the FreeBSD kernel a portion of memory is
reserved in order to place the metadata before booting.
In order to tell the loader to boot Xen and then the FreeBSD kernel the
following has to be added to the /boot/loader.conf file:
xen_cmdline="dom0_mem=1024M dom0_max_vcpus=2 dom0pvh=1 console=com1,vga"
xen_kernel="/boot/xen"
The first argument contains the command line that will be passed to the Xen
kernel, while the second argument is the path to the Xen kernel itself. This
can also be done manually from the loader command line, by for example
typing the following set of commands:
OK unload
OK load /boot/xen dom0_mem=1024M dom0_max_vcpus=2 dom0pvh=1 console=com1,vga
OK load kernel
OK load zfs
OK load if_tap
OK load ...
OK boot
Sponsored by: Citrix Systems R&D
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D517
For the Forth bits:
Submitted by: Julien Grall <julien.grall AT citrix.com>
Features by CPUID as CPUID.80000008H:EAX[7:0], into variable cpu_maxphyaddr.
Reviewed by: alc
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
cache for whole page containing modified pte, and more, only last page
in the series of the consequtive pages is flushed (i.e. the affected
mappings should be larger than 2MB).
Avoid excessive flushing and do missed neccessary flushing, by
splitting invalidation and unmapping. For now, flush exactly the
range of the changed pte. This is still somewhat bigger than
neccessary, since pte is 8 bytes, while cache flush line is at least
32 bytes.
The originator of the issue reports that after the change,
'dmar_bus_dmamap_unload went from 13,288 cycles down to
3,257. dmar_bus_dmamap_load_buffer went from 9,686 cycles down to
3,517. and I am now able to get line 1GbE speed with Netperf TCP
(even with 1K message size).'
Diagnosed and tested by: Nadav Amit <nadav.amit@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
In my case on the test machine, I have hierarchy of
pcib2 (PCIe port on host bridge with PCIe capability) -> pci2 ->
pcib3 (ITE PCIe/PCI bridge) -> pci3 -> em1
The device to check PCIe capability is pcib2 and not pcib3, as it is
currently done in the code. Also, in case of the bridge, we shall
step to pcib2 for the loop iteration, since pcib3 does not carry PCIe
capability info and would force wrong recalculation of rid.
Also change the returned requester to the PCIe bus which provides port
for the bridge. This only results in changing
hw.busdma.pciX.X.X.X.bounce tunable to force identity-mapped context
for the device.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
after dmar driver was converted to use rids. The bus component to
calculate context page must be taken from the requestor rid, which is
a bridge, and not from the device bus number.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
showing up on Haswell-class CPUs
From the Intel SDM, "Table 3-20. Feature Information Returned in the
ECX Register"
11 | SDBG | A value of 1 indicates the processor supports
IA32_DEBUG_INTERFACE MSR for silicon debug.
Submitted by: jiashiun@gmail.com
Reviewed by: jhb neel
MFC after: 2 weeks
kernel via the global cpuset_domain[] array. To export these to userland,
add a CPU_WHICH_DOMAIN level that can be used to fetch the mask for a
specific domain. Add a -d flag to cpuset(1) that can be used to fetch
the mask for a given domain.
Differential Revision: https://reviews.freebsd.org/D1232
Submitted by: jeff (kernel bits)
Reviewed by: adrian, jeff
code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly
identical and simply redefine a number of constants and helper subroutines;
a generic implementation will make it easier to implement features around
kernel core dumps. This change does not alter any minidump code and should
have no functional impact.
PR: 193873
Differential Revision: https://reviews.freebsd.org/D904
Submitted by: Conrad Meyer <conrad.meyer@isilon.com>
Reviewed by: jhibbits (earlier version)
Sponsored by: EMC / Isilon Storage Division
may also halt in C2 and not just C3 (it seems that in some cases the BIOS
advertises its C3 state as a C2 state in _CST). Just play it safe and
disable both C2 and C3 states if a user forces the use of the TSC as the
timecounter on such CPUs.
PR: 192316
Differential Revision: https://reviews.freebsd.org/D1441
No objection from: jkim
MFC after: 1 week
By default Xen binds all event channels to vCPU#0, and FreeBSD only shuffles
the interrupt sources once, at the end of the boot process. Since new event
channels might be created after this point (because new devices or backends
are added), try to automatically shuffle them at creation time.
This does not affect VIRQ or IPI event channels, that are already bound to a
specific vCPU as requested by the caller.
Sponsored by: Citrix Systems R&D
Mask the event channel source before trying to bind it to a CPU, this
prevents stray interrupts from firing while assigning them and hitting the
KASSERT in xen_intr_handle_upcall.
Sponsored by: Citrix Systems R&D
This allows the Grant-table code to attach directly to the xenpv bus,
allowing us to remove the grant-table initialization done in xenpv.
Sponsored by: Citrix Systems R&D
When running as a Xen PVH Dom0 we need to add custom buses that override
some of the functionality present in the ACPI PCI Bus and the PCI Bus. We
currently override the ACPI PCI Bus, but not the PCI Bus, so add a new
override for the PCI Bus and share the generic functions between them.
Reported by: David P. Discher <dpd@dpdtech.com>
Sponsored by: Citrix Systems R&D
conf/files.amd64:
- Add the new files.
x86/xen/xen_pci_bus.c:
- Generic file that contains the PCI overrides so they can be used by the
several PCI specific buses.
xen/xen_pci.h:
- Prototypes for the generic overried functions.
dev/xen/pci/xen_pci.c:
- Xen specific override for the PCI bus.
dev/xen/pci/xen_acpi_pci.c:
- Xen specific override for the ACPI PCI bus.
If the SCI is remapped to a non-ISA global interrupt notify the ACPI
subsystem about the override.
Reported by: David P. Discher <dpd@dpdtech.com>
Sponsored by: Citrix Systems R&D
- Dump an NT_X86_XSTATE note if XSAVE is in use. This note is designed
to match what Linux does in that 1) it dumps the entire XSAVE area
including the fxsave state, and 2) it stashes a copy of the current
xsave mask in the unused padding between the fxsave state and the
xstate header at the same location used by Linux.
- Teach readelf() to recognize NT_X86_XSTATE notes.
- Change PT_GET/SETXSTATE to take the entire XSAVE state instead of
only the extra portion. This avoids having to always make two
ptrace() calls to get or set the full XSAVE state.
- Add a PT_GET_XSTATE_INFO which returns the length of the current
XSTATE save area (so the size of the buffer needed for PT_GETXSTATE)
and the current XSAVE mask (%xcr0).
Differential Revision: https://reviews.freebsd.org/D1193
Reviewed by: kib
MFC after: 2 weeks
support for AVX on i386.
- Similar to amd64, move the FPU save area out of the PCB and instead
store saved FPU state in a variable-sized buffer after the PCB on the
stack.
- To support the variable PCB location, alter the locore code to only use
the bottom-most page of proc0stack for init386(). init386() returns
the correct stack pointer to locore which adjusts the stack for thread0
before calling mi_startup().
- Don't bother setting cr3 in thread0's pcb in locore before calling
init386(). It wasn't used (init386() overwrote it at the end) and
it doesn't work with the variable-sized FPU save area.
- Remove the new-bus attachment from npx. This was only ever useful for
external co-processors using IRQ13, but those have not been supported
for several years. npxinit() is now called much earlier during boot
(init386()) similar to amd64.
- Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE.
- npxsave() is now only called from context switch contexts so it can
use XSAVEOPT.
Differential Revision: https://reviews.freebsd.org/D1058
Reviewed by: kib
Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
- Move the existing code to x86/x86/identcpu.c since it is x86-specific.
- If the CPUID2_HV flag is set, assume a hypervisor is present and query
the 0x40000000 leaf to determine the hypervisor vendor ID. Export the
vendor ID and the highest supported hypervisor CPUID leaf via
hv_vendor[] and hv_high variables, respectively. The hv_vendor[]
array is also exported via the hw.hv_vendor sysctl.
- Merge the VMWare detection code from tsc.c into the new probe in
identcpu.c. Add a VM_GUEST_VMWARE to identify vmware and use that in
the TSC code to identify VMWare.
Differential Revision: https://reviews.freebsd.org/D1010
Reviewed by: delphij, jkim, neel
If bootverbose is enabled, a detailed list is provided; otherwise, a
single-line summary is displayed.
Differential Revision: https://reviews.freebsd.org/D1008
Reviewed by: jhb, neel
MFC after: 1 week
This device is only attached to priviledged domains, and allows the
toolstack to interact with Xen. The two functions of the privcmd
interface is to allow the execution of hypercalls from user-space, and
the mapping of foreign domain memory.
Sponsored by: Citrix Systems R&D
i386/include/xen/hypercall.h:
amd64/include/xen/hypercall.h:
- Introduce a function to make generic hypercalls into Xen.
xen/interface/xen.h:
xen/interface/memory.h:
- Import the new hypercall XENMEM_add_to_physmap_range used by
auto-translated guests to map memory from foreign domains.
dev/xen/privcmd/privcmd.c:
- This device has the following functions:
- Allow user-space applications to make hypercalls into Xen.
- Allow user-space applications to map memory from foreign domains,
this is accomplished using the newly introduced hypercall
(XENMEM_add_to_physmap_range).
xen/privcmd.h:
- Public ioctl interface for the privcmd device.
x86/xen/hvm.c:
- Remove declaration of hypercall_page, now it's declared in
hypercall.h.
conf/files:
- Add the privcmd device to the build process.
This is needed by the event channel user-space device, that requires
registering event channels without unmasking them. intr_add_handler
will unconditionally unmask the event channel, so we avoid calling it
if no filter/handler is provided, and then the user will be in charge
of calling it when ready.
In order to do this, we need to change the opaque type
xen_intr_handle_t to contain the event channel port instead of the
opaque cookie returned by intr_add_handler, since now registration of
event channels without handlers are allowed. The cookie will now be
stored inside of the private xenisrc struct. Also, introduce a new
function called xen_intr_add_handler that allows adding a
filter/handler after the event channel has been registered.
Sponsored by: Citrix Systems R&D
x86/xen/xen_intr.c:
- Leave the event channel without a handler if no filter/handler is
provided to xen_intr_bind_isrc.
- Don't perform an evtchn_mask_port, intr_add_handler will already do
it.
- Change the opaque type xen_intr_handle_t to contain a pointer to
the event channel port number, and make the necessary changes to
related functions.
- Introduce a new function called xen_intr_add_handler that can be
used to add filter/handlers to an event channel after registration.
xen/xen_intr.h:
- Add prototype of xen_intr_add_handler.
when that happens, we happily access our resource array out of
bounds. Make sure we stay within the MAX_ROMS limit.
While here, bump MAX_ROMS from 16 to 32 to minimize the chance
of leaving option ROMs unaccounted for.
Obtained from: Juniper Networks, Inc.
- Wrong integer type was specified.
- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.
- Logical OR where binary OR was expected.
- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.
- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.
- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.
- Updated "EXAMPLES" section in SYSCTL manual page.
MFC after: 3 days
Sponsored by: Mellanox Technologies
Add support for AMD's nested page tables in pmap.c:
- Provide the correct bit mask for various bit fields in a PTE (e.g. valid bit)
for a pmap of type PT_RVI.
- Add a function 'pmap_type_guest(pmap)' that returns TRUE if the pmap is of
type PT_EPT or PT_RVI.
Add CPU_SET_ATOMIC_ACQ(num, cpuset):
This is used when activating a vcpu in the nested pmap. Using the 'acquire'
variant guarantees that the load of the 'pm_eptgen' will happen only after
the vcpu is activated in 'pm_active'.
Add defines for various AMD-specific MSRs.
Submitted by: Anish Gupta (akgupt3@gmail.com)
in userland rename in-kernel getenv()/setenv() to kern_setenv()/kern_getenv().
This fixes a namespace collision with libc symbols.
Submitted by: kmacy
Tested by: make universe
Add the Xen specific implementation of pci_child_added to the Xen PCI
bus. This is needed so FreeBSD can register the devices it finds with
the hypervisor.
Sponsored by: Citrix Systems R&D
x86/xen/xen_pci.c:
- Add the Xen pci_child_added method.
This patch adds support for MSI interrupts when running on Xen. Apart
from adding the Xen related code needed in order to register MSI
interrupts this patch also makes the msi_init function a hook in
init_ops, so different MSI implementations can have different
initialization functions.
Sponsored by: Citrix Systems R&D
xen/interface/physdev.h:
- Add the MAP_PIRQ_TYPE_MULTI_MSI to map multi-vector MSI to the Xen
public interface.
x86/include/init.h:
- Add a hook for setting custom msi_init methods.
amd64/amd64/machdep.c:
i386/i386/machdep.c:
- Set the default msi_init hook to point to the native MSI
initialization method.
x86/xen/pv.c:
- Set the Xen MSI init hook when running as a Xen guest.
x86/x86/local_apic.c:
- Call the msi_init hook instead of directly calling msi_init.
xen/xen_intr.h:
x86/xen/xen_intr.c:
- Introduce support for registering/releasing MSI interrupts with
Xen.
- The MSI interrupts will use the same PIC as the IO APIC interrupts.
xen/xen_msi.h:
x86/xen/xen_msi.c:
- Introduce a Xen MSI implementation.
x86/xen/xen_nexus.c:
- Overwrite the default MSI hooks in the Xen Nexus to use the Xen MSI
implementation.
x86/xen/xen_pci.c:
- Introduce a Xen specific PCI bus that inherits from the ACPI PCI
bus and overwrites the native MSI methods.
- This is needed because when running under Xen the MSI messages used
to configure MSI interrupts on PCI devices are written by Xen
itself.
dev/acpica/acpi_pci.c:
- Lower the quality of the ACPI PCI bus so the newly introduced Xen
PCI bus can take over when needed.
conf/files.i386:
conf/files.amd64:
- Add the newly created files to the build process.
When the FreeBSD kernel is loaded from Xen the symtab and strtab are
not loaded the same way as the native boot loader. This patch adds
three new global variables to ddb that can be used to specify the
exact position and size of those tables, so they can be directly used
as parameters to db_add_symbol_table. A new helper is introduced, so callers
that used to set ksym_start and ksym_end can use this helper to set the new
variables.
It also adds support for loading them from the Xen PVH port, that was
previously missing those tables.
Sponsored by: Citrix Systems R&D
Reviewed by: kib
ddb/db_main.c:
- Add three new global variables: ksymtab, kstrtab, ksymtab_size that
can be used to specify the position and size of the symtab and
strtab.
- Use those new variables in db_init in order to call db_add_symbol_table.
- Move the logic in db_init to db_fetch_symtab in order to set ksymtab,
kstrtab, ksymtab_size from ksym_start and ksym_end.
ddb/ddb.h:
- Add prototype for db_fetch_ksymtab.
- Declate the extern variables ksymtab, kstrtab and ksymtab_size.
x86/xen/pv.c:
- Add support for finding the symtab and strtab when booted as a Xen
PVH guest. Since Xen loads the symtab and strtab as NetBSD expects
to find them we have to adapt and use the same method.
amd64/amd64/machdep.c:
arm/arm/machdep.c:
i386/i386/machdep.c:
mips/mips/machdep.c:
pc98/pc98/machdep.c:
powerpc/aim/machdep.c:
powerpc/booke/machdep.c:
sparc64/sparc64/machdep.c:
- Use the newly introduced db_fetch_ksymtab in order to set ksymtab,
kstrtab and ksymtab_size.
code. There are only a handful of MSRs common between the two so there isn't
too much duplicate functionality.
The VT-x code has the following types of MSRs:
- MSRs that are unconditionally saved/restored on every guest/host context
switch (e.g., MSR_GSBASE).
- MSRs that are restored to guest values on entry to vmx_run() and saved
before returning. This is an optimization for MSRs that are not used in
host kernel context (e.g., MSR_KGSBASE).
- MSRs that are emulated and every access by the guest causes a trap into
the hypervisor (e.g., MSR_IA32_MISC_ENABLE).
Reviewed by: grehan
than u_char.
Migrate post_filter to use an int for a CPU rather than u_char.
Change intr_event_bind() to use an int for CPU rather than u_char.
It touches the ppc, sparc64, arm and mips machdep code but it should
(hah!) be a no-op.
Tested:
* i386, AMD64 laptops
Reviewed by: jhb
The suspend/resume of event channels is already handled by the xen_intr_pic.
If those methods are set on the PIRQ PIC they are just called twice, which
breaks proper resume. This fix restores migration of FreeBSD guests to a
working state.
Sponsored by: Citrix Systems R&D
the local APIC in initializecpu() and re-enables it if the APIC code
decides to use the local APIC after all. Rework this workaround
slightly so that initializecpu() won't re-disable the local APIC if
it is called after the APIC code re-enables the local APIC.
It is the compaction bitmask, with the highest bit defining if compact
format of the xsave area is used at all.
Adjust the definition of struct xstate_hdr, provide define for bit 63.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
%eax report.
Print the XSAVE features 0xd/1 in the boot banner. The printcpuinfo()
is executed late enough so that XSAVE is already enabled.
There is no known to me off the shelf hardware that implements any
feature bits except XSAVEOPT, the list is taken from SDM rev. 50. The
banner printing will allow us to note the hardware arrival.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
resume that is a superset of a pcb. Move the FPU state out of the pcb and
into this new structure. As part of this, move the FPU resume code on
amd64 into a C function. This allows resumectx() to still operate only on
a pcb and more closely mirrors the i386 code.
Reviewed by: kib (earlier version)
and into the TSC probe routine.
- Initialize cpu_exthigh once in finishidentcpu() which is called
before printcpuinfo() (and matches the behavior on amd64).
of this patch, resumectx() called npxresume() directly, but that doesn't
work because resumectx() runs with a non-standard %cs selector. Instead,
all of the FPU suspend/resume handling is done in C.
MFC after: 1 week
After r269510 the IO APIC and ATPIC initialization is done at the same
order, which means atpic_init can be called before the IO APIC has
been initalized. In that case the ATPIC will take over the interrupt
sources, preventing the IO APIC from registering them.
Reported by: David Wolfskill <david@catwhisker.org>
Tested by: David Wolfskill <david@catwhisker.org>,
Trond Endrestøl <Trond.Endrestol@fagskolen.gjovik.no>
Sponsored by: Citrix Systems R&D
Also disable a couple of ACPI devices that are not usable under Dom0.
To this end a couple of booleans are added that allow disabling ACPI
specific devices.
Sponsored by: Citrix Systems R&D
Reviewed by: jhb
x86/xen/xen_nexus.c:
- Return BUS_PROBE_SPECIFIC in the Xen Nexus attachement routine to
force the usage of the Xen Nexus.
- Attach the ACPI bus when running as Dom0.
dev/acpica/acpi_cpu.c:
dev/acpica/acpi_hpet.c:
dev/acpica/acpi_timer.c
- Add a variable that gates the addition of the devices.
x86/include/init.h:
- Declare variables that control the attachment of ACPI cpu, hpet and
timer devices.
Allow a privileged Xen guest (Dom0) to parse the MADT ACPI interrupt
overrides and register them with the interrupt subsystem.
Also add a Xen specific implementation for bus_config_intr that
registers interrupts on demand for all the vectors less than
FIRST_MSI_INT.
Sponsored by: Citrix Systems R&D
x86/xen/pvcpu_enum.c:
- Use helper functions from x86/acpica/madt.c in order to parse
interrupt overrides from the MADT.
- Walk the MADT and register any interrupt override with the
interrupt subsystem.
x86/xen/xen_nexus.c:
- Add a custom bus_config_intr method for Xen that intercepts calls
to configure unset interrupts and registers them on the fly (if the
vector is < FIRST_MSI_INT).
Split a portion of the code in madt_parse_interrupt_override to a
separate function, that is public and can be used from other code.
This will be needed by the Xen port, since FreeBSD needs to parse the
interrupt overrides and notify Xen about them.
This commit should not introduce any functional change.
Sponsored by: Citrix Systems R&D
Reviewed by: jhb, gibbs
x86/acpica/madt.c:
- Introduce madt_parse_interrupt_values() that parses the intr
information from ACPI and returns the triggering and the polarity.
This is a subset of the functionality that used to be part of
madt_parse_interrupt_override().
- Make madt_found_sci_override a global variable that can be used
from other files.
x86/include/acpica_machdep.h:
- Prototype of madt_parse_interrupt_values.
- Extern declaration of madt_found_sci_override.
Lower the quality of the MADT ACPI enumerator, so on Xen Dom0 we can
force the usage of the Xen mptable enumerator even when ACPI is
detected.
This is needed because Xen might restrict the number of vCPUs
available to Dom0, but the MADT ACPI table parsed in FreeBSD is the
native one (which enumerates all the CPUs available in the system).
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
x86/acpica/madt.c:
- Lower MADT enumerator quality to -50.
x86/xen/pvcpu_enum.c:
- Rise Xen PV enumerator to 0.
This change inserts the Xen interrupt subsystem (event channels)
initialization between the system interrupt initialization and the IO
APIC source registration.
This is needed when running on Dom0, that routes physical interrupts
on top of event channels, so that the interrupt sources found during
IO APIC initialization can be registered using the Xen interrupt
subsystem.
The resulting order in the SI_SUB_INTR stage is the following:
- System intr initialization
- Xen intr initalization
- IO APIC source registration
Sponsored by: Citrix Systems R&D
x86/x86/local_apic.c:
- Change order of apic_setup_io to be called after xen interrupt
subsystem is setup.
x86/xen/xen_intr.c:
- Init Xen event channels before apic_setup_io.
Add a new DDB command to dump all registered event channels.
Sponsored by: Citrix Systems R&D
x86/xen/xen_intr.c:
- Add a new xen_evtchn command to DDB in order to dump all
information related to event channels.
Mask all event channels during initialization. This is done so that we
don't receive spurious interrupts while dynamically registering new
event channels. There's a small window during registration where an
event channel can fire before we have attached a handler to it.
Sponsored by: Citrix Systems R&D
x86/xen/xen_intr.c:
- Mask all event channels on init.
This allows Dom0 to manage physical hardware, redirecting the
physical interrupts to event channels.
Sponsored by: Citrix Systems R&D
x86/xen/xen_intr.c:
- Expand struct xenisrc to hold the level and triggering of PIRQ
event channels.
- Implement missing methods in xen_intr_pirq_pic.
- Allow xen_intr_alloc_isrc to take a vector parameter that globally
identifies the interrupt. This is only used for PIRQs that are
bound to a specific hardware IRQ.
- Introduce xen_register_pirq used to register IO APIC legacy PIRQ
interrupts.
- Add support for the dynamic PIRQ EOI map, this shared memory is
modified by Xen (if it suppoorts that feature), and notifies the
guest if an EOI is needed or not. If it's not available fall back
to the old implementation using PHYSDEVOP_irq_status_query.
- Rename xen_intr_isrc_count to xen_intr_auto_vector_count and
replace it's usages.
- Align static variables by name.
xen/xen_intr.h:
- Add prototype for xen_register_pirq.
features. If bootverbose is enabled, a detailed list is provided;
otherwise, a single-line summary is displayed.
- Add read-only sysctls for optional VT-x capabilities used by bhyve
under a new hw.vmm.vmx.cap node. Move a few exiting sysctls that
indicate the presence of optional capabilities under this node.
CR: https://phabric.freebsd.org/D498
Reviewed by: grehan, neel
MFC after: 1 week
This commit does not add error returns to minidumpsys() or
textdump_dumpsys(); those can also be added later.
Submitted by: Conrad Meyer (EMC / Isilon storage division)
HSD131 erratum in [1]) at a considerable rate. So filter these (default),
unless logging is enabled. Unfortunately, there really is no better way to
reasonably implement suppressing these errors than to just skipping them
in mca_log(). Given that they are reported for bank 0, they'd need to be
masked in MSR_MC0_CTL. However, P6 family processors require that register
to be set to either all 0s or all 1s, disabling way more than the one error
in question when using all 0s there. Alternatively, it could be masked for
the corresponding CMCI, but that still wouldn't keep the periodic scanner
from detecting these spurious errors. Apart from that, register contents of
MSR_MC0_CTL{,2} don't seem to be publicly documented, neither in the Intel
Architectures Developer's Manual nor in the Haswell datasheets.
Note that while HSD131 actually is only about C0-stepping as of revision
014 of the Intel desktop 4th generation processor family specification
update, these corrected errors also have been observed with D0-stepping
aka "Haswell Refresh".
1: http://www.intel.de/content/dam/www/public/us/en/documents/specification-updates/4th-gen-core-family-desktop-specification-update.pdf
Reviewed by: jhb
MFC after: 3 days
Sponsored by: Bally Wulff Games & Entertainment GmbH
This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation
This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h
Discussed at: BSDcan
These changes prevent sysctl(8) from returning proper output,
such as:
1) no output from sysctl(8)
2) erroneously returning ENOMEM with tools like truss(1)
or uname(1)
truss: can not get etype: Cannot allocate memory
there is an environment variable which shall initialize the SYSCTL
during early boot. This works for all SYSCTL types both statically and
dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
be used in the case a tunable sysctl has a custom initialisation
function allowing the sysctl to still be marked as a tunable. The
kernel SYSCTL API is mostly the same, with a few exceptions for some
special operations like iterating childrens of a static/extern SYSCTL
node. This operation should probably be made into a factored out
common macro, hence some device drivers use this. The reason for
changing the SYSCTL API was the need for a SYSCTL parent OID pointer
and not only the SYSCTL parent OID list pointer in order to quickly
generate the sysctl path. The motivation behind this patch is to avoid
parameter loading cludges inside the OFED driver subsystem. Instead of
adding special code to the OFED driver subsystem to post-load tunables
into dynamically created sysctls, we generalize this in the kernel.
Other changes:
- Corrected a possibly incorrect sysctl name from "hw.cbb.intr_mask"
to "hw.pcic.intr_mask".
- Removed redundant TUNABLE statements throughout the kernel.
- Some minor code rewrites in connection to removing not needed
TUNABLE statements.
- Added a missing SYSCTL_DECL().
- Wrapped two very long lines.
- Avoid malloc()/free() inside sysctl string handling, in case it is
called to initialize a sysctl from a tunable, hence malloc()/free() is
not ready when sysctls from the sysctl dataset are registered.
- Bumped FreeBSD version to indicate SYSCTL API change.
MFC after: 2 weeks
Sponsored by: Mellanox Technologies
Fix the gate in xen_pv_lapic_ipi_vectored to prevent access to element
at position nitems(xen_ipis).
Sponsored by: Citrix Systems R&D
Coverity ID: 1223203
Approved by: gibbs
The functions' definitions are protected by #ifdef SMP.
Keeping apic_ops.ipi_*() methods NULL would allow to catch the use
on UP machines.
Reviewed by: royger
Sponsored by: The FreeBSD Foundation
This is needed because syscons depends on ISA.
Sponsored by: Citrix Systems R&D
Approved by: gibbs
x86/isa/isa.c:
- Allow the ISA bus to attach to xenpv.
Create the necessary hooks in order to provide a Xen PV APIC
implementation that can be used on PVH. Most of the lapic ops
shouldn't be called on Xen, since we trap those operations at a higher
layer.
Sponsored by: Citrix Systems R&D
Approved by: gibbs
x86/xen/hvm.c:
x86/xen/xen_apic.c:
- Move IPI related code to xen_apic.c
x86/xen/xen_apic.c:
- Introduce Xen PV APIC implementation, most of the functions of the
lapic interface should never be called when running as PV(H) guest,
so make sure FreeBSD panics when trying to use one of those.
- Define the Xen APIC implementation in xen_apic_ops.
xen/xen_pv.h:
- Extern declaration of the xen_apic struct.
x86/xen/pv.c:
- Use xen_apic_ops as apic_ops when running as PVH guest.
conf/files.amd64:
conf/files.i386:
- Include the xen_apic.c file in the build of i386/amd64 kernels
using XENHVM.
This is needed for Xen PV(H) guests, since there's no hardware lapic
available on this kind of domains. This commit should not change
functionality.
Sponsored by: Citrix Systems R&D
Reviewed by: jhb
Approved by: gibbs
amd64/include/cpu.h:
amd64/amd64/mp_machdep.c:
i386/include/cpu.h:
i386/i386/mp_machdep.c:
- Remove lapic_ipi_vectored hook from cpu_ops, since it's now
implemented in the lapic hooks.
amd64/amd64/mp_machdep.c:
i386/i386/mp_machdep.c:
- Use lapic_ipi_vectored directly, since it's now an inline function
that will call the appropiate hook.
x86/x86/local_apic.c:
- Prefix bare metal public lapic functions with native_ and mark them
as static.
- Define default implementation of apic_ops.
x86/include/apicvar.h:
- Declare the apic_ops structure and create inline functions to
access the hooks, so the change is transparent to existing users of
the lapic_ functions.
x86/xen/hvm.c:
- Switch to use the new apic_ops.
BUS_DMA_KMEM_ALLOC. They serve the same purpose, but using the flag
means that the map can be NULL again, which in turn enables significant
optimizations for the common case of no bouncing.
Obtained from: Netflix, Inc.
MFC after: 3 days
boundary. This was addressed several years ago by creating a parent
tag hierarchy for the root buses that set the boundary restriction
for appropriate buses and allowed child deviced to inherit it.
Somewhere along the way, this restriction was turned into a case for
marking the tag as a candidate for needing bounce buffers, instead
of just splitting the segment along the boundary line. This flag
also causes all maps associated with this tag to be non-NULL, which
in turn causes bus_dmamap_sync() to take the slow path of function
pointer indirection to discover that there's no bouncing work to
do. The end result is a lot of pages set aside in bounce pools
that will never be used, and a slow path for data buffers in nearly
every DMA-capable PCIe device. For example, our workload at Netflix
was spending nearly 1% of all CPU time going through this slow path.
Fix this problem by being more selective about when to set the
COULD_BOUNCE flag. Only set it when the boundary restriction
exists and the consumer cannot do more than a single DMA segment
at once. This fixes the case of dynamic buffers (mbufs, bio's)
but doesn't address static buffers allocated from bus_dmamem_alloc().
That case will be addressed in the future.
For those interested, this was discovered thanks to Dtrace Flame
Graphs.
Discussed with: jhb, kib
Obtained from: Netflix, Inc.
MFC after: 3 days
XSAVE Extended Features for AVX512 and MPX (Memory Protection Extensions).
Obtained from: Intel's Instruction Set Extensions Programming Reference
(March 2014)
Under the hood the VT-d spec is really implemented in terms of
PCI RIDs instead of bus/slot/function, even though the spec makes
pains to convert back to bus/slot/function in examples. However
working with bus/slot/function is not correct when PCI ARI is
in use, so convert to using RIDs in most cases. bus/slot/function
will only be used when reporting errors to a user.
Reviewed by: kib
MFC after: 2 months
Sponsored by: Sandvine Inc.
My PCI RID changes somehow got intermixed with my PCI ARI patch when I
committed it. I may have accidentally applied a patch to a non-clean
working tree. Revert everything while I figure out what went wrong.
Pointy hat to: rstone
Under the hood the VT-d spec is really implemented in terms of
PCI RIDs instead of bus/slot/function, even though the spec makes
pains to convert back to bus/slot/function in examples. However
working with bus/slot/function is not correct when PCI ARI is
in use, so convert to using RIDs in most cases. bus/slot/function
will only be used when reporting errors to a user.
Reviewed by: kib
Sponsored by: Sandvine Inc.
