Commit Graph

1375 Commits

Author SHA1 Message Date
Dmitry Chagin
6fdf04a2be smp: Drop confusing braces and return statement as panic() is never returns.
Reviewed by:		imp, kib
Differential Revision:	https://reviews.freebsd.org/D38235
MFC after:		1 week
2023-01-29 15:33:16 +03:00
Konstantin Belousov
11989314dc x86: add more definitions for XCR0 bits
This covers all currently defined bits, adding PKRU and TILE.

Reviewed by:	jhb, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D38219
2023-01-27 19:44:49 +02:00
Corvin Köhne
122405c903
x86: ignore stepping for APL30 errata
The issue is present in all apollolake cpus and it doesn't look like
there'll be a fix in the future.

See
https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/pentium-celeron-n-series-j-series-datasheet-spec-update.pdf

MFC after:		1 week
Sponsored by:		Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D37621
2023-01-12 10:08:17 +01:00
Konstantin Belousov
45ac7755a7 amd64: identify small cores
Reviewed by:	alc
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37770
2023-01-01 00:09:45 +02:00
Li-Wen Hsu
611c6b233d
Complete retire cp(4)
And fix the LINT build.

Sponsored by:	The FreeBSD Foundation

Fixes:	895992bb66
2022-12-14 11:38:55 +08:00
Mateusz Guzik
c3f1a13902 Retire broken GPROF support from the kernel
The option is not even recognized and with that patched it does not
compile. Even if it did work, it would be prohibitively expensive to
use.

Interested parties can use pmcstat or dtrace instead.
2022-11-15 14:17:10 +00:00
Warner Losh
1d21f64149 bhyve: Implement MSR_MISC_FEATURES_ENABLES
Linux reads MISC_FEATURES_ENABLES to manage the CPUID faulting feature
(undocumented in the Intel SDM, but documented in 323850-004 (Intel
Virtualization Technology FlexMigration Application Note). Since bhyve
doesn't emulate this feature, we always return 0. Neither does bhyve
support the MONITOR/MWAIT fault bit also in this MSR (which is
documented in the sdm), so always return 0.

Sponsored by:		Netflix
Reviewed by:		jhb
Differential Revision:	https://reviews.freebsd.org/D36602
2022-10-27 11:34:41 -06:00
Ed Maste
c8113dad7e Increase MAX_APIC_ID safeguard to 0x800
MAX_APIC_ID must be at least twice MAXCPU.  Increase it to 0x800 so that
it is possible to set MAXCPU to 512 or 1024 in a custom kernel config
file.

Note that increasing this limit does not itself cause any allocations
to be larger; it just allows madt_parse_cpu() to process higher APIC
IDs.

APIC IDs may be sparse and so we can waste memory.  This is independent
of this change, but becomes more of an issue as the maximum APIC ID
grows.  This should be addressed with future work.

Reviewed by:	royger
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D37067
2022-10-27 12:33:34 -04:00
Konstantin Belousov
829145388b x86/include/elf.h: make inclusion blocks for elf32.h and elf64.h similar
They were copy-pasted when x86/include/elf.h file was merged from its
i386 and amd64 counterparts.  Having the text around inclusions
significantly different is somewhat confusing.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37085
2022-10-25 19:00:44 +03:00
Konstantin Belousov
5f00525dfc i386: move hard-coded load address for PIE below default linker base
both for i386 native and compat32 amd64.  We know the ld-elf.so.1 size
in advance, it fits there.  Trying to push it up after the end of a
binary cannot work reliably and eventually fail for large binaries.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37085
2022-10-25 19:00:44 +03:00
Colin Percival
b7761f1f08 x86/busdma: Limit reserved pages if low nsegs
When bus_dmamap_create is called, if bouncing might be required we
reserve enough pages for a maximum-length request, subject to the
MAX_BPAGES constraint (32 MB on amd64; 32 MB or 2 MB on i386
depending on the amount of RAM).

Since pages used for bouncing are typically non-consecutive, each
bounced page will typically constitute a busdma segment; as such, we
are unlikely to ever successfully use more pages than the nsegments
limit.  Limit the number of pages reserved to nsegments.

On FreeBSD/Firecracker, this reduces bounce page memory consumption
from 32 MB to 512 kB, making VMs with 128 MB of RAM usable.

Reviewed by:	imp, mav
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D37082
2022-10-21 22:47:33 -07:00
Colin Percival
13f34e211b PVH: Set bootmethod to PVH
Now that we can PVH boot on a non-Xen hypervisor, we shouldn't set
machdep.bootmethod to "XEN".  Instead, set it to "PVH"; there are
other ways to discern the hypervisor.

Reviewed by:	royger
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D36191
2022-10-17 23:02:22 -07:00
Colin Percival
c4a4011c74 PVH: support whitespace cmdline splitting
For historical reasons, Xen kernel command lines have options
separated by commas.  Every other FreeBSD platform uses whitespace;
this is also necessary in PVH in order to support the Firecracker
VMM.  Allow options to be separated by any combination of commas
and whitespace.

Reviewed by:	imp
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D36190
2022-10-17 23:02:22 -07:00
Colin Percival
a8ea154064 x86: Distinguish Xen from non-Xen PVH boots
The PVH boot protocol, introduced by Xen, is now used by some non-Xen
platforms (e.g. the Firecracker VM) as well.  In order to accommodate
these, we use CPUID to detect Xen and only perform Xen-specific setup
when running on that platform.

The "isxen" function duplicates some work done by identcpu.c later in
the boot process; but we need it here since this is the very first C
code which runs when PVH booting (even before hammer_time).

In many places the existing code had
        xc_printf(...);
        HYPERVISOR_shutdown(SHUTDOWN_crash);
making use of Xen functionality to print a message and shut down; in
the places where this idiom can be reached in the non-xen case, we
replace it idiom with a CRASH(...) macro which calls those in the Xen
case and halts in the non-Xen case.

Reviewed by:	royger
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D35801
2022-10-17 23:02:22 -07:00
Colin Percival
023a025b5c x86: Add support for PVH version 1 memmap
Version 0 of PVH booting uses a Xen hypercall to retrieve the system
memory map; in version 1 the memory map can be provided via the
start_info structure.

Using the memory map from the version 1 start_info structure allows
FreeBSD to use PVH booting on systems other than Xen, e.g. on the
Firecracker VM.

Reviewed by:	royger
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D35800
2022-10-17 23:02:22 -07:00
Colin Percival
d1ca8cc638 x86: Add MPTABLE_LINUX_BUG_COMPAT option
Linux has two bugs in its handling of the x86 MP table:
1. It assumes that there is always 640 kB of base memory, and looks for
the MP table in the top kB of this even if the memory map indicates
that memory location does not exist.
2. It ignores that entry_count field and instead iterates through the
MP table by scanning until it runs out of bytes in the table.

The Firecracker VM (and probably other related VMs) relies on both of
these bugs.  With the MPTABLE_LINUX_BUG_COMPAT option, we search for
the MP table at address 639k even if that isn't in the memory map; and
replace a zeroed entry_count with a value computed from scanning the
table until we run out of table bytes.

Reviewed by:	imp
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D35799
2022-10-17 23:02:22 -07:00
Colin Percival
2297a1633d Add NO_LEGACY_PCIB kernel option to i386, amd64
On systems without a PCI bus, legacy_pcib_identify by default creates
one anyway:
    legacy_pcib_identify: no bridge found, adding pcib0 anyway

This commit adds a kernel option NO_LEGACY_PCIB which disables this,
allowing systems to be fully PCI-free.

Reviewed by:	imp
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D35798
2022-10-17 23:02:22 -07:00
John Baldwin
a9fca3b987 Fix various places which cast a pointer to a vm_paddr_t or vice versa.
GCC warns about the mismatched sizes on i386 where vm_paddr_t is 64
bits.

Reviewed by:	imp, markj
Differential Revision:	https://reviews.freebsd.org/D36750
2022-10-03 16:10:41 -07:00
John Baldwin
f49fd63a6a kmem_malloc/free: Use void * instead of vm_offset_t for kernel pointers.
Reviewed by:	kib, markj
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D36549
2022-09-22 15:09:19 -07:00
John Baldwin
7ae99f80b6 pmap_unmapdev/bios: Accept a pointer instead of a vm_offset_t.
This matches the return type of pmap_mapdev/bios.

Reviewed by:	kib, markj
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D36548
2022-09-22 15:08:52 -07:00
Konstantin Belousov
fd25c62278 i386: check that trap() and syscall() run on the thread kstack
and not on the trampoline stack.  This is a useful way to ensure that
we did not enabled interrupts while on user %cr3 or trampoline stack.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2022-09-14 18:46:32 +03:00
Gordon Bergling
9755e244c9 x86: Correct a typo in source code comment
- s/occured/occurred/

MFC after:	3 days
2022-09-04 13:36:53 +02:00
Colin Percival
02ab915ae0 lapic_init: Reduce LOOPS
While I'm here, instrument lapic_init with TSLOG so it shows up (or
typically not, after this change) on flamecharts.

Reviewed by:	kib
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D36186
2022-08-13 15:28:09 -07:00
Mateusz Guzik
648edd6378 x86: remove MP_WATCHDOG
It does not work with ULE, which is the default scheduler for over a
decade.

Reviewed by:	emaste, kib
Differential Revision:	https://reviews.freebsd.org/D36094
2022-08-11 21:35:32 +00:00
Emmanuel Vadot
821b850a3b x86: Remove redundant parentheses
Reported by:	avg
Sponsored by:	Beckhoff Automation GmbH & Co. KG
MFC after:	1 week
MFC-With:	b223c1f1a0 ("x86: Add another cpuid for Apollo Lake errata APL30")
2022-08-09 09:46:50 +02:00
Corvin Köhne
b223c1f1a0 x86: Add another cpuid for Apollo Lake errata APL30
Sponsored by:	Beckhoff Automation GmbH & Co. KG
MFC after:	1 week
2022-08-09 09:07:59 +02:00
Alan Cox
7f46deccbe x86/iommu: Reduce the number of queued invalidation interrupts
Restructure dmar_qi_task() so as to reduce the number of invalidation
completion interrupts.  Specifically, because processing completed
invalidations in dmar_qi_task() can take quite some time, don't reenable
completion interrupts until processing has completed a first time. Then,
check a second time after reenabling completion interrupts, so that
any invalidations that complete just before interrupts are reenabled
do not linger until a future invalidation might raise an interrupt.
(Recent changes have made checking for completed invalidations cheap; no
locking is required.)

Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D36054
2022-08-06 13:05:58 -05:00
Alexander Motin
ac64943ca8 mca: Add sysctl to mute corrected errors.
Setting hw.mca.log_corrected to 0 will mute corrected errors logging
except ones marked as reaching Yellow threshold by hardware.

MFC after:	1 week
2022-08-05 13:48:05 -04:00
Alan Cox
4670f90846 iommu_gas: Eliminate redundant parameters and push down lock acquisition
Since IOMMU map entries store a reference to the domain in which they
reside, there is no need to pass the domain to iommu_gas_free_entry(),
iommu_gas_free_space(), and iommu_gas_free_region().

Push down the acquisition and release of the IOMMU domain lock into
iommu_gas_free_space() and iommu_gas_free_region().

Both of these changes allow for simplifications in the callers of the
functions without really complicating the functions themselves.
Moreover, the latter change eliminates the direct use of the IOMMU
domain lock from the x86-specific DMAR code.

Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D35995
2022-07-30 14:28:48 -05:00
Alan Cox
42736dc44d x86/iommu: Reduce DMAR lock contention
Replace the DMAR unit's tlb_flush TAILQ by a custom list implementation
that enables dmar_qi_task() to dequeue entries without holding the DMAR
lock.

Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D35951
2022-07-29 00:11:33 -05:00
Alan Cox
c251563470 x86/iommu: Correct a recent change to iommu_domain_unload_entry()
Correct 8bc3673847.  When iommu_domain_unload_entry() performs a
synchronous IOTLB invalidation, it must call dmar_domain_free_entry()
to remove the entry from the domain's RB_TREE.

Push down the acquisition and release of the DMAR lock into the
recently introduced function dmar_qi_invalidate_sync_locked() and
remove the _locked suffix.

MFC with:	8bc3673847
2022-07-26 01:07:21 -05:00
Alan Cox
8bc3673847 iommu_gas: Eliminate a possible case of use-after-free
Eliminate a possible case of use-after-free in an error handling path
after a mapping failure.  Specifically, eliminate IOMMU_MAP_ENTRY_QI_NF
and instead perform the IOTLB invalidation synchronously.  Otherwise,
when iommu_domain_unload_entry() is called and told not to free the
IOMMU map entry, the caller could free the entry before dmar_qi_task()
is finished with it.

Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D35878
2022-07-25 11:14:58 -05:00
Dimitry Andric
eadef926b0 Adjust linux_vdso_{cpu,tsc}_selector_idx() definitions to avoid clang 15 warnings
With clang 15, the following -Werror warnings are produced:

    sys/x86/linux/linux_vdso_selector_x86.c:44:28: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
    linux_vdso_tsc_selector_idx()
                               ^
                                void
    sys/x86/linux/linux_vdso_selector_x86.c:62:28: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
    linux_vdso_cpu_selector_idx()
                               ^
                                void

This is because linux_vdso_tsc_selector_idx() and
linux_vdso_cpu_selector_idx are declared with (void) argument lists, but
defined with empty argument lists. Make the definitions match the
declarations.

MFC after:	3 days
2022-07-25 00:40:13 +02:00
Alan Cox
4eaaacc755 x86/iommu: Shrink the critical section in dmar_qi_task()
It is safe to test and clear the Invalidation Wait Descriptor
Complete flag before acquiring the DMAR lock in dmar_qi_task(),
rather than waiting until the lock is held.

Reviewed by:	kib
MFC after:	2 weeks
2022-07-18 22:23:13 -05:00
Colin Percival
05350f0936 x86: Remove 1 second DELAY from cpu_reset
On SMP systems, cpu_reset broadcasts a message telling the APs to stop
themselves, and then the BSP waits 1 second before actually resetting
itself; this behaviour dates back to 1998-05-17.

I assume that this delay was added in order to allow the APs to stop
themselves before the BSP resets; but we wait until the APs have all
acknowledged entering the "stopped" state, so it no longer seems to
serve any purpose.

Reviewed by:	jhb, kib
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D35797
2022-07-18 17:23:25 -07:00
Mitchell Horne
c84c5e00ac ddb: annotate some commands with DB_CMD_MEMSAFE
This is not completely exhaustive, but covers a large majority of
commands in the tree.

Reviewed by:	markj
Sponsored by:	Juniper Networks, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D35583
2022-07-18 22:06:09 +00:00
Alan Cox
da55f86c61 x86/iommu: Eliminate redundant wrappers
Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D35832
2022-07-16 18:05:37 -05:00
Alan Cox
db0110a536 iommu: Shrink the iommu map entry structure
Eliminate the unroll_entry field from struct iommu_map_entry, shrinking
the struct by 16 bytes on 64-bit architectures.

Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D35769
2022-07-15 22:24:52 -05:00
Mark Johnston
03f868b163 x86: Add a required store-load barrier in cpu_idle()
ULE's tdq_notify() tries to avoid delivering IPIs to the idle thread.
In particular, it tries to detect whether the idle thread is running.
There are two mechanisms for this:
- tdq_cpu_idle, an MI flag which is set prior to calling cpu_idle().  If
  tdq_cpu_idle == 0, then no IPI is needed;
- idle_state, an x86-specific state flag which is updated after
  cpu_idleclock() is called.

The implementation of the second mechanism is racy; the race can cause a
CPU to go to sleep with pending work.  Specifically, cpu_idle_*() set
idle_state = STATE_SLEEPING, then check for pending work by loading the
tdq_load field of the CPU's runqueue.  These operations can be reordered
so that the idle thread observes tdq_load == 0, and tdq_notify()
observes idle_state == STATE_RUNNING.

Some counters indicate that the idle_state check in tdq_notify()
frequently elides an IPI.  So, fix the problem by inserting a fence
after the store to idle_state, immediately before idling the CPU.

PR:		264867
Reviewed by:	mav, kib, jhb
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35777
2022-07-14 10:28:01 -04:00
Mark Johnston
ece453d5fa eventtimer: Simplify KTR traces
Stop including the current CPU in all event messages, since it's already
saved in KTR log entries and thus is redundant.  All eventtimer traces
occur in a context where CPU migration is not possible.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-07-11 15:58:43 -04:00
Mitchell Horne
258958b3c7 ddb: use _FLAGS command macros where appropriate
Some command definitions were forced to use DB_FUNC in order to specify
their required flags, CS_OWN or CS_MORE. Use the new macros to simplify
these.

Reviewed by:	markj, jhb
MFC after:	3 days
Sponsored by:	Juniper Networks, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D35582
2022-07-05 11:56:55 -03:00
Dmitry Chagin
03473e8ec8 linux(4): Use saved cpu feature bits
MFC after:		3 days
2022-07-04 23:42:07 +03:00
Warner Losh
26031009cf amd64/efi: Stop falling back to hints for RSDP
All boot loaders for the last 6 years set acpi.rsdp in addition to the
hints. This was planned for removal ~5 years ago. Belatedly remove it
from here.

Sponsored by:		Netflix
Reviewed by:		jhb
Differential Revision:	https://reviews.freebsd.org/D35633
2022-07-02 08:02:12 -06:00
Roger Pau Monné
77cb05db0c x86/xen: stop assuming kernel memory loading order in PVH
Do not assume that start_info will always be loaded at the highest
memory address, and instead check the position of all the loaded
elements in order to find the last loaded one, and thus a likely safe
place to use as early boot allocation memory space.

Reported by: markj, cperciva
Sponsored by: Citrix Systems R&D
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D35628
2022-06-30 08:53:16 +02:00
Dmitry Chagin
050f5a8405 amd64: Reload CPU ext features after resume or cr4 changes
Reviewed by:		kib
Differential revision:	https://reviews.freebsd.org/D35555
MFC after:		2 weeks
2022-06-29 10:34:43 +03:00
Roger Pau Monné
091febc04a xen/blkback: do not use x86 CPUID in generic code
Move checker for whether Xen creates IOMMU mappings for foreign pages
into a helper that's defined in arch-specific code.

Reported by: Elliott Mitchell <ehem+freebsd@m5p.com>
Fixes: 1d528f95e8 ('xen/blkback: remove bounce buffering mode')
Sponsored by: Citrix Systems R&D
2022-06-28 09:51:57 +02:00
John Baldwin
15a6642da6 x86 mptable: Include <x86/legacvar.h> for legacy_get_pcibus().
Fixes:		b076d8d54c mptable_hostb: Use legacy_get_pcibus() to fetch PCI bus number.
MFC after:	1 week
2022-06-23 15:00:12 -07:00
Mitchell Horne
8701571df9 set_cputicker: use a bool
The third argument to this function indicates whether the supplied
ticker is fixed or variable, i.e. requiring calibration. Give this
argument a type and name that better conveys this purpose.

Reviewed by:	kib, markj
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35459
2022-06-23 15:15:11 -03:00
John Baldwin
b076d8d54c mptable_hostb: Use legacy_get_pcibus() to fetch PCI bus number.
The mptable_hostb driver is a child of legacy0 and has legacy bus
ivars, not PCI or PCI bridge ivars.

PR:		264819
Reported by:	Dennis Clarke <dclarke@blastwave.org>
Diagnosed by:	avg
Reviewed by:	avg
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D35548
2022-06-23 10:49:09 -07:00
Mark Johnston
f6b799a86b Fix the test used to wait for AP startup on x86, arm64, riscv
On arm64, testing pc_curpcb != NULL is not correct since pc_curpcb is
set in pmap_switch() while the bootstrap stack is still in use.  As a
result, smp_after_idle_runnable() can free the boot stack prematurely.

Take a different approach: use smp_rendezvous() to wait for all APs to
acknowledge an interrupt.  Since APs must not enable interrupts until
they've entered the scheduler, i.e., switched off the boot stack, this
provides the right guarantee without depending as much on the
implementation of cpu_throw().  And, this approach applies to all
platforms, so convert x86 and riscv as well.

Reported by:	mmel
Tested by:	mmel
Reviewed by:	kib
Fixes:		8db2e8fd16 ("Remove the secondary_stacks array in arm64 and riscv kernels.")
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35435
2022-06-15 11:38:04 -04:00
Dmitry Chagin
4a6c2d075d linux(4): Properly restore the thread signal mask after signal delivery on i386
Replace sigframe sf_extramask by native sigset_t and use it to
store/restore the thread signal mask without conversion to/from
Linux signal mask.

Pointy hat to:		dchagin
MFC after:		2 weeks
2022-05-30 20:03:49 +03:00
Corvin Köhne
7468332f55 x86/mp: don't create empty cpu groups
When some APICs are disabled by tunables, some cpu groups could end up
empty. An empty cpu group causes the system to panic because not all
functions handle them correctly. Additionally, it's wasted time to
handle and inspect empty cpu groups. Therefore, just don't create them.

Reviewed by:	kib, avg, cem
Sponsored by:	Beckhoff Automation GmbH & Co. KG
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D24927
2022-05-30 11:21:46 +02:00
Dmitry Chagin
9016ec056a linux(4): Deduplicate bsd_to_linux_trapcode()
As bsd_to_linux_trapcode() is common for x86 Linuxulators,
move it under x86/linux.

MFC after:		2 weeks
2022-05-23 13:16:58 +03:00
Dmitry Chagin
2434137f69 linux(4): Deduplicate translate_traps()
As translate_traps() is common for x86 Linuxulators,
move it under x86/linux.

MFC after:		2 weeks
2022-05-23 13:16:26 +03:00
Dmitry Chagin
6e826d27c3 linux(4): Better naming for ucontext field of struct rt_sigframe
To reduce sendsig code difference and to avoid confusing me,
rename sf_sc to sf_uc to match the content.

MFC after:		2 weeks
2022-05-15 21:06:47 +03:00
Dmitry Chagin
21f2461741 linux(4): Move sigframe definitions to separate headers
The signal trampoine-related definitions are used only in the MD part
of code, wherefore moved from everywhere used linux.h to separate MD
headers.

MFC after:		2 weeks
2022-05-15 21:03:01 +03:00
Dmitry Chagin
5a6a4fb284 linux(4): Implement vdso getcpu for x86.
This is modeled after f2395455 (by kib@).

MFC after:		2 weeks
2022-05-08 17:20:52 +03:00
Dmitry Chagin
332eca05b5 linux(4): Refactor vdso_gettc_x86 includes.
Factor out includes from common vdso_gettc_x86 file to the corresponding
MD files.

MFC after:		2 weeks
2022-05-08 17:20:51 +03:00
John Baldwin
80d2b3de16 x86: Remove unused devclass arguments to DRIVER_MODULE. 2022-05-06 15:46:58 -07:00
John Baldwin
b3407dcc58 cpufreq: Remove unused devclass arguments to DRIVER_MODULE. 2022-05-06 15:39:29 -07:00
John Baldwin
09fd3b43ad Remove isa_devclass from ISA bus drivers. 2022-05-06 15:39:28 -07:00
Dmitry Chagin
fe2c9f83a6 Remove dead code.
is_physical_memory() dead since 235a54de.

Reviewed by:		markj
Differential revision:	https://reviews.freebsd.org/D35056
MFC after:		2 weeks
2022-04-26 19:40:59 +03:00
John Baldwin
d4ab3a8d4f busdma_bounce: Add free_bounce_pages helper function.
Deduplicate code to iterate over the bpages list in a bus_dmamap_t
freeing bounce pages during bus_dmamap_unload.

Reviewed by:	imp
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D34967
2022-04-21 10:42:14 -07:00
John Baldwin
489e8f24a5 smbios/vpd: Use devclass_find to lookup devclass in module event handler.
While here, use a modern function declaration for smbios_modevent and
vpd_modevent.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D34996
2022-04-21 10:29:14 -07:00
Kornel Duleba
06f659c39d dmar: Disable PMR in driver attach routine
Previously it was disabled right before translation was enabled.
This way the disable logic is still executed even when translation
is not be activated, e.g. with hw.iommu.dma=0 tunable set.
On some platforms we need to disable PMR in order for core dump to work.
At the same time it was observed that enabling translation has
a significant impact on network performance.
With this patch PMR can be disabled, with IOMMU translation not being
turned on by appending the following to the loader.conf:

hw.dmar.enable=1
hw.dmar.pmr.disable=1
hw.dmar.dma=0

Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D34907
2022-04-20 09:40:28 +02:00
John Baldwin
3d6f4411e4 Remove checks for <sys/cdefs.h> being included.
These files no longer depend on the macros required when these checks
were added.

PR:		263102 (exp-run)
Reviewed by:	brooks, imp, emaste
Differential Revision:	https://reviews.freebsd.org/D34804
2022-04-12 10:06:18 -07:00
John Baldwin
56f5947a71 Remove checks for __GNUCLIKE_ASM assuming it is always true.
All supported compilers (modern versions of GCC and clang) support
this.

Many places didn't have an #else so would just silently do the wrong
thing.  Ancient versions of icc (the original motivation for this) are
no longer a compiler FreeBSD supports.

PR:		263102 (exp-run)
Reviewed by:	brooks, imp
Differential Revision:	https://reviews.freebsd.org/D34797
2022-04-12 10:05:45 -07:00
Mark Johnston
aa597d4049 i386: Fix the nodevice apic build
PR:		263124
Fixes:		62d09b46ad ("x86: Defer LAPIC calibration until after timecounters are available")
Reviewed by:	kib, jhb, emaste
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34830
2022-04-08 11:47:52 -04:00
John Baldwin
354ef278e9 powernow(4): Fix unused variable warnings by using the variables. 2022-04-06 16:45:28 -07:00
John Baldwin
89abc0fbbd x86 bounce_bus_dma_tag_destroy: Silence set but unused warning. 2022-04-06 16:45:27 -07:00
Gordon Bergling
bba12ee453 xen(4): Fix a few typos in source code comments
- s/querried/queried/

MFC after:	3 days
2022-03-28 19:37:20 +02:00
John Baldwin
931983ee08 x86: Add a NT_X86_SEGBASES register set.
This register set contains the values of the fsbase and gsbase
registers.  Note that these registers can already be controlled
individually via ptrace(2) via MD operations, so the main reason for
adding this is to include these register values in core dumps.  In
particular, this will enable looking up the value of TLS variables
from core dumps in gdb.

The value of NT_X86_SEGBASES was chosen to match the value of
NT_386_TLS on Linux.  The notes serve similar purposes, but FreeBSD
will never dump a note equivalent to NT_386_TLS (which dumps a single
segment descriptor rather than a pair of addresses) and picking a
currently-unused value in the NT_X86_* range could result in a future
conflict.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D34650
2022-03-24 11:36:19 -07:00
Roger Pau Monné
1ca34862dc x86/tsc: fetch frequency from CPUID when running on Xen
Introduce a helper to fetch the TSC frequency from CPUID when running
under Xen.

Since the TSC can also be initialized early when running as a Xen
guest pull out the call to tsc_init() from the
early_clock_source_init() handlers and place it in clock_init(), as
otherwise all handlers would call tsc_init() anyway.

Reviewed by: markj
Sponsored by: Citrix Systems R&D
Differential revision: https://reviews.freebsd.org/D34581
2022-03-18 10:21:04 +01:00
Roger Pau Monné
396a8479b0 x86/xen: fix CPUID signature
MFC: 3 days
Reviewed by: cem
Sponsored by: Citrix Systems R&D
Differential revision: https://reviews.freebsd.org/D34580
2022-03-17 12:56:36 +01:00
Zhenlei Huang
ba46c6c4b7 x86: Correctly report unexpected cache level
Reviewed by:	rpokala, emaste
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D34577
2022-03-16 16:30:38 -04:00
Mark Johnston
075e2779ac x86: Defer early TSC timecounter calibration to SI_SUB_CPU
If we can't determine the TSC frequency using CPU registers, we need to
give a chance for Hyper-V drivers to register a timecounter (during
SI_SUB_HYPERVISOR) since an emulated 8254 might not be available.
Thus, split probe_tsc_freq() into early and late stages, and wait until
the latter to attempt calibration using a reference clock.

Fixes:		84369dd523 ("x86: Probe the TSC frequency earlier")
Reported and tested by:	khng, Shawn Webb
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34444
2022-03-04 19:34:43 -05:00
Mark Johnston
84369dd523 x86: Probe the TSC frequency earlier
This lets us use the TSC to implement early DELAY, limiting the use of
the sometimes-unreliable 8254 PIT.

PR:		262155
Reviewed by:	emaste
Tested by:	emaste, mike tancsa <mike@sentex.net>, Stefan Hegnauer <stefan.hegnauer@gmx.ch>
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34367
2022-03-01 09:39:35 -05:00
Elliott Mitchell
ad7dd51499 xen: switch to use headers in contrib
These headers originate with the Xen project and shouldn't be mixed with
the main portion of the FreeBSD kernel. Notably they shouldn't be the
target of clean-up commits.

Switch to use the headers in sys/contrib/xen.

Reviewed by: royger
2022-02-07 10:11:56 +01:00
Elliott Mitchell
b6da4ec609 xen: remove leftover bits missed in commit ac3ede5371
These fields are now unused, remove them.

Fixes: ac3ede5371 ('x86/xen: remove PVHv1 code')
Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D31206
2022-02-07 10:06:27 +01:00
Takanori Watanabe
eb815a7419 atrtc: Install address space handler for \_SB and its descendant.
SystemCMOS address space is accessible for system wide.
 So install address handler in \_SB space.

Reviewed by: jhb

Differential Revision: https://reviews.freebsd.org/D33892
2022-01-21 15:32:30 +09:00
Roger Pau Monné
e0516c7553 x86/apic: remove apic_ops
All supported Xen instances by FreeBSD provide a local APIC
implementation, so there's no need to replace the native local APIC
implementation anymore.

Leave just the ipi_vectored hook in order to be able to override it
with an implementation based on event channels if the underlying local
APIC is not virtualized by hardware. Note the hook cannot use ifuncs,
because at the point where ifuncs are resolved the kernel doesn't yet
know whether it will benefit from using the optimization.

Sponsored by: Citrix Systems R&D
Reviewed by: kib
Differential revision: https://reviews.freebsd.org/D33917
2022-01-18 10:19:04 +01:00
Roger Pau Monné
2450da6776 x86/xen: use x{2}APIC if virtualized by hardware
Instead of using event channels or hypercalls to deal with IPIs and
NMIs.

Using a hardware virtualized APIC should be faster than using any PV
interface, since the VM exit can be avoided.

Xen exposes whether the domain is using hardware assisted x{2}APIC
emulation in a CPUID bit.

Sponsored by: Citrix Systems R&D
2022-01-18 10:18:22 +01:00
Roger Pau Monné
ad15eeeaba x86/xen: fallback when VCPUOP_send_nmi is not available
It has been reported that on some AWS instances VCPUOP_send_nmi
returns -38 (ENOSYS). The hypercall is only available for HVM guests
in Xen 4.7 and newer. Add a fallback to use the native NMI sending
procedure when VCPUOP_send_nmi is not available, so that the NMI is
not lost.

Reported and Tested by: avg
MFC after: 1 week
Fixes: b2802351c1 ('xen: fix dispatching of NMIs')
Sponsored by: Citrix Systems R&D
2022-01-17 11:06:40 +01:00
Colin Percival
de1292c6ff Use CPUID leaf 0x40000010 for local APIC freq
Some VM systems announce the frequency of the local APIC via the
CPUID leaf 0x40000010.  Using this allows us to boot slightly
faster by avoiding the need for timer calibration.

Reviewed by:	markj
Sponsored by:	https://www.patreon.com/cperciva
2022-01-14 17:30:17 -08:00
Colin Percival
4a432614f6 TSC: Use 0x40000010 CPUID leaf for all VM types
While this CPUID leaf was originally only used by VMWare, other
hypervisors now also use it to announce the TSC frequency to guests.

This speeds up the boot process by 100 ms in EC2 and other systems,
by allowing the early calibration DELAY to be skipped.

Reviewed by:	markj
Sponsored by:	https://www.patreon.com/cperciva
2022-01-14 17:30:17 -08:00
Colin Percival
fd980feb57 Detect CPU type before asking VMWare for TSC freq
This allows us to set tsc_is_invariant and select appropriately
fenced versions of RDTSC based on the CPU type.

Reviewed by:	markj
Sponsored by:	https://www.patreon.com/cperciva
2022-01-14 17:30:17 -08:00
Austin Zhang
e1ef6c0ef2 atrtc: reads Century field from FADT table
The ACPI spec describes the FADT->Century field as:

    The RTC CMOS RAM index to the century of data value (hundred and
    thousand year decimals).  If this field contains a zero, then the
    RTC centenary feature is not supported.  If this field has a non-zero
    value, then this field contains an index into RTC RAM space that
    OSPM can use to program the centenary field.

Use this field to decide whether to program the CENTURY register
of the CMOS RTC device.

Reviewed by:	akumar3@isilon.com, dab, vangyzen
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D33667

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2022-01-13 11:24:00 -06:00
Roger Pau Monné
7d06c761c8 x86/madt: allow Xen guest to use x2APIC mode
The old bogus Xen versions that would deliver a GPF when writing to
the LAPIC MSR are likely retired, so it's safe to enable x2APIC
unconditionally now if available.

Tested by: avg
Reviewed by: kib
Sponsored by: Citrix Systems R&D
Differential revision: https://reviews.freebsd.org/D33877
2022-01-13 17:15:24 +01:00
Roger Pau Monné
ca46f3289d xen: use an hypercall for shutdown and reboot
When running as a Xen guest it's easier to use an hypercall in order
to do power management operations (power off, power cycle). Do this
for all supported guest types (HVM and PVH). Note that for HVM the
power operation could also be done using ACPI, but there's no reason
to differentiate between PVH and HVM.

While there fix the shutdown handler to properly differentiate between
power cycle and power off requests.

Reported by: Freddy DISSAUX
MFC: 1 week
Sponsored by: Citrix Systems R&D
2022-01-13 16:54:30 +01:00
Colin Percival
c2705ceaeb x86: Speed up clock calibration
Prior to this commit, the TSC and local APIC frequencies were calibrated
at boot time by measuring the clocks before and after a one-second sleep.
This was simple and effective, but had the disadvantage of *requiring a
one-second sleep*.

Rather than making two clock measurements (before and after sleeping) we
now perform many measurements; and rather than simply subtracting the
starting count from the ending count, we calculate a best-fit regression
between the target clock and the reference clock (for which the current
best available timecounter is used). While we do this, we keep track
of an estimate of the uncertainty in the regression slope (aka. the ratio
of clock speeds), and stop measuring when we believe the uncertainty is
less than 1 PPM.

In order to avoid the risk of aliasing resulting from the data-gathering
loop synchronizing with (a multiple of) the frequency of the reference
clock, we add some additional spinning depending upon the iteration number.

For numerical stability and simplicity of implementation, we make use of
floating-point arithmetic for the statistical calculations.

On the author's Dell laptop, this reduces the time spent in calibration
from 2000 ms to 29 ms; on an EC2 c5.xlarge instance, it is reduced from
2000 ms to 2.5 ms.

Reviewed by:	bde (previous version), kib
MFC after:	1 month
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D33802
2022-01-12 12:34:07 -08:00
John Baldwin
7def1e10b3 bus_dma: Deduplicate locking helper functions.
- Move busdma_lock_mutex to subr_bus_dma.c.

- Move _busdma_lock_dflt to subr_bus_dma.c.  This function was named a
  couple of different things previously.  It is not a public API but
  an internal helper used in place of a NULL pointer.  The prototype
  is in <sys/bus_dma.h> as not all backends include
  <sys/bus_dma_internal.h>.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D33694
2022-01-05 13:50:40 -08:00
John Baldwin
85b4607324 Deduplicate bus_dma bounce code.
Move mostly duplicated code in various MD bus_dma backends to support
bounce pages into sys/kern/subr_busdma_bounce.c.  This file is
currently #include'd into the backends rather than compiled standalone
since it requires access to internal members of opaque bus_dma
structures such as bus_dmamap_t and bus_dma_tag_t.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D33684
2022-01-05 13:50:40 -08:00
Mark Johnston
0e494a9e3f x86: Skip late calibration if our reference timer has low quality
Some AMD Geode-based systems end up using the 8254 PIT to calibrate the
TSC during late calibration, which doesn't work because that
timecounter's mask (65535) is much smaller than its frequency (1193182).
Moreover, early calibration is done against the 8254 timer anyway.

Work around the problem by simply using early calibration results if no
high-quality timecounters exist.

PR:		260868
Fixes:		22875f8879 ("x86: Implement deferred TSC calibration")
Reported and tested by:	mike@sentex.net, Stefan Hegnauer <stefan.hegnauer@gmx.ch>
Reviewed by:	imp, kib
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33730
2022-01-03 13:00:50 -05:00
Colin Percival
698727d637 Fix variable name: freq_khz -> freq
An earlier version of this code computed the TSC frequency in kHz.
When the code was changed to compute the frequency more accurately,
the variable name was not updated.

Reviewed by:	markj
Fixes:		22875f8879 x86: Implement deferred TSC calibration
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D33696
2022-01-02 13:07:53 -08:00
Colin Percival
9cb3288287 Skip TSC calibration if exact value known
It's possible that the "early" TSC calibration gave us a value which
is known to be exact; in that case, skip the later re-calibration.

MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D33695
2022-01-02 13:07:53 -08:00
Doug Moore
f1e7a532d1 busdma: _bus_dmamap_addseg repaired
A recent change introduced a one-off error into a test allowing
coalescing chunks into segments.  This fixes that error.

broke a check in _bus_dmamap_addseg on many architectures. This change makes it clear that it is not a particular range that is being boundary-checked, but the proposed union of the two adjacent ranges.
Reported by:	se
Reviewed by:	se
Fixes:	c606ab59e7 vm_extern: use standard address checkers everywhere
Differential Revision:	https://reviews.freebsd.org/D33715
2022-01-02 12:37:05 -06:00
Doug Moore
b7810e05ff x86-busdma - Add missing paren
Reported by:	jenkins
Fixes:	c606ab59e7 vm_extern: use standard address checkers everywhere
2021-12-31 02:33:54 -06:00
Doug Moore
c606ab59e7 vm_extern: use standard address checkers everywhere
Define simple functions for alignment and boundary checks and use them
everywhere instead of having slightly different implementations
scattered about. Define them in vm_extern.h and use them where
possible where vm_extern.h is included.

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D33685
2021-12-30 22:09:08 -06:00
Stefan Eßer
e2650af157 Make CPU_SET macros compliant with other implementations
The introduction of <sched.h> improved compatibility with some 3rd
party software, but caused the configure scripts of some ports to
assume that they were run in a GLIBC compatible environment.

Parts of sched.h were made conditional on -D_WITH_CPU_SET_T being
added to ports, but there still were compatibility issues due to
invalid assumptions made in autoconfigure scripts.

The differences between the FreeBSD version of macros like CPU_AND,
CPU_OR, etc. and the GLIBC versions was in the number of arguments:
FreeBSD used a 2-address scheme (one source argument is also used as
the destination of the operation), while GLIBC uses a 3-adderess
scheme (2 source operands and a separately passed destination).

The GLIBC scheme provides a super-set of the functionality of the
FreeBSD macros, since it does not prevent passing the same variable
as source and destination arguments. In code that wanted to preserve
both source arguments, the FreeBSD macros required a temporary copy of
one of the source arguments.

This patch set allows to unconditionally provide functions and macros
expected by 3rd party software written for GLIBC based systems, but
breaks builds of externally maintained sources that use any of the
following macros: CPU_AND, CPU_ANDNOT, CPU_OR, CPU_XOR.

One contributed driver (contrib/ofed/libmlx5) has been patched to
support both the old and the new CPU_OR signatures. If this commit
is merged to -STABLE, the version test will have to be extended to
cover more ranges.

Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do
no longer require that option.

The FreeBSD version has been bumped to 1400046 to reflect this
incompatible change.

Reviewed by:	kib
MFC after:	2 weeks
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D33451
2021-12-30 12:20:32 +01:00
Mark Johnston
0ecda8d5ae x86: Do not attempt to calibrate the LAPIC timer if no APIC is present
Reported and tested by:	Michael Butler <imb@protected-networks.net>
Reviewed by:	jhb, kib
Fixes:	62d09b46ad ("x86: Defer LAPIC calibration until after timecounters are available")
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33669
2021-12-28 17:47:49 -05:00