It does not work with ULE, which is the default scheduler for over a
decade.
Reviewed by: emaste, kib
Differential Revision: https://reviews.freebsd.org/D36094
Restructure dmar_qi_task() so as to reduce the number of invalidation
completion interrupts. Specifically, because processing completed
invalidations in dmar_qi_task() can take quite some time, don't reenable
completion interrupts until processing has completed a first time. Then,
check a second time after reenabling completion interrupts, so that
any invalidations that complete just before interrupts are reenabled
do not linger until a future invalidation might raise an interrupt.
(Recent changes have made checking for completed invalidations cheap; no
locking is required.)
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D36054
Since IOMMU map entries store a reference to the domain in which they
reside, there is no need to pass the domain to iommu_gas_free_entry(),
iommu_gas_free_space(), and iommu_gas_free_region().
Push down the acquisition and release of the IOMMU domain lock into
iommu_gas_free_space() and iommu_gas_free_region().
Both of these changes allow for simplifications in the callers of the
functions without really complicating the functions themselves.
Moreover, the latter change eliminates the direct use of the IOMMU
domain lock from the x86-specific DMAR code.
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35995
Replace the DMAR unit's tlb_flush TAILQ by a custom list implementation
that enables dmar_qi_task() to dequeue entries without holding the DMAR
lock.
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35951
Correct 8bc3673847. When iommu_domain_unload_entry() performs a
synchronous IOTLB invalidation, it must call dmar_domain_free_entry()
to remove the entry from the domain's RB_TREE.
Push down the acquisition and release of the DMAR lock into the
recently introduced function dmar_qi_invalidate_sync_locked() and
remove the _locked suffix.
MFC with: 8bc3673847
Eliminate a possible case of use-after-free in an error handling path
after a mapping failure. Specifically, eliminate IOMMU_MAP_ENTRY_QI_NF
and instead perform the IOTLB invalidation synchronously. Otherwise,
when iommu_domain_unload_entry() is called and told not to free the
IOMMU map entry, the caller could free the entry before dmar_qi_task()
is finished with it.
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35878
With clang 15, the following -Werror warnings are produced:
sys/x86/linux/linux_vdso_selector_x86.c:44:28: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
linux_vdso_tsc_selector_idx()
^
void
sys/x86/linux/linux_vdso_selector_x86.c:62:28: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
linux_vdso_cpu_selector_idx()
^
void
This is because linux_vdso_tsc_selector_idx() and
linux_vdso_cpu_selector_idx are declared with (void) argument lists, but
defined with empty argument lists. Make the definitions match the
declarations.
MFC after: 3 days
It is safe to test and clear the Invalidation Wait Descriptor
Complete flag before acquiring the DMAR lock in dmar_qi_task(),
rather than waiting until the lock is held.
Reviewed by: kib
MFC after: 2 weeks
On SMP systems, cpu_reset broadcasts a message telling the APs to stop
themselves, and then the BSP waits 1 second before actually resetting
itself; this behaviour dates back to 1998-05-17.
I assume that this delay was added in order to allow the APs to stop
themselves before the BSP resets; but we wait until the APs have all
acknowledged entering the "stopped" state, so it no longer seems to
serve any purpose.
Reviewed by: jhb, kib
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D35797
This is not completely exhaustive, but covers a large majority of
commands in the tree.
Reviewed by: markj
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35583
Eliminate the unroll_entry field from struct iommu_map_entry, shrinking
the struct by 16 bytes on 64-bit architectures.
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35769
ULE's tdq_notify() tries to avoid delivering IPIs to the idle thread.
In particular, it tries to detect whether the idle thread is running.
There are two mechanisms for this:
- tdq_cpu_idle, an MI flag which is set prior to calling cpu_idle(). If
tdq_cpu_idle == 0, then no IPI is needed;
- idle_state, an x86-specific state flag which is updated after
cpu_idleclock() is called.
The implementation of the second mechanism is racy; the race can cause a
CPU to go to sleep with pending work. Specifically, cpu_idle_*() set
idle_state = STATE_SLEEPING, then check for pending work by loading the
tdq_load field of the CPU's runqueue. These operations can be reordered
so that the idle thread observes tdq_load == 0, and tdq_notify()
observes idle_state == STATE_RUNNING.
Some counters indicate that the idle_state check in tdq_notify()
frequently elides an IPI. So, fix the problem by inserting a fence
after the store to idle_state, immediately before idling the CPU.
PR: 264867
Reviewed by: mav, kib, jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35777
Stop including the current CPU in all event messages, since it's already
saved in KTR log entries and thus is redundant. All eventtimer traces
occur in a context where CPU migration is not possible.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Some command definitions were forced to use DB_FUNC in order to specify
their required flags, CS_OWN or CS_MORE. Use the new macros to simplify
these.
Reviewed by: markj, jhb
MFC after: 3 days
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35582
All boot loaders for the last 6 years set acpi.rsdp in addition to the
hints. This was planned for removal ~5 years ago. Belatedly remove it
from here.
Sponsored by: Netflix
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D35633
Do not assume that start_info will always be loaded at the highest
memory address, and instead check the position of all the loaded
elements in order to find the last loaded one, and thus a likely safe
place to use as early boot allocation memory space.
Reported by: markj, cperciva
Sponsored by: Citrix Systems R&D
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D35628
The third argument to this function indicates whether the supplied
ticker is fixed or variable, i.e. requiring calibration. Give this
argument a type and name that better conveys this purpose.
Reviewed by: kib, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35459
The mptable_hostb driver is a child of legacy0 and has legacy bus
ivars, not PCI or PCI bridge ivars.
PR: 264819
Reported by: Dennis Clarke <dclarke@blastwave.org>
Diagnosed by: avg
Reviewed by: avg
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35548
On arm64, testing pc_curpcb != NULL is not correct since pc_curpcb is
set in pmap_switch() while the bootstrap stack is still in use. As a
result, smp_after_idle_runnable() can free the boot stack prematurely.
Take a different approach: use smp_rendezvous() to wait for all APs to
acknowledge an interrupt. Since APs must not enable interrupts until
they've entered the scheduler, i.e., switched off the boot stack, this
provides the right guarantee without depending as much on the
implementation of cpu_throw(). And, this approach applies to all
platforms, so convert x86 and riscv as well.
Reported by: mmel
Tested by: mmel
Reviewed by: kib
Fixes: 8db2e8fd16 ("Remove the secondary_stacks array in arm64 and riscv kernels.")
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35435
Replace sigframe sf_extramask by native sigset_t and use it to
store/restore the thread signal mask without conversion to/from
Linux signal mask.
Pointy hat to: dchagin
MFC after: 2 weeks
When some APICs are disabled by tunables, some cpu groups could end up
empty. An empty cpu group causes the system to panic because not all
functions handle them correctly. Additionally, it's wasted time to
handle and inspect empty cpu groups. Therefore, just don't create them.
Reviewed by: kib, avg, cem
Sponsored by: Beckhoff Automation GmbH & Co. KG
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24927
The signal trampoine-related definitions are used only in the MD part
of code, wherefore moved from everywhere used linux.h to separate MD
headers.
MFC after: 2 weeks
Deduplicate code to iterate over the bpages list in a bus_dmamap_t
freeing bounce pages during bus_dmamap_unload.
Reviewed by: imp
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D34967
While here, use a modern function declaration for smbios_modevent and
vpd_modevent.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D34996
Previously it was disabled right before translation was enabled.
This way the disable logic is still executed even when translation
is not be activated, e.g. with hw.iommu.dma=0 tunable set.
On some platforms we need to disable PMR in order for core dump to work.
At the same time it was observed that enabling translation has
a significant impact on network performance.
With this patch PMR can be disabled, with IOMMU translation not being
turned on by appending the following to the loader.conf:
hw.dmar.enable=1
hw.dmar.pmr.disable=1
hw.dmar.dma=0
Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D34907
These files no longer depend on the macros required when these checks
were added.
PR: 263102 (exp-run)
Reviewed by: brooks, imp, emaste
Differential Revision: https://reviews.freebsd.org/D34804
All supported compilers (modern versions of GCC and clang) support
this.
Many places didn't have an #else so would just silently do the wrong
thing. Ancient versions of icc (the original motivation for this) are
no longer a compiler FreeBSD supports.
PR: 263102 (exp-run)
Reviewed by: brooks, imp
Differential Revision: https://reviews.freebsd.org/D34797
PR: 263124
Fixes: 62d09b46ad ("x86: Defer LAPIC calibration until after timecounters are available")
Reviewed by: kib, jhb, emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34830
This register set contains the values of the fsbase and gsbase
registers. Note that these registers can already be controlled
individually via ptrace(2) via MD operations, so the main reason for
adding this is to include these register values in core dumps. In
particular, this will enable looking up the value of TLS variables
from core dumps in gdb.
The value of NT_X86_SEGBASES was chosen to match the value of
NT_386_TLS on Linux. The notes serve similar purposes, but FreeBSD
will never dump a note equivalent to NT_386_TLS (which dumps a single
segment descriptor rather than a pair of addresses) and picking a
currently-unused value in the NT_X86_* range could result in a future
conflict.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D34650
