The PCB_FP_STARTED is used to indicate that the current VFP context
has been used since either 1. the start of the thread, or 2. exiting
a kernel FP context.
When case 2 was added to the kernel this could cause incorrect results
to be returned when a thread exits the kernel FP context and fill_fpregs
is called before it has restored the VFP state, e.g. by trappin on a
userspace VFP instruction.
In both of the cases the base save area is still valid so reduce the
use of the PCB_FP_STARTED flag check to help decide if we need to
store the current threads VFP state.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D37994
Commit 86a994d653 initialized
thread0.td_kstack_pages to KSTACK_PAGES. Due to the lack of an
include of opt_kstack_pages.h it used the fallback value of 4 from
machine/param.h. This meant that increasing KSTACK_PAGES in the kernel
config resulted in a panic in _epoch_enter_preempt as the following
assertion was false during network stack setup:
MPASS((vm_offset_t)et >= td->td_kstack &&
(vm_offset_t)et + sizeof(struct epoch_tracker) <=
td->td_kstack + td->td_kstack_pages * PAGE_SIZE);
Switch to initializing with kstack_pages following other architectures.
Reviewed by: imp, markj
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D38048
The SPDX-License-Identifier was wrong in the Arm CoreLink CMN-600
driver files. It used the incorrect FreeBSD variant of the BSD-2-Clause
identifier. According to [1] all files should use BSD-2-Clause.
[1] https://tools.spdx.org/app/check_license/
Reported by: emaste
Sponsored by: Arm Ltd
In 0a9a4d2cd6 a check for OPT_ACPI was added to the hwpmc Makefile
to fix loading the module in a kernel where ACPI has been disabled.
This broke loading the module when ACPI was enabled in the build as
OPT_ACPI isn't a Makefile macro so was always disabled.
Move this check to the C files where the DEV_ACPI macro does exist.
Reviewed by: gnn
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D37773
The td_ast member is an int so only 4 bytes, yet we were using an 8 byte
load and thus also got td_inhibitors in the upper bits. The code prior
to the commit that introduced td_ast did also do a bogus 8 byte load of
td_flags but masked the flags so arguably was correct, if dodgy. Now
that we're using the right width for the load we can also fold the
immediate offset back into the load; because td_ast is at an odd
multiple of 4 bytes from the start of struct thread the normal scaled
load couldn't be used with such an immediate offset when doing an 8 byte
load due to its limited immediate range, but we can use a scaled load
once more now that the offset is a multiple of the load width.
Reviewed by: andrew, kib
Fixes: c6d31b8306 ("AST: rework")
Differential Revision: https://reviews.freebsd.org/D37751
The order of asserting/deasserting the resets doesn't matter so use
the new hwreset_array to manage them all.
Reviewed by: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37358
The SCMI specification describes a set of standard interfaces for power,
performance and system management.
SCMI is extensible and provides interfaces to access functions which are
often implemented in firmwares in the System Control Processor (SCP).
This implements Shared Memory-based transfer, which is one of the ways on
how messages are exchanged between agents and the platform.
This includes a driver for ARM Message Handling Unit (MHU) Doorbell, which
is a mechanism that the caller can use to alert the callee of the presence
of a message.
The support implements clock management interface. For instance this allows
us to control HDMI pixel clock on ARM Morello Board.
Tested on ARM Morello Board.
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D37316
Reviewed by: manu
Sponsored by: UKRI
Partially from: https://reviews.freebsd.org/D36027
This can be eventually improved or simplified or fixed if necessary.
Following devices work with proper drivers and with the necessary clocks:
Native networking via eqos driver
USB3 and USB2
PCIe support is working but a bit picky about what hardware it supports (but so is Linux)
SD & (e)MMC
With the EDK2 loader video also works
Supported hardwares are Quartz64, NanoPI R5S and Firefly Station P2, more to come as DTS files gets done.
On amd64, don't abort promotion due to a missing accessed bit in a
mapping before possibly write protecting that mapping. Previously,
in some cases, we might not repromote after madvise(MADV_FREE) because
there was no write fault to trigger the repromotion. Conversely, on
arm64, don't pointlessly, yet harmlessly, write protect physical pages
that aren't part of the physical superpage.
Don't count aborted promotions due to explicit promotion prohibition
(arm64) or hardware errata (amd64) as ordinary promotion failures.
Reviewed by: kib, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36916
When Linux loads a new kernel via kexec, somtiems it must reserve memory
for devices that are still active (and typically can't be reset or
shutdown). When present, this table is a linked list of ranges that are
still in use that the OS must avoid using.
Mark these areas as reserved.
This is part of the GICv3 workaround code where we must use the PA
addresses already programmed into the GICv3 when we take over. This part
ensure we don't allocate the mmeory for anything else.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D37440
It would be nice to be able to pass an arbitrary pointer to the callback
code. Add one, and pass NULL in all the places that we do that today.
As noted by andrew@, we should likely refactor this into MI code and use
it here and amd64, but for the future.
Sponsored by: Netflix
Reviewed by: rpokala
Differential Revision: https://reviews.freebsd.org/D37439
When PV_STATS is defined, freed is used. Otherwise it isn't. Mark it as
__pvused and define __pvused appropriately.
Sponsored by: Netflix
Reviewed by: tsoome, rpokala, andrew
Differential Revision: https://reviews.freebsd.org/D37438
As with amd64 pmap introduce per-superpage locks backed by pages
allocated by their respective domains.
This significiantly reduces lock contantion from pmap when running
poudriere on a 160 core Ampere Altra server
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36310
A misaligned frame pointer is certainly not a valid frame pointer and
with strict alignment enabled (as on CHERI) can cause panics when it is
loaded from later in the code.
This is a recommit of 40e0fa10f5 with
is_aligned() corrected to __is_aligned().
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D34646
The RK3328 dts doesn't have the glue node so we need the dwc3 driver
to attach directly.
Differential Revision: https://reviews.freebsd.org/D37396
Sponsored by: Beckhoff Automation GmbH & Co. KG
These were originally in locore.S as they are only needed so we have
a valid value to put into the vbar_el2 register. As these will soon
be used by bhyve so move them to a new file as we already have with
the EL1 exception vectors in exception.S.
Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Zero the vttbr_el2 register on each CPU so we can tell if we are
running the host or guest kernel from a hypervisor.
Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
For completeness add accessors for the MIDR field. As the field is
always 0xf on arm64 it is unneeded in the current MICR handling, but
will be used in the vmm module for bhyve.
Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: The FreeBSD Foundation
These all work on stage 1 tables. Rename them so we can add similar
functions that operate on stage 2 tables.
Reviewed by: alc, markj, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37302
When modifying a stage 2 mapping we may need to call into the
hypervisor to invalidate the TLB. Until it is known if the cost of
this operation is less than the performance gains superpages offers
disable their use.
Reviewed by: kib. markj
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37299
A misaligned frame pointer is certainly not a valid frame pointer and
with strict alignment enabled (as on CHERI) can cause panics when it is
loaded from later in the code.
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D34646
The M1 has no EL3, so we're limited to a spin-table implementation if we
want to eventually use bhyve on it. Implement spin-table now, but note
that we still prefer PSCI where possible.
Reviewed by: mmel
Differential Revision: https://reviews.freebsd.org/D34661
The pull-up/pull-down register offset was wrong on the Rockchip rk356x.
It was set such that the driver would modify the IOMUX control registers.
This seems to work with the current device tree files, but fails with
upstream files. Fix the offset so the later calculation has the correct
offset for the pull-up/pull-down control register.
Sponsored by: The FreeBSD Foundation
With PERTHREAD_SSP configured, the compiler's stack-smashing protection
uses a per-thread canary value instead of a global value. The value is
stored in td->td_md.md_canary; the sp_el0 register always contains a
pointer to that value, and certain functions selected by the compiler
will store the canary value on the stack as a part of the function
prologue (and will verify the copy as part of the epilogue). In
particular, the thread structure may be accessed.
This happens to occur in data_abort(), which leads to the same problem
addressed by commit 2c10be9e06 ("arm64: Handle translation faults for
thread structures"). This commit fixes that directly, by disabling SSP
in data_abort() and a couple of related functions by using a function
attribute. It also moves the update of sp_el0 out of C code in case
the compiler decides to start checking the canary in pmap_switch()
someday.
A different solution might be to move the canary value to the PCB, which
currently lives on the kernel stack and isn't subject to the same
problem as thread structures (if only because guard pages inhibit
superpage promotion). However, there isn't any particular reason the
PCB has to live on the stack today; on amd64 it is embedded in struct
thread, reintroducing the same problem. Keeping the reference canary
value at the top of the stack is also rather dubious since it could be
clobbered by a sufficiently large stack overflow.
A third solution could be to go back to the approach of commit
5aa5420ff2, and modify UMA to use the direct map for thread structures
even if KASAN is enabled. But, transient promotions and demotions in
the direct map are possible too.
Reviewed by: alc, kib, andrew
MFC after: 1 month
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37255
This reverts commit fe36346a89.
The arm64 Hyper-v code now checks it is running under Hyper-v before
calling into the hypervisor.
Sponsored by: The FreeBSD Foundation
The break-before-make requirement poses a problem when promoting or
demoting mappings containing thread structures: a CPU may raise a
translation fault while accessing curthread, and data_abort() accesses
the thread again before pmap_fault() can translate the address and
return.
Normally this isn't a problem because we have a hack to ensure that
slabs used by the thread zone are always accessed via the direct map,
where promotions and demotions are rare. However, this hack doesn't
work properly with UMA_MD_SMALL_ALLOC disabled, as is the case with
KASAN configured (since our KASAN implementation does not shadow the
direct map and so tries to force the use of the kernel map wherever
possible).
Fix the problem by modifying data_abort() to handle translation faults
in the kernel map without dereferencing "td", i.e., curthread, and
without enabling interrupts. pmap_klookup() has special handling for
translation faults which makes it safe to call in this context. Then,
revert the aforementioned hack.
Reviewed by: kevans, alc, kib, andrew
MFC after: 1 month
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37231
