When vm_map_remove() is called from vm_swapout_map_deactivate_pages()
due to swapout, PKRU attributes for the removed range must be kept
intact. Provide a variant of pmap_remove(), pmap_map_delete(), to
allow pmap to distinguish between real removes of the UVA mappings
and any other internal removes, e.g. swapout.
For non-amd64, pmap_map_delete() is stubbed by define to pmap_remove().
Reported by: andrew
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39556
To allow it to be used before ENTRY we need to ensure the symbol is
in the .text section. It also needs to be aligned correctly.
While here mark the symbol type as a function as in the ENTRY macro.
Reported by: jrtc27
Sponsored by: Arm Ltd
Handle data-lanes property for pcie phy and set it accordingly.
This makes devices attached to pcie3 work properly.
For some RK3568 based boards, RTL8125B based device is
connected it. So with this, realtek-re-kmod driver attaches
and works.
Partially obtained from OpenBSD.
Tested on NanoPI-R5S, FireFly Station P2 boards.
NETLINK is going to replace rtsock and a number of other ioctl/sysctl interfaces.
In-base utilies such as route(8), netstat(8) and soon ifconfig(8)
are being converted to use netlink sockets as a transport between
kernel and userland.
In the current configuration, it still possible have the kernel
without NETLINK (`nooptions NETLINK`) and use the aforementioned
utilies by buidling the world with `WITHOUT_NETLINK` src.conf knob.
However, this approach does not cover the cases when person unintentionally
builds a custom kernel without netlink and tries to use the standard userland.
This change adds `option NETLINK` to the default options for each
architecture, fixing the custom kernel issue.
For arm, this change uses `std.armv6` and `std.armv7` (netlink already in)
instead of DEFAULTS.
Reviewed By: imp
Differential Revision: https://reviews.freebsd.org/D39339
Use the GICD_SIZE macro (0x10000), which is half the size of the current
fixed-sized mapping (128 * 1024 == 0x20000).
In ARM64 Hyper-V instances, it seems the Distributor's registers are
located immediately preceding a range of physical memory in the bus
address space. Thus, when ram0 is attaching and attempts to reserve
SYS_RES_MEMORY resources corresponding to its physmem ranges, it fails,
because the first 0x10000 bytes of this range are already owned by gic0.
PR: 270415
Reported by: whu
Tested by: whu
Differential Revision: https://reviews.freebsd.org/D39260
init_pagetables is mapped into the segment containing the BSS, but does
not get zeroed by locore. It is used for bootstrap page table pages.
It happens that the bootstrap kernel stack is also placed in that
section, but there's no reason it shouldn't live in the BSS, so move it
there. No functional change intended.
Reviewed by: andrew
MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D39367
The ENTRY macro adds instructions to the start of a function but not
EENTRY. To use these instructions in both functions move the EENTRY
use before the ENTRY use.
Sponsored by: Arm Ltd
On arm64, the PCB is stored at the top of the thread stack. For thread0
this comes from the static "initstack" region, which is placed in the
.init_pagetable section, which is not part of the BSS and thus doesn't
get zeroed by locore. (See the comment in ldscript.arm64.) It is thus
possible for the pcb_flags field to be uninitialized, which can result
in PCB_SINGLE_STEP being set.
Fix this by simply initializing the field. A separate commit will move
initstack out of the .init_pagetable section, since it has no reason to
be there, but it is preferable to explicitly initialize PCB fields
anyway. In particular, regular kernel stacks are not zeroed upon
allocation, so we should be consistent here.
Reviewed by: andrew
MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D39343
Rather than falling through to the default case handle the unknown
exception with its own panic message. As ESR_EL1 is zero for this
exception stop printing it.
Sponsored by: Arm Ltd
Previously this would zero out x18 in the pcb, now it's attacking the
innocent pcb_onfault -- drop it entirely.
This technically fixes
e605b87a9e ("Save only callee-saved registers in pcb"), but it's
harmless until the below commit trims down pcb_x.
Reported by: mmel
Reviewed by: andrew, mmel
Fixes: 1c1f31a5e5 ("Remove unused registes from the arm pcb")
Differential Revision: https://reviews.freebsd.org/D39277
This entails:
- Marking some obvious candidates for __nosanitizeaddress
- Similar trap frame markings as amd64, for similar reasons
- Shadow map implementation
The shadow map implementation is roughly similar to what was done on
amd64, with some exceptions. Attempting to use available space at
preinit_map_va + PMAP_PREINIT_MAPPING_SIZE (up to the end of that range,
as depicted in the physmap) results in odd failures, so we instead
search the physmap for free regions that we can carve out, fragmenting
the shadow map as necessary to try and fit as much as we need for the
initial kernel map. pmap_bootstrap_san() is thus after
pmap_bootstrap(), which still included some technically reserved areas
of the memory map that needed to be included in the DMAP.
The odd failure noted above may be a bug, but I haven't investigated it
all that much.
Initial work by mhorne with additional fixes from kevans and markj.
Reviewed by: andrew, markj
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D36701
These were kept for ABI reasons. Remove them and bump __FreeBSD_version
so debuggers can be updated to use the new layout.
Reviewed by: jhb
Sponsored by: Arm Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35378
The size of the spsr field in struct reg has changed. Mask the bits
that userspace doesn't know about out as they may be invalid.
While here add a comment why we don't need compat support in set_regs.
Sponsored by: Arm Ltd
It was previously possible for the fault address register to get
clobbered before it was saved. This small window occurred when an
additional exception was encountered inside the exception handler,
overwriting the previous value.
Commit f29942229d ("Read the arm64 far early in el0 exceptions")
patched this issue, but avoided changing the trapframe since this could
be considered a KBI change in FreeBSD 13.
Revert the above fix and save the fault address in the trapframe
instead. This saves the fault address even earlier in the exception
handling process, and is a more robust and simple fix.
Reviewed by: andrew, jhb, jrtc27
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D38984
For the Exception Syndrome Register, ESR_ELx, the upper 32b were
previously unused, but now may contain additional exception info as of
Armv8.7 (FEAT_LS64).
Extend ESR from u32->u64 in exception handling code to support this. In
addition, also extend Saved Program Status Register SPSR_ELx in the same
way to allow for future extensions.
Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D38983
sig_atomic_t is defined as a long and thus is 64-bit on arm64. For some
reason its limit was incorrectly specified as a 32-bit number. This had
the unfortunate side effect of causing gnulib to override most of the
definitions in stdint.h. On CheriBSD this breaks all software that uses
gnulib in annoying and hard to debug ways.
Technically updating the limits might be an ABI change, but these
defines are largely unused (the only use in tree is in the libc++ test
suite where it's use an assertion that will fail due to this bug).
Further, since the underlying type remains the same, we're just
increasing the range of values a paranoid program might use.
Reviewed by: andrew, emaste
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D39193
Add macros for offsets of macros we set in the arm64 pcb pcb_x array.
This will simplift reducing the size of this array in a later change.
Sponsored by: Arm Ltd
Make a pass at the various nexus implementations, fixing some very minor
style issues, obsolete comments, etc.
The method declaration section has become unwieldy in many respects.
Attempt to tame it by:
- Using generated method typedefs
- Grouping methods roughly by category, and then alphabetically.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D38495
Adding a missing include file, which provides the definition of
SYSCTL_INT.
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D39149
Move device memory to a weaker type. The new device memory type allows
the system to acknowledge a write to a device before the write has
completed. This is inline with VM_MEMATTR_DEVICE on armv6/armv7.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D38945
To allow for debugging after changing the arm64 VM_MEMATTR_DEVICE
memory type add a new set of tunables to tell the kernel to use
non-posted memory.
This adds the following tunables:
- kern.force_nonposted: When set to non-zero the kernel will use
non-posted memory for all device allocations.
- hint.<dev>.<unit>.force_nonposted: As above, however only forces
non-posted memory on the named device.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D38944
We only ever build a 4 level page table for the Arm SMMU. Remove the
support for a 3 level table.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D38949
These are SMMU (and MALI GPU) specific. Give them a SMMU specific name.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D38948
The fields we need to adjust are different in stage 1 and stage 2
tables. Handle this by adding variables to hold the bits to check,
set, and clear.
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37399
Consolidate add_efi_map_entry() and exclude_efi_map_entry() into a
single function, handle_efi_map_entry(), so that the exact set of entry
types handled is the same in the addition or exclusion cases. Before,
exclude_efi_map_entry() had a 'default' case that would exclude all
entry types that were not listed explicitly in the switch statement.
Logically, we do not need to exclude a range that could not possibly be
added to physmem, and we do not need to exclude bus ranges that are not
physical memory, for example EFI_MD_TYPE_IOMEM.
Since physmem's ram0 device will reserve bus memory resources for its
owned ranges, this was preventing attachment of the watchdog device on
the RPI4B. For some reason its region of memory-mapped I/O appeared in
the EFI memory map (with the aforementioned EFI_MD_TYPE_IOMEM type).
This change fixes the attachment issue, as we prevent the physmem API
from messing with this range of bus space.
PR: 270044
Reported by: karels, Mark Millard
Reviewed by: andrew, karels, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39003
To invalidate stage 2 mappings on arm64 we may need to call into the
hypervisor so add a function pointer that bhyve can use to implement
this.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37254
Hyper-V does not provide Mellanox hardware, some of Azure's instances
do, thus the configuration to enable them does not belong in the generic
std.hyperv config.
Fixes: 15e7fa83ef ("arm64: Hyper-V: Add vPCI and Mellanox driver modules into build")
This is the only mapping remaining which needs to respect nonposted-mmio
to avoid breaking the boot on Apple silicon.
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D38920
To allow hardware to work around a broken memory bus where we need to
support the nonposted-mmio flag.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D34333
On arm64 PCI config memory is expected to be mapped with a non-posted
device type. To handle this use the new bus_map_resource support in
arm64 to map memory with the new VM_MEMATTR_DEVICE_NP attribute. This
memory has already been allocated and activated, it just needs to be
mapped.
Reviewed by: kevans, mmel
Differential Revision: https://reviews.freebsd.org/D30079
