In the arm64 busdma we have an internal flag to signal when a tag is
for a cache-coherent device. In this case we don't need to adjust the
size and alignment of allocated buffers to be within a cache line.
The cache line adjustment was incorrectly using the coherent flag
passed in to bus_dma_tag_create and not the internal flag. Fix it to
use the latter to reduce the memory usage slightly.
Reviewed by: bz
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34763
To simplify the creation of the direct map (DMAP) region on arm64 move
it from the pre-C code into pmap. This simplifies the DMAP creation
as we can use the notmal index macros, and should reduce the number
of pages needed to hold the level 1 tables to just those needed.
Reviewed by: alc, dch
Tested by: dch, kevans
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34568
This more clearly differentiates system call arguments from integer
registers and return values. On current architectures it has no effect,
but on architectures where pointers are not integers (CHERI) and may
not even share registers (CHERI-MIPS) it is necessiary to differentiate
between system call arguments (syscallarg_t) and integer register values
(register_t).
Obtained from: CheriBSD
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D33780
On arm64 we can ask the hardware to perform cache operations from
userspace. These require read permission however when the memory is
unmapped the kernel will receive a write exception. Add a check to
see if the cause of the exception is from the cache and pass a memory
read fault type to the vm subsystem.
PR: 262836
Reported by: dch
Sponsored by: The FreeBSD Foundation
Following ARMARM sec D5.2.11, which says:
> Where an instruction results in an update to a System register,
> as is the case with the AT * address translation instructions,
> explicit synchronization must be performed before the result is
> guaranteed to be visible to subsequent direct reads of the
> PAR_EL1.
Reviewed By: andrew
MFC after: 3 weeks
Sponsored by: Ampere Computing
Differential Revision: https://reviews.freebsd.org/D34665
Bits 43:0 of the TLBI operand are bits 55:12 of the VA. Leaving
bits 63:55 of the VA in bits 51:44 of the operand might wind up
setting the TTL field (47:44) and accidentally restricting which
translation levels are flushed in the TLB.
Reviewed By: andrew
MFC after: 3 days
Sponsored by: Ampere Computing
Differential Revision: https://reviews.freebsd.org/D34664
This register set exposes the per-thread TLS register. It matches the
layout used by Linux on arm64. Linux does not implement this note for
32-bit arm.
Reviewed by: andrew, markj
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D34595
This includes adding support for NT_ARM_VFP for 32-bit binaries
running under aarch64 kernels both for ptrace(), and coredumps via the
kernel and gcore.
Reviewed by: andrew, markj
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D34448
Similar to fill_fpregs(), only invoke vfp_save_state() for curthread.
While here, zero the buffer if FP hasn't been started to avoid leaking
kernel stack memory.
Reviewed by: andrew, markj
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D34525
It's unneeded as it was just used to align KERNBASE to a level 2
block start address. KERNBASE was already aligned correctly.
Sponsored by: The FreeBSD Foundation
When the page size the kernel is built for is not the same as
EFI_PAGE_SIZE we need to increment the page index at a faster rate.
Add this adjustment to the arm64 EFIRT support in preperation for
experimental 16k PAGE_SIZE support.
Sponsored by: The FreeBSD Foundation
To support cc -pg on arm64 we need to implement .mcount. As clang and
gcc think it is function like it just needs to load the arguments
to _mcount and call it.
On gcc the first argument is passed in x0, however this is missing on
clang so we need to load it from the stack. As it's the caller return
address this will be at a known location.
PR: 262709
Reviewed by: emaste (earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34634
The ITS is defined to be disabled on a warm reset, but we may be coming
in via another kernel/hypervisor type setup where the ITS has been
previously configured then relinquished to the next kernel in the chain.
If it's enabled, the later configuration of GITS_BASER will almost
certainly fail -- clear it to prevent that.
Reviewed by: andrew
Sponsored by: Ampere Computing
Submitted by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34546
When using 16k or 64k pages atop will shift the address by more than
the needed amount for a tlbi instruction. Replace this with a new macro
to shift the address by 12 and use PAGE_SIZE in the for loop to let the
code work with any page size.
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34516
To allow for a future 16k or 64k page size we need to tell libkvm which
is being used. Add a flag field in unused space in minidumphdr and use
it to signal between the different options.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34548
When moving from the l1 index to l0 index we need to use the l1 shift
value not the l0 shift value. With 4k pages they are identical, however
with 16k pages we only have 2 l0 entries so the shift value is incorrect.
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34517
In particular, use a generic wrapper around struct regset rather than
requiring per-regset helpers. This helper replaces the MI
__elfN(note_prstatus) and __elfN(note_fpregset) helpers. It also
removes the need to explicitly dump NT_ARM_ADDR_MASK in the arm64
__elfN(dump_thread).
Reviewed by: markj, emaste
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D34446
When creating the DMAP region we may need to create level 2 page table
entries at the start and end of a block of memory. The code to do this
was almost identical so we can merge into a single function.
Sponsored by: The FreeBSD Foundation
They are in a different order to the TCR_TG1 values but appear to have
been copied incorrectly.
While here use TCR_TG0_4K in locore.S to make it explicit the userspace
page size is 4K.
Sponsored by: The FreeBSD Foundation
We assume EFI_PAGE_SIZE is the same as PAGE_SIZE, however this may not
be the case. Use the former when working with a list of pages from the
UEFI firmware so the correct size is used.
This will be needed on arm64 where PAGE_SIZE could be 16k or 64k in the
future. The other architectures have been updated to be consistent.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34510
We assume the pointer returned from get_pcpu will be consistent even
if the thread is moved to a new CPU. Fix this by partially reverting
63c858a04d to make get_pcpu a function again.
Sponsored by: The FreeBSD Foundation
The arm64 unknown exception will be raised when we execute an
instruction that id invalid or disabled. To help debug these print
the instruction that failed.
Sponsored by: The FreeBSD Foundation
To help with switching to a vdso sigtramp switch to passing through the
sigcode function when entering a signal. This ensures the return address
is within the function.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33690
We need to not include the MI _san.h files when builing some parts of
the kernel. Fix these checks in the arm64 header files.
Sponsored by: The FreeBSD Foundation
This can be used by debuggers to find which bits in a virtual address
should be masked off to get a canonical address. This is currently used
by the Pointer Authentication Code support to get its mask. It could also
be used if we support Top Byte Ignore for the same purpose.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34302
Use the new ofw_iicbus_set_devinfo() method to implant an OFW compatibility
string for a manually created RTC sub-device.
MFC after: 4 weeks
Reported by: archimedes.gaviola_at_gmail.com
bscott_at_bunyatech.com.au
Add a static assert for the siginfo{,32}_t, mcontext{,32}_t and
ucontext{,32}_t sizes. These are de-facto ABI options and cannot change
size ever.
Reviewed by: kib, andrew, jhb
Differential Revision: https://reviews.freebsd.org/D32958
We should clear the single step flag when entering a signal hander and
set it when returning. This fixes the ptrace__PT_STEP_with_signal test.
While here add support for userspace to set the single step bit as on
x86. This can be used by userspace for self tracing.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34170
When debugging 32-bit programs a debugger may insert a instruction that
will raise the undefined instruction trap. The kernel handles these
by raising a SIGTRAP, however the code was incorrect.
Fix this by using the expected TRAP_BRKPT signal code.
Sponsored by: The FreeBSD Foundation
