Commit Graph

1755 Commits

Author SHA1 Message Date
Andrew Turner
b6fd96ed0b Add the PMCR_EL0.N arm64 register field
Sponsored by:	Innovate UK
2022-03-11 11:21:09 +00:00
Andrew Turner
a1b4e4fa9a Correct the location of the arm64 PMCR register
This was one of the last registers to not be in alphabetical order in
armreg.h. Fix this to make it easier to find.

Sponsored by:	Innovate UK
2022-03-11 11:21:09 +00:00
Andrew Turner
2abeef7300 Sort the M* and P* arm64 registers
Sponsored by:	Innovate UK
2022-03-11 11:21:09 +00:00
John Baldwin
6b71405bfe Store core dump notes for all valid register sets for FreeBSD processes.
In particular, use a generic wrapper around struct regset rather than
requiring per-regset helpers.  This helper replaces the MI
__elfN(note_prstatus) and __elfN(note_fpregset) helpers.  It also
removes the need to explicitly dump NT_ARM_ADDR_MASK in the arm64
__elfN(dump_thread).

Reviewed by:	markj, emaste
Sponsored by:	University of Cambridge, Google, Inc.
Differential Revision:	https://reviews.freebsd.org/D34446
2022-03-10 15:40:19 -08:00
Andrew Turner
5055ffaeba Remove an unneeded memset from the arm64 pmap
There is no need to zero pagetable_dmap as we already did it when
creating the page tables in locore.S

Sponsored by:	The FreeBSD Foundation
2022-03-10 19:13:49 +00:00
Andrew Turner
ba3b602001 Split out creating the arm64 L2 dmap entries
When creating the DMAP region we may need to create level 2 page table
entries at the start and end of a block of memory. The code to do this
was almost identical so we can merge into a single function.

Sponsored by:	The FreeBSD Foundation
2022-03-10 18:17:06 +00:00
Andrew Turner
854d5a4f72 Remove l1ptfrom pmap_early_vtophys on arm64
The first argument was unused as we use an address translation
instruction to get the physical address.

Sponsored by:	The FreeBSD Foundation
2022-03-10 18:17:06 +00:00
Andrew Turner
a19acfd56c Fix the arm64 DMAP start address in a comment
Sponsored by:	The FreeBSD Foundation
2022-03-10 11:00:44 +00:00
Andrew Turner
f62e099ed5 Fix the TCR_TG0 values
They are in a different order to the TCR_TG1 values but appear to have
been copied incorrectly.

While here use TCR_TG0_4K in locore.S to make it explicit the userspace
page size is 4K.

Sponsored by:	The FreeBSD Foundation
2022-03-10 10:54:39 +00:00
Andrew Turner
9cf15aefb9 Fix the spelling of EFI_PAGE_SIZE
We assume EFI_PAGE_SIZE is the same as PAGE_SIZE, however this may not
be the case. Use the former when working with a list of pages from the
UEFI firmware so the correct size is used.

This will be needed on arm64 where PAGE_SIZE could be 16k or 64k in the
future. The other architectures have been updated to be consistent.

Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34510
2022-03-10 10:43:54 +00:00
Andrew Turner
e3f7087489 Add more arm64 PAC identification fields
Sponsored by:	The FreeBSD Foundation
2022-03-10 10:43:48 +00:00
Andrew Turner
ed30663426 Make the arm64 get_pcpu a function again
We assume the pointer returned from get_pcpu will be consistent even
if the thread is moved to a new CPU. Fix this by partially reverting
63c858a04d to make get_pcpu a function again.

Sponsored by:	The FreeBSD Foundation
2022-03-08 11:54:58 +00:00
Andrew Turner
e793a55a74 Print the instruction for the unknown exception
The arm64 unknown exception will be raised when we execute an
instruction that id invalid or disabled. To help debug these print
the instruction that failed.

Sponsored by:	The FreeBSD Foundation
2022-03-08 11:54:58 +00:00
Andrew Turner
db3a1eec37 Rework the arm64 sigtramp code to be a trampoline
To help with switching to a vdso sigtramp switch to passing through the
sigcode function when entering a signal. This ensures the return address
is within the function.

Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33690
2022-03-07 10:03:07 +00:00
Andrew Turner
d8da59ddf8 Fix the sanitizer header checks on arm64
We need to not include the MI _san.h files when builing some parts of
the kernel. Fix these checks in the arm64 header files.

Sponsored by:	The FreeBSD Foundation
2022-02-23 14:50:56 +00:00
Andrew Turner
ec1ecf78ae Add the char and short atomic operations on arm64
Sponsored by:	The FreeBSD Foundation
2022-02-23 14:45:28 +00:00
Andrew Turner
6713be3159 Add NT_ARM_ADDR_MASK
This can be used by debuggers to find which bits in a virtual address
should be masked off to get a canonical address. This is currently used
by the Pointer Authentication Code support to get its mask. It could also
be used if we support Top Byte Ignore for the same purpose.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34302
2022-02-22 17:10:35 +00:00
Emmanuel Vadot
90288dcb4c arm64: conf: Remove options EXT_RESOURCES
It is now unused in kernel code.

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D33836
2022-02-21 17:29:09 +01:00
Michal Meloun
a534b50e24 tegra: Improve the detection of the secondary function (RTC) of the MAX77620.
Use the new ofw_iicbus_set_devinfo() method to implant an OFW compatibility
string for a manually created RTC sub-device.

MFC after:	4 weeks
Reported by:	archimedes.gaviola_at_gmail.com
		bscott_at_bunyatech.com.au
2022-02-20 12:26:04 +01:00
Warner Losh
3988ca5aab aarch64: Add static asssert for context size
Add a static assert for the siginfo{,32}_t, mcontext{,32}_t and
ucontext{,32}_t sizes. These are de-facto ABI options and cannot change
size ever.

Reviewed by:		kib, andrew, jhb
Differential Revision:	https://reviews.freebsd.org/D32958
2022-02-10 14:32:20 -07:00
Andrew Turner
31cf95cec7 Stop single stepping in signal handers on arm64
We should clear the single step flag when entering a signal hander and
set it when returning. This fixes the ptrace__PT_STEP_with_signal test.

While here add support for userspace to set the single step bit as on
x86. This can be used by userspace for self tracing.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34170
2022-02-07 15:03:23 +00:00
Andrew Turner
67dc576bae Fix the signal code on 32-bit breakpoints on arm64
When debugging 32-bit programs a debugger may insert a instruction that
will raise the undefined instruction trap. The kernel handles these
by raising a SIGTRAP, however the code was incorrect.

Fix this by using the expected TRAP_BRKPT signal code.

Sponsored by:	The FreeBSD Foundation
2022-02-07 11:56:04 +00:00
Gordon Bergling
b6724f7004 tegra: Fix a common typo in source code comments
- s/ajusted/adjusted/

MFC after:	3 days
2022-02-06 17:31:05 +01:00
Andrew Turner
664640ba6c Sort the names of the arm64 debug registers
While here clean up the names for the naming convention of the other
registers in this file.

Reviewed by:	kib, mhorne (earlier version)
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34060
2022-02-04 10:49:27 +00:00
Jesper Schmitz Mouridsen
ea07ba1170 sys/arm64/iommu/iommu_pmap.c readd sys/systm.h
after d950c5898a UINT64_C and bzero were no longer defined

Approved by:	kib
Differential Revision:	https://reviews.freebsd.org/D34155
2022-02-03 20:03:29 +01:00
Mitchell Horne
4e1bc961bb arm64, riscv: handle RB_KDB
This allows entering the debugger at the earliest possible time, if
the '-d' argument is passed to the kernel.

Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D34120
2022-02-01 13:59:54 -04:00
Andrew Turner
548a2ec49b Add PT_GETREGSET
This adds the PT_GETREGSET and PT_SETREGSET ptrace types. These can be
used to access all the registers from a specified core dump note type.
The NT_PRSTATUS and NT_FPREGSET notes are initially supported. Other
machine-dependant types are expected to be added in the future.

The ptrace addr points to a struct iovec pointing at memory to hold the
registers along with its length. On success the length in the iovec is
updated to tell userspace the actual length the kernel wrote or, if the
base address is NULL, the length the kernel would have written.

Because the data field is an int the arguments are backwards when
compared to the Linux PTRACE_GETREGSET call.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19831
2022-01-27 11:40:34 +00:00
Andrew Turner
2ad1999722 Add the Armv8.3-SPE registers 2022-01-19 12:07:35 +00:00
Mark Johnston
758d98debe exec: Remove the stack gap implementation
ASLR stack randomization will reappear in a forthcoming commit.  Rather
than inserting a random gap into the stack mapping, the entire stack
mapping itself will be randomized in the same way that other mappings
are when ASLR is enabled.

No functional change intended, as the stack gap implementation is
currently disabled by default.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33704
2022-01-17 16:11:54 -05:00
Mark Johnston
706f4a81a8 exec: Introduce the PROC_PS_STRINGS() macro
Rather than fetching the ps_strings address directly from a process'
sysentvec, use this macro.  With stack address randomization the
ps_strings address is no longer fixed.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33704
2022-01-17 16:11:54 -05:00
Mark Johnston
3fc21fdd5f sysent: Add a sv_psstringssz field to struct sysentvec
The size of the ps_strings structure varies between ABIs, so this is
useful for computing the address of the ps_strings structure relative to
the top of the stack when stack address randomization is enabled.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33704
2022-01-17 11:42:07 -05:00
Brooks Davis
0910a41ef3 Revert "syscallarg_t: Add a type for system call arguments"
Missed issues in truss on at least armv7 and powerpcspe need to be
resolved before recommit.

This reverts commit 3889fb8af0.
This reverts commit 1544e0f5d1.
2022-01-12 23:29:20 +00:00
Brooks Davis
3889fb8af0 sysent: regen for syscallarg_t 2022-01-12 22:51:25 +00:00
Brooks Davis
1544e0f5d1 syscallarg_t: Add a type for system call arguments
This more clearly differentiates system call arguments from integer
registers and return values. On current architectures it has no effect,
but on architectures where pointers are not integers (CHERI) and may
not even share registers (CHERI-MIPS) it is necessiary to differentiate
between system call arguments (syscallarg_t) and integer register values
(register_t).

Obtained from:	CheriBSD

Reviewed by:	imp, kib
Differential Revision:	https://reviews.freebsd.org/D33780
2022-01-12 22:51:25 +00:00
Andrew Turner
85b7c566f1 Add arm64 pointer authentication support
Pointer authentication allows userspace to add instructions to insert
a Pointer Authentication Code (PAC) into a register based on an address
and modifier and check if the PAC is correct. If the check fails it will
either return an invalid address or fault to the kernel.

As many of these instructions are a NOP when disabled and in earlier
revisions of the architecture this can be used, for example, to sign
the return address before pushing it to the stack making Return-oriented
programming (ROP) attack more difficult on hardware that supports them.

The kernel manages five 128 bit signing keys: 2 instruction keys, 2 data
keys, and a generic key. The instructions then use one of these when
signing the registers. Instructions that use the first four store the
PAC in the register being signed, however the instructions that use the
generic key store the PAC in a separate register.

Currently all userspace threads share all the keys within a process
with a new set of userspace keys being generated when executing a new
process. This means a forked child will share its keys with its parent
until it calls an appropriate exec system call.

In the kernel we allow the use of one of the instruction keys, the ia
key. This will be used to sign return addresses in function calls.
Unlike userspace each kernel thread has its own randomly generated.

Thread0 has a static key as does the early code on secondary CPUs.
This should be safe as there is minimal user interaction with these
threads, however we could generate random keys when the Armv8.5
Random number generation instructions are present.

Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31261
2022-01-12 15:27:17 +00:00
Andrew Turner
78c347d9f3 Fix a typo in an arm64 comment
This was pointed out by markj in the review, but I missed it and forgot
to fix before pushing.

Reported by:	markj
Sponsored by:	The FreeBSD Foundation
2022-01-10 14:08:15 +00:00
Andrew Turner
77402d2834 Move instructions into the arm64 exception vectors
We have 32 instructions in each exception vector on arm64. Previously
only one was used to branch to the handler function. We can split the
start of these functions and move some of the instructions into the
vectors.

Reviewed by:	kib, markj
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33751
2022-01-10 10:51:16 +00:00
John Baldwin
7def1e10b3 bus_dma: Deduplicate locking helper functions.
- Move busdma_lock_mutex to subr_bus_dma.c.

- Move _busdma_lock_dflt to subr_bus_dma.c.  This function was named a
  couple of different things previously.  It is not a public API but
  an internal helper used in place of a NULL pointer.  The prototype
  is in <sys/bus_dma.h> as not all backends include
  <sys/bus_dma_internal.h>.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D33694
2022-01-05 13:50:40 -08:00
John Baldwin
85b4607324 Deduplicate bus_dma bounce code.
Move mostly duplicated code in various MD bus_dma backends to support
bounce pages into sys/kern/subr_busdma_bounce.c.  This file is
currently #include'd into the backends rather than compiled standalone
since it requires access to internal members of opaque bus_dma
structures such as bus_dmamap_t and bus_dma_tag_t.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D33684
2022-01-05 13:50:40 -08:00
Andrew Turner
459e0b92e8 Use the arm64 ttbr macro to define the ASID shift
Now we have a macro to descrive the arm64 TTBR registers use it in the
ASID_TO_OPERAND pmap macro.

Sponsored by:	The FreeBSD Foundation
2022-01-05 14:56:25 +00:00
Andrew Turner
0accd72674 Add the arm64 TTBR fields to armreg.h
Sponsored by:	The FreeBSD Foundation
2022-01-05 14:56:25 +00:00
Andrew Turner
ce1e5d0d5e Add a missing space in the arm64 sendsig
Sponsored by:	The FreeBSD Foundation
2022-01-05 12:38:54 +00:00
Andrew Turner
fbdca14953 Remove the sigcode base check on arm64
This was added with the shared page support. It is unused as we always
enable the shared page on FreeBSD executables on arm64.
2022-01-05 11:57:38 +00:00
Hubert Mazur
f21c469dbc Add driver for marvell, a37x0 peripheral clocks
Add driver for Marvell, Armada-37xx peripheral clock.
Register clocks for various peripheral devices in
north bridge or south bridge domain. Dump clock's
domain while verbose boot.

Reviewed by:
Obtained from:		Semihalf
Differential revision:	https://reviews.freebsd.org/D32294
2022-01-04 13:26:35 +01:00
Hubert Mazur
127e4d9524 Add driver for armada, 37x0-tbg-clock
Driver for tbg clocks. Read reference frequency from parent
and modify it depending on parameters read from register.

Reviewed by:		manu
Obtained from:		Semihalf
Differential revision:	https://reviews.freebsd.org/D32293
2022-01-04 13:26:34 +01:00
Hubert Mazur
01b9c59fd2 Add driver for marvell, armada-3700 oscillator
Driver registers new clock device. Clock frequency is set depending
on tenth bit's value obtained from syscon register. Full information
about the clock is dumped if bootverbose is enabled.

Driver was tested on EspressoBin.

Reviewed by:		manu
Obtained from:		Semihalf
Differential revision:	https://reviews.freebsd.org/D32292
2022-01-04 13:26:34 +01:00
Alan Cox
4ccd6c137f arm64: Implement final level only TLB invalidations
A feature of arm64's instruction for TLB invalidation is the ability
to determine whether cached intermediate entries, i.e., L{0,1,2}_TABLE
entries, are invalidated in addition to the final entry, e.g., an
L3_PAGE entry.

Update pmap_invalidate_{page,range}() to support both types of
invalidation, allowing the caller to determine which type of
invalidation is performed.

Update the callers to request the appropriate type of invalidation.

Eliminate redundant TLB invalidations in pmap_abort_ptp() and
pmap_remove_l3_range().

Add a comment to pmap_invalidate_all() making clear that it always
invalidates entries at all levels.

As expected, these changes result in a tiny yet measurable
performance improvement.

Reviewed by:	kib, markj
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D33705
2022-01-03 13:14:18 -06:00
Jessica Clarke
3582b9e372 arm64: Check for intrng-reported errors in gicv3_its
Currently, any errors when adding a PIC child handler are ignored,
instead just continuing on to registering that PIC as an MSI, and
ignoring any errors that occur for that too.

Reviewed by:	andrew
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D33342
2022-01-03 17:09:42 +00:00
Doug Moore
f1e7a532d1 busdma: _bus_dmamap_addseg repaired
A recent change introduced a one-off error into a test allowing
coalescing chunks into segments.  This fixes that error.

broke a check in _bus_dmamap_addseg on many architectures. This change makes it clear that it is not a particular range that is being boundary-checked, but the proposed union of the two adjacent ranges.
Reported by:	se
Reviewed by:	se
Fixes:	c606ab59e7 vm_extern: use standard address checkers everywhere
Differential Revision:	https://reviews.freebsd.org/D33715
2022-01-02 12:37:05 -06:00
Mark Johnston
f04a096049 exec: Simplify sv_copyout_strings implementations a bit
Simplify control flow around handling of the execpath length and signal
trampoline.  Cache the sysentvec pointer in a local variable.

No functional change intended.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33703
2021-12-31 12:50:15 -05:00