Commit Graph

1453 Commits

Author SHA1 Message Date
Mitchell Horne
5742f2d89c arm64: adjust comments in dbg_monitor_exit()
These comments were copied from dbg_monitor_enter(), but the intended
modifications weren't made. Update them to reflect what this code
actually does.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2021-04-13 14:41:31 -03:00
Mitchell Horne
a2a8b582bd arm64: clear debug registers after execve(2)
This is both intuitive and required, as any previous breakpoint settings
may not be applicable to the new process.

Reported by:	arichardson
Reviewed by:	kib
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D29672
2021-04-13 14:41:03 -03:00
Mark Johnston
f66a1f4074 genet: Use device_t in preference to struct device *
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2021-04-12 09:31:58 -04:00
Andrew Turner
5998328e55 Clean up the style in the arm64 bus.h
MFC after:	2 weeks
Sponsored by:	Innovate UK
2021-04-08 10:27:11 +00:00
Mitchell Horne
1fd001db9c arm64: clear debug register state on fork
Following the analogous change for amd64 and i386 in 8223717ce6,
ensure that new processes start with these registers inactive.

PR:		254661
Reported by:	Michał Górny
Reviewed by:	kib, emaste
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D29630
2021-04-08 09:41:41 -03:00
Andrew Turner
d6a53211a7 Discard the arm64 VFP state before resetting it
When resetting the VFP state we need to discard any old state so we don't
try to save it on a context switch. Move this first so resetting the pcb
is safe to perform outside a critical section.

Reviewed by:	arichardson
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D29401
2021-04-08 07:51:26 +00:00
Greg V
a29bff7a52 smbios: support getting address from EFI
On some systems (e.g. Lenovo ThinkPad X240, Apple MacBookPro12,1)
the SMBIOS entry point is not found in the <0xFFFFF space.

Follow the SMBIOS spec and use the EFI Configuration Table for
locating the entry point on EFI systems.

Reviewed by:	rpokala, dab
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D29276
2021-04-07 14:46:29 -05:00
Justin Hibbits
16e549ebe2 Merge the QorIQ GPIO drivers between arm and powerpc
Summary:
They're nearly identical, so don't use two copies.  Merge the newer
driver into the older one, and move it to a common location.

Add the Semihalf and associated copyrights in addition to mine, since
it's a non-trivial amount of code merged.

Reviewed By: mw
Differential Revision: https://reviews.freebsd.org/D29520
2021-04-05 10:35:15 -05:00
Mitchell Horne
7446b0888d gdb: report specific stop reason for watchpoints
The remote protocol allows for implementations to report more specific
reasons for the break in execution back to the client [1]. This is
entirely optional, so it is only implemented for amd64, arm64, and i386
at the moment.

[1] https://sourceware.org/gdb/current/onlinedocs/gdb/Stop-Reply-Packets.html

Reviewed by:	jhb
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
NetApp PR:	51
Differential Revision:	https://reviews.freebsd.org/D29174
2021-03-30 11:36:41 -03:00
Mitchell Horne
4beb385813 gdb: allow setting/removing hardware watchpoints
Handle the 'z' and 'Z' remote packets for manipulating hardware
watchpoints.

This could be expanded quite easily to support hardware or software
breakpoints as well.

https://sourceware.org/gdb/onlinedocs/gdb/Packets.html

Reviewed by:	cem, markj
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
NetApp PR:	51
Differential Revision:	https://reviews.freebsd.org/D29173
2021-03-30 11:36:41 -03:00
Mitchell Horne
9d81dd5404 ddb: replace watchpoint set/clear functions
Use the new kdb variants. Print more specific error messages.

Reviewed by:	jhb, markj
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D29156
2021-03-29 12:05:44 -03:00
Mitchell Horne
3ef68bc62c arm64: implement kdb watchpoint functions
Add wrappers around the debug_monitor interface, to be consumed by MI
kernel debugger code.  Update dbg_setup_watchpoint() and
dbg_remove_watchpoint() to return specific error codes, not just -1.

Reviewed by:	jhb, kib, markj
MFC after:	3 weeks
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D29155
2021-03-29 12:05:43 -03:00
Mitchell Horne
720dc6bcb5 Consolidate machine/endian.h definitions
This change serves two purposes.

First, we take advantage of the compiler provided endian definitions to
eliminate some long-standing duplication between the different versions
of this header. __BYTE_ORDER__ has been defined since GCC 4.6, so there
is no need to rely on platform defaults or e.g. __MIPSEB__ to determine
endianness. A new common sub-header is added, but there should be no
changes to the visibility of these definitions.

Second, this eliminates the hand-rolled __bswapNN() routines, again in
favor of the compiler builtins. This was done already for x86 in
e6ff6154d2. The benefit here is that we no longer have to maintain our
own implementations on each arch, and can instead rely on the compiler
to emit appropriate instructions or libcalls, as available. This should
result in equivalent or better code generation. Notably 32-bit arm will
start using the `rev` instruction for these routines, which is available
on armv6+.

PR:		236920
Reviewed by:	arichardson, imp
Tested by:	bdragon (BE powerpc)
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D29012
2021-03-26 19:00:22 -03:00
Andrew Turner
df7549775c Replace the arm64 initial_fpcr with a macro
This value was never changed from its default value. Replace it with a
macro.

Sponsored by:	Innovate UK
2021-03-23 17:12:43 +00:00
Emmanuel Vadot
6bcba8dac9 arm64: Only check for freebsd,dts-version if we are booted in FDT mode.
Reported by:	  andrew
2021-03-23 16:37:25 +01:00
Emmanuel Vadot
63f344024a arm64: Check dtb version against the one we're expecting to find
Do for arm64 what was done for armv7 in e63faa9ba8
2021-03-23 15:24:53 +01:00
Mark Johnston
3ead60236f Generalize bus_space(9) and atomic(9) sanitizer interceptors
Make it easy to define interceptors for new sanitizer runtimes, rather
than assuming KCSAN.  Lay a bit of groundwork for KASAN and KMSAN.

When a sanitizer is compiled in, atomic(9) and bus_space(9) definitions
in atomic_san.h are used by default instead of the inline
implementations in the platform's atomic.h.  These definitions are
implemented in the sanitizer runtime, which includes
machine/{atomic,bus}.h with SAN_RUNTIME defined to pull in the actual
implementations.

No functional change intended.

MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2021-03-22 22:21:53 -04:00
Jason A. Harmening
d22883d715 Remove PCPU_INC
e4b8deb222 removed the last in-tree uses of PCPU_INC().  Its
potential benefit is also practically nonexistent.  Non-x86
platforms already implement it as PCPU_ADD(..., 1), and according
to [0] there are no recent x86 processors for which the 'inc'
instruction provides a performance benefit over the equivalent
memory-operand form of the 'add' instruction.  The only remaining
benefit of 'inc' is smaller instruction size, which in this case
is inconsequential given the limited number of per-CPU data consumers.

[0]: https://www.agner.org/optimize/instruction_tables.pdf

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D29308
2021-03-20 19:23:59 -07:00
John Baldwin
3b57ddb029 Rename linux_set_upcall_kse() to linux_set_upcall().
This matches the rename of cpu_set_upcall_kse() in
5c2cf81845.

Reviewed by:	kib, emaste
MFC after:	1 week
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D29295
2021-03-18 12:14:34 -07:00
Mike Karels
2bdcf62377 genet: Fix problem with forwarding some TCP/IPv6 packets
TCP/IPv6 packets to be forwarded can be laid out with only the Ethernet
header in the first mbuf, and these packets are lost.  There was a
previous hack to pullup ICMPv6 packets with such a layout for the
same reason.  Generalize, and pullup any IPv6 packets with only the
Ethernet header in the first mbuf.  Possibly this should also include
IPv4, but that situation has not been observed to fail.

PR:		254060
Reported by:	denis at h3q.com
MFC after:	3 days
2021-03-17 19:25:43 -05:00
Ed Maste
5be27cbf16 arm64: implement COMPAT_FREEBSD32 fill/set dbregs/fpregs
On FreeBSD/arm fill_fpregs, fill_dbregs are stubs that zero the reg
struct and return success. set_fpregs and set_dbregs do nothing and
return success.

Provide the same implementation for arm64 COMPAT_FREEBSD32.

Reviewed by:	andrew
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29314
2021-03-17 16:46:50 -04:00
John Baldwin
0723b40915 aarch64: Clear TLS registers during exec().
These are not stored in the trapframe so must be cleared explicitly.

This is similar to one of the MIPS changes in 822d2d6ac9.

Reviewed by:	andrew
Obtained from:	CheriBSD
MFC after:	1 week
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D28711
2021-03-17 13:19:04 -07:00
Peter Jeremy
07564e1762 arm64: Add support for the RK805/RK808 RTC
Implement a driver for the RTC embedded in the RK805/RK808 power
management system used for RK3328 and RK3399 SoCs.

Based on experiments on my RK808, setting the time doesn't alter the
internal/inaccessible sub-second counter, therefore there's no point
in calling clock_schedule().

Based on an earlier revision by andrew.

Reviewed by:	manu
Differential Revision:	https://reviews.freebsd.org/D22692
Sponsored by:	Google
MFC after:	1 week
2021-03-13 09:06:04 +11:00
Olivier Houchard
c328f64d81 arm64: Fix COMPAT_FREEBSD32.
The ENTRY() macro was modified by commit
28d945204e to add an optional NOP instruction
at the beginning of the function. It is of course an arm64 instruction, so
unsuitable for the 32bits sigcode. So just use EENTRY() instead for
aarch32_sigcode. This should fix receiving signals when running 32bits
binaries on FreeBSD/arm64.

MFC After: 1 week
2021-03-10 19:06:42 +01:00
Alex Richardson
953a7d7c61 Arch64: Clear VFP state on execve()
I noticed that many of the math-related tests were failing on AArch64.
After a lot of debugging, I noticed that the floating point exception flags
were not being reset when starting a new process. This change resets the
VFP inside exec_setregs() to ensure no VFP register state is leaked from
parent processes to children.

This commit also moves the clearing of fpcr that was added in 65618fdda0
from fork() to execve() since that makes more sense: fork() can retain
current register values, but execve() should result in a well-defined
clean state.

Reviewed By:	andrew
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D29060
2021-03-10 12:44:42 +00:00
Mark Johnston
435c7cfb24 Rename _cscan_atomic.h and _cscan_bus.h to atomic_san.h and bus_san.h
Other kernel sanitizers (KMSAN, KASAN) require interceptors as well, so
put these in a more generic place as a step towards importing the other
sanitizers.

No functional change intended.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D29103
2021-03-08 12:39:06 -05:00
Alex Richardson
0072e5e0f3 sys/arm64/arm64/vfp.c: Fix -Wunused and -Wpointer-sign warnings
These are off by default but were flagged by my IDE while adding some
debugging printfs for D29060.
2021-03-04 18:25:44 +00:00
Andrew Turner
23553d6b94 Fix creating the early arm64 level 2 blocks
In 48ba9b2669 we switched from creating level 1 blocks to smaller
level 2 blocks when creating the early arm64 page tables. On issue
was that they had a different meaning for register x7. The former used
it to hold page table attributes, while the latter held just the memory
type. This caused these attributes to be incorrectly shifted.

Fix this by changing the meaning of x7 to hold the block attributes
and fix the only caller that used the old meaning.

Most hardware seems to have handled the bits being off however qemu
failed to boot as reserved bits that should be zero were being set and
qemu fails to clear these when translating from a virtual address to a
physical address.

Sponsored by:	Innovate UK
2021-03-04 14:39:12 +00:00
Andrew Turner
28d945204e Handle functions that use a nop in the arm64 fbt
To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.

Reviewed by:	gnn
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D28132
2021-03-03 14:18:03 +00:00
Andrew Turner
48ba9b2669 Use L2 blocks when in the identity map
This reduces the memory mapped to be closer to the minimal memory
needed to enable the MMU.

Reviewed by:	mmel
Sponsored by:	Innovate UK
Differential Revision:://reviews.freebsd.org/D27765
2021-03-03 14:18:03 +00:00
Alex Richardson
0e4ff0acbe AArch64: Don't set flush-subnormals-to-zero flag on startup
This flag has been set on startup since 65618fdda0.
However, This causes some of the math-related tests to fail as they report
zero instead of a tiny number. This fixes at least
/usr/tests/lib/msun/ldexp_test and possibly others.
Additionally, setting this flag prevents printf() from printing subnormal
numbers in decimal form.
See also https://www.openwall.com/lists/musl/2021/02/26/1

PR:		253847
Reviewed By:	mmel
Differential Revision: https://reviews.freebsd.org/D28938
2021-03-01 14:27:30 +00:00
Mitchell Horne
e152c88273 arm64: add definition for IS_SSTEP_TRAP()
arm64 has a distinct exception code for single-step, so we can use this
to detect when an unexpected SS trap is encountered, or when an expected
one is not. See db_stop_at_pc().

Reviewed by:	markj, jhb
MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28942
2021-03-01 10:04:23 -04:00
Mitchell Horne
bd0b7cbf5a arm64: update kdb_thrctx->pcb_lr with BKPT_SKIP
This value should be kept in sync with updates to kdb_frame->tf_elr,
since it is queried by PC_REGS() in several places.

Reviewed by:	markj, jhb
MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28943
2021-03-01 10:04:22 -04:00
Mitchell Horne
874635e381 arm64: fix hardware single-stepping from EL1
The main issue is that debug exceptions must to be disabled for the
entire duration that SS bit in MDSCR_EL1 is set. Otherwise, a
single-step exception will be generated immediately. This can occur
before returning from the debugger (when MDSCR is written to) or before
re-entering it after the single-step (when debug exceptions are unmasked
in the exception handler).

Solve this by delaying the unmask to C code for EL1, and avoid unmasking
at all while handling debug exceptions, thus avoiding any recursive
debug traps.

Reviewed by:	markj, jhb
MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28944
2021-03-01 10:04:22 -04:00
Mark Johnston
17d0f830dd arm64: Include NUMA locality info in the CPU topology
The scheduler uses this topology to try and preserve locality when
migrating threads between CPUs and when performing work stealing.
Ensure that on NUMA systems it will at least take the NUMA topology into
account.

Reviewed by:	mmel
Submitted by:	Klara, Inc.
Sponsored by:	Ampere Computing
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D28579
2021-02-18 10:51:38 -05:00
John Baldwin
67932460c7 Add a VA_IS_CLEANMAP() macro.
This macro returns true if a provided virtual address is contained
in the kernel's clean submap.

In CHERI kernels, the buffer cache and transient I/O map are allocated
as separate regions.  Abstracting this check reduces the diff relative
to FreeBSD.  It is perhaps slightly more readable as well.

Reviewed by:	kib
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D28710
2021-02-17 16:32:11 -08:00
D Scott Phillips
3046eb03cc gicv3_its: Don't restrict target CPUs based on SRAT
ACPI Sec 5.2.16.5 (SRAT, GIC Interrupt Translation Service (ITS)
Affinity Structure) says:

> The GIC ITS Affinity Structure provides the association between
> a GIC ITS and a proximity domain. This enables the OSPM to
> discover the memory that is closest to the ITS, and use that in
> allocating its management tables and command queue.

Previously the ITS driver was using the proximity domain to
restrict which CPUs can be targeted by an LPI. We keep that logic
just for the original dual socket ThunderX which cannot forward
LPIs between sockets.

We also use the SRAT entry for its intended purpose of attempting
to allocate ITS table structures near the ITS.

Reviewed by:	andrew
Sponsored by:	Ampere Computing LLC
Differential Revision:	https://reviews.freebsd.org/D28340
2021-02-17 13:54:51 -08:00
D Scott Phillips
c720435062 gicv3_its: Leave LPI interrupts enabled during handling
This follows the behavior on x86 where edge triggered interrupts are
not disabled when executing the handler. Because the ITS is a shared
resource, contention for the command queue lock can be substantial.

Suggested by:	gallatin
Reviewed by:	andrew
Tested by:	gallatin
Sponsored by:	Ampere Computing LLC
Differential Revision:	https://reviews.freebsd.org/D28709
2021-02-17 14:08:38 -05:00
Mitchell Horne
39df2b5682 arm64: use macros to access special register values 2021-02-17 12:09:45 -04:00
Mitchell Horne
f2583be110 arm64: extend struct db_reg to include watchpoint registers
The motivation is to provide access to these registers from userspace
via ptrace(2) requests PT_GETDBREGS and PT_SETDBREGS.

This change breaks the ABI of these particular requests, but is
justified by the fact that the intended consumers (debuggers) have not
been taught to use them yet. Making this change now enables active
upstream work on lldb to begin using this interface, and take advantage
of the hardware debugging registers available on the platform.

PR:		252860
Reported by:	Michał Górny (mgorny@gentoo.org)
Reviewed by:	andrew, markj (earlier version)
Tested by:	Michał Górny (mgorny@gentoo.org)
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28415
2021-02-17 12:05:00 -04:00
Mitchell Horne
bd012c7159 arm64: handle watchpoint exceptions from EL0
This is a prerequisite to allowing the use of hardware watchpoints for
userspace debuggers.

This is also a slight departure from the x86 behaviour, since `si_addr`
returns the data address that triggered the watchpoint, not the
address of the instruction that was executed. Otherwise, there is no
straightforward way for the application to determine which watchpoint
was triggered. Make a note of this in the siginfo(3) man page.

Reviewed by:	jhb, markj (earlier version)
Tested by:	Michał Górny (mgorny@gentoo.org)
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28561
2021-02-17 12:05:00 -04:00
Mitchell Horne
de2b942280 arm64: validate breakpoint registers
In particular, we want to disallow setting breakpoints on kernel
addresses from userspace. The control register fields are validated or
ignored as appropriate.

Reviewed by:	markj
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28560
2021-02-17 12:05:00 -04:00
Mark Johnston
0fc8a79672 linux: Unmap the VDSO page when unloading
linux_shared_page_init() creates an object and grabs and maps a single
page to back the VDSO.  When destroying the VDSO object, we failed to
destroy the mapping and free KVA.  Fix this.

Reviewed by:	kib
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28696
2021-02-16 09:40:02 -05:00
Emmanuel Vadot
3b721b4495 arm64: Add a SOC_BRCM_NS2 option
Only compile files needed for this platform if the option is enabled in the
kernel config file.
Add the option to GENERIC.

MFC after:	3 days
2021-02-10 21:15:50 +01:00
Edward Tomasz Napierala
fb8c2f743a arm64: optimize set_syscall_retval()
Microoptimize set_syscall_retval() for arm64 by predicting
the return value to be zero.  This is similar to what has
been done for other architectures

Reviewed By:	emaste, mhorne
Differential Revision: https://reviews.freebsd.org/D26991
2021-02-08 10:46:47 +00:00
Andrew Turner
13ec5a6da0 Add support for arm64 nGnRE device memory
On arm64 we can select how strongly we order device memory. Currently
we use the strongest type of non-Gathering, non-Reordering, no Early
write acknowledgement. This is equivalent to VM_MEMATTR_SO in the 32-bit
arm code.

Create a new memory type to remove the no Early write acknowledgement
option to create a memory attribute that is equivalent to the arm
VM_MEMATTR_DEVICE.

Keep the the old nGnRnE memory as what we provide for VM_MEMATTR_DEVICE
until we can test nGnRE on more hardware. A method for dynamically
switching back may be needed as at least one vendor is known to have
broken nGnRE memory.

Sponsored by:	Innovate UK
2021-02-05 12:25:56 +00:00
Ed Maste
375d797b81 Enable pvscsi and vmx in arm64 GENERIC
FreeBSD pvscsi and vmx work with VMware ESXi Arm "Fling"; provide these
in GENERIC for a convenient out-of-the-box experience.

PR:		253202
Reported by:	Vincent Milum Jr
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2021-02-04 19:21:38 -05:00
Michal Meloun
65618fdda0 arm64: Initialize VFP control register.
The RW fields in this register reset to architecturally unknown values,
so initialize these to the proper rounding and denormal mode.
MFC after:	1 week
2021-02-03 16:27:30 +01:00
Jessica Clarke
05985a7f80 arm64: Improve DDB backtrace support
The existing implementation relies on each trap handler saving a normal
stack frame record, which is a waste of time and space when we're
already saving a trapframe to the stack. It's also wrong as it currently
saves LR not ELR.

Instead of patching it up, rewrite it based on the RISC-V implementation
with inspiration from the amd64 implementation for how to handle
vectored traps to provide an improved implementation. This includes
compressing the information down to one line like other architectures
rather than the highly-verbose old form that repeats itself by printing
LR and FP in one frame only to print them as PC and SP in the next. It
also includes printing out actually useful information about the traps
that occurred, though FAR is not saved in the trapframe so we cannot
print it (in general it can be clobbered between when the trap happened
and now), only ESR.

The AAPCS also allows the stack frame record to be located anywhere in
the frame, not just the top, so the caller's SP is not at a fixed offset
from the callee's FP like on almost all other architectures in
existence. This means there is no way to derive the caller's SP in the
unwinder, and so we have to drop that bit of (unused) state everywhere.

Reviewed by:	jhb, markj
Differential Revision:	https://reviews.freebsd.org/D28026
2021-02-01 14:15:57 +00:00
Allan Jude
f9322ff6ef Enable KERN_TLS by default in ARM64 GENERIC
Sponsored-by:	Ampere Computing
Submitted-by:	Klara, Inc.
2021-01-28 20:48:37 +00:00