This silences an "unused label" warning as well as fixes the attach fail
path that wasn't releasing resources.
Submitted by: Nicholas O'Brien <nickisobrien_gmail.com>
Sponsored by: Axiado
Differential Revision: https://reviews.freebsd.org/D24004
Use __riscv_flen instead of __riscv_float_abi_soft. While the latter works for
userland (and one could argue it's more correct), it fails for the kernel. We
compile the kernel with -mabi=lp64 (eg soft float abi) to avoid floating point
instructions in the kernel. We also compile the kernel -march=rv64imafdc for
hard float kernels (eg those with options FPE), but with -march=rv64imac for
softfloat kernels (eg those with FPE). Since we do this, in the kernel (as in
userland) __riscv_flen will be defined for 'riscv64' and not for 'riscv64sf'.
This also removes the -DMACHINE_ARCH hack now that it's no longer needed.
Longer term, we should return the ABI from the sysctl hw.machine_arch like on
amd64 for i386 binaries.
Suggested by: mhorne@
Differential Revision: https://reviews.freebsd.org/D23813
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.
This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.
Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT
Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718
GENERICSF is just like GENERIC, only creates a soft-float kernel. Omit it from the
universe build for now.
Reviewed by: philip
Differential Revision: https://reviews.freebsd.org/D23812
MACHINE_ARCH sets the hw.machine_arch sysctl in the kernel. In userspace
it sets MACHINE_ARCH in bmake, which bsd.cpu.mk uses to configure the
target ABI for ports.
For riscv64sf builds (i.e. soft-float) that needs to be riscv64sf, but
the sysctl didn't reflect that. It is static.
Set the define from the riscv makefile so that we correctly reflect our
actual build (i.e. riscv64 or riscv64sf), depending on what TARGET_ARCH
we were built with.
That still doesn't satisfy userspace builds (e.g. bmake), so check if
we're building with a software-floating point toolchain there. That
check doesn't work in the kernel, because it never uses floating point.
Reviewed by: philip (previous version), mhorne
Sponsored by: Axiado
Differential Revision: https://reviews.freebsd.org/D23741
This is taken from the arm64 version, with the following simplifications:
- Our current pmap implementation uses a 3-level paging scheme
- The "mode" field has been omitted since RISC-V PTEs don't encode
typical mode attributes
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D23594
Always use the kdb_thr_ctx() for db_trace_thread() as on other
architectures. Initialize pcb_ra to be the sepc from the saved
trapframe rather than the saved ra to avoid skipping a frame.
Reviewed by: mhorne, br
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23513
While here, add an extra line of information for exceptions and
interrupts and compress the per-frame line down to one line to match
other architectures.
Reviewed by: mhorne, br
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23508
While cause codes higher than 16 are reserved, the exception code
field of the register is defined to be all bits but the upper-most
bit.
Reviewed by: mhorne
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23510
While here, remove a local variable to avoid the CSR read in non-debug
kernels.
Reviewed by: mhorne
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23511
In practice this discarded all characters entered at the DDB prompt.
Reviewed by: br
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23509
This fixes continuing from debug.kdb.enter=1 after enabling the use of
compressed instructions since the compiler can emit the two byte
c.ebreak instead of the 4 byte ebreak.
Reviewed by: br
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23507
This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.
PR: 243711
Reviewed by: jhb, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23512
The PRCI exports tlclk as a constant fixed divisor clock, defined as 1/2
of the coreclk frequency. In older FU540 device trees (such as the one
provided by SiFive), tlclk is represented as its own entity, and is
automatically registered as a fixed-divisor-clock. Unfortunately the
upstream FU540 device tree (that we have in our tree) represents tlclk
as an output of the PRCI block, and we must register it manually. At
worst, users of the old device tree will end up with an unreferenced
duplicate of tlclk.
This fixes device attachment for the SiFive UART on newer device trees,
since it references tlclk via the PRCI.
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D23406
Add two additional compat strings that can be used to identify the PRCI.
With newer device trees the PRCI has two parents, hfclk and rtcclk, so
allow the driver to attach when more than one parent is found.
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D23405
The PRCI module exports three PLLs. Currently only the coreclk/corepll
is registered, so add the logic to register the DDR (memory) and GEMGX
(ethernet) clocks as well. These clocks are unused at the moment.
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D23404
Rather than trying to blacklist which bits userland can't write to via
sigreturn() or setcontext(), only permit changes to whitelisted bits.
- Permit arbitrary writes to bits in the user-writable USTATUS
register that shadows SSTATUS.
- Ignore changes in write-only bits maintained by the CPU.
- Ignore the user-supplied value of the FS field used to track
floating point state and instead set it to a value matching the
actions taken by set_fpcontext().
Discussed with: mhorne
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23338
After r355784 the td_oncpu field is no longer synchronized by the thread
lock, so the stack capture interrupt cannot be delievered precisely.
Fix this using a loop which drops the thread lock and restarts if the
wrong thread was sampled from the stack capture interrupt handler.
Change the implementation to use a regular interrupt instead of an NMI.
Now that we drop the thread lock, there is no advantage to the latter.
Simplify the KPIs. Remove stack_save_td_running() and add a return
value to stack_save_td(). On platforms that do not support stack
capture of running threads, stack_save_td() returns EOPNOTSUPP. If the
target thread is running in user mode, stack_save_td() returns EBUSY.
Reviewed by: kib
Reported by: mjg, pho
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23355
The stack pointer is swapped with the sscratch CSR just before the
jump to cpu_exception_handler_user where the first instruction swaps
it again. The two swaps together are a no-op, but the csr swap
instructions can be expensive (e.g. on Bluespec RISC-V cores csr swap
instructions force a full pipeline stall).
Reported by: jrtc27
Reviewed by: br
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23394
cpu_switch/throw() and savectx() do not save or restore any values in
these fields which mostly held non-callee-save registers.
makectx() copied these fields from kdb_frame, but they weren't used
except for PC_REGS using pcb_sepc. Change PC_REGS to use
kdb_frame->tf_sepc directly instead.
Reviewed by: br
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23395
The SD bit is defined as the MSB of the sstatus register, meaning its
position will vary depending on the CSR's length. Previously, there were
two (unused) defines for this, for the 32 and 64-bit cases, but their
definitions were swapped.
Consolidate these into one define: SSTATUS_SD, and make the definition
dependent on the value of __riscv_xlen.
Reviewed by: br
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D23402
Previously, this check was only in sys_sigreturn() which meant that
user applications could write invalid values to the register via
setcontext() or swapcontext().
Reviewed by: mhorne
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23219
arm64 and riscv were only saving and restoring floating point
registers for sendsig() and sys_sigreturn(), but not for getcontext(),
setcontext(), and swapcontext().
While here, remove an always-false check for uap being NULL from
sys_sigreturn().
Reviewed by: br, mhorne
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23218
As part of the RISC-V ABI, the gp register is expected to initialized
with the address of __global_pointer$ as early as possible. This allows
loads and stores from .sdata to be relaxed based on the value of gp. In
locore.S we do this initialization twice, once each for va and mpva.
However, in both cases the initialization is preceded by an la
instruction, which in theory could be relaxed by the linker.
Move the initialization of gp to be slightly earlier (before la
cpu_exception_handler), and add an additional gp initialization at the
very beginning of _start, before virtual memory is set up.
Reported by: jrtc27
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D23139
as these functions should do zero-extend.
Discovered by running pci_read_cap(), and by hint from James Clarke.
Reviewed by: James Clarke <jrtc27@jrtc27.com>
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D23236
This is a follow up to r356481. In locore.S, before virtual memory is
set up, we should avoid using indirect address lookups through the GOT.
Therefore we need to convert uses of the la instruction to lla, which
always generates an auipc/addi pair of instructions. This conversion was
done for the BSP case, but not the AP case, resulting in a fault
somewhere before mpva and a failure to bring APs online.
Reported by: lwhsu
Reviewed by: lwhsu, jrtc27 (accepted in a comment)
Differential Revision: https://reviews.freebsd.org/D23138
lld on RISC-V is not yet able to handle undefined weak symbols for
non-PIC code in the code model (medany/medium) used by the RISC-V
kernel.
Both GCC and clang emit an auipc / addi pair of instructions to
generate an address relative to the current PC with a 31-bit offset.
Undefined weak symbols need to have an address of 0, but the kernel
runs with PC values much greater than 2^31, so there is no way to
construct a NULL pointer as a PC-relative value. The bfd linker
rewrites the instruction pair to use lui / addi with values of 0 to
force a NULL pointer address. (There are similar cases for 'ld'
becoming auipc / ld that bfd rewrites to lui / ld with an address of
0.)
To work around this, compile the kernel with -fPIE when using lld.
This does not make the kernel position-independent, but it does
force the compiler to indirect address lookups through GOT entries
(so auipc / ld against a GOT entry to fetch the address). This
adds extra memory indirections for global symbols, so should be
disabled once lld is finally fixed.
A few 'la' instructions in locore that depend on PC-relative
addressing to load physical addresses before paging is enabled have to
use auipc / addi and not indirect via GOT entries, so change those to
use 'lla' which always uses auipc / addi for both PIC and non-PIC.
Submitted by: jrtc27
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23064
Implement support for the UART as found on the SiFive FU540. It should
also work on, but has not been tested with, the FU310.
Reviewed by: philip
Sponsored by: Axiado
There's no point in checking for absent CPUs if we're not going to do anything
about either the present or absent case. This loop can just be removed.
Reviewed by: philip
Sponsored by: Axiado
With WITNESS enabled we see the following warning:
lock order reversal: (sleepable after non-sleepable)
1st 0xffffffd0847c7210 fu540spi0 (fu540spi0) @
/usr/home/kp/axiado/hornet-freebsd/src/sys/riscv/sifive/fu540_spi.c:297
2nd 0xffffffc00372bb30 Clock topology lock (Clock topology lock) @
/usr/home/kp/axiado/hornet-freebsd/src/sys/dev/extres/clk/clk.c:1137
stack backtrace:
#0 0xffffffc0002a579e at witness_checkorder+0xb72
#1 0xffffffc0002a5556 at witness_checkorder+0x92a
#2 0xffffffc000254c7a at _sx_slock_int+0x66
#3 0xffffffc00025537a at _sx_slock+0x8
#4 0xffffffc000123022 at clk_get_freq+0x38
#5 0xffffffc0005463e4 at __clzdi2+0x2bb8
#6 0xffffffc00014af58 at randomdev_getkey+0x76e
#7 0xffffffc0001278b0 at simplebus_add_device+0x7ee
#8 0xffffffc00027c9a8 at device_attach+0x2e6
#9 0xffffffc00027c634 at device_probe_and_attach+0x7a
#10 0xffffffc00027d76a at bus_generic_attach+0x10
#11 0xffffffc00014aab0 at randomdev_getkey+0x2c6
#12 0xffffffc00027c9a8 at device_attach+0x2e6
#13 0xffffffc00027c634 at device_probe_and_attach+0x7a
#14 0xffffffc00027d76a at bus_generic_attach+0x10
#15 0xffffffc000278bd2 at config_intrhook_oneshot+0x52
#16 0xffffffc000278b3e at config_intrhook_establish+0x146
#17 0xffffffc000278cf2 at config_intrhook_disestablish+0xfe
The clock topology lock can sleep, which means we cannot attempt to
acquire it while holding the non-sleepable mutex.
Fix that by retrieving the clock speed once, during attach and not every
time during SPI transaction setup.
Submitted by: kp
Sponsored by: Axiado
Due to clang and LLD's tendency to use a PLT for builtins, and as they
don't have full support for EABI, we sometimes have to deal with a PLT in
.ko files in a clang-built kernel.
As such, augment the in-kernel linker to support jump table processing.
As there is no particular reason to support lazy binding in kernel modules,
only implement Secure-PLT immediate binding.
As part of these changes, add elf_cpu_parse_dynamic() to the MD API of the
in-kernel linker (except on platforms that use raw object files.)
The new function will allow MD code to act on MD tags in _DYNAMIC.
Use this new function in the PowerPC MD code to ensure BSS-PLT modules using
PLT will be rejected during insertion, and to poison the runtime resolver to
ensure we get a clear panic reason if a call is made to the resolver.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D22608
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created. See the man page for
details on how to initialize a template.
Templates do not support tag filters. Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine. Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D22906
The SiFive FU540 Power Reset Clocking Interrupt block contains a PLL
that turns the input crystal (33.3MHz) into a 1-1.5GHz clock.
This clock in turn is divided by two to produce the tlclk, which is fed
into devices such as the SPI and I2C controllers.
Register a new clock device for the PRCI so that those devices can
read the correct clock through the clk framework.
Submitted by: kp
Sponsored by: Axiado
fix an assert violation introduced in r355784. Without this spinlock_exit()
may see owepreempt and switch before reducing the spinlock count. amd64
had been optimized to do a single critical enter/exit regardless of the
number of spinlocks which avoided the problem and this optimization had
not been applied elsewhere.
Reported by: emaste
Suggested by: rlibby
Discussed with: jhb, rlibby
Tested by: manu (arm64)
This is a 32-bit structure embedded in each vm_page, consisting mostly
of page queue state. The use of a structure makes it easy to store a
snapshot of a page's queue state in a stack variable and use cmpset
loops to update that state without requiring the page lock.
This change merely adds the structure and updates references to atomic
state fields. No functional change intended.
Reviewed by: alc, jeff, kib
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22650
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.
