Commit Graph

447 Commits

Author SHA1 Message Date
John Baldwin
1dce7d9e7e Skip the vm.pmap.kernel_maps sysctl by default.
This sysctl node can generate very verbose output, so don't trigger it
for sysctl -a or sysctl vm.pmap.

Reviewed by:	markj, kib
Differential Revision:	https://reviews.freebsd.org/D27504
2020-12-18 20:41:23 +00:00
Mitchell Horne
25de8fb6ca riscv: report additional known SBI implementations
These implementation IDs are defined in the SBI spec, so we should print
their name if detected.

Submitted by:	Danjel Qyteza <danq1222@gmail.com>
Reviewed by:	jhb, kp
Differential Revision:	https://reviews.freebsd.org/D27660
2020-12-18 20:10:30 +00:00
Alexander V. Chernikov
d5fe384b4d Enable ROUTE_MPATH support in GENERIC kernels.
Ability to load-balance traffic over multiple path is a must-have thing for routers.
It may be used by the servers to balance outgoing traffic over multiple default gateways.

The previous implementation, RADIX_MPATH stayed in the shadow for too long.
It was not well maintained, which lead us to a vicious circle - people were using
 non-contiguous mask or firewalls to achieve similar goals. As a result, some routing
 daemons implementation still don't have multipath support enabled for FreeBSD.

Turning on ROUTE_MPATH by default would fix it. It will allow to reduce networking
 feature gap to other operating systems. Linux and OpenBSD enabled similar support
 at least 5 years ago.

ROUTE_MPATH does not consume memory unless actually used. It enables around ~1k LOC.

It does not bring any behaviour changes for userland.
Additionally, feature is (temporarily) turned off by the net.route.multipath sysctl
 defaulting to 0.

Differential Revision:	https://reviews.freebsd.org/D27428
2020-12-14 22:23:08 +00:00
Mitchell Horne
f7cc0eae7e riscv: small counter(9) improvements
Prefer atomics to critical section. This reduces the cost of the
increment operation and removes the possibility of it being interrupted
by counter_u64_zero().

Use CPU_FOREACH() macro to skip absent CPUs.

Replace hand-rolled address calculation with zpcpu_get().

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D27536
2020-12-11 20:01:45 +00:00
Mitchell Horne
5a28499f2f riscv: handle debug.debugger_on_trap for fatal page faults
Allows recovery or diagnosis of a fatal page fault before panicking the
system.

Reviewed by:	jhb, kp
Differential Revision:	https://reviews.freebsd.org/D27534
2020-12-10 22:20:20 +00:00
John Baldwin
9b9e7f4c51 Stack unwinding robustness fixes for RISC-V.
- Push the kstack_contains check down into unwind_frame() so that it
  is honored by DDB and DTrace.

- Check that the trapframe for an exception frame is contained in the
  traced thread's kernel stack for DDB traces.

Reviewed by:	markj
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D27357
2020-12-08 17:57:18 +00:00
John Baldwin
5941edfcdc Add a kstack_contains() helper function.
This is useful for stack unwinders which need to avoid out-of-bounds
reads of a kernel stack which can trigger kernel faults.

Reviewed by:	kib, markj
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D27356
2020-12-01 17:04:46 +00:00
Alex Richardson
acf8792067 Add .cfi_{start,end}proc for RISC-V assembly functions
This allows GDB to print more useful backtraces when setting a breakpoint
on an assembly function.

Reviewed By:	jhb
Differential Revision: https://reviews.freebsd.org/D27177
2020-11-26 17:37:22 +00:00
Mitchell Horne
08241f9192 riscv: always initialize the static kernel environment
Ensure we initialize the static environment when not booting via
loader(8), and provide a static buffer if this is the case. This fixes
two issues.

First, performing the initialization ensures that kenv variables set in
the kernel's config file are honored. Previously, any new or overridden
values were ignored.

Second, providing the static buffer allows variables to be set in the
device tree's bootargs property of the chosen node. This can be set by
u-boot or by QEMU's '-append' flag. Attempting to this prior to this
change resulted in an early panic, since the static environment had no
buffer backing it.

Submitted by:	syrinx (earlier version)
Reviewed by:	kp
Differential Revision:	https://reviews.freebsd.org/D25034
2020-11-20 15:21:10 +00:00
Mitchell Horne
caaddb88e8 riscv: set kernel_pmap hart mask more precisely
In pmap_bootstrap(), we fill kernel_pmap->pm_active since it is
invariably active on all harts. However, this marks it as active even
for harts that don't exist in the system, which can cause issue when the
mask is passed to the SBI firmware via sbi_remote_sfence_vma().
Specifically, the SBI spec allows SBI_ERR_INVALID_PARAM to be returned
when an invalid hart is set in the mask.

The latest version of OpenSBI does not have this issue, but v0.6 does,
and this is triggering a recently added KASSERT in CI. Switch to only
setting bits in pm_active for harts that enter the system.

Reported by:	Jenkins
Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D27080
2020-11-05 00:52:52 +00:00
Alan Cox
9b4e77cb97 Tidy up the #includes. Recent changes, such as the introduction of
VM_ALLOC_WAITOK and vm_page_unwire_noq(), have eliminated the need for
many of the #includes.

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D27052
2020-11-02 19:20:06 +00:00
Edward Tomasz Napierala
b1497fb649 Optimize set_syscall_retval for riscv by predicting the return
value to be zero.

Reviewed by:	mhorne, kp
MFC after:	2 weeks
Sponsored by:	EPSRC
Differential Revision:	https://reviews.freebsd.org/D26990
2020-10-29 15:36:20 +00:00
Kristof Provost
eb81dfb3af riscv: Minor cleanup in startup code
- remove setting of register value which is not used until the next value is
   set
 - Use the L2_SHIFT constant when setting up L2 superpages

Submitted by:	Antonin Houska <ah AT melesmeles DOT cz>
2020-10-27 12:44:49 +00:00
Mitchell Horne
89f3492919 riscv: make use of SBI legacy replacement extensions
Version 0.2 of the SBI specification [1] marked the existing SBI
functions as "legacy" in order to move to a newer calling convention. It
also introduced a set of replacement extensions for some of the legacy
functionality. In particular, the TIME, IPI, and RFENCE extensions
implement and extend the semantics of their legacy counterparts, while
conforming to the newer version of the spec.

Update our SBI code to use the new replacement extensions when
available, and fall back to the legacy ones. These will eventually be
dropped, when support for version 0.2 is ubiquitous.

[1] https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc

Submitted by:	Danjel Q. <danq1222@gmail.com>
Reviewed by:	kp
Differential Revision:	https://reviews.freebsd.org/D26953
2020-10-26 19:13:22 +00:00
Mitchell Horne
6b35ff5fcb riscv: remove sbi_clear_ipi()
S-mode software has write access to the SIP.SSIP bit, so instead of
making a second round-trip through the SBI we can clear it ourselves.
The SBI spec has deprecated this function for this exactly this reason.

Submitted by:	Danjel Q. <danq1222@gmail.com
Reviewed by:	kp
Differential Revision:	https://reviews.freebsd.org/D26952
2020-10-26 19:06:30 +00:00
Mitchell Horne
cb8e067818 riscv: improve exception code naming
The existing names were inherited from arm64, but we should prefer
RISC-V terminology. Change the prefix to SCAUSE, and further change the
names to better match the RISC-V spec and be more consistent with one
another. Also, remove two codes that are not defined for S-mode (machine
and hypervisor ecall).

While here, apply style(9) to some condition checks.

Reviewed by:	kp
Discussed with: jrtc27
Differential Revision:	https://reviews.freebsd.org/D26918
2020-10-24 20:57:13 +00:00
Mitchell Horne
02a37049b4 riscv: zero reserved PTE bits for L2 PTEs
As was done for L3 PTEs in r362853, mask out the reserved bits when
extracting the physical address from an L2 PTE. Future versions of the
spec or custom implementations may make use of these reserved bits, in
which case the resulting physical address could be incorrect.

Submitted by:	Nathaniel Filardo <nwf20@cl.cam.ac.uk>
Reviewed by:	kp, mhorne
Differential Revision:	https://reviews.freebsd.org/D26607
2020-10-17 17:31:06 +00:00
Mitchell Horne
ce4900bc8a Simplify preload_dump() condition
Hiding this feature behind RB_VERBOSE is gratuitous. The tunable is enough
to limit its use to only those who explicitly request it.

Suggested by:	kevans
2020-10-15 20:21:15 +00:00
Konstantin Belousov
6f3b523c9a Avoid dump_avail[] redefinition.
Move dump_avail[] extern declaration and inlines into a new header
vm/vm_dumpset.h.  This fixes default gcc build for mips.

Reviewed by:	alc, scottph
Tested by:	kevans (previous version)
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D26741
2020-10-14 22:51:40 +00:00
Conrad Meyer
f8e8a06d23 random(4) FenestrasX: Push root seed version to arc4random(3)
Push the root seed version to userspace through the VDSO page, if
the RANDOM_FENESTRASX algorithm is enabled.  Otherwise, there is no
functional change.  The mechanism can be disabled with
debug.fxrng_vdso_enable=0.

arc4random(3) obtains a pointer to the root seed version published by
the kernel in the shared page at allocation time.  Like arc4random(9),
it maintains its own per-process copy of the seed version corresponding
to the root seed version at the time it last rekeyed.  On read requests,
the process seed version is compared with the version published in the
shared page; if they do not match, arc4random(3) reseeds from the
kernel before providing generated output.

This change does not implement the FenestrasX concept of PCPU userspace
generators seeded from a per-process base generator.  That change is
left for future discussion/work.

Reviewed by:	kib (previous version)
Approved by:	csprng (me -- only touching FXRNG here)
Differential Revision:	https://reviews.freebsd.org/D22839
2020-10-10 21:52:00 +00:00
Mitchell Horne
eff4c46e28 RISC-V LINT kernel config
Create the RISC-V NOTES and LINT files. As of r366559, LINT configs are
no longer generated but checked in to the tree.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D26502
2020-10-09 14:45:41 +00:00
Mitchell Horne
22e6a67086 Add a routine to dump boot metadata
The boot metadata (also referred to as modinfo, or preload metadata)
provides information about the size and location of the kernel,
pre-loaded modules, and other metadata (e.g. the EFI framebuffer) to be
consumed during by the kernel during early boot. It is encoded as a
series of type-length-value entries and is usually constructed by
loader(8) and passed to the kernel. It is also faked on some
architectures when booted by other means.

Although much of the module information is available via kldstat(8),
there is no easy way to debug the metadata in its entirety. Add some
routines to parse this data and allow it to be printed to the console
during early boot or output via a sysctl.

Since the output can be lengthly, printing to the console is gated
behind the debug.dump_modinfo_at_boot kenv variable as well as the
BOOTVERBOSE flag. The sysctl to print the metadata is named
debug.dump_modinfo.

Reviewed by:	tsoome
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D26687
2020-10-08 18:02:05 +00:00
Edward Tomasz Napierala
5319fa1b3e Remove yet another useless assignment, adding a KASSERT just in case.
Reviewed by:	kp
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26698
2020-10-08 11:04:32 +00:00
Mitchell Horne
8481aab1ac Print symbol index for unsupported relocation types
It is unlikely, but possible, that an unrecognized or unsupported
relocation type is encountered while trying to load a kernel module. If
this occurs we should offer the symbol index as a hint to the user.

While here, fix some small style issues.

Reviewed by:	markj, kib (amd64 part, in D26701)
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
2020-10-07 18:48:10 +00:00
Edward Tomasz Napierala
29c4e4b1af Don't use critical section when calling intr_irq_handler() - that function
enters critical section by itself anyway.

Reviewed by:	kp
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26694
2020-10-07 12:11:11 +00:00
Jessica Clarke
2152743f11 riscv: Remove outdated condition in page_fault_handler
Since r366355 and r366284 we panic on access faults rather than treating
them like page faults so this condition is never true.

Reviewed by:	jhb (mentor), markj, mhorne
Approved by:	jhb (mentor), markj, mhorne
Differential Revision:	https://reviews.freebsd.org/D26686
2020-10-06 13:03:31 +00:00
Jessica Clarke
105708ca1c riscv: Handle supervisor instruction page faults
We should never take instruction page faults when in the kernel, but by
using the standard page fault code we should get a more-informative
message about faulting on a NOFAULT page rather than branching to the
default case here and printing an "Unknown kernel exception ..."
message.

Reviewed by:	jhb (mentor), markj
Approved by:	jhb (mentor), markj
Differential Revision:	https://reviews.freebsd.org/D26685
2020-10-06 13:02:20 +00:00
Jessica Clarke
da8944d96d riscv: De-Arm a few names
These names were inherited from the arm64 port and should be changed to
the RISC-V terminology.

Reviewed by:	jhb (mentor), kp, markj
Approved by:	jhb (mentor), kp, markj
Differential Revision:	https://reviews.freebsd.org/D26671
2020-10-06 12:56:29 +00:00
Edward Tomasz Napierala
f157761902 Drop useless assignment, and add a KASSERT to make sure it really was useless.
Reviewed by:	nick, jhb
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26649
2020-10-05 18:41:35 +00:00
Edward Tomasz Napierala
f726515758 Optimize riscv's cpu_fetch_syscall_args(), making it possible
for the compiler to inline the memcpy.

Reviewed by:	arichardson, mhorne
MFC after:	2 weeks
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26528
2020-10-03 13:01:07 +00:00
Kristof Provost
75f022774f riscv: handle access faults in user mode
Access faults in user mode are treated like TLB misses, which leads to an
endless loop of faults. It's less serious than the same fault in kernel mode,
because we can just terminate the process, but that's not ideal.

Treat user mode access faults as a bus error.

Suggested by:	jrtc27
Reviewed by:	br, jhb
Sponsored by:	Axiado
Differential Revision:	https://reviews.freebsd.org/D26621
2020-10-02 07:30:11 +00:00
Kristof Provost
57712c0b76 riscv: Add memmmap so we can mmap /dev/mem
Reviewed by:	mhorne
Sponsored by:	Axiado
Differential Revision:	https://reviews.freebsd.org/D26622
2020-10-01 15:04:55 +00:00
Kristof Provost
0d3aa0fb64 riscv: Panic on PMP errors
Load/store/fetch access exceptions always indicate a violation of a PMP
rule. We can't treat those as page faults, because updating the page
table and trying again will only result in exactly the same access
exception recurring. This leaves us in an endless exception loop.

We cannot recover from these exceptions, so panic instead.

Reviewed by:	jhb
Sponsored by:	Axiado
Differential Revision:	https://reviews.freebsd.org/D26544
2020-09-30 08:23:43 +00:00
Jessica Clarke
7de649170f riscv: Define __PCI_REROUTE_INTERRUPT
Every other architecture defines this and this is required for
interrupts to work when using QEMU's PCI VirtIO devices (which all
report an interrupt line of 0) for two reasons.

Firstly, interrupt line 0 is wrong; they use one of 0x20-0x23 with the
lines being cycled across devices like normal. Moreover, RISC-V uses
INTRNG, whose IRQs are virtual as indices into its irq_map, so even if
we have the right interrupt line we still need to try and route the
interrupt in order to ultimately call into intr_map_irq and get back a
unique index into the map for the given line, otherwise we will use
whatever happens to be in irq_map[line] (which for QEMU where the line
is initialised to 0 results in using the first allocated interrupt,
namely the RTC on IRQ 11 at time of commit).

Note that pci_assign_interrupt will still do the wrong thing for INTRNG
when using a tunable, as it will bypass INTRNG entirely and use the
tunable's value as the index into irq_map, when it should instead
(indirectly) call intr_map_irq to allocate a new entry for the given
IRQ and treat the tunable as stating the physical line in use, which is
what one would expect. This, however, is a problem shared by all INTRNG
architectures, and not exclusive to RISC-V.

Reviewed by:	kib
Approved by:	kib
Differential Revision:	https://reviews.freebsd.org/D26564
2020-09-30 02:21:38 +00:00
Edward Tomasz Napierala
1e2521ffae Get rid of sa->narg. It serves no purpose; use sa->callp->sy_narg instead.
Reviewed by:	kib
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26458
2020-09-27 18:47:06 +00:00
Mark Johnston
78257765f2 Add a vmparam.h constant indicating pmap support for large pages.
Enable SHM_LARGEPAGE support on arm64.

Reviewed by:	alc, kib
Sponsored by:	Juniper Networks, Inc., Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D26467
2020-09-23 19:34:21 +00:00
Nick O'Brien
e1c8f8f87d riscv: Trap cleanup - use nitems()
No functional changes, just cleanup.

Reviewed by:	kp
Approved by:	kp (mentor)
Sponsored by:	Axiado
2020-09-23 18:54:14 +00:00
Mitchell Horne
3994f5bc18 RISC-V: build SiFive drivers and DTB in GENERIC
In the spirit of the GENERIC config, we should include the drivers required to
run on most supported platforms.

Reviewed by:	kp
Differential Revision:	https://reviews.freebsd.org/D26501
2020-09-22 13:00:02 +00:00
D Scott Phillips
00e6614750 Sparsify the vm_page_dump bitmap
On Ampere Altra systems, the sparse population of RAM within the
physical address space causes the vm_page_dump bitmap to be much
larger than necessary, increasing the size from ~8 Mib to > 2 Gib
(and overflowing `int` for the size).

Changing the page dump bitmap also changes the minidump file
format, so changes are also necessary in libkvm.

Reviewed by:	jhb
Approved by:	scottl (implicit)
MFC after:	1 week
Sponsored by:	Ampere Computing, Inc.
Differential Revision:	https://reviews.freebsd.org/D26131
2020-09-21 22:21:59 +00:00
D Scott Phillips
ab041f713a Move vm_page_dump bitset array definition to MI code
These definitions were repeated by all architectures, with small
variations. Consolidate the common definitons in machine
independent code and use bitset(9) macros for manipulation. Many
opportunities for deduplication remain in the machine dependent
minidump logic. The only intended functional change is increasing
the bit index type to vm_pindex_t, allowing the indexing of pages
with address of 8 TiB and greater.

Reviewed by:	kib, markj
Approved by:	scottl (implicit)
MFC after:	1 week
Sponsored by:	Ampere Computing, Inc.
Differential Revision:	https://reviews.freebsd.org/D26129
2020-09-21 22:20:37 +00:00
Michal Meloun
3182062142 Add missing assignment forgotten in r365899
Noticed by:	mav
MFC after:	1 month
MFC with:	r365899
2020-09-20 15:11:52 +00:00
Michal Meloun
95a85c125d Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions.
One problem with the bus_space_read_N() and bus_space_write_N() family of
functions is that they provide no protection against exceptions which can
occur when no physical hardware or device responds to the read or write
cycles. In such a situation, the system typically would panic due to a
kernel-mode bus error. The bus_space_peek_N() and bus_space_poke_N() family
of functions provide a mechanism to handle these exceptions gracefully
without the risk of crashing the system.

Typical example is access to PCI(e) configuration space in bus enumeration
function on badly implemented PCI(e) root complexes (RK3399 or Neoverse
N1 N1SDP and/or access to PCI(e) register when device is in deep sleep state.

This commit adds a real implementation for arm64 only. The remaining
architectures have bus_space_peek()/bus_space_poke() emulated by using
bus_space_read()/bus_space_write() (without exception handling).

MFC after:	1 month
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D25371
2020-09-19 11:06:41 +00:00
Edward Tomasz Napierala
70890254b3 Get rid of sv_errtbl and SV_ABI_ERRNO().
Reviewed by:	kib
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26388
2020-09-17 11:39:33 +00:00
Scott Long
74c781ed91 Refine the busdma template interface. Provide tools for filling in fields
that can be extended, but also ensure compile-time type checking.  Refactor
common code out of arch-specific implementations.  Move the mpr and mps
drivers to this new API.  The template type remains visible to the consumer
so that it can be allocated on the stack, but should be considered opaque.
2020-09-14 05:58:12 +00:00
John Baldwin
39585a4c10 Disable WITNESS for spin locks by default.
This matches all other architectures and removes substantial overhead.

Reported by:	arichardson (indirectly)
Reviewed by:	imp, arichardson
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26403
2020-09-11 00:06:16 +00:00
Mitchell Horne
7c7b8f577e RISC-V: fix some mismatched format specifiers
RISC-V is currently built with -Wno-format, which is how these went
undetected. Address them now before re-enabling those warnings.

Differential Revision:	https://reviews.freebsd.org/D26319
2020-09-08 13:21:13 +00:00
Brooks Davis
46b974a9db Round TF_SIZE up to the stack alignment (16-bytes).
The kernel adjusts the stack by TF_SIZE and the RISC-V ABI requires
that it remain 16-byte aligned.

Reported by:	CHERI, jrtc27
Reviewed by:	mhorne
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D26328
2020-09-04 21:55:22 +00:00
Mark Johnston
847ab36bf2 Include the psind in data returned by mincore(2).
Currently we use a single bit to indicate whether the virtual page is
part of a superpage.  To support a forthcoming implementation of
non-transparent 1GB superpages, it is useful to provide more detailed
information about large page sizes.

The change converts MINCORE_SUPER into a mask for MINCORE_PSIND(psind)
values, indicating a mapping of size psind, where psind is an index into
the pagesizes array returned by getpagesizes(3), which in turn comes
from the hw.pagesizes sysctl.  MINCORE_PSIND(1) is equal to the old
value of MINCORE_SUPER.

For now, two bits are used to record the page size, permitting values
of MAXPAGESIZES up to 4.

Reviewed by:	alc, kib
Sponsored by:	Juniper Networks, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D26238
2020-09-02 18:16:43 +00:00
Mark Johnston
2d838cd867 Add the MEM_EXTRACT_PADDR ioctl to /dev/mem.
This allows privileged userspace processes to find information about the
physical page backing a given mapping.  It is useful in applications
such as DPDK which perform some of their own memory management.

Reviewed by:	kib, jhb (previous version)
MFC after:	2 weeks
Sponsored by:	Juniper Networks, Inc.
Sponsored by:	Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D26237
2020-09-02 18:12:47 +00:00
Kristof Provost
3bebdc0564 riscv: very large dma mappings can cause integer overflow
Fix the return type for _bus_dmamap_addseg().
Based on the same fix done for arm64 in r348571.

Sponsored by:	Axiado
2020-09-02 11:33:31 +00:00