This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.
PR: 243711
Reviewed by: jhb, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23512
Submitted by: Bora Özarslan <borako.ozarslan@gmail.com>
Submitted by: Yang Wang <2333@outlook.jp>
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19917
When panicing because of an unhandled data abort from the kernel it is
useful to know the register state and faulting address to aid debugging.
Print these registers before calling panic.
Sponsored by: DARPA, AFRL
Once all CPUs are online, determine if they all support LSE atomics and
set lse_supported to indicate this. For now the atomic(9)
implementations are still always inlined, though it would be preferable
to create out-of-line functions to avoid text bloat. This was not done
here since big.little systems exist in which some CPUs implement LSE
while others do not, and ifunc resolution must occur well before this
scenario can be detected. It does seem unlikely that FreeBSD will
ever run on such platforms, however, so converting atomic(9) to use
ifuncs is probably a good next step.
Add a LSE_ATOMICS arm64 kernel configuration option to unconditionally
select LSE-based atomic(9) implementations when the target system is
known.
Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23325
These make use of the cas*, ld* and swp instructions added in ARMv8.1.
Testing shows them to be significantly more performant than LL/SC-based
implementations.
No functional change here since the wrappers still unconditionally
select the _llsc variants.
Reviewed by: andrew, kib
MFC after: 1 month
Submitted by: Ali Saidi <alisaidi@amazon.com> (original version)
Differential Revision: https://reviews.freebsd.org/D23324
Add a _llsc suffix for the existing LL/SC-based implementations and add
trivial wrappers. This is in preparation for supporting LSE-based
atomic(9) implementations.
No functional change intended.
Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23323
Parameterize the macros by type width as well as acq/rel semantics.
This makes modifying the implementations much less tedious and
error-prone and makes it easier to support alternate LSE-based
implementations. No functional change intended.
Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23322
In r357324 most of the use of gi_irq was moved to gi_lpi. Complete this
with the last few places we need the IRQ value and create gi_id for the
per-device value we need.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
When there are multiple GICv3 ITS devices we don't know which vmem is for
which device. Use device_get_nameunit to get a per-device name.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
This uses UMA to allocate space. It causes issues when there are multiple
ITS devices in the system where interrupts are not allocated from a low
address on some interrupt controllers. Disabling the quantum cache fixes
this on the Neoverse N1 SDP.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
After r355784 the td_oncpu field is no longer synchronized by the thread
lock, so the stack capture interrupt cannot be delievered precisely.
Fix this using a loop which drops the thread lock and restarts if the
wrong thread was sampled from the stack capture interrupt handler.
Change the implementation to use a regular interrupt instead of an NMI.
Now that we drop the thread lock, there is no advantage to the latter.
Simplify the KPIs. Remove stack_save_td_running() and add a return
value to stack_save_td(). On platforms that do not support stack
capture of running threads, stack_save_td() returns EOPNOTSUPP. If the
target thread is running in user mode, stack_save_td() returns EBUSY.
Reviewed by: kib
Reported by: mjg, pho
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23355
The GICv3 Software Overview suggests when allocating a new MSI/MSI-X
interrupt we need to call MAPD followed by MAPTI. Unfortunately the code
would place a MOVI command between these. This is invalid as it needs
values set by the MAPTI to be present.
Re-order so we allocate a temporary CPU for the interrupt, then use the
MAPTI command to assign the MSI to it.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
When there are multiple ITS devices in the system we would allocate a
configuration table for each, however only one table is needed as all the
ITS devices share this.
Allocate a table only when the global table is unset.
While here fix the type of this to be a pointer to a uint8_t array as the
entries are all 8 bits wide.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
When mapping MSI/MSI-X interrupts throught he Arm IORT ACPI tables we may
need to ignore an interrupt controller even if it is within the bounds the
entry describes. When the SMMUv3 is not using GSIV (non-MSI/MSI-X)
interrupts we need to read the defined field. The Performance Monitoring
Counter Group always ignores the first table entry.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
When using the processor ID value we mask off the low and high bits that
should be zero. Unfortunatly we don't shift the ID value so it won't be
affected. Add the shift when reading the ID as this will need to align
with the address based target value.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Doing so can cause deadlocks or panics during boot, if an interrupt
handler accesses uninitialized per-CPU scheduler structures. This seems
to occur frequently when running under QEMU or AWS. The idle threads
are set up to release a spinlock section and enable interrupts in
fork_exit(), so there is no need to enable interrupts earlier.
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23328
Static relocations for the immediate operand of a branch instruction
must be applied.
In a patch which implements LSE-based atomic(9) operations using ifuncs,
-zifunc-noplt reduces system CPU usage during a buildkernel by several
percent.
Also fix elf_reloc_internal() to return an error if symbol lookup fails.
Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17392
This method will set the desired voltaged based on values in the DTS.
It will not enable the regulator, this is the job of either a consumer
or regnode_set_constraint SYSINIT if the regulator is boot_on or always_on.
Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23216
r355473 vastly improved the readability and cleanliness of these Makefiles.
Every single one of them follows the same pattern and duplicates the exact
same logic.
Now that we have GENERATED/SRCS, split SRCS up into the two parameters we'll
use for ${MAKESYSCALLS} rather than assuming a specific ordering of SRCS and
include a common sysent.mk to handle the rest. This makes it less tedious to
make sweeping changes.
Some default values are provided for GENERATED/SYSENT_*; almost all of these
just use a 'syscalls.master' and 'syscalls.conf' in cwd, and they all use
effectively the same filenames with an arbitrary prefix. Most ABIs will be
able to get away with just setting GENERATED_PREFIX and including
^/sys/conf/sysent.mk, while others only need light additions. kern/Makefile
is the notable exception, as it doesn't take a SYSENT_CONF and the generated
files are spread out between ^/sys/kern and ^/sys/sys, but it otherwise fits
the pattern enough to use the common version.
Reviewed by: brooks, imp
Nice!: emaste
Differential Revision: https://reviews.freebsd.org/D23197
arm64 and riscv were only saving and restoring floating point
registers for sendsig() and sys_sigreturn(), but not for getcontext(),
setcontext(), and swapcontext().
While here, remove an always-false check for uap being NULL from
sys_sigreturn().
Reviewed by: br, mhorne
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23218
The gpio controller in rockchips SoC in a child of the pinctrl driver
and cannot control pullups and pulldowns.
Use the new fdt_pinctrl interface for accessing pin capabilities and
setting them.
We can now report that every pins is capable of being IN or OUT function
and PULLUP PULLDOWN.
If the pin isn't in gpio mode no changes will be allowed.
Reviewed by: ganbold (previous version)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22849
When either makesyscalls.lua or syscalls.master changes, all of the
${GENERATED} targets are now out-of-date. With make jobs > 1, this means we
will run the makesyscalls script in parallel for the same ABI, generating
the same set of output files.
Prior to r356603 , there is a large window for interlacing output for some
of the generated files that we were generating in-place rather than staging
in a temp dir. After that, we still should't need to run the script more
than once per-ABI as the first invocation should update all of them. Add
.ORDER to do so cleanly.
Reviewed by: brooks
Discussed with: sjg
Differential Revision: https://reviews.freebsd.org/D23099
These will reportedly be used in future uma changes.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D23019
The two switch regulator are always 3.0V.
Add a special case in get_voltage that if min=max we directly
return the value without calculating it.
Reviewed by: mmel
Differential Revision: https://reviews.freebsd.org/D23004
- Add all the Cortex-A CPU ID register values I can find.
- Add the Neoverse-N1 ID regiser value [1]
- Sort macros by register value.
PR: 243065
Submitted by: Ali Saidi <alisaidi AT amazon.com> [1]
Sponsored by: DARPA, AFRL (other than [1])
vm.kvm_size and vm.kvm_free are read only and marked as MPSAFE on i386
already. Mark them as that on amd64 and arm64 too to avoid locking Giant.
Reviewed by: kib (mentor)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D23039
mapping to the old read-only page with a mapping to the new read-write page.
To destroy the old mapping, pmap_enter() must destroy its page table and PV
entries and invalidate its TLB entry. This change simply invalidates that
TLB entry a little earlier, specifically, on amd64 and arm64, before the PV
list lock is held.
Reviewed by: kib, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23027
syscall is to query the CPU number and the NUMA domain the calling
thread is currently running on. The third argument is ignored.
It doesn't do anything regarding scheduling - it's literally
just a way to query the current state, without any guarantees
you won't get rescheduled an opcode later.
This unbreaks Java from CentOS 8
(java-11-openjdk-11.0.5.10-0.el8_0.x86_64).
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22972
entry's access flag and dirty state, and enable this feature when it's
available.
Ensure that we don't overlook a dirty state update that is concurrent
with a call to pmap_enter(). (Previously, all dirty state updates would
have occurred with the containing pmap's lock held, so a page table entry's
dirty state could not have changed while pmap_enter() held that same lock.)
Reviewed by: andrew, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22907
copy_file_range(2) is implemented natively since r350315, make it available
for Linux binaries too.
Reviewed by: kib (mentor), trasz (previous version)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D22959
This driver configure the registers in the GRF according to the value
of the regulators for the platform.
Some IP can run with either 3.0V or 1.8V, if we don't configure them
correctly according to the external voltage used they will not work.
It's only done at boot time for now and might be needed at runtime for
IP like sdmmc.
Reviewed by: mmel
Tested On: RockPro64, Firefly-RK3399 (gonzo), AIO-3288 (mmel)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22854
The RK808 driver was missing the LDO and switch regulators.
Add support for them.
Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22852
Due to clang and LLD's tendency to use a PLT for builtins, and as they
don't have full support for EABI, we sometimes have to deal with a PLT in
.ko files in a clang-built kernel.
As such, augment the in-kernel linker to support jump table processing.
As there is no particular reason to support lazy binding in kernel modules,
only implement Secure-PLT immediate binding.
As part of these changes, add elf_cpu_parse_dynamic() to the MD API of the
in-kernel linker (except on platforms that use raw object files.)
The new function will allow MD code to act on MD tags in _DYNAMIC.
Use this new function in the PowerPC MD code to ensure BSS-PLT modules using
PLT will be rejected during insertion, and to poison the runtime resolver to
ensure we get a clear panic reason if a call is made to the resolver.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D22608
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created. See the man page for
details on how to initialize a template.
Templates do not support tag filters. Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine. Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D22906
On arm64 the eret instruction is used to return from an exception handler.
Some implementations may speculate past this instruction into the next
function. As the user may control many registers in these functions add
a synchronisation barrier sequence after the eret instruction to stop these
CPUs from speculating out of the exception handler.
PR: 242676
Submitted by: Anthony Steinhauser <asteinhauser@google.com> (previous version)
MFC after: 1 week
incrementing (and decrementing) the ref_count on kernel page table pages.
They should not do this. Kernel page table pages are expected to have a
fixed ref_count. Address this problem by refactoring pmap_alloc{_l2,pde}()
and their callers. This also eliminates some duplicated code from the
callers.
Correctly implement PMAP_ENTER_NOREPLACE in pmap_enter_{l2,pde}() on kernel
mappings.
Reduce code duplication by defining a function, pmap_abort_ptp(), for
handling a common error case.
Handle a possible page table page leak in pmap_copy(). Suppose that we are
determining whether to copy a superpage mapping. If we abort because there
is already a mapping in the destination pmap at the current address, then
simply decrementing the page table page's ref_count is correct, because the
page table page must have a ref_count > 1. However, if we abort because we
failed to allocate a PV entry, this might be a just allocated page table
page that has a ref_count = 1, so we should call pmap_abort_ptp().
Simplify error handling in pmap_enter_quick_locked().
Reviewed by: kib, markj (an earlier)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22763
The maxpin counter starts at 0, fix one by one error.
This is still not totally correct for some banks in some SoC that have
fewer pins but this will be dealt with in another commit.
MFC after: 3 days
r351187 change the SYSCON_WRITE to SYSCON_MODIFY but didn't changed the
mask variable that used to hold the bitmask in the upper 16 bits of the
register that control which bits are changed. So we ended up clearing
bit from the upper 16bits half which are always 0 after a read.
Use the correct bit mask for bits that we want to clear.
MFC after: 3 days
fix an assert violation introduced in r355784. Without this spinlock_exit()
may see owepreempt and switch before reducing the spinlock count. amd64
had been optimized to do a single critical enter/exit regardless of the
number of spinlocks which avoided the problem and this optimization had
not been applied elsewhere.
Reported by: emaste
Suggested by: rlibby
Discussed with: jhb, rlibby
Tested by: manu (arm64)
than "/compat/linux". Useful when you have several compat directories
with different Linux versions and you don't want to clash with files
installed by linux-c7 packages.
Reviewed by: bcr (manpages)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22574
PHYS_TO_VM_PAGE() operation that always returns the same vm_page_t out of
the loop. (Since arm64 is configured as VM_PHYSSEG_SPARSE, the
implementation of PHYS_TO_VM_PAGE() is more costly than that of
VM_PHYSSEG_DENSE platforms, like amd64.)
MFC after: 1 week
Unfortunately, there are some limitations:
- memory aperture of his controller is only 16MiB, so it is nearly
unusable for graphic cards
- every attempt to generate type 1 config cycle always causes trap.
These config cycles are disabled now and we don't support cards
with PCIe switch.
- in some cases, attempt to do config cycle to (probably) not-yet ready
card also causes trap. This cannot be detected at runtime, but it seems
like very rare issue.
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D22724
over the usual fsync(2).
This silences some warnings when running "apt-get upgrade".
Reviewed by: brooks, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22371
This code is non-obvious when reading for the first time. To help with
understanding of it add comments explaining what it's doing.
While here use macros from armreg.h rather than magic numbers.
Sponsored by: DARPA, AFRL
Only bank 0 and bank 2 are different than other rockchip SoC, fix this.
While here remove some debug printfs that where added in r355648
MFC after: 3 days
X-MFC-With: r355648
Those clocks are always enable by default and are not really explained
in the TRM but the reason we had them is that they have the periph clock
as a parent and those parent should never be disable which can happen
if we disable all the childs. The current childs are the sd/emmc/sdio clocks
so the board will hang if we disable them.
MFC after: 1 month
We used to include the hisi version if soc_hisi_hi6220 was present,
include the altera version if dwmmc_altera was present and include
the rockchip version if soc_rockchip_rk3328 was present.
Now every version have it's own device directive.
The rockchip version isn't named dwmmc_rockchip because all other
rockchip driver are named rk_XXX.
MFC after: 1 month
This is a 32-bit structure embedded in each vm_page, consisting mostly
of page queue state. The use of a structure makes it easy to store a
snapshot of a page's queue state in a stack variable and use cmpset
loops to update that state without requiring the page lock.
This change merely adds the structure and updates references to atomic
state fields. No functional change intended.
Reviewed by: alc, jeff, kib
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22650
Partially revert r354741 and r354754 and go back to allocating a
fixed-size chunk of stack space for the auxiliary vector. Keep
sv_copyout_auxargs but change it to accept the address at the end of
the environment vector as an input stack address and no longer
allocate room on the stack. It is now called at the end of
copyout_strings after the argv and environment vectors have been
copied out.
This should fix a regression in r354754 that broke the stack alignment
for newer Linux amd64 binaries (and probably broke Linux arm64 as
well).
Reviewed by: kib
Tested on: amd64 (native, linux64 (only linux-base-c7), and i386)
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22695
In some cases, like is locked bootstrap or device's inability to boot from
removable media, we cannot use standard boot sequence and is necessary to
boot kernel directly from U-Boot.
Discussed with: jhibbits
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D13861
Use the power of variable to avoid spelling out source and generated
files too many times. The previous Makefiles were hard to read, hard to
edit, and badly formatted.
Reviewed by: kevans, emaste
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22714
Interrupt based driver, implements SPI mode and clock configuration.
Tested on espressobin and SG-3200.
Sponsored by: Rubicon Communications, LLC (Netgate)
preempted after an "ic" or "tlbi" instruction but before it performed a
"dsb" instruction. The "ic" and "tlbi" instructions have unusual
synchronization requirements. If the old thread migrates to a new
processor, its completion of a "dsb" instruction on that new processor does
not guarantee that the "ic" or "tlbi" instructions performed on the old
processor have completed.
This issue is not restricted to the kernel. Since locore.S sets the UCI bit
in SCTLR, user-space programs can perform "ic ivau" instructions (as well as
some forms of the "dc" instruction).
Reviewed by: andrew, kib, markj, mmel
X-MFC with: r355145
Differential Revision: https://reviews.freebsd.org/D22622
This controller is a bit tricky as the STOP condition must be indicated in
the last tranferred byte, some devices will not like the repeated start
behavior of this controller. A proper fix to this issue is in the works.
This driver works in polling mode, can be used early in the boot (required
in some cases).
Tested on espressobin/SG-1100 and the SG-3200.
Obtained from: pfSense
Sponsored by: Rubicon Communications, LLC (Netgate)
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.
- Use ustringp for the location of the argv and environment strings
and allow destp to travel further down the stack for the stackgap
and auxv regions.
- Update the Linux copyout_strings variants to move destp down the
stack as was done for the native ABIs in r263349.
- Stop allocating a space for a stack gap in the Linux ABIs. This
used to hold translated system call arguments, but hasn't been used
since r159992.
Reviewed by: kib
Tested on: md64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22501
RK3399 PLLs have three modes :
- Normal, where they behave normally and their freq is calculated based on
the registers values.
- Slow, where the PLL freq is 24Mhz (well, the external oscillator).
- Deep Slow, used for suspend where the freq is 32Khz.
We used to put every CPU related PLL in normal mode but it can cause problem
if the firmware didn't setup the clocks register correctly.
And even if it did but left the pll in slow or deep slow mode that might be
because the PMIC suppling voltage for the CPU haven't been configured yet
and we cannot do that at this point.
So remove the ability to set PLLs to normal mode at boot to avoid any problems.
After discussing with mmel@, it was clear this is insufficient to address
all the needs. mmel@ will commit his original patch, from
https://reviews.freebsd.org/D13861, and the additions needed from r354714
will be made afterward.
Requested by: mmel
Sponsored by: Juniper Networks, Inc.
Remove the duplicate macros that defined a subset of the VM_MEMATTR values.
While here use VM_MEMATTR macros when filling in the MAIR register.
Reviewed by: alc, markj
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22241
processor in pmap_invalidate_{all,page,range}(). These functions are using
an instruction that broadcasts the TLB invalidation to every processor, so
even if a thread migrates in the middle of one of these functions every
processor will still perform the required TLB invalidations.
Reviewed by: andrew, markj
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D22502
- implement of all but mmc clocks. MMC clocks will be added later by own commit.
- use 'link' clock type for external clocks.
- use macros for initialization of structure's named members.
MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22441
This should have been done back when it was added, but it was not. It only
really adds an extra entry for memory mapping bits in bcm2835_vcbus.c, so
nothing too extensive yet.
Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in
the FreeBSD kernel. It is a useful tool for finding data races between
threads executing on different CPUs.
This can be enabled by enabling KCSAN in the kernel config, or by using the
GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later
needs a compiler change to allow -fsanitize=thread that KCSAN uses.
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22315
invalidation in reclaim_pv_chunk(). (2) Prevent an "invalid ASID" assertion
failure in reclaim_pv_chunk(). The detailed explanation for this change is
provided by r354792.
X-MFC with: r354792
flua is bootstrapped as part of the build for those on older
versions/revisions that don't yet have flua installed. Once upgraded past
r354833, "make sysent" will again naturally work as expected.
Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D21894
The purpose of this option is to make it easier to track down memory
corruption bugs by reducing the number of malloc(9) types that might
have recently been associated with a given chunk of memory. However, it
increases fragmentation and is disabled in release kernels.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
As on amd64, there is no need for mappings created by these functions to
be executable.
Reviewed by: alc, andrew
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22141
invalidation in pmap_remove_all(). (2) Prevent an "invalid ASID" assertion
failure in pmap_remove_all().
The architecture definition specifies that the TLB will not cache mappings
that don't have the "AF" bit set, so pmap_remove_all() needn't issue a TLB
invalidation for mappings that don't have the "AF" bit set.
We allocate ASIDs lazily. Specifically, we don't allocate an ASID for a
pmap until we are activating it. Now, consider what happens on a fork().
Before we activate the child's pmap, we use pmap_copy() to copy mappings
from the parent's pmap to the child's. These new mappings have their "AF"
bits cleared. Suppose that the page daemon decides to reclaim a page that
underlies one of these new mappings. Previously, the pmap_invalidate_page()
performed by pmap_remove_all() on a mapping in the child's pmap would fail
an assertion because that pmap hasn't yet been assigned an ASID. However,
we don't need to issue a TLB invalidation for such mappings because they
can't possibly be cached in the TLB.
Reported by: bob prohaska <fbsd@www.zefox.net>
Reviewed by: markj
MFC after: 1 week
X-MFC-before: r354286
Differential Revision: https://reviews.freebsd.org/D22388
Change the FreeBSD ELF ABIs to use this new hook to copyout ELF auxv
instead of doing it in the sv_fixup hook. In particular, this new
hook allows the stack space to be allocated at the same time the auxv
values are copied out to userland. This allows us to avoid wasting
space for unused auxv entries as well as not having to recalculate
where the auxv vector is by walking back up over the argv and
environment vectors.
Reviewed by: brooks, emaste
Tested on: amd64 (amd64 and i386 binaries), i386, mips, mips64
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22355
Summary:
Boot arm64 kernel using booti command from U-boot. booti can relocate initrd
image into higher ram addresses, therefore align the initrd load address to 1GiB
and create VA = PA map for it. Create L2 pagetable entries to copy the initrd
image into KVA.
(parts of the code in https://reviews.freebsd.org/D13861 was referred and used
as appropriate)
Submitted by: Siddharth Tuli <siddharthtuli_gmail.com>
Reviewed by: manu
Sponsored by: Juniper Networks, Inc
Differential Revision: https://reviews.freebsd.org/D22255
For any size that isn't page-aligned, we end up not pre-allocating enough
for a single mapping because we truncate the size instead of rounding up to
make sure the last bit is accounted for, leaving us one page shy of what we
need to fulfill a request.
Differential Revision: https://reviews.freebsd.org/D22288
There is no reason why the pmap_invalidate_all() in pmap_remove_pages()
must be performed before the final PV list lock release. Move it past
the lock release.
Eliminate a stale comment from pmap_page_test_mappings(). We implemented
a modified bit in r350004.
MFC after: 1 week
- add support for log2 based dividers
- use proper write mask when writing to divider register
MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22283
- add support for fractional dividers
- allow to declare fixed and linked clock
MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22282
These are direct copies of the 32 bit functions, adjusted ad needed.
While here fix atomic_fcmpset_16 to use the valid load and store exclusive
instructions.
Sponsored by: DARPA, AFRL
This function will call the regnode_check_voltage method for a given regulator
and check if the desired voltage in reachable by it.
Also adds a default method that check the std_param and which should be enough
for most regulators and add it as the method for axp* rk805 and fixed regulators.
Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22260
Some userland libraries incude machine/pcb.h and this needs the full
definition of struct debug_monitor_state. To allow this to work move
stuct debug_monitor_state out of the _KERNEL guard.
Sponsored by: DARPA, AFRL
switching. The indirect costs being unnecessary TLB misses that are
incurred when ASIDs are not used. In fact, currently, when we perform a
context switch on one processor, we issue a broadcast TLB invalidation that
flushes the TLB contents on every processor.
Mark all user-space ("ttbr0") page table entries with the non-global flag so
that they are cached in the TLB under their ASID.
Correct an error in pmap_pinit0(). The pointer to the root of the page
table was being initialized to the root of the kernel-space page table
rather than a user-space page table. However, the root of the page table
that was being cached in process 0's md_l0addr field correctly pointed to a
user-space page table. As long as ASIDs weren't being used, this was
harmless, except that it led to some unnecessary page table switches in
pmap_switch(). Specifically, other kernel processes besides process 0 would
have their md_l0addr field set to the root of the kernel-space page table,
and so pmap_switch() would actually change page tables when switching
between process 0 and other kernel processes.
Implement a workaround for Cavium erratum 27456 affecting ThunderX machines.
(I would like to thank andrew@ for providing the code to detect the affected
machines.)
Address integer overflow in the definition of TCR_ASID_16.
Setup TCR according to the PARange and ASIDBits fields from
ID_AA64MMFR0_EL1. Previously, TCR_ASID_16 was unconditionally set.
Modify build_l1_block_pagetable so that lower attributes, such as ATTR_nG,
can be specified as a parameter.
Eliminate some unused code.
Earlier versions were tested to varying degrees by: andrew, emaste, markj
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D21922
Implement get/fill_dbregs on arm64. This is used by ptrace with the
PT_GETDBREGS and PT_SETDBREGS requests. It allows userspace to set hardware
breakpoints.
The struct dbreg is based on Linux to ease adding hardware breakpoint
support to debuggers.
Reviewed by: jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22195
Altough in the comment above the pmap_change_attr() it was mentioned
that VA could be in KV or DMAP memory space. However,
pmap_change_attr_locked() was accepting only the values inside the DMAP
memory range.
To fix that, the condition check was changed so also the va inside the
KV memory range would be accepted.
The sample use case that wasn't supported is the PCI Device that has the
BAR which should me mapped with the Write Combine attribute - for
example BAR2 of the ENA network controller on the A1 instances on AWS.
Tested on A1 AWS instance and changed ENA BAR2 mapped resource to be
write-combined memory region.
Differential Revision: https://reviews.freebsd.org/D22055
MFC after: 2 weeks
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Arm64 allows us to create execute only mappings. To make sure userspace is
unable to accidentally execute kernel code set the user execute never
bit in the kernel page tables.
MFC after: 1 week
Sponsored by: DARPA, AFRL
Summary:
ARM64 currently treats all data abort exceptions as page faults. This
can cause infinite loops on non-page fault faults, such as alignment faults.
Since kernel-side alignment faults should be avoided, this adds support directly
to the el0 fault handler, instead of the data_abort() handler.
Test Plan: Tested on rpi3, with a misaligned ldm test.
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D22133
We may want to mask exceptions when in userspace. This was previously
impossible as threads are created with all exceptions unmasked and
signals expected userspace to mask any. Fix these by copying the
mask state on thread creation and allow exceptions to be masked on
signal return, as long as they don't change.
Sponsored by: DARPA, AFRL
To allow consistent values to be used in both the kernel and userspace
create a function for these to be read from the kernel. They use a newly
created macro with the name of the ID register to read. For now there is
redundant information in the user_regs array as it still holds the CRm and
Op2 values, however this will be fixed in a later change.
This will be used by ptrace to allow hardware breakpoints in userspace.
Sponsored by: DARPA, AFRL
These instructions are used to access the registers described in armreg.h,
and will be used in a future change to create a per-register identification
macro.
Sponsored by: DARPA, AFRL
The debug monitor register state is now stored in a struct and updated
when required. Currently there is only a kernel state, however a
per-process state will be added in a future change.
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22128
Previously we would call data_abort on all data and instruction aborts
however this is incorrect for most abort types. Move to use an array
of function pointers to allow for more handlers to be easily added.
Reviewed by: jhibbits
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22170
Because of the previous naming scheme the old ID_AA64PFR0_EL1 macro
collided with a potential macro for the register of the same name. To fix
this collision rename these macros.
Sponsored by: DARPA, AFRL
Declare retry in the function scope.
Rename it to retry as there is a timeout function which was
causing to code to compile.
Reported by: jhibbits
MFC after: 1 month
X-MFC-WITH: r354089
The RK805 regs array was being allocated before it's required size was
known, causing the driver to use memory it didn't own. That memory
was subsequently allocated and used elsewhere causing later fatal data
aborts in rk805_map().
Whilst I'm here, add a sanity check to catch unsupported PMICs (this
shouldn't ever get hit because the probe should have failed).
Reviewed by: manu
MFC after: 1 week
Sponsored by: Google
This is a driver for the USB3 PHY present in the RK3399.
While the phy support DP (Display Port) the driver doesn't has we have
no driver to test this with for now.
All the lane and pll configuration is just magic values from rockchip.
While the manual have some info on those registers it's really hard to
understand how to calculate those values (if there is a way).
MFC after: 1 month
As with amd64 NUMA is required for reasonable operation on big-iron
arm64 systems and is expected to have no significant impact on small
systems. Enable it now for wider testing in advance of FreeBSD 13.0.
You can use the 'vm.ndomains' sysctl to see if multiple domains are in
use - for example (from Cavium/Marvell ThunderX2):
# sysctl vm.ndomains
vm.ndomains: 2
No objection: manu
Sponsored by: The FreeBSD Foundation
When we raise a data abort from the kernel we need to enable interrupts,
however we shouldn't be doing this when in the kernel debugger. In this
case interrupts can lead to a further panic as they don't expect to be
run from such a context.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
in simple multifunction driver.
- follow interrupt changes in DT. Split old ICU driver to function oriented
parts and add drivers for newly defined parts (system error interrupts).
- Many drivers are children of simple multifunction driver. But after r349596
simple MF driver doesn't longer exports memory resources, and all children
must use syscon interface to access their registers. Adapt affected
drivers to this fact.
MFC after: 3 weeks
Move futex_mtx to linux_common.ko for amd64 and aarch64 along
with respective list/mutex init/destroy.
PR: 240989
Reported by: Alex S <iwtcex@gmail.com>
Debugnet is a simplistic and specialized panic- or debug-time reliable
datagram transport. It can drive a single connection at a time and is
currently unidirectional (debug/panic machine transmit to remote server
only).
It is mostly a verbatim code lift from netdump(4). Netdump(4) remains
the only consumer (until the rest of this patch series lands).
The INET-specific logic has been extracted somewhat more thoroughly than
previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as
much as possible as is protocol-independent, remains in debugnet.c. The
separation is not perfect and future improvement is welcome. Supporting
INET6 is a long-term goal.
Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to
'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the
generic module would be more confusing than the refactoring.
The only functional change here is the mbuf allocation / tracking. Instead
of initiating solely on netdump-configured interface(s) at dumpon(8)
configuration time, we watch for any debugnet-enabled NIC for link
activation and query it for mbuf parameters at that time. If they exceed
the existing high-water mark allocation, we re-allocate and track the new
high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone.
In a future patch in this series, this will allow initiating netdump from
panic ddb(4) without pre-panic configuration.
No other functional change intended.
Reviewed by: markj (earlier version)
Some discussion with: emaste, jhb
Objection from: marius
Differential Revision: https://reviews.freebsd.org/D21421
After r352110 the page lock no longer protects a page's identity, so
there is no purpose in locking the page in pmap_mincore(). Instead,
if vm.mincore_mapped is set to the non-default value of 0, re-lookup
the page after acquiring its object lock, which holds the page's
identity stable.
The change removes the last callers of vm_page_pa_tryrelock(), so
remove it.
Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21823
Arm updates these with each new architecture revision. To help keep them
updated use a collection of tables to hold the needed information to
decode these registers.
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22020
callers hold it.
This simplifies pmap code and removes a dependency on the object lock.
Reviewed by: kib, markj
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D21596
busy acquires while held.
This allows code that would need to acquire and release a very large number
of page busy locks to use the old mechanism where busy is only checked and
not held. This comes at the cost of false positives but never false
negatives which the single consumer, vm_fault_soft_fast(), handles.
Reviewed by: kib
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D21592
This adds support for H6 SoC.
Add a CCU driver for H6 that support all PLLs and most of the clocks
that we are intersted in for now (i2c, mmc, usb, etc ...)
MFC after: 1 month
Summary: Add trivial 32-bit arm cores on aarch64 support for gcore. This
doesn't handle fpregs.
Reviewed by: #arm, andrew
Sponsored by: Juniper Networks, Inc
Differential Revision: https://reviews.freebsd.org/D21947
We only support host mode so those functions are just added so
we won't panic when generic-{e,o}hci will set the phy to host mode.
MFC after: 1 month
X-MFC-With: r353062
