1086 Commits

Author SHA1 Message Date
Scott Long
757d4fbaa7 Introduce the concept of busdma tag templates. A template can be allocated
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created.  See the man page for
details on how to initialize a template.

Templates do not support tag filters.  Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine.  Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.

Reviewed by:	imp, kib
Differential Revision:	https://reviews.freebsd.org/D22906
2019-12-24 14:48:46 +00:00
Alan Cox
7c237b7c3a Correct a mistakenly inverted condition in r355833.
Noticed by:	kib
X-MFC with:	r355833
2019-12-20 20:46:26 +00:00
Andrew Turner
494278bb26 Stop speculation past an eret instruction
On arm64 the eret instruction is used to return from an exception handler.
Some implementations may speculate past this instruction into the next
function. As the user may control many registers in these functions add
a synchronisation barrier sequence after the eret instruction to stop these
CPUs from speculating out of the exception handler.

PR:		242676
Submitted by:	Anthony Steinhauser <asteinhauser@google.com> (previous version)
MFC after:	1 week
2019-12-19 08:52:16 +00:00
Alan Cox
48ef33180a When pmap_enter_{l2,pde}() are called to create a kernel mapping, they are
incrementing (and decrementing) the ref_count on kernel page table pages.
They should not do this.  Kernel page table pages are expected to have a
fixed ref_count.  Address this problem by refactoring pmap_alloc{_l2,pde}()
and their callers.  This also eliminates some duplicated code from the
callers.

Correctly implement PMAP_ENTER_NOREPLACE in pmap_enter_{l2,pde}() on kernel
mappings.

Reduce code duplication by defining a function, pmap_abort_ptp(), for
handling a common error case.

Handle a possible page table page leak in pmap_copy().  Suppose that we are
determining whether to copy a superpage mapping.  If we abort because there
is already a mapping in the destination pmap at the current address, then
simply decrementing the page table page's ref_count is correct, because the
page table page must have a ref_count > 1.  However, if we abort because we
failed to allocate a PV entry, this might be a just allocated page table
page that has a ref_count = 1, so we should call pmap_abort_ptp().

Simplify error handling in pmap_enter_quick_locked().

Reviewed by:	kib, markj (an earlier)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22763
2019-12-18 18:21:39 +00:00
Ian Lepore
a856c60bdd Remove unnecessary MODULE_DEPEND() from imx_i2c.c, and also from rk_i2c
where it got copied to.
2019-12-17 17:03:03 +00:00
Emmanuel Vadot
bc188bd4f8 arm64: rockchip: rk_gpio: Fix pin number
The maxpin counter starts at 0, fix one by one error.
This is still not totally correct for some banks in some SoC that have
fewer pins but this will be dealt with in another commit.

MFC after:	3 days
2019-12-17 10:57:31 +00:00
Emmanuel Vadot
3df0c02623 arm64: rockchip: rk_pinctrl: Fix clear bits in SYSCON_MODIFY
r351187 change the SYSCON_WRITE to SYSCON_MODIFY but didn't changed the
mask variable that used to hold the bitmask in the upper 16 bits of the
register that control which bits are changed. So we ended up clearing
bit from the upper 16bits half which are always 0 after a read.
Use the correct bit mask for bits that we want to clear.

MFC after:	3 days
2019-12-17 10:55:28 +00:00
Jeff Roberson
a94ba188c3 Repeat the spinlock_enter/exit pattern from amd64 on other architectures to
fix an assert violation introduced in r355784.  Without this spinlock_exit()
may see owepreempt and switch before reducing the spinlock count.  amd64
had been optimized to do a single critical enter/exit regardless of the
number of spinlocks which avoided the problem and this optimization had
not been applied elsewhere.

Reported by:	emaste
Suggested by:	rlibby
Discussed with:	jhb, rlibby
Tested by:	manu (arm64)
2019-12-16 20:15:04 +00:00
Edward Tomasz Napierala
b5f20658ee Add compat.linux.emul_path, so it can be set to something other
than "/compat/linux".  Useful when you have several compat directories
with different Linux versions and you don't want to clash with files
installed by linux-c7 packages.

Reviewed by:	bcr (manpages)
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22574
2019-12-16 20:07:04 +00:00
Alan Cox
68ca966558 Apply a small optimization to pmap_remove_l3_range(). Specifically, hoist a
PHYS_TO_VM_PAGE() operation that always returns the same vm_page_t out of
the loop.  (Since arm64 is configured as VM_PHYSSEG_SPARSE, the
implementation of PHYS_TO_VM_PAGE() is more costly than that of
VM_PHYSSEG_DENSE platforms, like amd64.)

MFC after:	1 week
2019-12-15 22:41:57 +00:00
Michal Meloun
dfd1d0fcab Add driver for Rockchip PCIe root complex found in RK3399 SOC.
Unfortunately, there are some limitations:
- memory aperture of his controller is only 16MiB, so it is nearly
  unusable for graphic cards
- every attempt to generate type 1 config cycle always causes trap.
  These config cycles are disabled now and we don't support cards
  with PCIe switch.
- in some cases, attempt to do config cycle to (probably) not-yet ready
  card also causes trap. This cannot be detected at runtime, but it seems
  like very rare issue.

MFC after:	3 weeks
Differential Revision:  https://reviews.freebsd.org/D22724
2019-12-14 14:56:34 +00:00
Edward Tomasz Napierala
cf69fe66d4 Add sync_file_range(2) implementation to linux(4); it's a thin wrapper
over the usual fsync(2).

This silences some warnings when running "apt-get upgrade".

Reviewed by:	brooks, emaste
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22371
2019-12-14 13:37:17 +00:00
Edward Tomasz Napierala
0cde2b3239 Regen after r355752.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22371
2019-12-14 13:32:37 +00:00
Edward Tomasz Napierala
0610f417a4 Fix definitions for linuxulator's sync_file_range(2).
Reviewed by:	brooks, emaste
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22371
2019-12-14 13:30:43 +00:00
Andrew Turner
65565c9784 Add comments and macros to the tcr_el1 setting code to help understand it.
This code is non-obvious when reading for the first time. To help with
understanding of it add comments explaining what it's doing.

While here use macros from armreg.h rather than magic numbers.

Sponsored by:	DARPA, AFRL
2019-12-12 18:27:54 +00:00
Emmanuel Vadot
e63adc105d arm64: rockchip: rk_pinctrl: Fix parse_bias for RK3399
Only bank 0 and bank 2 are different than other rockchip SoC, fix this.
While here remove some debug printfs that where added in r355648

MFC after:	3 days
X-MFC-With:	r355648
2019-12-12 13:21:43 +00:00
Emmanuel Vadot
064486c05f arm64: rockchip: rk_pinctrl: Add bias parsing based on the SoC type
Not all rockchip have the same value for pullup/pulldown so add a function
per SoC and call the right one to have the proper value.

MFC after:	3 days
2019-12-12 13:02:22 +00:00
Emmanuel Vadot
6fe4e8bdbd arm64: rk3328: Add the *clk_peri_niu clocks
Those clocks are always enable by default and are not really explained
in the TRM but the reason we had them is that they have the periph clock
as a parent and those parent should never be disable which can happen
if we disable all the childs. The current childs are the sd/emmc/sdio clocks
so the board will hang if we disable them.

MFC after:	1 month
2019-12-11 18:39:05 +00:00
Emmanuel Vadot
946c511ee5 arm64: Add explicit devices for dwmmc variant
We used to include the hisi version if soc_hisi_hi6220 was present,
include the altera version if dwmmc_altera was present and include
the rockchip version if soc_rockchip_rk3328 was present.
Now every version have it's own device directive.
The rockchip version isn't named dwmmc_rockchip because all other
rockchip driver are named rk_XXX.

MFC after:	1 month
2019-12-11 18:36:07 +00:00
Mark Johnston
5cff1f4dc3 Introduce vm_page_astate.
This is a 32-bit structure embedded in each vm_page, consisting mostly
of page queue state.  The use of a structure makes it easy to store a
snapshot of a page's queue state in a stack variable and use cmpset
loops to update that state without requiring the page lock.

This change merely adds the structure and updates references to atomic
state fields.  No functional change intended.

Reviewed by:	alc, jeff, kib
Sponsored by:	Netflix, Intel
Differential Revision:	https://reviews.freebsd.org/D22650
2019-12-10 18:14:50 +00:00
John Baldwin
d8010b1175 Copy out aux args after the argument and environment vectors.
Partially revert r354741 and r354754 and go back to allocating a
fixed-size chunk of stack space for the auxiliary vector.  Keep
sv_copyout_auxargs but change it to accept the address at the end of
the environment vector as an input stack address and no longer
allocate room on the stack.  It is now called at the end of
copyout_strings after the argv and environment vectors have been
copied out.

This should fix a regression in r354754 that broke the stack alignment
for newer Linux amd64 binaries (and probably broke Linux arm64 as
well).

Reviewed by:	kib
Tested on:	amd64 (native, linux64 (only linux-base-c7), and i386)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22695
2019-12-09 19:17:28 +00:00
Emmanuel Vadot
670f36540d arm64: Use the kenv provided by loader
Otherwise we have an empty kenv a likely cannot boot.

Submitted by:	kevans
MFC after:	1 month
X-MFC-With:	355487
2019-12-09 15:23:05 +00:00
Michal Meloun
5641eda2f3 Add support for booting kernel directly from U-Boot using booti command.
In some cases, like is locked bootstrap or device's inability to boot from
removable media, we cannot use standard boot sequence and is necessary to
boot kernel directly from U-Boot.

Discussed with:	jhibbits
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D13861
2019-12-07 16:14:23 +00:00
Brooks Davis
af796bfa71 sysent: Reduce duplication and improve readability.
Use the power of variable to avoid spelling out source and generated
files too many times.  The previous Makefiles were hard to read, hard to
edit, and badly formatted.

Reviewed by:	kevans, emaste
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22714
2019-12-06 23:59:23 +00:00
Luiz Otavio O Souza
8cfe2a7ad3 Add the SPI driver for the Marvell Armada 37x0 SoC.
Interrupt based driver, implements SPI mode and clock configuration.

Tested on espressobin and SG-3200.

Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-12-06 12:55:39 +00:00
Alan Cox
1bef4955bc On a context switch, handle the possibility that the old thread was
preempted after an "ic" or "tlbi" instruction but before it performed a
"dsb" instruction.  The "ic" and "tlbi" instructions have unusual
synchronization requirements.  If the old thread migrates to a new
processor, its completion of a "dsb" instruction on that new processor does
not guarantee that the "ic" or "tlbi" instructions performed on the old
processor have completed.

This issue is not restricted to the kernel.  Since locore.S sets the UCI bit
in SCTLR, user-space programs can perform "ic ivau" instructions (as well as
some forms of the "dc" instruction).

Reviewed by:	andrew, kib, markj, mmel
X-MFC with:	r355145
Differential Revision:	https://reviews.freebsd.org/D22622
2019-12-05 19:25:49 +00:00
Luiz Otavio O Souza
8a82d5675d Add the I2C driver for the Armada 37x0.
This controller is a bit tricky as the STOP condition must be indicated in
the last tranferred byte, some devices will not like the repeated start
behavior of this controller.  A proper fix to this issue is in the works.

This driver works in polling mode, can be used early in the boot (required
in some cases).

Tested on espressobin/SG-1100 and the SG-3200.

Obtained from:	pfSense
Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-12-05 00:56:03 +00:00
Warner Losh
f86e60008b Regularize my copyright notice
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
  All Rights Reserved on same line as other copyright holders (but not
  me). Other such holders are also listed last where it's clear.
2019-12-04 16:56:11 +00:00
Li-Wen Hsu
0386b6c8a0 Fix arm64 build after r355373
Sponsored by:	The FreeBSD Foundation
2019-12-04 08:21:54 +00:00
John Baldwin
31174518d2 Use uintptr_t instead of register_t * for the stack base.
- Use ustringp for the location of the argv and environment strings
  and allow destp to travel further down the stack for the stackgap
  and auxv regions.
- Update the Linux copyout_strings variants to move destp down the
  stack as was done for the native ABIs in r263349.
- Stop allocating a space for a stack gap in the Linux ABIs.  This
  used to hold translated system call arguments, but hasn't been used
  since r159992.

Reviewed by:	kib
Tested on:	md64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22501
2019-12-03 23:17:54 +00:00
Emmanuel Vadot
e213223c9b Remove "all rights reserved" from copyright for the file I own.
Some of the files have both me and Jared McNeill and he gave me
permission to remove it from his files too.
2019-12-03 21:00:45 +00:00
Emmanuel Vadot
f141cc31dc arm64: rockchip: rl3399: Remove the ability to put the PLL in normal mode at boot
RK3399 PLLs have three modes :
 - Normal, where they behave normally and their freq is calculated based on
   the registers values.
 - Slow, where the PLL freq is 24Mhz (well, the external oscillator).
 - Deep Slow, used for suspend where the freq is 32Khz.

We used to put every CPU related PLL in normal mode but it can cause problem
if the firmware didn't setup the clocks register correctly.
And even if it did but left the pll in slow or deep slow mode that might be
because the PMIC suppling voltage for the CPU haven't been configured yet
and we cannot do that at this point.
So remove the ability to set PLLs to normal mode at boot to avoid any problems.
2019-12-03 19:18:32 +00:00
Justin Hibbits
750d951f5a revert r354714 "Boot arm64 kernel using booti command from U-boot."
After discussing with mmel@, it was clear this is insufficient to address
all the needs.  mmel@ will commit his original patch, from
https://reviews.freebsd.org/D13861, and the additions needed from r354714
will be made afterward.

Requested by:	mmel
Sponsored by:	Juniper Networks, Inc.
2019-12-02 15:07:06 +00:00
Andrew Turner
ef3e1e13b6 Use the VM_MEMATTR macros to describe the MAIR offsets.
Remove the duplicate macros that defined a subset of the VM_MEMATTR values.
While here use VM_MEMATTR macros when filling in the MAIR register.

Reviewed by:	alc, markj
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22241
2019-11-29 16:14:32 +00:00
Emmanuel Vadot
19b38c94dc arm64: rockchip: rk3328: Add TSADC clocks
Add the clocks so we can use the rk_tsadc driver to monitor
the cpu temperature.
2019-11-28 20:46:24 +00:00
Emmanuel Vadot
a0cb4996c6 arm64: rockchip: tsadc: Do not free the sysctl context is it wasn't created
MFC after:	3 weeks
X-MFC-With:	r355173
2019-11-28 20:17:03 +00:00
Michal Meloun
6903d37518 Add driver for temperature sensors found in RK32898, RK3328 and RK3399 SoC's.
MFC after:	3 weeks
Reviewed by:	manu
Differential Revision:  https://reviews.freebsd.org/D22442
2019-11-28 17:01:31 +00:00
Alan Cox
7b3c31acbe There is no reason why we need to pin the underlying thread to its current
processor in pmap_invalidate_{all,page,range}().  These functions are using
an instruction that broadcasts the TLB invalidation to every processor, so
even if a thread migrates in the middle of one of these functions every
processor will still perform the required TLB invalidations.

Reviewed by:	andrew, markj
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D22502
2019-11-27 20:33:49 +00:00
Michal Meloun
fa2ec6b439 Finish implementation of RK3299 clocks.
- implement of all but mmc clocks. MMC clocks will be added later by own commit.
- use 'link' clock type for external clocks.
- use macros for initialization of structure's named members.

MFC after:	3 weeks
Reviewed by:	manu
Differential Revision:  https://reviews.freebsd.org/D22441
2019-11-26 17:56:39 +00:00
Kyle Evans
0227a14997 arm64/NOTES: add SOC_BRCM_BCM2838
This should have been done back when it was added, but it was not. It only
really adds an extra entry for memory mapping bits in bcm2835_vcbus.c, so
nothing too extensive yet.
2019-11-23 03:38:26 +00:00
Andrew Turner
68cad68149 Add kcsan_md_unsupported from NetBSD.
It's used to ignore virtual addresses that may have a different physical
address depending on the CPU.

Sponsored by:	DARPA, AFRL
2019-11-21 13:22:23 +00:00
Andrew Turner
849aef496d Port the NetBSD KCSAN runtime to FreeBSD.
Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in
the FreeBSD kernel. It is a useful tool for finding data races between
threads executing on different CPUs.

This can be enabled by enabling KCSAN in the kernel config, or by using the
GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later
needs a compiler change to allow -fsanitize=thread that KCSAN uses.

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22315
2019-11-21 11:22:08 +00:00
Alan Cox
48c22f0eb2 Until every possible root cause for an "invalid ASID" assertion failure is
resolved, assign every pmap a valid ASID when it is first initialized.
2019-11-20 16:32:13 +00:00
Alan Cox
6b71afbf7f Achieve two goals at once: (1) Avoid an unnecessary broadcast TLB
invalidation in reclaim_pv_chunk(). (2) Prevent an "invalid ASID" assertion
failure in reclaim_pv_chunk().  The detailed explanation for this change is
provided by r354792.

X-MFC with:	r354792
2019-11-19 19:05:05 +00:00
Andrew Turner
5008941522 Return 0 from ptrace_set_pc as it now completes successfully.
Sponsored by:	DARPA, AFRL
2019-11-19 13:28:59 +00:00
Andrew Turner
5d1349914a Allow ptrace to set the probram counter on arm64.
Sponsored by:	DARPA, AFRL
2019-11-19 13:25:46 +00:00
Andrew Turner
a2bb7f7a62 Fix the definition of bus_space_read_stream_8 on arm64.
This is currently unused, however will be when the Kernel Concurrency
Sanitizer (KCSAN) is imported from NetBSD.

Sponsored by:	DARPA, AFRL
2019-11-19 10:57:44 +00:00
Kyle Evans
f22a592111 Convert in-tree sysent targets to use new makesyscalls.lua
flua is bootstrapped as part of the build for those on older
versions/revisions that don't yet have flua installed. Once upgraded past
r354833, "make sysent" will again naturally work as expected.

Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D21894
2019-11-18 23:28:23 +00:00
John Baldwin
03b0d68c72 Check for errors from copyout() and suword*() in sv_copyout_args/strings.
Reviewed by:	brooks, kib
Tested on:	amd64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22401
2019-11-18 20:07:43 +00:00
Mark Johnston
85e06c728c Set MALLOC_DEBUG_MAXZONES=1 in GENERIC-NODEBUG configurations.
The purpose of this option is to make it easier to track down memory
corruption bugs by reducing the number of malloc(9) types that might
have recently been associated with a given chunk of memory.  However, it
increases fragmentation and is disabled in release kernels.

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2019-11-18 20:03:28 +00:00