Commit Graph

1126 Commits

Author SHA1 Message Date
Emmanuel Vadot
29ee738052 allwinner: Remove a10_ehci driver
We have generic-ehci since r353062 so use it.

MFC after:	1 month
X-MFC-With:	r353062
2019-10-03 18:58:15 +00:00
Emmanuel Vadot
361a394828 arm64: rockchip: rk805: Switch to iicdev_{readfrom,writeto}
This simpify the code a bit.
2019-10-01 18:30:06 +00:00
Alan Cox
ff7a11a2c7 In short, pmap_enter_quick_locked("user space", ..., VM_PROT_READ) doesn't
work.  More precisely, it doesn't set ATTR_AP(ATTR_AP_USER) in the page
table entry, so any attempt to read from the mapped page by user space
generates a page fault.  This problem has gone unnoticed because the page
fault handler, vm_fault(), will ultimately call pmap_enter(), which
replaces the non-working page table entry with one that has
ATTR_AP(ATTR_AP_USER) set.

This change reduces the number of page faults during a "buildworld" by
about 19.4%.

Reviewed by:	andrew, markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21841
2019-10-01 15:33:47 +00:00
Emmanuel Vadot
6e5eac8cc0 arm64: rockchip: rk_clk_pll: Check mode on recalc
If the pll is in slow or deep slow mode return the correct frequency.
2019-09-30 15:01:09 +00:00
Emmanuel Vadot
d46c04051d arm64: rockchip: correct reset value
If bit is 0 the reset is not asserted.
Also register our self as a reset provider, this was commented
in r352850

Reported by:	mmel
2019-09-30 15:00:22 +00:00
Emmanuel Vadot
c3e25952b2 arm64: rockchip: Add usb2phy driver
This driver is for the usb phy present on rockchip SoC.
It only support RK3399 and host mode for now.
The driver expose the usb clock needed by the usb controller.
2019-09-28 22:25:21 +00:00
Emmanuel Vadot
2a5509bf25 arm64: rockchip: Fix map_gpio
The map_gpio function wasn't correct, the first element is the pin
and not the phandle.
2019-09-28 22:21:16 +00:00
Emmanuel Vadot
78a2a6b613 arm64: rockchip: Implement resets
Module resets where not implemented when rockchip clocks were commited.
Implement them.
Since all resets registers are contiguous a driver only need to give
the start offset and the number of resets. This avoid to have to declare
every resets.
2019-09-28 22:19:52 +00:00
Emmanuel Vadot
4b1d162d96 arm64: rockchip: rk3399: Add usb2 clocks 2019-09-28 22:17:26 +00:00
Alan Cox
0432618600 Eliminate redundant calls to critical_enter() and critical_exit() from
pmap_update_entry().  It suffices that interrupts are blocked.

Reviewed by:	andrew, markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21753
2019-09-28 17:16:03 +00:00
Mark Johnston
d4586dd328 Implement pmap_page_is_mapped() correctly on arm64 and riscv.
We must also check for large mappings.  pmap_page_is_mapped() is
mostly used in assertions, so the problem was not very noticeable.

Reviewed by:	alc
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21824
2019-09-27 23:37:01 +00:00
Konstantin Belousov
df08823d07 Improve MD page fault handlers.
Centralize calculation of signal and ucode delivered on unhandled page
fault in new function vm_fault_trap().  MD trap_pfault() now almost
always uses the signal numbers and error codes calculated in
consistent MI way.

This introduces the protection fault compatibility sysctls to all
non-x86 architectures which did not have that bug, but apparently they
were already much more wrong in selecting delivered signals on
protection violations.

Change the delivered signal for accesses to mapped area after the
backing object was truncated.  According to POSIX description for
mmap(2):
   The system shall always zero-fill any partial page at the end of an
   object. Further, the system shall never write out any modified
   portions of the last page of an object which are beyond its
   end. References within the address range starting at pa and
   continuing for len bytes to whole pages following the end of an
   object shall result in delivery of a SIGBUS signal.

   An implementation may generate SIGBUS signals when a reference
   would cause an error in the mapped object, such as out-of-space
   condition.
Adjust according to the description, keeping the existing
compatibility code for SIGSEGV/SIGBUS on protection failures.

For situations where kernel cannot handle page fault due to resource
limit enforcement, SIGBUS with a new error code BUS_OBJERR is
delivered.  Also, provide a new error code SEGV_PKUERR for SIGSEGV on
amd64 due to protection key access violation.

vm_fault_hold() is renamed to vm_fault().  Fixed some nits in
trap_pfault()s like mis-interpreting Mach errors as errnos.  Removed
unneeded truncations of the fault addresses reported by hardware.

PR:	211924
Reviewed by:	alc
Discussed with:	jilles, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21566
2019-09-27 18:43:36 +00:00
Kyle Evans
d19f028e33 sysent: regenerate after r352693 2019-09-25 17:30:28 +00:00
Mark Johnston
b119329d81 Complete the removal of the "wire_count" field from struct vm_page.
Convert all remaining references to that field to "ref_count" and update
comments accordingly.  No functional change intended.

Reviewed by:	alc, kib
Sponsored by:	Intel, Netflix
Differential Revision:	https://reviews.freebsd.org/D21768
2019-09-25 16:11:35 +00:00
Alan Cox
2b28ec59ac In case a translation fault on the kernel address space occurs from
within a critical section, we must perform a lock-free check on the
faulting address.

Reported by:	andrew
Reviewed by:	andrew, markj
X-MFC with:	r350579
Differential Revision:	https://reviews.freebsd.org/D21685
2019-09-21 19:51:57 +00:00
Mark Johnston
e8bcf6966b Revert r352406, which contained changes I didn't intend to commit. 2019-09-16 15:04:45 +00:00
Mark Johnston
41fd4b9422 Fix a couple of nits in r352110.
- Remove a dead variable from the amd64 pmap_extract_and_hold().
- Fix grammar in the vm_page_wire man page.

Reported by:	alc
Reviewed by:	alc, kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D21639
2019-09-16 15:03:12 +00:00
Ruslan Bukin
ee41e38d42 Add support for Intel Stratix 10 platform.
Intel Stratix 10 SoC includes a quad-core arm64 cluster and FPGA fabric.

This adds support for reconfiguring FPGA.

Accessing FPGA core of this SoC require the level of privilege EL3,
while kernel runs in EL1 (lower) level of privilege.

This provides an Intel service layer interface that uses SMCCC to pass
queries to the secure-monitor (EL3).

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D21454
2019-09-13 16:50:57 +00:00
Ed Maste
a8ce8f0ae5 Update comments and ordering in linux*_dummy.c
- sort alphabetically
- getcpu arrived in Linux 2.6.19
- fanotify_* arrived in 2.6.36
2019-09-11 17:56:48 +00:00
Ed Maste
6626e12a62 linuxulator: add stub arm64 linux_genassym.c
This will be fleshed out in the future but allows us to build the arm64
linuxulator using the same infrastructure as x86.
2019-09-11 17:29:44 +00:00
Ed Maste
eff9d7b799 linuxulator: memfd_create first appeared in Linux 3.17
Reference: http://man7.org/linux/man-pages/man2/memfd_create.2.html
2019-09-11 17:05:49 +00:00
Ed Maste
1515fe49f2 linuxulator: seccomp syscall first appeared in Linux 3.17
Reference: http://man7.org/linux/man-pages/man2/seccomp.2.html
2019-09-11 17:04:13 +00:00
Ed Maste
2eb6ef203a linux: add trivial renameat2 implementation
Just return EINVAL if flags != 0.  The Linux man page documents one
case of EINVAL as "The filesystem does not support one of the flags in
flags."

After r351723 userland binaries will try using new system calls.

Reported by:	mjg
Reviewed by:	mjg, trasz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21590
2019-09-11 13:01:59 +00:00
Ed Maste
65ab1fdd21 regen linuxulator sysent after r352208 2019-09-11 12:58:53 +00:00
Ed Maste
427b1baec0 make linux_renameat2 args consistent with linux_renameat
Use 'dfd' consistently for a directory fd.
2019-09-11 12:58:06 +00:00
Mark Johnston
fee2a2fa39 Change synchonization rules for vm_page reference counting.
There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator.  In
particular, holding the page's object lock is sufficient to prevent the
page from being freed; holding the busy lock or a wiring is sufficent as
well.  These references are protected by the page lock, which must
therefore be acquired for many per-page operations.  This results in
false sharing since the page locks are external to the vm_page
structures themselves and each lock protects multiple structures.

Transition to using an atomically updated per-page reference counter.
The object's reference is counted using a flag bit in the counter.  A
second flag bit is used to atomically block new references via
pmap_extract_and_hold() while removing managed mappings of a page.
Thus, the reference count of a page is guaranteed not to increase if the
page is unbusied, unmapped, and the object's write lock is held.  As
a consequence of this, the page lock no longer protects a page's
identity; operations which move pages between objects are now
synchronized solely by the objects' locks.

The vm_page_wire() and vm_page_unwire() KPIs are changed.  The former
requires that either the object lock or the busy lock is held.  The
latter no longer has a return value and may free the page if it releases
the last reference to that page.  vm_page_unwire_noq() behaves the same
as before; the caller is responsible for checking its return value and
freeing or enqueuing the page as appropriate.  vm_page_wire_mapped() is
introduced for use in pmap_extract_and_hold().  It fails if the page is
concurrently being unmapped, typically triggering a fallback to the
fault handler.  vm_page_wire() no longer requires the page lock and
vm_page_unwire() now internally acquires the page lock when releasing
the last wiring of a page (since the page lock still protects a page's
queue state).  In particular, synchronization details are no longer
leaked into the caller.

The change excises the page lock from several frequently executed code
paths.  In particular, vm_object_terminate() no longer bounces between
page locks as it releases an object's pages, and direct I/O and
sendfile(SF_NOCACHE) completions no longer require the page lock.  In
these latter cases we now get linear scalability in the common scenario
where different threads are operating on different files.

__FreeBSD_version is bumped.  The DRM ports have been updated to
accomodate the KPI changes.

Reviewed by:	jeff (earlier version)
Tested by:	gallatin (earlier version), pho
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20486
2019-09-09 21:32:42 +00:00
Ruslan Bukin
50c365c49a Include dwgpio to the build.
Sponsored by:	DARPA, AFRL
2019-09-04 15:55:44 +00:00
Konstantin Belousov
a2a0f90654 Centralize __pcpu definitions.
Many extern struct pcpu <something>__pcpu declarations were
copied/pasted in sources.  The issue is that the definition is MD, but
it cannot be provided by machine/pcpu.h due to actual struct pcpu
defined in sys/pcpu.h later than the inclusion of machine/pcpu.h.
This forced the copying when other code needed direct access to
__pcpu.  There is no way around it, due to machine/pcpu.h supplying
part of struct pcpu fields.

To work around the problem, add a new machine/pcpu_aux.h header, which
should fill any needed MD definitions after struct pcpu definition is
completed. This allows to remove copies of __pcpu spread around the
source.  Also on x86 it makes it possible to remove work arounds like
OFFSETOF_CURTHREAD or clang specific warnings supressions.

Reported and tested by:	lwhsu, bcran
Reviewed by:	imp, markj (previous version)
Discussed with:	jhb
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D21418
2019-08-29 07:25:27 +00:00
Emmanuel Vadot
0178a63ffe arm64: rk3399: pinctrl: Add gpio banks and fix iomux
Since r351187 the pinctrl driver need to know the gpio bank as it
directly attach the gpio driver to handle some setup that might
be present in the dts, add the gpio banks table for rk3399.
While here fix some IOMUX definition that prevented to boot
on RK3399 as pinctrl wasn't configured correctly.

Submitted by:	mmel (original version)
MFC after:	2 weeks
MFC With:	r351187
2019-08-27 18:00:01 +00:00
Emmanuel Vadot
8a27f35631 arm64: rk3328: pinctrl: Add gpio banks and fix iomux
Since r351187 the pinctrl driver need to know the gpio bank as it
directly attach the gpio driver to handle some setup that might
be present in the dts, add the gpio banks table for rk3328.
While here fix some IOMUX definition that prevented to boot
on RK3328 as pinctrl wasn't configured correctly.

Submitted by:	mmel (original version)
MFC after:	2 weeks
MFC With:	r351187
2019-08-27 17:59:09 +00:00
Michal Meloun
f69c06c303 Add support for RK3288 into existing RockChip drivers.
This patch ensures only minimal level of compatibility necessary to boot
on RK3288 based boards. GPIO and pinctrl interaction, missing in current
implementation, will be improved by own patch in the near future.

MFC after:	2 weeks
MFC with:	r351452
2019-08-27 09:20:01 +00:00
Michael Tuexen
7d143bd2df Identify eMAG CPU used in Ampere Computing systems.
Reviewed by:		emaste@
MFC after:		1 week
Differential Revision:	https://reviews.freebsd.org/D21314
2019-08-26 16:12:14 +00:00
Michal Meloun
38b7749a82 Improve rk_pinctrl driver:
- add support for 'output-low', 'output-high', 'output-low' and
  'output-enable' properties. These are use in RK3288 DT files
- add support for RK3288
- to reduce overall file size, use local macros for initialization
  of pinctrl description structures.

MFC after:	2 weeks
2019-08-18 09:19:33 +00:00
Michal Meloun
c6ed46a518 Improve rk_i2c driver:
- Properly handle IIC_M_NOSTOP and IIC_M_NOSTART flags.
- add polling mode, so driver can be used even if interrupts are not
  enabled (this is necessary for proper support of PMICs).
- add support for RK3288

MFC after:	2 weeks
2019-08-18 09:11:43 +00:00
Emmanuel Vadot
51154edc56 arm64: Add EspressoBin DTB to the build
This will compile the espressobin dts to a dtb file and this will be
install in /boot/dtb/marvell/ during installkernel.

MFC after:	1 week
2019-08-16 21:11:43 +00:00
Ed Maste
0760b4c242 aarch64: make pmap_change_attr public like on other platforms
Submitted by:	Greg V <greg@unrelenting.technology>
Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D20787
2019-08-16 18:21:31 +00:00
Jeff Roberson
2194393787 Move phys_avail definition into MI code. It is consumed in the MI layer and
doing so adds more flexibility with less redundant code.

Reviewed by:	jhb, markj, kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D21250
2019-08-16 00:45:14 +00:00
Warner Losh
09813007a5 ukbdmap.h rule was identical on all platforms, so move them into sys/conf/files.
This allows us to remove 'nodevice ukbd' from the arm64 NOTES file.
2019-08-12 23:25:14 +00:00
Warner Losh
c1ab04fce5 Floppy driver really only works on x86
Move the floppy driver to the x86 specific notes file.

Reviewed by: jhb, manu, jhibbits, emaste
Differential Revision: https://reviews.freebsd.org/D21208
2019-08-12 22:58:50 +00:00
Warner Losh
99e1c5ab38 Move sc out of the global file
x86 needs sc, as does sparc64. powerpc doesn't use it by default, but some old
powermac notebooks do not work with vt yet for reasons unknonw. Even so, I've
removed it from powerpc LINT. It's not in daily use there, and the intent is to
100% switch to vt now that it works for that platform to limit support burden.

All the other architectures omit some or all of the screen savers from their
lint config. Move them to the x86 NOTES files and remove the exclusions. This
reduces slightly the number of savers sparc64 compiles, but since they are in
GENERIC, the overage is adequate and if someone reaelly wants to sort them out
in sparc64 they can sweat the details and the testing.

Reviewed by: jhb (earlier version), manu (earlier version), jhibbits
Differential Revision: https://reviews.freebsd.org/D21233
2019-08-12 22:58:44 +00:00
Warner Losh
56abac3fc3 First pass at a LINT for arm64.
Create a rough and ready NOTES file from GENERIC, remove the duplication from
sys/conf/NOTES and add relevant no* directives to make this compile.

Reviewed by: jhb, manu (earlier versions that differed only in comments)
Differential Revision: https://reviews.freebsd.org/D21184
2019-08-08 17:55:19 +00:00
Warner Losh
83959d613a Make arm64 32-bit mode compile with COMPAT_43
The COMPAT_43 option isn't quite like the other compat options, and arm64 makes
attempts to support it in 64-bit mode. In 32-bit compat mode, however, two
syscall implementations that COMPAT_FREEBSD32 assumes will be there are
missing. Provide implementations for these: ofreebsd32_sigreturn (which we'll
never encounter, so implement it as nosys as is done in kern_sig.c) and
ofreebsd32_getpagesize, where we'll always return 4096 since that's the only
PAGE_SIZE we support, similar to how the ia32 implementation does things.

Reviewed by: manu@
Differential Revision: https://reviews.freebsd.org/D21192
2019-08-08 17:48:07 +00:00
Alan Cox
56e66ce802 Ordinarily, during a superpage promotion or demotion within a pmap, the
pmap's lock ensures that other operations on the pmap don't observe the
old mapping being broken before the new mapping is established.  However,
pmap_kextract() doesn't acquire the kernel pmap's lock, so it may observe
the broken mapping.  And, if it does, it returns an incorrect result.

This revision implements a lock-free solution to this problem in
pmap_update_entry() and pmap_kextract() because pmap_kextract() can't
acquire the kernel pmap's lock.

Reported by:	andrew, greg_unrelenting.technology
Reviewed by:	andrew, markj
Tested by:	greg_unrelenting.technology
X-MFC with:	r350579
Differential Revision:	https://reviews.freebsd.org/D21169
2019-08-08 06:26:34 +00:00
Alan Cox
061b729fd4 Enable superpage promotion within the kernel pmap.
Reviewed by:	markj
X-MFC after:	r350004
Differential Revision:	https://reviews.freebsd.org/D21149
2019-08-05 02:44:04 +00:00
Alan Cox
072a067fb8 Because of AArch64's weak memory consistency model, we need to include a
memory barrier between the stores for initializing a page table page and
the store for adding that page to the page table.  Otherwise, a page table
walk by another processor's MMU could see the page table page before it
sees the initialized entries.

Simplify pmap_growkernel().  In particular, eliminate an unnecessary TLB
invalidation.

Reviewed by:	andrew, markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21126
2019-08-02 22:36:42 +00:00
Mark Johnston
407b6a4e44 Use ATTR_DBM even when hardware dirty bit management is not enabled.
The ARMv8 reference manual only states that the bit is reserved in
this case; following Linux's example, use it instead of a
software-defined bit for the purpose of indicating that a managed
mapping is writable.

Reviewed by:	alc, andrew
MFC after:	r350004
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21121
2019-08-01 22:48:06 +00:00
Alan Cox
43ded0a321 In pmap_advise(), when we encounter a superpage mapping, we first demote the
mapping and then destroy one of the 4 KB page mappings so that there is a
potential trigger for repromotion.  Currently, we destroy the first 4 KB
page mapping that falls within the (current) superpage mapping or the
virtual address range [sva, eva).  However, I have found empirically that
destroying the last 4 KB mapping produces slightly better results,
specifically, more promotions and fewer failed promotion attempts.
Accordingly, this revision changes pmap_advise() to destroy the last 4 KB
page mapping.  It also replaces some nearby uses of boolean_t with bool.

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D21115
2019-07-31 05:38:39 +00:00
Ed Maste
305b9efefc linuxulator: rename linux_locore.s to .asm
It is assembled using "${CC} -x assembler-with-cpp", which by convention
(bsd.suffixes.mk) uses the .asm extension.

This is a portion of the review referenced below (D18344).  That review
also renamed linux_support.s to .S, but that is a functional change
(using the compiler's integrated assembler instead of as) and will be
revisited separately.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18344
2019-07-30 17:18:31 +00:00
Ruslan Bukin
f808f2ce3e Add support for the SD/MMC controller found in Terasic DE10-Pro
(an Intel Stratix 10 GX/SX FPGA Development Kit).

Set the bus speed manually due to lack of clock management support.

Sponsored by:	DARPA, AFRL
2019-07-30 12:51:14 +00:00
Mark Johnston
1587dc37fa Have arm64's pmap_fault() handle WnR faults on dirty PTEs.
If we take a WnR permission fault on a managed, writeable and dirty
PTE, simply return success without calling the main fault handler.  This
situation can occur if multiple threads simultaneously access a clean
writeable mapping and trigger WnR faults; losers of the race to mark the
PTE dirty would end up calling the main fault handler, which had no work
to do.

Reported by:	alc
Reviewed by:	alc
MFC with:	r350004
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21097
2019-07-29 21:21:53 +00:00
Mark Johnston
96e90e5f14 Remove an unneeded trunc_page() in pmap_fault().
Reported by:	alc
MFC with:	r350004
Sponsored by:	The FreeBSD Foundation
2019-07-29 20:31:28 +00:00
Ruslan Bukin
4be6714234 Add glue driver for Altera SOCFPGA Ethernet MAC (EMAC) found in
Terasic DE10-Pro (an Intel Stratix 10 GX/SX FPGA Development Kit).

The Altera EMAC is an instance of Synopsys DesignWare Gigabit MAC.

This driver sets correct clock range for MDIO interface on Intel Stratix 10
platform.

This is required due to lack of support for clock manager device for
this platform that could tell us the clock frequency value for ethernet
clock domain.

Sponsored by:	DARPA, AFRL
2019-07-29 16:32:23 +00:00
Alan Cox
b3d6ea5bc2 Implement pmap_advise(). (Without a working pmap_advise() implementation
madvise(MADV_DONTNEED) and madvise(MADV_FREE) are NOPs.)

Reviewed by:	markj
X-MFC after:	r350004
Differential Revision:	https://reviews.freebsd.org/D21062
2019-07-26 05:07:09 +00:00
Alan Cox
5d18382b72 Simplify the handling of superpages in pmap_clear_modify(). Specifically,
if a demotion succeeds, then all of the 4KB page mappings within the
superpage-sized region must be valid, so there is no point in testing the
validity of the 4KB page mapping that is going to be write protected.

Deindent the nearby code.

Reviewed by:	kib, markj
Tested by:	pho (amd64, i386)
X-MFC after:	r350004 (this change depends on arm64 dirty bit emulation)
Differential Revision:	https://reviews.freebsd.org/D21027
2019-07-25 22:02:55 +00:00
Andrew Turner
ac4e582795 As with r350241 use the new UL macro on the main register mask.
MFC after:	1 week
Sponsored by:	DARPA, AFRL
2019-07-23 14:52:46 +00:00
Andrew Turner
f31c5955e2 Ensure the arm64 ID register fields are 64 bit types.
Previously only some of the ID register fields were 64 bit. To allow
for a script to generate these mark them all 64 bit. To allow for their
use in assembly we need to use the UINT64_C macro via a new UL macro
to stop the lines from being too long.

MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D20977
2019-07-23 14:40:37 +00:00
Alan Cox
f606d835de With the introduction of software dirty bit emulation for managed mappings,
we should test ATTR_SW_DBM, not ATTR_AP_RW, to determine whether to set
PGA_WRITEABLE.  In effect, we are currently setting PGA_WRITEABLE based on
whether the dirty bit is preset, not whether the mapping is writeable.
Correct this mistake.

Reviewed by:	markj
X-MFC with:	r350004
Differential Revision:	https://reviews.freebsd.org/D21013
2019-07-21 17:00:19 +00:00
Alan Cox
1a4cb969d1 Introduce pmap_store(), and use it to replace pmap_load_store() in places
where the page table entry was previously invalid.  (Note that I did not
replace pmap_load_store() when it was followed by a TLB invalidation, even
if we are not using the return value from pmap_load_store().)

Correct an error in pmap_enter().  A test for determining when to set
PGA_WRITEABLE was always true, even if the mapping was read only.

In pmap_enter_l2(), when replacing an empty kernel page table page by a
superpage mapping, clear the old l2 entry and issue a TLB invalidation.  My
reading of the ARM architecture manual leads me to believe that the TLB
could hold an intermediate entry referencing the empty kernel page table
page even though it contains no valid mappings.

Replace a couple direct uses of atomic_clear_64() by the new
pmap_clear_bits().

In a couple comments, replace the term "paging-structure caches", which is
an Intel-specific term for the caches that hold intermediate entries in the
page table, with wording that is more consistent with the ARM architecture
manual.

Reviewed by:	markj
X-MFC after:	r350004
Differential Revision:	https://reviews.freebsd.org/D20998
2019-07-21 03:26:26 +00:00
Emmanuel Vadot
d5fdfa2c8a arm64: Implement HWCAP
Add HWCAP support for arm64.
defines are the same as in Linux and a userland program can use
elf_aux_info to get the data.
We only save the common denominator for all cores in case the
big and little cluster have different support (this is known to
exists even if we don't support those SoCs in FreeBSD)

Differential Revision:	https://reviews.freebsd.org/D17137
2019-07-20 14:29:11 +00:00
Ganbold Tsagaankhuu
ea01660f1c Add driver for Rockchip RK3399 eMMC PHY.
Tested on NanoPC-T4 board.

Reviewed by:	manu
Differential Revision:	https://reviews.freebsd.org/D20840
2019-07-20 02:03:31 +00:00
Andrew Turner
f1fbf9c3b1 Rename arm64 macros in preperation for a script to generate them.
I have a script to generate most of the ID_AA64* macros from the Arm
XML source [1]. In preperation for using this we need to clean up the
macros to be in line with what the script will generate. This is the
first step, rename the macros to follow the names in said XML.

[1] https://developer.arm.com/architectures/cpu-architecture/a-profile/exploration-tools

MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D20976
2019-07-18 13:58:04 +00:00
Mark Johnston
7af2abed6a Always use the software DBM bit for now.
r350004 added most of the machinery needed to support hardware DBM
management, but it did not intend to actually enable use of the hardware
DBM bit.

Reviewed by:	andrew
MFC with:	r350004
Sponsored by:	The FreeBSD Foundation
2019-07-16 15:41:09 +00:00
Mark Johnston
32e09b04ce Fix the arm64 page table entry attribute mask.
It did not include the DBM or contiguous bits.

Reported by:	andrew
Reviewed by:	andrew
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-07-16 15:38:01 +00:00
Mark Johnston
9da9cb48e9 Propagate attribute changes during demotion.
After r349117 and r349122, some mapping attribute changes do not trigger
superpage demotion. However, pmap_demote_l2() was not updated to ensure
that the replacement L3 entries carry any attribute changes that
occurred since promotion.

Reported and tested by:	manu
Reviewed by:	alc
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20965
2019-07-16 14:40:49 +00:00
John Baldwin
c18ca74916 Don't pass error from syscallenter() to syscallret().
syscallret() doesn't use error anymore.  Fix a few other places to permit
removing the return value from syscallenter() entirely.
- Remove a duplicated assertion from arm's syscall().
- Use td_errno for amd64_syscall_ret_flush_l1d.

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D2090
2019-07-15 21:25:16 +00:00
Mark Johnston
ca2cae0b4d Implement software access and dirty bit management for arm64.
Previously the arm64 pmap did no reference or modification tracking;
all mappings were treated as referenced and all read-write mappings
were treated as dirty.  This change implements software management
of these attributes.

Dirty bit management is implemented to emulate ARMv8.1's optional
hardware dirty bit modifier management, following a suggestion from alc.
In particular, a mapping with ATTR_SW_DBM set is logically writeable and
is dirty if the ATTR_AP_RW_BIT bit is clear.  Mappings with
ATTR_AP_RW_BIT set are write-protected, and a write access will trigger
a permission fault.  pmap_fault() handles permission faults for such
mappings and marks the page dirty by clearing ATTR_AP_RW_BIT, thus
mapping the page read-write.

Reviewed by:	alc
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20907
2019-07-15 17:13:32 +00:00
Alan Cox
f884dbbcfc Revert r349442, which was a workaround for bus errors caused by an errant
TLB entry.  Specifically, at the start of pmap_enter_quick_locked(), we
would sometimes have a TLB entry for an invalid PTE, and we would need to
issue a TLB invalidation before exiting pmap_enter_quick_locked().  However,
we should never have a TLB entry for an invalid PTE.  r349905 has addressed
the root cause of the problem, and so we no longer need this workaround.

X-MFC after:	r349905
2019-07-13 16:32:19 +00:00
Konstantin Belousov
30b3018d48 Provide protection against starvation of the ll/sc loops when accessing userpace.
Casueword(9) on ll/sc architectures must be prepared for userspace
constantly modifying the same cache line as containing the CAS word,
and not loop infinitely.  Otherwise, rogue userspace livelocks the
kernel.

To fix the issue, change casueword(9) interface to return new value 1
indicating that either comparision or store failed, instead of relying
on the oldval == *oldvalp comparison.  The primitive no longer retries
the operation if it failed spuriously.  Modify callers of
casueword(9), all in kern_umtx.c, to handle retries, and react to
stops and requests to terminate between retries.

On x86, despite cmpxchg should not return spurious failures, we can
take advantage of the new interface and just return PSL.ZF.

Reviewed by:	andrew (arm64, previous version), markj
Tested by:	pho
Reported by:	https://xenbits.xen.org/xsa/advisory-295.txt
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D20772
2019-07-12 18:43:24 +00:00
Mark Johnston
194a6e146d Apply some light cleanup to uses of pmap_pte_dirty().
- Check for ATTR_SW_MANAGED before anything else.
- Use pmap_pte_dirty() in pmap_remove_pages().

No functional change intended.

Reviewed by:	alc
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-07-12 15:24:25 +00:00
Mark Johnston
a9da8477af Fix some ISS bit definitions for data aborts.
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-07-11 15:36:59 +00:00
Alan Cox
46a7f2ebd4 According to Section D5.10.3 "Maintenance requirements on changing System
register values" of the architecture manual, an isb instruction should be
executed after updating ttbr0_el1 and before invalidating the TLB.  The
lack of this instruction in pmap_activate() appears to be the reason why
andrew@ and I have observed an unexpected TLB entry for an invalid PTE on
entry to pmap_enter_quick_locked().  Thus, we should now be able to revert
the workaround committed in r349442.

Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20904
2019-07-11 02:43:23 +00:00
Mark Johnston
f84a04c8bc Rename pmap_page_dirty() to pmap_pte_dirty().
This is a precursor to implementing dirty bit management.

Discussed with:	alc
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-07-10 22:52:26 +00:00
Alan Cox
d1dde35baa Introduce pmap_clear(), which zeroes a page table entry, and use it, instead
of pmap_load_clear(), in places where we don't care about the page table
entry's prior contents.

Eliminate an unnecessary pmap_load() from pmap_remove_all().  Instead, use
the value returned by the pmap_load_clear() on the very next line.  (In the
future, when we support "hardware dirty bit management", using the value
from the pmap_load() rather than the pmap_load_clear() would have actually
been an error because the dirty bit could potentially change between the
pmap_load() and the pmap_load_clear().)

A KASSERT() in pmap_enter(), which originated in the amd64 pmap, was meant
to check the value returned by the pmap_load_clear() on the previous line.
However, we were ignoring the value returned by the pmap_load_clear(), and
so the KASSERT() was not serving its intended purpose.  Use the value
returned by the pmap_load_clear() in the KASSERT().

MFC after:	2 weeks
2019-07-09 20:28:53 +00:00
Mark Johnston
eeacb3b02f Merge the vm_page hold and wire mechanisms.
The hold_count and wire_count fields of struct vm_page are separate
reference counters with similar semantics.  The remaining essential
differences are that holds are not counted as a reference with respect
to LRU, and holds have an implicit free-on-last unhold semantic whereas
vm_page_unwire() callers must explicitly determine whether to free the
page once the last reference to the page is released.

This change removes the KPIs which directly manipulate hold_count.
Functions such as vm_fault_quick_hold_pages() now return wired pages
instead.  Since r328977 the overhead of maintaining LRU for wired pages
is lower, and in many cases vm_fault_quick_hold_pages() callers would
swap holds for wirings on the returned pages anyway, so with this change
we remove a number of page lock acquisitions.

No functional change is intended.  __FreeBSD_version is bumped.

Reviewed by:	alc, kib
Discussed with:	jeff
Discussed with:	jhb, np (cxgbe)
Tested by:	pho (previous version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D19247
2019-07-08 19:46:20 +00:00
Alan Cox
b9e19259d7 Three changes to pmap_enter():
1. Use _pmap_alloc_l3() instead of pmap_alloc_l3() in order to handle the
possibility that a superpage mapping for "va" was created while we slept.
(This is derived from the amd64 version.)

2. Eliminate code for allocating kernel page table pages.  Kernel page
table pages are preallocated by pmap_growkernel().

3. Eliminate duplicated unlock operations when KERN_RESOURCE_SHORTAGE is
returned.

MFC after:	2 weeks
2019-07-07 06:06:48 +00:00
Alan Cox
d8d0bf06a7 Restructure cache_handle_range to avoid repeated barriers. Specifically,
restructure cache_handle_range so that all of the data cache operations are
performed before any instruction cache operations.  Then, we only need one
barrier between the data and instruction cache operations and one barrier
after the instruction cache operations.

On an Amazon EC2 a1.2xlarge instance, this simple change reduces the time
for a "make -j8 buildworld" by 9%.

Reviewed by:	andrew
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20848
2019-07-05 20:01:06 +00:00
Ganbold Tsagaankhuu
40c30bf337 Subclass Rockchip's General Register Files driver from Simple MFD driver. 2019-07-03 03:42:51 +00:00
Alan Cox
3612c1f045 Implement pmap_copy(). (This includes the changes applied to the amd64
pmap_copy() in r349585.)

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D20790
2019-07-02 23:02:52 +00:00
Emmanuel Vadot
6c4395e3b5 arm64: efi: Map memory IO region as device
Reviewed by:	andrew
Sponsored by:	Ampere Computing, LLC
2019-07-01 22:11:56 +00:00
Navdeep Parhar
57f317e60a Display the approximate space needed when a minidump fails due to lack
of space.

Reviewed by:	kib@
MFC after:	2 weeks
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D20801
2019-06-30 03:14:04 +00:00
Olivier Houchard
84322e3ee3 In get_fpcontext32() and set_fpcontext32(), we can't just use memcpy() to
copy the VFP registers.
arvm7 VFP uses 32 64bits fp registers (but those could be used in pairs to
make 16 128bits registers), while aarch64 uses 32 128bits fp registers, so
we have to copy the value of each register.
2019-06-26 22:06:40 +00:00
Alan Cox
1d3423d914 Revert one of the changes from r349323. Specifically, undo the change
that replaced a pmap_invalidate_page() with a dsb(ishst) in
pmap_enter_quick_locked().  Even though this change is in principle
correct, I am seeing occasional, spurious bus errors that are only
reproducible without this pmap_invalidate_page().  (None of adding an
isb, "upgrading" the dsb to wait on loads as well as stores, or
disabling superpage mappings eliminates the bus errors.)  Add an XXX
comment explaining why the pmap_invalidate_page() is being performed.

Discussed with:	     andrew, markj
2019-06-26 21:43:41 +00:00
Olivier Houchard
b726d74fce Fix debugging of 32bits arm binaries on arm64.
In set_regs32()/fill_regs32(), we have to get/set SP and LR from/to
tf_x[13] and tf_x[14].
set_regs() and fill_regs() may be called for a 32bits process, if the process
is ptrace'd from a 64bits debugger. So, in set_regs() and fill_regs(), get
or set PC and SPSR from where the debugger expects it, from tf_x[15] and
tf_x[16].
2019-06-26 16:56:56 +00:00
Jayachandran C.
b0f79e328e arm64 acpi_iort: add some error handling
Print warnings for some bad kernel configurations (like NUMA disabled
with multiple domains). Check and report some firmware errors (like
incorrect proximity domain entries).

Differential Revision:	https://reviews.freebsd.org/D20416
2019-06-24 21:24:55 +00:00
Jayachandran C.
c66524f07d arm64 gicv3_its: enable all ITS blocks for a CPU
We now support multiple ITS blocks raising interrupts to a CPU.
Add all available CPUs to the ITS when no NUMA information is
available.

This reverts the check added in r340602, at that tim we did not
suppport multiple ITS blocks for a CPU.

Differential Revision:	https://reviews.freebsd.org/D20417
2019-06-24 21:13:45 +00:00
Jayachandran C.
893caf588a arm64 gic: Drop unused GICV3_IVAR_REDIST_VADDR
Now that GICV3_IVAR_REDIST is available, GICV3_IVAR_REDIST_VADDR
is unused and can be removed. Drop the define and add a comment.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D20454
2019-06-24 21:00:28 +00:00
Alan Cox
22c7bcb842 pmap_enter_quick_locked() never replaces a valid mapping, so it need not
perform a TLB invalidation.  A barrier suffices.  (See r343876.)

Add a comment to pmap_enter_quick_locked() in order to highlight the
fact that it does not replace valid mappings.

Correct a typo in one of pmap_enter()'s comments.

MFC after:	1 week
2019-06-23 21:06:56 +00:00
Alan Cox
36c5a4cb3f Introduce pmap_remove_l3_range() and use it in two places:
(1) pmap_remove(), where it eliminates redundant TLB invalidations by
pmap_remove() and pmap_remove_l3(), and (2) pmap_enter_l2(), where it may
optimize the TLB invalidations by batching them.

Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D12725
2019-06-22 16:26:38 +00:00
Conrad Meyer
c363b16c63 sys: Remove DEV_RANDOM device option
Remove 'device random' from kernel configurations that reference it (most).
Replace perhaps mistaken 'nodevice random' in two MIPS configs with 'options
RANDOM_LOADABLE' instead.  Document removal in UPDATING; update NOTES and
random.4.

Reviewed by:	delphij, markm (previous version)
Approved by:	secteam(delphij)
Differential Revision:	https://reviews.freebsd.org/D19918
2019-06-21 00:16:30 +00:00
Alan Cox
0b3c5f1bc9 Correct an error in r349122. pmap_unwire() should update the pmap's wired
count, not its resident count.

X-MFC with:	r349122
2019-06-19 03:33:00 +00:00
Alan Cox
29c25bd781 Eliminate a redundant call to pmap_invalidate_page() from
pmap_ts_referenced().

MFC after:	14 days
Differential Revision:	https://reviews.freebsd.org/D12725
2019-06-17 01:58:25 +00:00
Alan Cox
c8f59059d8 Three changes to arm64's pmap_unwire():
Implement wiring changes on superpage mappings.  Previously, a superpage
mapping was unconditionally demoted by pmap_unwire(), even if the wiring
change applied to the entire superpage mapping.

Rewrite a comment to use the arm64 names for bits in a page table entry.
Previously, the bits were referred to by their x86 names.

Use atomic_"op"_64() instead of atomic_"op"_long() to update a page table
entry in order to match the prevailing style in this file.

MFC after:	10 days
2019-06-16 22:13:27 +00:00
Alan Cox
bf13f9d279 Three enhancements to arm64's pmap_protect():
Implement protection changes on superpage mappings.  Previously, a superpage
mapping was unconditionally demoted by pmap_protect(), even if the
protection change applied to the entire superpage mapping.

Precompute the bit mask describing the protection changes rather than
recomputing it for every page table entry that is changed.

Skip page table entries that already have the requested protection changes
in place.

Reviewed by:	andrew, kib
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D20657
2019-06-16 16:45:01 +00:00
Alan Cox
3b97569c31 Previously, when pmap_remove_pages() destroyed a dirty superpage mapping,
it only called vm_page_dirty() on the first of the superpage's constituent
4KB pages.  This revision corrects that error, calling vm_page_dirty() on
all of superpage's constituent 4KB pages.

MFC after:	3 days
2019-06-15 17:26:42 +00:00
Julian Elischer
c749d68596 Lightly hide the 'var' inside the macros to read the arm special registers.
I just happenned to have 3rd party code using 'var' as the output variable
which drew my attention to this. variables defined inside macros should be
prefixed to avoid getting shadowed varable wanrings from clang.
2019-06-15 00:47:39 +00:00
Alan Cox
26066aef10 Batch the TLB invalidations that are performed by pmap_protect() rather
than performing them one at a time.

MFC after:	10 days
2019-06-14 22:06:43 +00:00
Alan Cox
aa4a1d3a55 Change the arm64 pmap so that updates to the global count of wired pages are
not performed directly by the pmap.  Instead, they are performed by
vm_page_free_pages_toq().  (This is the same approach that we use on x86.)

Reviewed by:	kib, markj
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D20627
2019-06-14 04:01:08 +00:00
Alan Cox
a04cd5cdee Change pmap_demote_l2_locked() so that it removes the superpage mapping on a
demotion failure.  Otherwise, some callers to pmap_demote_l2_locked(), such
as pmap_protect(), may leave an incorrect mapping in place on a demotion
failure.

Change pmap_demote_l2_locked() so that it handles addresses that are not
superpage aligned.  Some callers to pmap_demote_l2_locked(), such as
pmap_protect(), may not pass a superpage aligned address.

Change pmap_enter_l2() so that it correctly calls vm_page_free_pages_toq().
The arm64 pmap is updating the count of wired pages when freeing page table
pages, so pmap_enter_l2() should pass false to vm_page_free_pages_toq().

Optimize TLB invalidation in pmap_remove_l2().

Reviewed by:	kib, markj (an earlier version)
Discussed with:	andrew
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D20585
2019-06-12 20:38:49 +00:00
Bjoern A. Zeeb
8c9ca28b31 A bit of code hygiene (no functional changes).
Hide unused code under #ifdef notyet (in one case the only caller is under
that same ifdef), or if it is arm (not arm64) specific code under the
__arm__ ifdef to not yield -Wunused-function warnings during the arm64
kernel compile.

MFC after:	2 weeks
2019-06-10 23:25:40 +00:00
Luiz Otavio O Souza
8c62ce83cc Add the GPIO driver for the North/South bridge in Marvell Armada 37x0.
The A3700 has a different GPIO controller and thus, do not use the old (and
shared) code for Marvell.

The pinctrl driver, also part of the controller, is not supported yet (but
the implementation should be straightforward).

Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-06-10 21:27:21 +00:00
Alan Cox
fd2dae0a30 Implement an alternative solution to the amd64 and i386 pmap problem that we
previously addressed in r348246.

This pmap problem also exists on arm64 and riscv.  However, the original
solution developed for amd64 and i386 cannot be used on arm64 and riscv.  In
particular, arm64 and riscv do not define a PG_PROMOTED flag in their level
2 PTEs.  (A PG_PROMOTED flag makes no sense on arm64, where unlike x86 or
riscv we are required to break the old 4KB mappings before making the 2MB
mapping; and on riscv there are no unused bits in the PTE to define a
PG_PROMOTED flag.)

This commit implements an alternative solution that can be used on all four
architectures.  Moreover, this solution has two other advantages.  First, on
older AMD processors that required the Erratum 383 workaround, it is less
costly.  Specifically, it avoids unnecessary calls to pmap_fill_ptp() on a
superpage demotion.  Second, it enables the elimination of some calls to
pagezero() in pmap_kernel_remove_{l2,pde}().

In addition, remove a related stale comment from pmap_enter_{l2,pde}().

Reviewed by:	kib, markj (an earlier version)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20538
2019-06-09 03:36:10 +00:00
Tycho Nightingale
88e9fbe568 very large dma mappings can cause integer overflow
Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20505
2019-06-03 19:19:35 +00:00
Brooks Davis
4af6033324 makesyscalls.sh: always use absolute path for syscalls.conf
syscalls.conf is included using "." which per the Open Group:

 If file does not contain a <slash>, the shell shall use the search
 path specified by PATH to find the directory containing file.

POSIX shells don't fall back to the current working directory.

Submitted by:	Nathaniel Wesley Filardo <nwf20@cl.cam.ac.uk>
Reviewed by:	bdrewery
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D20476
2019-05-30 20:56:23 +00:00
Jayachandran C.
33e4f8169d arm64 gicv3_its: Fix a typo
Fix 'Cavium' spelling in errata description.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D20418
2019-05-30 01:39:07 +00:00
Jayachandran C.
55d9048856 gicv3_its: do LPI init only once per CPU
The initialization required for LPIs (setting up pending tables etc.)
has to be done just once per CPU, even in the case where there are
multiple ITS blocks associated with the CPU.

Add a flag lpi_enabled in the per-cpu distributor info for this and
use it to ensure that we call its_init_cpu_lpi() just once.

This enables us to support platforms where multiple GIC ITS blocks
can generate LPIs to a CPU.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19844
2019-05-30 01:32:00 +00:00
Jayachandran C.
db359ad3d6 gicv3_its: refactor LPI init into a new function
Move the per-cpu LPI intialization to a separate function. This is
in preparation for a commit that does LPI init only once for a CPU,
even when there are multiple ITS blocks associated with the CPU.

No functional changes in this commit.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19843
2019-05-30 01:24:47 +00:00
Jayachandran C.
a9b702ddde gic_v3: consolidate per-cpu redistributor information
Update 'struct gic_redists' to consolidate all per-cpu redistributor
information into a new 'struct redist_pcpu'. Provide a new interface
(GICV3_IVAR_REDIST) for the GIC driver, which can be used to retrieve
the per-cpu data.

This per-cpu redistributor struct will be later used to improve the
GIC ITS setup.

While there, remove some unused fields in gic_v3_var.h interface.
No functional changes.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19842
2019-05-30 01:21:08 +00:00
Jayachandran C.
87820437f9 arm64 nexus: remove incorrect warning
acpi_config_intr() will be called when an arm64 system booted with ACPI.
We do the interrupt mapping for ACPI interrupts in nexus_acpi_map_intr()
on arm64, so acpi_config_intr() has to just return success without
printing this error message.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19432
2019-05-26 23:04:21 +00:00
Conrad Meyer
daec92844e Include ktr.h in more compilation units
Similar to r348026, exhaustive search for uses of CTRn() and cross reference
ktr.h includes.  Where it was obvious that an OS compat header of some kind
included ktr.h indirectly, .c files were left alone.  Some of these files
clearly got ktr.h via header pollution in some scenarios, or tinderbox would
not be passing prior to this revision, but go ahead and explicitly include it
in files using it anyway.

Like r348026, these CUs did not show up in tinderbox as missing the include.

Reported by:	peterj (arm64/mp_machdep.c)
X-MFC-With:	r347984
Sponsored by:	Dell EMC Isilon
2019-05-21 20:38:48 +00:00
Conrad Meyer
e2e050c8ef Extract eventfilter declarations to sys/_eventfilter.h
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions.  The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended).  Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed.  __FreeBSD_version has been bumped.
2019-05-20 00:38:23 +00:00
Doug Moore
60645781d6 Implement the ffs and fls functions, and their longer counterparts, in
cpufunc, in terms of __builtin_ffs and the like, for arm64
architectures, and use those, rather than the simple libkern
implementations, in building arm64 kernels.

Tested by: greg_unrelenting.technology (earlier version)
Reviewed by: alc
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D20250
2019-05-17 15:52:17 +00:00
Konstantin Belousov
7c5a46a1bc Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.
In all practical situations, the resolver visibility is static.

Requested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	so (emaste)
Differential revision:	https://reviews.freebsd.org/D20281
2019-05-16 22:20:54 +00:00
Tycho Nightingale
b961c0f244 Allow loading the same DMA address multiple times without any prior
unload for the LinuxKPI.

Reviewed by:	kib, zeising
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20181
2019-05-16 17:41:16 +00:00
Justin Hibbits
2471860668 arm64: Add the rename interpreter path for compat32 ld-elf
Let arm64 and arm32 dynamic binaries coexist.  Match all other compat32
archs.

Reviewed by:	manu
Sponsored by:	Juniper Networks, Inc
2019-05-16 17:04:29 +00:00
Konstantin Belousov
98fc918f41 arm64: bzero buffer for ucontext in freebsd32_swapcontext().
This change is the same as r340994 for amd64.

PR:	237922
Submitted by:	Young <yangx92@hotmail.com>
MFC after:	3 days
2019-05-16 13:00:35 +00:00
Dmitry Chagin
c5156c7785 Linuxulator depends on a fundamental kernel settings such as SMP. Many
of them listed in opt_global.h which is not generated while building
modules outside of a kernel and such modules never match real cofigured
kernel.

So, we should prevent our users from building obviously defective modules.

Therefore, remove the root cause of the building of modules outside of a
kernel - the possibility of building modules with DEBUG or KTR flags.
And remove all of DEBUG printfs as it is incomplete and in threaded
programms not informative, also a half of system call does not have DEBUG
printf. For debuging Linux programms we have dtrace, ktr and ktrace ability.

PR:		222861
Reviewed by:	trasz
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20178
2019-05-13 18:24:29 +00:00
Emmanuel Vadot
595d5338fa arm64: rockchip: Don't always put PLL to normal mode
We used to put every PLL in normal mode (meaning that the output would
be the result of the PLL configuration) instead of slow mode (the output
is equal to the external oscillator frequency, 24-26Mhz) but this doesn't
work for most of the PLLs as when we put them into normal mode the registers
configuring the output frequency haven't been set.
Add a normal_mode member in clk_pll_def/clk_pll_sc struct and if it's true
we then set the PLL to normal mode.
For now only set it to the LPLL and BPLL (Little cluster PLL and Big cluster
PLL respectively).

Reviewed by:	ganbold
Differential Revision:	https://reviews.freebsd.org/D20174
2019-05-10 16:45:17 +00:00
Andrew Gallatin
542970fa2d Remove IPSEC from GENERIC due to performance issues
Having IPSEC compiled into the kernel imposes a non-trivial
performance penalty on multi-threaded workloads due to IPSEC
refcounting. In my benchmarks of multi-threaded UDP
transmit (connected sockets), I've seen a roughly 20% performance
penalty when the IPSEC option is included in the kernel (16.8Mpps
vs 13.8Mpps with 32 senders on a 14 core / 28 HTT Xeon
2697v3)). This is largely due to key_addref() incrementing and
decrementing an atomic reference count on the default
policy. This cause all CPUs to stall on the same cacheline, as it
bounces between different CPUs.

Given that relatively few users use ipsec, and that it can be
loaded as a module, it seems reasonable to ask those users to
load the ipsec module so as to avoid imposing this penalty on the
GENERIC kernel. Its my hope that this will make FreeBSD look
better in "out of the box" benchmark comparisons with other
operating systems.

Many thanks to ae for fixing auto-loading of ipsec.ko when
ifconfig tries to configure ipsec, and to cy for volunteering
to ensure the the racoon ports will load the ipsec.ko module

Reviewed by:	cem, cy, delphij, gnn, jhb, jpaetzel
Differential Revision:	https://reviews.freebsd.org/D20163
2019-05-09 22:38:15 +00:00
Emmanuel Vadot
052073c382 Add support for USB 3.0 XHCI via ACPI
Ampere eMAG systems have XHCI just described in ACPI, not on PCI.

Submitted by:	Greg V <greg@unrelenting.technology>
Reviewed by:	andrew
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D19986
2019-05-08 15:42:39 +00:00
Kyle Evans
251a32b5b2 tun/tap: merge and rename to tuntap
tun(4) and tap(4) share the same general management interface and have a lot
in common. Bugs exist in tap(4) that have been fixed in tun(4), and
vice-versa. Let's reduce the maintenance requirements by merging them
together and using flags to differentiate between the three interface types
(tun, tap, vmnet).

This fixes a couple of tap(4)/vmnet(4) issues right out of the gate:
- tap devices may no longer be destroyed while they're open [0]
- VIMAGE issues already addressed in tun by kp

[0] emaste had removed an easy-panic-button in r240938 due to devdrn
blocking. A naive glance over this leads me to believe that this isn't quite
complete -- destroy_devl will only block while executing d_* functions, but
doesn't block the device from being destroyed while a process has it open.
The latter is the intent of the condvar in tun, so this is "fixed" (for
certain definitions of the word -- it wasn't really broken in tap, it just
wasn't quite ideal).

ifconfig(8) also grew the ability to map an interface name to a kld, so
that `ifconfig {tun,tap}0` can continue to autoload the correct module, and
`ifconfig vmnet0 create` will now autoload the correct module. This is a
low overhead addition.

(MFC commentary)

This may get MFC'd if many bugs in tun(4)/tap(4) are discovered after this,
and how critical they are. Changes after this are likely easily MFC'd
without taking this merge, but the merge will be easier.

I have no plans to do this MFC as of now.

Reviewed by:	bcr (manpages), tuexen (testing, syzkaller/packetdrill)
Input also from:	melifaro
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D20044
2019-05-08 02:32:11 +00:00
Ed Maste
0e26cd440f make sysent after r347228
Regenerate to add @generated tag in generated files.
2019-05-07 18:10:21 +00:00
Ed Maste
e0bfdf599d Reformat arm64 linux syscalls.master per current style
Equivalent to r339958 for sys/kern/syscalls.master.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14858
2019-05-06 16:07:14 +00:00
Konstantin Belousov
ecaed009a9 arm64: Properly restore PAN when done with userspace access in casueword.
Approved by:	andrew
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-05-04 19:40:30 +00:00
Ganbold Tsagaankhuu
07be3e3dc9 Add emmc clock definitions for Rockchip RK3399 SoC. 2019-05-04 10:48:44 +00:00
Dmitry Chagin
d151344dbf In order to reduce duplication between MD parts of the Linuxulator
move bits that are MI out into the headers in compat/linux.
For that remove bogus _packed attribute from struct l_sockaddr
and use MI types for struct members.

And continue to move into the linux_common module a code that is
intended for both Linuxulator modules (both instruction set - 32 & 64 bit)
or for external modules like linsysfs or linprocfs.

To avoid header pollution introduce new sys/compat/linux_common.h header.

Reviewed by:	emaste
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20137
2019-05-03 08:42:49 +00:00
Conrad Meyer
d6745408c7 Add a COMPAT_FREEBSD12 kernel option.
Use it wherever COMPAT_FREEBSD11 is currently specified, like r309749.

Reviewed by:	imp, jhb, markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20120
2019-05-02 18:10:23 +00:00
Emmanuel Vadot
5b1309542e arm64: Add support for NanoPI NEO2
Add overlay files and activate devicetree file for NanoPi NEO2 featuring
Allwinner H5 ARM64 core.
To enable sound, dma and codec drivers are enabled for build.

Submitted by:	Manuel Stühn (freebsdnewbie@freenet.de)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20129
2019-05-02 12:56:13 +00:00
Andrew Turner
fa19730c61 Restore x18 in efi_arch_leave.
Some UEFI implementations trash this register and, as we use it as a
platform register, the kernel doesn't save it before calling into the UEFI
runtime services. As we have a copy in tpidr_el1 restore from there when
exiting the EFI environment.

PR:		237234, 237055
Reviewed by:	manu
Tested On:	Ampere eMAG
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Sponsored by:	Ampere Computing (hardware)
Differential Revision:	https://reviews.freebsd.org/D20127
2019-05-01 17:12:49 +00:00
Ganbold Tsagaankhuu
65f1fc3f3f Add a hw.model sysctl oid for arm64 which reports the CPU model similar to armv6/7.
Reviewed by:	andrew, manu
Differential Revision:	https://reviews.freebsd.org/D20123
2019-05-01 14:20:31 +00:00
Ganbold Tsagaankhuu
c4a350ffea Add SY8106A Buck Regulator and Allwinner CIR devices to GENERIC arm64 kernel. 2019-04-20 03:21:47 +00:00
Jung-uk Kim
278f0de60d Merge ACPICA 20190329. 2019-03-29 20:21:28 +00:00
Andrew Turner
40fcc4760e Sort printing of the ID registers on arm64 to be identical to the
documentation. This will simplify checking new fields when they are added.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2019-03-25 18:02:04 +00:00
Marcin Wojtas
e54926b254 Enable etherswitchcfg and e6000sw driver in arm64 build
After latest binding update, this patch enables usage of
the switch on Armada 3720 EspressoBin, so compile it
by default with arm64 GENERIC.

A patch was extracted from https://reviews.freebsd.org/D19036

Submitted by: Bert JW Regeer <xistence@0x58.com>
Reviewed by: manu
2019-03-23 02:53:47 +00:00
Colin Percival
4782b90e4a Add nvme support to the arm64 GENERIC kernel.
Submitted by:	Greg V
Differential Revision:	https://reviews.freebsd.org/D19657
2019-03-22 06:36:40 +00:00
Marcin Wojtas
8b0f6c4215 Add bus_release_resource() method to nexus on arm64
The nexus module was missing method for releasing bus resources. As a
result, it couldn't be released and the bus_release_resource() call would
return ENXIO.

Next call to bus_alloc_resource() for the same resource was returning
error, because it wasn't released previously and it was still busy.

The implementation of the nexus_release_resource() is the same as for
arm architecture.

Submitted by:  Michal Krawczyk <mk@semihalf.com>
Reported-by:   Greg V <greg@unrelenting.technology>
Tested-by:     cperciva, Greg V <greg@unrelenting.technology>
Obtained from: Semihalf
MFC after:     2 weeks
Sponsored by:  Amazon, Inc.
Differential revision: https://reviews.freebsd.org/D19641
2019-03-21 10:51:36 +00:00
Konstantin Belousov
fd8d844f76 amd64 KPTI: add control from procctl(2).
Add the infrastructure to allow MD procctl(2) commands, and use it to
introduce amd64 PTI control and reporting.  PTI mode cannot be
modified for existing pmap, the knob controls PTI of the new vmspace
created on exec.

Requested by:	jhb
Reviewed by:	jhb, markj (previous version)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:44:33 +00:00
Konstantin Belousov
6f1fe3305a amd64: Add md process flags and first P_MD_PTI flag.
PTI mode for the process pmap on exec is activated iff P_MD_PTI is set.

On exec, the existing vmspace can be reused only if pti mode of the
pmap matches the P_MD_PTI flag of the process.  Add MD
cpu_exec_vmspace_reuse() callback for exec_new_vmspace() which can
vetoed reuse of the existing vmspace.

MFC note: md_flags change struct proc KBI.

Reviewed by:	jhb, markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:31:01 +00:00
Edward Tomasz Napierala
1699546def Remove sv_pagesize, originally introduced with r100384.
In all of the architectures we have today, we always use PAGE_SIZE.
While in theory one could define different things, none of the
current architectures do, even the ones that have transitioned from
32-bit to 64-bit like i386 and arm. Some ancient mips binaries on
other systems used 8k instead of 4k, but we don't support running
those and likely never will due to their age and obscurity.

Reviewed by:	imp (who also contributed the commit message)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19280
2019-03-01 16:16:38 +00:00
Emmanuel Vadot
a5120cf00c arm64: rockchip: rk3399_pll: Fix the recalc function
The plls frequency are now correctly calculated in fractional mode
and integer mode.
While here add some debug printfs (disabled by default)
Tested with powerd on the little cluster on a RockPro64.

MFC after:	1 week
2019-03-01 13:05:37 +00:00
Andrew Turner
01a8235ea6 Add the hw.ncpu tunable to arm64.
This allows us to limit the number of CPUs to use, e.g. to debug problems
seen when enabling multiple clusters.

Reviewed by:	manu
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19404
2019-02-28 14:40:43 +00:00
Emmanuel Vadot
7032095101 xhci_mv: Move the driver to generic_xhci
Marvell XHCI is in fact generic-xhci, so move the driver and
add the compatible string.
While here, get and enable the phy if the dtb provide one.
The xhci bindings state that phys should be in a 'phys' property but
Marvell DTS uses 'usb-phy', only add support for 'usb-phy' for now.

Sponsored-by:      Rubicon Communications, LCC ("Netgate")
2019-02-27 21:04:40 +00:00
Emmanuel Vadot
722b978677 arm64: rockchip: rk3399_clk: Add sd clock definitions
MFC after:	1 week
2019-02-27 17:29:38 +00:00
Emmanuel Vadot
3a17f3007e arm64: rockchip: clk_pll: Multiple improvement
Remove the mode_val from the clock definition as it's a bit unreadable.
Use mode_shift to represent which bit control the mode in the register.
Simplify some case where we can avoid a register read before changing it.
Set the PLL back to normal mode after the PLL have stabilized.

Discussed with:	 mmel
MFC after:	1 week
2019-02-27 14:20:28 +00:00
Emmanuel Vadot
4e5222d1cc arm64: rockchip: rk3399_pll: Fix copy paste
RK3399 PLLs don't have mode_reg, use the correct register.

MFC after:	1 week
2019-02-26 17:20:03 +00:00
Emmanuel Vadot
2db22920c5 arm64: rockchip: rk3399_pll: Switch to slow mode when changing the freq
Like r344578 but for RK3399.
This solve some hangs when switching between frequency.

MFC after:	1 week
2019-02-26 17:08:51 +00:00
Emmanuel Vadot
38bf16ebe9 arm64: rockchip: rk_pinctrl: Fix two banks in RK3328
The last two banks don't have 3 bits for the pin function but only 2.
This fixes eMMC on the Rock64.

MFC after:	1 week
2019-02-26 15:29:16 +00:00
Emmanuel Vadot
c78444cd82 arm64: rockchip: rk805: Map the regulator
No map function was provided before so every regulator lookup resolved
the regulator with id 1, as it uses the default mapper, which is wrong.
Correctly map the regulators.
While here remove some debug printfs and make them disable by default.

MFC after:	1 week
2019-02-26 13:18:14 +00:00
Emmanuel Vadot
c99a9196f2 arm64: rockchip: rk805: Add LDO regulators
Add the 3 LDO regulator found in the RK805 Power Management IC.

MFC after:	1 week
2019-02-26 13:17:09 +00:00
Emmanuel Vadot
caecd45840 arm64: rockchip: rk3328_pll: Multiple improvement
RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here, when switching PLL frequency, first switch it to slow mode.
When set to slow mode the PLL clock will be the external oscillator.
Changing the PLL parameters while its output is used can cause hang (sometimes).

MFC after:	1 week
2019-02-26 13:16:05 +00:00
Emmanuel Vadot
dae337c2c2 arm64: rockchip: clk: ARM CLK improvement
RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here set the parent after changing its freqeuncy, this reduce the time
between changing the parent and changing the divider for the arm clock.

MFC after:	1 week
2019-02-26 13:15:31 +00:00
Emmanuel Vadot
554c9209e3 arm64: rockchip: clk: rk_clk_composite: Properly use the mask bits
RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here add some debug printf useful for debuging clock problems

MFC after:	1 week
2019-02-26 13:14:49 +00:00
Emmanuel Vadot
f4f53f2b4d arm64: rockchip: clk: Set the write mask when setting the clock mux
RockChip clocks have a write mask in the upper 16bits of the mux register
which wasn't set in the set_mux function.
Also the wrong parent was tested instead of the real current one, when
switch parent, test with the current one before.

Pointy Hat:    manu
MFC after:	1 week
2019-02-25 17:40:00 +00:00
Konstantin Belousov
e7a9df16e6 Add kernel support for Intel userspace protection keys feature on
Skylake Xeons.

See SDM rev. 68 Vol 3 4.6.2 Protection Keys and the description of the
RDPKRU and WRPKRU instructions.

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D18893
2019-02-20 09:51:13 +00:00
Konstantin Belousov
72091bb393 Enable enabling ASLR on non-x86 architectures.
Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation
2019-02-14 14:44:53 +00:00
Emmanuel Vadot
f6f8a42129 arm64: Fix compile when removing SOC_ROCKCHIP_* options
Make every rockchip file depend on the multiple soc_rockchip options
While here make rk_i2c and rk_gpio depend on their device options.

Reported by:	sbruno
2019-02-10 08:14:06 +00:00
Conrad Meyer
7e804fd5c5 Revert r343713 temporarily
The COVERAGE option breaks xtoolchain-gcc GENERIC kernel early boot
extremely badly and hasn't been fixed for the ~week since it was committed.
Please enable for GENERIC only when it doesn't do that.

Related fallout reported by:	lwhsu, tuexen (pr 235611)
2019-02-10 07:54:46 +00:00
Andrew Turner
4f33c38083 Add missing data barriers after storeing a new valid pagetable entry.
When moving from an invalid to a valid entry we don't need to invalidate
the tlb, however we do need to ensure the store is ordered before later
memory accesses. This is because this later access may be to a virtual
address within the newly mapped region.

Add the needed barriers to places where we don't later invalidate the
tlb. When we do invalidate the tlb there will be a barrier to correctly
order this.

This fixes a panic on boot on ThunderX2 when INVARIANTS is turned off:
panic: vm_fault_hold: fault on nofault entry, addr: 0xffff000040c11000

Reported by:	jchandra
Tested by:	jchandra
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19097
2019-02-07 20:58:45 +00:00
Andrew Turner
8308d2a251 Add a missing data barrier to the start of arm64_tlb_flushID.
We need to ensure the page table store has happened before the tlbi.

Reported by:	jchandra
Tested by:	jchandra
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19097
2019-02-07 20:50:39 +00:00
Jayachandran C.
73d8c81f38 arm64 gicv3: add IORT and NUMA support
acpi_iort.c has added support to query GIC proximity and MSI XREF
ID for GIC ITS blocks. Use this when GIC ITS blocks are initialized
from ACPI.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D18003
2019-02-07 03:01:54 +00:00
Jayachandran C.
9088a4751c arm64 acpi: Add support for IORT table
Add new file arm64/acpica/acpi_iort.c to support the "IO Remapping
Table" (IORT). The table is specified in ARM document "ARM DEN 0049D"
titled "IO Remapping Table Platform Design Document".  The IORT table
has information on the associations between PCI root complexes, SMMU
blocks and GIC ITS blocks in the system.

The changes are to parse and save the information in the IORT table.
The API to use this information is added to sys/dev/acpica/acpivar.h.

The acpi_iort.c also has code to check the GIC ITS nodes seen in the
IORT table with corresponding entries in MADT table (for validity)
and with entries in SRAT table (for proximity information).

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D18002
2019-02-07 02:30:33 +00:00
Andrew Turner
634a8a8873 Enable COVERAGE and KCOV by default on arm64 and amd64.
This allows userspace to trace the kernel using the coverage sanitizer
found in clang. It will also allow other coverage tools to be built as
modules and attach into the same framework.

Sponsored by:	DARPA, AFRL
2019-02-03 12:46:27 +00:00
Konstantin Belousov
c75f49f7d8 Make iflib a loadable module.
iflib is already a module, but it is unconditionally compiled into the
kernel.  There are drivers which do not need iflib(4), and there are
situations where somebody might not want iflib in kernel because of
using the corresponding driver as module.

Reviewed by:	marius
Discussed with:	erj
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D19041
2019-01-31 19:05:56 +00:00
David E. O'Brien
09efc56d66 Follow arm[32] and sparc64 KAPI and provide the FreeBSD standard spelling
across all architectures for this header.

Reviewed by:	stevek
Obtained from:	Juniper Networks
2019-01-29 20:10:27 +00:00
Andrew Turner
524553f56d Extract the coverage sanitizer KPI to a new file.
This will allow multiple consumers of the coverage data to be compiled
into the kernel together. The only requirement is only one can be
registered at a given point in time, however it is expected they will
only register when the coverage data is needed.

A new kernel conflig option COVERAGE is added. This will allow kcov to
become a module that can be loaded as needed, or compiled into the
kernel.

While here clean up the #include style a little.

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18955
2019-01-29 11:04:17 +00:00
Oleksandr Tymoshenko
00a4815fc6 [rpi] Reorganize spigen(4) overlays for Raspberry Pi
- Remove CS=2 entry from spigen-rpi2 since it didn't work
- Add spigen-rpi3 overlay for Raspberry Pi 3
- Enable rpi overlay modules for GENERIC kernel on aarch64

PR:		233489
Submitted by:	bobf@mrp3.com
Reviewed by:	db
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D16088
2019-01-16 01:08:34 +00:00
Andrew Turner
cd0c606fda Ensure the I-Cache is correctly handled in arm64_icache_sync_range
The cache_handle_range macro to handle the arm64 instruction and data
cache operations would return when it was complete. This causes problems
for arm64_icache_sync_range and arm64_icache_sync_range_checked as they
assume they can execute the i-cache handling instruction after it has been
called.

Fix this by making this assumption correct.

While here add missing instruction barriers and adjust the style to
match the rest of the assembly.

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18838
2019-01-15 09:48:18 +00:00
Olivier Houchard
e8d5909c39 Don't forget to add the needed #includes.
Pointy hat to:	cognet
2019-01-13 23:41:56 +00:00
Olivier Houchard
9cd27257d5 Introduce cpu_icache_sync_range_checked(), that does the same thing as
cpu_icache_sync_range(), except that it sets pcb_onfault to catch any page
fault, as doing cache maintenance operations for non-mapped generates a
data abort, and use it in freebsd32_sysarch(), so that a userland program
attempting to sync the icache with unmapped addresses doesn't crash the
kernel.

Spotted out by:	andrew
2019-01-13 23:29:46 +00:00
Olivier Houchard
8c9c3144cc Impleent COMPAT_FREEBSD32 for arm64.
This is based on early work by andrew@.
2019-01-13 19:49:46 +00:00
Andrew Turner
b3c0d957a2 Add support for the Clang Coverage Sanitizer in the kernel (KCOV).
When building with KCOV enabled the compiler will insert function calls
to probes allowing us to trace the execution of the kernel from userspace.
These probes are on function entry (trace-pc) and on comparison operations
(trace-cmp).

Userspace can enable the use of these probes on a single kernel thread with
an ioctl interface. It can allocate space for the probe with KIOSETBUFSIZE,
then mmap the allocated buffer and enable tracing with KIOENABLE, with the
trace mode being passed in as the int argument. When complete KIODISABLE
is used to disable tracing.

The first item in the buffer is the number of trace event that have
happened. Userspace can write 0 to this to reset the tracing, and is
expected to do so on first use.

The format of the buffer depends on the trace mode. When in PC tracing just
the return address of the probe is stored. Under comparison tracing the
comparison type, the two arguments, and the return address are traced. The
former method uses on entry per trace event, while the later uses 4. As
such they are incompatible so only a single mode may be enabled.

KCOV is expected to help fuzzing the kernel, and while in development has
already found a number of issues. It is required for the syzkaller system
call fuzzer [1]. Other kernel fuzzers could also make use of it, either
with the current interface, or by extending it with new modes.

A man page is currently being worked on and is expected to be committed
soon, however having the code in the kernel now is useful for other
developers to use.

[1] https://github.com/google/syzkaller

Submitted by:	Mitchell Horne <mhorne063@gmail.com> (Earlier version)
Reviewed by:	kib
Testing by:	tuexen
Sponsored by:	DARPA, AFRL
Sponsored by:	The FreeBSD Foundation (Mitchell Horne)
Differential Revision:	https://reviews.freebsd.org/D14599
2019-01-12 11:21:28 +00:00
Andrew Turner
80e21aabea Fix the location of td->td_frame at the top of the kernel stack.
In cpu_thread_alloc we would allocate space for the trap frame at the top of
the kernel stack. This is just below the pcb, however due to a missing cast
the pointer arithmetic would use the pcb size, not the trapframe size. As
the pcb is larger than the trapframe this is safe, however later in cpu_fork
we include the case leading to the two disagreeing on the location.

Fix by using the same arithmetic in both locations.

Found by:	An early KASAN patch
Sponsored by:	DARPA, AFRL
2019-01-11 11:32:46 +00:00
Fedor Uporov
6651cf410c Fix errno values returned from DUMMY_XATTR linuxulator calls
Reported by: weiss@uni-mainz.de
Reviewed by: markj
MFC after: 1 day
Differential Revision: https://reviews.freebsd.org/D18812
2019-01-11 07:58:25 +00:00
Andrew Turner
9c871ab54a Fix a comment, pushed onto is two words.
While here make the comments sentences.

Sponsored by:	DARPA, AFRL
2019-01-10 16:31:07 +00:00
Andrew Turner
0f72efaac7 Pass VM_PROT_EXECUTE to vm_fault for instruction faults.
We need to tell vm_fault the reason for the fault was because we tried to
execute from the memory location. Without this it may return with success
as we only request read-only memory, then we return to the same location
and try to execute from the same memory address. This leads to an infinite
loop raising the same fault and returning to the same invalid location.

MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18511
2018-12-27 14:14:41 +00:00
Mateusz Guzik
628888f0e0 Remove iBCS2, part2: general kernel
Reviewed by:	kib (previous version)
Sponsored by:	The FreeBSD Foundation
2018-12-19 21:57:58 +00:00
Emmanuel Vadot
2d5e81fbd0 mv_thermal: Add thermal driver for AP806 and CP110 thermal sensor
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2018-12-12 22:33:05 +00:00
Emmanuel Vadot
44d027bb5d arm64: Add mv_cp110_icu and mv_cp110_gicp
icu is a interrupt concentrator in the CP110 block and gicp
is a gic extension to allow interrupts in the CP block to be turned
into GIC SPI interrupts

Sponsored by:	Rubicon Communications, LLC ("Netgate")
2018-12-12 22:08:43 +00:00
Emmanuel Vadot
4f3a5b510b arm64: mv_gpio: Add Marvell 8K support
While here put the interrupts setup in it's own function

Sponsored by:	Rubicon Communications, LCC ("Netgate")
2018-12-12 22:02:57 +00:00
Emmanuel Vadot
2d2a085222 arm64: mvebu_pinctrl: Add driver for Marvell Pinmux Controller
Add a driver compatible with Marvell mvebu-pinctrl and add ap806-pinctrl
support.

Sponsored by:	Rubicon Communications, LCC ("Netgate")
2018-12-12 22:00:05 +00:00
Emmanuel Vadot
75a01d59f4 arm64: Add new SoC type MARVELL_8K
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2018-12-12 21:58:30 +00:00
Emmanuel Vadot
277a038d0d arm64: allwinner: Add pwm driver
Add a pwm driver for Allwinner PWM
Add pwm and aw_pwm to the GENERIC kernel
2018-12-12 20:58:43 +00:00
Andrew Turner
1242588828 Only read the ACPI proximity tabled on arm64 when we are booting from
ACPI.

Sponsored by:	DARPA, AFRL
2018-12-11 11:13:11 +00:00
Jayachandran C.
ec63066407 arm64: add ACPI based NUMA support
Use the newly defined SRAT/SLIT parsing APIs in arm64 to support
ACPI based NUMA.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D17943
2018-12-08 19:42:01 +00:00
Andreas Tobler
905bbe376c Build the dtb for the rock64 board.
Reviewed by:	manu@
2018-12-02 19:36:20 +00:00
Emmanuel Vadot
04f9b8a116 Add Silergy SYR827 PMIC driver
SYR827 is a PMIC that can output a voltage from 0.7125V to 1.5V in 12.5mV steps
It's controlled via I2C.

MFC after:	1 month
2018-12-01 20:31:49 +00:00
Emmanuel Vadot
affb46a826 arm64: rockchip: rk805: Add basic support for RK808 PMIC
RK808 PMIC is the companion chip for RK3399 SoC.
Add basic regulator support in RK805 since they are similar.

MFC after:	1 month
2018-12-01 20:31:05 +00:00
Emmanuel Vadot
1d47648a76 arm64: rockchip: rk_i2c: Use correct clock
While here add RK3399 support and call clk_set_assigned to set the correct
clock set in the DTS.

MFC after:	1 month
2018-12-01 20:29:42 +00:00
Emmanuel Vadot
36ae7efe61 arm64/rockchip: add RK3399 support
Add CRU (Clock and Reset Unit) driver for RK3399.
Add support in rk_pinctrl driver.

Submitted by:  Greg V <greg@unrelenting.technology> (Original version)
Differential Revision: https://reviews.freebsd.org/D16732

MFC after:	1 month
2018-12-01 20:28:16 +00:00
Emmanuel Vadot
c3aec23870 arm64: rockchip: Add RK3399_CLK_PLL
PLLs on the RK3399 are different than the ones on the RK3328.
Add a new type and some dedicated recalc and set_freq functions.
Rename the RK3328 dedicated rk_clk_pll function with rk3328_ prefix.

MFC after:	1 month
2018-12-01 20:26:59 +00:00
Eric van Gyzen
984969cd96 Fix reporting of SS_ONSTACK
Fix reporting of SS_ONSTACK in nested signal delivery when sigaltstack()
is used on some architectures.

Add a unit test for this.  I tested the test by introducing the bug
on amd64.  I did not test it on other architectures.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D18347
2018-11-30 22:44:33 +00:00
Emmanuel Vadot
fc06a872ec arm64: rockchip: armclk: Do not change parent freq if CLK_SET_DRYRUN is set
MFC after:	3 days
2018-11-29 19:11:35 +00:00
Emmanuel Vadot
006f540e44 arm64: Add rk_i2c and rk805 to GENERIC
MFC after:	1 month
2018-11-28 13:54:39 +00:00
Emmanuel Vadot
cee1927ea2 Add RK805 PMIC Support
RK805 is the companion PMIC for RK3328 SoC.
Add a driver for it with most of it's regulators supported.

MFC after:	1 month
2018-11-28 13:53:43 +00:00
Emmanuel Vadot
da3d2dce91 rk_clk_composite: Fix set_mux
Pointy Hat:	  manu
2018-11-28 13:25:22 +00:00
Eric van Gyzen
4d5a108409 Prevent kernel stack disclosure in signal delivery
On arm64 and riscv platforms, sendsig() failed to zero the signal
frame before copying it out to userspace.  Zero it.

On arm, I believe all the contents of the frame were initialized,
so there was no disclosure.  However, explicitly zero the whole frame
because that fact could inadvertently change in the future,
it's more clear to the reader, and I could be wrong in the first place.

MFC after:	2 days
Security:	similar to FreeBSD-EN-18:12.mem and CVE-2018-17155
Sponsored by:	Dell EMC Isilon
2018-11-26 20:52:53 +00:00
Emmanuel Vadot
6bff96c117 arm64: Add evdev support to GENERIC 2018-11-26 17:59:25 +00:00
Mark Johnston
cafcdfd00f Handle kernel superpage mappings in pmap_remove_l2().
PR:		233088
Reviewed by:	alc, andrew, kib
Tested by:	sbruno
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17981
2018-11-20 15:12:37 +00:00
Jayachandran C.
be7af100ad gitv3_its: fixes for multiple GIC ITS blocks
First pass of support for multiple GIC ITS blocks with ACPI.
Changes are to:
 * register the correct subset of interrupts with pic_register
   in case of ACPI.
 * initialize just the cpu interface for the first ITS, when
   domain information is not avialable. This has to be done
   until we split the per-CPU init to do LPI setup just once.
 * remove duplicate check for the GIC ITS domain, the sc_cpus
   are setup from domain, so the check again in per-CPU init
   seems unnecessary.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D17841
2018-11-19 03:52:56 +00:00
John Baldwin
c5e797a836 Drop the legacy ELF brandinfo for the old rtld from arm64 and riscv.
These architectures never shipped binaries with an rtld path of
/usr/libexec/ld-elf.so.1.

Reviewed by:	markj
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D17876
2018-11-07 18:28:55 +00:00
Andrew Turner
3869df5d71 Add the KUBSAN options to the arm64 and amd64 GENERIC kernel config files.
As the kernel file size may be too large to run with a stock loader comment
them out for now.

Sponsored by:	DARPA, AFRL
2018-11-06 17:47:58 +00:00