Commit Graph

975 Commits

Author SHA1 Message Date
br
a1a1b09414 Add glue driver for Altera SOCFPGA Ethernet MAC (EMAC) found in
Terasic DE10-Pro (an Intel Stratix 10 GX/SX FPGA Development Kit).

The Altera EMAC is an instance of Synopsys DesignWare Gigabit MAC.

This driver sets correct clock range for MDIO interface on Intel Stratix 10
platform.

This is required due to lack of support for clock manager device for
this platform that could tell us the clock frequency value for ethernet
clock domain.

Sponsored by:	DARPA, AFRL
2019-07-29 16:32:23 +00:00
alc
467944ce45 Implement pmap_advise(). (Without a working pmap_advise() implementation
madvise(MADV_DONTNEED) and madvise(MADV_FREE) are NOPs.)

Reviewed by:	markj
X-MFC after:	r350004
Differential Revision:	https://reviews.freebsd.org/D21062
2019-07-26 05:07:09 +00:00
alc
41dcc1dad1 Simplify the handling of superpages in pmap_clear_modify(). Specifically,
if a demotion succeeds, then all of the 4KB page mappings within the
superpage-sized region must be valid, so there is no point in testing the
validity of the 4KB page mapping that is going to be write protected.

Deindent the nearby code.

Reviewed by:	kib, markj
Tested by:	pho (amd64, i386)
X-MFC after:	r350004 (this change depends on arm64 dirty bit emulation)
Differential Revision:	https://reviews.freebsd.org/D21027
2019-07-25 22:02:55 +00:00
andrew
a68b656d72 As with r350241 use the new UL macro on the main register mask.
MFC after:	1 week
Sponsored by:	DARPA, AFRL
2019-07-23 14:52:46 +00:00
andrew
e52d3fea4b Ensure the arm64 ID register fields are 64 bit types.
Previously only some of the ID register fields were 64 bit. To allow
for a script to generate these mark them all 64 bit. To allow for their
use in assembly we need to use the UINT64_C macro via a new UL macro
to stop the lines from being too long.

MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D20977
2019-07-23 14:40:37 +00:00
alc
cfe8b54602 With the introduction of software dirty bit emulation for managed mappings,
we should test ATTR_SW_DBM, not ATTR_AP_RW, to determine whether to set
PGA_WRITEABLE.  In effect, we are currently setting PGA_WRITEABLE based on
whether the dirty bit is preset, not whether the mapping is writeable.
Correct this mistake.

Reviewed by:	markj
X-MFC with:	r350004
Differential Revision:	https://reviews.freebsd.org/D21013
2019-07-21 17:00:19 +00:00
alc
19eb42c206 Introduce pmap_store(), and use it to replace pmap_load_store() in places
where the page table entry was previously invalid.  (Note that I did not
replace pmap_load_store() when it was followed by a TLB invalidation, even
if we are not using the return value from pmap_load_store().)

Correct an error in pmap_enter().  A test for determining when to set
PGA_WRITEABLE was always true, even if the mapping was read only.

In pmap_enter_l2(), when replacing an empty kernel page table page by a
superpage mapping, clear the old l2 entry and issue a TLB invalidation.  My
reading of the ARM architecture manual leads me to believe that the TLB
could hold an intermediate entry referencing the empty kernel page table
page even though it contains no valid mappings.

Replace a couple direct uses of atomic_clear_64() by the new
pmap_clear_bits().

In a couple comments, replace the term "paging-structure caches", which is
an Intel-specific term for the caches that hold intermediate entries in the
page table, with wording that is more consistent with the ARM architecture
manual.

Reviewed by:	markj
X-MFC after:	r350004
Differential Revision:	https://reviews.freebsd.org/D20998
2019-07-21 03:26:26 +00:00
manu
8d6dc914b6 arm64: Implement HWCAP
Add HWCAP support for arm64.
defines are the same as in Linux and a userland program can use
elf_aux_info to get the data.
We only save the common denominator for all cores in case the
big and little cluster have different support (this is known to
exists even if we don't support those SoCs in FreeBSD)

Differential Revision:	https://reviews.freebsd.org/D17137
2019-07-20 14:29:11 +00:00
ganbold
3da75da272 Add driver for Rockchip RK3399 eMMC PHY.
Tested on NanoPC-T4 board.

Reviewed by:	manu
Differential Revision:	https://reviews.freebsd.org/D20840
2019-07-20 02:03:31 +00:00
andrew
f90826df9f Rename arm64 macros in preperation for a script to generate them.
I have a script to generate most of the ID_AA64* macros from the Arm
XML source [1]. In preperation for using this we need to clean up the
macros to be in line with what the script will generate. This is the
first step, rename the macros to follow the names in said XML.

[1] https://developer.arm.com/architectures/cpu-architecture/a-profile/exploration-tools

MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D20976
2019-07-18 13:58:04 +00:00
markj
07347a7f8f Always use the software DBM bit for now.
r350004 added most of the machinery needed to support hardware DBM
management, but it did not intend to actually enable use of the hardware
DBM bit.

Reviewed by:	andrew
MFC with:	r350004
Sponsored by:	The FreeBSD Foundation
2019-07-16 15:41:09 +00:00
markj
079d038a05 Fix the arm64 page table entry attribute mask.
It did not include the DBM or contiguous bits.

Reported by:	andrew
Reviewed by:	andrew
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-07-16 15:38:01 +00:00
markj
5f4e2bae6a Propagate attribute changes during demotion.
After r349117 and r349122, some mapping attribute changes do not trigger
superpage demotion. However, pmap_demote_l2() was not updated to ensure
that the replacement L3 entries carry any attribute changes that
occurred since promotion.

Reported and tested by:	manu
Reviewed by:	alc
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20965
2019-07-16 14:40:49 +00:00
jhb
895d57ec60 Don't pass error from syscallenter() to syscallret().
syscallret() doesn't use error anymore.  Fix a few other places to permit
removing the return value from syscallenter() entirely.
- Remove a duplicated assertion from arm's syscall().
- Use td_errno for amd64_syscall_ret_flush_l1d.

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D2090
2019-07-15 21:25:16 +00:00
markj
cdc0469c0e Implement software access and dirty bit management for arm64.
Previously the arm64 pmap did no reference or modification tracking;
all mappings were treated as referenced and all read-write mappings
were treated as dirty.  This change implements software management
of these attributes.

Dirty bit management is implemented to emulate ARMv8.1's optional
hardware dirty bit modifier management, following a suggestion from alc.
In particular, a mapping with ATTR_SW_DBM set is logically writeable and
is dirty if the ATTR_AP_RW_BIT bit is clear.  Mappings with
ATTR_AP_RW_BIT set are write-protected, and a write access will trigger
a permission fault.  pmap_fault() handles permission faults for such
mappings and marks the page dirty by clearing ATTR_AP_RW_BIT, thus
mapping the page read-write.

Reviewed by:	alc
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20907
2019-07-15 17:13:32 +00:00
alc
0b570c6d9c Revert r349442, which was a workaround for bus errors caused by an errant
TLB entry.  Specifically, at the start of pmap_enter_quick_locked(), we
would sometimes have a TLB entry for an invalid PTE, and we would need to
issue a TLB invalidation before exiting pmap_enter_quick_locked().  However,
we should never have a TLB entry for an invalid PTE.  r349905 has addressed
the root cause of the problem, and so we no longer need this workaround.

X-MFC after:	r349905
2019-07-13 16:32:19 +00:00
kib
ea314818c6 Provide protection against starvation of the ll/sc loops when accessing userpace.
Casueword(9) on ll/sc architectures must be prepared for userspace
constantly modifying the same cache line as containing the CAS word,
and not loop infinitely.  Otherwise, rogue userspace livelocks the
kernel.

To fix the issue, change casueword(9) interface to return new value 1
indicating that either comparision or store failed, instead of relying
on the oldval == *oldvalp comparison.  The primitive no longer retries
the operation if it failed spuriously.  Modify callers of
casueword(9), all in kern_umtx.c, to handle retries, and react to
stops and requests to terminate between retries.

On x86, despite cmpxchg should not return spurious failures, we can
take advantage of the new interface and just return PSL.ZF.

Reviewed by:	andrew (arm64, previous version), markj
Tested by:	pho
Reported by:	https://xenbits.xen.org/xsa/advisory-295.txt
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D20772
2019-07-12 18:43:24 +00:00
markj
e7f77dfbc0 Apply some light cleanup to uses of pmap_pte_dirty().
- Check for ATTR_SW_MANAGED before anything else.
- Use pmap_pte_dirty() in pmap_remove_pages().

No functional change intended.

Reviewed by:	alc
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-07-12 15:24:25 +00:00
markj
8ba13f1606 Fix some ISS bit definitions for data aborts.
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-07-11 15:36:59 +00:00
alc
f3e8efe2d4 According to Section D5.10.3 "Maintenance requirements on changing System
register values" of the architecture manual, an isb instruction should be
executed after updating ttbr0_el1 and before invalidating the TLB.  The
lack of this instruction in pmap_activate() appears to be the reason why
andrew@ and I have observed an unexpected TLB entry for an invalid PTE on
entry to pmap_enter_quick_locked().  Thus, we should now be able to revert
the workaround committed in r349442.

Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20904
2019-07-11 02:43:23 +00:00
markj
98074aa6dd Rename pmap_page_dirty() to pmap_pte_dirty().
This is a precursor to implementing dirty bit management.

Discussed with:	alc
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-07-10 22:52:26 +00:00
alc
6af2f0abb4 Introduce pmap_clear(), which zeroes a page table entry, and use it, instead
of pmap_load_clear(), in places where we don't care about the page table
entry's prior contents.

Eliminate an unnecessary pmap_load() from pmap_remove_all().  Instead, use
the value returned by the pmap_load_clear() on the very next line.  (In the
future, when we support "hardware dirty bit management", using the value
from the pmap_load() rather than the pmap_load_clear() would have actually
been an error because the dirty bit could potentially change between the
pmap_load() and the pmap_load_clear().)

A KASSERT() in pmap_enter(), which originated in the amd64 pmap, was meant
to check the value returned by the pmap_load_clear() on the previous line.
However, we were ignoring the value returned by the pmap_load_clear(), and
so the KASSERT() was not serving its intended purpose.  Use the value
returned by the pmap_load_clear() in the KASSERT().

MFC after:	2 weeks
2019-07-09 20:28:53 +00:00
markj
039f74039e Merge the vm_page hold and wire mechanisms.
The hold_count and wire_count fields of struct vm_page are separate
reference counters with similar semantics.  The remaining essential
differences are that holds are not counted as a reference with respect
to LRU, and holds have an implicit free-on-last unhold semantic whereas
vm_page_unwire() callers must explicitly determine whether to free the
page once the last reference to the page is released.

This change removes the KPIs which directly manipulate hold_count.
Functions such as vm_fault_quick_hold_pages() now return wired pages
instead.  Since r328977 the overhead of maintaining LRU for wired pages
is lower, and in many cases vm_fault_quick_hold_pages() callers would
swap holds for wirings on the returned pages anyway, so with this change
we remove a number of page lock acquisitions.

No functional change is intended.  __FreeBSD_version is bumped.

Reviewed by:	alc, kib
Discussed with:	jeff
Discussed with:	jhb, np (cxgbe)
Tested by:	pho (previous version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D19247
2019-07-08 19:46:20 +00:00
alc
6b5e2b6223 Three changes to pmap_enter():
1. Use _pmap_alloc_l3() instead of pmap_alloc_l3() in order to handle the
possibility that a superpage mapping for "va" was created while we slept.
(This is derived from the amd64 version.)

2. Eliminate code for allocating kernel page table pages.  Kernel page
table pages are preallocated by pmap_growkernel().

3. Eliminate duplicated unlock operations when KERN_RESOURCE_SHORTAGE is
returned.

MFC after:	2 weeks
2019-07-07 06:06:48 +00:00
alc
1b381cfc3c Restructure cache_handle_range to avoid repeated barriers. Specifically,
restructure cache_handle_range so that all of the data cache operations are
performed before any instruction cache operations.  Then, we only need one
barrier between the data and instruction cache operations and one barrier
after the instruction cache operations.

On an Amazon EC2 a1.2xlarge instance, this simple change reduces the time
for a "make -j8 buildworld" by 9%.

Reviewed by:	andrew
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20848
2019-07-05 20:01:06 +00:00
ganbold
ee002f95a3 Subclass Rockchip's General Register Files driver from Simple MFD driver. 2019-07-03 03:42:51 +00:00
alc
b233a7ed74 Implement pmap_copy(). (This includes the changes applied to the amd64
pmap_copy() in r349585.)

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D20790
2019-07-02 23:02:52 +00:00
manu
0ee80ab0e2 arm64: efi: Map memory IO region as device
Reviewed by:	andrew
Sponsored by:	Ampere Computing, LLC
2019-07-01 22:11:56 +00:00
np
a33a56c7b5 Display the approximate space needed when a minidump fails due to lack
of space.

Reviewed by:	kib@
MFC after:	2 weeks
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D20801
2019-06-30 03:14:04 +00:00
cognet
f18057177b In get_fpcontext32() and set_fpcontext32(), we can't just use memcpy() to
copy the VFP registers.
arvm7 VFP uses 32 64bits fp registers (but those could be used in pairs to
make 16 128bits registers), while aarch64 uses 32 128bits fp registers, so
we have to copy the value of each register.
2019-06-26 22:06:40 +00:00
alc
8646c1fa6b Revert one of the changes from r349323. Specifically, undo the change
that replaced a pmap_invalidate_page() with a dsb(ishst) in
pmap_enter_quick_locked().  Even though this change is in principle
correct, I am seeing occasional, spurious bus errors that are only
reproducible without this pmap_invalidate_page().  (None of adding an
isb, "upgrading" the dsb to wait on loads as well as stores, or
disabling superpage mappings eliminates the bus errors.)  Add an XXX
comment explaining why the pmap_invalidate_page() is being performed.

Discussed with:	     andrew, markj
2019-06-26 21:43:41 +00:00
cognet
dff664a855 Fix debugging of 32bits arm binaries on arm64.
In set_regs32()/fill_regs32(), we have to get/set SP and LR from/to
tf_x[13] and tf_x[14].
set_regs() and fill_regs() may be called for a 32bits process, if the process
is ptrace'd from a 64bits debugger. So, in set_regs() and fill_regs(), get
or set PC and SPSR from where the debugger expects it, from tf_x[15] and
tf_x[16].
2019-06-26 16:56:56 +00:00
jchandra
d4ca47be39 arm64 acpi_iort: add some error handling
Print warnings for some bad kernel configurations (like NUMA disabled
with multiple domains). Check and report some firmware errors (like
incorrect proximity domain entries).

Differential Revision:	https://reviews.freebsd.org/D20416
2019-06-24 21:24:55 +00:00
jchandra
4dbc8bbbad arm64 gicv3_its: enable all ITS blocks for a CPU
We now support multiple ITS blocks raising interrupts to a CPU.
Add all available CPUs to the ITS when no NUMA information is
available.

This reverts the check added in r340602, at that tim we did not
suppport multiple ITS blocks for a CPU.

Differential Revision:	https://reviews.freebsd.org/D20417
2019-06-24 21:13:45 +00:00
jchandra
6dd56b2c13 arm64 gic: Drop unused GICV3_IVAR_REDIST_VADDR
Now that GICV3_IVAR_REDIST is available, GICV3_IVAR_REDIST_VADDR
is unused and can be removed. Drop the define and add a comment.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D20454
2019-06-24 21:00:28 +00:00
alc
7cd42a206e pmap_enter_quick_locked() never replaces a valid mapping, so it need not
perform a TLB invalidation.  A barrier suffices.  (See r343876.)

Add a comment to pmap_enter_quick_locked() in order to highlight the
fact that it does not replace valid mappings.

Correct a typo in one of pmap_enter()'s comments.

MFC after:	1 week
2019-06-23 21:06:56 +00:00
alc
f228ac131c Introduce pmap_remove_l3_range() and use it in two places:
(1) pmap_remove(), where it eliminates redundant TLB invalidations by
pmap_remove() and pmap_remove_l3(), and (2) pmap_enter_l2(), where it may
optimize the TLB invalidations by batching them.

Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D12725
2019-06-22 16:26:38 +00:00
cem
aae886fade sys: Remove DEV_RANDOM device option
Remove 'device random' from kernel configurations that reference it (most).
Replace perhaps mistaken 'nodevice random' in two MIPS configs with 'options
RANDOM_LOADABLE' instead.  Document removal in UPDATING; update NOTES and
random.4.

Reviewed by:	delphij, markm (previous version)
Approved by:	secteam(delphij)
Differential Revision:	https://reviews.freebsd.org/D19918
2019-06-21 00:16:30 +00:00
alc
4bc7fb5303 Correct an error in r349122. pmap_unwire() should update the pmap's wired
count, not its resident count.

X-MFC with:	r349122
2019-06-19 03:33:00 +00:00
alc
6c4c499cff Eliminate a redundant call to pmap_invalidate_page() from
pmap_ts_referenced().

MFC after:	14 days
Differential Revision:	https://reviews.freebsd.org/D12725
2019-06-17 01:58:25 +00:00
alc
dd738ba879 Three changes to arm64's pmap_unwire():
Implement wiring changes on superpage mappings.  Previously, a superpage
mapping was unconditionally demoted by pmap_unwire(), even if the wiring
change applied to the entire superpage mapping.

Rewrite a comment to use the arm64 names for bits in a page table entry.
Previously, the bits were referred to by their x86 names.

Use atomic_"op"_64() instead of atomic_"op"_long() to update a page table
entry in order to match the prevailing style in this file.

MFC after:	10 days
2019-06-16 22:13:27 +00:00
alc
e9b66164d9 Three enhancements to arm64's pmap_protect():
Implement protection changes on superpage mappings.  Previously, a superpage
mapping was unconditionally demoted by pmap_protect(), even if the
protection change applied to the entire superpage mapping.

Precompute the bit mask describing the protection changes rather than
recomputing it for every page table entry that is changed.

Skip page table entries that already have the requested protection changes
in place.

Reviewed by:	andrew, kib
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D20657
2019-06-16 16:45:01 +00:00
alc
1c176f7393 Previously, when pmap_remove_pages() destroyed a dirty superpage mapping,
it only called vm_page_dirty() on the first of the superpage's constituent
4KB pages.  This revision corrects that error, calling vm_page_dirty() on
all of superpage's constituent 4KB pages.

MFC after:	3 days
2019-06-15 17:26:42 +00:00
julian
a54ff7ccd8 Lightly hide the 'var' inside the macros to read the arm special registers.
I just happenned to have 3rd party code using 'var' as the output variable
which drew my attention to this. variables defined inside macros should be
prefixed to avoid getting shadowed varable wanrings from clang.
2019-06-15 00:47:39 +00:00
alc
5d89b521c5 Batch the TLB invalidations that are performed by pmap_protect() rather
than performing them one at a time.

MFC after:	10 days
2019-06-14 22:06:43 +00:00
alc
4fd6fe044c Change the arm64 pmap so that updates to the global count of wired pages are
not performed directly by the pmap.  Instead, they are performed by
vm_page_free_pages_toq().  (This is the same approach that we use on x86.)

Reviewed by:	kib, markj
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D20627
2019-06-14 04:01:08 +00:00
alc
18116a4713 Change pmap_demote_l2_locked() so that it removes the superpage mapping on a
demotion failure.  Otherwise, some callers to pmap_demote_l2_locked(), such
as pmap_protect(), may leave an incorrect mapping in place on a demotion
failure.

Change pmap_demote_l2_locked() so that it handles addresses that are not
superpage aligned.  Some callers to pmap_demote_l2_locked(), such as
pmap_protect(), may not pass a superpage aligned address.

Change pmap_enter_l2() so that it correctly calls vm_page_free_pages_toq().
The arm64 pmap is updating the count of wired pages when freeing page table
pages, so pmap_enter_l2() should pass false to vm_page_free_pages_toq().

Optimize TLB invalidation in pmap_remove_l2().

Reviewed by:	kib, markj (an earlier version)
Discussed with:	andrew
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D20585
2019-06-12 20:38:49 +00:00
bz
8797c3d838 A bit of code hygiene (no functional changes).
Hide unused code under #ifdef notyet (in one case the only caller is under
that same ifdef), or if it is arm (not arm64) specific code under the
__arm__ ifdef to not yield -Wunused-function warnings during the arm64
kernel compile.

MFC after:	2 weeks
2019-06-10 23:25:40 +00:00
loos
8b044e9dba Add the GPIO driver for the North/South bridge in Marvell Armada 37x0.
The A3700 has a different GPIO controller and thus, do not use the old (and
shared) code for Marvell.

The pinctrl driver, also part of the controller, is not supported yet (but
the implementation should be straightforward).

Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-06-10 21:27:21 +00:00
alc
0b597963c6 Implement an alternative solution to the amd64 and i386 pmap problem that we
previously addressed in r348246.

This pmap problem also exists on arm64 and riscv.  However, the original
solution developed for amd64 and i386 cannot be used on arm64 and riscv.  In
particular, arm64 and riscv do not define a PG_PROMOTED flag in their level
2 PTEs.  (A PG_PROMOTED flag makes no sense on arm64, where unlike x86 or
riscv we are required to break the old 4KB mappings before making the 2MB
mapping; and on riscv there are no unused bits in the PTE to define a
PG_PROMOTED flag.)

This commit implements an alternative solution that can be used on all four
architectures.  Moreover, this solution has two other advantages.  First, on
older AMD processors that required the Erratum 383 workaround, it is less
costly.  Specifically, it avoids unnecessary calls to pmap_fill_ptp() on a
superpage demotion.  Second, it enables the elimination of some calls to
pagezero() in pmap_kernel_remove_{l2,pde}().

In addition, remove a related stale comment from pmap_enter_{l2,pde}().

Reviewed by:	kib, markj (an earlier version)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20538
2019-06-09 03:36:10 +00:00
tychon
e0b11c4354 very large dma mappings can cause integer overflow
Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20505
2019-06-03 19:19:35 +00:00
brooks
786a385786 makesyscalls.sh: always use absolute path for syscalls.conf
syscalls.conf is included using "." which per the Open Group:

 If file does not contain a <slash>, the shell shall use the search
 path specified by PATH to find the directory containing file.

POSIX shells don't fall back to the current working directory.

Submitted by:	Nathaniel Wesley Filardo <nwf20@cl.cam.ac.uk>
Reviewed by:	bdrewery
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D20476
2019-05-30 20:56:23 +00:00
jchandra
ce38ee5193 arm64 gicv3_its: Fix a typo
Fix 'Cavium' spelling in errata description.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D20418
2019-05-30 01:39:07 +00:00
jchandra
03c5d90f7c gicv3_its: do LPI init only once per CPU
The initialization required for LPIs (setting up pending tables etc.)
has to be done just once per CPU, even in the case where there are
multiple ITS blocks associated with the CPU.

Add a flag lpi_enabled in the per-cpu distributor info for this and
use it to ensure that we call its_init_cpu_lpi() just once.

This enables us to support platforms where multiple GIC ITS blocks
can generate LPIs to a CPU.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19844
2019-05-30 01:32:00 +00:00
jchandra
21280ab5e7 gicv3_its: refactor LPI init into a new function
Move the per-cpu LPI intialization to a separate function. This is
in preparation for a commit that does LPI init only once for a CPU,
even when there are multiple ITS blocks associated with the CPU.

No functional changes in this commit.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19843
2019-05-30 01:24:47 +00:00
jchandra
281e5a5253 gic_v3: consolidate per-cpu redistributor information
Update 'struct gic_redists' to consolidate all per-cpu redistributor
information into a new 'struct redist_pcpu'. Provide a new interface
(GICV3_IVAR_REDIST) for the GIC driver, which can be used to retrieve
the per-cpu data.

This per-cpu redistributor struct will be later used to improve the
GIC ITS setup.

While there, remove some unused fields in gic_v3_var.h interface.
No functional changes.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19842
2019-05-30 01:21:08 +00:00
jchandra
fdfda4909e arm64 nexus: remove incorrect warning
acpi_config_intr() will be called when an arm64 system booted with ACPI.
We do the interrupt mapping for ACPI interrupts in nexus_acpi_map_intr()
on arm64, so acpi_config_intr() has to just return success without
printing this error message.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D19432
2019-05-26 23:04:21 +00:00
cem
3038f1af7b Include ktr.h in more compilation units
Similar to r348026, exhaustive search for uses of CTRn() and cross reference
ktr.h includes.  Where it was obvious that an OS compat header of some kind
included ktr.h indirectly, .c files were left alone.  Some of these files
clearly got ktr.h via header pollution in some scenarios, or tinderbox would
not be passing prior to this revision, but go ahead and explicitly include it
in files using it anyway.

Like r348026, these CUs did not show up in tinderbox as missing the include.

Reported by:	peterj (arm64/mp_machdep.c)
X-MFC-With:	r347984
Sponsored by:	Dell EMC Isilon
2019-05-21 20:38:48 +00:00
cem
250e158ddf Extract eventfilter declarations to sys/_eventfilter.h
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions.  The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended).  Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed.  __FreeBSD_version has been bumped.
2019-05-20 00:38:23 +00:00
dougm
3cfe83e1e0 Implement the ffs and fls functions, and their longer counterparts, in
cpufunc, in terms of __builtin_ffs and the like, for arm64
architectures, and use those, rather than the simple libkern
implementations, in building arm64 kernels.

Tested by: greg_unrelenting.technology (earlier version)
Reviewed by: alc
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D20250
2019-05-17 15:52:17 +00:00
kib
653b035c89 Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.
In all practical situations, the resolver visibility is static.

Requested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	so (emaste)
Differential revision:	https://reviews.freebsd.org/D20281
2019-05-16 22:20:54 +00:00
tychon
b267afaa6b Allow loading the same DMA address multiple times without any prior
unload for the LinuxKPI.

Reviewed by:	kib, zeising
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20181
2019-05-16 17:41:16 +00:00
jhibbits
cc98d74ced arm64: Add the rename interpreter path for compat32 ld-elf
Let arm64 and arm32 dynamic binaries coexist.  Match all other compat32
archs.

Reviewed by:	manu
Sponsored by:	Juniper Networks, Inc
2019-05-16 17:04:29 +00:00
kib
142b7761cc arm64: bzero buffer for ucontext in freebsd32_swapcontext().
This change is the same as r340994 for amd64.

PR:	237922
Submitted by:	Young <yangx92@hotmail.com>
MFC after:	3 days
2019-05-16 13:00:35 +00:00
dchagin
f7e99603a7 Linuxulator depends on a fundamental kernel settings such as SMP. Many
of them listed in opt_global.h which is not generated while building
modules outside of a kernel and such modules never match real cofigured
kernel.

So, we should prevent our users from building obviously defective modules.

Therefore, remove the root cause of the building of modules outside of a
kernel - the possibility of building modules with DEBUG or KTR flags.
And remove all of DEBUG printfs as it is incomplete and in threaded
programms not informative, also a half of system call does not have DEBUG
printf. For debuging Linux programms we have dtrace, ktr and ktrace ability.

PR:		222861
Reviewed by:	trasz
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20178
2019-05-13 18:24:29 +00:00
manu
2c4607867d arm64: rockchip: Don't always put PLL to normal mode
We used to put every PLL in normal mode (meaning that the output would
be the result of the PLL configuration) instead of slow mode (the output
is equal to the external oscillator frequency, 24-26Mhz) but this doesn't
work for most of the PLLs as when we put them into normal mode the registers
configuring the output frequency haven't been set.
Add a normal_mode member in clk_pll_def/clk_pll_sc struct and if it's true
we then set the PLL to normal mode.
For now only set it to the LPLL and BPLL (Little cluster PLL and Big cluster
PLL respectively).

Reviewed by:	ganbold
Differential Revision:	https://reviews.freebsd.org/D20174
2019-05-10 16:45:17 +00:00
gallatin
d0514c0542 Remove IPSEC from GENERIC due to performance issues
Having IPSEC compiled into the kernel imposes a non-trivial
performance penalty on multi-threaded workloads due to IPSEC
refcounting. In my benchmarks of multi-threaded UDP
transmit (connected sockets), I've seen a roughly 20% performance
penalty when the IPSEC option is included in the kernel (16.8Mpps
vs 13.8Mpps with 32 senders on a 14 core / 28 HTT Xeon
2697v3)). This is largely due to key_addref() incrementing and
decrementing an atomic reference count on the default
policy. This cause all CPUs to stall on the same cacheline, as it
bounces between different CPUs.

Given that relatively few users use ipsec, and that it can be
loaded as a module, it seems reasonable to ask those users to
load the ipsec module so as to avoid imposing this penalty on the
GENERIC kernel. Its my hope that this will make FreeBSD look
better in "out of the box" benchmark comparisons with other
operating systems.

Many thanks to ae for fixing auto-loading of ipsec.ko when
ifconfig tries to configure ipsec, and to cy for volunteering
to ensure the the racoon ports will load the ipsec.ko module

Reviewed by:	cem, cy, delphij, gnn, jhb, jpaetzel
Differential Revision:	https://reviews.freebsd.org/D20163
2019-05-09 22:38:15 +00:00
manu
9028c47957 Add support for USB 3.0 XHCI via ACPI
Ampere eMAG systems have XHCI just described in ACPI, not on PCI.

Submitted by:	Greg V <greg@unrelenting.technology>
Reviewed by:	andrew
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D19986
2019-05-08 15:42:39 +00:00
kevans
0f415eea65 tun/tap: merge and rename to tuntap
tun(4) and tap(4) share the same general management interface and have a lot
in common. Bugs exist in tap(4) that have been fixed in tun(4), and
vice-versa. Let's reduce the maintenance requirements by merging them
together and using flags to differentiate between the three interface types
(tun, tap, vmnet).

This fixes a couple of tap(4)/vmnet(4) issues right out of the gate:
- tap devices may no longer be destroyed while they're open [0]
- VIMAGE issues already addressed in tun by kp

[0] emaste had removed an easy-panic-button in r240938 due to devdrn
blocking. A naive glance over this leads me to believe that this isn't quite
complete -- destroy_devl will only block while executing d_* functions, but
doesn't block the device from being destroyed while a process has it open.
The latter is the intent of the condvar in tun, so this is "fixed" (for
certain definitions of the word -- it wasn't really broken in tap, it just
wasn't quite ideal).

ifconfig(8) also grew the ability to map an interface name to a kld, so
that `ifconfig {tun,tap}0` can continue to autoload the correct module, and
`ifconfig vmnet0 create` will now autoload the correct module. This is a
low overhead addition.

(MFC commentary)

This may get MFC'd if many bugs in tun(4)/tap(4) are discovered after this,
and how critical they are. Changes after this are likely easily MFC'd
without taking this merge, but the merge will be easier.

I have no plans to do this MFC as of now.

Reviewed by:	bcr (manpages), tuexen (testing, syzkaller/packetdrill)
Input also from:	melifaro
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D20044
2019-05-08 02:32:11 +00:00
emaste
206ba42431 make sysent after r347228
Regenerate to add @generated tag in generated files.
2019-05-07 18:10:21 +00:00
emaste
ef0f4aa09f Reformat arm64 linux syscalls.master per current style
Equivalent to r339958 for sys/kern/syscalls.master.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14858
2019-05-06 16:07:14 +00:00
kib
58510286a0 arm64: Properly restore PAN when done with userspace access in casueword.
Approved by:	andrew
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-05-04 19:40:30 +00:00
ganbold
e4bfebd8e9 Add emmc clock definitions for Rockchip RK3399 SoC. 2019-05-04 10:48:44 +00:00
dchagin
90cdaa0665 In order to reduce duplication between MD parts of the Linuxulator
move bits that are MI out into the headers in compat/linux.
For that remove bogus _packed attribute from struct l_sockaddr
and use MI types for struct members.

And continue to move into the linux_common module a code that is
intended for both Linuxulator modules (both instruction set - 32 & 64 bit)
or for external modules like linsysfs or linprocfs.

To avoid header pollution introduce new sys/compat/linux_common.h header.

Reviewed by:	emaste
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20137
2019-05-03 08:42:49 +00:00
cem
09c46e6d56 Add a COMPAT_FREEBSD12 kernel option.
Use it wherever COMPAT_FREEBSD11 is currently specified, like r309749.

Reviewed by:	imp, jhb, markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20120
2019-05-02 18:10:23 +00:00
manu
1f1b2ec3c5 arm64: Add support for NanoPI NEO2
Add overlay files and activate devicetree file for NanoPi NEO2 featuring
Allwinner H5 ARM64 core.
To enable sound, dma and codec drivers are enabled for build.

Submitted by:	Manuel Stühn (freebsdnewbie@freenet.de)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20129
2019-05-02 12:56:13 +00:00
andrew
b278d30ff0 Restore x18 in efi_arch_leave.
Some UEFI implementations trash this register and, as we use it as a
platform register, the kernel doesn't save it before calling into the UEFI
runtime services. As we have a copy in tpidr_el1 restore from there when
exiting the EFI environment.

PR:		237234, 237055
Reviewed by:	manu
Tested On:	Ampere eMAG
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Sponsored by:	Ampere Computing (hardware)
Differential Revision:	https://reviews.freebsd.org/D20127
2019-05-01 17:12:49 +00:00
ganbold
dbc342b6b4 Add a hw.model sysctl oid for arm64 which reports the CPU model similar to armv6/7.
Reviewed by:	andrew, manu
Differential Revision:	https://reviews.freebsd.org/D20123
2019-05-01 14:20:31 +00:00
ganbold
a7e393156a Add SY8106A Buck Regulator and Allwinner CIR devices to GENERIC arm64 kernel. 2019-04-20 03:21:47 +00:00
jkim
c30ced85d1 Merge ACPICA 20190329. 2019-03-29 20:21:28 +00:00
andrew
0c946c6523 Sort printing of the ID registers on arm64 to be identical to the
documentation. This will simplify checking new fields when they are added.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2019-03-25 18:02:04 +00:00
mw
d8bc0028e7 Enable etherswitchcfg and e6000sw driver in arm64 build
After latest binding update, this patch enables usage of
the switch on Armada 3720 EspressoBin, so compile it
by default with arm64 GENERIC.

A patch was extracted from https://reviews.freebsd.org/D19036

Submitted by: Bert JW Regeer <xistence@0x58.com>
Reviewed by: manu
2019-03-23 02:53:47 +00:00
cperciva
be4e05cbf3 Add nvme support to the arm64 GENERIC kernel.
Submitted by:	Greg V
Differential Revision:	https://reviews.freebsd.org/D19657
2019-03-22 06:36:40 +00:00
mw
6bc222605a Add bus_release_resource() method to nexus on arm64
The nexus module was missing method for releasing bus resources. As a
result, it couldn't be released and the bus_release_resource() call would
return ENXIO.

Next call to bus_alloc_resource() for the same resource was returning
error, because it wasn't released previously and it was still busy.

The implementation of the nexus_release_resource() is the same as for
arm architecture.

Submitted by:  Michal Krawczyk <mk@semihalf.com>
Reported-by:   Greg V <greg@unrelenting.technology>
Tested-by:     cperciva, Greg V <greg@unrelenting.technology>
Obtained from: Semihalf
MFC after:     2 weeks
Sponsored by:  Amazon, Inc.
Differential revision: https://reviews.freebsd.org/D19641
2019-03-21 10:51:36 +00:00
kib
71140c5be4 amd64 KPTI: add control from procctl(2).
Add the infrastructure to allow MD procctl(2) commands, and use it to
introduce amd64 PTI control and reporting.  PTI mode cannot be
modified for existing pmap, the knob controls PTI of the new vmspace
created on exec.

Requested by:	jhb
Reviewed by:	jhb, markj (previous version)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:44:33 +00:00
kib
9b5673f116 amd64: Add md process flags and first P_MD_PTI flag.
PTI mode for the process pmap on exec is activated iff P_MD_PTI is set.

On exec, the existing vmspace can be reused only if pti mode of the
pmap matches the P_MD_PTI flag of the process.  Add MD
cpu_exec_vmspace_reuse() callback for exec_new_vmspace() which can
vetoed reuse of the existing vmspace.

MFC note: md_flags change struct proc KBI.

Reviewed by:	jhb, markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:31:01 +00:00
trasz
1a536e4f24 Remove sv_pagesize, originally introduced with r100384.
In all of the architectures we have today, we always use PAGE_SIZE.
While in theory one could define different things, none of the
current architectures do, even the ones that have transitioned from
32-bit to 64-bit like i386 and arm. Some ancient mips binaries on
other systems used 8k instead of 4k, but we don't support running
those and likely never will due to their age and obscurity.

Reviewed by:	imp (who also contributed the commit message)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19280
2019-03-01 16:16:38 +00:00
manu
68a8109f87 arm64: rockchip: rk3399_pll: Fix the recalc function
The plls frequency are now correctly calculated in fractional mode
and integer mode.
While here add some debug printfs (disabled by default)
Tested with powerd on the little cluster on a RockPro64.

MFC after:	1 week
2019-03-01 13:05:37 +00:00
andrew
21d07a1dd3 Add the hw.ncpu tunable to arm64.
This allows us to limit the number of CPUs to use, e.g. to debug problems
seen when enabling multiple clusters.

Reviewed by:	manu
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19404
2019-02-28 14:40:43 +00:00
manu
ab251adb8b xhci_mv: Move the driver to generic_xhci
Marvell XHCI is in fact generic-xhci, so move the driver and
add the compatible string.
While here, get and enable the phy if the dtb provide one.
The xhci bindings state that phys should be in a 'phys' property but
Marvell DTS uses 'usb-phy', only add support for 'usb-phy' for now.

Sponsored-by:      Rubicon Communications, LCC ("Netgate")
2019-02-27 21:04:40 +00:00
manu
101bf68bc2 arm64: rockchip: rk3399_clk: Add sd clock definitions
MFC after:	1 week
2019-02-27 17:29:38 +00:00
manu
9a5226a11b arm64: rockchip: clk_pll: Multiple improvement
Remove the mode_val from the clock definition as it's a bit unreadable.
Use mode_shift to represent which bit control the mode in the register.
Simplify some case where we can avoid a register read before changing it.
Set the PLL back to normal mode after the PLL have stabilized.

Discussed with:	 mmel
MFC after:	1 week
2019-02-27 14:20:28 +00:00
manu
4d0588ce6e arm64: rockchip: rk3399_pll: Fix copy paste
RK3399 PLLs don't have mode_reg, use the correct register.

MFC after:	1 week
2019-02-26 17:20:03 +00:00
manu
2aca625774 arm64: rockchip: rk3399_pll: Switch to slow mode when changing the freq
Like r344578 but for RK3399.
This solve some hangs when switching between frequency.

MFC after:	1 week
2019-02-26 17:08:51 +00:00
manu
f5b0c58813 arm64: rockchip: rk_pinctrl: Fix two banks in RK3328
The last two banks don't have 3 bits for the pin function but only 2.
This fixes eMMC on the Rock64.

MFC after:	1 week
2019-02-26 15:29:16 +00:00
manu
8822e00f7d arm64: rockchip: rk805: Map the regulator
No map function was provided before so every regulator lookup resolved
the regulator with id 1, as it uses the default mapper, which is wrong.
Correctly map the regulators.
While here remove some debug printfs and make them disable by default.

MFC after:	1 week
2019-02-26 13:18:14 +00:00
manu
836eb97045 arm64: rockchip: rk805: Add LDO regulators
Add the 3 LDO regulator found in the RK805 Power Management IC.

MFC after:	1 week
2019-02-26 13:17:09 +00:00
manu
ed8d62c038 arm64: rockchip: rk3328_pll: Multiple improvement
RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here, when switching PLL frequency, first switch it to slow mode.
When set to slow mode the PLL clock will be the external oscillator.
Changing the PLL parameters while its output is used can cause hang (sometimes).

MFC after:	1 week
2019-02-26 13:16:05 +00:00
manu
5761a7675b arm64: rockchip: clk: ARM CLK improvement
RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here set the parent after changing its freqeuncy, this reduce the time
between changing the parent and changing the divider for the arm clock.

MFC after:	1 week
2019-02-26 13:15:31 +00:00
manu
c78c3e1e8a arm64: rockchip: clk: rk_clk_composite: Properly use the mask bits
RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here add some debug printf useful for debuging clock problems

MFC after:	1 week
2019-02-26 13:14:49 +00:00