Commit Graph

1383 Commits

Author SHA1 Message Date
Andrew Turner
c085d2ea97 Add DDB_CTF to the arm64 and riscv kernel configs
This allows DTrace fbt probes to find arguments.

Sponsored by:	Innovate UK
2020-08-05 11:54:51 +00:00
Alexander Motin
855e49f3b0 Add initial driver for ACPI Platform Error Interfaces.
APEI allows platform to report different kinds of errors to OS in several
ways.  We've found that Supermicro X10/X11 motherboards report PCIe errors
appearing on hot-unplug via this interface using NMI.  Without respective
driver it ended up in kernel panic without any additional information.

This driver introduces support for the APEI Generic Hardware Error Source
reporting via NMI, SCI or polling.  It decodes the reported errors and
either pass them to pci(4) for processing or just logs otherwise.  Errors
marked as fatal still end up in kernel panic, but some more informative.

When somebody get to native PCIe AER support implementation both of the
reporting mechanisms should get common error recovery code.  Since in our
case errors happen when the device is already gone, there is nothing to
recover, so the code just clears the error statuses, practically ignoring
the otherwise destructive NMIs in nicer way.

MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	iXsystems, Inc.
2020-07-27 21:19:41 +00:00
Alex Richardson
b798ef6490 Include TMPFS in all the GENERIC kernel configs
Being able to use tmpfs without kernel modules is very useful when building
small MFS_ROOT kernels without a real file system.
Including TMPFS also matches arm/GENERIC and the MIPS std.MALTA configs.

Compiling TMPFS only adds 4 .c files so this should not make much of a
difference to NO_MODULES build times (as we do for our minimal RISC-V
images).

Reviewed By: br (earlier version for riscv), brooks, emaste
Differential Revision: https://reviews.freebsd.org/D25317
2020-07-24 08:40:04 +00:00
John Baldwin
d7d14db9c5 Set si_trapno to the exception code from esr.
Reviewed by:	kib
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D25771
2020-07-23 21:40:03 +00:00
John Baldwin
a1119d08b9 Add missing space after switch.
Reviewed by:	br, emaste
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D25778
2020-07-22 22:51:14 +00:00
Edward Tomasz Napierala
3e9a214260 Regen after r363304.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2020-07-18 11:31:31 +00:00
Edward Tomasz Napierala
8d1d017175 Add a trivial linux(4) splice(2) implementation, which simply
returns EINVAL.  Fixes grep (grep-3.1-2build1).

PR:		kern/218699
Reported by:	avos
Reviewed by:	emaste
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25636
2020-07-18 11:28:40 +00:00
Ruslan Bukin
f3856e6881 Add acpi_iort_map_pci_smmuv3().
This new function allows us to find the SMMU instance assigned
for a particular PCI RID.

Reviewed by:	andrew
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D25687
2020-07-17 14:51:51 +00:00
Mark Johnston
e64080e79c Switch from SCTP to SCTP_SUPPORT in GENERIC configs.
This removes SCTP from in-tree kernel configuration files.  Now, SCTP
can be enabled by simply loading the module, as discussed on
freebsd-net@.

Reviewed by:	tuexen
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25611
2020-07-16 15:09:04 +00:00
Andrew Turner
a7f1b0cac9 Print the arm64 registers in more exception handling panics
It can be useful to get a dump of all registers when investigating why we
received an exception that we are unable to handle. In these cases we
already call panic, however we don't always print the registers.

Add calls to print_registers and print esr and far when applicable.

Sponsored by:	Innovate UK
2020-07-14 18:50:48 +00:00
Mark Johnston
b356ddf076 Add a driver for the SafeXcel EIP-97.
The EIP-97 is a packet processing module found on the ESPRESSObin.  This
commit adds a crypto(9) driver for the crypto and hash engine in this
device.  An initial skeleton driver that could attach and submit
requests was written by loos and others at Netgate, and the driver was
finished by me.

Support for separate AAD and output buffers will be added in a separate
commit, to simplify merging to stable/12 (where those features don't
exist).

Reviewed by:	gnn, jhb
Feedback from:	andrew, cem, manu
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D25417
2020-07-14 14:09:29 +00:00
Mark Johnston
329d975c0c Print arm64 physmem info during boot.
PR:		243682
Reviewed by:	andrew, emaste
MFC after:	1 week
Event:		July 2020 Bugathon
Differential Revision:	https://reviews.freebsd.org/D25625
2020-07-13 17:05:44 +00:00
Andrew Turner
fcf7a48191 Rerun kernel ifunc resolvers after all CPUs have started
On architectures that use RELA relocations it is safe to rerun the ifunc
resolvers on after all CPUs have started, but while they are sill parked.

On arm64 with big.LITTLE this is needed as some SoCs have shipped with
different ID register values the big and little clusters meaning we were
unable to rely on the register values from the boot CPU.

Add support for rerunning the resolvers on arm64 and amd64 as these are
both RELA using architectures.

Reviewed by:	kib
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D25455
2020-07-05 14:38:22 +00:00
Andrew Turner
e4fc3b653a Read the CPU 0 arm64 ID registers early in initarm
We also update the kernel view early in the boot. This will allow the
use of the common kernel view in ifunc resolvers.

Sponsored by:	Innovate UK
2020-07-01 16:57:57 +00:00
Andrew Turner
eeada9221b Move ID reading signatures to a better header
The functions to read the common user and kernel ID registers should be
in cpu.h rather than undefined.h as they are related to CPU details and
used by undefined instruction handlers.

Sponsored by:	Innovate UK
2020-07-01 16:17:51 +00:00
Andrew Turner
9eb07d5627 Read the arm64 ID registers earlier in the boot process.
Also move parsing the registers to just after the secondary CPUs have
started. This means the kernel register view from all CPUs is available
after the CPU SYSINITs have finished, e.g. for use by ifunc resolvers.

Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D25505
2020-07-01 15:17:45 +00:00
Oleksandr Tymoshenko
94bc2117b4 Add i.MX 8M Quad support
- Add CCM driver and clocks implementations for i.MX 8M
- Add GPC driver for iMX8
- Add clock tree for i.MX 8M Quad
- Add clocks support and new compat strings (where required) for existing i.MX 6 UART, I2C, and GPIO drivers
- Enable aarch64-compatible drivers form i.MX 6 in arm64 GENERIC kernel config
- Add dtb/imx8 kernel module with DTBs for Nitrogen8M and iMX8MQ EVK

With this patch both Nitrogen8M and iMX8MQ EVK boot with NFS root up to multiuser login prompt

Reviewed by:	manu
Differential Revision:	https://reviews.freebsd.org/D25274
2020-07-01 00:33:16 +00:00
Ruslan Bukin
1ea7952510 Coresight: provide device_attach method for FDT bus.
Sponsored by:	DARPA, AFRL
2020-06-29 12:59:09 +00:00
Andrew Turner
b639b3b195 Fix the spelling of identify in the arm64 identcpu code
Sponsored by:	Innovate UK
2020-06-29 09:37:07 +00:00
Andrew Turner
45e999d918 Create a kernel arm64 ID register view
In preparation for using ifuncs in the kernel is is useful to have a common
view of the arm64 ID registers across all CPUs. Add this and extract the
logic for finding the lower value of two fields to a new helper function.

Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D25463
2020-06-29 09:08:36 +00:00
Kyle Evans
5403f186a7 linuxolator: implement memfd_create syscall
This effectively mirrors our libc implementation, but with minor fudging --
name needs to be copied in from userspace, so we just copy it straight into
stack-allocated memfd_name into the correct position rather than allocating
memory that needs to be cleaned up.

The sealing-related fcntl(2) commands, F_GET_SEALS and F_ADD_SEALS, have
also been implemented now that we support them.

Note that this implementation is still not quite at feature parity w.r.t.
the actual Linux version; some caveats, from my foggy memory:

- Need to implement SHM_GROW_ON_WRITE, default for memfd (in progress)
- LTP wants the memfd name exposed to fdescfs
- Linux allows open() of an fdescfs fd with O_TRUNC to truncate after dup.
  (?)

Interested parties can install and run LTP from ports (devel/linux-ltp) to
confirm any fixes.

PR:		240874
Reviewed by:	kib, trasz
Differential Revision:	https://reviews.freebsd.org/D21845
2020-06-29 03:09:14 +00:00
Oleksandr Tymoshenko
4c95d46303 Configure rx_delay/tx_delay values for RK3399/RK3328 GMAC
For 1000Mb mode to work reliably TX/RX delays need to be configured
between the TX/RX clock and the respective signals on the PHY
to compensate for differing trace lengths on the PCB.

Reviewed by:	manu
MFC after:	1 week
2020-06-28 21:11:10 +00:00
Andrew Turner
23e42a83c1 Use EFI memory map to determine attributes for Acpi mappings on arm64.
AcpiOsMapMemory is used for device memory when e.g. an _INI method wants
to access physical memory, however, aarch64 pmap_mapbios is hardcoded to
writeback. Search for the correct memory type to use in pmap_mapbios.

Submitted by:	Greg V <greg_unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D25201
2020-06-28 15:03:07 +00:00
Edward Tomasz Napierala
a39cdcd7e7 Regen.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2020-06-27 14:43:29 +00:00
Edward Tomasz Napierala
308e194cbf Add proper types for linux message queue syscalls; mostly taken
from 32-bit Linuxulator.

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25386
2020-06-27 14:42:08 +00:00
Edward Tomasz Napierala
36507f85dc Add syscall definitions for linux xattr syscalls.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25387
2020-06-27 14:39:44 +00:00
Edward Tomasz Napierala
8036e7876d Adjust types of linuxulator syscalls, to match include/linux/syscalls.h
in vanilla Linux git tree.

Reviewed by:	markj
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25385
2020-06-27 14:37:36 +00:00
Ed Maste
e46cf959d6 arm64 armreg.h: fix TCR_TBI1 definition
Submitted by:	Greg V <greg@unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D25411
2020-06-23 15:32:05 +00:00
Brandon Bergren
40b664f64b [PowerPC] More relocation fixes
It turns out relocating the symbol table itself can cause issues, like fbt
crashing because it applies the offsets to the kernel twice.

This had been previously brought up in rS333447 when the stoffs hack was
added, but I had been unaware of this and reimplemented symtab relocation.

Instead of relocating the symbol table, keep track of the relocation base
in ddb, so the ddb symbols behave like the kernel linker-provided symbols.

This is intended to be NFC on platforms other than PowerPC, which do not
use fully relocatable kernels. (The relbase will always be 0)

 * Remove the rest of the stoffs hack.
 * Remove my half-baked displace_symbol_table() function.
 * Extend ddb initialization to cope with having a relocation offset on the
   kernel symbol table.
 * Fix my kernel-as-initrd hack to work with booke64 by using a temporary
   mapping to access the data.
 * Fix another instance of __powerpc__ that is actually RELOCATABLE_KERNEL.
 * Change the behavior or X_db_symbol_values to apply the relocation base
   when updating valp, to match link_elf_symbol_values() behavior.

Reviewed by:	jhibbits
Sponsored by:	Tag1 Consulting, Inc.
Differential Revision:	https://reviews.freebsd.org/D25223
2020-06-21 03:39:26 +00:00
Edward Tomasz Napierala
bafd96b8dd Regen after r362440.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2020-06-20 18:31:02 +00:00
Edward Tomasz Napierala
52c81be11a Add linux_madvise(2) instead of having Linux apps call the native
FreeBSD madvise(2) directly.  While some of the flag values match,
most don't.

PR:		kern/230160
Reported by:	markj
Reviewed by:	markj
Discussed with:	brooks, kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25272
2020-06-20 18:29:22 +00:00
Michal Meloun
188aee740f Finish renaming in if_dwc.
By using DWC TRM terminology, normal descriptor format should be named
extended and alternate descriptor format should be named normal.

Should not been functional change.

MFC after:	4 weeks
2020-06-19 18:34:27 +00:00
Michal Meloun
a67687fcd8 Use native-sized accesses when accessing memory from kdb.
Not all MMIO mapped devices supports byte access.

MFC after:	4 weeks
2020-06-19 16:26:42 +00:00
Mike Karels
349eddbd07 Add support for bcm54213PE in brgphy.
This chip is used in the Rasperry Pi 4, and is supported by the if_genet
driver. Currently we use the ukphy mii driver, this patch switches over
to the brgphy mii driver instead. To support the rgmii-rxid phy mode,
which is now the default in the Linux dtb, we add support for clock
skewing.

These changes are taken from OpenBSD and NetBSD, except for the bailout
in brgphy_bcm54xx_clock_delay() in rgmii mode, which was found necessary
after testing.

Submitted by:	Robert Crowston, crowston at protomail.com
Differential Revision:	https://reviews.freebsd.org/D25251
2020-06-18 23:57:10 +00:00
Andrew Turner
3a6413d81e Support pmap_extract_and_hold on arm64 stage 2 mappings
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D24469
2020-06-17 19:45:05 +00:00
Ruslan Bukin
c9ea007c3b Complete the ACPI support for ARM Coresight:
o Parse the ACPI DSD (Device Specific Data) graph property and record
  device connections.
o Split-out FDT support to a separate file.
o Get the corresponding (FDT/ACPI) Coresight platform data in
  the device drivers.

Sponsored by:	DARPA, AFRL
2020-06-17 15:54:51 +00:00
Andrew Turner
f3e9395d0c Add all the TCR_EL1 fields
These will be used when adding support for new Armv8 extensions.

Sponsored by:	Innovate UK
2020-06-17 11:56:10 +00:00
Ruslan Bukin
72842e4697 Coresight replicator:
o Add a header file;
o Split-out FDT attachment to a separate file;
o Add ACPI attachment.

Sponsored by:	DARPA, AFRL
2020-06-12 17:31:38 +00:00
Ruslan Bukin
a132ec9f8a ARM Coresight Trace Memory Controller (TMC):
o Split-out FDT attachment to a separate file;
o Add ACPI attachment.

Sponsored by:	DARPA, AFRL
2020-06-12 13:59:58 +00:00
Andrew Turner
400c0119a7 Teach the arm64 vfp.h about struct thread.
Ensure struct thread is defined in vfp.h. In some cases it is not and stops
the kernel from building.

Sponsored by:	Innovate UK
2020-06-12 10:43:21 +00:00
Ruslan Bukin
d06110e566 Shorten the filename of the coresight replicator driver.
Sponsored by:	DARPA, AFRL
2020-06-11 21:52:06 +00:00
Ruslan Bukin
5637d889e3 ARM Coresight Funnel device:
o Split-out FDT attachment to a separate file;
o Add ACPI attachment;
o Add support for the Static Funnel device.

Sponsored by:	DARPA, AFRL
2020-06-10 14:28:36 +00:00
Ruslan Bukin
b6f7bae402 ARM Embedded Trace Macrocell v4.x driver:
o Split-out FDT attachment to a separate file;
o Add ACPI attachment.

Sponsored by:	DARPA, AFRL
2020-06-09 16:43:16 +00:00
Ruslan Bukin
b65c190c40 Fix style: wrap long lines.
Sponsored by:	DARPA, AFRL
2020-06-09 16:06:10 +00:00
Ruslan Bukin
b1670691e8 Rename coresight drivers: use underscores in filenames.
Sponsored by:	DARPA, AFRL
2020-06-09 15:56:41 +00:00
Mitchell Horne
cd9207569f Remove remnant of arm's ELF trampoline
The trampoline code used for loading gzipped a.out kernels on arm was
removed in r350436. A portion of this code allowed for DDB to find the
symbol tables when booting without loader(8), and some of this was
untouched in the removal. Remove it now.

Differential Revision:	https://reviews.freebsd.org/D24950
2020-05-31 14:43:04 +00:00
Mike Karels
51cefda170 genet: workaround for problem with ICMPv6 echo replies
The ICMPv6 echo reply is constructed with the IPv6 header too close to
the beginning of a packet for an Ethernet header to be prepended, so we
end up with an mbuf containing just the Ethernet header.  The GENET
controller doesn't seem to handle this, with or without transmit checksum
offload.  At least until we have chip documentation, do a pullup to
satisfy the chip.  Hopefully this can be fixed properly in the future.
2020-05-30 02:09:36 +00:00
Mike Karels
0add2a5229 genet: fix issues with transmit checksum offload
Fix problem with ICMP echo replies: check only deferred data checksum
flags, and not the received checksum status bits, when checking whether
a packet has a deferred checksum; otherwise echo replies are corrupted
because the received checksum status bits are still present.

Fix some unhandled cases in packet shuffling for checksum offload.
2020-05-30 02:02:34 +00:00
Andrew Turner
2cb0e95f48 Support creating and using arm64 pmap at stage 2
Add minimal support for creating stage 2 IPA -> PA mappings. For this we
need to:

 - Create a new vmid set to allocate a vmid for each Virtual Machine
 - Add the missing stage 2 attributes
 - Use these in pmap_enter to create a new mapping
 - Handle stage 2 faults

The vmid set is based on the current asid set that was generalised in
r358328. It adds a function pointer for bhyve to use when the kernel needs
to reset the vmid set. This will need to call into EL2 and invalidate the
TLB.

The stage 2 attributes have been added. To simplify setting these fields
two new functions are added to get the memory type and protection fields.
These are slightly different on stage 1 and stage 2 tables. We then use
them in pmap_enter to set the new level 3 entry to be stored.

The D-cache on all entries is cleaned to the point of coherency. This is
to allow the data to be visible to the VM. To allow for userspace to load
code when creating a new executable entry an invalid entry is created. When
the VM tried to use it the I-cache is invalidated. As the D-cache has
already been cleaned this will ensure the I-cache is synchronised with the
D-cache.

When the hardware implements a VPIPT I-cache we need to either have the
correct VMID set or invalidate it from EL2. As the host kernel will have
the wrong VMID set we need to call into EL2 to clean it. For this a second
function pointer is added that is called when this invalidation is needed.

Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D23875
2020-05-27 08:00:38 +00:00
Eric Joyner
71d104536b ice(4): Introduce new driver for Intel E800 Ethernet controllers
The ice(4) driver is the driver for the Intel E8xx series Ethernet
controllers; currently with codenames Columbiaville and
Columbia Park.

These new controllers support 100G speeds, as well as introducing
more queues, better virtualization support, and more offload
capabilities. Future work will enable virtual functions (like
in ixl(4)) and the other functionality outlined above.

For full functionality, the kernel should be compiled with
"device ice_ddp" like in the amd64 NOTES file, and/or
ice_ddp_load="YES" should be added to /boot/loader.conf so that
the DDP package file included in this commit can be downloaded
to the adapter. Otherwise, the adapter will fall back to a single
queue mode with limited functionality.

A man page for this driver will be forthcoming.

MFC after:	1 month
Relnotes:	yes
Sponsored by:	Intel Corporation
Differential Revision:	https://reviews.freebsd.org/D21959
2020-05-26 23:35:10 +00:00
Conrad Meyer
852c303b61 copystr(9): Move to deprecate (attempt #2)
This reapplies logical r360944 and r360946 (reverting r360955), with fixed
copystr() stand-in replacement macro.  Eventually the goal is to convert
consumers and kill the macro, but for a first step it helps if the macro is
correct.

Prior commit message:

Unlike the other copy*() functions, it does not serve to copy from one
address space to another or protect against potential faults.  It's just
an older incarnation of the now-more-common strlcpy().

Add a coccinelle script to tools/ which can be used to mechanically
convert existing instances where replacement with strlcpy is trivial.
In the two cases which matched, fuse_vfsops.c and union_vfsops.c, the
code was further refactored manually to simplify.

Replace the declaration of copystr() in systm.h with a small macro
wrapper around strlcpy (with correction from brooks@ -- thanks).

Remove N redundant MI implementations of copystr.  For MIPS, this
entailed inlining the assembler copystr into the only consumer,
copyinstr, and making the latter a leaf function.

Reviewed by:		jhb (earlier version)
Discussed with:		brooks (thanks!)
Differential Revision:	https://reviews.freebsd.org/D24672
2020-05-25 16:40:48 +00:00
Marcin Wojtas
1e6005d807 Introduce VF610 I2C controller support.
NXP LS1046A contains I2C controller compatible with Vybrid VF610.
Existing Vybrid MVF600 driver can be used to support it. For that purpose
declare driver as ofw_iicbus and add methods associated with ofw_iicbus.

For VF610 add dynamic clock prescaler calculation using clock information
from clock driver and clock frequency requested in device tree.

On the occasion add detach function and add additional error handling
in i2c_attach function.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24361
2020-05-25 15:21:38 +00:00
Marcin Wojtas
a5dfa67db1 Add GPIO support for QorIQ boards.
This patch adds a GPIO controller support targeted for NXP LS1046A
SoC. The driver implements the following features:
 * setting direction of each pin (IN or OUT)
 * setting the mode of output pins (PUSHPULL or OPENDRAIN)
 * setting the state of each output pin (1 or 0)
 * reading the state of each input pin (1 or 0)

Submitted by: Kamil Koczurek <kek@semihalf.com>
              Dawid Gorecki <dgr@semihalf.com>
Reviewed by: manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24353
2020-05-25 14:55:37 +00:00
Marcin Wojtas
eacff8a248 Add LS1046A clockgen driver.
Driver provides probe and attach functions for LS1046A clockgen and passes
configuration information to QorIQ clockgen class. It may be used as
a reference implementation for different QorIQ clockgen devices.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: mmel, manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24352
2020-05-25 14:45:18 +00:00
Marcin Wojtas
b8cb0864dc Add QorIQ platform clockgen driver.
This patch adds classes and functions that can be used with various NXP
QorIQ Layerscape SoCs.

As for the clock topology - there is single platform PLL, which supplies
clocks for the peripheral bus and additional PLLs for CPU cores. There
can be multiple core PLLs (For example - LS1046A has two PLLs - CGAPLL1
and CGAPLL2). Each PLL has fixed dividers on output. The core PLLs
are not accessible from dts.

This is a preparation patch for NXP LS1046A SoC support.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: mmel
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24351
2020-05-25 14:31:32 +00:00
Andrew Turner
ad020198ff Stop performing a full icache sync when the DIC and IDC flags are set
The DIC and IDC bits in the CTR_EL0 register signal to the kernel when it
can relax the instruction cache synchronisation operations. The IDC bit
means we can relax cleaning the data cache to the point of unification
while the DIC bit means we don't need to invalidate the instruction cache
for data coherence. In both cases an appropriate barrier is still needed.

For now only implement the case where both bits are set, as is the case
on the Neoverse-N1 as used in the Amazon AWS Graviton 2 CPU. Note that
this behaviour is a optional on the N1 so we may later need to implement
only one or the other bit being set.

There is a tunable to disable each flag on boot.

Testing on a 4 core Graviton 2 instance found a significant improvement
in sys and real time when running "make buildkernel -j4", with no
significant difference in user time.

Reviewed by:	markj
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D24853
2020-05-19 16:04:27 +00:00
Andrew Turner
13628ada98 Create MSI/MSI-X isrcs as needed in the GICv3 ITS driver
Previously we would create an isrc for each MSI/MSI-X interrupt. This
causes issues for other interrupt sources in the system, e.g. a GPIO
driver, as they may be unable to allocate interrupts. This works around
this by allocating the isrc only when needed.

Reported by:	alisaidi@amazon.com
Reviewed by:	mmel
Sponsored by:	Innovaate UK
Differential Revision:	https://reviews.freebsd.org/D24876
2020-05-19 15:27:20 +00:00
Andrew Turner
fd1f4df2be Remove arm64_idcache_wbinv_range as it's unused.
Sponsored by:	Innovate UK
2020-05-15 13:33:48 +00:00
Mark Johnston
e76aab6ae2 Call acpi_pxm_set_proximity_info() slightly earlier on x86.
This function is responsible for setting pc_domain in each pcpu
structure.  Call it from the main function that starts APs, rather than
a separate SYSINIT.  This makes it easier to close the window where
UMA's per-CPU slab allocator may be called while pc_domain is
uninitialized.  In particular, the allocator uses pc_domain to allocate
domain-local pages, so allocations before this point end up using domain
0 for everything.

Reviewed by:	kib
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24757
2020-05-14 16:07:27 +00:00
Andrew Turner
bf610960c6 Fix the name reported when the core supports a 64-bit CCIDX 2020-05-12 21:00:13 +00:00
Conrad Meyer
051fc58cb3 Revert r360944 and r360946 until reported issues can be resolved
Reported by:	cy
2020-05-12 04:34:26 +00:00
Conrad Meyer
580744621f copystr(9): Move to deprecate [2/2]
Unlike the other copy*() functions, it does not serve to copy from one
address space to another or protect against potential faults.  It's just
an older incarnation of the now-more-common strlcpy().

Add a coccinelle script to tools/ which can be used to mechanically
convert existing instances where replacement with strlcpy is trivial.
In the two cases which matched, fuse_vfsops.c and union_vfsops.c, the
code was further refactored manually to simplify.

Replace the declaration of copystr() in systm.h with a small macro
wrapper around strlcpy.

Remove N redundant MI implementations of copystr.  For MIPS, this
entailed inlining the assembler copystr into the only consumer,
copyinstr, and making the latter a leaf function.

Reviewed by:	jhb
Differential Revision:	https://reviews.freebsd.org/D24672
2020-05-11 22:57:21 +00:00
Mark Johnston
1a5b3b5102 Simplify arm64's pmap_bootstrap() a bit.
locore constructs an L2 page mapping the kernel and preloaded data
starting a KERNBASE (the same as VM_MIN_KERNEL_ADDRESS on arm64).
initarm() and pmap_bootstrap() use the preloaded metadata to
tell it where it can start allocating from.

pmap_bootstrap() currently iterates over the L2 page to find the last
valid entry, but doesn't do anything with the result.  Remove the loop
and zap some now-unused local variables.

MFC after:	2 weeks
Sponsored by:	Juniper Networks, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D24559
2020-05-06 15:01:06 +00:00
Michal Meloun
015f223e0b Export tracing facility of GIC500 ITS block.
Possibility of tracing of processing message based interrupts is very
useful for debugging of PCIe driver, mainly for its MSI part.

MFC after: 1 week
2020-04-29 14:31:25 +00:00
Michal Meloun
726122cfda Don't try to re-initialize already preseted regulator.
Don't set initial voltage for regulators having their voltage already
in allowed range. As side effect of this change, we don't try to set
initial voltage for fixed voltage regulators - these don't have impemented
voltage set  method so their initialization has always failed.

MFC after:	3 weeks
2020-04-29 13:45:21 +00:00
Michal Meloun
4c291cb857 Multiple fixes for rockchip iodomain driver:
- always initialize selector of voltage signaling standard.
  Various versions of U-boot leaves voltage signaling standard settings
  for PMUIO2 domain in different state.  Always initialize it
  into expected state.
- start the driver as early as possible, the IO domains should be
  initialized before other drivers are attached.
- rename RK3399 register to its name founds in TRM.

This is the second part of fixes for serial port corruption observed after
DT 5.6 import.

Reviewed by:	manu
MFC after:	1 week
2020-04-29 13:43:15 +00:00
Emmanuel Vadot
1dc1adf0a4 arm64: rockchip: rk805: Use a tailq for the attached regulator
Store the attached regulator in a tailq to later find them in ofw_map.
While here, do not attempt to attach a regulator without a name, a node
might exists but if it doesn't have a name the regulator is unused.

MFC after:	1 month
2020-04-25 15:34:48 +00:00
Michal Meloun
a3fc40936a Reorder initialization steps for given pin.
If pin is switched from fixed function to GPIO, it should have prepared
direction, pull-up/down and default value before function gets switched.
Otherwise we may produce unwanted glitch on output pin.
Right order of drive strength settings is questionable, but I think that
is slightly safer to do it also before function switch.

This fixes serial port corruption observed after DT 5.6 import.

MFC after:	1 week
2020-04-25 09:17:49 +00:00
Mark Johnston
b6fd3860b5 Fix handling of 1GB mappings in the arm64 minidump code.
For such mappings we need to dump 512 page table pages, not one, and
they need to be included in the pmap size recorded in the minidump
header.

MFC after:	2 weeks
Sponsored by:	Juniper Networks, Klara Inc.
2020-04-24 18:47:57 +00:00
Mark Johnston
1d6638472b Remove an obsolete TODO comment from several minidump implementations.
The comment referenced a non-existent function, and these minidump
implementations already buffer discontiguous physical data pages by
mapping them into a single VA range that gets passed to the dump device,
so there is no real advantage in batching calls to blk_write().

The RISC-V and MIPS minidump implementations still write a page at a
time and so would benefit from some form of batching.

MFC after:	2 weeks
Sponsored by:	Juniper Networks, Klara Inc.
2020-04-24 18:47:42 +00:00
Emmanuel Vadot
fb0d255111 arm64: rockchip: Fix TSADC on RK3328
The TSADC familiy is a little bit more complex than V2 and V3.
Early revision do not use syscon and do not use qsel (RK3288).
Next revision still do not use syscon but uses qsel (RK3328).
Final revision use both.

Submitted by:	peterj
MFC after:	1 month
2020-04-23 19:16:20 +00:00
Mike Karels
2cd0c52978 Add genet driver for Raspberry Pi 4B Ethernet
Add driver for Broadcom "GENET" version 5, as found in BCM-2711 on
Raspberry Pi 4B. The driver is derived in part from the bcmgenet.c
driver in NetBSD, along with bcmgenetreg.h.

Reviewed by:	manu
Obtained from:	in part from NetBSD
Relnotes:	yes, note addition
Differential Revision:	https://reviews.freebsd.org/D24436
2020-04-22 00:42:10 +00:00
Mitchell Horne
49439183ce Convert arm's physmem interface to MI code
The arm_physmem interface found in arm's MD code provides a convenient
set of routines for adding/excluding physical memory regions and
initializing important kernel globals such as Maxmem, realmem,
phys_avail[], and dump_avail[]. It is especially convenient for FDT
systems, since we can use FDT parsing functions and pass the result
directly to one of these physmem routines. This interface is already in
use on arm and arm64, and can be used to simplify this early
initialization on RISC-V as well.

This requires only a couple trivial changes:
  - Move arm_physmem_kernel_addr to arm/machdep.c. It is unused on arm64,
    and manipulated entirely in arm MD code.
  - Convert arm32_btop/arm64_btop to atop. This is equivalently defined
    on all architectures.
  - Drop the "arm" prefix.

Reviewed by:	manu, emaste ("looks reasonable")
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D24153
2020-04-19 00:12:30 +00:00
Brooks Davis
b24e6ac8b7 Convert canary, execpathp, and pagesizes to pointers.
Use AUXARGS_ENTRY_PTR to export these pointers.  This is a followup to
r359987 and r359988.

Reviewed by:	jhb
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24446
2020-04-16 21:53:17 +00:00
Brooks Davis
562894f0dc Centralize compatability translation macros.
Copy the CP, PTRIN, etc macros from freebsd32.h into a sys/abi_compat.h
and replace existing definitation with includes where required. This
eliminates duplicate code and allows Linux and FreeBSD compatability
headers to be included in the same files.

Input from:	cem, jhb
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24275
2020-04-14 20:30:48 +00:00
Wojciech Macek
36c1a37655 Add MDIO PHY driver for NS2 ARM64 platform.
Obtained from:         Semihalf
Authored by:           Kornel Duleba <mindal@semihalf.com>
Approved by:           wma
Differential Revision: https://reviews.freebsd.org/D21335
2020-04-06 05:48:58 +00:00
John Baldwin
59838c1a19 Retire procfs-based process debugging.
Modern debuggers and process tracers use ptrace() rather than procfs
for debugging.  ptrace() has a supserset of functionality available
via procfs and new debugging features are only added to ptrace().
While the two debugging services share some fields in struct proc,
they each use dedicated fields and separate code.  This results in
extra complexity to support a feature that hasn't been enabled in the
default install for several years.

PR:		244939 (exp-run)
Reviewed by:	kib, mjg (earlier version)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D23837
2020-04-01 19:22:09 +00:00
Conrad Meyer
ca0ec73c11 Expand generic subword atomic primitives
The goal of this change is to make the atomic_load_acq_{8,16},
atomic_testandset{,_acq}_long, and atomic_testandclear_long primitives
available in MI-namespace.

The second goal is to get this draft out of my local tree, as anything that
requires a full tinderbox is a big burden out of tree.  MD specifics can be
refined individually afterwards.

The generic implementations may not be ideal for your architecture; feel
free to implement better versions.  If no subword_atomic definitions are
needed, the include can be removed from your arch's machine/atomic.h.
Generic definitions are guarded by defined macros of the same name.  To
avoid picking up conflicting generic definitions, some macro defines are
added to various MD machine/atomic.h to register an existing implementation.

Include _atomic_subword.h in arm and arm64 machine/atomic.h.

For some odd reason, KCSAN only generates some versions of primitives.
Generate the _acq variants of atomic_load.*_8, atomic_load.*_16, and
atomic_testandset.*_long.  There are other questionably disabled primitives,
but I didn't run into them, so I left them alone.  KCSAN is only built for
amd64 in tinderbox for now.

Add atomic_subword implementations of atomic_load_acq_{8,16} implemented
using masking and atomic_load_acq_32.

Add generic atomic_subword implementations of atomic_testandset_long(),
atomic_testandclear_long(), and atomic_testandset_acq_long(), using
atomic_fcmpset_long() and atomic_fcmpset_acq_long().

On x86, add atomic_testandset_acq_long as an alias for
atomic_testandset_long.

Reviewed by:	kevans, rlibby (previous versions both)
Differential Revision:	https://reviews.freebsd.org/D22963
2020-03-25 23:12:43 +00:00
Mark Johnston
8db2e8fd16 Remove the secondary_stacks array in arm64 and riscv kernels.
Instead, dynamically allocate a page for the boot stack of each AP when
starting them up, like we do on x86.  This shrinks the bss by
MAXCPU*KSTACK_PAGES pages, which corresponds to 4MB on arm64 and 256KB
on riscv.

Duplicate the logic used on x86 to free the bootstacks, by using a
sysinit to wait for each AP to switch to a thread before freeing its
stack.

While here, mark some static MD variables as such.

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	Juniper Networks, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D24158
2020-03-24 18:43:23 +00:00
Brandon Bergren
3069380898 [PowerPC][Book-E] Fix missing load base in elf_cpu_parse_dynamic().
When I implemented MD DYNAMIC parsing, I was originally passing a
linker_file_t so that the MD code could relocate pointers.

However, it turns out this isn't even filled in until later, so it was
always 0.

Just pass the load base (ef->address) directly, as that's really the only
thing we were interested in in the first place.

This fixes a crash on RB800 where it was trying to write to an unmapped
address when updating the GOT.

Reviewed by:	jhibbits
Sponsored by:	Tag1 Consulting, Inc.
Differential Revision:	https://reviews.freebsd.org/D24105
2020-03-18 02:58:18 +00:00
Michal Meloun
711b7264f4 Add the missing brackets to the logical expression.
Reported by:	clang10 via dim
MFC with:	r355755
2020-03-09 13:36:45 +00:00
Andrew Turner
3a1c1a303e Add more are64 special register fields
Obtained from:	https://github.com/FreeBSD-UPB/freebsd
2020-03-06 16:00:35 +00:00
Andrew Turner
db724d9005 Update the hypervisor registers
- Add more registers needed by bhyve [1]
 - Move EL2 registers from armreg.h to hypervisor.h
 - Add the register name to hypervisor.h

Obtained from:	https://github.com/FreeBSD-UPB/freebsd [1]
2020-03-06 14:46:50 +00:00
Andrew Turner
0c4be71a59 Mark the arm64 machdep.h as kernel only
None of this is useful for userspace.

Sponsored by:	Innovate UK
2020-03-05 10:52:16 +00:00
Andrew Turner
0e6627350e Fix the spelling of aliasing.
Sponsored by:	Innovate UK
2020-03-03 15:31:40 +00:00
Andrew Turner
411531cf30 Move the arm64 cache identification to identcpu.c
This allows us to call it on a per-CPU basis and to warn if the details
are different across CPUs.

While here read the L1 I-Cache type and store this for use later by pmap.

Sponsored by:	Innovate UK
2020-03-03 15:25:01 +00:00
Andrew Turner
2923027c15 Fix the spelling of the VIPT cache type field
Sponsored by:	Innovate UK
2020-03-03 12:50:45 +00:00
Andrew Turner
228b87bc31 Store the boot exception level on arm64 so it can be queried later
A hypervisor, e.g. bhyve, will need to know what exception levelthe kernel
was in when it started booting. If it was EL2 we can then enable said
hypervisor.

Store the boot exception level and allow the kernel to later query it.

Obtained from:	https://github.com/FreeBSD-UPB/freebsd (earlier version)
Sponsored by:	Innovate UK
2020-03-03 08:28:16 +00:00
Andrew Turner
80b9f30590 Add a space missed in r358545
Sponsored by:	Innovate UK
2020-03-02 14:36:15 +00:00
Andrew Turner
15f8f7200f Generate the offsets for struct arm64_bootparams and use it in locore.S
This removes one place with hard coded offsets in locore.S

Sponsored by:	Innovate UK
2020-03-02 14:06:50 +00:00
Dimitry Andric
b95cee35d3 Fix the following -Werror warning from clang 10.0.0:
sys/arm64/arm64/identcpu.c:1170:5: error: misleading indentation; statement is not part of the previous 'if' [-Werror,-Wmisleading-indentation]
                                break;
                                ^
sys/arm64/arm64/identcpu.c:1168:4: note: previous statement is here
                        if (fv[j].desc[0] != '\0')
                        ^

The break should be after the if statement, indented one level less.

MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D23871
2020-02-29 14:11:27 +00:00
Andrew Turner
efc228ae21 Fix the cache type identification
DIC and IDC are supported when the field bits are set.

Sponsored by:	Innovate UK
2020-02-26 15:56:07 +00:00
Pawel Biernacki
7029da5c36 Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE.  All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by:	kib (mentor, blanket)
Commented by:	kib, gallatin, melifaro
Differential Revision:	https://reviews.freebsd.org/D23718
2020-02-26 14:26:36 +00:00
Andrew Turner
a4c354bdd5 Teach the arm64 ident CPU code to print non-ID registers
Add support for non-ID registers when printing CPU information. This is
used with the cache type register to print details of the cache on boot.

Sponsored by:	Innovate UK
2020-02-26 13:22:23 +00:00
Andrew Turner
249ecb6338 Generalise the arm64 ASID allocator.
The requirements of an Address Space ID allocator and a Virtual Machine ID
allocator are similar. Generalise the former code so it can be used with
the latter.

Reviewed by:	alc (previous version)
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D23831
2020-02-26 11:50:24 +00:00
Andrew Turner
23982ffd47 Start to support multiple stages in the arm64 pmap.
On arm64 the stage 1 and stage 2 pte formats are similar enough we can
reuse the pmap code for both. As they are only similar and not identical
we need to know if we are managing stage 1 or stage 2 tables.

Add an enum to store this information and a check to make sure it is
set to stage 1 when we manage stage 1 pte fields.

Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D23830
2020-02-26 11:47:24 +00:00
Andrew Turner
c32e28d5e7 Add more arm64 CTR_EL0 register fields
While here make the _SIZE macros return the size in bytes, not the log2
of the size

Sponsored by:	Innovate UK
2020-02-26 11:29:03 +00:00
Andrew Turner
d153d023f5 Split out the stage 1 pte bits and add the stage 2 bits
In preperation for adding bhyve support to arm64 we need to split the
stage 1 and stage 2 pte fields to allow future changes to create stage 2
page tables.

MFC after:	1 month
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D23669
2020-02-24 16:45:31 +00:00
Olivier Houchard
b0254d9afb debug_monitor: Avoid setting the PSR_D flag for 32bits binaries.
In dbg_monitor_exit(), avoid setting the PSR_D bit if the process is
a 32bits binary. PSR_D is an aarch64-only flags, and for aarch32 processes,
it means "run in big endian".
This should make COMPAT_FREEBSD32 run much better on arm64.
2020-02-24 16:25:11 +00:00
Pawel Biernacki
8eea36ae5b Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (13 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Approved by:	kib (mentor, blanket)
Differential Revision:	https://reviews.freebsd.org/D23635
2020-02-24 10:45:22 +00:00
Emmanuel Vadot
83198172dd arm64: rockchip: rk808: Only init regulator not enabled
If a regulator is already enabled, do not set its value to the minimum
supported on the board.
This fixes booting on rock64 where we set some regulator to the minimal value
while the IPs needs more based on what the bootloader configured.

MFC after:	1 week
2020-02-24 10:40:35 +00:00
Emmanuel Vadot
d71e2ff2dc arm64: rockchip: rk_i2c: Bump to DELAY to 1000
In polling mode with use DELAY to wait for interrupts. The value was
too low for RK3328.

MFC after:	1 week
2020-02-24 10:38:38 +00:00
Mateusz Guzik
f50a2474b7 arm64: remove no longer needed atomic_load_ptr casts 2020-02-14 23:17:18 +00:00
Ruslan Bukin
667c3fc0f6 Add PCI Express driver for the ARM Neoverse N1 System Development
Platform (N1SDP).

Neoverse N1 is a high-performance ARM microarchitecture designed
by the ARM Holdings for the server market.

The PCI part on N1SDP was shipped untested and suffers from some
integration issues.

For instance accessing to not existing BDFs causes System Error
(SError) exception. To mitigate this, the firmware scans the bus,
catches SErrors and creates a table with valid BDFs. That allows
us to filter-out accesses to invalid BDFs in this driver.

Also the root complex config space (BDF == 0) has an unusual
location in memory map, so remapping accesses to it is required.

Finally, the config space is restricted to 32-bit accesses only.

This was tested on the ARM boxes kindly provided by the ARM Ltd
to the DARPA CHERI Project.

In collaboration with:	andrew
Reviewed by:	andrew
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D23349
2020-02-11 15:12:09 +00:00
Andrew Turner
eb43a03acb Implement the Linux/arm64 VDSO gettimeofday and clock_gettime
These are used by Centos 7 and will be needed for the
emulators/linux_base-c7 port.

Sponsored by:	DARPA, AFRL
2020-02-08 13:35:56 +00:00
Mark Johnston
c3d326fd44 Define MAXCPU consistently between the kernel and KLDs.
This reverts r177661.  The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway.  Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.

PR:		243711
Reviewed by:	jhb, kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23512
2020-02-05 19:08:21 +00:00
Ed Maste
fc7510aef7 linuxulator: implement sendfile
Submitted by:	Bora Özarslan <borako.ozarslan@gmail.com>
Submitted by:	Yang Wang <2333@outlook.jp>
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19917
2020-02-05 16:53:02 +00:00
Andrew Turner
9bf9b8b93a Print useful debug data on unhandled kernel fault on arm64
When panicing because of an unhandled data abort from the kernel it is
useful to know the register state and faulting address to aid debugging.
Print these registers before calling panic.

Sponsored by:	DARPA, AFRL
2020-02-04 12:33:00 +00:00
Mark Johnston
a83c682b36 Dynamically select LSE-based atomic(9)s on arm64.
Once all CPUs are online, determine if they all support LSE atomics and
set lse_supported to indicate this.  For now the atomic(9)
implementations are still always inlined, though it would be preferable
to create out-of-line functions to avoid text bloat.  This was not done
here since big.little systems exist in which some CPUs implement LSE
while others do not, and ifunc resolution must occur well before this
scenario can be detected.  It does seem unlikely that FreeBSD will
ever run on such platforms, however, so converting atomic(9) to use
ifuncs is probably a good next step.

Add a LSE_ATOMICS arm64 kernel configuration option to unconditionally
select LSE-based atomic(9) implementations when the target system is
known.

Reviewed by:	andrew, kib
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation, Amazon (hardware)
Differential Revision:	https://reviews.freebsd.org/D23325
2020-02-03 18:23:50 +00:00
Mark Johnston
920de6a15f Add LSE-based atomic(9) implementations.
These make use of the cas*, ld* and swp instructions added in ARMv8.1.
Testing shows them to be significantly more performant than LL/SC-based
implementations.

No functional change here since the wrappers still unconditionally
select the _llsc variants.

Reviewed by:	andrew, kib
MFC after:	1 month
Submitted by:	Ali Saidi <alisaidi@amazon.com> (original version)
Differential Revision:	https://reviews.freebsd.org/D23324
2020-02-03 18:23:35 +00:00
Mark Johnston
c1fced6800 Add wrappers for arm64 atomics.
Add a _llsc suffix for the existing LL/SC-based implementations and add
trivial wrappers.  This is in preparation for supporting LSE-based
atomic(9) implementations.

No functional change intended.

Reviewed by:	andrew, kib
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation, Amazon (hardware)
Differential Revision:	https://reviews.freebsd.org/D23323
2020-02-03 18:23:14 +00:00
Mark Johnston
3ad6c736cf Provide a single implementation for each of the arm64 atomic(9) ops.
Parameterize the macros by type width as well as acq/rel semantics.
This makes modifying the implementations much less tedious and
error-prone and makes it easier to support alternate LSE-based
implementations.  No functional change intended.

Reviewed by:	andrew, kib
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation, Amazon (hardware)
Differential Revision:	https://reviews.freebsd.org/D23322
2020-02-03 18:22:59 +00:00
Andrew Turner
59606417c4 Remove the GICv3 ITS irq and replace it with an ID
In r357324 most of the use of gi_irq was moved to gi_lpi. Complete this
with the last few places we need the IRQ value and create gi_id for the
per-device value we need.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-02-03 14:38:19 +00:00
Andrew Turner
7877018c2c Use a unique name for the GICv3 ITS vmem
When there are multiple GICv3 ITS devices we don't know which vmem is for
which device. Use device_get_nameunit to get a per-device name.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-02-03 13:50:55 +00:00
Andrew Turner
a58fc7cb88 Disable the use of the quantum cache in the GICv3 ITS
This uses UMA to allocate space. It causes issues when there are multiple
ITS devices in the system where interrupts are not allocated from a low
address on some interrupt controllers. Disabling the quantum cache fixes
this on the Neoverse N1 SDP.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-02-03 13:47:41 +00:00
Mark Johnston
1c29da0279 Reimplement stack capture of running threads on i386 and amd64.
After r355784 the td_oncpu field is no longer synchronized by the thread
lock, so the stack capture interrupt cannot be delievered precisely.
Fix this using a loop which drops the thread lock and restarts if the
wrong thread was sampled from the stack capture interrupt handler.

Change the implementation to use a regular interrupt instead of an NMI.
Now that we drop the thread lock, there is no advantage to the latter.

Simplify the KPIs.  Remove stack_save_td_running() and add a return
value to stack_save_td().  On platforms that do not support stack
capture of running threads, stack_save_td() returns EOPNOTSUPP.  If the
target thread is running in user mode, stack_save_td() returns EBUSY.

Reviewed by:	kib
Reported by:	mjg, pho
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23355
2020-01-31 15:43:33 +00:00
Andrew Turner
80ba579b00 Call the MAPTI command earlier in the ITS driver
The GICv3 Software Overview suggests when allocating a new MSI/MSI-X
interrupt we need to call MAPD followed by MAPTI. Unfortunately the code
would place a MOVI command between these. This is invalid as it needs
values set by the MAPTI to be present.

Re-order so we allocate a temporary CPU for the interrupt, then use the
MAPTI command to assign the MSI to it.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-01-31 11:33:11 +00:00
Andrew Turner
e93448dd69 Only create one ITS configuration table
When there are multiple ITS devices in the system we would allocate a
configuration table for each, however only one table is needed as all the
ITS devices share this.

Allocate a table only when the global table is unset.

While here fix the type of this to be a pointer to a uint8_t array as the
entries are all 8 bits wide.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-01-31 10:30:13 +00:00
Andrew Turner
45951bf0a9 Ignore the SMMUv3 and PMCG interrupt controller in the IORT tables
When mapping MSI/MSI-X interrupts throught he Arm IORT ACPI tables we may
need to ignore an interrupt controller even if it is within the bounds the
entry describes. When the SMMUv3 is not using GSIV (non-MSI/MSI-X)
interrupts we need to read the defined field. The Performance Monitoring
Counter Group always ignores the first table entry.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-01-31 09:51:38 +00:00
Andrew Turner
9c7eb5ca62 Shift the ITS processor ID after reading it.
When using the processor ID value we mask off the low and high bits that
should be zero. Unfortunatly we don't shift the ID value so it won't be
affected. Add the shift when reading the ID as this will need to align
with the address based target value.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2020-01-30 18:49:19 +00:00
Ganbold Tsagaankhuu
5e1e7e6192 Enable USB3 support for Rockchip RK3328 SoC.
Reviewed by:	manu
2020-01-29 09:36:59 +00:00
Ganbold Tsagaankhuu
87fcb283c5 Add USB3 related clock definitions for Rockchip RK3328 SoC.
Reviewed by:	manu
2020-01-29 08:46:35 +00:00
Mark Johnston
b6cf94ae07 Print missing ID_AA64PFR{0,1}_EL1 register fields.
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23213
2020-01-23 16:10:38 +00:00
Mark Johnston
32a1f92f2c arm64: Don't enable interrupts in init_secondary().
Doing so can cause deadlocks or panics during boot, if an interrupt
handler accesses uninitialized per-CPU scheduler structures.  This seems
to occur frequently when running under QEMU or AWS.  The idle threads
are set up to release a spinlock section and enable interrupts in
fork_exit(), so there is no need to enable interrupts earlier.

Reviewed by:	kib
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23328
2020-01-23 16:07:27 +00:00
Mark Johnston
9ddf3d342b Add relocation handling required for -zifunc-noplt to work on arm64.
Static relocations for the immediate operand of a branch instruction
must be applied.

In a patch which implements LSE-based atomic(9) operations using ifuncs,
-zifunc-noplt reduces system CPU usage during a buildkernel by several
percent.

Also fix elf_reloc_internal() to return an error if symbol lookup fails.

Reviewed by:	andrew
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17392
2020-01-21 17:45:49 +00:00
Emmanuel Vadot
f70961bf86 rk805: Add a regnode_init method
This method will set the desired voltaged based on values in the DTS.
It will not enable the regulator, this is the job of either a consumer
or regnode_set_constraint SYSINIT if the regulator is boot_on or always_on.

Reviewed by:	mmel
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D23216
2020-01-19 19:56:50 +00:00
Kyle Evans
05d7dd739c sysent targets: further cleanup and deduplication
r355473 vastly improved the readability and cleanliness of these Makefiles.
Every single one of them follows the same pattern and duplicates the exact
same logic.

Now that we have GENERATED/SRCS, split SRCS up into the two parameters we'll
use for ${MAKESYSCALLS} rather than assuming a specific ordering of SRCS and
include a common sysent.mk to handle the rest. This makes it less tedious to
make sweeping changes.

Some default values are provided for GENERATED/SYSENT_*; almost all of these
just use a 'syscalls.master' and 'syscalls.conf' in cwd, and they all use
effectively the same filenames with an arbitrary prefix. Most ABIs will be
able to get away with just setting GENERATED_PREFIX and including
^/sys/conf/sysent.mk, while others only need light additions. kern/Makefile
is the notable exception, as it doesn't take a SYSENT_CONF and the generated
files are spread out between ^/sys/kern and ^/sys/sys, but it otherwise fits
the pattern enough to use the common version.

Reviewed by:	brooks, imp
Nice!:		emaste
Differential Revision:	https://reviews.freebsd.org/D23197
2020-01-18 20:37:45 +00:00
John Baldwin
ffbf0e3ce4 Save and restore floating point registers in get/set_mcontext().
arm64 and riscv were only saving and restoring floating point
registers for sendsig() and sys_sigreturn(), but not for getcontext(),
setcontext(), and swapcontext().

While here, remove an always-false check for uap being NULL from
sys_sigreturn().

Reviewed by:	br, mhorne
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D23218
2020-01-17 19:01:59 +00:00
Emmanuel Vadot
bcd380e88b arm64: rockchip: Add RK3399 PWM driver
Add a driver for the pwm controller in the RK3399 SoC

Submitted by:	bdragon (original version)
Reviewed by:	ganbold (previous version)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D19046
2020-01-16 21:25:13 +00:00
Emmanuel Vadot
3ee778c15e arm64: rockchip: Add new interface for rk_pinctrl
The gpio controller in rockchips SoC in a child of the pinctrl driver
and cannot control pullups and pulldowns.
Use the new fdt_pinctrl interface for accessing pin capabilities and
setting them.
We can now report that every pins is capable of being IN or OUT function
and PULLUP PULLDOWN.
If the pin isn't in gpio mode no changes will be allowed.

Reviewed by:	ganbold (previous version)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D22849
2020-01-16 21:21:20 +00:00
Kyle Evans
1171c633fb Set .ORDER for makesyscalls generated files
When either makesyscalls.lua or syscalls.master changes, all of the
${GENERATED} targets are now out-of-date. With make jobs > 1, this means we
will run the makesyscalls script in parallel for the same ABI, generating
the same set of output files.

Prior to r356603 , there is a large window for interlacing output for some
of the generated files that we were generating in-place rather than staging
in a temp dir. After that, we still should't need to run the script more
than once per-ABI as the first invocation should update all of them. Add
.ORDER to do so cleanly.

Reviewed by:	brooks
Discussed with:	sjg
Differential Revision:	https://reviews.freebsd.org/D23099
2020-01-10 18:24:17 +00:00
Andrew Turner
f2792e5e55 Add atomic_testandset/clear on arm64.
These will reportedly be used in future uma changes.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D23019
2020-01-09 10:26:36 +00:00
Emmanuel Vadot
546652fbe3 rk805: Add regnode_status method
This allow consumers to check if the regulator is enable or not.

Reviewed by:	mmel
Differential Revision:	https://reviews.freebsd.org/D23005
2020-01-08 11:30:03 +00:00
Emmanuel Vadot
d194b63134 rk808: Add min/max for the switch regulators
The two switch regulator are always 3.0V.
Add a special case in get_voltage that if min=max we directly
return the value without calculating it.

Reviewed by:	mmel
Differential Revision:	https://reviews.freebsd.org/D23004
2020-01-08 11:29:22 +00:00
Andrew Turner
1b02a76602 Add more Arm arm64 CPU identification values
- Add all the Cortex-A CPU ID register values I can find.
 - Add the Neoverse-N1 ID regiser value [1]
 - Sort macros by register value.

PR:		243065
Submitted by:	Ali Saidi <alisaidi AT amazon.com> [1]
Sponsored by:	DARPA, AFRL (other than [1])
2020-01-06 20:57:59 +00:00
Pawel Biernacki
a1d7296784 sysctl: mark more nodes as MPSAFE
vm.kvm_size and vm.kvm_free are read only and marked as MPSAFE on i386
already. Mark them as that on amd64 and arm64 too to avoid locking Giant.

Reviewed by:	kib (mentor)
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D23039
2020-01-06 10:52:13 +00:00
Alan Cox
1c3a241032 When a copy-on-write fault occurs, pmap_enter() is called on to replace the
mapping to the old read-only page with a mapping to the new read-write page.
To destroy the old mapping, pmap_enter() must destroy its page table and PV
entries and invalidate its TLB entry.  This change simply invalidates that
TLB entry a little earlier, specifically, on amd64 and arm64, before the PV
list lock is held.

Reviewed by:	kib, markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D23027
2020-01-04 19:50:25 +00:00
Andrew Turner
6d26116baa Add the 8 and 16 bit atomic load/store functions with a barrier on arm64.
Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22966
2020-01-03 10:03:36 +00:00
Ian Lepore
fce985b0bd Add the xdma framework and pl330 dma drivers to arm and arm64 lint builds. 2020-01-02 22:26:54 +00:00
Emmanuel Vadot
84e404763f arm64: rockchip: Add a module for rk_spi
The spi node doesn't lives under a simple-bus compatible node so we need
OFWBUS_PNP_INFO instead of SIMPLEBUS_PNP_INFO.
2020-01-02 17:44:41 +00:00
Edward Tomasz Napierala
cc50333011 Add basic getcpu(2) support to linuxulator. The purpose of this
syscall is to query the CPU number and the NUMA domain the calling
thread is currently running on.  The third argument is ignored.
It doesn't do anything regarding scheduling - it's literally
just a way to query the current state, without any guarantees
you won't get rescheduled an opcode later.

This unbreaks Java from CentOS 8
(java-11-openjdk-11.0.5.10-0.el8_0.x86_64).

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22972
2019-12-31 22:01:08 +00:00
Alan Cox
b0a0152ab9 Determine whether the MMU hardware is capable of updating a page table
entry's access flag and dirty state, and enable this feature when it's
available.

Ensure that we don't overlook a dirty state update that is concurrent
with a call to pmap_enter().  (Previously, all dirty state updates would
have occurred with the containing pmap's lock held, so a page table entry's
dirty state could not have changed while pmap_enter() held that same lock.)

Reviewed by:	andrew, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22907
2019-12-30 20:30:31 +00:00
Pawel Biernacki
54666dffa8 linux(4): implement copy_file_range(2)
copy_file_range(2) is implemented natively since r350315, make it available
for Linux binaries too.

Reviewed by:	kib (mentor), trasz (previous version)
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D22959
2019-12-30 18:11:06 +00:00
Edward Tomasz Napierala
ee0fe82ee2 Implement Linux syslog(2) syscall; just enough to make Linux dmesg(8)
utility work.

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22465
2019-12-29 15:53:55 +00:00
Emmanuel Vadot
6db3672c08 arm64: rockchip: Add driver for the io domain
This driver configure the registers in the GRF according to the value
of the regulators for the platform.
Some IP can run with either 3.0V or 1.8V, if we don't configure them
correctly according to the external voltage used they will not work.
It's only done at boot time for now and might be needed at runtime for
IP like sdmmc.

Reviewed by:	mmel
Tested On:	RockPro64, Firefly-RK3399 (gonzo), AIO-3288 (mmel)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D22854
2019-12-28 15:30:50 +00:00
Emmanuel Vadot
b68dfe6949 arm64: rockchip: rk808: Add remaining regulators
The RK808 driver was missing the LDO and switch regulators.
Add support for them.

Reviewed by:	mmel
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D22852
2019-12-28 15:28:39 +00:00
Brandon Bergren
7821a820d0 [PowerPC] Implement Secure-PLT jump table processing for ppc32.
Due to clang and LLD's tendency to use a PLT for builtins, and as they
don't have full support for EABI, we sometimes have to deal with a PLT in
.ko files in a clang-built kernel.

As such, augment the in-kernel linker to support jump table processing.

As there is no particular reason to support lazy binding in kernel modules,
only implement Secure-PLT immediate binding.

As part of these changes, add elf_cpu_parse_dynamic() to the MD API of the
in-kernel linker (except on platforms that use raw object files.)

The new function will allow MD code to act on MD tags in _DYNAMIC.

Use this new function in the PowerPC MD code to ensure BSS-PLT modules using
PLT will be rejected during insertion, and to poison the runtime resolver to
ensure we get a clear panic reason if a call is made to the resolver.

Reviewed by:	jhibbits
Differential Revision:	https://reviews.freebsd.org/D22608
2019-12-24 15:56:24 +00:00
Scott Long
757d4fbaa7 Introduce the concept of busdma tag templates. A template can be allocated
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created.  See the man page for
details on how to initialize a template.

Templates do not support tag filters.  Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine.  Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.

Reviewed by:	imp, kib
Differential Revision:	https://reviews.freebsd.org/D22906
2019-12-24 14:48:46 +00:00
Alan Cox
7c237b7c3a Correct a mistakenly inverted condition in r355833.
Noticed by:	kib
X-MFC with:	r355833
2019-12-20 20:46:26 +00:00
Andrew Turner
494278bb26 Stop speculation past an eret instruction
On arm64 the eret instruction is used to return from an exception handler.
Some implementations may speculate past this instruction into the next
function. As the user may control many registers in these functions add
a synchronisation barrier sequence after the eret instruction to stop these
CPUs from speculating out of the exception handler.

PR:		242676
Submitted by:	Anthony Steinhauser <asteinhauser@google.com> (previous version)
MFC after:	1 week
2019-12-19 08:52:16 +00:00
Alan Cox
48ef33180a When pmap_enter_{l2,pde}() are called to create a kernel mapping, they are
incrementing (and decrementing) the ref_count on kernel page table pages.
They should not do this.  Kernel page table pages are expected to have a
fixed ref_count.  Address this problem by refactoring pmap_alloc{_l2,pde}()
and their callers.  This also eliminates some duplicated code from the
callers.

Correctly implement PMAP_ENTER_NOREPLACE in pmap_enter_{l2,pde}() on kernel
mappings.

Reduce code duplication by defining a function, pmap_abort_ptp(), for
handling a common error case.

Handle a possible page table page leak in pmap_copy().  Suppose that we are
determining whether to copy a superpage mapping.  If we abort because there
is already a mapping in the destination pmap at the current address, then
simply decrementing the page table page's ref_count is correct, because the
page table page must have a ref_count > 1.  However, if we abort because we
failed to allocate a PV entry, this might be a just allocated page table
page that has a ref_count = 1, so we should call pmap_abort_ptp().

Simplify error handling in pmap_enter_quick_locked().

Reviewed by:	kib, markj (an earlier)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22763
2019-12-18 18:21:39 +00:00
Ian Lepore
a856c60bdd Remove unnecessary MODULE_DEPEND() from imx_i2c.c, and also from rk_i2c
where it got copied to.
2019-12-17 17:03:03 +00:00
Emmanuel Vadot
bc188bd4f8 arm64: rockchip: rk_gpio: Fix pin number
The maxpin counter starts at 0, fix one by one error.
This is still not totally correct for some banks in some SoC that have
fewer pins but this will be dealt with in another commit.

MFC after:	3 days
2019-12-17 10:57:31 +00:00
Emmanuel Vadot
3df0c02623 arm64: rockchip: rk_pinctrl: Fix clear bits in SYSCON_MODIFY
r351187 change the SYSCON_WRITE to SYSCON_MODIFY but didn't changed the
mask variable that used to hold the bitmask in the upper 16 bits of the
register that control which bits are changed. So we ended up clearing
bit from the upper 16bits half which are always 0 after a read.
Use the correct bit mask for bits that we want to clear.

MFC after:	3 days
2019-12-17 10:55:28 +00:00
Jeff Roberson
a94ba188c3 Repeat the spinlock_enter/exit pattern from amd64 on other architectures to
fix an assert violation introduced in r355784.  Without this spinlock_exit()
may see owepreempt and switch before reducing the spinlock count.  amd64
had been optimized to do a single critical enter/exit regardless of the
number of spinlocks which avoided the problem and this optimization had
not been applied elsewhere.

Reported by:	emaste
Suggested by:	rlibby
Discussed with:	jhb, rlibby
Tested by:	manu (arm64)
2019-12-16 20:15:04 +00:00
Edward Tomasz Napierala
b5f20658ee Add compat.linux.emul_path, so it can be set to something other
than "/compat/linux".  Useful when you have several compat directories
with different Linux versions and you don't want to clash with files
installed by linux-c7 packages.

Reviewed by:	bcr (manpages)
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22574
2019-12-16 20:07:04 +00:00
Alan Cox
68ca966558 Apply a small optimization to pmap_remove_l3_range(). Specifically, hoist a
PHYS_TO_VM_PAGE() operation that always returns the same vm_page_t out of
the loop.  (Since arm64 is configured as VM_PHYSSEG_SPARSE, the
implementation of PHYS_TO_VM_PAGE() is more costly than that of
VM_PHYSSEG_DENSE platforms, like amd64.)

MFC after:	1 week
2019-12-15 22:41:57 +00:00
Michal Meloun
dfd1d0fcab Add driver for Rockchip PCIe root complex found in RK3399 SOC.
Unfortunately, there are some limitations:
- memory aperture of his controller is only 16MiB, so it is nearly
  unusable for graphic cards
- every attempt to generate type 1 config cycle always causes trap.
  These config cycles are disabled now and we don't support cards
  with PCIe switch.
- in some cases, attempt to do config cycle to (probably) not-yet ready
  card also causes trap. This cannot be detected at runtime, but it seems
  like very rare issue.

MFC after:	3 weeks
Differential Revision:  https://reviews.freebsd.org/D22724
2019-12-14 14:56:34 +00:00
Edward Tomasz Napierala
cf69fe66d4 Add sync_file_range(2) implementation to linux(4); it's a thin wrapper
over the usual fsync(2).

This silences some warnings when running "apt-get upgrade".

Reviewed by:	brooks, emaste
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22371
2019-12-14 13:37:17 +00:00
Edward Tomasz Napierala
0cde2b3239 Regen after r355752.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22371
2019-12-14 13:32:37 +00:00
Edward Tomasz Napierala
0610f417a4 Fix definitions for linuxulator's sync_file_range(2).
Reviewed by:	brooks, emaste
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22371
2019-12-14 13:30:43 +00:00
Andrew Turner
65565c9784 Add comments and macros to the tcr_el1 setting code to help understand it.
This code is non-obvious when reading for the first time. To help with
understanding of it add comments explaining what it's doing.

While here use macros from armreg.h rather than magic numbers.

Sponsored by:	DARPA, AFRL
2019-12-12 18:27:54 +00:00
Emmanuel Vadot
e63adc105d arm64: rockchip: rk_pinctrl: Fix parse_bias for RK3399
Only bank 0 and bank 2 are different than other rockchip SoC, fix this.
While here remove some debug printfs that where added in r355648

MFC after:	3 days
X-MFC-With:	r355648
2019-12-12 13:21:43 +00:00
Emmanuel Vadot
064486c05f arm64: rockchip: rk_pinctrl: Add bias parsing based on the SoC type
Not all rockchip have the same value for pullup/pulldown so add a function
per SoC and call the right one to have the proper value.

MFC after:	3 days
2019-12-12 13:02:22 +00:00
Emmanuel Vadot
6fe4e8bdbd arm64: rk3328: Add the *clk_peri_niu clocks
Those clocks are always enable by default and are not really explained
in the TRM but the reason we had them is that they have the periph clock
as a parent and those parent should never be disable which can happen
if we disable all the childs. The current childs are the sd/emmc/sdio clocks
so the board will hang if we disable them.

MFC after:	1 month
2019-12-11 18:39:05 +00:00
Emmanuel Vadot
946c511ee5 arm64: Add explicit devices for dwmmc variant
We used to include the hisi version if soc_hisi_hi6220 was present,
include the altera version if dwmmc_altera was present and include
the rockchip version if soc_rockchip_rk3328 was present.
Now every version have it's own device directive.
The rockchip version isn't named dwmmc_rockchip because all other
rockchip driver are named rk_XXX.

MFC after:	1 month
2019-12-11 18:36:07 +00:00
Mark Johnston
5cff1f4dc3 Introduce vm_page_astate.
This is a 32-bit structure embedded in each vm_page, consisting mostly
of page queue state.  The use of a structure makes it easy to store a
snapshot of a page's queue state in a stack variable and use cmpset
loops to update that state without requiring the page lock.

This change merely adds the structure and updates references to atomic
state fields.  No functional change intended.

Reviewed by:	alc, jeff, kib
Sponsored by:	Netflix, Intel
Differential Revision:	https://reviews.freebsd.org/D22650
2019-12-10 18:14:50 +00:00
John Baldwin
d8010b1175 Copy out aux args after the argument and environment vectors.
Partially revert r354741 and r354754 and go back to allocating a
fixed-size chunk of stack space for the auxiliary vector.  Keep
sv_copyout_auxargs but change it to accept the address at the end of
the environment vector as an input stack address and no longer
allocate room on the stack.  It is now called at the end of
copyout_strings after the argv and environment vectors have been
copied out.

This should fix a regression in r354754 that broke the stack alignment
for newer Linux amd64 binaries (and probably broke Linux arm64 as
well).

Reviewed by:	kib
Tested on:	amd64 (native, linux64 (only linux-base-c7), and i386)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22695
2019-12-09 19:17:28 +00:00
Emmanuel Vadot
670f36540d arm64: Use the kenv provided by loader
Otherwise we have an empty kenv a likely cannot boot.

Submitted by:	kevans
MFC after:	1 month
X-MFC-With:	355487
2019-12-09 15:23:05 +00:00
Michal Meloun
5641eda2f3 Add support for booting kernel directly from U-Boot using booti command.
In some cases, like is locked bootstrap or device's inability to boot from
removable media, we cannot use standard boot sequence and is necessary to
boot kernel directly from U-Boot.

Discussed with:	jhibbits
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D13861
2019-12-07 16:14:23 +00:00
Brooks Davis
af796bfa71 sysent: Reduce duplication and improve readability.
Use the power of variable to avoid spelling out source and generated
files too many times.  The previous Makefiles were hard to read, hard to
edit, and badly formatted.

Reviewed by:	kevans, emaste
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22714
2019-12-06 23:59:23 +00:00
Luiz Otavio O Souza
8cfe2a7ad3 Add the SPI driver for the Marvell Armada 37x0 SoC.
Interrupt based driver, implements SPI mode and clock configuration.

Tested on espressobin and SG-3200.

Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-12-06 12:55:39 +00:00
Alan Cox
1bef4955bc On a context switch, handle the possibility that the old thread was
preempted after an "ic" or "tlbi" instruction but before it performed a
"dsb" instruction.  The "ic" and "tlbi" instructions have unusual
synchronization requirements.  If the old thread migrates to a new
processor, its completion of a "dsb" instruction on that new processor does
not guarantee that the "ic" or "tlbi" instructions performed on the old
processor have completed.

This issue is not restricted to the kernel.  Since locore.S sets the UCI bit
in SCTLR, user-space programs can perform "ic ivau" instructions (as well as
some forms of the "dc" instruction).

Reviewed by:	andrew, kib, markj, mmel
X-MFC with:	r355145
Differential Revision:	https://reviews.freebsd.org/D22622
2019-12-05 19:25:49 +00:00
Luiz Otavio O Souza
8a82d5675d Add the I2C driver for the Armada 37x0.
This controller is a bit tricky as the STOP condition must be indicated in
the last tranferred byte, some devices will not like the repeated start
behavior of this controller.  A proper fix to this issue is in the works.

This driver works in polling mode, can be used early in the boot (required
in some cases).

Tested on espressobin/SG-1100 and the SG-3200.

Obtained from:	pfSense
Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-12-05 00:56:03 +00:00
Warner Losh
f86e60008b Regularize my copyright notice
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
  All Rights Reserved on same line as other copyright holders (but not
  me). Other such holders are also listed last where it's clear.
2019-12-04 16:56:11 +00:00
Li-Wen Hsu
0386b6c8a0 Fix arm64 build after r355373
Sponsored by:	The FreeBSD Foundation
2019-12-04 08:21:54 +00:00
John Baldwin
31174518d2 Use uintptr_t instead of register_t * for the stack base.
- Use ustringp for the location of the argv and environment strings
  and allow destp to travel further down the stack for the stackgap
  and auxv regions.
- Update the Linux copyout_strings variants to move destp down the
  stack as was done for the native ABIs in r263349.
- Stop allocating a space for a stack gap in the Linux ABIs.  This
  used to hold translated system call arguments, but hasn't been used
  since r159992.

Reviewed by:	kib
Tested on:	md64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22501
2019-12-03 23:17:54 +00:00
Emmanuel Vadot
e213223c9b Remove "all rights reserved" from copyright for the file I own.
Some of the files have both me and Jared McNeill and he gave me
permission to remove it from his files too.
2019-12-03 21:00:45 +00:00
Emmanuel Vadot
f141cc31dc arm64: rockchip: rl3399: Remove the ability to put the PLL in normal mode at boot
RK3399 PLLs have three modes :
 - Normal, where they behave normally and their freq is calculated based on
   the registers values.
 - Slow, where the PLL freq is 24Mhz (well, the external oscillator).
 - Deep Slow, used for suspend where the freq is 32Khz.

We used to put every CPU related PLL in normal mode but it can cause problem
if the firmware didn't setup the clocks register correctly.
And even if it did but left the pll in slow or deep slow mode that might be
because the PMIC suppling voltage for the CPU haven't been configured yet
and we cannot do that at this point.
So remove the ability to set PLLs to normal mode at boot to avoid any problems.
2019-12-03 19:18:32 +00:00
Justin Hibbits
750d951f5a revert r354714 "Boot arm64 kernel using booti command from U-boot."
After discussing with mmel@, it was clear this is insufficient to address
all the needs.  mmel@ will commit his original patch, from
https://reviews.freebsd.org/D13861, and the additions needed from r354714
will be made afterward.

Requested by:	mmel
Sponsored by:	Juniper Networks, Inc.
2019-12-02 15:07:06 +00:00
Andrew Turner
ef3e1e13b6 Use the VM_MEMATTR macros to describe the MAIR offsets.
Remove the duplicate macros that defined a subset of the VM_MEMATTR values.
While here use VM_MEMATTR macros when filling in the MAIR register.

Reviewed by:	alc, markj
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22241
2019-11-29 16:14:32 +00:00
Emmanuel Vadot
19b38c94dc arm64: rockchip: rk3328: Add TSADC clocks
Add the clocks so we can use the rk_tsadc driver to monitor
the cpu temperature.
2019-11-28 20:46:24 +00:00
Emmanuel Vadot
a0cb4996c6 arm64: rockchip: tsadc: Do not free the sysctl context is it wasn't created
MFC after:	3 weeks
X-MFC-With:	r355173
2019-11-28 20:17:03 +00:00
Michal Meloun
6903d37518 Add driver for temperature sensors found in RK32898, RK3328 and RK3399 SoC's.
MFC after:	3 weeks
Reviewed by:	manu
Differential Revision:  https://reviews.freebsd.org/D22442
2019-11-28 17:01:31 +00:00
Alan Cox
7b3c31acbe There is no reason why we need to pin the underlying thread to its current
processor in pmap_invalidate_{all,page,range}().  These functions are using
an instruction that broadcasts the TLB invalidation to every processor, so
even if a thread migrates in the middle of one of these functions every
processor will still perform the required TLB invalidations.

Reviewed by:	andrew, markj
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D22502
2019-11-27 20:33:49 +00:00
Michal Meloun
fa2ec6b439 Finish implementation of RK3299 clocks.
- implement of all but mmc clocks. MMC clocks will be added later by own commit.
- use 'link' clock type for external clocks.
- use macros for initialization of structure's named members.

MFC after:	3 weeks
Reviewed by:	manu
Differential Revision:  https://reviews.freebsd.org/D22441
2019-11-26 17:56:39 +00:00
Kyle Evans
0227a14997 arm64/NOTES: add SOC_BRCM_BCM2838
This should have been done back when it was added, but it was not. It only
really adds an extra entry for memory mapping bits in bcm2835_vcbus.c, so
nothing too extensive yet.
2019-11-23 03:38:26 +00:00
Andrew Turner
68cad68149 Add kcsan_md_unsupported from NetBSD.
It's used to ignore virtual addresses that may have a different physical
address depending on the CPU.

Sponsored by:	DARPA, AFRL
2019-11-21 13:22:23 +00:00
Andrew Turner
849aef496d Port the NetBSD KCSAN runtime to FreeBSD.
Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in
the FreeBSD kernel. It is a useful tool for finding data races between
threads executing on different CPUs.

This can be enabled by enabling KCSAN in the kernel config, or by using the
GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later
needs a compiler change to allow -fsanitize=thread that KCSAN uses.

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22315
2019-11-21 11:22:08 +00:00
Alan Cox
48c22f0eb2 Until every possible root cause for an "invalid ASID" assertion failure is
resolved, assign every pmap a valid ASID when it is first initialized.
2019-11-20 16:32:13 +00:00
Alan Cox
6b71afbf7f Achieve two goals at once: (1) Avoid an unnecessary broadcast TLB
invalidation in reclaim_pv_chunk(). (2) Prevent an "invalid ASID" assertion
failure in reclaim_pv_chunk().  The detailed explanation for this change is
provided by r354792.

X-MFC with:	r354792
2019-11-19 19:05:05 +00:00
Andrew Turner
5008941522 Return 0 from ptrace_set_pc as it now completes successfully.
Sponsored by:	DARPA, AFRL
2019-11-19 13:28:59 +00:00
Andrew Turner
5d1349914a Allow ptrace to set the probram counter on arm64.
Sponsored by:	DARPA, AFRL
2019-11-19 13:25:46 +00:00
Andrew Turner
a2bb7f7a62 Fix the definition of bus_space_read_stream_8 on arm64.
This is currently unused, however will be when the Kernel Concurrency
Sanitizer (KCSAN) is imported from NetBSD.

Sponsored by:	DARPA, AFRL
2019-11-19 10:57:44 +00:00
Kyle Evans
f22a592111 Convert in-tree sysent targets to use new makesyscalls.lua
flua is bootstrapped as part of the build for those on older
versions/revisions that don't yet have flua installed. Once upgraded past
r354833, "make sysent" will again naturally work as expected.

Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D21894
2019-11-18 23:28:23 +00:00
John Baldwin
03b0d68c72 Check for errors from copyout() and suword*() in sv_copyout_args/strings.
Reviewed by:	brooks, kib
Tested on:	amd64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22401
2019-11-18 20:07:43 +00:00
Mark Johnston
85e06c728c Set MALLOC_DEBUG_MAXZONES=1 in GENERIC-NODEBUG configurations.
The purpose of this option is to make it easier to track down memory
corruption bugs by reducing the number of malloc(9) types that might
have recently been associated with a given chunk of memory.  However, it
increases fragmentation and is disabled in release kernels.

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2019-11-18 20:03:28 +00:00
Mark Johnston
c7181c5ab0 Implement vm.pmap.kernel_maps for arm64.
Reviewed by:	alc
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22142
2019-11-18 15:37:01 +00:00
Mark Johnston
d2624609e6 Let arm64 pmap_qenter() and pmap_kenter() unconditionally set NX.
As on amd64, there is no need for mappings created by these functions to
be executable.

Reviewed by:	alc, andrew
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22141
2019-11-18 15:36:46 +00:00
Alan Cox
8a19cf92cf Achieve two goals at once: (1) Avoid an unnecessary broadcast TLB
invalidation in pmap_remove_all(). (2) Prevent an "invalid ASID" assertion
failure in pmap_remove_all().

The architecture definition specifies that the TLB will not cache mappings
that don't have the "AF" bit set, so pmap_remove_all() needn't issue a TLB
invalidation for mappings that don't have the "AF" bit set.

We allocate ASIDs lazily.  Specifically, we don't allocate an ASID for a
pmap until we are activating it.  Now, consider what happens on a fork().
Before we activate the child's pmap, we use pmap_copy() to copy mappings
from the parent's pmap to the child's.  These new mappings have their "AF"
bits cleared.  Suppose that the page daemon decides to reclaim a page that
underlies one of these new mappings.  Previously, the pmap_invalidate_page()
performed by pmap_remove_all() on a mapping in the child's pmap would fail
an assertion because that pmap hasn't yet been assigned an ASID.  However,
we don't need to issue a TLB invalidation for such mappings because they
can't possibly be cached in the TLB.

Reported by:	bob prohaska <fbsd@www.zefox.net>
Reviewed by:	markj
MFC after:	1 week
X-MFC-before:	r354286
Differential Revision:	https://reviews.freebsd.org/D22388
2019-11-17 17:38:53 +00:00