Commit Graph

345 Commits

Author SHA1 Message Date
Andrew Turner
0f0b7aa289 Fix a typo in a string in a KASSERT to sanity check the CPU IDs.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-25 15:59:31 +00:00
Andrew Turner
8ec2b5df73 Rework how we number CPUs on arm64 to try and keep clusters together.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-25 15:57:13 +00:00
Andrew Turner
abf5bdb441 Enable the generic OHCI driver on arm64
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-07-25 14:49:15 +00:00
Alexander Motin
fb112f72a8 Add more UEFI/e820 memory types from latest specifications.
This is only cosmetics.

MFC after:	2 weeks
2016-07-24 09:15:11 +00:00
Andrew Turner
e0f50f02e4 Implement bus_print_child to print the resources used by the ITS driver.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-15 13:25:47 +00:00
Andrew Turner
5d94bbc241 Finish removing the non-INTRNG support from sys/arm64.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:31:29 +00:00
Andrew Turner
ec1172ed92 Remove the non-INTRNG support from the ThunderX PCIe drivers.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:23:49 +00:00
Andrew Turner
c55795369b Move gic_v3_irqsrc into the GICv3 driver source as it's only needed there.
Remove unused macros from the GICv3 header.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:16:51 +00:00
Andrew Turner
a509d56d7f Move structures only used by the GICv3 ITS driver from a shared header to
the ITS driver file. There is no need for other drivers to need to know
about these structures.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:10:54 +00:00
Andrew Turner
b30fd6a080 Remove the non-INTRNG support from the GICv3 interrupt controller driver.
This is no longer needed.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:05:25 +00:00
Andrew Turner
07b8a57120 Add memmmap on arm64 so we can mmap /dev/mem and /dev/kmem.
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-07-13 23:03:34 +00:00
Andrey A. Chernov
239e5577a2 Undo r302601, WCHAR_MAX may not be a valid wchar value. 2016-07-12 04:20:44 +00:00
Andrey A. Chernov
713a61670c I don't know why unsigned int is choosed for wchar_t here, but WCHAR_MAX
should be <= WINT_MAX. It is bigger, __UINT_MAX > INT32_MAX
2016-07-12 00:37:48 +00:00
Ed Schouten
d96aeddf2f Don't forget to set sa->narg for CloudABI system calls.
It turns out that this value is not used within the system call code
under normal conditions, except when using tracing tools like ktrace.
If we forget to set this value, it is set to random garbage. This may
cause ktrace to hang indefinitely, making it impossible to kill.

Reported by: Michael Plass
PR: 210800
MFC before: 11.0-RELEASE
2016-07-08 20:09:21 +00:00
Andrew Turner
b760df65c8 Remove the old pre-INTRNG arm64 interrupt framework. GENERIC was switched
to INTRNG in r301565 with the old code no longer being built by default with
no reports of issues on any supported hardware.

Approved by:	re (gjb)
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-07-06 16:20:10 +00:00
Wojciech Macek
e46f2622b4 ARM64: fix DMAP calculation
Use arithmetic operators instead of logical. This fixes
    DMAP ranges calculation for ThunderX Dual Socket.

    Obtained from:         Semihalf
    Sponsored by:          Cavium
    Reviewed by:           zbb
    Differential Revision: https://reviews.freebsd.org/D7023
    Approved by:           re (gjb)
2016-06-30 04:58:19 +00:00
Andrew Turner
add8a9c2d2 Fix a race when the hardware has raised an exception with an unknown or
uncategorised reason. We need to read the fault address register before
enabling interrupts as the interrupt handler may cause this register to
change.

Approved by:	re (marius, kib)
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-06-22 12:05:08 +00:00
Konstantin Belousov
5c2cf81845 Update comments for the MD functions managing contexts for new
threads, to make it less confusing and using modern kernel terms.

Rename the functions to reflect current use of the functions, instead
of the historic KSE conventions:
  cpu_set_fork_handler -> cpu_fork_kthread_handler (for kthreads)
  cpu_set_upcall -> cpu_copy_thread (for forks)
  cpu_set_upcall_kse -> cpu_set_upcall (for new threads creation)

Reviewed by:	jhb (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	re (hrs)
Differential revision:	https://reviews.freebsd.org/D6731
2016-06-16 12:05:44 +00:00
Andrew Turner
04ff877e4c Switch arm64 to use intrng by default. The old interrupt handling code can
still be used, however this is expected to be removed soon.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-06-07 20:14:08 +00:00
Svatopluk Kraus
ad5244ece1 INTRNG - change the way how an interrupt mapping data are provided
to the framework in OFW (FDT) case.

This is a follow-up to r301451.

Differential Revision:	https://reviews.freebsd.org/D6634
2016-06-05 16:20:12 +00:00
Andrew Turner
550d01a211 Add the GICv3 ITS intrng driver. As the interface to the interrupt
framework has significantly changed the driver has moved to a new file.
While it shares some code with the existing driver this has been modified
to work better with the intrng framework.

This has been tested on the ThunderX servers in the netperf cluster and has
been used to boot them for other testing, including DTrace and hwpmc.

With this we can use intrng on all supported arm64 platforms I was able to
test on. It is expected we will move to intrng soon, and disable the old
arm64 interrupt framework.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6437
2016-06-03 10:28:06 +00:00
Zbigniew Bodek
32fb8300d4 Return real error value instead of hard-coded ENXIO (fix after r300149)
It is possible to return real error value in case of gic_v3_attach()
failure instead of hard-coded ENXIO.

Obtained from:	Semihalf
Sponsored by:	Cavium
2016-06-01 08:20:10 +00:00
Andrew Turner
920c8a57c0 dpcpu_init should have also passed in the calculated cpuid, not the
devicetree ID.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-31 19:17:32 +00:00
Andrew Turner
e0db04b3de Allow the kernel to boot on a CPU where the devicetree has numbered it with
a non-zero ID. To do this we increment the cpuid of any CPUs with a smaller
devicetree ID by one to stop them conflicting with the boot CPU.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-31 18:45:52 +00:00
Andrew Turner
cea2a7fe6d Enable setting BF_COHERENT on DMA tags. This allows the kernel to start
using the cache handling functions.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-31 16:28:56 +00:00
Andrew Turner
2760c2cad4 Mark the ThunderX and generic PCI drivers as cache-coherent when we know
this to be the case. This will mean we don't try and handle the cache in
bus_dmamap_sync when it is not needed.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6605
2016-05-31 09:15:21 +00:00
Andrew Turner
02759f14f2 Extract the correct bits from the GICD_TYPER register. The interrupt count
is encoded in the bottom 5 bits.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-20 15:43:51 +00:00
Andrew Turner
da257dda10 Add more useful GICv3 register definitions. While here fix
GITS_CBASER_CACHE_MASK to use the correct shift macro.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-20 13:11:07 +00:00
Andrew Turner
98670cad72 Filter out BUS_DMASYNC_POSTWRITE sync operations, there is nothing for us
to do on these.

Reported by:	wma
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-20 12:38:48 +00:00
Andrew Turner
fa13831f86 Enable NEW_PCIB on arm64.
Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-20 08:43:18 +00:00
Andrew Turner
6d2f7ef617 Handle PCI_RES_BUS on the generic and ThunderX PCIe drivers. This has been
tested on the Pass 1.1 and 2.0 ThunderX machines in the Netperf cluster.

Reviewed by:	jhb
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6453
2016-05-20 08:29:00 +00:00
Andrew Turner
b7c02deed2 Define PCI_RES_BUS for NEW_PCIB
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-19 14:00:18 +00:00
Andrew Turner
9346e9130d Return the struct intr_pic pointer from intr_pic_register. This will be
needed in later changes where we may not be able to lock the pic list lock
to perform a lookup, e.g. from within interrupt context.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-18 15:05:44 +00:00
Zbigniew Bodek
b02a807b46 Add support for MSI/MSIX deallocation on GICv3-ITS
Allow to deallocate previously allocated ITS device along with
its interrupts. Interrupt numbers are being freed when the last
LPI number is no longer busy.

Reviewed by:	wma
Obtained from:	Semihalf
Sponsored by:	Cavium
Differential Revision:	https://reviews.freebsd.org/D6351
2016-05-18 10:09:07 +00:00
Bjoern A. Zeeb
1f420ed469 The GIC (v2 at least) has a bit in the TYPER register to indicate whether the GIC
supports the Security Extensions or not. This bit is not the same as the CPU one.
Currently we are not checking for either before trying to write to the special
registers.  This can lead to problems on hardware or simulators that do not
provide the security extensions.  Add the missing checks. Their interactions with
the CPU flag is not entirely clear to me but using a macro will make it easier
to quickly adjust the condition once the CPU bits are sorted as well.

Reviewed by:	br
Sponsored by:	DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D6397
2016-05-17 13:12:26 +00:00
Andrew Turner
e34357f56d Add an arm64 kernel config to help testing intrng. It is expected this
config will only last a few weeks until we switch to this interrupt
framework.

Obtained from:	 ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-17 12:48:30 +00:00
Andrew Turner
9a38d21983 Clean up the GICv3 intrng code:
* In gic_v3_attach free the correct data on failure.
 * Implement gic_v3_teardown_intr.
 * Update the panic string when enabling/disabling an invalid interrupt.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-17 12:46:50 +00:00
Andrew Turner
88f7980a81 Add intrng support to the GICv3 driver. It lacks ITS support so won't handle
MSI or MSI-X interrupts, however this is enought to boot FreeBSD under the
ARM Foundation Model with a GICv3 interrupt controller.

Approved by:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-16 14:07:43 +00:00
Andrew Turner
2b48ec8e6d Move the call to intr_pic_init_secondary to the same place as in the
non-intrng case.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 12:02:06 +00:00
Andrew Turner
72b3f638b7 Add support for intrng to arm64. As the GICv3 drivers will need to be
updated, and until further testing can be done, this is disabled for now.

It is expected arm64 will switch to this interface, and the old interface
will be removed before 11.0 is released.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-16 10:48:51 +00:00
Andrew Turner
4e1763d34c Teach the ThunderX PCI PEM driver about intrng. This will be used later
when arm64 is supported by intrng.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 10:03:57 +00:00
Andrew Turner
1e43b18c4b Add a pcib interface for use by interrupt controllers that need to
translate the pci rid to a controller ID. The translation could be based
on the 'msi-map' OFW property, a similar ACPI option, or hard-coded for
hardware lacking the above options.

Reviewed by:	wma
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 09:31:44 +00:00
Andrew Turner
5db4448f06 Add support to the arm64 busdma to handle the cache. For now this is
disabled, however when we enable it it will default to assume memory is
not cache-coherent, unless either the tag was created or the parent was
marked as cache-coherent.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-13 16:03:50 +00:00
Andrew Turner
f82b6f5255 Rename the internal BUC_DMA_* flags to BF_* so they won't conflict with
the flags in sys/bus_dma.h.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-12 15:24:36 +00:00
Andrew Turner
2b45fde379 Restrict the memory barriers in bus_dmamap_sync to just the operations
where it's needed.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-12 12:13:28 +00:00
Andrew Turner
9615213bef Call busdma_swi from swi_vm as is done from other architectures.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 18:48:47 +00:00
Andrew Turner
b7a7ee3af1 On arm64 always create a bus_dmamap_t object. This will be use to hold the
list of memory that the kernel will need to sync when operating with a
non-cache coherent DMA engine.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 16:53:41 +00:00
Andrew Turner
7ca01e8f89 Add data barriers to the arm64 bus_dmamap_sync function. We need these
to ensure ordering between the CPU and device. As the CPU and DMA target
may be in different shareability domains they need to be full system
barriers.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 14:59:54 +00:00
Zbigniew Bodek
46542fb652 Fix I/O coherence issues on ThunderX when SMP is disabled
To maintain coherence between cache and DMA memory appropriate
shareability flags need to be set in the PTE regardless of SMP
option.

Reviewed by:	wma
Obtained from:	Semihalf
Sponsored by:	Cavium
Differential Revision: https://reviews.freebsd.org/D6231
2016-05-11 13:23:56 +00:00
Andrew Turner
fb853e4270 Push the logic to talk with the MSI/MSI-X interrupt controller to the FDT
attachment. This is where it will live when we import intrng as it will
need to look at either the msi-parent or msi-map FDT properties.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-10 15:45:59 +00:00