Commit Graph

332 Commits

Author SHA1 Message Date
ed
887bfdc0a4 Don't forget to set sa->narg for CloudABI system calls.
It turns out that this value is not used within the system call code
under normal conditions, except when using tracing tools like ktrace.
If we forget to set this value, it is set to random garbage. This may
cause ktrace to hang indefinitely, making it impossible to kill.

Reported by: Michael Plass
PR: 210800
MFC before: 11.0-RELEASE
2016-07-08 20:09:21 +00:00
andrew
27c3442d81 Remove the old pre-INTRNG arm64 interrupt framework. GENERIC was switched
to INTRNG in r301565 with the old code no longer being built by default with
no reports of issues on any supported hardware.

Approved by:	re (gjb)
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-07-06 16:20:10 +00:00
wma
2da75cc60b ARM64: fix DMAP calculation
Use arithmetic operators instead of logical. This fixes
    DMAP ranges calculation for ThunderX Dual Socket.

    Obtained from:         Semihalf
    Sponsored by:          Cavium
    Reviewed by:           zbb
    Differential Revision: https://reviews.freebsd.org/D7023
    Approved by:           re (gjb)
2016-06-30 04:58:19 +00:00
andrew
4c327142ba Fix a race when the hardware has raised an exception with an unknown or
uncategorised reason. We need to read the fault address register before
enabling interrupts as the interrupt handler may cause this register to
change.

Approved by:	re (marius, kib)
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-06-22 12:05:08 +00:00
kib
496a3b1f65 Update comments for the MD functions managing contexts for new
threads, to make it less confusing and using modern kernel terms.

Rename the functions to reflect current use of the functions, instead
of the historic KSE conventions:
  cpu_set_fork_handler -> cpu_fork_kthread_handler (for kthreads)
  cpu_set_upcall -> cpu_copy_thread (for forks)
  cpu_set_upcall_kse -> cpu_set_upcall (for new threads creation)

Reviewed by:	jhb (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	re (hrs)
Differential revision:	https://reviews.freebsd.org/D6731
2016-06-16 12:05:44 +00:00
andrew
9fcae3cab6 Switch arm64 to use intrng by default. The old interrupt handling code can
still be used, however this is expected to be removed soon.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-06-07 20:14:08 +00:00
skra
612e6958e4 INTRNG - change the way how an interrupt mapping data are provided
to the framework in OFW (FDT) case.

This is a follow-up to r301451.

Differential Revision:	https://reviews.freebsd.org/D6634
2016-06-05 16:20:12 +00:00
andrew
a00165d816 Add the GICv3 ITS intrng driver. As the interface to the interrupt
framework has significantly changed the driver has moved to a new file.
While it shares some code with the existing driver this has been modified
to work better with the intrng framework.

This has been tested on the ThunderX servers in the netperf cluster and has
been used to boot them for other testing, including DTrace and hwpmc.

With this we can use intrng on all supported arm64 platforms I was able to
test on. It is expected we will move to intrng soon, and disable the old
arm64 interrupt framework.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6437
2016-06-03 10:28:06 +00:00
zbb
e284c69912 Return real error value instead of hard-coded ENXIO (fix after r300149)
It is possible to return real error value in case of gic_v3_attach()
failure instead of hard-coded ENXIO.

Obtained from:	Semihalf
Sponsored by:	Cavium
2016-06-01 08:20:10 +00:00
andrew
9d76db4d37 dpcpu_init should have also passed in the calculated cpuid, not the
devicetree ID.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-31 19:17:32 +00:00
andrew
05549901e3 Allow the kernel to boot on a CPU where the devicetree has numbered it with
a non-zero ID. To do this we increment the cpuid of any CPUs with a smaller
devicetree ID by one to stop them conflicting with the boot CPU.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-31 18:45:52 +00:00
andrew
ff690eb3c3 Enable setting BF_COHERENT on DMA tags. This allows the kernel to start
using the cache handling functions.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-31 16:28:56 +00:00
andrew
83cf90e9e3 Mark the ThunderX and generic PCI drivers as cache-coherent when we know
this to be the case. This will mean we don't try and handle the cache in
bus_dmamap_sync when it is not needed.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6605
2016-05-31 09:15:21 +00:00
andrew
1860a225ca Extract the correct bits from the GICD_TYPER register. The interrupt count
is encoded in the bottom 5 bits.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-20 15:43:51 +00:00
andrew
8dc7f1d8c7 Add more useful GICv3 register definitions. While here fix
GITS_CBASER_CACHE_MASK to use the correct shift macro.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-20 13:11:07 +00:00
andrew
bddd81e012 Filter out BUS_DMASYNC_POSTWRITE sync operations, there is nothing for us
to do on these.

Reported by:	wma
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-20 12:38:48 +00:00
andrew
81413fa2b4 Enable NEW_PCIB on arm64.
Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-20 08:43:18 +00:00
andrew
353b498d9d Handle PCI_RES_BUS on the generic and ThunderX PCIe drivers. This has been
tested on the Pass 1.1 and 2.0 ThunderX machines in the Netperf cluster.

Reviewed by:	jhb
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6453
2016-05-20 08:29:00 +00:00
andrew
659d635514 Define PCI_RES_BUS for NEW_PCIB
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-19 14:00:18 +00:00
andrew
5755bcaba9 Return the struct intr_pic pointer from intr_pic_register. This will be
needed in later changes where we may not be able to lock the pic list lock
to perform a lookup, e.g. from within interrupt context.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-18 15:05:44 +00:00
zbb
6fac2e64af Add support for MSI/MSIX deallocation on GICv3-ITS
Allow to deallocate previously allocated ITS device along with
its interrupts. Interrupt numbers are being freed when the last
LPI number is no longer busy.

Reviewed by:	wma
Obtained from:	Semihalf
Sponsored by:	Cavium
Differential Revision:	https://reviews.freebsd.org/D6351
2016-05-18 10:09:07 +00:00
bz
083ca1a6af The GIC (v2 at least) has a bit in the TYPER register to indicate whether the GIC
supports the Security Extensions or not. This bit is not the same as the CPU one.
Currently we are not checking for either before trying to write to the special
registers.  This can lead to problems on hardware or simulators that do not
provide the security extensions.  Add the missing checks. Their interactions with
the CPU flag is not entirely clear to me but using a macro will make it easier
to quickly adjust the condition once the CPU bits are sorted as well.

Reviewed by:	br
Sponsored by:	DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D6397
2016-05-17 13:12:26 +00:00
andrew
b4b3af68b5 Add an arm64 kernel config to help testing intrng. It is expected this
config will only last a few weeks until we switch to this interrupt
framework.

Obtained from:	 ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-17 12:48:30 +00:00
andrew
bc26af2c81 Clean up the GICv3 intrng code:
* In gic_v3_attach free the correct data on failure.
 * Implement gic_v3_teardown_intr.
 * Update the panic string when enabling/disabling an invalid interrupt.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-17 12:46:50 +00:00
andrew
e7850ad5c7 Add intrng support to the GICv3 driver. It lacks ITS support so won't handle
MSI or MSI-X interrupts, however this is enought to boot FreeBSD under the
ARM Foundation Model with a GICv3 interrupt controller.

Approved by:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-16 14:07:43 +00:00
andrew
04f04889ae Move the call to intr_pic_init_secondary to the same place as in the
non-intrng case.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 12:02:06 +00:00
andrew
58f81b7be8 Add support for intrng to arm64. As the GICv3 drivers will need to be
updated, and until further testing can be done, this is disabled for now.

It is expected arm64 will switch to this interface, and the old interface
will be removed before 11.0 is released.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-16 10:48:51 +00:00
andrew
26e3729fc0 Teach the ThunderX PCI PEM driver about intrng. This will be used later
when arm64 is supported by intrng.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 10:03:57 +00:00
andrew
cb60d23248 Add a pcib interface for use by interrupt controllers that need to
translate the pci rid to a controller ID. The translation could be based
on the 'msi-map' OFW property, a similar ACPI option, or hard-coded for
hardware lacking the above options.

Reviewed by:	wma
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 09:31:44 +00:00
andrew
297d2b0fcc Add support to the arm64 busdma to handle the cache. For now this is
disabled, however when we enable it it will default to assume memory is
not cache-coherent, unless either the tag was created or the parent was
marked as cache-coherent.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-13 16:03:50 +00:00
andrew
6d7e8f7834 Rename the internal BUC_DMA_* flags to BF_* so they won't conflict with
the flags in sys/bus_dma.h.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-12 15:24:36 +00:00
andrew
131c1c256e Restrict the memory barriers in bus_dmamap_sync to just the operations
where it's needed.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-12 12:13:28 +00:00
andrew
8c3616af7a Call busdma_swi from swi_vm as is done from other architectures.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 18:48:47 +00:00
andrew
4ec608a6fe On arm64 always create a bus_dmamap_t object. This will be use to hold the
list of memory that the kernel will need to sync when operating with a
non-cache coherent DMA engine.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 16:53:41 +00:00
andrew
c1c72529c8 Add data barriers to the arm64 bus_dmamap_sync function. We need these
to ensure ordering between the CPU and device. As the CPU and DMA target
may be in different shareability domains they need to be full system
barriers.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 14:59:54 +00:00
zbb
bc6d6fd49a Fix I/O coherence issues on ThunderX when SMP is disabled
To maintain coherence between cache and DMA memory appropriate
shareability flags need to be set in the PTE regardless of SMP
option.

Reviewed by:	wma
Obtained from:	Semihalf
Sponsored by:	Cavium
Differential Revision: https://reviews.freebsd.org/D6231
2016-05-11 13:23:56 +00:00
andrew
e08747edd2 Push the logic to talk with the MSI/MSI-X interrupt controller to the FDT
attachment. This is where it will live when we import intrng as it will
need to look at either the msi-parent or msi-map FDT properties.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-10 15:45:59 +00:00
jhb
eb663acb54 Native PCI-express HotPlug support.
PCI-express HotPlug support is implemented via bits in the slot
registers of the PCI-express capability of the downstream port along
with an interrupt that triggers when bits in the slot status register
change.

This is implemented for FreeBSD by adding HotPlug support to the
PCI-PCI bridge driver which attaches to the virtual PCI-PCI bridges
representing downstream ports on HotPlug slots. The PCI-PCI bridge
driver registers an interrupt handler to receive HotPlug events. It
also uses the slot registers to determine the current HotPlug state
and drive an internal HotPlug state machine. For simplicty of
implementation, the PCI-PCI bridge device detaches and deletes the
child PCI device when a card is removed from a slot and creates and
attaches a PCI child device when a card is inserted into the slot.

The PCI-PCI bridge driver provides a bus_child_present which claims
that child devices are present on HotPlug-capable slots only when a
card is inserted. Rather than requiring a timeout in the RC for
config accesses to not-present children, the pcib_read/write_config
methods fail all requests when a card is not present (or not yet
ready).

These changes include support for various optional HotPlug
capabilities such as a power controller, mechanical latch,
electro-mechanical interlock, indicators, and an attention button.
It also includes support for devices which require waiting for
command completion events before initiating a subsequent HotPlug
command. However, it has only been tested on ExpressCard systems
which support surprise removal and have none of these optional
capabilities.

PCI-express HotPlug support is conditional on the PCI_HP option
which is enabled by default on arm64, x86, and powerpc.

Reviewed by:	adrian, imp, vangyzen (older versions)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D6136
2016-05-05 22:26:23 +00:00
zbb
726fcfebab Fix GICv3 build after r299090
Obtained from:	Semihalf
Sponsored by:	Cavium
2016-05-05 17:51:14 +00:00
andrew
6cf149ba99 Disable ACPI on arm64 ad it has only had minimal testing and is causing
boot issues when booting with FDT. It is planned to re-enable this at a
later date.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-26 14:21:39 +00:00
br
778cc5a811 Move arm's devmap to some generic place, so it can be used
by other architectures.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D6091
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-04-26 11:53:37 +00:00
andrew
b40ffdceea Use the yield instruction in the arm64 cpu_spinwait. This instruction is
a hint to the hardware the software is not performing a task.

Sponsored by:	ABT Systems Ltd
2016-04-25 17:32:08 +00:00
pfg
729533413f sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
2016-04-21 19:57:40 +00:00
andrew
afb6bf1779 Group the ThunderX PCIe PEM newbus methods to help find them.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-20 14:12:40 +00:00
andrew
cb592fde3d Pull out the MSI/MSI-X handling calls to simplify future intrng
integration.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-20 13:23:06 +00:00
pfg
2056617741 arm: for pointers replace 0 with NULL.
These are mostly cosmetical, no functional change.

Found with devel/coccinelle.
2016-04-15 14:30:40 +00:00
andrew
474f44319d Fix the types for the start, end, and count arguments to
arm_gic_fdt_alloc_resource. These were the old u_long where they should be
rman_res_t. Both of these are the same size on arm64 so this is just for
correctness, and would not have led to incorrect behaviour.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-14 14:44:23 +00:00
andrew
6d3c93b78f Set the upper limit of the DMAP region to the limit of RAM as was found in
the physmap. This will reduce the likelihood of an issue where we have
device memory mapped in the DMAP. This can only happen if it is within the
same 1G block of normal memory.

Reviewed by:	kib
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5938
2016-04-14 10:43:28 +00:00
emaste
f015fe6ded arm64: Avoid null dereference in its_init_cpu
its_init_cpu() is called from gic_v3_init_secondary(), and its_sc will
be NULL if its did not attach.

Sponsored by:	The FreeBSD Foundation
2016-04-13 21:00:00 +00:00
andrew
fcf07e4e7b Document the memory ranges within the kernel region to help with debugging
to track down which region an address is from.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-13 11:43:03 +00:00