Commit Graph

405 Commits

Author SHA1 Message Date
Andrew Turner
4e1763d34c Teach the ThunderX PCI PEM driver about intrng. This will be used later
when arm64 is supported by intrng.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 10:03:57 +00:00
Andrew Turner
1e43b18c4b Add a pcib interface for use by interrupt controllers that need to
translate the pci rid to a controller ID. The translation could be based
on the 'msi-map' OFW property, a similar ACPI option, or hard-coded for
hardware lacking the above options.

Reviewed by:	wma
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-16 09:31:44 +00:00
Andrew Turner
5db4448f06 Add support to the arm64 busdma to handle the cache. For now this is
disabled, however when we enable it it will default to assume memory is
not cache-coherent, unless either the tag was created or the parent was
marked as cache-coherent.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-05-13 16:03:50 +00:00
Andrew Turner
f82b6f5255 Rename the internal BUC_DMA_* flags to BF_* so they won't conflict with
the flags in sys/bus_dma.h.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-12 15:24:36 +00:00
Andrew Turner
2b45fde379 Restrict the memory barriers in bus_dmamap_sync to just the operations
where it's needed.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-12 12:13:28 +00:00
Andrew Turner
9615213bef Call busdma_swi from swi_vm as is done from other architectures.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 18:48:47 +00:00
Andrew Turner
b7a7ee3af1 On arm64 always create a bus_dmamap_t object. This will be use to hold the
list of memory that the kernel will need to sync when operating with a
non-cache coherent DMA engine.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 16:53:41 +00:00
Andrew Turner
7ca01e8f89 Add data barriers to the arm64 bus_dmamap_sync function. We need these
to ensure ordering between the CPU and device. As the CPU and DMA target
may be in different shareability domains they need to be full system
barriers.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-11 14:59:54 +00:00
Zbigniew Bodek
46542fb652 Fix I/O coherence issues on ThunderX when SMP is disabled
To maintain coherence between cache and DMA memory appropriate
shareability flags need to be set in the PTE regardless of SMP
option.

Reviewed by:	wma
Obtained from:	Semihalf
Sponsored by:	Cavium
Differential Revision: https://reviews.freebsd.org/D6231
2016-05-11 13:23:56 +00:00
Andrew Turner
fb853e4270 Push the logic to talk with the MSI/MSI-X interrupt controller to the FDT
attachment. This is where it will live when we import intrng as it will
need to look at either the msi-parent or msi-map FDT properties.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-05-10 15:45:59 +00:00
John Baldwin
82cb5c3b5b Native PCI-express HotPlug support.
PCI-express HotPlug support is implemented via bits in the slot
registers of the PCI-express capability of the downstream port along
with an interrupt that triggers when bits in the slot status register
change.

This is implemented for FreeBSD by adding HotPlug support to the
PCI-PCI bridge driver which attaches to the virtual PCI-PCI bridges
representing downstream ports on HotPlug slots. The PCI-PCI bridge
driver registers an interrupt handler to receive HotPlug events. It
also uses the slot registers to determine the current HotPlug state
and drive an internal HotPlug state machine. For simplicty of
implementation, the PCI-PCI bridge device detaches and deletes the
child PCI device when a card is removed from a slot and creates and
attaches a PCI child device when a card is inserted into the slot.

The PCI-PCI bridge driver provides a bus_child_present which claims
that child devices are present on HotPlug-capable slots only when a
card is inserted. Rather than requiring a timeout in the RC for
config accesses to not-present children, the pcib_read/write_config
methods fail all requests when a card is not present (or not yet
ready).

These changes include support for various optional HotPlug
capabilities such as a power controller, mechanical latch,
electro-mechanical interlock, indicators, and an attention button.
It also includes support for devices which require waiting for
command completion events before initiating a subsequent HotPlug
command. However, it has only been tested on ExpressCard systems
which support surprise removal and have none of these optional
capabilities.

PCI-express HotPlug support is conditional on the PCI_HP option
which is enabled by default on arm64, x86, and powerpc.

Reviewed by:	adrian, imp, vangyzen (older versions)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D6136
2016-05-05 22:26:23 +00:00
Zbigniew Bodek
42638f9c5a Fix GICv3 build after r299090
Obtained from:	Semihalf
Sponsored by:	Cavium
2016-05-05 17:51:14 +00:00
Andrew Turner
ef6b515b69 Disable ACPI on arm64 ad it has only had minimal testing and is causing
boot issues when booting with FDT. It is planned to re-enable this at a
later date.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-26 14:21:39 +00:00
Ruslan Bukin
30b72b6871 Move arm's devmap to some generic place, so it can be used
by other architectures.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D6091
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-04-26 11:53:37 +00:00
Andrew Turner
ad76caf642 Use the yield instruction in the arm64 cpu_spinwait. This instruction is
a hint to the hardware the software is not performing a task.

Sponsored by:	ABT Systems Ltd
2016-04-25 17:32:08 +00:00
Pedro F. Giffuni
d9c9c81c08 sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
2016-04-21 19:57:40 +00:00
Andrew Turner
be95c24986 Group the ThunderX PCIe PEM newbus methods to help find them.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-20 14:12:40 +00:00
Andrew Turner
cf9c1ecfd2 Pull out the MSI/MSI-X handling calls to simplify future intrng
integration.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-20 13:23:06 +00:00
Pedro F. Giffuni
41b610a8ee arm: for pointers replace 0 with NULL.
These are mostly cosmetical, no functional change.

Found with devel/coccinelle.
2016-04-15 14:30:40 +00:00
Andrew Turner
a48bd01151 Fix the types for the start, end, and count arguments to
arm_gic_fdt_alloc_resource. These were the old u_long where they should be
rman_res_t. Both of these are the same size on arm64 so this is just for
correctness, and would not have led to incorrect behaviour.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-14 14:44:23 +00:00
Andrew Turner
f8a39033c2 Set the upper limit of the DMAP region to the limit of RAM as was found in
the physmap. This will reduce the likelihood of an issue where we have
device memory mapped in the DMAP. This can only happen if it is within the
same 1G block of normal memory.

Reviewed by:	kib
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5938
2016-04-14 10:43:28 +00:00
Ed Maste
6b580fd720 arm64: Avoid null dereference in its_init_cpu
its_init_cpu() is called from gic_v3_init_secondary(), and its_sc will
be NULL if its did not attach.

Sponsored by:	The FreeBSD Foundation
2016-04-13 21:00:00 +00:00
Andrew Turner
0b4890b034 Document the memory ranges within the kernel region to help with debugging
to track down which region an address is from.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-13 11:43:03 +00:00
Andrew Turner
9853d10411 Increase the arm64 kernel address space to 512GB, and the DMAP region to
2TB. The latter can be increased in 512GB chunks by adjusting the lower
address, however more work will be needed to increase the former.

There is still some work needed to only create a DMAP region for the RAM
address space as on ARM architectures all mappings should have the same
memory attributes, and these will be different for device and normal memory.

Reviewed by:	kib
Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5859
2016-04-13 09:44:32 +00:00
Pedro F. Giffuni
74b8d63dcc Cleanup unnecessary semicolons from the kernel.
Found with devel/coccinelle.
2016-04-10 23:07:00 +00:00
Zbigniew Bodek
017b6ebc0a Fix interrupts delivery on ThunderX for VF IDs beyond 8
SR-IOV devices usually use Alternative Routing ID (ARI).
In that case slot/device is always assumed to be 0 and
function/identifier is extended to 8 bits.

Fix interrupts delivery to VF IDs beyond 8 by using a correct
DevID if ARI is enabled.

Reviewed by:   jhb, wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5855
2016-04-07 10:36:50 +00:00
Andrew Turner
5d58666c93 Use PHYS_IN_DMAP to check if a physical address is within the DMAP region.
Approved by:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-06 14:16:37 +00:00
Andrew Turner
70e7278593 Cleanup the early pagetable creation code in preperation for increasing
the size of the arm64 DMAP region.

Approved by:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-06 14:12:00 +00:00
Andrew Turner
c7d4b461b3 Allow vmparam.h to be included from assembly files on arm64.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-06 14:08:10 +00:00
Ed Schouten
ab83575070 Make CloudABI's way of doing TLS more friendly to userspace emulators.
We're currently seeing how hard it would be to run CloudABI binaries on
operating systems cannot be modified easily (Windows, Mac OS X). The
idea is that we want to just run them without any sandboxing. Now
that CloudABI executables are PIE, this is already a bit easier, but TLS
is still problematic:

- CloudABI executables want to write to the %fs, which typically
  requires extra system calls by the emulator every time it needs to
  switch between CloudABI's and its own TLS.

- If CloudABI executables overwrite the %fs base unconditionally, it
  also becomes harder for the emulator to store a backup of the old
  value of %fs. To solve this, let's no longer overwrite %fs, but just
  %fs:0.

As CloudABI's C library does not use a TCB, this space can now be used
by an emulator to keep track of its internal state. The executable can
now safely overwrite %fs:0, as long as it makes sure that the TCB is
copied over to the new TLS area.

Ensure that there is an initial TLS area set up when the process starts,
only containing a bogus TCB. We don't really care about its contents on
FreeBSD.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D5836
2016-04-06 11:11:31 +00:00
Wojciech Macek
1c7c13aa0e Implement dtrace_getupcstack in ARM64
Allow using DTRACE for performance analysis of userspace
applications - the function call stack can be captured.
This is almost an exact copy of AMD64 solution.

Obtained from:         Semihalf
Sponsored by:          Cavium
Reviewed by:           emaste, gnn, jhibbits
Differential Revision: https://reviews.freebsd.org/D5779
2016-04-06 05:13:36 +00:00
Andrew Turner
53b832b091 Add a table to map from the FreeBSD CPUID space to the GIC CPUID space. On
many SoCs these two are the same, however there is no requirement for this
to be the case, e.g. on the ARM Juno we boot on what the GIC thinks of as
CPU 2, but FreeBSD numbers it CPU 0.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-04 17:04:33 +00:00
Andrew Turner
bc5a80161c Reduce the diff for when we switch to intrng. The IPI interrupts will be
split out to multiple handlers.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-04-04 15:13:17 +00:00
Wojciech Macek
4d1dd74a50 arm64: pagezero improvement
This change has been provided to improve pagezero call performance.

Submitted by:          Dominik Ermel <der@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          Cavium
Reviewed by:           kib
Differential Revision: https://reviews.freebsd.org/D5741
2016-04-04 07:16:43 +00:00
Wojciech Macek
73ffb5e8a4 Add bzero.S to ARM64 machdep
Add fille missing from https://svnweb.freebsd.org/changeset/base/297536
2016-04-04 07:11:33 +00:00
Wojciech Macek
db27818234 arm64: bzero optimization
This optimization attempts to utylize as wide as possible register store instructions to zero large buffers.
The implementation, if possible, will use 'dc zva' to zero buffer by cache lines.

Speedup: 60x faster memory zeroing

Submitted by:          Dominik Ermel <der@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          Cavium
Reviewed by:           kib
Differential Revision: https://reviews.freebsd.org/D5726
2016-04-04 07:06:20 +00:00
Ed Schouten
4a8b3b18cc Make Position Independent Executables work for CloudABI.
- Set BI_CAN_EXEC_DYN, so we can execute ET_DYN ELF files in addition to
  regular ET_EXECs.
- Provide an AT_BASE entry in the auxiliary vector, so the executable
  knows at which address it got loaded and can apply relocations.
2016-03-31 18:52:00 +00:00
Andrew Turner
f2f21faf62 Add support for 4 level pagetables. The userland address space has been
increased to 256TiB. The kernel address space can also be increased to be
the same size, but this will be performed in a later change.

To help work with an extra level of page tables two new functions have
been added, one to file the lowest level table entry, and one to find the
block/page level. Both of these find the entry for a given pmap and virtual
address.

This has been tested with a combination of buildworld, stress2 tests, and
by using sort to consume a large amount of memory by sorting /dev/zero. No
new issues are known to be present from this change.

Reviewed by:	kib
Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5720
2016-03-31 11:07:24 +00:00
Andrew Turner
6c5b1ed4b6 Read the CPU ID for the current CPU from the GIC. The GIC may have a
different ID space than the kernel. Because of this we need to read the
ID from the hardware. The hardware will provide this value to the CPU by
reading any of the first 8 Interrupt Processor Targets Registers.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5706
2016-03-29 13:51:26 +00:00
Wojciech Macek
f379b4636a arm64: Fixing user space boudary checking in copyinout.S
Big buffer size could cause integer overflow and as a result
attempt to copy beyond VM_USERMAX_ADDRESS.

Fixing copyinstr boundary checking where compared value has been
overwritten by accident when setting fault handler.

Submitted by:          Dominik Ermel <der@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          Cavium
Reviewed by:           kib
Differential Revision: https://reviews.freebsd.org/D5719
2016-03-24 13:28:33 +00:00
Wojciech Macek
f3e730a1e5 ARM64 copyinout improvements
The first of set of patches.
Use wider load/stores when aligned buffer is being copied.

In a simple test:
  dd if=/dev/zero of=/dev/null bs=1M count=1024
the performance jumped from 410MB/s up to 3.6GB/s.

TODO:
 - better handling of unaligned buffers (WiP)
 - implement similar mechanism to bzero

Submitted by:          Dominik Ermel <der@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          Cavium
Reviewed by:           kib, andrew, emaste
Differential Revision: https://reviews.freebsd.org/D5664
2016-03-23 13:29:52 +00:00
Andrew Turner
f6c7371c81 Use the saved program state register to detect when an exception frame is
from userpsace. Previously we could have triggered a panic by trying to
jump to a kernel address from userland as the trap handling code thought we
received an ast in kernel mode.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-03-22 08:36:25 +00:00
Andrew Turner
b011fce09f Move the opt_ files to be included first so their definitions can be used
from within all further included files.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-03-18 16:32:22 +00:00
Andrew Turner
a9056bbb93 Rename COUNT_IPI to INTR_IPI_COUNT to reduce the diff with intrng.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-03-18 16:29:58 +00:00
Andrew Turner
4d00d27b3a Reduce the diff with intrng by renaming similar functions. This is a noop,
but will help move to use the common interrupt handling code later.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-03-18 16:18:29 +00:00
Andrew Turner
7285efe8cd Remove the invalid L0_BLOCK definition. ARMv8 doesn't support block
translation in the level 0 descriptor.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-03-18 10:01:25 +00:00
Wojciech Macek
f54153bb08 pmap arm64: fixing pmap_invalidate_range
It seems that if range within one page is given this page will not be
invalidated at all. Clean it up.

Submitted by:          Dominik Ermel <der@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          Cavium
Reviewed by:           wma, zbb
Approved by:           cognet (mentor)
Differential Revision: https://reviews.freebsd.org/D5569
2016-03-14 07:26:38 +00:00
John Baldwin
6fc8053f1a Fix reporting of the CloudABI ABI in kdump.
- Advertise the word size for CloudABI ABIs via the SV_LP64 flag.  All of
  the other ABIs include either SV_ILP32 or SV_LP64.
- Fix kdump to not assume a 32-bit ABI if the ABI flags field is non-zero
  but SV_LP64 isn't set.  Instead, only assume a 32-bit ABI if SV_ILP32 is
  set and fallback to the unknown value of "00" if neither SV_LP64 nor
  SV_ILP32 is set.

Reviewed by:	kib, ed
Differential Revision:	https://reviews.freebsd.org/D5560
2016-03-09 18:38:30 +00:00
Bjoern A. Zeeb
3d9ac4ecfc Force re-routing PCI interrupts (this is for legacy INTx not MSI).
Need this for gem5, but was not needed on real hadrware (yet) as it
was always MSI.

Reviewed by:		andrew, jhb
Discovered by:		andrew
Sponsored by:		DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D5494
2016-03-02 15:20:42 +00:00
Wojciech Macek
52ea10af69 Improve ThunderX PEM driver to work on pass2 revision
Things changed:
    * do not allocate 4GB of SLI space, because it's the waste of
      system resources. Allocate only small portions when needed.
    * provide own implementation of activate_resource which performs
      address translation between PCI bus and host PA address space.
      This is temporary solution, should be replaced by bus_map_resource
      once implemented.

Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           jhb
Differential revision: https://reviews.freebsd.org/D5294
2016-03-02 08:39:59 +00:00
Wojciech Macek
a5d461a398 Get memory ranges from FDT if no EFI API is available on ARM64
Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Approved by:           cognet (mentor)
Reviewed by:           andrew, wma
Differential revision: https://reviews.freebsd.org/D5408
2016-03-01 12:50:24 +00:00
Wojciech Macek
b2552c46b6 Enable SRE_EL2 on ARM64
Enable system register access for EL2. Alpine-V2 is
the first device requiring this to be enabled.
It is also in-sync with Linux initialization code,
and compatible with Alpine-V2 uboot requirements.

Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Approved by:           cognet (mentor)
Reviewed by:           wma
Differential revision: https://reviews.freebsd.org/D5394
2016-03-01 08:15:00 +00:00
Wojciech Macek
30d9287468 Add uart 8250 device to GENERIC arm64 configuration
Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Approved by:           cognet (mentor)
Reviewed by:           zbb, wma
Differential revision: https://reviews.freebsd.org/D5406
2016-03-01 07:06:36 +00:00
Justin Hibbits
e665eafb25 Correct the memory rman ranges to be to BUS_SPACE_MAXADDR
Summary:
As part of the migration of rman_res_t to be typed to uintmax_t, memory ranges
must be clamped appropriately for the bus, to prevent completely bogus addresses
from being used.

This is extracted from D4544.

Reviewed By: cem
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5134
2016-03-01 02:59:06 +00:00
Wojciech Macek
2445e7c84e Restore ThunderX Pass1.1 PCI changes removed by r295962
If Enhanced Allocation is not used, we can't allocate any random
    range. All internal devices have hardcoded place where they can
    be located within PCI address space. Fortunately, we can read
    this value from BAR.

Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb
Differential revision: https://reviews.freebsd.org/D5455
2016-02-26 12:16:11 +00:00
Wojciech Macek
fb05500b24 Make pci_host_generic and thunderx_pci common
* provided OFW interface for pci_host_generic (for handling devices which are present in DTS under the PCI node)
  * removed support for internal PCI from arm64/cavium
  * cleaned up and made most of the code common

Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb
Differential revision: https://reviews.freebsd.org/D5261
2016-02-24 06:05:30 +00:00
Wojciech Macek
0422bce8b2 Add Intel 10Gb support to ARM64 GENERIC kernel config
Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb
Differential revision: https://reviews.freebsd.org/D5347
2016-02-22 13:34:43 +00:00
Svatopluk Kraus
35a0bc1260 As <machine/vmparam.h> is included from <vm/vm_param.h>, there is no
need to include it explicitly when <vm/vm_param.h> is already included.

Suggested by:	alc
Reviewed by:	alc
Differential Revision:	https://reviews.freebsd.org/D5379
2016-02-22 09:08:04 +00:00
Svatopluk Kraus
d6849317c5 As <machine/param.h> is included from <sys/param.h>, there is no need
to include it explicitly when <sys/param.h> is already included.

Reviewed by:	alc, kib
Differential Revision:	https://reviews.freebsd.org/D5378
2016-02-22 09:04:36 +00:00
Svatopluk Kraus
a1e1814d76 As <machine/pmap.h> is included from <vm/pmap.h>, there is no need to
include it explicitly when <vm/pmap.h> is already included.

Reviewed by:	alc, kib
Differential Revision:	https://reviews.freebsd.org/D5373
2016-02-22 09:02:20 +00:00
Justin Hibbits
7915adb560 Introduce a RMAN_IS_DEFAULT_RANGE() macro, and use it.
This simplifies checking for default resource range for bus_alloc_resource(),
and improves readability.

This is part of, and related to, the migration of rman_res_t from u_long to
uintmax_t.

Discussed with:	jhb
Suggested by:	marcel
2016-02-20 01:32:58 +00:00
Zbigniew Bodek
b998c9656b Introduce bus_get_bus_tag() method
Provide bus_get_bus_tag() for sparc64, powerpc, arm, arm64 and mips
nexus and its children in order to return a platform specific default tag.

This is required to ensure generic correctness of the bus_space tag.
It is especially needed for arches where child bus tag does not match
the parent bus tag. This solves the problem with ppc architecture
where the PCI bus tag differs from parent bus tag which is big-endian.

This commit is a part of the following patch:
https://reviews.freebsd.org/D4879

Submitted by:  Marcin Mazurek <mma@semihalf.com>
Obtained from: Semihalf
Sponsored by:  Annapurna Labs
Reviewed by:   jhibbits, mmel
Differential Revision: https://reviews.freebsd.org/D4879
2016-02-18 13:00:04 +00:00
Wojciech Macek
98bc9384c5 Fix ThunderX external PEM bus offset
Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb
Differential revision: https://reviews.freebsd.org/D5293
2016-02-18 11:26:08 +00:00
Svatopluk Kraus
1413a3ab64 Remove pd_prot and pd_cache members from struct arm_devmap_entry.
The struct is used for definition of static device mappings which
should always have same protection and attributes.
2016-02-17 12:36:24 +00:00
Andrew Turner
45fd186285 Allow callers of OF_decode_addr to get the size of the found mapping. This
will allow for code that uses the old fdt_get_range and fdt_regsize
functions to find a range, map it, access, then unmap to replace this, up
to and including the map, with a call to OF_decode_addr.

As this function should only be used in the early boot code the unmap is
mostly do document we no longer need the mapping as it's a no-op, at least
on arm.

Reviewed by:	jhibbits
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D5258
2016-02-16 15:18:12 +00:00
Zbigniew Bodek
9ccaab6db5 Support PEM that is not a PCI endpoint on ThunderX
Some chip revisions don't have their external PCIe buses
behind the internal bridge. Add support for FDT-configurable
PEMs but keep ability for PCIe enumeration.

Reviewed by:   andrew, wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5285
2016-02-16 11:43:57 +00:00
Andrew Turner
770fd1c976 Only update curthread and curpcb after we have finished using the old
values.

If switching from a thread that used floating-point registers to a thread
that is still running, but holding the blocked_lock lock we would switch
the curthread to the new (running) thread, then call critical_enter. This
will non-atomically increment td_critnest, and later call critical_exit to
non-atomically decrement this value.

This can happen at the same time as the new thread is still running on the
old core, also calling these functions. In this case there will be a race
between these non-atomic operations. This can be an issue as we could loose
one of these operations leading to the value to not return to zero.

If, later on, we then hit a data abort we check if the td_critnest is zero.
If this check fails we will panic the kernel.

This has been observed when running pcmstat on a Cavium ThunderX. The pcm
thread will use the blocked_lock lock and there is a high chance userspace
will use the floating-point registers. When, later on, pmcstat triggers a
data abort we will hit this panic.

The fix is to update these values after storing the floating-point state.
This means we use the correct curthread while storing the state so it will
not be an issue that the changes to td_critnest are non-atomic.

Sponsored by:	ABT Systems Ltd
2016-02-12 12:38:04 +00:00
Zbigniew Bodek
6cd36342c0 Support interrupts binding in GICv3 and ITS
- Add MOVI command and routine for the LPI migration
- Allow to search for the ITS device descriptor using
  not only devID but also LPI number.
- Bind SPIs in the Distributor
- Don't bind its_dev to collection. Keep track of the collection
  IDs for each LPI.

Reviewed by:   wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5231
2016-02-11 12:04:58 +00:00
Zbigniew Bodek
907a0579aa Implement finer locking in ITS
- Change locks' names to be more suitable
- Don't use blocking mutex. Lock only basic operations such
  as lists or bitmaps modifications.

Reviewed by:   wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5230
2016-02-11 12:03:11 +00:00
Zbigniew Bodek
47a1ff355e Initially bind all interrupts to the boot CPU when using GICv3
This should be done by routing all interrupts to CPU0,
different assignment will be induced by either interrupts
shuffling or bus_bind_intr().

Reviewed by:   wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5229
2016-02-11 12:01:33 +00:00
Zbigniew Bodek
55bdcadded Call pmc_hook() correctly in the ARM64 interrupt handler
pmc_hook() was called only in case of the stray interrupt but should
rather be called on each interrupt. Move in to the arm_cpu_intr()
handler, out of the critical section too.

Reviewed by:   br
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5161
2016-02-11 11:59:32 +00:00
Zbigniew Bodek
be7aab76ec Introduce bus_bind_intr method for ARM64
It can be used to bind specific interrupt to a particular CPU.
Requires PIC support for interrupts binding.

Reviewed by:   wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5122
2016-02-11 11:58:27 +00:00
Zbigniew Bodek
513411c9f5 Fix bugs in interrupts allocation on ARM64
Separate interrupt descriptors lookup from allocation. It was possible
to perform config on non-existing interrupt simply by allocating spurious
descriptor.
Must lock the interrupt descriptors table lookup to avoid mismatches.
This ought to prevent trouble while setting up new interrupt
and dispatching existing one.
Use spin mutex rather than sleep mutex. This is mainly due to lock in
arm_dispatch_intr.
This should be eventually changed to a lock-less solution without
walking through a linked list on each interrupt.

Reviewed by:   andrew, wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5121
2016-02-11 11:57:13 +00:00
Zbigniew Bodek
8133eda921 Minor clean-ups for ARM64 GICv3 and GIC drivers
GICv3:
- move ICC_SGI1R_EL1 definitions to armreg.h and use proper system
  register's names
GIC:
- remove unused functions

Reviewed by:   andrew
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5119
2016-02-11 11:55:37 +00:00
Wojciech Macek
c7fc655f3f ARM64 disassembler: support for LDR instructions
Implemented disassembly for a whole bunch of
    various ldr instructions.

Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb
Differential revision: https://reviews.freebsd.org/D5217
2016-02-11 06:50:11 +00:00
Andrew Turner
d664515e68 Revert an arm64 change that sneaked in with r295464. 2016-02-10 10:28:33 +00:00
Andrew Turner
907fe11655 Update of the Allwinner drivers to:
* Use the Linux compat string
 * Use EARLY_DRIVER_MODULE to attach at the right time
 * Add a generic A10 kernel config file
 * A20 now use generic_timer
 * Add two new dts files for Olimex boards
 * Update our custom DTS file for A10 and A20 to use the same compatible
   property names as the vendor ones.

Submitted by:	Emmanuel Vadot <manu@bidouilliste.com>
Differential Revision:	https://reviews.freebsd.org/D4792
2016-02-10 09:19:29 +00:00
Gleb Smirnoff
b28cc462ad Include sys/_task.h into uma_int.h, so that taskqueue.h isn't a
requirement for uma_int.h.

Suggested by:	jhb
2016-02-09 20:22:35 +00:00
Andrew Turner
f75ef22a66 Use designated initialisers for the db disassembler interface to help with
finding the struct when searching for one of its members.
2016-02-09 10:18:22 +00:00
Wojciech Macek
c00a03a11d Ignore invalid page descriptors in ARM64 pmap_mincore
Prevent the function from null-pointer-dereference when unexisting
    mapping is being processed.

Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb, cognet
Differential revision: https://reviews.freebsd.org/D5228
2016-02-09 06:26:27 +00:00
Andrew Turner
55beb2a538 Implement kdb_cpu_sync_icache on arm64.
Sponsored by:	ABT Systems Ltd
2016-02-05 15:38:28 +00:00
Andrew Turner
f13ec4b40e Enable checking of the stack alignment. The stack should be aligned to a
16-byte value. With this the hardware will check if a memory access uses
an incorrectly aligned stack pointer as the base address.

Sponsored by:	ABT Systems Ltd
2016-02-04 17:22:15 +00:00
Ruslan Bukin
46c9b07f22 Fix build. 2016-02-04 11:52:53 +00:00
Gleb Smirnoff
6c95c7903c Fix build. 2016-02-04 03:55:41 +00:00
Andrew Turner
729ac0ee8c Handle a misaligned stack pointer exception from userspace. The exception
still needs to be enabled, but this will help with testing.

Sponsored by:	ABT Systems Ltd
2016-02-03 17:00:19 +00:00
Andrew Turner
3df911720b Increase the size of PHYS_AVAIL_SIZE to allow firmware to provide a large
number of physical memory locations we can access. This is the case on
some HiKey boards that may have UEFI reserved memory dispersed through the
physical space.

Sponsored by:	ABT Systems Ltd
2016-02-02 17:59:43 +00:00
Andrew Turner
332b343376 Ensure we don't overflow the phys_avail array. Some firmware may provide
more memory locations than we have space to record.

Sponsored by:	ABT Systems Ltd
2016-02-02 17:57:15 +00:00
Andrew Turner
50d9282660 Increase the space we use after the kernel to 8MiB. On 2GiB HiKey board we
would try to access data past this point stopping the boot.

Sponsored by:	ABT Systems Ltd
2016-02-02 16:35:37 +00:00
Andrew Turner
2c0ed87ef8 Only look for the ranges property when we have children. This fixes booting
on systems with a gicv2, but no PCIe so no gicv2m.

Sponsored by:	ABT Systems Ltd
2016-02-02 16:32:44 +00:00
Andrew Turner
87e19994e1 Implement single stepping on arm64. We need to set the single step bits in
the processor and debug state registers. A flag has been added to the pcb
to tell us when to enable single stepping for a given thread.

Reviewed by:	kib
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D4730
2016-02-02 10:28:56 +00:00
Andrew Turner
5df33bf4df Correctly handle the case where copystr(9) is given a string longer than
the passed in length. In this case we need to return ENAMETOOLONG.
2016-02-02 10:11:56 +00:00
Zbigniew Bodek
4111e0a100 Fix sending IPI to all CPUs on ARM64
There is no explanation why IPI ID is incremented here by "16".
This should have been removed in r285533 but somehow survived.

Reviewed by:   wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5120
2016-01-29 18:43:51 +00:00
Wojciech Macek
8a1867f4aa Framework for ARM64 instruction disassembler
Provide an easy to use framework for ARM64 DDB disassembler.
    This commit does not contain full list of instruction opcodes.

Obtained from:         Semihalf
Sponsored by:          Cavium
Approved by:           cognet (mentor)
Reviewed by:           zbb, andrew, cognet
Differential revision: https://reviews.freebsd.org/D5114
2016-01-29 13:06:30 +00:00
Zbigniew Bodek
db7cfc199e Fix VNIC enumeration after r294993 and r294990
ofw_bus_get_node() must be tested against negative values since
missing parent bus method will result in calling the default method
which simply returns (-1): sys/dev/ofw/ofw_bus_if.m
This was lost in the review process.

Obtained from: Semihalf
Sponsored by:  Cavium
2016-01-28 16:58:49 +00:00
Zbigniew Bodek
68a594774e Add FDT bus capabilities to ThunderX PCI driver
New ThunderX firmware incorporates modified DTB that presents
different device hierarchy. In the new device tree, MDIO
devices are below two additional buses that oddly hang on
PCI bridge.

Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5069
2016-01-28 15:40:56 +00:00
Zbigniew Bodek
88c5cdf401 Correct alloc_ and release_resource methods in thunder_pcie driver
- Avoid using BUS_ macros as bus_generic_ functions should be used instead.
- Fix mistaken device_t pointers in thunder_pcie_alloc_resource.
  Should use dev->parent method and allocate resource for child device

Reviewed by:   wma
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5068
2016-01-28 15:38:02 +00:00
Zbigniew Bodek
bc5758b633 Divide ThunderX PCIe driver to general and FDT part
- Separate FDT and general PCIe driver parts
- Drop some irrelevant printfs that cannot be displayed in
  FDT attach
- Move ranges parsing to FDT portion of PCIe code

Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5067
2016-01-28 15:34:13 +00:00
Wojciech Macek
8eb447cc45 Fix mutex releasing in ARM64 cpu_switch
The code should be comparing pointers, not any data
    gathered from a blocked_lock.

Spotted by:            cognet
Approved by:           zbb, cognet (mentor)
Differential revision: https://reviews.freebsd.org/D5100
2016-01-28 12:00:17 +00:00
John Baldwin
aa949be551 Convert ss_sp in stack_t and sigstack to void *.
POSIX requires these members to be of type void * rather than the
char * inherited from 4BSD.  NetBSD and OpenBSD both changed their
fields to void * back in 1998.  No new build failures were reported
via an exp-run.

PR:		206503 (exp-run)
Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D5092
2016-01-27 17:55:01 +00:00
Justin Hibbits
2dd1bdf183 Convert rman to use rman_res_t instead of u_long
Summary:
Migrate to using the semi-opaque type rman_res_t to specify rman resources.  For
now, this is still compatible with u_long.

This is step one in migrating rman to use uintmax_t for resources instead of
u_long.

Going forward, this could feasibly be used to specify architecture-specific
definitions of resource ranges, rather than baking a specific integer type into
the API.

This change has been broken out to facilitate MFC'ing drivers back to 10 without
breaking ABI.

Reviewed By: jhb
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5075
2016-01-27 02:23:54 +00:00
Zbigniew Bodek
004ae5cbbd Simplify GICv3 related drivers' naming
Rename gic_v3_ instances to simply use 'gic' and 'its'.
The information about the controller's revision is printed
in the device announcement during boot anyway.
The intention behind this change is to avoid somewhat misleading
GIC instances naming such as:
    gic_v30
    gic_v31
    ...
etc.

Submitted by:  Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5016
2016-01-25 15:18:32 +00:00
Zbigniew Bodek
7ea5004ba7 Create proper FDT attachment for GICv2m
Avoid probing GICv2m to any parent bus/driver. Instead, match
GICv2m driver with FDT complatible strings as not every GIC
has a MSI controller in the form of GICv2m extension.

Submitted by:  Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by:  Cavium
Differential Revision: https://reviews.freebsd.org/D5015
2016-01-25 15:10:43 +00:00
Andrew Turner
60f9d31cd3 Stop including fdt_common.h in the arm64 code. We don't use anything from
it, however may have relied on header pollution to pull in the needed
headers through it

Sponsored by:	ABT Systems Ltd
2016-01-22 16:35:01 +00:00
Zbigniew Bodek
16aec470eb Revert r294267 to avoid using experimental VFS_AIO in ARM64's GENERIC
Remove VFS_AIO from the ARM64's GENERIC as it can be used
as a loadable module.
2016-01-20 11:34:22 +00:00
Zbigniew Bodek
107ef92ffe Enable AIO interface on ARM64 platforms
Add VFS_AIO to generic config to allow using of high-performance
asynchronous disk AIO operation.

Reviewed by:   imp
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4979
2016-01-18 14:11:34 +00:00
Zbigniew Bodek
6f675c9d20 Update ThunderX PCIe driver to fit new DTS layout
In recent EFI the DTS entries changed for PCIe controller.
This commit fixes internal PCIe, external is yet TBD.

Submitted by:  Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4976
2016-01-18 13:31:29 +00:00
Dmitry Chagin
038c720553 Implement vsyscall hack. Prior to 2.13 glibc uses vsyscall
instead of vdso. An upcoming linux_base-c6 needs it.

Differential Revision:  https://reviews.freebsd.org/D1090

Reviewed by:	kib, trasz
MFC after:	1 week
2016-01-09 20:18:53 +00:00
Ian Lepore
a195a8e177 Fix the arm64 build by adding an all-important '&' to get a pointer.
I'm not sure how I missed the error when I test-built here, I guess the
pointy hat must have slipped down over my eyes.
2016-01-02 21:13:14 +00:00
Nathan Whitehorn
9f4a7eae43 Make using the #address-cells property on the interrupt parent in device
tree parsing opt-out rather than opt-in. All FDT-based systems as well as
PowerPC systems with real Open Firmware use the CHRP-derived binding that
includes it, which makes SPARC the odd man out here. Making it opt-out
avoids astonishment on new platform bring up.
2016-01-02 19:28:35 +00:00
Ian Lepore
002ef20ca8 Add an OF_decode_addr() implementation for arm64.
Discussed with:		andrew
2016-01-02 19:14:19 +00:00
Ian Lepore
69dcb7e771 Make the 'env' directive described in config(5) work on all architectures,
providing compiled-in static environment data that is used instead of any
data passed in from a boot loader.

Previously 'env' worked only on i386 and arm xscale systems, because it
required the MD startup code to examine the global envmode variable and
decide whether to use static_env or an environment obtained from the boot
loader, and set the global kern_envp accordingly.  Most startup code wasn't
doing so.  Making things even more complex, some mips startup code uses an
alternate scheme that involves calling init_static_kenv() to pass an empty
buffer and its size, then uses a series of kern_setenv() calls to populate
that buffer.

Now all MD startup code calls init_static_kenv(), and that routine provides
a single point where envmode is checked and the decision is made whether to
use the compiled-in static_kenv or the values provided by the MD code.

The routine also continues to serve its original purpose for mips; if a
non-zero buffer size is passed the routine installs the empty buffer ready
to accept kern_setenv() values.  Now if the size is zero, the provided buffer
full of existing env data is installed.  A NULL pointer can be passed if the
boot loader provides no env data; this allows the static env to be installed
if envmode is set to do so.

Most of the work here is a near-mechanical change to call the init function
instead of directly setting kern_envp.  A notable exception is in xen/pv.c;
that code was originally installing a buffer full of preformatted env data
along with its non-zero size (like mips code does), which would have allowed
kern_setenv() calls to wipe out the preformatted data.  Now it passes a zero
for the size so that the buffer of data it installs is treated as
non-writeable.
2016-01-02 02:53:48 +00:00
Andrew Turner
5f0a5fefc6 Decode and print the ID_AA64* registers on boot. These registers hold
information on what the core supports. In most cases these will be
identical across most CPUs in the SoC, however there may be the case where,
with a big.LITTLE setup they may differ. In this case we print the
decoded data on all CPUs.

Reviewed by:	kib
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D4725
2015-12-30 17:36:34 +00:00
Andrew Turner
91569f1f38 Increase the size and alignment of the setjmp buffer. This will allow for
possible future CPU extentions with larger registers.

jmp_buf's size and alignment are baked into the ABI of third party libraries
and thus are very hard to change later so it is best to waste a small amount
of space now.

Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D3956
2015-12-23 15:22:44 +00:00
Andrew Turner
122493cf95 Support the variant of the interrupt-map property where the parent bus has
the #address-cells property set. For this we need to read more data before
the parent interrupt description.

this is only enabled on arm64 for now as it's not quite compliant with the
ePAPR spec. We should use a default of 2 where the #address-cells property
is missing, however this will need further testing across architectures.

Obtained from:	ABT Systems Ltd
Sponsored by:	SoftIron Inc
Differential Revision:	https://reviews.freebsd.org/D4518
2015-12-17 17:00:04 +00:00
Andrew Turner
68fe2d8f70 Add support for MSI interrupts to the gicv2m controller. The allocation
is still quite simplistic, it just increments a counter to use the next
interrupt.

Obtained from:	ABT Systems Ltd
Sponsored by:	SoftIron Inc
2015-12-14 16:07:20 +00:00
Andrew Turner
90257a2b6c Skip restoring more registers when returning from an exception taken in
the kernel. These registers are all callee saved, and as such will be
restored before returning to the exception handler.

Userland still needs these registers to be restored as they may be changed
by the kernel and we don't currently track these places.
2015-12-14 10:06:01 +00:00
Andrew Turner
a38fc8ca6a Sort the list of NICs after the mii options. While here add the msk driver
as it has now been tested.

Sponsored by:	SoftIron Inc
2015-12-11 15:25:47 +00:00
Andrew Turner
4b771f1b99 Add support for the GICv2M extension to the GICv2 interrupt controller.
This is (oddly) specified in the ARM Server Base System Architecture. It
extends the GICv2 to support MSI and MSI-X interrupts, however only the
latter are currently supported.

Only the FDT attachment is currently supported, however the attachment
and core driver are split to help adding ACPI support in the future.

Obtained from:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	SoftIron Inc
2015-12-10 16:40:38 +00:00
Andrew Turner
6d8433cb00 Remove pic_map_msix from ARM64, it is unneeded as all mappings happen
through pic_map_msi (without the x).

Sponsored by:	ABT Systems Ltd
2015-12-10 15:51:02 +00:00
Andrew Turner
c1d5e7a1f1 Reduce the numbers of levels of indirection in arm64 pcib drivers by making
the MSI & MSI-X handler functions be directly callable from the driver
methods.

Sponsored by:	ABT Systems Ltd
2015-12-10 13:19:30 +00:00
Konstantin Belousov
4d22d07a07 Add support for usermode (vdso-like) gettimeofday(2) and
clock_gettime(2) on ARMv7 and ARMv8 systems which have architectural
generic timer hardware. It is similar how the RDTSC timer is used in
userspace on x86.

Fix a permission problem where generic timer access from EL0 (or
userspace on v7) was not properly initialized on APs.

For ARMv7, mark the stack non-executable. The shared page is added for
all arms (including ARMv8 64bit), and the signal trampoline code is
moved to the page.

Reviewed by:	andrew
Discussed with:	emaste, mmel
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D4209
2015-12-07 12:20:26 +00:00
Andrew Turner
9d4de283d5 Rework the exception entry/return functions to make them valid frames to be
unwound through. For this we need the frame pointer (x29) to point to the
location on the stack where we stored the previous frame pointer, and link
register. To simplify this the stack pointer is only adjusted by addition
and subtraction, and not through the use of post increment on loads and
stores.

The updated frame layout is:

+------------+
| lr -- x30  |
+------------+
| fp -- x29  | <-- x29 points at this
+------------+
| Trap frame |
| ...        |
|            | <-- sp points at this
+------------+

The only difference is the first two items, and setting of x29.

Sponsored by:	ABT Systems Ltd
2015-12-01 12:37:04 +00:00
Andrew Turner
119a353e3d Rework the atomic code to reduce the repetition. This merges some of the
atomic functions where they are almost identical, or have acquire/release
semantics.

While here clean these function up. The cbnz instruction doesn't change
the condition flags so drop cc, however they should have memory added to the
clobber list.

Reviewed by:	kib
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D4318
2015-12-01 12:27:36 +00:00
Andrew Turner
1e888d7870 Print useful information when we hit a data abort we can't handle. This
prints the trap frame, along with the exception syndrome and fault address
registers. Even though esr is 64-bits here it is only 32-bits in hardware
so only print the valid 32-bits.

While here also print esr and far when appropriate after printing the trap
frame.

Sponsored by:	ABT Systems Ltd
2015-12-01 09:52:41 +00:00
Konstantin Belousov
4424a685c2 Shorten conditional branch code.
Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
2015-11-30 10:18:15 +00:00
Konstantin Belousov
5cb72aa7ec Remove unneeded instructions.
Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
2015-11-30 10:17:13 +00:00
Konstantin Belousov
724f4b62b0 Remove sv_prepsyscall, sv_sigsize and sv_sigtbl members of the struct
sysent.

sv_prepsyscall is unused.

sv_sigsize and sv_sigtbl translate signal number from the FreeBSD
namespace into the ABI domain.  It is only utilized on i386 for iBCS2
binaries.  The issue with this approach is that signals for iBCS2 were
delivered with the FreeBSD signal frame layout, which does not follow
iBCS2.  The same note is true for any other potential user if
sv_sigtbl.  In other words, if ABI needs signal number translation, it
really needs custom sv_sendsig method instead.

Sponsored by:	The FreeBSD Foundation
2015-11-28 08:49:07 +00:00
Ed Maste
20e0b5fcce Correct arm64 gic_v3 sizeof argument
No functional change as 'struct resource *' and 'struct resource **'
have the same size, but the former is the proper type.

PR:		204768
Submitted by:	David Binderman
2015-11-26 21:05:55 +00:00
Andrew Turner
ac4dad9e2f Add support for moving the DMAP range. This is needed as some AMD SoCs
place physical memory at an address outside the old DMAP range. This is an
issue as we rely on being able to move from PA -> VA using this range.

Obtained from:	Patrick Wildt <patrick@bitrig.org> (earlier version)
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D3885
2015-11-24 11:01:43 +00:00
Svatopluk Kraus
eae22c4430 Revert r291142.
The not quite consistent logic for bounce pages allocation is utilizited
by re(4) interface which can hang now.

Approved by:	kib (mentor)
2015-11-23 11:19:00 +00:00
Svatopluk Kraus
6fa7734d6f Fix BUS_DMA_MIN_ALLOC_COMP flag logic. When bus_dmamap_t map is being
created for bus_dma_tag_t tag, bounce pages should be allocated
only if needed.

Before the fix, they were allocated always if BUS_DMA_COULD_BOUNCE flag
was set but BUS_DMA_MIN_ALLOC_COMP not. As bounce pages are never freed,
it could cause memory exhaustion when a lot of such tags together with
their maps were created.

Note that there could be more maps in one tag by current design.
However BUS_DMA_MIN_ALLOC_COMP flag is tag's flag. It's set after
bounce pages are allocated. Thus, they are allocated only for first
tag's map which needs them.

Approved by:	kib (mentor)
2015-11-21 19:55:01 +00:00
Marius Strobl
ec2fbee752 Avoid a NULL pointer dereference in bounce_bus_dmamap_unload() when
the map has been created via bounce_bus_dmamem_alloc(). In that case
bus_dmamap_unload(9) typically isn't called during normal operation
but still should be during detach, cleanup from failed attach etc.

Submitted by:	yongari
MFC after:	3 days
2015-11-21 02:08:47 +00:00
Marius Strobl
8fd47ac11c Avoid a NULL pointer dereference in bounce_bus_dmamap_sync() when the
map has been created via bounce_bus_dmamem_alloc(). Even for coherent
DMA - which bus_dmamem_alloc(9) typically is used for -, calling of
bus_dmamap_sync(9) isn't optional.

PR:		188899 (non-original problem)
MFC after:	3 days
2015-11-20 02:23:35 +00:00
Ed Maste
f51f84a628 Fix typo in message from arm64 ITS workaround 2015-11-09 01:49:25 +00:00
Ed Maste
22a00274ac Sync arm64 GENERIC whitespace/comments with amd64
Sponsored by:	The FreeBSD Foundation
2015-11-08 21:08:31 +00:00
Ed Maste
13d3ad4d71 arm64: add igb(4) to GENERIC
We have em(4) in GENERIC already and so also supporting the related
igb(4) makes sense.

Sponsored by:	The FreeBSD Foundation
2015-11-07 04:46:34 +00:00
Andrew Turner
1df3f63ac5 Mark the arm64 nexus devices to be attached early. This allows multipass
to work correctly. Without it the pass quickly moves to the final pass
before the nexus device attaches.

Sponsored by:	ABT Systems Ltd
2015-11-06 14:36:21 +00:00
Ian Lepore
53f93ed3ff Fix an alignment check that is wrong in half the busdma implementations.
This will enable the elimination of a workaround in the USB driver that
artifically allocates buffers twice as big as they need to be (which
actually saves memory for very small buffers on the buggy platforms).

When deciding how to allocate a dma buffer, armv4, armv6, mips, and
x86/iommu all correctly check for the tag alignment <= maxsize as enabling
simple uma/malloc based allocation.  Powerpc, sparc64, x86/bounce, and
arm64/bounce were all checking for alignment < maxsize; on those platforms
when alignment was equal to the max size it would fall back to page-based
allocators even for very small buffers.

This change makes all platforms use the <= check.  It should be noted that
on all platforms other than arm[v6] and mips, this check is relying on
undocumented behavior in malloc(9) that if you allocate a block of a given
size it will be aligned to the next larger power-of-2 boundary.  There is
nothing in the malloc(9) man page that makes that explicit promise (but the
busdma code has been relying on this behavior all along so I guess it works).

Arm and mips code uses the allocator in kern/subr_busdma_buffalloc.c, which
does explicitly implement this promise about size and alignment.  Other
platforms probably should switch to the aligned allocator.
2015-11-02 23:37:19 +00:00
Andrew Turner
deeaa1c566 Make the arm64_cpu driver quiet as it adds no new information.
Only report the register used to start each cpu in bootverbose.

Sponsored by:	ABT Systems Ltd
2015-11-02 16:43:26 +00:00
Andrew Turner
4dd6ed5ce6 Mark functions as such. This means we call them directly rather than have
the dynamic linker copy them, but not relocate them at the new location.
This allows us to run sqlite3 without it crashing.

Sponsored by:	ABT Systems Ltd
2015-10-27 22:24:57 +00:00
Zbigniew Bodek
db94e32ba7 Fix bus numbering in ThunderX ITS quirk
Internal busses (thus ECAM access) should be mapped to
all values from 0 to 143.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D3753
2015-10-25 23:27:08 +00:00
Zbigniew Bodek
71e2c1d4c0 Add support for unspecified ranges on ThunderX system
When one tries to allocate a resource with unspecified range,
read already configured BAR values (by UEFI or whatever).
This is necessary to make VNIC VFs working and to allow them to be
properly allocated.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D3752
2015-10-25 23:22:40 +00:00
Ed Maste
5280a92784 arm64: remove exception instruction length assertion
From the (now removed) comment:

 * It is unclear in some cases if the bit is implementation defined.
 * The Foundation Model and QEMU disagree on if the IL bit should
 * be set when we are in a data fault from the same EL and the ISV
 * bit (bit 24) is also set.

Instead of adding even more special cases just remove the assertion.

Approved by:	andrew
Sponsored by:	The FreeBSD Foundation
2015-10-23 15:24:00 +00:00
Jason A. Harmening
a50730587b Remove unclear comment about address truncation in busdma. Add (hopefully much clearer) comment at declaration of PHYS_TO_VM_PAGE().
Noted by:	avg
2015-10-23 12:03:25 +00:00
Ed Schouten
2ff069a7bc Add support for CloudABI on ARM64.
It turns out that it is pretty easy to make CloudABI work on ARM64. We
essentially only need to copy over the sysvec from AMD64 and ensure that
we use ARM64 specific registers.

As there is an overlap between function argument and return registers,
we do need to extend cloudabi64_schedtail() to only set its values if
we're actually forking. Not when we're creating a new thread.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D3917
2015-10-22 11:09:25 +00:00
Jason A. Harmening
d394b026f6 Use pmap_quick* functions in arm64 busdma to make bounce buffer synchronization more flexible and avoid borrowing UVAs for userspace buffers. This is mostly equivalent to r286785 and r286787 for x86.
Differential Revision:	https://reviews.freebsd.org/D3870
2015-10-21 19:44:20 +00:00
Ed Maste
598cd16442 arm64: Enable CTF for DTrace support
Sponsored by:	The FreeBSD Foundation
2015-10-21 19:08:16 +00:00
Andrew Turner
80c4b9e575 Use 4 levels of page tables when enabling the MMU. This will allow us to
boot on an SoC that places physical memory at an address past where three
levels of page tables can access in an identity mapping.

Submitted by:   Wojciech Macek <wma@semihalf.com>,
                Patrick Wildt <patrick@bitrig.org>
Differential Revision:	https://reviews.freebsd.org/D3885 (partial)
Differential Revision:	https://reviews.freebsd.org/D3744
2015-10-19 13:20:23 +00:00
Zbigniew Bodek
32f849ca1e Add ThunderX VNIC to arm64/GENERIC kernel
Add vnic to enabled networking cards and enable SR-IOV by the way.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
2015-10-18 22:13:21 +00:00
Andrew Turner
22c6adff09 Correctly align the stack. The early csu assumed we passed the aux vector
in through the stack pointer, however this may have been misaligned
causing some userland applications to crash. A workaround was committed in
r284707 where userland would check if the aux vector was passed using the
old or new ABI and adjust the stack if needed. As 4 months have passed it
is time to move to the new ABI, with the expectation the compat code in csu
and the runtime linker to be removed in the future.

Sponsored by:	ABT Systems Ltd
2015-10-18 13:23:21 +00:00