Commit Graph

381 Commits

Author SHA1 Message Date
Jared McNeill
1567e084b3 Add support for Allwinner A64.
Reviewed by:	andrew, manu
Relnotes:	yes
2016-08-25 13:59:19 +00:00
Andrew Turner
d5911afb79 Map coherent memory in a non-coherent dma tag as uncached. This is similar
to what the 32-bit arm code does, with the exception that it always assumes
the tag is non-coherent.

Tested by:	jmcneill
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-08-25 10:53:03 +00:00
Andrew Turner
aa9872243a Allow superpages on arm64 to be enabled. It is disabled for now to allow
for more testing across more hardware.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-24 13:00:50 +00:00
Andrew Turner
262432bbb1 Add support to promote and demote managed superpages. This is the last part
needed before enabling superpages on arm64. This code is based on the amd64
pmap with changes as needed to handle the differences between the two
architectures.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-24 12:57:40 +00:00
Andrew Turner
3969d2f921 Teach the parts of the arm64 pmap that need to iterate over pages to also
iterate over superpages. We don't yet create these, but soon will.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-24 12:32:18 +00:00
Ed Schouten
48734c99d3 Convert pointers obtained from the threadattr_t structure with TO_PTR().
In all of these source files, the userspace pointer size corresponds
with the kernelspace pointer size, meaning that casting directly works.
As I'm planning on making 32-bit execution on 64-bit systems work as
well, use TO_PTR() here as well, so that the changes between source
files remain minimal.
2016-08-24 10:13:18 +00:00
Andrew Turner
e0a66c6379 Also adjust the virtual address passed to vm_page_pa_tryrelock.
Reported by:	alc
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-23 16:37:34 +00:00
Andrew Turner
56d9fe8567 Map memory as read-only in pmap_enter_quick_locked as is done in other
pmap implementations.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-23 16:20:56 +00:00
Andrew Turner
f17394c340 If we find we have a superpage in pmap_enter_quick_locked return without
trying to add a new level 3 page.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-23 16:12:25 +00:00
Andrew Turner
dd9faf6dc4 Include the offset the virtual address is within an L1 or L2 block when
finding the vm_page_t in pmap_extract_and_hold. Previously it would return
the vm_page_t of the first page in a block. This would cause issues when,
for example, fsck reads from a device into the middle of a superpage. In
this case the read call would write to the start of the block, and not to
the buffer passed in.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-23 15:48:27 +00:00
Andrew Turner
724c3e47f6 Fix the arm64 non-SMP build, active_irq is a uint64_t so cast it through
a uintmax_t.

Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-08-22 19:05:11 +00:00
Andrew Turner
1d479540a5 Ensure map is valid, even before userland exists and the fault address
register points to an address in the userland range.

Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	the FreeBSD Foundation
2016-08-22 18:19:46 +00:00
Andrew Turner
8e7fcbc322 Fix pmap_update_entry, pmap_invalidate_range takes the end address, not
the size.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-22 18:12:44 +00:00
Andrew Turner
c7d40f1843 Use switch statements in pmap_remove_pages. While only one level of
pagetable is supported more will be added soon to support removing
superpages.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-22 14:53:39 +00:00
Andrew Turner
a3c1bc2eae Use pmap_update_entry in pmap_enter when updating an entry with a new
physical address. This is required when either mapping is writeable.

While here remove an unneeded call to pmap_pde, we already have the pde
from earlier in the function.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-22 12:56:40 +00:00
Andrew Turner
aed6b9701b Add sysctls to report on superpages statistics. While here add extra
logging to these paths.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-22 12:17:40 +00:00
Andrew Turner
109dddc015 Add a size argument to pmap_update_entry.
Make use of this in pmap_promote_l2.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-22 10:50:30 +00:00
Andrew Turner
1d32c3222e Add KASSERTS in pmap_alloc_l3 to ensure we are not encountering superpages
when we don't yet expect them;

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-22 10:21:09 +00:00
Michal Meloun
895c8b1c39 INTRNG: Rework handling with resources. Partially revert r301453.
- Read interrupt properties at bus enumeration time and store
   it into global mapping table.
 - At bus_activate_resource() time, given mapping entry is resolved and
   connected to real interrupt source. A copy of mapping entry is attached
   to given resource.
 - At bus_setup_intr() time, mapping entry stored in resource is used
   for delivery of requested interrupt configuration.
 - For MSI/MSIX interrupts, mapping entry is created within
   pci_alloc_msi()/pci_alloc_msix() call.
 - For legacy PCI interrupts, mapping entry must be created within
   pcib_route_interrupt() by pcib driver itself.

Reviewed by: nwhitehorn, andrew
Differential Revision: https://reviews.freebsd.org/D7493
2016-08-19 10:52:39 +00:00
Konstantin Belousov
1680854946 Implement userspace gettimeofday(2) with HPET timecounter.
Right now, userspace (fast) gettimeofday(2) on x86 only works for
RDTSC.  For older machines, like Core2, where RDTSC is not C2/C3
invariant, and which fall to HPET hardware, this means that the call
has both the penalty of the syscall and of the uncached hw behind the
QPI or PCIe connection to the sought bridge.  Nothing can me done
against the access latency, but the syscall overhead can be removed.
System already provides mappable /dev/hpetX devices, which gives
straight access to the HPET registers page.

Add yet another algorithm to the x86 'vdso' timehands. Libc is updated
to handle both RDTSC and HPET.  For HPET, the index of the hpet device
to mmap is passed from kernel to userspace, index might be changed and
libc invalidates its mapping as needed.

Remove cpu_fill_vdso_timehands() KPI, instead require that
timecounters which can be used from userspace, to provide
tc_fill_vdso_timehands{,32}() methods.  Merge i386 and amd64
libc/<arch>/sys/__vdso_gettc.c into one source file in the new
libc/x86/sys location.  __vdso_gettc() internal interface is changed
to move timecounter algorithm detection into the MD code.

Measurements show that RDTSC even with the syscall overhead is faster
than userspace HPET access.  But still, userspace HPET is three-four
times faster than syscall HPET on several Core2 and SandyBridge
machines.

Tested by:	Howard Su <howard0su@gmail.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D7473
2016-08-17 09:52:09 +00:00
Andrew Turner
2bafd72fdb Add the ARMv8.1 identification registers to the list we print when booting.
MFC after:	1 week
Sponsored by:	ABT Systems Ltd
2016-08-15 09:23:08 +00:00
Andrew Turner
510a3f1b79 Implement promotions and demotions in the arm64 pmap code. For now we don't
promote memory as I am not sure all the demotion cases are handled, however
it is useful to implement pmap_page_set_memattr. This is used, for example,
when mapping uncached memory for bus_dma(9).

pmap_page_set_memattr needs to demote the DMAP region as on ARM we need to
ensure all mappings to the same physical address have the same attributes.

Reviewed by:	kib
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6987
2016-08-12 10:29:34 +00:00
Ed Schouten
13b4b4df98 Provide the CloudABI vDSO to its executables.
CloudABI executables already provide support for passing in vDSOs. This
functionality is used by the emulator for OS X to inject system call
handlers. On FreeBSD, we could use it to optimize calls to
gettimeofday(), etc.

Though I don't have any plans to optimize any system calls right now,
let's go ahead and already pass in a vDSO. This will allow us to
simplify the executables, as the traditional "syscall" shims can be
removed entirely. It also means that we gain more flexibility with
regards to adding and removing system calls.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D7438
2016-08-10 21:02:41 +00:00
Ed Schouten
66bd4c2eeb Make cpu_set_user_tls() work when called on the running thread.
On all the other architectures, this function can also be called on the
currently running thread. In this case, we shouldn't fix up the address
in the PCB, but also patch up the register itself. Otherwise it will not
become active and will simply become overwritten by the next switch.

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D7437
2016-08-10 15:45:25 +00:00
Andrew Turner
4cf6e978b4 Uncomment the vm.kvm_size and vm.kvm_free sysctls. These work as expected so
there is no reason to leave them commented out.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-10 10:36:11 +00:00
Andrew Turner
4088f71f9b Implement pmap_align_superpage on arm64 based on the amd64 implementation.
This will be needed when superpage support is added.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-10 10:13:34 +00:00
Andrew Turner
b0316526b0 Remove the pvh_global_lock lock from the arm64 pmap. It is unneeded on arm64
as invalidation will have completed before the pmap_invalidate_* functions
have complete.

Discussed with:	alc, kib
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-04 13:49:36 +00:00
Andrew Turner
27c9d42531 Remove trailing whitespace from the arm64 pmap
Obtained from:	ABT Systems Ltd
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
2016-08-02 15:26:46 +00:00
Andrew Turner
698c14e189 Add a kernel variable to let the user to select their preferred order
between ACPI and FDT. This will be needed on machines with both, e.g. the
SoftIron Overdrive 3000. The kernel will accept one or more comma separated
values of either 'acpi' or 'fdt'. Any other values are skipped.

To set it the user can either set it on the loader command line, or
in loader.conf e.g. in loader.conf:
kern.cfg.order=acpi,fdt

This will try using ACPI then FDT. If none of the selected options work the
kernel tries to use one to get the serial console, then panics.

Reviewed by:	emaste (earlier version)
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D7274
2016-08-01 12:17:44 +00:00
Andrew Turner
49a92cd4a5 Add the fields for the PAR_EL1 register. This is used when performing an
address lookup with the AT instructions.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-01 10:36:58 +00:00
Andrew Turner
63512a1276 Add the Data Fault Status Code values to the ESR_ELx registers for when the
fault code is a Data Abort.

Obtained from:	AT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-31 18:58:20 +00:00
Andrew Turner
ed24579135 Extract the common parts of pmap_kenter_device to a new function. This will
be used when superpage support is added.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-31 17:53:09 +00:00
Andrew Turner
7592321ad1 Fix the comment above pmap_invalidate_page. tlbi will invalidate the tlb
on all CPUs.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-31 14:59:44 +00:00
Andrew Turner
c5b3b20907 Relax the barriers around a TLB invalidation to only wait on
inner-shareable memory accesses. There is no need for full system barriers.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-31 12:59:10 +00:00
Andrew Turner
eda295b9e5 Add a generic EHCI USB driver based on the Allwinner A10 driver. It is ACPI
only for now, but wouldn't be too difficult to add support for FDT.

Reviewed by:	hselasky
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D7352
2016-07-29 08:50:36 +00:00
Andrew Turner
1af4207389 Remove an unused variable.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-25 16:18:20 +00:00
Andrew Turner
0f0b7aa289 Fix a typo in a string in a KASSERT to sanity check the CPU IDs.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-25 15:59:31 +00:00
Andrew Turner
8ec2b5df73 Rework how we number CPUs on arm64 to try and keep clusters together.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-25 15:57:13 +00:00
Andrew Turner
abf5bdb441 Enable the generic OHCI driver on arm64
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2016-07-25 14:49:15 +00:00
Alexander Motin
fb112f72a8 Add more UEFI/e820 memory types from latest specifications.
This is only cosmetics.

MFC after:	2 weeks
2016-07-24 09:15:11 +00:00
Andrew Turner
e0f50f02e4 Implement bus_print_child to print the resources used by the ITS driver.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-15 13:25:47 +00:00
Andrew Turner
5d94bbc241 Finish removing the non-INTRNG support from sys/arm64.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:31:29 +00:00
Andrew Turner
ec1172ed92 Remove the non-INTRNG support from the ThunderX PCIe drivers.
Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:23:49 +00:00
Andrew Turner
c55795369b Move gic_v3_irqsrc into the GICv3 driver source as it's only needed there.
Remove unused macros from the GICv3 header.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:16:51 +00:00
Andrew Turner
a509d56d7f Move structures only used by the GICv3 ITS driver from a shared header to
the ITS driver file. There is no need for other drivers to need to know
about these structures.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:10:54 +00:00
Andrew Turner
b30fd6a080 Remove the non-INTRNG support from the GICv3 interrupt controller driver.
This is no longer needed.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-07-14 17:05:25 +00:00
Andrew Turner
07b8a57120 Add memmmap on arm64 so we can mmap /dev/mem and /dev/kmem.
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-07-13 23:03:34 +00:00
Andrey A. Chernov
239e5577a2 Undo r302601, WCHAR_MAX may not be a valid wchar value. 2016-07-12 04:20:44 +00:00
Andrey A. Chernov
713a61670c I don't know why unsigned int is choosed for wchar_t here, but WCHAR_MAX
should be <= WINT_MAX. It is bigger, __UINT_MAX > INT32_MAX
2016-07-12 00:37:48 +00:00
Ed Schouten
d96aeddf2f Don't forget to set sa->narg for CloudABI system calls.
It turns out that this value is not used within the system call code
under normal conditions, except when using tracing tools like ktrace.
If we forget to set this value, it is set to random garbage. This may
cause ktrace to hang indefinitely, making it impossible to kill.

Reported by: Michael Plass
PR: 210800
MFC before: 11.0-RELEASE
2016-07-08 20:09:21 +00:00