Commit Graph

116 Commits

Author SHA1 Message Date
Andrew Turner
8f7467734a Call into kdb on a data abort when it's enabled. This is needed to
correctly handle trying to access an invalid address in the debugger.

While here document that the breakpoint handler is supposed to fall
through to the following case.

Sponsored by:	ABT Systems Ltd
2015-08-25 16:49:10 +00:00
Andrew Turner
65780a4020 Correctly update the address to write to after storing the stack pointer
in the kernels setjmp function.

Sponsored by:	ABT Systems Ltd
2015-08-25 16:43:32 +00:00
Andrew Turner
9b9266e8e7 Check the frame pointer is within the kernel before accessing it. 2015-08-24 17:28:19 +00:00
Andrew Turner
67905e0aec Add support for pmap_sync_icache on arm64.
Reviewed by:	emaste, imp (both earlier version)
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3438
2015-08-24 12:01:39 +00:00
Andrew Turner
db36b645a0 Add pages used by the PV entries to minidumps. 2015-08-24 10:55:54 +00:00
Andrew Turner
0987c184a1 Handle the breakpoint instruction in userland by sending a SIGTRAP signal. 2015-08-23 13:18:56 +00:00
Andrew Turner
52afd687c3 Add the kernel support for minidumps on arm64.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3318
2015-08-20 12:49:56 +00:00
Andrew Turner
b0d415fc85 Remove pmap_kenter from pmap.h, the function doesn't exist. 2015-08-20 12:07:44 +00:00
Andrew Turner
9a0de24b5e Add pmap_get_tables to get the page tables for a given virtual address. This
will be used for minidump support.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-08-20 12:05:42 +00:00
Andrew Turner
5c714b29ff Add the arm64 minidump header. This was missed from r286953. 2015-08-20 11:26:26 +00:00
Zbigniew Bodek
a8d377b809 Increase MAXCPU in ARM64
Increase MAXCPU number to the maximum known value the existing
hardware can support.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3405
2015-08-19 11:59:41 +00:00
Zbigniew Bodek
04ae5bbe55 Remove redundant mp_naps from ARM64 secondary CPU start-up code
The global variable has been only used for CPU startup ordering
which is not needed anyway.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3296
2015-08-19 11:45:45 +00:00
Zbigniew Bodek
c50231a494 Add SMP support to GICv3 and ITS drivers
Introduce supprot for SMP to GICv3 and ITS drivers.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3299
2015-08-19 10:36:36 +00:00
Ed Maste
8d5bb77429 Correct comment typo noted by erik 2015-08-18 11:52:45 +00:00
Ed Maste
5a060174cd Remove register dump from arm64 el0 unknown exception
An exception with an unknown reasion is the expected result of the
attempted execution of an instruction bit pattern that has no allocated
instruction.

Sponsored by:	The FreeBSD Foundation
2015-08-18 10:07:03 +00:00
Andrew Turner
edc2af7807 Add support for bus_space_read_region and bus_space_write_region. This is
needed for the dwc USB controller driver.

Sponsored by:	ABT Systems Ltd
2015-08-14 09:55:25 +00:00
Ed Maste
13d06fd808 Remove arm64 workaround for Clang 3.4 crash 2015-08-13 19:05:18 +00:00
Ed Maste
ccd285e76c arm64: turn unknown el0 exception into a SIGILL
It seems we get EXCP_UNKNOWN from QEMU when executing zeroed memory.
Print a register dump here and signal illegal instruction. Also print
a register dump for other invalid exceptions, before panic.

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3370
2015-08-13 13:21:00 +00:00
Andrew Turner
b1bacc1cbd Add the CNTHCTL_EL2 register bits missed in r286674 2015-08-12 17:09:57 +00:00
Andrew Turner
f03aa10f47 Set the counter-timer virtual offset to a know value, it may not have been
set by the boot code and are reset to an implementation defined value that
may be unknown.

Sponsored by:	ABT Systems Ltd
2015-08-12 17:06:22 +00:00
Andrew Turner
c8d372efdd Check the correct value in db_validate_address, pmap_extract returns 0 on
failure.

Sponsored by:	ABT Systems Ltd
2015-08-11 12:32:17 +00:00
Pedro F. Giffuni
6bc3fe5f4e Clean out some externally visible "more then" grammar
MFC after:	3 days
2015-08-11 03:12:09 +00:00
Zbigniew Bodek
bfc3978594 Add support for external PCIe (PEM) on Cavium's ThunderX
Reviewed by:   jhb
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3257
2015-08-08 21:32:03 +00:00
Zbigniew Bodek
6ff9127b86 Add Ethernet and block device drivers to ARM64 GENERIC 2015-08-08 20:56:30 +00:00
Zbigniew Bodek
d943d79a36 Introduce support for internal PCIe for Cavium's ThunderX
This driver supports internal PCIe Root Complex on
Cavium ThunderX Pass 1.1 hardware.

Reviewed by:   andrew, jhb
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3031
2015-08-08 20:34:55 +00:00
Andrew Turner
756e25bfd7 Fill in dump_avail based on the physical memory from EFI.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-08-06 14:49:23 +00:00
Ed Maste
fc8c856029 Rationalize BSD license on sys/*/include/in_cksum.h
Remove the advertising clause from the Regents of the University of
California's license, per the letter dated July 22, 1999.

Update clause numbering.
2015-08-05 19:05:12 +00:00
Ed Maste
96226a9aa7 Rationalize BSD license on sys/*/include/float.h
Remove the advertising clause from the Regents of the University of
California's license, per the letter dated July 22, 1999.

Update clause numbering.
2015-08-05 17:05:35 +00:00
Jason A. Harmening
713841afb2 Add two new pmap functions:
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)

These will create and destroy a temporary, CPU-local KVA mapping of a specified page.

Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread

Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page

The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.

NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.

Reviewed by:	kib
Approved by:	kib (mentor)
Differential Revision:	http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
Zbigniew Bodek
4cbca60875 Add missing exception number to EL0 sync. abort on ARM64
When doing a data abort from userland it is possible to get
more than one data abort inside the same exception level.
Add an appropriate exception number to allow nesting of
data_abort handler for EL0.

Reviewed by:   andrew
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3276
2015-08-03 14:58:46 +00:00
Andrew Turner
f692e32555 Pass the pcb to store the vfp state in to vfp_save_state. This fixes a bug
in savectx where it will be used to store the current state however will
pass in a pcb when vfp_save_state expected a thread pointer.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-08-03 11:05:02 +00:00
Andrew Turner
9e2529043b Try to put the CPU into a low power state if we failed to otherwise halt
the system.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-31 15:54:34 +00:00
Andrew Turner
176739d3f5 Load the stack in stack_save and stack_save_td. This uses the generalised
unwind_frame function to read each stack frame until either the pc or stack
are no longer withing the kernel's address space.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-31 15:32:32 +00:00
Andrew Turner
36baf858c9 Add support for uma_small_alloc and uma_small_free, and make use of these.
This is copied from the amd64 version with minor changes. These should be
merged into a single file as from a quick look there are other copies of
the same file in other parts of the tree.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-31 14:17:26 +00:00
Andrew Turner
872df66596 Add memrw. This has had minimal testing, and will likely panic the kernel
when trying to read data from outside the DMAP region. I expect this panic
to be from within uiomove_fromphys, which needs to grow support to support
such addresses.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-31 13:39:51 +00:00
Andrew Turner
9b8c3c4f0b Add more atomic_swap_* functions.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-31 13:34:43 +00:00
Andrew Turner
71d72ea14f Add VIRT_IN_DMAP to check if a virtual address is from the DMAP range.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-31 13:32:25 +00:00
Zbigniew Bodek
a2b3dfad08 Apply erratum for mrs ICC_IAR1_EL1 speculative execution on ThunderX
ERRATUM:     22978, 23154
PASS (rev.): 1.0/1.1

Reviewed by:   imp
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3184
2015-07-31 10:00:45 +00:00
Andrew Turner
8df0053b7a Add enough of pmap_page_set_memattr to run gstat. It still needs to split
the DMAP 1G pages so we set the attributes only on the specified page.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-30 16:17:44 +00:00
Ed Maste
c547d650eb Add ARM64TODO markers to unimplemented functionality
Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D2389
2015-07-30 14:20:36 +00:00
Zbigniew Bodek
9028b18f75 Enable IRQ during syscalls on ARM64
FreeBSD provides a feature called Adaptive Mutexes, which allows
a thread to spin for a while when the mutex is taken instead of
immediately going to sleep. This causes issues when called from
syscall handler if interrupts are masked. If every other core
also attempts to access the same mutex there is a chance that
all of them are spinning on the same lock at the same time.
If interrupts are disabled, no kernel preemtion can occur and
the system becomes unresponsive.

This patch enables interrupts when syscall is being executed
and masks them as soon as it is completed.

Reviewed by:   andrew
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3246
2015-07-30 13:59:38 +00:00
Zbigniew Bodek
cf89e8c919 Add quirk for ThunderX ITS device table size
Limit the number of supported device IDs to 0x100000
in order to decrease the size of the ITS device table so
that it matches with the HW capabilities.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3131
2015-07-29 11:22:19 +00:00
Andrew Turner
4027d3d62a Teach the GICv2 driver about the Qualcomm GICv2 compatible string.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-21 18:08:10 +00:00
Zbigniew Bodek
3ed97a1a52 Add some more explanation to r285752
Add brief commentary to vendor-specific devid function in ITS
and remove redundant spaces by the way.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
2015-07-21 17:14:24 +00:00
Zbigniew Bodek
9920b3aa95 Don't allow malloc() to wait for resource while holding a lock in ITS
malloc() should not go to sleep in case of lack of resource while
the kernel thread is holding a non-sleepable lock.

- change malloc() flags to M_NOWAIT in such cases implement
  lpi_free_chunk() routine as it will be needed when ITT
  allocation fails in its_device_alloc_locked()
- do not increase verbosity of this code since upper layers will
  communicate an error if the interrupt setup fails

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3121
2015-07-21 15:28:07 +00:00
Zbigniew Bodek
1fe6a1a25a Add support for vendor specific function for PCI devid acquisition in ITS
It is possible that some HW will use different PCI devids,
hence allow to replace the default domain🚌slot:func schema
by implementing and registering custom function.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3118
2015-07-21 14:47:23 +00:00
Zbigniew Bodek
52b584bc15 Implement get_cyclecount() on ARM64
Use Vritual Counter register associated with Generic Timer to
read the cyclecount.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3134
2015-07-21 12:50:45 +00:00
Zbigniew Bodek
13aaea2fd7 Improve ARM64 CPU_MATCH
Add a method to identify CPU based on RAW MIDR value.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3117
2015-07-21 12:15:00 +00:00
Zbigniew Bodek
b8bbefed30 Fix possible coherency issues between PEs related to I-cache
Basing on B.2.3.4:
Synchronization and coherency issues between data and
instruction accesses.

To ensure that modified instructions are visible to all PEs
(Processing Elements) in a shareability domain one need to
perform following sequence:
    1. Clean D-cache
    2. Ensure the visibility of data cleaned from cache
    3. Invalidate I-cache
    4. Ensure completion
    5. In SMP system PE must issue isb to ensure execution of the
       modified instructions

Reviewed by:   andrew
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3106
2015-07-17 14:33:47 +00:00
Zbigniew Bodek
ab89029bd0 Fix secondary stacks calculation on ARM64
Secondary stack calculation is modified to provide
stack_top = secondary_stacks + (cpu_id) * PAGE_SIZE * KSTACK_PAGES
because on ARM64 the stack grows to lower memory addresses.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3107
2015-07-17 14:08:08 +00:00