Commit Graph

179 Commits

Author SHA1 Message Date
Konstantin Belousov
724f4b62b0 Remove sv_prepsyscall, sv_sigsize and sv_sigtbl members of the struct
sysent.

sv_prepsyscall is unused.

sv_sigsize and sv_sigtbl translate signal number from the FreeBSD
namespace into the ABI domain.  It is only utilized on i386 for iBCS2
binaries.  The issue with this approach is that signals for iBCS2 were
delivered with the FreeBSD signal frame layout, which does not follow
iBCS2.  The same note is true for any other potential user if
sv_sigtbl.  In other words, if ABI needs signal number translation, it
really needs custom sv_sendsig method instead.

Sponsored by:	The FreeBSD Foundation
2015-11-28 08:49:07 +00:00
Ed Maste
20e0b5fcce Correct arm64 gic_v3 sizeof argument
No functional change as 'struct resource *' and 'struct resource **'
have the same size, but the former is the proper type.

PR:		204768
Submitted by:	David Binderman
2015-11-26 21:05:55 +00:00
Andrew Turner
ac4dad9e2f Add support for moving the DMAP range. This is needed as some AMD SoCs
place physical memory at an address outside the old DMAP range. This is an
issue as we rely on being able to move from PA -> VA using this range.

Obtained from:	Patrick Wildt <patrick@bitrig.org> (earlier version)
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D3885
2015-11-24 11:01:43 +00:00
Svatopluk Kraus
eae22c4430 Revert r291142.
The not quite consistent logic for bounce pages allocation is utilizited
by re(4) interface which can hang now.

Approved by:	kib (mentor)
2015-11-23 11:19:00 +00:00
Svatopluk Kraus
6fa7734d6f Fix BUS_DMA_MIN_ALLOC_COMP flag logic. When bus_dmamap_t map is being
created for bus_dma_tag_t tag, bounce pages should be allocated
only if needed.

Before the fix, they were allocated always if BUS_DMA_COULD_BOUNCE flag
was set but BUS_DMA_MIN_ALLOC_COMP not. As bounce pages are never freed,
it could cause memory exhaustion when a lot of such tags together with
their maps were created.

Note that there could be more maps in one tag by current design.
However BUS_DMA_MIN_ALLOC_COMP flag is tag's flag. It's set after
bounce pages are allocated. Thus, they are allocated only for first
tag's map which needs them.

Approved by:	kib (mentor)
2015-11-21 19:55:01 +00:00
Marius Strobl
ec2fbee752 Avoid a NULL pointer dereference in bounce_bus_dmamap_unload() when
the map has been created via bounce_bus_dmamem_alloc(). In that case
bus_dmamap_unload(9) typically isn't called during normal operation
but still should be during detach, cleanup from failed attach etc.

Submitted by:	yongari
MFC after:	3 days
2015-11-21 02:08:47 +00:00
Marius Strobl
8fd47ac11c Avoid a NULL pointer dereference in bounce_bus_dmamap_sync() when the
map has been created via bounce_bus_dmamem_alloc(). Even for coherent
DMA - which bus_dmamem_alloc(9) typically is used for -, calling of
bus_dmamap_sync(9) isn't optional.

PR:		188899 (non-original problem)
MFC after:	3 days
2015-11-20 02:23:35 +00:00
Ed Maste
f51f84a628 Fix typo in message from arm64 ITS workaround 2015-11-09 01:49:25 +00:00
Ed Maste
22a00274ac Sync arm64 GENERIC whitespace/comments with amd64
Sponsored by:	The FreeBSD Foundation
2015-11-08 21:08:31 +00:00
Ed Maste
13d3ad4d71 arm64: add igb(4) to GENERIC
We have em(4) in GENERIC already and so also supporting the related
igb(4) makes sense.

Sponsored by:	The FreeBSD Foundation
2015-11-07 04:46:34 +00:00
Andrew Turner
1df3f63ac5 Mark the arm64 nexus devices to be attached early. This allows multipass
to work correctly. Without it the pass quickly moves to the final pass
before the nexus device attaches.

Sponsored by:	ABT Systems Ltd
2015-11-06 14:36:21 +00:00
Ian Lepore
53f93ed3ff Fix an alignment check that is wrong in half the busdma implementations.
This will enable the elimination of a workaround in the USB driver that
artifically allocates buffers twice as big as they need to be (which
actually saves memory for very small buffers on the buggy platforms).

When deciding how to allocate a dma buffer, armv4, armv6, mips, and
x86/iommu all correctly check for the tag alignment <= maxsize as enabling
simple uma/malloc based allocation.  Powerpc, sparc64, x86/bounce, and
arm64/bounce were all checking for alignment < maxsize; on those platforms
when alignment was equal to the max size it would fall back to page-based
allocators even for very small buffers.

This change makes all platforms use the <= check.  It should be noted that
on all platforms other than arm[v6] and mips, this check is relying on
undocumented behavior in malloc(9) that if you allocate a block of a given
size it will be aligned to the next larger power-of-2 boundary.  There is
nothing in the malloc(9) man page that makes that explicit promise (but the
busdma code has been relying on this behavior all along so I guess it works).

Arm and mips code uses the allocator in kern/subr_busdma_buffalloc.c, which
does explicitly implement this promise about size and alignment.  Other
platforms probably should switch to the aligned allocator.
2015-11-02 23:37:19 +00:00
Andrew Turner
deeaa1c566 Make the arm64_cpu driver quiet as it adds no new information.
Only report the register used to start each cpu in bootverbose.

Sponsored by:	ABT Systems Ltd
2015-11-02 16:43:26 +00:00
Andrew Turner
4dd6ed5ce6 Mark functions as such. This means we call them directly rather than have
the dynamic linker copy them, but not relocate them at the new location.
This allows us to run sqlite3 without it crashing.

Sponsored by:	ABT Systems Ltd
2015-10-27 22:24:57 +00:00
Zbigniew Bodek
db94e32ba7 Fix bus numbering in ThunderX ITS quirk
Internal busses (thus ECAM access) should be mapped to
all values from 0 to 143.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D3753
2015-10-25 23:27:08 +00:00
Zbigniew Bodek
71e2c1d4c0 Add support for unspecified ranges on ThunderX system
When one tries to allocate a resource with unspecified range,
read already configured BAR values (by UEFI or whatever).
This is necessary to make VNIC VFs working and to allow them to be
properly allocated.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D3752
2015-10-25 23:22:40 +00:00
Ed Maste
5280a92784 arm64: remove exception instruction length assertion
From the (now removed) comment:

 * It is unclear in some cases if the bit is implementation defined.
 * The Foundation Model and QEMU disagree on if the IL bit should
 * be set when we are in a data fault from the same EL and the ISV
 * bit (bit 24) is also set.

Instead of adding even more special cases just remove the assertion.

Approved by:	andrew
Sponsored by:	The FreeBSD Foundation
2015-10-23 15:24:00 +00:00
Jason A. Harmening
a50730587b Remove unclear comment about address truncation in busdma. Add (hopefully much clearer) comment at declaration of PHYS_TO_VM_PAGE().
Noted by:	avg
2015-10-23 12:03:25 +00:00
Ed Schouten
2ff069a7bc Add support for CloudABI on ARM64.
It turns out that it is pretty easy to make CloudABI work on ARM64. We
essentially only need to copy over the sysvec from AMD64 and ensure that
we use ARM64 specific registers.

As there is an overlap between function argument and return registers,
we do need to extend cloudabi64_schedtail() to only set its values if
we're actually forking. Not when we're creating a new thread.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D3917
2015-10-22 11:09:25 +00:00
Jason A. Harmening
d394b026f6 Use pmap_quick* functions in arm64 busdma to make bounce buffer synchronization more flexible and avoid borrowing UVAs for userspace buffers. This is mostly equivalent to r286785 and r286787 for x86.
Differential Revision:	https://reviews.freebsd.org/D3870
2015-10-21 19:44:20 +00:00
Ed Maste
598cd16442 arm64: Enable CTF for DTrace support
Sponsored by:	The FreeBSD Foundation
2015-10-21 19:08:16 +00:00
Andrew Turner
80c4b9e575 Use 4 levels of page tables when enabling the MMU. This will allow us to
boot on an SoC that places physical memory at an address past where three
levels of page tables can access in an identity mapping.

Submitted by:   Wojciech Macek <wma@semihalf.com>,
                Patrick Wildt <patrick@bitrig.org>
Differential Revision:	https://reviews.freebsd.org/D3885 (partial)
Differential Revision:	https://reviews.freebsd.org/D3744
2015-10-19 13:20:23 +00:00
Zbigniew Bodek
32f849ca1e Add ThunderX VNIC to arm64/GENERIC kernel
Add vnic to enabled networking cards and enable SR-IOV by the way.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
2015-10-18 22:13:21 +00:00
Andrew Turner
22c6adff09 Correctly align the stack. The early csu assumed we passed the aux vector
in through the stack pointer, however this may have been misaligned
causing some userland applications to crash. A workaround was committed in
r284707 where userland would check if the aux vector was passed using the
old or new ABI and adjust the stack if needed. As 4 months have passed it
is time to move to the new ABI, with the expectation the compat code in csu
and the runtime linker to be removed in the future.

Sponsored by:	ABT Systems Ltd
2015-10-18 13:23:21 +00:00
Andrew Turner
6aa751cff1 Replace build_section_pagetable with build_l1_block_pagetable as it takes
an extra argument to specify the number of 1GiB pages to map. This should
be a nop as we are only mapping a single page, but when we move to use an
extra level of page tables we will be able to map a second block, e.g. if
the kernel was loaded over a 1GiB boundary.
2015-10-17 19:52:17 +00:00
Andrew Turner
81b4133adf Rename build_block_pagetable to build_l2_block_pagetable in preperation
for adding support for 4 levels of page tables.

Obtained from:	Patrick Wildt <patrick@bitrig.org>
2015-10-17 14:07:47 +00:00
Konstantin Belousov
153bf0bce0 Add checks for kernel VA accesses to the copyin(9) and related
functions on arm64.

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D3907
2015-10-17 13:20:42 +00:00
Ed Schouten
632b5263c5 Properly set the return value for casueword to 0 upon success.
While trying to get multithreading working for CloudABI on aarch64, I
noticed that compare-and-exchange operations in kernelspace would always
fail. It turns out that we don't properly set the return value to 0 when
the compare and exchange succeeds.

Approved by:	andrew
Differential Revision:	https://reviews.freebsd.org/D3899
2015-10-15 17:50:28 +00:00
Konstantin Belousov
8598392cd1 Build changes that allow the modules on arm64.
- Move the required kernel compiler flags from Makefile.arm64 to kern.mk.
- Build arm64 modules as PIC; non-PIC relocations in .o for shared object
  output cannot be handled.
- Do not try to install aarch64 symlink.
- A hack for arm64 to avoid ld -r stage.  See the comment for the explanation.
  Some functionality is lost, like ctf handling, but hopefully will be
  restored after newer linker is available.

Reviewed by:	andrew, emaste
Tested by:	andrew (on real hardware)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3796
2015-10-08 17:42:08 +00:00
Konstantin Belousov
00a53ebe08 Implement in-kernel relocator for the arm64 module linker.
It is decided to go with the shared object file format for modules on
arm64, due to the Aarch64 instruction set details.  Combination of the
signed 28-bit offset in the branch instructions encoding together with
the supported memory model of compilers makes the relocatable object
support impossible or at least too hard.

Reviewed by:	andrew, emaste
Tested by:	andrew (on real hardware)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3796
2015-10-08 16:58:01 +00:00
Konstantin Belousov
fbc7b10762 Make the copyright notice in the file to match reality. Use the
recommended FreeBSD license text.

Approved by:	andrew
Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3846
2015-10-08 16:46:11 +00:00
Andrew Turner
8bdcc09641 When trying to execute from a misaligned address raise a SIGBUS with the
invalid address alignment code.

Obtained from:	EuroBSDCon
Sponsored by:	ABT Systems Ltd
2015-10-04 21:16:45 +00:00
Konstantin Belousov
b86860001b When asserting IL bit in exception syndrome register, print the raw
register value.

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
2015-10-04 12:52:30 +00:00
Andrew Turner
9439047657 Use pmap_load more consistently. While here try to only load the data once
when we reuse the same data.

Obtained from:	EuroBSDCon Devsummit
Sponsored by:	ABT Systems Ltd
2015-10-01 10:43:40 +00:00
Andrew Turner
7991717238 Add the ENTRY/END entries around the exception handlers.
Obtained from:	EuroBSDCon Devsummit
Sponsored by:	ABT Systems Ltd
2015-10-01 09:44:15 +00:00
Alan Cox
9f86aba61c Exploit r288122 to address a cosmetic issue. Since PV chunk pages don't
belong to a vm object, they can't be paged out.  Since they can't be paged
out, they are never enqueued in a paging queue.  Nonetheless, passing
PQ_INACTIVE to vm_page_unwire() creates the appearance that these pages
are being enqueued in the inactive queue.  As of r288122, we can avoid
this false impression by passing PQ_NONE.

Submitted by:	kmacy (an earlier version)
Differential Revision:	https://reviews.freebsd.org/D1674
2015-09-26 07:18:05 +00:00
Ed Maste
7fc0791608 Remove apostrophe from AP's for consistency (arm64 mp_machdep) 2015-09-25 21:04:31 +00:00
Ed Maste
2ed39a2216 Add pass device to arm64 GENERIC (for smartctl)
Sponsored by:	The FreeBSD Foundation
2015-09-22 21:43:08 +00:00
Konstantin Belousov
37864a5451 Call ast when handling irq from userspace, otherwise we could miss
reschedule.  Right now arm_cpu_intr() does critical_exit() as the last
action, so the impact is not serious.

Remove duplicated interrupt disable in restore_registers macro, when
returning to usermode.  The do_ast macro disabled interrupts for us.

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3714
2015-09-22 18:30:06 +00:00
Andrew Turner
9063d39e49 Don't restore interrupts when we are about to disable them in the next
instruction.
2015-09-22 16:46:34 +00:00
Konstantin Belousov
d50c68b2f0 Re-check for new ast after ast was handled. We should not return to
usermode with pending asts.

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3667
2015-09-22 16:29:55 +00:00
Ed Maste
a9ee805d45 arm64 ThunderX PCIe workaround: enumerate only one slot for now
Otherwise an em(4) NIC is detected 32 times.

Submitted by:	wma@semihalf.com
Obtained from:	Semihalf
Differential Revision:	https://reviews.freebsd.org/D3706
2015-09-22 12:56:34 +00:00
Konstantin Belousov
0510aedcd5 Do not execute exception handlers with disabled interrupts.
We should not call vm_fault(), or send a signal, with interrupts
disabled.  MI kernel code is not prepared for such environment, not to
mention that this increases system latency, since code appears to be
executing as being under spinlock.

The FAR register for data aborts is read before the interrupts are
enabled, to avoid its corruption due to nested exception or context
switch.

Add asserts, similar to the checks done by other architectures, about
not taking page faults in non-sleepable contexts, rather than die with
late and somewhat confusing witness diagnostic.

Reviewed by:	andrew
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3669
2015-09-18 17:09:59 +00:00
Konstantin Belousov
6ced378970 Clear exclusive monitors when handling data aborts, the monitors are
in unknown state per spec.

Reviewed by:	andrew (previous version)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3668
2015-09-18 16:52:18 +00:00
Andrew Turner
cf8b6f76a2 Don't read the floating-point registers for now. We will need to enable the
VFP around the read of these instructions as they may raise an exception.

Sponsored by:	ABT Systems Ltd
2015-09-18 13:44:15 +00:00
Zbigniew Bodek
7b1c945350 Block secondary ITS instances from attaching on ARM64
Currently FreeBSD supports only single PIC controller. Some systems
that have more than one (like ThunderX dual-socket) fails to boot.
Disable other PICes until proper handling is implemented in the
generic interrupt code.

Reviewed by:   imp
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3682
2015-09-16 23:59:45 +00:00
Zbigniew Bodek
dbf66ac207 Release memory for CPUs that fail to init on ARM64
cpu_init_fdt will now release memory allocated for structures
serving CPUs that have failed to init.

Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3297
2015-09-16 23:46:20 +00:00
Ed Maste
9b76bf99d5 arm64: add xhci driver and umass/ukbd to GENERIC for Cavium ThunderX
Sponsored by:	The FreeBSD Foundation
2015-09-15 19:59:35 +00:00
Mark Johnston
610141cebb Add stack_save_td_running(), a function to trace the kernel stack of a
running thread.

It is currently implemented only on amd64 and i386; on these
architectures, it is implemented by raising an NMI on the CPU on which
the target thread is currently running. Unlike stack_save_td(), it may
fail, for example if the thread is running in user mode.

This change also modifies the kern.proc.kstack sysctl to use this function,
so that stacks of running threads are shown in the output of "procstat -kk".
This is handy for debugging threads that are stuck in a busy loop.

Reviewed by:	bdrewery, jhb, kib
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D3256
2015-09-11 03:54:37 +00:00
Konstantin Belousov
1fa6712471 Do not hold the process around the vm_fault() call from the trap()s.
The only operation which is prevented by the hold is the kernel stack
swapout for the faulted thread, which should be fine to allow.

Remove useless checks for NULL curproc or curproc->p_vmspace from the
trap_pfault() wrappers on x86 and powerpc.

Reviewed by:	alc (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2015-09-10 17:46:48 +00:00