Commit Graph

429 Commits

Author SHA1 Message Date
Andrew Turner
0c5434ae58 Pull the common FDT interrupt values into a new header rather than be magic
numbers.

Sponsored by:	DARPA, AFRL
2016-10-26 15:18:08 +00:00
Andrew Turner
2eb8079722 Only release CPUs when they exist.
MFC after:	1 week
Sponsored by:	DARPA, AFRL
2016-10-26 12:30:53 +00:00
Andrew Turner
2e620e70f9 Use nitems to get the correct number of registers to read when dumping
them. Previously this would walk past the end of the array and print
whatever happened to be after the trapframe struct.

MFC after:	1 week
Sponsored by:	DARPA, AFRL
2016-10-26 12:29:56 +00:00
Andrew Turner
c2dd354bf4 Create a new PSCI error code and use it to signal that starting the CPU is
impossible as the PSCI firmware is missing.

Sponsored by:	ABT Systmes Ltd
2016-10-25 14:18:27 +00:00
Andrew Turner
02e1c9861d Import the Cortex String memcpy and memmove into the kernel. On ThunderX
these show a 9-10% reduction in user and system time for a buildworld -j48.

Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-10-25 14:04:35 +00:00
Andrew Turner
9eb0ccbb54 Increase CACHE_LINE_SHIFT to 7 as cache lines are 128 bytes on ThunderX.
MFC after:	1 week
Sponsored by:	ABT Systems Ltd
2016-10-24 13:44:24 +00:00
Wojciech Macek
7902c8dca8 Driver for PCI Ethernet NIC on Alpine V1 and V2.
Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Reviewed by:           wma
Differential Revision: https://reviews.freebsd.org/D7814
2016-10-20 11:31:11 +00:00
Andrew Turner
6ed982a221 Add support for the fpu_kern(9) KPI on arm64. It hooks into the existing
VFP code to store the old context, with lazy loading of the new context
when needed.

FPU_KERN_NOCTX is missing as this is unused in the crypto code this has
been tested with, and I am unsure on the requirements of the UEFI
Runtime Services.

Reviewed by:	kib
Obtained from:	ABT Systeems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8276
2016-10-20 09:22:10 +00:00
Oleksandr Tymoshenko
3ab1c20e5b Move SOC_BRCM_BCM2837 to GENERIC-UP since does not support SMP 2016-10-19 02:03:30 +00:00
Oleksandr Tymoshenko
232de4a9c2 Retire RPI3 kernel config in favour of GENERIC-UP
Policy for FreeBSD/arm64 kernel config is the same one as for x86
architectures: provide GENERIC kernel bootable on as many systems
as possible. Since there is no SMP support for RPi 3 yet new kernel
config was introduced: GENERIC-UP, which is effectively GENERIC with
SMP option disabled
2016-10-18 23:48:47 +00:00
Andrew Turner
4c247b9719 Add PCB_FP_USERMASK so we can mask off floating point flags that should
not be sent to userspace, for example the future flag to tell when we are
using floating point in the kernel.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-10-18 13:39:55 +00:00
Andrew Turner
d88a17dd8f Fix the build, struct vfpstate needs to be visible to userspace as it's
part of struct pcb which is in a header used in libutil.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-10-15 16:29:06 +00:00
Andrew Turner
403ddccb10 Check we are in a critical section when calling vfp_discard. As we may call
it with a NULL thread pointer only check when it is non-NULL.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2016-10-15 09:54:22 +00:00
Oleksandr Tymoshenko
745c4aa5e8 Make BRCM2837 port conform FreeBSD/ARM64 guidelines
- Rename SOC_BCM2837 to SOC_BRCM_BCM2837, put it to opt_soc.h
- do not use files.XXX files, just move required sources to
    conf/files.arm64 and make them depend on soc_brcm_bcm2837

Suggested by: andrew
2016-10-14 22:23:03 +00:00
Andrew Turner
401d3029af Create macros for the MAIR memory attributes. While here add an uncached
memory type, however the VM code still needs to be taught about this.

MFC after:	1 week
Sponsored by:	ABT Systems Ltd
2016-10-14 22:20:14 +00:00
John Baldwin
31dc1e9681 Drop support for using mmap() with /dev/kmem.
Using the device pager with /dev/kmem is not stable since KVA mappings
are transient, but the device pager caches the PA associated with a
given offset forever.  Interestingly, mips' implementation of
memmap() already refused requests for /dev/kmem.

Note that kvm_read/kvm_write do not use mmap, but use read and write on
/dev/kmem, so this should not affect libkvm users.

Reviewed by:	kib
MFC after:	2 months
2016-10-14 20:01:07 +00:00
Oleksandr Tymoshenko
65de32210c Remove redundant kern_clocksource.c. It is already in conf/files.arm64
Reported by:	andrew
2016-10-14 17:40:48 +00:00
Andrew Turner
8ff003011d Rework how we store the VFP registers in the pcb. This will be used when
creating a floating-point context within the kernel without having to move
the stored values in memory.

Sponsored by:	The FreeBSD Foundation
2016-10-14 15:53:48 +00:00
Oleksandr Tymoshenko
563193fd0c Add initial Raspberry Pi 3 support
RPI3 kernel config builds kernel compatible with latest upstream device
tree and firmware: https://github.com/raspberrypi/firmware/tree/master/boot
As of today it's 597c662a613df1144a6bc43e5f4505d83bd748ca

Default console is PL01x, so pi3-disable-bt dt overlay should be configured
in config.txt and stock U-Boot should be patched to use proper serial port.

Yet unsupported: SMP, VCHIQ, RNG driver. RNG requires some work due to
upstream device tree incompatibility.

Multiple people contributed to this work over time: db@, loos@, manu@
2016-10-14 03:37:35 +00:00
Andrew Turner
1890f18cbe Move printing the AArch64 ID registers to a new SYSINIT, the previous
location only prints them when booting on SMP with multiple cores.

MFC after:	1 week
Sponsored by:	DARPA, AFRL
2016-10-13 09:06:29 +00:00
Warner Losh
b2a7ac4802 Fix building on i386 and arm. But 'public domain' headers on the files
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.

Submitted by: kib@, dim@, zbb@, emaste@
2016-10-13 06:56:23 +00:00
Oleksandr Tymoshenko
c38fb7809f INTRNG: Propagate IRQ activation error to API consumer
Keep resource state consistent with INTRNG state - if intr_activate_irq
fails - deactivate resource and propagate error to calling function

Reviewed by:	mmel
2016-10-12 17:10:59 +00:00
Jonathan T. Looney
bd79708dbf In the TCP stack, the hhook(9) framework provides hooks for kernel modules
to add actions that run when a TCP frame is sent or received on a TCP
session in the ESTABLISHED state. In the base tree, this functionality is
only used for the h_ertt module, which is used by the cc_cdg, cc_chd, cc_hd,
and cc_vegas congestion control modules.

Presently, we incur overhead to check for hooks each time a TCP frame is
sent or received on an ESTABLISHED TCP session.

This change adds a new compile-time option (TCP_HHOOK) to determine whether
to include the hhook(9) framework for TCP. To retain backwards
compatibility, I added the TCP_HHOOK option to every configuration file that
already defined "options INET". (Therefore, this patch introduces no
functional change. In order to see a functional difference, you need to
compile a custom kernel without the TCP_HHOOK option.) This change will
allow users to easily exclude this functionality from their kernel, should
they wish to do so.

Note that any users who use a custom kernel configuration and use one of the
congestion control modules listed above will need to add the TCP_HHOOK
option to their kernel configuration.

Reviewed by:	rrs, lstewart, hiren (previous version), sjg (makefiles only)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D8185
2016-10-12 02:16:42 +00:00
Jung-uk Kim
493deb390b Merge ACPICA 20160930. 2016-10-04 20:27:15 +00:00
Konstantin Belousov
559a7b209a Add a way for the architecture to specify the calling ABI for methods
in the EFI Runtime Services Table.  On amd64, the calling conventions
are MS.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-09-21 10:35:44 +00:00
Wojciech Macek
a134b98878 Add support for SPI-mapped MSI interrupts in GICv3.
PIC_SETUP_INTR implementation in GICv3 did not allow
for setting up interrupts without included FDT
description. GICv2m-like MSI interrupts, which map
MSI messages to SPI interrupt lines, may not have
a description in FDT. Add support for such interrupts
by setting the trigger and polarity to the appropriate
values for MSI (edge, high) and get the hardware
IRQ number from the corresponding ISRC.

Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Reviewed by:           wma
Differential Revision: https://reviews.freebsd.org/D7662
2016-09-21 05:22:49 +00:00
Wojciech Macek
3502e7b37c Register GICv3 xref.
This allows other drivers to retrieve interrupt parent node.

Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Reviewed by:           wma, zbb
Differential Revision: https://reviews.freebsd.org/D7568
2016-09-21 05:15:50 +00:00
Wojciech Macek
988d9b3505 Add driver for PCIe root complex on Annapurna Alpine platform.
The driver subclasses pci-host-generic and additionally
performs configuration of vendor-specific PCIe registers.

Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Reviewed by:           wma
Differential Revision: https://reviews.freebsd.org/D7571
2016-09-20 11:11:06 +00:00
Konstantin Belousov
944e0bab86 Consolidate four efi_next_descriptor() definitions.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-09-18 17:38:02 +00:00
Andrew Turner
6de3c41534 Add a missing opt_ddb.h include from places that include debug_moditor.h
to fix the build.

Obtained from:	ABT Systems Ltd
X-MFC with:	r305771
Sponsored by:	The FreeBSD Foundation
2016-09-13 16:20:31 +00:00
Andrew Turner
5ed824174b Fix the arm64 kernel build when DDB is disabled, debug_monitor.c depends on
DDB, and is unused when it's disabled.

Obtained from:	ABT Systems Ltd
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2016-09-13 15:45:22 +00:00
Andrew Turner
8b52054063 Add a warning about a known erratum we have observed on ThunderX pass 1.1.
As this is evaluation hardware with only a few users, and there is a lack
of information add a warning when booting on this hardware.

Reported by:	cognet
Obtained from:	ABT Systems Ltd
MFC after:	Instant
Sponsored by:	The FreeBSD Foundation
2016-09-13 14:10:49 +00:00
Alan Cox
8cb0c1029d Various changes to pmap_ts_referenced()
Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and
into vm/pmap.h, and describe what its purpose is.  Eliminate the archaic
"XXX" comment about its value.  I don't believe that its exact value, e.g.,
5 versus 6, matters.

Update the arm64 and riscv pmap implementations of pmap_ts_referenced()
to opportunistically update the page's dirty field.

On amd64, use the PDE value already cached in a local variable rather than
dereferencing a pointer again and again.

Reviewed by:	kib, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
Andrew Turner
13db69623b Trap msr/mrs instructions. These are privileged arm64 instructions and
shouldn't normally be used.

Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-09-08 12:53:01 +00:00
Andrew Turner
e0c6c1d1fd Don't panic when we don't handle a userland exception, not all we may see
are currently handled.

Obtained from:	ABT Systems Ltd
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2016-09-08 12:39:03 +00:00
Andrew Turner
77c02eccb8 When synchronising the instruction and data caches we only need to clean
the data cache to the point of unification. This is the point where the
two caches are unified to a single unified cache so cleaning past here
is just extra unneeded work.

This was noticed when investigating r305545.

Reported by:	bz
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-09-07 16:46:54 +00:00
Andrew Turner
3b34364450 Only call cpu_icache_sync_range when inserting an executable page. If the
page is non-executable the contents of the i-cache are unimportant so this
call is just adding unneeded overhead when inserting pages.

While doing research using gem5 with an O3 pipeline and 1k/32k/1M iTLB/L1
iCache/L2 Bjoern Zeeb (bz@) observed a fairly high rate of calls into
arm64_icache_sync_range() from pmap_enter() along with a high number of
instruction fetches and iTLB/iCache hits.

Limiting the calls to arm64_icache_sync_range() to only executable pages,
we observe the iTLB and iCache Hit going down by about 43%. These numbers
are quite misleading when looked at alone as at the same time instructions
retired were reduced by 19.2% and instruction fetches were reduced by 38.8%.
Overall this reduced the runtime of the test program by 22.4%.

On Juno hardware, in steady-state, running the same test, using the cycle
count to determine runtime, we do see a reduction of up to 28.9% in runtime.

While these numbers certainly depend on the program executed, we expect an
overall performance improvement.

Reported by:	bz
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-09-07 16:22:05 +00:00
Wojciech Macek
9d6cd3d858 Introduce support for Annapurna Alpine CCU and NB devices
This commit adds drivers for Alpine Cache Coherency Unit
and North Bridge Service whose task is to configure
the system fabric and enable cache coherency.

Obtained from:         Semihalf
Submitted by:          Michal Stanek <mst@semihalf.com>
Sponsored by:          Annapurna Labs
Reviewed by:           wma
Differential Revision: https://reviews.freebsd.org/D7565
2016-09-07 05:34:41 +00:00
Jared McNeill
0fbb017195 Add generic device-tree cpufreq driver. 2016-09-06 21:18:14 +00:00
Andrew Turner
705cb30cae Enable superpages on arm64 by default. These seem to be stable, having
survived multiple world and kernel builds, and of poudriere building full
package sets.

I have observed a 3% reduction in buildworld times with superpages enabled,
however further testing is needed to see if this is observed in other
workloads.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-09-04 17:50:23 +00:00
Mark Johnston
dbbaf04f1e Remove support for idle page zeroing.
Idle page zeroing has been disabled by default on all architectures since
r170816 and has some bugs that make it seemingly unusable. Specifically,
the idle-priority pagezero thread exacerbates contention for the free page
lock, and yields the CPU without releasing it in non-preemptive kernels. The
pagezero thread also does not behave correctly when superpage reservations
are enabled: its target is a function of v_free_count, which includes
reserved-but-free pages, but it is only able to zero pages belonging to the
physical memory allocator.

Reviewed by:	alc, imp, kib
Differential Revision:	https://reviews.freebsd.org/D7714
2016-09-03 20:38:13 +00:00
Jared McNeill
d69d5ab04f Add support for Allwinner A64 thermal sensors. 2016-09-03 15:26:00 +00:00
Andrew Turner
af6936890a Add a pc_clock pcpu field and use it to implement cpu_est_clockrate. This
will allow drivers that manage the clock frequency to communicate this with
the reset of the kernel.

Reported by:	jmcneill
MFC after:	1 week
Sponsored by:	ABT Systems Ltd
2016-09-02 10:13:51 +00:00
Andrew Turner
2c97ce35a7 Fix arm64 superpages bugs in pmap_enter:
* Pass the correct virtual address when demoting a superpage
 * Use the correct l3 table after demoting a superpage
 * Remove an invalid KASSERT hit demoting then promoting a superpage [1]

With this it is believed that  superpages on arm64 is stable.

Reported by:	[1] cognet
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-09-01 10:26:06 +00:00
Andrew Turner
d953ec3222 Also handle instruction traps. We might hit these when the page we are
executing is being promoted to a superpage.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-31 13:58:32 +00:00
Andrew Turner
7090a45142 Because we need to use a break-before-make sequence when promoting pages
there is a short period where functions that walk the kernel page table
without locking them may see an invalid entry. One solution would be to add
locking to these functions, however some may be called from locations where
we are unable to sleep.

Until a better solution can be found stop promoting pages in the kernel
pmap so these functions work as expected.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-30 16:45:15 +00:00
Andrew Turner
bc3aca4bf9 Print both the kernel read and write translation in DDB when asking for
a virtual to physical translation. These may be different, e.g. when a
page is mapped as read-only.

MFC after:	1 month
Sponsored by:	ABT Systems Ltd
2016-08-27 10:30:20 +00:00
Andrew Turner
d92a1b6ab3 Fix an assert, it should check if, when moving from 1 l1 to 512 l2 blocks,
the l2 entry is a block type and not an l3 page.

While here fix the string to correct the level name and add a missing ')'.

Obtained from:	ABT Systems Ltd
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-08-25 14:42:29 +00:00
Jared McNeill
1567e084b3 Add support for Allwinner A64.
Reviewed by:	andrew, manu
Relnotes:	yes
2016-08-25 13:59:19 +00:00
Andrew Turner
d5911afb79 Map coherent memory in a non-coherent dma tag as uncached. This is similar
to what the 32-bit arm code does, with the exception that it always assumes
the tag is non-coherent.

Tested by:	jmcneill
Obtained from:	ABT Systems Ltd
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-08-25 10:53:03 +00:00