Commit Graph

2770 Commits

Author SHA1 Message Date
Wojciech Macek
55b823e52a PPC64: add CXGBE and remove AHCI from GENERIC64
Add CXGBE driver which is required for PowerNV system.
Also, remove AHCI which does not work in BigEndian.

Created by:            Wojciech Macek <wma@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          QCM Technologies
2018-01-17 09:33:16 +00:00
Wojciech Macek
6005affb74 PowerNV: workaround console on OPAL 5.4
FreeBSD prints text char-by-char, which is not what OPAL
is designed to. Poll events more frequently to avoid buffer
overflow and loosing data.

Created by:            Wojciech Macek <wma@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          QCM Technologies
2018-01-17 08:01:51 +00:00
Wojciech Macek
5c3e53ef19 PowerNV: make PowerNV PCIe working on a real hardware
Fixes:
- map all devices to PE0
- use 1:1 TCE mapping
- provide the same TCE mapping for all PEs (not only PE0)
- add TCE reset and alignment (required by OPAL)

Created by:            Wojciech Macek <wma@semihalf.com>
Obtained from:         Semihalf
Sponsored by:          QCM Technologies
2018-01-17 07:39:11 +00:00
Wojciech Macek
8fc8068eba PowerNV: XICS support for PowerNV/OPAL
Make XICS to be OPAL-aware.

Created by:            Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by:          Wojciech Macek <wma@semihalf.com>
Sponsored by:          FreeBSD Foundation
2018-01-16 06:24:19 +00:00
Justin Hibbits
e64428edf7 Make fsl_sata driver work on P1022
P1022 SATA controller may set the wrong CCR bit for a command completion.
This would previously cause an interrupt storm.  Solve this by marking all
commands complete, and letting the end_transaction deal with the successes.
Causes no problems on P5020.

While here, fix a minor bug in collision detection.  The Freescale SATA
controller only has 16 slots, not 32.
2018-01-16 04:50:23 +00:00
Pedro F. Giffuni
6d5bc1bcab powerpc: make some use of mallocarray(9).
Focus on code where we are doing multiplications within malloc(9). None of
these ire likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.

X-Differential revision: https://reviews.freebsd.org/D13837
2018-01-15 21:10:40 +00:00
Nathan Whitehorn
fc8ea4be2a Install the SLB miss trap-handling code in the SLB-based MMU driver set up,
to which it is specific, rather than in the generic AIM startup code. This
will be required to support the radix-table-based MMU introduced with POWER9.
2018-01-15 16:08:34 +00:00
Nathan Whitehorn
04329fa708 Move the pmap-specific code in copyinout.c that gets pointers to userland
buffers into a new pmap-module function pmap_map_user_ptr() that can
be implemented by the respective modules. This is required to implement
non-segment-based AIM-ish MMU systems such as the radix-tree page tables
introduced by POWER ISA 3.0 and present on POWER9.

Reviewed by:	jhibbits
2018-01-15 06:46:33 +00:00
Nathan Whitehorn
68b9c019aa Document places we assume that physical memory is direct-mapped at zero by
using a new macro PHYS_TO_DMAP, which deliberately has the same name as the
equivalent macro on amd64. This also sets the stage for moving the direct
map to another base address.
2018-01-13 23:14:53 +00:00
Justin Hibbits
4a20766452 Include only the headers needed
The extra headers came through evolution of the file.
2018-01-13 21:10:42 +00:00
Justin Hibbits
8e14018389 Add SPDX identifier to header
Reported by:	pfg
2018-01-13 17:25:48 +00:00
Nathan Whitehorn
222393d5ca Chase removal of FDT fixup code on PowerPC in r327907. 2018-01-13 03:09:05 +00:00
Justin Hibbits
e9f96ff457 Enable L2 cache on supported PowerQUICC and QorIQ platforms
Some PowerQUICC and QorIQ platforms have a L2 cache managed via the
memory-mapped configuration registers, and appear as a node in the device
tree.  This adds basic support to enable the cache.
2018-01-13 01:36:37 +00:00
Jeff Roberson
6f4acaf4c9 Add support for NUMA domains to bus dma tags. This causes all memory
allocated with a tag to come from the specified domain if it meets the
other constraints provided by the tag.  Automatically create a tag at
the root of each bus specifying the domain local to that bus if
available.

Reviewed by:	jhb, kib
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13545
2018-01-12 23:34:16 +00:00
Jeff Roberson
ab3185d15e Implement NUMA support in uma(9) and malloc(9). Allocations from specific
domains can be done by the _domain() API variants.  UMA also supports a
first-touch policy via the NUMA zone flag.

The slab layer is now segregated by VM domains and is precise.  It handles
iteration for round-robin directly.  The per-cpu cache layer remains
a mix of domains according to where memory is allocated and freed.  Well
behaved clients can achieve perfect locality with no performance penalty.

The direct domain allocation functions have to visit the slab layer and
so require per-zone locks which come at some expense.

Reviewed by:	Attilio (a slightly older version)
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
2018-01-12 23:25:05 +00:00
Wojciech Macek
504d9b6029 PowerNV: update OPAL driver
Update OPAL driver with:
- better console support
- proper AP configuration
- enhanced IRQ/OFW mapping
- RTC support

Created by:            Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by:          Wojciech Macek <wma@semihalf.com>
Sponsored by:          FreeBSD Foundation
2018-01-12 12:14:52 +00:00
Wojciech Macek
ac9b43252a PowerNV: initial support for PCIe host controller
Provide initial support for PCIe host controller as
well as for IOMMU mapping. This commit allows proper
bus enumeration, but does not guarantee DMA operations
are working.

Created by:            Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by:          Wojciech Macek <wma@semihalf.com>
Sponsored by:          FreeBSD Foundation
2018-01-12 07:55:49 +00:00
Wojciech Macek
fc1689021f PowerNV: add buffer for OPAL console
Avoid the lock in vtophys() by providing a static direct-mapped
spinlock- protected output buffer to use when the console driver
cannot acquire locks for some reason. This allows the idle thread
to use printf() (e.g. the SMP startup messages) without crashing
the kernel.

Created by:            Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by:          Wojciech Macek <wma@freebsd.org>
Sponsored by:          FreeBSD Foundation
2018-01-11 09:42:24 +00:00
Wojciech Macek
c024897601 PowerNV: set LPCR[LPES] correctly
Make sure to set LPCR[LPES] so that external interrupts set SRR0 and SRR1
instead of HSRR0 and HSRR1. Without this, external interrupt handlers would
get the wrong MSR value when executing, causing eventual madness.

Created by:            Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by:          Wojciech Macek <wma@freebsd.org>
Sponsored by:          FreeBSD Foundation
2018-01-11 09:39:38 +00:00
Wojciech Macek
01d7bda7b7 PowerNV: correctly start secondary CPUs
Fix AP startup, which was broken.

Created by:            Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by:          Wojciech Macek <wma@freebsd.org>
Sponsored by:          FreeBSD Foundation
2018-01-11 09:34:33 +00:00
Wojciech Macek
32d1354a39 PowerNV: add reset, poweroff, OPAL console
Add basic power control (reset, power off) and bind
ttyuX to opal console so that init will start login there.

Created by:            Nathan Whitehorn <nw@freebsd.org>
Submitted by:          Wojciech Macek <wma@freebsd.org>
Sponsored by:          FreeBSD Foundation
2018-01-11 09:26:28 +00:00
Wojciech Macek
fb3855e0e7 PowerNV: initial support for OPAL
OPAL is a dedicated firmware acting as a hypervisor.
Add generic functions to provide all access.

Created by:            Nathan Whitehorn <nw@freebsd.org>
Submitted by:          Wojciech Macek <wma@freebsd.org>
2018-01-11 07:40:06 +00:00
Landon J. Fuller
a1df0d9592 Fix minor locking issues in the Power Mac Uninorth PCI bridge driver.
- Call resource_int_value() once during attach, rather than within the
  pci_(read|write)_config() code path; this avoids taking a blocking mutex
  to read kenv variables.

- Use a spin lock to protect non-atomic config space accesses; this matches
  the behavior of Darwin's AppleMacRiscPCI driver.

Reviewed by:	jhibbits
Differential Revision:	https://reviews.freebsd.org/D13839
2018-01-10 22:19:11 +00:00
Nathan Whitehorn
566a135bd5 Add XHCI support to powerpc64 GENERIC. This is useful to get input devices
supported on newer POWER hardware and in graphical VMs run on the same,
which are typically XHCI-only. The 32-bit GENERIC kernel, which
does not run on hardware made in the last decade and is unlikely to
encounter XHCI devices, is left unchanged.

PR:		kern/224940
Submitted by:	Gustavo Romero
MFC after:	1 week
2018-01-09 19:41:10 +00:00
Nathan Whitehorn
09f07b0017 Revert r327360, which can cause boot problems on high-CPU-count (>60)
POWER8 and POWER9 systems, pending further analysis.

PR:		224841
2018-01-04 23:07:51 +00:00
Andreas Tobler
7e792cb8f5 The recent bump of MAXDSIZ made 32-bit binary execution on 64-bit powerpc fail.
The data segement was too big.

Add a fix-up function like on ia32 for MAXDSIZ.

While here, bring also the MAXSSIZ closer to amd64 and add an equal fix-up
function for MAXSSIZ.

Reviewed by:	jhibbits@
Obtained from:  jhibbits@
Differential Revision:	https://reviews.freebsd.org/D13753
2018-01-03 20:20:43 +00:00
Nathan Whitehorn
67530f82dd Fix reversed endianness that crept in at some point. Blue is now blue
instead of pink.

MFC after:	3 days
2018-01-02 03:59:46 +00:00
Nathan Whitehorn
3972f4c1d4 Remove PIR from PCPU data. It has an implementation-defined meaning that
is of limited utility outside of platform-specific code and can vary
at runtime when running as a hypervisor guest, so does not even have the
virtue of being a static identifier.

Reviewed by:	jhibbits
2017-12-31 20:23:39 +00:00
Nathan Whitehorn
4e05ac247c Fix 32-bit build. 2017-12-31 20:20:55 +00:00
Nathan Whitehorn
f81dfc7f6b Make newer binutils happy by using a bl-type branch instead of b, which
displeases it for some reason. LR is not relevant in this code, so just
do what it wants.
2017-12-31 20:10:08 +00:00
Nathan Whitehorn
ec75f647cc Provide relative, as well as absolute, addresses in trap panic panics. This
makes it easier to cross-correlate them with instruction listings without
worrying about where the kernel was relocated to.

MFC after:	1 week
2017-12-31 20:08:16 +00:00
Colin Percival
d5d7606c0c Use the TSLOG framework to record entry/exit timestamps for DELAY and
_vprintf; these functions are called in many places and can contribute
meaningfully to the total time spent booting.
2017-12-31 09:24:41 +00:00
Nathan Whitehorn
5261ac0eda Use data from the boot loader to pick the appropriate output graphics mode
instead of hard-coding a default. This information is passed implicitly by
the PS3 firmware and can be relied upon. Also adjust the default mode, if
somehow firmware doesn't pass one, to 1920x1080 from 720x480 since it is
2017.

MFC after:	2 weeks
2017-12-31 06:10:07 +00:00
Nathan Whitehorn
a891d21aac Make sure the first instruction of the low-memory spinloop is in the
cacheline being invalidated.

MFC after:	1 month
2017-12-31 05:38:19 +00:00
Nathan Whitehorn
3fca788024 Remove logic for early console with loader.ps3 now that loader.ps3 is dead. 2017-12-30 20:25:33 +00:00
Nathan Whitehorn
ba06dbb874 Change the way SMP startup works to match the new multi-AP features in
locore64.S introduced in r327358.

MFC after:	3 weeks
2017-12-30 20:24:33 +00:00
Nathan Whitehorn
f9d6e0a5d0 Enhance the CHRP/pSeries platform layer:
- Densely number CPUs to avoid systems with CPUs with very high ID numbers
- Always have the BSP be CPU 0 to avoid remnant brokenness with non-0 BSPs
  in other parts of the kernel.
- Improve parsing of the device tree CPU listings on SMT systems.
- Allow reboot via RTAS as well as OF for pSeries systems booted by FDT
  without functioning Open Firmware.

Obtained from:	projects/powernv
MFC after:	3 weeks
2017-12-29 21:09:17 +00:00
Nathan Whitehorn
70f654991a Add support for 64-bit PowerPC kernels to be directly loaded by kexec, which
is used as the bootloader on a number of PPC64 platforms. This involves the
following pieces:
- Making the first instruction a valid kernel entry point, since kexec
  ignores the ELF entry value. This requires a separate section and linker
  magic to prevent the linker from filling the beginning of the section
  with stubs.
- Adding an entry point at 0x60 past the first instruction for systems
  lacking firmware CPU shutdown support (notably PS3).
- Linker script changes to support the above.

MFC after:	1 month
2017-12-29 20:30:10 +00:00
Nathan Whitehorn
8469e0fe35 Maintain alignment of in-code 64-bit quantities by design rather than luck.
If these are not aligned, the linker has to emit a different type of
relocation that the early boot self-relocation code cannot handle, even
in principle, resulting in them being set to zero and the kernel crashing.

MFC after:	1 week
2017-12-29 20:25:15 +00:00
Nathan Whitehorn
2ad331874e Remove ELF note for Open Firmware. It is marked optional in a single 1996
draft of a never-finalized standard (CHRP) and is irrelevant in practice
on FreeBSD since we load the kernel with loader(8) on Open Firmware
platforms anyway. Moreover, loader(8), which is directly loaded by Open
Firmware, has never had an equivalent note.

MFC after:	2 weeks
2017-12-28 23:49:53 +00:00
Eitan Adler
caa7e52f3f kernel: Fix several typos and minor errors
- duplicate words
- typos
- references to old versions of FreeBSD

Reviewed by:	imp, benno
2017-12-27 03:23:21 +00:00
Justin Hibbits
87879ba805 Increase default MAXDSIZ to 32G on powerpc64
Linking LLVM now seems to require more than 1GB data size, so increase the
default to 32G, which matches amd64.

Reviewed by:	nwhitehorn
2017-12-20 16:49:45 +00:00
Nathan Whitehorn
d6716aa2af The highest-order bit of the bootloader cookie is 1, with the result that
the 32-bit cookie can be sign-extended on its way out of the loader and
through Open Firmware. If sign-extended, the in-kernel check of its value
would fail on 64-bit systems, resulting in a mountroot prompt. Solve this
by telling the kernel to ignore the high-order bits.

PR:		kern/224437
Submitted by:	Gustavo Romero
2017-12-19 16:45:40 +00:00
Konstantin Belousov
30d4f9e888 Add atomic_load(9) and atomic_store(9) operations.
They provide relaxed-ordered atomic access semantic.  Due to the
FreeBSD memory model, the operations are syntaxical wrappers around
the volatile accesses.  The volatile qualifier is used to ensure that
the access not optimized out and in turn depends on the volatile
semantic as implemented by supported compilers.

The motivation for adding the operation is to help people coming from
other systems or knowing the C11/C++ standards where atomics have
special type and require use of the special access operations.  It is
still the case that FreeBSD requires plain load and stores of aligned
integer types to be atomic.

Suggested by:	jhb
Reviewed by:	alc, jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D13534
2017-12-19 09:59:20 +00:00
Justin Hibbits
7cd4e55c43 Handle the Facility Unavailable exception as a SIGILL
Currently Facility Unavailable is absent and once an application
tries to use or access a register from a feature disabled in the
CPU it causes a kernel panic.

A simple test-case is:

int main() { asm volatile ("tbegin.;"); }

which will use TM (Hardware Transactional Memory) feature which
is not supported by the kernel and so will trigger the following
kernel panic:

----

fatal user trap:

    exception       = 0xf60 (unknown)
    srr0            = 0x10000890
    srr1            = 0x800000000000f032
    lr              = 0x100004e4
    curthread       = 0x5f93000
    pid = 1021, comm = htm

panic: unknown trap
cpuid = 40
KDB: stack backtrace:
Uptime: 3m18s
Dumping 10 MB (3 chunks)
    chunk 0: 11MB (2648 pages) ... ok
    chunk 1: 1MB (24 pages) ... ok
    chunk 2: 1MB (2 pages)panic: IOMMU mapping error: -4

cpuid = 40
Uptime: 3m18s

----

Since Hardware Transactional Memory is not yet supported by FreeBSD, treat
this as an illegal instruction.

PR:		224350
Submitted by:	Gustavo Romero <gromero_AT_ibm_DOT_com>
MFC after:	2 weeks
2017-12-15 04:11:20 +00:00
Justin Hibbits
9ee02cd6f8 Add identifier for POWER9 CPU to CPU list
Without the identifier in the list booting FreeBSD results in printing the
following (from a PowerKVM boot):

cpu0: Unknown PowerPC CPU revision 0x1201, 2550.00 MHz

For now, add the same feature list as POWER8.  As new capabilities are added to
support POWER9 specific features, they will be added to this.

PR:		224344
Submitted by:	Breno Leitao <breno_DOT_leitao_AT_gmail_DOT_com>
2017-12-14 20:01:04 +00:00
Justin Hibbits
bf1b92967f Decode some PowerPC trap registers
Decode on Book-E:
* ESR (Exception Syndrome Register)
* MCSR (Machine Check Status Register)

On AIM:
* MSSSR (Memory Subsystem Status Register)

Makes it easier to tell at a glance the type of trap and machine check
conditions now.
2017-12-12 03:16:10 +00:00
Mark Johnston
5bab623438 Pass the trap frame to fasttrap hooks.
The DTrace fasttrap entry points expect a struct reg containing the
register values of the calling thread. Perform the conversion in
fasttrap rather than in the trap handler: this reduces the number of
ifdefs and avoids wasting stack space for traps that don't involve
DTrace.

MFC after:	2 weeks
2017-12-11 19:21:39 +00:00
Justin Hibbits
713e844971 Retrieve the page outside of holding locks
pmap_track_page() only works with physical memory pages, which have a
constant vm_page_t address.  Microoptimize pmap_track_page() to perform one
less operation under the lock.
2017-12-10 04:43:27 +00:00
Justin Hibbits
94a9d7c3b9 Remove PTE VA mappings for tracked pages in 64-bit mode
This was done in 32-bit mode, but not duplicated when 64-bit mode was
brought in.  Without this, stale mappings can be left, leading to odd
crashes when the wrong VA is checked in XX_PhysToVirt() (dpaa(4)).
2017-12-08 03:49:53 +00:00