Skylake Xeons.
See SDM rev. 68 Vol 3 4.6.2 Protection Keys and the description of the
RDPKRU and WRPKRU instructions.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D18893
When sigreturn() restored a thread's context, SRR1 was being restored
to its previous value, but pcb_flags was not being touched.
This could cause a mismatch between the thread's MSR and its pcb_flags.
For instance, when the thread used the FPU for the first time inside
the signal handler, sigreturn() would clear SRR1, but not pcb_flags.
Then, the thread would return with the FPU bit cleared in MSR and,
the next time it tried to use the FPU, it would fail on a KASSERT
that checked if the FPU was disabled.
This change clears the FPU bit in both pcb_flags and frame->srr1,
as the code that restores the context expects to use the FPU trap
to re-enable it.
PR: 234539
Reported by: sbruno
Reviewed by: jhibbits, sbruno
Differential Revision: https://reviews.freebsd.org/D19166
Newer cores have the 'tlbilx' instruction, which doesn't broadcast over
CoreNet. This is significantly faster than walking the TLB to invalidate
the PID mappings. tlbilx with the arguments given takes 131 clock cycles to
complete, as opposed to 512 iterations through the loop plus tlbre/tlbwe at
each iteration.
MFC after: 3 weeks
At moea64_sync_icache(), when the 'va' argument has page size
alignment, round_page() will return the same value as 'va'.
This would cause 'len' to be 0 and thus an infinite loop.
With this change, 'lim' will always point to the next page boundary.
This issue occurred especially during debugging sessions, when a breakpoint
was placed on an exact page-aligned offset, for instance.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D19149
SoCs with e500v2 chips only have at most 2 cores, and there are no plans to
release any more e500v2-based SoCs. Clamping MAXCPU down to 2 saves 5MB of
data, and 1.5MB bss.
For direct mapped kernel addresses, ppc64 function was not
performing the dmap to physical conversion, before jumping
to the code that fetched the value from physical memory.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D19086
The QorIQ SoCs don't actually support multicast interrupts, and the
references state explicitly that multicast is undefined behavior. Avoid the
undefined behavior by binding to only a single CPU, using a quirk to
determine if this is necessary.
MFC after: 3 weeks
When running several builders in parallel, on QEMU, with 8GB of
memory, a fatal kernel trap (0x300 (data storage interrupt))
caused by llan driver is sometimes observed, when the system
starts to run out of swap space.
This happens because, at llan_intr(), a phyp call to add a
logical LAN buffer is always made when llan_add_rxbuf() fails,
even if it fails to allocate a new buffer.
PR: 235489
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D19084
It appears idling via 'wait' on e5500 causes strange behaviors, such as
top(1) simply hanging sporadically, until input. Until this can possibly be
sorted out (interrupt issue?), just don't idle on this hardware. The SoCs
are low power already, and the wait state doesn't save much anyway.
Currently, the trap code switches to the the temporary stack in the dbtrap
section. It works in most cases, but in the beginning of the execution, the
temp stack is being used, as starting in the powerpc_init() code.
In this current scenario, the stack is being overwritten, which causes the
return of breakpoint() to take abnormal execution.
This current patchset create a small stack to use by the dbtrap: codepath
avoiding the corruption of the temporary stack.
PR: 224872
Submitted by: breno.leitao_gmail.com
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D14484
The XIVE (External Interrupt Virtualization Engine) is a new interrupt
controller present in IBM's POWER9 processor. It's a very powerful,
very complex device using queues and shared memory to improve interrupt
dispatch performance in a virtualized environment.
This yields a ~10% performance improvment over the XICS emulation mode,
measured in both buildworld, and 'dd' from nvme to /dev/null.
Currently, this only supports native access.
MFC after: 1 month
iflib is already a module, but it is unconditionally compiled into the
kernel. There are drivers which do not need iflib(4), and there are
situations where somebody might not want iflib in kernel because of
using the corresponding driver as module.
Reviewed by: marius
Discussed with: erj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D19041
The powerpc_intr structure is not zero-initialized, so on an invariants
build would panic in the xics driver with an invalid pointer. Also fix the
xics driver to share the private data setup code between xics_enable() and
xics_bind().
Reported by: Leonardo Bianconi
If fsqrts is emulated with +INF as its argument, the 0 return value causes a
NULL pointer dereference, panicking the system. Follow the PowerISA and
return +INF with no FP exception.
MFC after: 1 week
Don't clobber the low part of the register restoring the high component of.
This could lead to very bad behavior if it's an ABI-affected register.
While here, also mark the asm volatile in the SPE high save case, to match
the load case.
Reported by: Branden Bergren (git_bdragon.rtk0.net)
MFC after: 1 week
Summary:
I was working on implementing ifuncs on powerpc64 elfv2 today, and I suddenly
realized that the reason I was having so much trouble with AT_HWCAP and
AT_HWCAP2 is they are missing from the sysentvec.
After adding them, the auxv is being filled like it should.
Submitted by: Brandon Bergren (git_bdragon.rtk0.net)
Differential Revision: https://reviews.freebsd.org/D18575
The IPI vector is static, and happens to be the most common interrupt by far
on some systems. Rather than searching for the interrupt every time, cache
the index.
This appears to yield a small performance boost, of about 8% reduction in
buildworld times, on my POWER9 system, when paired with r342975.
The XICS and XIVE need extra data beyond irq and vector. Rather than
performing a separate search, it's better for the general interrupt facility
to hold a private pointer, since the search already must be done anyway at
that level.
In r342771, I introduced a regression in Power by abusing the platform
smp_topo() method as a shortcut for providing the MI information needed for
the stated sysctls. The smp_topo() method was already called later by
sched_ule (under the name cpu_topo()), and initializes a static array of
scheduler topology information. I had skimmed the smp_topo_foo() functions
and assumed they were idempotent; empirically, they are not (or at least,
detect re-initialization and panic).
Do the cleaner thing I should have done in the first place and add a
platform method specifically for core- and thread-count probing.
Reported by: luporl via jhibbits
Reviewed by: luporl
X-MFC-With: r342771
Differential Revision: https://reviews.freebsd.org/D18777
With new sysctls (to the best of our ability do detect them). Restructured
smp.4 slightly for clarity (keep relevant stuff closer to the top) while
documenting.
Reviewed by: markj, jhibbits (ppc parts)
MFC after: 3 days
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D18322
Previous commits have made VM_MIN_KERNEL_ADDRESS its own separate entity,
and rebased the kernel around that address instead of KERNBASE. This commit
pulls the trigger to rebase KERNBASE to a physical load address. The
eventual goal is to align the address with the AIM KERNBASE, but at this
time that's not an option.
Currently a Book-E kernel must be loaded on a 64MB boundary, due to size
issues. The common load address is at the 64MB mark (0x04000000), so simply
make that the default KERNBASE.
As of this commit, Book-E kernels can be loaded and booted with ubldr.
MFC after: 3 weeks
Optimize the exception handler to only save and load the upper word of the
GPRs used in the emulating instruction. This reduces the save/load
overhead, and as a side effect does not overwrite the upper word of any
temporary register.
With this commit I am now able to run editors/abiword and math/gnumeric on a
e500-based system.
MFC after: 1 week
MFC With: r341752,r341751
The code was a near exact copy of the code in startup, but it doesn't need
the complexity since the kernel is already relocated. With
VM_MIN_KERNEL_ADDRESS as currently set to KERNBASE, this doesn't cause a
problem, because it's a zero offset. However, when KERNBASE is changed to a
physical load address, it then has a non-zero offset, and ends up with an
invalid stack pointer, causing the AP to hang.
This change adds a hypervisor trap handler for exception 0x1500 (soft patch),
normalizing all VSX registers and returning.
This avoids a kernel panic due to unknown exception.
Change made with the collaboration of leonardo.bianconi_eldorado.org.br,
that found out that this is a hypervisor exception and not a supervisor one,
and fixed this in the code.
Reviewed by: jhibbits, sbruno
Differential Revision: https://reviews.freebsd.org/D17806
mpr as a module for powerpc or mips. An upcoming commit will cause these
drivers to rely on the presence of 64bit atomic operations. Discussed
with jhibbits.
The POWER9 does not recognize 'or 27,27,27' as a thread priority NOP. On
earlier POWER architectures, this NOP would note to the processor to give up
resources if able, to improve performance of other threads.
All processors that support the thread priority NOPs recognize the
'or 31,31,31' NOP as very low priority, so use this to perform a similar
function, and not burn cycles on POWER9.
The jump slot is a function pointer, not a descriptor pointer, in ELFv2. Just
write the pointer itself over, not the contents of the pointer, which would be
the first instruction of the function.
The 'interrupts' property is actually 2 words, not one, on macgpio child
nodes. Open Firmware's getprop function might be returning the value
copied, not the total size of the property, but FDT's returns the total
size. Prior to this patch, this would cause the SYS_RES_IRQ resource list
to not be populated when running with the 'usefdt' loader variable set, to
convert the OFW device tree to a FDT. Since the property is always 2 words,
read both words, and ignore the second.
Tested by: Dennis Clarke (previous attempt)
MFC after: 2 weeks
Mark some buses as BUS_PASS_BUS, and some resources as BUS_PASS_RESOURCE.
This also decouples some resource attachment orderings from being races by
device tree ordering, instead relying on the bus pass to provide the
ordering.
This was originally intended to support multipass suspend/resume, but it's
also needed on PowerMacs when using fdt, as the device tree seems to get
created in reverse of the OFW tree.
Reviewed by: nwhitehorn (long ago)
Differential Revision: https://reviews.freebsd.org/D918
The same behavior was moved to machdep.c, paired with AIM's relocation,
making this redundant. With this, it's now possible to boot FreeBSD with
ubldr on a uboot Book-E platform, even with a
KERNBASE != VM_MIN_KERNEL_ADDRESS.
It's possible the fdt pointer was passed in via the metadata, as is done in
ubldr. Check for the fdt here, instead of working with a NULL fdt, and
panicking.
The metadata pointer will almost never be at or above 'btext', as btext is a
relocated symbol, so will be based at VM_MIN_KERNEL_ADDRESS, not at
KERNBASE. Check the address against kernload, where the kernel is
physically loaded.
Fix p-state setting on Power8 by removing the accidental double-indirection of
the pstate_ids table.
The pstate_ids table comes from the OF property "ibm,pstate-ids." On Power9,
the values happen to be identical to the indices, so the extra indirection was
harmless. On Power8, the values were out of the range [0, npstates], so
pmcr_set() would fail the spec[0] range check with EINVAL.
While here, include both the value and index in the driver-specific register
array as spec[0] and spec[1] respectively. They're redundant, but relatively
harmless, and it may aid debugging.
While here, fix the range check to exclude the index npstates, which is one
past the last valid index.
PR: 233693
Reported and tested by: sbruno
Reviewed by: jhibbits
Summary:
Currently, the upcall used to create threads assumes ELFv1.
Instead, we should check which sysentvec is in use on the process and act
accordingly.
This makes ELFv2 threaded processes work.
Submitted by: git_bdragon.rtk0.net
Differential Revision: https://reviews.freebsd.org/D18330
A related future change, which changes KERNBASE for Book-E for some reason
causes a "KERNBASE redefined" error with assym.inc, even though it only changed
the value of KERNBASE and nothing else. Since machine/vmparam.h is already
included in booke/locore.S, and the requisite guards are already in place for
properly handling KERNBASE in vmparam.h, just remove it from genassym, and
include vmparam.h in the AIM locore files.
Book-E kernels really run at VM_MIN_KERNEL_ADDRESS, which currently happens to
be the same as KERNBASE. KERNBASE is the linked address, which the loader also
takes to be the physical load address. Treat KERNBASE as a physical address,
not a virtual, and change virtual address references for KERNBASE to use
something more appropriate.
