Commit Graph

243 Commits

Author SHA1 Message Date
Pedro F. Giffuni
910c079886 sys/powerpc: make use of the howmany() macro when available.
We have a howmany() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
2016-04-26 14:44:49 +00:00
Pedro F. Giffuni
d9c9c81c08 sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
2016-04-21 19:57:40 +00:00
Justin Hibbits
f60708c9f7 Fix SMP booting for PowerPC Book-E
Summary:
PowerPC Book-E SMP is currently broken for unknown reasons.  Pull in
Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP.

This eliminates the shared software TLB1 table, replacing it with
tlb1_read_entry() function.

This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs
already released (ePAPR boot releases APs to a spin loop waiting on a specific
address).  This will be addressed in the near future by using the MPIC to reset
the AP into our own alternate boot address.

This does include a change to the dpaa/dtsec(4) driver, to mark the portals as
CPU-private.

Test Plan:
Tested on Amiga X5000/20 (P5020).  Boots, prints the following
messages:

 Adding CPU 0, pir=0, awake=1
 Waking up CPU 1 (dev=1)
 Adding CPU 1, pir=20, awake=1
 SMP: AP CPU #1 launched

top(1) shows CPU1 active.

Obtained from:	Semihalf
Relnotes:	Yes
Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
Justin Hibbits
f00e990465 VM_MAXUSER_ADDRESS is highest page start, not highest address.
In case a single page mapping is requested first, which might overlap the user
address space, fix the device map block to the next page.
2016-04-10 15:50:45 +00:00
Justin Hibbits
f2c3b7f2ba Restructure device mappings for Book-E.
Summary:
There is currently a 1GB hole between user and kernel address spaces
into which direct (1:1 PA:VA) device mappings go.  This appears to go largely
unused, leaving all devices to contend with the 128MB block at the end of the
32-bit space (0xf8000000-0xffffffff).  This easily fills up, and needs to be
densely packed.  However, dense packing wastes precious TLB1 space, of which
there are only 16 (e500v2) or 64(e5500) entries available.

Change this by using the 1GB space for all device mappings, and allow the kernel
to use the entire upper 1GB for KVA.  This also allows us to use sparse device
mappings, freeing up TLB entries.

Test Plan: Boot tested on p5020.

Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
Justin Hibbits
0aeed3e993 Add support for the Freescale dTSEC DPAA-based ethernet controller.
Freescale's QorIQ line includes a new ethernet controller, based on their
Datapath Acceleration Architecture (DPAA).  This uses a combination of a Frame
manager, Buffer manager, and Queue manager to improve performance across all
interfaces by being able to pass data directly between hardware acceleration
interfaces.

As part of this import, Freescale's Netcomm Software (ncsw) driver is imported.
This was an attempt by Freescale to create an OS-agnostic sub-driver for
managing the hardware, using shims to interface to the OS-specific APIs.  This
work was abandoned, and Freescale's primary work is in the Linux driver (dual
BSD/GPL license).  Hence, this was imported directly to sys/contrib, rather than
going through the vendor area.  Going forward, FreeBSD-specific changes may be
made to the ncsw code, diverging from the upstream in potentially incompatible
ways.  An alternative could be to import the Linux driver itself, using the
linuxKPI layer, as that would maintain parity with the vendor-maintained driver.
However, the Linux driver has not been evaluated for reliability yet, and may
have issues with the import, whereas the ncsw-based driver in this commit was
completed by Semihalf 4 years ago, and is very stable.

Other SoC modules based on DPAA, which could be added in the future:
* Security and Encryption engine (SEC4.x, SEC5.x)
* RAID engine

Additional work to be done:
* Implement polling mode
* Test vlan support
* Add support for the Pattern Matching Engine, which can do regular expression
  matching on packets.

This driver has been tested on the P5020 QorIQ SoC.  Others listed in the
dtsec(4) manual page are expected to work as the same DPAA engine is included in
all.

Obtained from:	Semihalf
Relnotes:	Yes
Sponsored by:	Alex Perez/Inertial Computing
2016-02-29 03:38:00 +00:00
Justin Hibbits
0f7aeab0e7 Implement pmap_change_attr() for PowerPC (Book-E only for now)
Summary:
Some drivers need special memory requirements.  X86 solves this with a
pmap_change_attr() API, which DRM uses for changing the mapping of the GART and
other memory regions.  Implement the same function for PowerPC.  AIM currently
does not need this, but will in the future for DRM, so a default is added for
that, for business as usual.  Book-E has some drivers coming down that do
require non-default memory coherency.  In this case, the Datapath Acceleration
Architecture (DPAA) based ethernet controller has 2 regions for the buffer
portals: cache-inhibited, and cache-enabled.  By default, device memory is
cache-inhibited.  If the cache-enabled memory regions are mapped
cache-inhibited, an alignment exception is thrown on access.

Test Plan:
Tested with a new driver to be added after this (DPAA dTSEC ethernet driver).
No alignment exceptions thrown, driver works as expected with this.

Reviewed By:	nwhitehorn
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
Justin Hibbits
debd17c5e7 Fix a panic bug that cropped up in the PTE rewrite.
PTE was getting overwritten by just the flags.

Pointy-hat to:	jhibbits
2016-02-16 02:13:55 +00:00
Justin Hibbits
64a982ea56 Migrate the PTE format for book-e to standardize on the 'indirect PTE' format
Summary:
The revised Book-E spec, adding the specification for the MMUv2 and e6500,
includes a hardware PTE layout for indirect page tables.  In order to support
this in the future, migrate the PTE format to match the MMUv2 hardware PTE
format.

Test Plan: Boot tested on a P5020 board.  Booted to multiuser mode.

Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
Justin Hibbits
49d36ba409 Older Book-E processors (e500v1/e500v2) don't support dcbzl.
The only difference between dcbzl and dcbz is dcbzl operates on native cache
line lengths regardless of L1CSR0[DCBZ32].  Since we don't change the cache line
size, the cacheline_size variable will reflect the used cache line length, and
dcbz will work as expected.
2016-01-26 04:41:18 +00:00
Justin Hibbits
30857edaea Fix a debug printf().
Somehow this printf() was missed in the conversion of vm_paddr_t to 64-bit, and
made it through until now.

Sponsored by:	Alex Perez/Inertial Computing
2016-01-26 03:52:14 +00:00
Justin Hibbits
e1a51e19e0 Revert a printf change from r294307.
Caused build failures with MPC85XX.

Pointy-hat to:	jhibbits
2016-01-19 23:35:12 +00:00
Justin Hibbits
ca31148c56 Hide most of the PTE initialization and management.
By confining the page table management to a handful of functions it'll be
easier to modify the page table scheme without affecting other functions.
This will be necessary when 64-bit support is added, and page tables become
much larger.
2016-01-19 03:07:25 +00:00
Nathan Whitehorn
ca496abd5a Remove dead code and dead comments, most notably the implemenation of the
now-obsolete setfault(). No NetBSD code exists in the AIM locore files, so
update the copyrights there.
2016-01-10 18:00:01 +00:00
Justin Hibbits
4717ada9cd Make arguments for booke_init() u_long, to match register width.
On powerpc64, pointers are 64 bits, so casting from uint32_t changes the integer
width.

The alternative was to use register_t, but I didn't see register_t used as
argument type for any other functions, though didn't look too closely.  u_long
was an acceptable alternative.  On 64-bit it's 64 bits, on 32-bit it's 32 bits.
2016-01-04 02:20:14 +00:00
Justin Hibbits
a917f636a2 Set the cacheline size before calling powerpc_init()
powerpc_init() initializes the mmu.  Since this may clear pages via
pmap_zero_page(), set the cacheline size before calling into it, so
pmap_zero_page() has the right cacheline size.  This isn't completely
necessary now, but will be when 64-bit book-e is completed.
2016-01-04 01:33:07 +00:00
Nathan Whitehorn
b0bf7fcd29 Bring CPU features list in line with the ABI requirements.
MFC after:	1 week
2016-01-02 18:15:10 +00:00
Justin Hibbits
3f068cbf5c Add platform support for QorIQ SoCs.
This includes the following changes:
* SMP kickoff for QorIQ (tested on P5020)
* Errata fixes for some silicon revisions
* Enables L2 (and L3 if available) caches
Obtained from:	Semihalf
Sponsored by:	Alex Perez/Inertial Computing
2015-12-30 03:43:25 +00:00
Justin Hibbits
eabf894627 Optimize zero_page for book-e mmu.
Instead of indirectly calling bzero() through mmu_booke_zero_page_area, zero the
full page the same way as the AIM pmap logic does: using dcbz.
2015-12-30 02:26:04 +00:00
Justin Hibbits
459021cc7d Rewrite tid_flush() in C.
There's no need for it to be in asm.  Also, by writing in C, and marking it
static in pmap.c, it saves a branch to the function itself, as it's only used in
one location.  The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
Justin Hibbits
b3936ebe07 Extend Book-E to support >4GB RAM
Summary:
With some additional changes for AIM, that could also support much
larger physmem sizes.  Given that 32-bit AIM is more or less obsolete, though,
it's not worth it at this time.

Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
Justin Hibbits
777d81af62 No need to reset tlb1 here, it gets reset again after BSS is cleared in
powerpc_init().

Also fix a comment typo (0x45 == E, not e)
2015-12-11 01:34:13 +00:00
Justin Hibbits
a39f10532a Add more interrupts handled for booke.
e500mc, e5500, and e6500 all use the normal FPU, with the same behavior as AIM
hardware.  e6500 also supports Altivec, so, although we don't yet have e6500
hardware to test on, add these IVORs as well.  Theoretically, since it boots the
same as a e5500, it should work, single-threaded, single-core, with full altivec
support as of this commit.

With this commit, and some other patches to be committed shortly FreeBSD now
boots on the P5020, single-core, all the way to user space, and should boot just
fine on e500mc.

Relnotes:	Yes (e500mc, e5500 support)
Sponsored by:	Alex Perez/Inertial Computing
2015-12-11 01:23:18 +00:00
Justin Hibbits
ef596c4021 trunc_page() goes through unsigned long, which is too short.
sizeof(unsigned long) < sizeof(vm_paddr_t) on Book-E, which uses 36-bit
addressing.  With this, a CCSR with a physical address above 4GB successfully
maps.

Sponsored by:	Alex Perez/Inertial Computing
2015-11-21 06:03:46 +00:00
Justin Hibbits
12d44566ae The TLB1 TSIZE is a multiple of 4, not 2, so shift 2 bits, not 1. 2015-08-29 06:52:14 +00:00
Justin Hibbits
afefc223b1 Extend pmap to support e500mc and e5500.
As part of this, clean up tlb1_init(), since bootinfo is always NULL here just
eliminate the loop altogether.

Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger
immediately following a smaller page, causing the mappings to overlap.  Instead,
break up the new mapping into smaller chunks.  The downside to this is that it
uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause
problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries).

Obtained from:	Semihalf (partial)
Sponsored by:	Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
Justin Hibbits
92f6e93414 Follow up to r287014
Missed these files, from the original diff.
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision:	https://reviews.freebsd.org/D3027
2015-08-22 07:27:06 +00:00
Justin Hibbits
b239c24baa Enhance book-e pmap for 36-bit physaddr
Summary:
This is (probably step 1) of enhancing the book-e pmap to support the full
36-bit physical address space on Freescale e500 and e5500 cores.

Thus far it has only been regression tested on one platform.  Since I only have
one other Book-E platform (e5500), that needs work beyond this, I haven't yet
tested it on this.

Test Plan: Regression tested on my RouterBoard RB800.

Reviewed By: marcel
Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
Justin Hibbits
7f7fcf5596 Add initial boot support for e500mc and e5500.
* Since r257190 the kernel must actually be loaded at a 64MB boundary, not 16MB.
* Don't program HID1 register on e500mc or e5500, they don't have this SPR.
* Set proper HID0 defaults for these new architectures.

There is still more work to be done for the various SoCs, and the PMAP code
still needs to be extended to 36-bit paddr, coming soon.

Obtained from:	Semihalf
Sponsored by:	Alex Perez/Inertial Computing
2015-08-21 02:41:35 +00:00
Justin Hibbits
13adf27426 Fix copy&paste. 2015-08-19 06:08:11 +00:00
Justin Hibbits
947c974b39 Save the registers at the correct offsets.
When merging the AIM and BookE trap.c files, the offsets for BookE's setfault
inadvertantly got munged.
2015-08-19 06:07:32 +00:00
Konstantin Belousov
edc8222303 Make kstack_pages a tunable on arm, x86, and powepc. On i386, the
initial thread stack is not adjusted by the tunable, the stack is
allocated too early to get access to the kernel environment. See
TD0_KSTACK_PAGES for the thread0 stack sizing on i386.

The tunable was tested on x86 only.  From the visual inspection, it
seems that it might work on arm and powerpc.  The arm
USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already
incorrect for the threads with non-default kstack size.  I only
changed the macros to use variable instead of constant, since I cannot
test.

On arm64, mips and sparc64, some static data structures are sized by
KSTACK_PAGES, so the tunable is disabled.

Sponsored by:	The FreeBSD Foundation
MFC after:	2 week
2015-08-10 17:18:21 +00:00
Justin Hibbits
38376983d9 Correct return type of booke_init() prototype. 2015-08-08 23:13:53 +00:00
Jason A. Harmening
713841afb2 Add two new pmap functions:
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)

These will create and destroy a temporary, CPU-local KVA mapping of a specified page.

Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread

Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page

The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.

NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.

Reviewed by:	kib
Approved by:	kib (mentor)
Differential Revision:	http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
Zbigniew Bodek
721555e7ee Fix KSTACK_PAGES issue when the default value was changed in KERNCONF
If KSTACK_PAGES was changed to anything alse than the default,
the value from param.h was taken instead in some places and
the value from KENRCONF in some others. This resulted in
inconsistency which caused corruption in SMP envorinment.

Ensure all places where KSTACK_PAGES are used the opt_kstack_pages.h
is included.

The file opt_kstack_pages.h could not be included in param.h
because was breaking the toolchain compilation.

Reviewed by:   kib
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3094
2015-07-16 10:46:52 +00:00
Justin Hibbits
3f3cffedce Merge booke and aim interrupt.c files.
Summary:
Both booke and AIM interrupt.c files contain nearly identical code.  This merges
the two files, to reduce duplication.

Reviewers: #powerpc, marcel

Reviewed By: marcel

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D2991
2015-07-06 05:08:57 +00:00
Justin Hibbits
0936003e3d Use the correct type for physical addresses.
On Book-E, physical addresses are actually 36-bits, not 32-bits.  This is
currently worked around by ignoring the top bits.  However, in some cases, the
boot loader configures CCSR to something above the 32-bit mark.  This is stage 1
in updating the pmap to handle 36-bit physaddr.
2015-07-04 19:00:38 +00:00
Justin Hibbits
6c8df58287 Unify booke and AIM machdep.
Much of the code was common to begin with.  There is one nit, which is likely
not an issue at all.  With the old code, the AIM machdep would __syncicache()
the entire kernel core at setup.  However, in the unified setup, that seems to
hang on the MPC7455, perhaps because it's running later than before.  Removing
this allows it to boot just fine.  Examining the code, the FreeBSD loader
already does syncicache of the full kernel, and each module loaded, so this
doesn't appear to be an actual problem.

Initial code by Nathan Whitehorn.
2015-04-30 01:24:25 +00:00
Justin Hibbits
a745246822 Implement hwpmc(4) for Freescale e500 core.
This supports e500v1, e500v2, and e500mc. Tested only on e500v2, but the
performance counters are identical across all, with e500mc having some
additional events.

Relnotes:	Yes
2015-04-18 21:39:17 +00:00
Justin Hibbits
d8f2c16bb3 Unbreak book-e, broken by the trap.c merge (missed this file). 2015-04-08 00:31:33 +00:00
Justin Hibbits
28cbb9b173 Unify Book-E and AIM trap.c
Summary:
Book-E and AIM trap.c are almost identical, except for a few bits.  This is step
1 in unifying them.

This also renumbers EXC_DEBUG, to not conflict with AIM vector numbers.  Since
this is the only one thus far that is used in the switch statement in trap(),
it's the only one renumbered.  If others get added to the switch, which conflict
with AIM numbers, they should also be renumbered.

Reviewers: #powerpc, marcel, nwhitehorn

Reviewed By: marcel

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D2215
2015-04-05 02:42:52 +00:00
Justin Hibbits
58cb9ed9e6 CCSRBAR_VA is mpc85xx-specific, so add guards, and include the proper header
file for it.

MFC after:	1 month
2015-03-31 05:29:44 +00:00
Justin Hibbits
cf0b1c3340 Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and
building without SMP causes a build failure.

MFC after:	1 month
2015-03-28 21:39:42 +00:00
Nathan Whitehorn
5c845fde2e Make 32-bit PowerPC kernels, like 64-bit PowerPC kernels, position-independent
executables. The goal here, not yet accomplished, is to let the e500 kernel
run under QEMU by setting KERNBASE to something that fits in low memory and
then having the kernel relocate itself at runtime.
2015-03-07 20:14:46 +00:00
Nathan Whitehorn
6df61aad0a Move IVOR setup from assembler to C, decreasing required assumptions about
address formats for trap handlers.
2015-03-05 05:53:08 +00:00
Nathan Whitehorn
f14cf38dbe The AIM DAR (data access fault address register) and Book-E DEAR registers
have the same meaning and occupy the same memory address in the trapframe
courtesy of union. Avoid some pointless #ifdef by spelling them both 'DAR'
in the trapframe.
2015-03-04 21:06:57 +00:00
Nathan Whitehorn
d1295abdc0 Move Book-E/AIM dependent bits for setting user PMAP during thread switch
out of cpu_switch() and into pmap_activate() where they belong. This also
removes all the #ifdef from cpu_switch().
2015-03-04 16:45:31 +00:00
Nathan Whitehorn
2c8f60acb3 Missed local diff. 2015-03-01 21:47:38 +00:00
Nathan Whitehorn
3846dae834 Initialize NX stack capabilities and direct map status in pmap like on AIM. 2015-03-01 21:23:23 +00:00
Nathan Whitehorn
bb80825435 Refactor PowerPC (especially AIM) init sequence to be less baroque.
MFC after:	2 months
2015-01-18 18:32:43 +00:00