Commit Graph

301 Commits

Author SHA1 Message Date
Justin Hibbits
81ef73fb1b Take into account mas7/8 when reading/writing TLB entries on e6500
Summary: Current booke/pmap code ignores mas7 and mas8 on e6500 CPU.

Submitted by:	Ivan Krivonos <int0dster_gmail.com>
Differential Revision: https://reviews.freebsd.org/D7606
2016-08-23 04:26:30 +00:00
Justin Hibbits
beacb09864 Skip HID1 initialization on e6500 cores, it doesn't exist.
With this, and some drivers removed, a T2080 dev board boots to mountroot.

Submitted by:	Ivan Krivonos <int0dster_AT_gmail.com>
2016-08-20 00:55:58 +00:00
Justin Hibbits
791578a84f Add missing pmap_kenter() method for book-e.
This isn't added to AIM yet, because it's not yet needed.  It's needed for
Book-E ePAPR boot support.

X-MFC With:	r304047
2016-08-13 18:57:14 +00:00
Justin Hibbits
cbc3c68d9a Add a kdb show command to print arbitrary SPRs on PowerPC
Summary:
There is often a need at the debugger to print arbitrary special
purpose registers (SPRs) on PowerPC.  Using a rewritable asm stub, print any SPR
provided on the command line.

Note, as there is no checking in this, attempting to print a nonexistent SPR
may cause a Program exception (illegal instruction, or boundedly undefined).

Note also that this relies on the kernel text pages being writable.  If in the
future this is made not the case, this will need to be reworked.

Test Plan:
Printing the Processor Version Register (PVR, SPR 287):

db> show spr 11f
SPR 287(11f): 80240012

Differential Revision: https://reviews.freebsd.org/D7403
2016-08-13 18:46:49 +00:00
Justin Hibbits
253902b44c Add ePAPR boot support for PowerPC book-E (MPC85xx) hardware
Summary:
u-boot, following the ePAPR specification, puts secondary cores into a
spinloop at boot, rather than leaving them shut off.  It then relies on the host
OS to write the correct values to a special spin table, located in coherent
memory (on newer implementations), or noncoherent memory (older
implementations).

This supports both implementations of ePAPR, as well as continuing to support
non-ePAPR booting, by first attempting to use the spintable, and falling back to
expecting non-started CPUs.

Test Plan:
Booted on a P5020 board.  Tested before and after the changes.
Before the changes, prints the error "SMP: CPU 1 already out of hold-off state!"
and panics shortly thereafter.  After the changes, same boot method lets it
complete boot.

Reviewed by:	nwhitehorn
MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D7494
2016-08-13 16:16:02 +00:00
Justin Hibbits
d071aa6df2 Use label math instead of hard-coding offsets for return addresses.
Though the chances of the code in these sections changing are low, future-proof
the sections and use label math.

Renumber the surrounding areas to avoid duplicate label numbers.
2016-07-23 02:27:42 +00:00
Justin Hibbits
78cc33fd32 Remove booke_enable_l3_cache declaration and remaining definition.
L3 cache is not defined by Book-E, so is platform specific.  Since it was
already moved for e500-based devices into mpc85xx in r292903, just eliminate it
altogether.  Any device that supports L3 cache should have its own platform
means to enable it.
2016-07-17 19:24:28 +00:00
Justin Hibbits
b926a14ca7 No need to include mpc85xx.h anymore, so remove it. 2016-07-17 19:19:50 +00:00
Pedro F. Giffuni
910c079886 sys/powerpc: make use of the howmany() macro when available.
We have a howmany() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
2016-04-26 14:44:49 +00:00
Pedro F. Giffuni
d9c9c81c08 sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
2016-04-21 19:57:40 +00:00
Justin Hibbits
f60708c9f7 Fix SMP booting for PowerPC Book-E
Summary:
PowerPC Book-E SMP is currently broken for unknown reasons.  Pull in
Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP.

This eliminates the shared software TLB1 table, replacing it with
tlb1_read_entry() function.

This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs
already released (ePAPR boot releases APs to a spin loop waiting on a specific
address).  This will be addressed in the near future by using the MPIC to reset
the AP into our own alternate boot address.

This does include a change to the dpaa/dtsec(4) driver, to mark the portals as
CPU-private.

Test Plan:
Tested on Amiga X5000/20 (P5020).  Boots, prints the following
messages:

 Adding CPU 0, pir=0, awake=1
 Waking up CPU 1 (dev=1)
 Adding CPU 1, pir=20, awake=1
 SMP: AP CPU #1 launched

top(1) shows CPU1 active.

Obtained from:	Semihalf
Relnotes:	Yes
Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
Justin Hibbits
f00e990465 VM_MAXUSER_ADDRESS is highest page start, not highest address.
In case a single page mapping is requested first, which might overlap the user
address space, fix the device map block to the next page.
2016-04-10 15:50:45 +00:00
Justin Hibbits
f2c3b7f2ba Restructure device mappings for Book-E.
Summary:
There is currently a 1GB hole between user and kernel address spaces
into which direct (1:1 PA:VA) device mappings go.  This appears to go largely
unused, leaving all devices to contend with the 128MB block at the end of the
32-bit space (0xf8000000-0xffffffff).  This easily fills up, and needs to be
densely packed.  However, dense packing wastes precious TLB1 space, of which
there are only 16 (e500v2) or 64(e5500) entries available.

Change this by using the 1GB space for all device mappings, and allow the kernel
to use the entire upper 1GB for KVA.  This also allows us to use sparse device
mappings, freeing up TLB entries.

Test Plan: Boot tested on p5020.

Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
Justin Hibbits
0aeed3e993 Add support for the Freescale dTSEC DPAA-based ethernet controller.
Freescale's QorIQ line includes a new ethernet controller, based on their
Datapath Acceleration Architecture (DPAA).  This uses a combination of a Frame
manager, Buffer manager, and Queue manager to improve performance across all
interfaces by being able to pass data directly between hardware acceleration
interfaces.

As part of this import, Freescale's Netcomm Software (ncsw) driver is imported.
This was an attempt by Freescale to create an OS-agnostic sub-driver for
managing the hardware, using shims to interface to the OS-specific APIs.  This
work was abandoned, and Freescale's primary work is in the Linux driver (dual
BSD/GPL license).  Hence, this was imported directly to sys/contrib, rather than
going through the vendor area.  Going forward, FreeBSD-specific changes may be
made to the ncsw code, diverging from the upstream in potentially incompatible
ways.  An alternative could be to import the Linux driver itself, using the
linuxKPI layer, as that would maintain parity with the vendor-maintained driver.
However, the Linux driver has not been evaluated for reliability yet, and may
have issues with the import, whereas the ncsw-based driver in this commit was
completed by Semihalf 4 years ago, and is very stable.

Other SoC modules based on DPAA, which could be added in the future:
* Security and Encryption engine (SEC4.x, SEC5.x)
* RAID engine

Additional work to be done:
* Implement polling mode
* Test vlan support
* Add support for the Pattern Matching Engine, which can do regular expression
  matching on packets.

This driver has been tested on the P5020 QorIQ SoC.  Others listed in the
dtsec(4) manual page are expected to work as the same DPAA engine is included in
all.

Obtained from:	Semihalf
Relnotes:	Yes
Sponsored by:	Alex Perez/Inertial Computing
2016-02-29 03:38:00 +00:00
Justin Hibbits
0f7aeab0e7 Implement pmap_change_attr() for PowerPC (Book-E only for now)
Summary:
Some drivers need special memory requirements.  X86 solves this with a
pmap_change_attr() API, which DRM uses for changing the mapping of the GART and
other memory regions.  Implement the same function for PowerPC.  AIM currently
does not need this, but will in the future for DRM, so a default is added for
that, for business as usual.  Book-E has some drivers coming down that do
require non-default memory coherency.  In this case, the Datapath Acceleration
Architecture (DPAA) based ethernet controller has 2 regions for the buffer
portals: cache-inhibited, and cache-enabled.  By default, device memory is
cache-inhibited.  If the cache-enabled memory regions are mapped
cache-inhibited, an alignment exception is thrown on access.

Test Plan:
Tested with a new driver to be added after this (DPAA dTSEC ethernet driver).
No alignment exceptions thrown, driver works as expected with this.

Reviewed By:	nwhitehorn
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
Justin Hibbits
debd17c5e7 Fix a panic bug that cropped up in the PTE rewrite.
PTE was getting overwritten by just the flags.

Pointy-hat to:	jhibbits
2016-02-16 02:13:55 +00:00
Justin Hibbits
64a982ea56 Migrate the PTE format for book-e to standardize on the 'indirect PTE' format
Summary:
The revised Book-E spec, adding the specification for the MMUv2 and e6500,
includes a hardware PTE layout for indirect page tables.  In order to support
this in the future, migrate the PTE format to match the MMUv2 hardware PTE
format.

Test Plan: Boot tested on a P5020 board.  Booted to multiuser mode.

Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
Justin Hibbits
49d36ba409 Older Book-E processors (e500v1/e500v2) don't support dcbzl.
The only difference between dcbzl and dcbz is dcbzl operates on native cache
line lengths regardless of L1CSR0[DCBZ32].  Since we don't change the cache line
size, the cacheline_size variable will reflect the used cache line length, and
dcbz will work as expected.
2016-01-26 04:41:18 +00:00
Justin Hibbits
30857edaea Fix a debug printf().
Somehow this printf() was missed in the conversion of vm_paddr_t to 64-bit, and
made it through until now.

Sponsored by:	Alex Perez/Inertial Computing
2016-01-26 03:52:14 +00:00
Justin Hibbits
e1a51e19e0 Revert a printf change from r294307.
Caused build failures with MPC85XX.

Pointy-hat to:	jhibbits
2016-01-19 23:35:12 +00:00
Justin Hibbits
ca31148c56 Hide most of the PTE initialization and management.
By confining the page table management to a handful of functions it'll be
easier to modify the page table scheme without affecting other functions.
This will be necessary when 64-bit support is added, and page tables become
much larger.
2016-01-19 03:07:25 +00:00
Nathan Whitehorn
ca496abd5a Remove dead code and dead comments, most notably the implemenation of the
now-obsolete setfault(). No NetBSD code exists in the AIM locore files, so
update the copyrights there.
2016-01-10 18:00:01 +00:00
Justin Hibbits
4717ada9cd Make arguments for booke_init() u_long, to match register width.
On powerpc64, pointers are 64 bits, so casting from uint32_t changes the integer
width.

The alternative was to use register_t, but I didn't see register_t used as
argument type for any other functions, though didn't look too closely.  u_long
was an acceptable alternative.  On 64-bit it's 64 bits, on 32-bit it's 32 bits.
2016-01-04 02:20:14 +00:00
Justin Hibbits
a917f636a2 Set the cacheline size before calling powerpc_init()
powerpc_init() initializes the mmu.  Since this may clear pages via
pmap_zero_page(), set the cacheline size before calling into it, so
pmap_zero_page() has the right cacheline size.  This isn't completely
necessary now, but will be when 64-bit book-e is completed.
2016-01-04 01:33:07 +00:00
Nathan Whitehorn
b0bf7fcd29 Bring CPU features list in line with the ABI requirements.
MFC after:	1 week
2016-01-02 18:15:10 +00:00
Justin Hibbits
3f068cbf5c Add platform support for QorIQ SoCs.
This includes the following changes:
* SMP kickoff for QorIQ (tested on P5020)
* Errata fixes for some silicon revisions
* Enables L2 (and L3 if available) caches
Obtained from:	Semihalf
Sponsored by:	Alex Perez/Inertial Computing
2015-12-30 03:43:25 +00:00
Justin Hibbits
eabf894627 Optimize zero_page for book-e mmu.
Instead of indirectly calling bzero() through mmu_booke_zero_page_area, zero the
full page the same way as the AIM pmap logic does: using dcbz.
2015-12-30 02:26:04 +00:00
Justin Hibbits
459021cc7d Rewrite tid_flush() in C.
There's no need for it to be in asm.  Also, by writing in C, and marking it
static in pmap.c, it saves a branch to the function itself, as it's only used in
one location.  The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
Justin Hibbits
b3936ebe07 Extend Book-E to support >4GB RAM
Summary:
With some additional changes for AIM, that could also support much
larger physmem sizes.  Given that 32-bit AIM is more or less obsolete, though,
it's not worth it at this time.

Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
Justin Hibbits
777d81af62 No need to reset tlb1 here, it gets reset again after BSS is cleared in
powerpc_init().

Also fix a comment typo (0x45 == E, not e)
2015-12-11 01:34:13 +00:00
Justin Hibbits
a39f10532a Add more interrupts handled for booke.
e500mc, e5500, and e6500 all use the normal FPU, with the same behavior as AIM
hardware.  e6500 also supports Altivec, so, although we don't yet have e6500
hardware to test on, add these IVORs as well.  Theoretically, since it boots the
same as a e5500, it should work, single-threaded, single-core, with full altivec
support as of this commit.

With this commit, and some other patches to be committed shortly FreeBSD now
boots on the P5020, single-core, all the way to user space, and should boot just
fine on e500mc.

Relnotes:	Yes (e500mc, e5500 support)
Sponsored by:	Alex Perez/Inertial Computing
2015-12-11 01:23:18 +00:00
Justin Hibbits
ef596c4021 trunc_page() goes through unsigned long, which is too short.
sizeof(unsigned long) < sizeof(vm_paddr_t) on Book-E, which uses 36-bit
addressing.  With this, a CCSR with a physical address above 4GB successfully
maps.

Sponsored by:	Alex Perez/Inertial Computing
2015-11-21 06:03:46 +00:00
Justin Hibbits
12d44566ae The TLB1 TSIZE is a multiple of 4, not 2, so shift 2 bits, not 1. 2015-08-29 06:52:14 +00:00
Justin Hibbits
afefc223b1 Extend pmap to support e500mc and e5500.
As part of this, clean up tlb1_init(), since bootinfo is always NULL here just
eliminate the loop altogether.

Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger
immediately following a smaller page, causing the mappings to overlap.  Instead,
break up the new mapping into smaller chunks.  The downside to this is that it
uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause
problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries).

Obtained from:	Semihalf (partial)
Sponsored by:	Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
Justin Hibbits
92f6e93414 Follow up to r287014
Missed these files, from the original diff.
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision:	https://reviews.freebsd.org/D3027
2015-08-22 07:27:06 +00:00
Justin Hibbits
b239c24baa Enhance book-e pmap for 36-bit physaddr
Summary:
This is (probably step 1) of enhancing the book-e pmap to support the full
36-bit physical address space on Freescale e500 and e5500 cores.

Thus far it has only been regression tested on one platform.  Since I only have
one other Book-E platform (e5500), that needs work beyond this, I haven't yet
tested it on this.

Test Plan: Regression tested on my RouterBoard RB800.

Reviewed By: marcel
Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
Justin Hibbits
7f7fcf5596 Add initial boot support for e500mc and e5500.
* Since r257190 the kernel must actually be loaded at a 64MB boundary, not 16MB.
* Don't program HID1 register on e500mc or e5500, they don't have this SPR.
* Set proper HID0 defaults for these new architectures.

There is still more work to be done for the various SoCs, and the PMAP code
still needs to be extended to 36-bit paddr, coming soon.

Obtained from:	Semihalf
Sponsored by:	Alex Perez/Inertial Computing
2015-08-21 02:41:35 +00:00
Justin Hibbits
13adf27426 Fix copy&paste. 2015-08-19 06:08:11 +00:00
Justin Hibbits
947c974b39 Save the registers at the correct offsets.
When merging the AIM and BookE trap.c files, the offsets for BookE's setfault
inadvertantly got munged.
2015-08-19 06:07:32 +00:00
Konstantin Belousov
edc8222303 Make kstack_pages a tunable on arm, x86, and powepc. On i386, the
initial thread stack is not adjusted by the tunable, the stack is
allocated too early to get access to the kernel environment. See
TD0_KSTACK_PAGES for the thread0 stack sizing on i386.

The tunable was tested on x86 only.  From the visual inspection, it
seems that it might work on arm and powerpc.  The arm
USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already
incorrect for the threads with non-default kstack size.  I only
changed the macros to use variable instead of constant, since I cannot
test.

On arm64, mips and sparc64, some static data structures are sized by
KSTACK_PAGES, so the tunable is disabled.

Sponsored by:	The FreeBSD Foundation
MFC after:	2 week
2015-08-10 17:18:21 +00:00
Justin Hibbits
38376983d9 Correct return type of booke_init() prototype. 2015-08-08 23:13:53 +00:00
Jason A. Harmening
713841afb2 Add two new pmap functions:
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)

These will create and destroy a temporary, CPU-local KVA mapping of a specified page.

Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread

Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page

The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.

NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.

Reviewed by:	kib
Approved by:	kib (mentor)
Differential Revision:	http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
Zbigniew Bodek
721555e7ee Fix KSTACK_PAGES issue when the default value was changed in KERNCONF
If KSTACK_PAGES was changed to anything alse than the default,
the value from param.h was taken instead in some places and
the value from KENRCONF in some others. This resulted in
inconsistency which caused corruption in SMP envorinment.

Ensure all places where KSTACK_PAGES are used the opt_kstack_pages.h
is included.

The file opt_kstack_pages.h could not be included in param.h
because was breaking the toolchain compilation.

Reviewed by:   kib
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3094
2015-07-16 10:46:52 +00:00
Justin Hibbits
3f3cffedce Merge booke and aim interrupt.c files.
Summary:
Both booke and AIM interrupt.c files contain nearly identical code.  This merges
the two files, to reduce duplication.

Reviewers: #powerpc, marcel

Reviewed By: marcel

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D2991
2015-07-06 05:08:57 +00:00
Justin Hibbits
0936003e3d Use the correct type for physical addresses.
On Book-E, physical addresses are actually 36-bits, not 32-bits.  This is
currently worked around by ignoring the top bits.  However, in some cases, the
boot loader configures CCSR to something above the 32-bit mark.  This is stage 1
in updating the pmap to handle 36-bit physaddr.
2015-07-04 19:00:38 +00:00
Justin Hibbits
6c8df58287 Unify booke and AIM machdep.
Much of the code was common to begin with.  There is one nit, which is likely
not an issue at all.  With the old code, the AIM machdep would __syncicache()
the entire kernel core at setup.  However, in the unified setup, that seems to
hang on the MPC7455, perhaps because it's running later than before.  Removing
this allows it to boot just fine.  Examining the code, the FreeBSD loader
already does syncicache of the full kernel, and each module loaded, so this
doesn't appear to be an actual problem.

Initial code by Nathan Whitehorn.
2015-04-30 01:24:25 +00:00
Justin Hibbits
a745246822 Implement hwpmc(4) for Freescale e500 core.
This supports e500v1, e500v2, and e500mc. Tested only on e500v2, but the
performance counters are identical across all, with e500mc having some
additional events.

Relnotes:	Yes
2015-04-18 21:39:17 +00:00
Justin Hibbits
d8f2c16bb3 Unbreak book-e, broken by the trap.c merge (missed this file). 2015-04-08 00:31:33 +00:00
Justin Hibbits
28cbb9b173 Unify Book-E and AIM trap.c
Summary:
Book-E and AIM trap.c are almost identical, except for a few bits.  This is step
1 in unifying them.

This also renumbers EXC_DEBUG, to not conflict with AIM vector numbers.  Since
this is the only one thus far that is used in the switch statement in trap(),
it's the only one renumbered.  If others get added to the switch, which conflict
with AIM numbers, they should also be renumbered.

Reviewers: #powerpc, marcel, nwhitehorn

Reviewed By: marcel

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D2215
2015-04-05 02:42:52 +00:00
Justin Hibbits
58cb9ed9e6 CCSRBAR_VA is mpc85xx-specific, so add guards, and include the proper header
file for it.

MFC after:	1 month
2015-03-31 05:29:44 +00:00
Justin Hibbits
cf0b1c3340 Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and
building without SMP causes a build failure.

MFC after:	1 month
2015-03-28 21:39:42 +00:00
Nathan Whitehorn
5c845fde2e Make 32-bit PowerPC kernels, like 64-bit PowerPC kernels, position-independent
executables. The goal here, not yet accomplished, is to let the e500 kernel
run under QEMU by setting KERNBASE to something that fits in low memory and
then having the kernel relocate itself at runtime.
2015-03-07 20:14:46 +00:00
Nathan Whitehorn
6df61aad0a Move IVOR setup from assembler to C, decreasing required assumptions about
address formats for trap handlers.
2015-03-05 05:53:08 +00:00
Nathan Whitehorn
f14cf38dbe The AIM DAR (data access fault address register) and Book-E DEAR registers
have the same meaning and occupy the same memory address in the trapframe
courtesy of union. Avoid some pointless #ifdef by spelling them both 'DAR'
in the trapframe.
2015-03-04 21:06:57 +00:00
Nathan Whitehorn
d1295abdc0 Move Book-E/AIM dependent bits for setting user PMAP during thread switch
out of cpu_switch() and into pmap_activate() where they belong. This also
removes all the #ifdef from cpu_switch().
2015-03-04 16:45:31 +00:00
Nathan Whitehorn
2c8f60acb3 Missed local diff. 2015-03-01 21:47:38 +00:00
Nathan Whitehorn
3846dae834 Initialize NX stack capabilities and direct map status in pmap like on AIM. 2015-03-01 21:23:23 +00:00
Nathan Whitehorn
bb80825435 Refactor PowerPC (especially AIM) init sequence to be less baroque.
MFC after:	2 months
2015-01-18 18:32:43 +00:00
Mark Johnston
bdb9ab0dd9 Factor out duplicated code from dumpsys() on each architecture into generic
code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly
identical and simply redefine a number of constants and helper subroutines;
a generic implementation will make it easier to implement features around
kernel core dumps. This change does not alter any minidump code and should
have no functional impact.

PR:		193873
Differential Revision:	https://reviews.freebsd.org/D904
Submitted by:	Conrad Meyer <conrad.meyer@isilon.com>
Reviewed by:	jhibbits (earlier version)
Sponsored by:	EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
Andreas Tobler
a00b2ba8fe Fix kernel build for booke. 2014-12-10 20:23:19 +00:00
Roger Pau Monné
c98a2727cc ddb: allow specifying the exact address of the symtab and strtab
When the FreeBSD kernel is loaded from Xen the symtab and strtab are
not loaded the same way as the native boot loader. This patch adds
three new global variables to ddb that can be used to specify the
exact position and size of those tables, so they can be directly used
as parameters to db_add_symbol_table. A new helper is introduced, so callers
that used to set ksym_start and ksym_end can use this helper to set the new
variables.

It also adds support for loading them from the Xen PVH port, that was
previously missing those tables.

Sponsored by: Citrix Systems R&D
Reviewed by:	kib

ddb/db_main.c:
 - Add three new global variables: ksymtab, kstrtab, ksymtab_size that
   can be used to specify the position and size of the symtab and
   strtab.
 - Use those new variables in db_init in order to call db_add_symbol_table.
 - Move the logic in db_init to db_fetch_symtab in order to set ksymtab,
   kstrtab, ksymtab_size from ksym_start and ksym_end.

ddb/ddb.h:
 - Add prototype for db_fetch_ksymtab.
 - Declate the extern variables ksymtab, kstrtab and ksymtab_size.

x86/xen/pv.c:
 - Add support for finding the symtab and strtab when booted as a Xen
   PVH guest. Since Xen loads the symtab and strtab as NetBSD expects
   to find them we have to adapt and use the same method.

amd64/amd64/machdep.c:
arm/arm/machdep.c:
i386/i386/machdep.c:
mips/mips/machdep.c:
pc98/pc98/machdep.c:
powerpc/aim/machdep.c:
powerpc/booke/machdep.c:
sparc64/sparc64/machdep.c:
 - Use the newly introduced db_fetch_ksymtab in order to set ksymtab,
   kstrtab and ksymtab_size.
2014-09-25 08:28:10 +00:00
Konstantin Belousov
39ffa8c138 Change pmap_enter(9) interface to take flags parameter and superpage
mapping size (currently unused).  The flags includes the fault access
bits, wired flag as PMAP_ENTER_WIRED, and a new flag
PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep.

For powerpc aim both 32 and 64 bit, fix implementation to ensure that
the requested mapping is created when PMAP_ENTER_NOSLEEP is not
specified, in particular, wait for the available memory required to
proceed.

In collaboration with:	alc
Tested by:	nwhitehorn (ppc aim32 and booke)
Sponsored by:	The FreeBSD Foundation and EMC / Isilon Storage Division
MFC after:	2 weeks
2014-08-08 17:12:03 +00:00
Alan Cox
a695d9b25b Retire pmap_change_wiring(). We have never used it to wire virtual pages.
We continue to use pmap_enter() for that.  For unwiring virtual pages, we
now use pmap_unwire(), which unwires a range of virtual addresses instead
of a single virtual page.

Sponsored by:	EMC / Isilon Storage Division
2014-08-03 20:40:51 +00:00
Alan Cox
a844c68fc2 Implement pmap_unwire(). See r268327 for the motivation behind this change. 2014-07-13 16:27:57 +00:00
Bryan Drewery
44f1c91610 Rename global cnt to vm_cnt to avoid shadowing.
To reduce the diff struct pcu.cnt field was not renamed, so
PCPU_OP(cnt.field) is still used. pc_cnt and pcpu are also used in
kvm(3) and vmstat(8). The goal was to not affect externally used KPI.

Bump __FreeBSD_version_ in case some out-of-tree module/code relies on the
the global cnt variable.

Exp-run revealed no ports using it directly.

No objection from:	arch@
Sponsored by:	EMC / Isilon Storage Division
2014-03-22 10:26:09 +00:00
Nathan Whitehorn
a3845e4242 Avoid spurious compiler warning about an uninitialized variable. 2014-02-01 20:06:52 +00:00
Nathan Whitehorn
c1cb22d755 Rearchitect platform memory map parsing to make it less
Open Firmware-centric:
- Keep the static list of regions in platform.c instead of ofw_machdep.c
- Move various merging and sorting operations to platform.c as well
- Move apple_hacks code out of ofw_machdep.c and into platform_powermac.c,
  where it belongs
- Move CHRP-specific dynamic-reconfiguration memory parsing into
  platform_chrp.c instead of pretending it is shared code
2013-12-01 19:43:15 +00:00
Nathan Whitehorn
a0d0d6d88b badaddr() is used only in the grackle PCI driver, so move its definition
there. Clean up a spurious setfault() declaration as well.
2013-11-27 22:01:09 +00:00
Nathan Whitehorn
e537388b84 Unify handling of illegal instruction faults between AIM and Book-E. This
allows FPU emulation on AIM as well as providing support for the mfpvr
and lwsync instructions from userland on e500 cores. lwsync, in particular,
is required for many C++ programs to work correctly.

MFC after:	1 week
2013-11-17 15:12:03 +00:00
Nathan Whitehorn
debe445512 Split the function of the PCB_FPU flags into two: PCB_FPU now indicates that
the actual FPU is enabled, while PCB_FPREGS indicates that the FPU state
structure in the PCB is valid. This separation reflects the situation on
FPU-less systems in which the FP state is used by the emulator but we don't
actually want to try to turn on the non-existant FPU.

Use this flag to save and restore FP regs properly on both AIM and Book-E.
As a side effect, this sets up hard-FP and Altivec on Book-E CPUs with such
abilities except for a trap handler to call enable_fpu()/enable_altivec().
2013-11-17 14:44:22 +00:00
Nathan Whitehorn
52cfe485fb Move CCSR discovery into the platform module, while simultaneously making
it more flexible about how the CCSR range is found. With this change, the
stock MPC85XX will boot on a Routerboard 800.

Hardware donated by:	Benjamin Perrault
2013-11-17 02:03:36 +00:00
Nathan Whitehorn
b899f5d5c9 Make sure that TLB1 mappings are aligned correctly. 2013-11-17 01:59:42 +00:00
Nathan Whitehorn
e39c26a950 Use the same implementation of copyinout.c for both AIM and Book-E. This
fixes some bugs in both implementations related to validity checks on
mapping bounds.
2013-11-11 23:37:16 +00:00
Nathan Whitehorn
bdac436008 Follow up r223485, which made AIM use the ABI thread pointer instead of
PCPU fields for curthread, by doing the same to Book-E. This closes
some potential races switching between CPUs. As a side effect, it turns out
the AIM and Book-E swtch.S implementations were the same to within a few
registers, so move that to powerpc/powerpc.

MFC after: 3 months
2013-11-11 17:37:50 +00:00
Nathan Whitehorn
302acc2e5f Rename the "bare" platform "mpc85xx", which is what it actually is, and
add actual platform probing based on PVR. Still needs a little more work:
in particular, the CCRS setup should move here.

Also turn "bare" into a truly bare platform that doesn't pretend to know how
to do anything except get the memory map. This should also be enhanced to
process the FDT reserved memory list, but that is for another day.
2013-11-11 16:14:25 +00:00
Nathan Whitehorn
0f3406e079 Do not panic if pmap_mincore() is called. This prevents crashing userland
binaries from bringing down the kernel.

MFC after:	3 days
2013-11-06 14:36:38 +00:00
Nathan Whitehorn
0b8a792e0b Make devices with registers into the KVA region work reliably. Without this,
previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could
shadow device maps in TLB1. Add a big block comment about some of the
caveats with this approach.
2013-10-26 20:57:26 +00:00
Nathan Whitehorn
b5c192462c Handle (in a slightly ugly way) ePAPR-type loaders that just place a
device tree into r3. Rather than worrying about mapping that tree, reserving
its space in the global physical memory space, etc., just copy it to some
memory after the kernel.
2013-10-26 19:50:40 +00:00
Nathan Whitehorn
d2a406dd46 Bump initial TLB size. The kernel is not necessarily less than 16 MB any
more.
2013-10-26 19:49:09 +00:00
Nathan Whitehorn
33724f17d2 Interrelated improvements to early boot mappings:
- Remove explicit requirement that the SOC registers be found except as an
  optimization (although the MPC85XX LAW drivers still require they be found
  externally, which should change).
- Remove magic CCSRBAR_VA value.
- Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and
  systems not in real-mode or global 1:1 maps in early boot.
- Allow pmap_mapdev() on Book-E to reissue previous addresses if the
  area is already mapped. Additionally have it check all mappings, not
  just the CCSR area.

This allows the console on e500 systems to actually work on systems where
the boot loader was not kind enough to set up a 1:1 mapping before starting
the kernel.
2013-10-26 18:18:14 +00:00
Nathan Whitehorn
928554a24f Fix concurrency issues with TLB1 updates and make pmap_kextract() search
TLB1 mappings as well, which is required for the console to work after
r257111.
2013-10-26 16:49:41 +00:00
Nathan Whitehorn
8bcd59f2c0 Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr()
is slightly more complicated and is left unimplemented for now. Also
prevent pmap_mapdev() from mapping over the kernel and KVA regions if
devices happen to have high physical addresses.

MFC after:	2 weeks
2013-10-26 14:52:55 +00:00
Nathan Whitehorn
1c02b4c9fb A quick addendum: the standard says that timebase-frequency can be either
32 or 64 bits, so allow either.
2013-10-23 14:34:04 +00:00
Nathan Whitehorn
d26eb2c194 If the device tree directly contains the timebase frequency, use it. This
property is required by ePAPR, but maintain the fallback to bus-frequency
for compatibility.

MFC after:	2 weeks
2013-10-23 14:28:59 +00:00
Nathan Whitehorn
c8d23639da Make hard-wired TLB allocations be at minimum one page. This is required by
some implementations, most notably (in my case) QEMU's e500 emulation.
2013-10-21 22:25:54 +00:00
Nathan Whitehorn
58ac6f25fa Avoid sign overflow if there are more than 2 GB of RAM. 2013-10-20 23:02:16 +00:00
Nathan Whitehorn
228f09b3ef Replace the two almost-exactly-identical AIM and Book-E clock.c
implementations with a single one after the application of a very small
amount of #ifdef.
2013-10-20 16:37:03 +00:00
Nathan Whitehorn
1cfdc97153 Unify the AIM and Book-E vm_machdep.c implementations, which previously
differed only with respect to the AIM version not following style(9) and
some additional features for 64-bit systems and machines with direct maps
in the AIM implementation that are no-ops on Book-E (at least for now).
2013-10-20 16:14:03 +00:00
Gleb Smirnoff
255c1caae3 - Create kern.ipc.sendfile namespace, and put the new "readhead" OID
there as "kern.ipc.sendfile.readahead".
- Push all nsfbuf related tunables into MD code. Don't move them
  to new namespace in favor of POLA.

Reviewed by:	scottl
Approved by:	re (gjb)
2013-09-22 13:36:52 +00:00
Alan Cox
deb179bb4c The pmap function pmap_clear_reference() is no longer used. Remove it.
pmap_clear_reference() has had exactly one caller in the kernel for
several years, more precisely, since FreeBSD 8.  Now, that call no
longer exists.

Approved by:	re (kib)
Sponsored by:	EMC / Isilon Storage Division
2013-09-20 04:30:18 +00:00
Konstantin Belousov
e68c64f0ba Revert r254501. Instead, reuse the type stability of the struct pmap
which is the part of struct vmspace, allocated from UMA_ZONE_NOFREE
zone.  Initialize the pmap lock in the vmspace zone init function, and
remove pmap lock initialization and destruction from pmap_pinit() and
pmap_release().

Suggested and reviewed by:	alc (previous version)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2013-08-22 18:12:24 +00:00
Attilio Rao
c7aebda8a1 The soft and hard busy mechanism rely on the vm object lock to work.
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.

Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
  and vm_page_grab are being executed.  This will be very helpful
  once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag

The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.

Sponsored by:	EMC / Isilon storage division
Discussed with:	alc
Reviewed by:	jeff, kib
Tested by:	gavin, bapt (older version)
Tested by:	pho, scottl
2013-08-09 11:11:11 +00:00
Jeff Roberson
5df87b21d3 Replace kernel virtual address space allocation with vmem. This provides
transparent layering and better fragmentation.

 - Normalize functions that allocate memory to use kmem_*
 - Those that allocate address space are named kva_*
 - Those that operate on maps are named kmap_*
 - Implement recursive allocation handling for kmem_arena in vmem.

Reviewed by:	alc
Tested by:	pho
Sponsored by:	EMC / Isilon Storage Division
2013-08-07 06:21:20 +00:00
Andrey V. Elsukov
05d1f5bce0 Introduce new structure sfstat for collecting sendfile's statistics
and remove corresponding fields from struct mbstat. Use PCPU counters
and SFSTAT_INC() macro for update these statistics.

Discussed with:	glebius
2013-07-15 06:16:57 +00:00
Attilio Rao
9af6d512f5 o Relax locking assertions for vm_page_find_least()
o Relax locking assertions for pmap_enter_object() and add them also
  to architectures that currently don't have any
o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade
  operation on the per-object rwlock
o Use all the mechanisms above to make vm_map_pmap_enter() to work
  mostl of the times only with readlocks.

Sponsored by:	EMC / Isilon storage division
Reviewed by:	alc
2013-05-21 20:38:19 +00:00
Alan Cox
658f180b3b Relax the object locking assertion in pmap_enter_locked().
Reviewed by:	attilio
Sponsored by:	EMC / Isilon Storage Division
2013-05-17 18:59:00 +00:00
Konstantin Belousov
e8a4a618cf Add pmap function pmap_copy_pages(), which copies the content of the
pages around, taking array of vm_page_t both for source and
destination.  Starting offsets and total transfer size are specified.

The function implements optimal algorithm for copying using the
platform-specific optimizations.  For instance, on the architectures
were the direct map is available, no transient mappings are created,
for i386 the per-cpu ephemeral page frame is used.  The code was
typically borrowed from the pmap_copy_page() for the same
architecture.

Only i386/amd64, powerpc aim and arm/arm-v6 implementations were
tested at the time of commit. High-level code, not committed yet to
the tree, ensures that the use of the function is only allowed after
explicit enablement.

For sparc64, the existing code has known issues and a stab is added
instead, to allow the kernel linking.

Sponsored by:	The FreeBSD Foundation
Tested by:	pho (i386, amd64), scottl (amd64), ian (arm and arm-v6)
MFC after:	2 weeks
2013-03-14 20:18:12 +00:00
Attilio Rao
89f6b8632c Switch the vm_object mutex to be a rwlock. This will enable in the
future further optimizations where the vm_object lock will be held
in read mode most of the time the page cache resident pool of pages
are accessed for reading purposes.

The change is mostly mechanical but few notes are reported:
* The KPI changes as follow:
  - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
  - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
  - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
  - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
    (in order to avoid visibility of implementation details)
  - The read-mode operations are added:
    VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
    VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
* The vm/vm_pager.h namespace pollution avoidance (forcing requiring
  sys/mutex.h in consumers directly to cater its inlining functions
  using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
  consumers now must include also sys/rwlock.h.
* zfs requires a quite convoluted fix to include FreeBSD rwlocks into
  the compat layer because the name clash between FreeBSD and solaris
  versions must be avoided.
  At this purpose zfs redefines the vm_object locking functions
  directly, isolating the FreeBSD components in specific compat stubs.

The KPI results heavilly broken by this commit.  Thirdy part ports must
be updated accordingly (I can think off-hand of VirtualBox, for example).

Sponsored by:	EMC / Isilon storage division
Reviewed by:	jeff
Reviewed by:	pjd (ZFS specific review)
Discussed with:	alc
Tested by:	pho
2013-03-09 02:32:23 +00:00
Alexander Motin
fdc5dd2d2f MFcalloutng:
Switch eventtimers(9) from using struct bintime to sbintime_t.
Even before this not a single driver really supported full dynamic range of
struct bintime even in theory, not speaking about practical inexpediency.
This change legitimates the status quo and cleans up the code.
2013-02-28 13:46:03 +00:00
Attilio Rao
dc1558d1cd Merge from vmobj-rwlock:
VM_OBJECT_LOCKED() macro is only used to implement a custom version
of lock assertions right now (which likely spread out thanks to
copy and paste).
Remove it and implement actual assertions.

Sponsored by:	EMC / Isilon storage division
Reviewed by:	alc
Tested by:	pho
2013-02-27 18:12:13 +00:00