536 Commits

Author SHA1 Message Date
nwhitehorn
f632a769ed The first argument of dcbz interprets r0 as a literal zero, not the second.
This worked before by accident.

MFC after:	1 week
2009-12-03 20:55:09 +00:00
nwhitehorn
451e17ae41 Add a CPU features framework on PowerPC and simplify CPU setup a little
more. This provides three new sysctls to user space:
hw.cpu_features - A bitmask of available CPU features
hw.floatingpoint - Whether or not there is hardware FP support
hw.altivec - Whether or not Altivec is available

PR:		powerpc/139154
MFC after:	10 days
2009-11-28 17:33:19 +00:00
alc
dcb93e6c95 Simplify the invocation of vm_fault(). Specifically, eliminate the flag
VM_FAULT_DIRTY.  The information provided by this flag can be trivially
inferred by vm_fault().

Discussed with:	kib
2009-11-27 20:24:11 +00:00
nwhitehorn
cb5daaa321 Garbage collect some code that was never compiled in to handle Altivec
during traps. It predates actual Altivec support and was never used.
2009-11-22 20:45:15 +00:00
nwhitehorn
9cdb628414 Provide a real fix to the too-many-translations problem when booting
from CD on 64-bit hardware to replace existing band-aids. This occurred
when the preloaded mdroot required too many mappings for the static
buffer.

Since we only use the translations buffer once, allocate a dynamic
buffer on the stack. This early in the boot process, the call chain
is quite short and we can be assured of having sufficient stack space.

Reviewed by:	grehan
2009-11-12 15:19:09 +00:00
kib
3cf53f181e Extract the code that records syscall results in the frame into MD
function cpu_set_syscall_retval().

Suggested by:	marcel
Reviewed by:	marcel, davidxu
PowerPC, ARM, ia64 changes:	marcel
Sparc64 tested and reviewed by:	marius, also sunv reviewed
MIPS tested by:	gonzo
MFC after:	1 month
2009-11-10 11:43:07 +00:00
nwhitehorn
f44a14ea63 Spell sz correctly.
Pointed out by:	jmallett
2009-11-09 21:12:28 +00:00
nwhitehorn
95e8f111b4 Increase the size of the OFW translations buffer to handle G5 systems
that use many translation regions in firmware, and add bounds checking
to prevent buffer overflows in case even the new value is exceeded.

Reported by:	Jacob Lambert
MFC after:	3 days
2009-11-09 14:26:23 +00:00
nwhitehorn
113d2ed8a6 Unbreak cpu_switch(). The register allocator in my brain is clearly
broken. Also, Altivec context switching worked before only by accident,
but should work now by design.
2009-10-31 20:59:13 +00:00
nwhitehorn
2612f797ab Remove an unnecessary sync that crept in the last commit. 2009-10-31 18:04:34 +00:00
nwhitehorn
2eaf4c92f4 Fix a race in casuword() exposed by csup. casuword() non-atomically read
the current value of its argument before atomically replacing it, which
could occasionally return the wrong value on an SMP system. This resulted
in user mutex operations hanging when using threaded applications.
2009-10-31 17:59:24 +00:00
nwhitehorn
b6af1984be Loop on blocked threads when using ULE scheduler, removing an
XXX MP comment.
2009-10-31 17:55:48 +00:00
nwhitehorn
56fee338a9 Garbage collect set_user_sr(), which is declared static inline and
never called.
2009-10-31 17:46:50 +00:00
nwhitehorn
68bae1c5ee Turn off Altivec data-stream prefetching before going into power-save
mode on those CPUs that need it.
2009-10-29 14:22:09 +00:00
kib
ce081b037e In r197963, a race with thread being selected for signal delivery
while in kernel mode, and later changing signal mask to block the
signal, was fixed for sigprocmask(2) and ptread_exit(3). The same race
exists for sigreturn(2), setcontext(2) and swapcontext(2) syscalls.

Use kern_sigprocmask() instead of direct manipulation of td_sigmask to
reschedule newly blocked signals, closing the race.

Reviewed by:	davidxu
Tested by:	pho
MFC after:	1 month
2009-10-27 10:47:58 +00:00
nwhitehorn
a6de972d9b Remove debugging printf that snuck in here.
Pointy hat to:	me
2009-10-23 21:44:46 +00:00
nwhitehorn
1ba71fb08a Add some more paranoia to setting HID registers, and update the AIM
clock routines to work better with SMP. This makes SMP work fully and
stably on an Xserve G5.

Obtained from:	Book-E (clock bits)
2009-10-23 21:36:33 +00:00
nwhitehorn
8f8a90522c Do not map the trap vectors into the kernel's address space. They are
only used in real mode and keeping them mapped only serves to make NULL
a valid address, which results in silent NULL pointer deferences.

Suggested by:   Patrick Kerharo
Obtained from:	projects/ppc64
2009-10-23 14:27:40 +00:00
nwhitehorn
377d7d4bff Add SMP support on U3-based G5 systems. This does not yet work perfectly:
at least on my Xserve, getting the decrementer and timebase on APs to tick
requires setting up a clock chip over I2C, which is not yet done.

While here, correct the 64-bit tlbie function to set the CPU to 64-bit
mode correctly.

Hardware donated by:	grehan
2009-10-23 03:17:02 +00:00
marcel
51bb720939 o Introduce vm_sync_icache() for making the I-cache coherent with
the memory or D-cache, depending on the semantics of the platform.
    vm_sync_icache() is basically a wrapper around pmap_sync_icache(),
    that translates the vm_map_t argumument to pmap_t.
o   Introduce pmap_sync_icache() to all PMAP implementation. For powerpc
    it replaces the pmap_page_executable() function, added to solve
    the I-cache problem in uiomove_fromphys().
o   In proc_rwmem() call vm_sync_icache() when writing to a page that
    has execute permissions. This assures that when breakpoints are
    written, the I-cache will be coherent and the process will actually
    hit the breakpoint.
o   This also fixes the Book-E PMAP implementation that was missing
    necessary locking while trying to deal with the I-cache coherency
    in pmap_enter() (read: mmu_booke_enter_locked).

The key property of this change is that the I-cache is made coherent
*after* writes have been done. Doing it in the PMAP layer when adding
or changing a mapping means that the I-cache is made coherent *before*
any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
nwhitehorn
0297b355f6 Don't assume that physical addresses are identity mapped. This allows
the second processor on G5 systems to start. Note that SMP is still
non-functional on these systems because of IPI delivery problems.
2009-10-18 17:22:08 +00:00
nwhitehorn
68c3c08fe8 Correct another typo. Actually save the condition register instead
of overwriting r12 by mistake.
2009-10-11 16:44:58 +00:00
nwhitehorn
c4b6b5951f Correct a typo here and actually save DSISR instead of overwriting it. 2009-10-11 16:41:39 +00:00
nwhitehorn
04a1b965bd Increase the size of the page table on 64-bit PowerPC machines as a
bandaid to prevent exhaustion of the primary and secondary hash groups
in the event of extreme stress on the PMAP layer (e.g. a forkbomb). This
wastes memory, and should be revised to properly handle PTEG spills instead.

Suggested by:	grehan
Approved by:	re (kensmith)
2009-07-12 04:07:52 +00:00
jeff
5bc3a65e40 Implement a facility for dynamic per-cpu variables.
- Modules and kernel code alike may use DPCPU_DEFINE(),
   DPCPU_GET(), DPCPU_SET(), etc. akin to the statically defined
   PCPU_*.  Requires only one extra instruction more than PCPU_* and is
   virtually the same as __thread for builtin and much faster for shared
   objects.  DPCPU variables can be initialized when defined.
 - Modules are supported by relocating the module's per-cpu linker set
   over space reserved in the kernel.  Modules may fail to load if there
   is insufficient space available.
 - Track space available for modules with a one-off extent allocator.
   Free may block for memory to allocate space for an extent.

Reviewed by:    jhb, rwatson, kan, sam, grehan, marius, marcel, stas
2009-06-23 22:42:39 +00:00
grehan
695df43388 Get the gdb/psim emulator functioning again.
aim/machdep.c:
  - the	RI status register bit needs to be set when	doing the mtmsrd 64-bit
    instruction	test
  - psim doesn't implement the dcbz instruction	so the run-time	cacheline
    test fails.	Set the	cachline size to 32 to avoid infinite loops in
    future calls to __syncicache()

aim/platform_chrp.c:
  - if after iterating through / and a name property of "cpus" still isn't
    found, just	search directly	for '/cpus'.
  - psim doesn't put a "reg" property on it's cpu nodes, so assume 0
    since it is	uniprocessor-only at this point

powerpc/openpic.c
  - the	number of CPUs reported	is 1 too many on psim's	openpic

Reviewed by:	nwhitehorn
MFC after:	1 week (openpic part)
2009-06-10 12:47:54 +00:00
nwhitehorn
26eb077d03 Introduce support for cpufreq on PowerPC with the dynamic frequency
switching capabilities of the MPC7447A and MPC7448.
2009-05-31 09:01:23 +00:00
marcel
8b09116a5a Add cpu_flush_dcache() for use after non-DMA based I/O so that a
possible future I-cache coherency operation can succeed. On ARM
for example the L1 cache can be (is) virtually mapped, which
means that any I/O that uses temporary mappings will not see the
I-cache made coherent. On ia64 a similar behaviour has been
observed. By flushing the D-cache, execution of binaries backed
by md(4) and/or NFS work reliably.
For Book-E (powerpc), execution over NFS exhibits SIGILL once in
a while as well, though cpu_flush_dcache() hasn't been implemented
yet.

Doing an explicit D-cache flush as part of the non-DMA based I/O
read operation eliminates the need to do it as part of the
I-cache coherency operation itself and as such avoids pessimizing
the DMA-based I/O read operations for which D-cache are already
flushed/invalidated. It also allows future optimizations whereby
the bcopy() followed by the D-cache flush can be integrated in a
single operation, which could be implemented using on-chips DMA
engines, by-passing the D-cache altogether.
2009-05-18 18:37:18 +00:00
raj
e648274e7c PowerPC common SMP startup and time base rework.
- make mftb() shared, rewrite in C, provide complementary mttb()
- adjust SMP startup per the above, additional comments, minor naming
  changes
- eliminate redundant TB defines, other minor cosmetics

Reviewed by:	marcel, nwhitehorn
Obtained from:	Freescale, Semihalf
2009-05-14 16:48:25 +00:00
nwhitehorn
dc1ac440de Factor out platform dependent things unrelated to device drivers into a
new platform module. These are probed in early boot, and have the
responsibility of determining the layout of physical memory, determining
the CPU timebase frequency, and handling the zoo of SMP mechanisms
found on PowerPC.

Reviewed by:	marcel, raj
Book-E parts by: raj
2009-05-14 00:34:26 +00:00
raj
9d122f9cba Zero PCB during early AIM PowerPC init.
When memory is not zero'ed by firmware, uninitialized PCB can have bogus
contents, which appear as a saved onfault condition, Altivec context to
restore etc. and lead to corruption/crashes. This commit fixes such issues.

Submitted by:	Michal Mazur arg ! semihalf dot com
Tested by:	Andreas Tobler andreast-list ! fgznet dot ch
2009-04-24 08:57:54 +00:00
nwhitehorn
5db25627cf Fix a typo in the SRR1 comparison for program exceptions. While here,
replace magic numbers with constants to keep this from happening again.

Without this fix, some programs would occasionally get SIGTRAP instead
of SIGILL on an illegal instruction. This affected Altivec detection
in pixman, and possibly other software.

Reported by:	Andreas Tobler
MFC after:	1 week
2009-04-19 06:30:00 +00:00
nwhitehorn
97b415b1b8 Changing the overflow trap to use bla to branch to dbtrap in r190946 was
bogus. Revert to a branch that does not set LR. It's been a long week...
2009-04-14 04:15:56 +00:00
nwhitehorn
32d233ecc5 Rework the way we get the cacheline size. Instead of having a table of
CPUs known to use 128 byte cache lines and defaulting to 32, use the dcbz
instruction to measure it. Also make dcbz behave the way you would
expect on PPC 970.
2009-04-12 03:03:55 +00:00
nwhitehorn
8a26716497 Fix recognition of kernel-mode traps that pass through the KDB trap handler
but do not actually invoke KDB. This includes recoverable machine checks
encountered in kernel mode.

This patch causes machines with Grackle host-PCI bridges to be able to
correctly enumerate them again.

MFC after:	3 days
2009-04-11 20:43:41 +00:00
nwhitehorn
18eefdb2b2 Fix the build when KDB is disabled. The second instance of rfi in
trap_subr.S that is patched at runtime to rfid on 64-bit systems
is inside KDB-specific code, so don't patch it without KDB.
2009-04-05 21:52:13 +00:00
marcel
43f12bcbe6 Perform a dummy stwcx. when we switch contexts. The context
being switched out may hold a reservation. The stwcx. will
clear the reservation. This is architecturally recommended.

The scenario this addresses is as follows:
1. Thread 1 performs a lwarx and as such holds a reservation.
2. Thread 1 gets switched out (before doing the matching
   stwcx.) and thread 2 is switched in.
3. Thread 2 performs a stwcx. to the same reservation granule.
   This will succeed because the processor has the reservation
   even though thread 2 didn't do the lwarx.

Note that on some processors the address given the stwcx. is
not checked. On these processors the mere condition of having
a reservation would cause the stwcx. to succeed, irrespective
of whether the addresses are the same. The dummy stwcx. is
especially important for those processors.
2009-04-04 22:23:03 +00:00
nwhitehorn
ef1e56b6d4 Add support for 64-bit PowerPC CPUs operating in the 64-bit bridge mode
provided, for example, on the PowerPC 970 (G5), as well as on related CPUs
like the POWER3 and POWER4.

This also adds support for various built-in hardware found on Apple G5
hardware (e.g. the IBM CPC925 northbridge).

Reviewed by:    grehan
2009-04-04 00:22:44 +00:00
nwhitehorn
95651d9cb6 Change the PVO zone for fictitious pages to the unmanaged PVO zone, to match
the unmanaged flag set in the PVO attributes. Without doing this,
pmap_remove() could try to remove fictitious pages (like those created
by mmap of physical memory) from the wrong UMA zone, causing a panic.

Reported by:	Justin Hibbits
MFC after:	1 week
2009-03-11 03:19:19 +00:00
nwhitehorn
28bdf866e3 Fix comment: we write the trap vector to SPRG3, not SPRG0. 2009-02-23 19:31:48 +00:00
nwhitehorn
78b3ec3583 Add Altivec support for supported CPUs. This is derived from the FPU support
code, and also reducing the size of trapcode to fit inside a 32 byte handler
slot.

Reviewed by:	grehan
MFC after:	2 weeks
2009-02-20 17:48:40 +00:00
nwhitehorn
3fcef6d9c2 Modularize the Open Firmware client interface to allow run-time switching
of OFW access semantics, in order to allow future support for real-mode
OF access and flattened device frees. OF client interface modules are
implemented using KOBJ, in a similar way to the PPC PMAP modules.

Because we need Open Firmware to be available before mutexes can be used on
sparc64, changes are also included to allow KOBJ to be used very early in
the boot process by only using the mutex once we know it has been initialized.

Reviewed by:    marius, grehan
2008-12-20 00:33:10 +00:00
marcel
322737440f Add support for kernel profiling for both AIM and BookE.
Obtained from:	Juniper Networks, Inc (BookE support).
2008-10-27 02:36:03 +00:00
nwhitehorn
0d091e41de Convert PowerPC AIM PCI and nexus busses to standard OFW bus interface. This
simplifies certain device attachments (Kauai ATA, for instance), and makes
possible others on new hardware.

On G5 systems, there are several otherwise standard PCI devices
(Serverworks SATA) that will not allow their interrupt properties to be
written, so this information must be supplied directly from Open Firmware.

Obtained from:	sparc64
2008-10-14 14:54:14 +00:00
nwhitehorn
2a5e536b9e Allow the cacheline size on PowerPC to be set at runtime. This is essential for
supporting 64-bit CPUs, which often have 128-byte cache lines instead of the
standard 32.
2008-09-24 00:28:46 +00:00
nwhitehorn
d5de411ace In preparation for PowerPC G5 support, allow PVO objects to contain page
table entries for both the 32-bit and 64-bit AIM MMUs.
2008-09-23 03:02:57 +00:00
marcel
5657e9d778 o When not making a translation cache-inhibit and guarded (PTE_I|PTE_G)
make it memory-coherency enforced (PTE_M). This is required for SMP
   to work.
o  Serialize tlbie operations and implement the tlbie operation in a
   function called tlbie(). Hardware can end up in a live-lock if
   between the tlbsync and subsequent sync on one processor another
   processor executes a tlbie or tlbsync.
o  Eliminate the following defines:
	TLBIE, TLBSYNC, SYNC and EIEIO
   Use either inline assembly statements or inline functions defined
   in <machine/cpufunc.h>
2008-09-16 19:16:33 +00:00
marcel
c304425191 Rewrite cpudep_ap_bootstrap(). We now enable L3, L2, L1D and L1I
caches if not yet enabed. This is required for coherency and
atomic operations to work, not to mention performance. We use the
L2 and L3 cache settings of the BSP to configure the APs caches.
Can't be bad.

Program NAP and not DOZE. DOZE is present only on earlier CPUs
and the bit is reserved on the MPC7441 & MPC7451. NAP will do
bus snooping to keep caches coherent.

Program the PIR with the cpuid. This may not be necessary...
2008-09-16 17:22:16 +00:00
marcel
79892756cd o In decr_get_timecount() only read the low timebase register.
We're only returning a 32-bit counter.
o  In decr_intr(), manually perform LICM, so that we don't test
   a loop invariant condition inside a loop.
o  Include <machine/smp.h>
2008-09-16 17:11:33 +00:00
marcel
be22078803 Set pcpup->pc_curthread and pcpup->pc_curpcb before calling
pmap_activate. While pmap_activate doesn't need either, we
do need a valid curthread if we enable KTR_PMAP.
2008-09-16 17:03:52 +00:00