Commit Graph

721 Commits

Author SHA1 Message Date
Nathan Whitehorn
46e93cbbc5 Add an extra invariant here which was useful on 64-bit CPUs. 2011-11-17 15:48:12 +00:00
Alan Cox
fbd80bd047 Refactor the code that performs physically contiguous memory allocation,
yielding a new public interface, vm_page_alloc_contig().  This new function
addresses some of the limitations of the current interfaces, contigmalloc()
and kmem_alloc_contig().  For example, the physically contiguous memory that
is allocated with those interfaces can only be allocated to the kernel vm
object and must be mapped into the kernel virtual address space.  It also
provides functionality that vm_phys_alloc_contig() doesn't, such as wiring
the returned pages.  Moreover, unlike that function, it respects the low
water marks on the paging queues and wakes up the page daemon when
necessary.  That said, at present, this new function can't be applied to all
types of vm objects.  However, that restriction will be eliminated in the
coming weeks.

From a design standpoint, this change also addresses an inconsistency
between vm_phys_alloc_contig() and the other vm_phys_alloc*() functions.
Specifically, vm_phys_alloc_contig() manipulated vm_page fields that other
functions in vm/vm_phys.c didn't.  Moreover, vm_phys_alloc_contig() knew
about vnodes and reservations.  Now, vm_page_alloc_contig() is responsible
for these things.

Reviewed by:	kib
Discussed with:	jhb
2011-11-16 16:46:09 +00:00
Nathan Whitehorn
8a4cf006f4 Fix a bug where the pmap_cpu_bootstrap() ap argument could be clobbered.
Luckily, it mostly wasn't important, so this didn't cause major problems.
Also improve register reuse when setting up trap frames very slightly.

Submitted by:	Justin Hibbits <chmeeedalf at gmail dot com>
MFC after:	5 days
2011-11-09 13:48:23 +00:00
Konstantin Belousov
26ccf4f10f Inline the syscallenter() and syscallret(). This reduces the time measured
by the syscall entry speed microbenchmarks by ~10% on amd64.

Submitted by:	jhb
Approved by:	re (bz)
MFC after:	2 weeks
2011-09-11 16:05:09 +00:00
Konstantin Belousov
3407fefef6 Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic
flags field. Updates to the atomic flags are performed using the atomic
ops on the containing word, do not require any vm lock to be held, and
are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9)
functions are provided to modify afalgs.

Document the changes to flags field to only require the page lock.

Introduce vm_page_reference(9) function to provide a stable KPI and
KBI for filesystems like tmpfs and zfs which need to mark a page as
referenced.

Reviewed by:    alc, attilio
Tested by:      marius, flo (sparc64); andreast (powerpc, powerpc64)
Approved by:	re (bz)
2011-09-06 10:30:11 +00:00
Konstantin Belousov
d98d0ce27a - Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag
to VPO_UNMANAGED (and also making the flag protected by the vm object
  lock, instead of vm page queue lock).
- Mark the fake pages with both PG_FICTITIOUS (as it is now) and
  VPO_UNMANAGED. As a consequence, pmap code now can use use just
  VPO_UNMANAGED to decide whether the page is unmanaged.

Reviewed by:	alc
Tested by:	pho (x86, previous version), marius (sparc64),
    marcel (arm, ia64, powerpc), ray (mips)
Sponsored by:	The FreeBSD Foundation
Approved by:	re (bz)
2011-08-09 21:01:36 +00:00
Andreas Tobler
42f2270475 This a follow up commit from r224216 for powerpc 32-bit. Increase
the storage size for sintrcnt/sintrnames to .long.

Reviewed by: nwhitehorn
Approved by: re (kib)
2011-07-25 20:10:01 +00:00
Attilio Rao
732772c701 On 64 bit architectures size_t is 8 bytes, thus it should use an 8 bytes
storage.
Fix the sintrcnt/sintrnames specification.

No MFC is previewed for this patch.

Reported, reviewed and tested by:	marcel
Approved by:	re (kib)
2011-07-19 12:41:57 +00:00
Attilio Rao
521ea19d1c - Remove the eintrcnt/eintrnames usage and introduce the concept of
sintrcnt/sintrnames which are symbols containing the size of the 2
  tables.
- For amd64/i386 remove the storage of intr* stuff from assembly files.
  This area can be widely improved by applying the same to other
  architectures and likely finding an unified approach among them and
  move the whole code to be MI. More work in this area is expected to
  happen fairly soon.

No MFC is previewed for this patch.

Tested by:	pluknet
Reviewed by:	jhb
Approved by:	re (kib)
2011-07-18 15:19:40 +00:00
Attilio Rao
cfdfd32d34 MFC 2011-06-26 17:30:46 +00:00
Nathan Whitehorn
1b17fa33dc Revert r223479. It is unnecessary and served only to slightly ameliorate
some manifestations of the bug actually fixed in r223485.
2011-06-26 15:08:14 +00:00
Attilio Rao
de138ec703 MFC 2011-06-24 16:35:40 +00:00
Nathan Whitehorn
e69dff491d Use the ABI-mandated thread pointer register (r2 for ppc32, r13 for ppc64)
instead of a PCPU field for curthread. This averts a race on SMP systems
with a high interrupt rate where the thread looking up the value of
curthread could be preempted and migrated between obtaining the PCPU
pointer and reading the value of pc_curthread, resulting in curthread being
observed to be the current thread on the thread's original CPU. This played
merry havoc with the system, in particular with mutexes. Many thanks to
jhb for helping me work this one out.

Note that Book-E is in principle susceptible to the same problem, but has
not been modified yet due to lack of Book-E hardware.

MFC after:	2 weeks
2011-06-23 22:21:28 +00:00
Nathan Whitehorn
045aee08f3 Clear any outstanding atomic reservations when traps are taken. This fixes
some interesting bugs (mostly on SMP systems) with atomic operations
silently failing in interrupt heavy situations, especially when using
overflow pages.
2011-06-23 16:34:41 +00:00
Andreas Tobler
dcf496e844 Fix merge typo. 2011-06-23 09:46:12 +00:00
Attilio Rao
c7c2767e33 Remove pc_other_cpus and pc_cpumask usage from powerpc support.
Tested and reviewed by:	andreast
2011-06-16 07:27:13 +00:00
Attilio Rao
3bce356ea4 MFC 2011-06-04 22:05:20 +00:00
Nathan Whitehorn
4770f5380e Fix a typo derived from a mismerge from mmu_oea that would cause
pmap_sync_icache() to sync random (possibly uncached or nonexisting!)
memory, causing kernel page faults or machine checks, most easily
triggered by using GDB. While here, add an additional safeguard to only
sync cacheable memory.

MFC after:	2 days
2011-06-04 03:22:16 +00:00
Attilio Rao
d7073a2b3b MFC 2011-06-03 17:09:15 +00:00
Nathan Whitehorn
cd507188bc Quantities stored on the stack on ppc64 tend to be twice as large as on
ppc32, so make the early stack correspondingly twice as big.
2011-06-03 00:11:13 +00:00
Nathan Whitehorn
17763042e4 The POWER7 has only 32 SLB slots instead of 64, like other supported
64-bit PowerPC CPUs. Add infrastructure to support variable numbers of
SLB slots and move the user slot from 63 to 0, so that it is always
available.
2011-06-02 14:25:52 +00:00
Nathan Whitehorn
1dff98d9bb If running under a hypervisor, don't yell at the user about starting
unknown CPU types, instead relying on the hypervisor to have given us a
reasonable environment.
2011-06-02 14:23:36 +00:00
Nathan Whitehorn
20ae1015b9 Explicitly initialize the first thread's MSR to PSL_KERNSET. 2011-06-02 14:21:20 +00:00
Nathan Whitehorn
6dd24ab3f1 Include the modules area in the mapped kernel code. This fixes the kernel's
access to modules and loader metadata when started from real mode, but
without a direct map.
2011-06-02 14:19:18 +00:00
Nathan Whitehorn
97f7cde42c Remove some dead code: unnecessary isyncs and memory sorting, which are
handled in mtmsr() and mem_regions(), respectively.
2011-06-02 14:15:44 +00:00
Nathan Whitehorn
1787909001 MFpseries:
Renovate and improve the AIM Open Firmware support:
- Add RTAS (Run-Time Abstraction Services) support, found on all IBM systems
  and some Apple ones
- Improve support for 32-bit real mode Open Firmware systems
- Pull some more OF bits over from the AIM directory
- Fix memory detection on IBM LPARs and systems with more than one /memory
  node (by andreast@)
2011-06-02 14:12:37 +00:00
Attilio Rao
7fcdc9a26f MFC 2011-05-26 17:38:00 +00:00
Nathan Whitehorn
2ec6a5984c Add a missing isync. 2011-05-26 14:34:22 +00:00
Attilio Rao
7e7a34e520 MFC 2011-05-16 16:34:03 +00:00
Nathan Whitehorn
43db7b0eab Remove a useless check that served only to make 64-bit PPC systems
unbootable after r221855.

Submitted by:	andreast
MFC after:	1 week
2011-05-16 03:32:40 +00:00
Attilio Rao
c47dd3db8c Add the powerpc support.
Note that there is a dirty hack for calling openpic_write(), but
nwhitehorn approved it.

Discussed with:	nwhitehorn
2011-05-09 16:16:15 +00:00
Andreas Tobler
c819dfaeba Add leading zeros when printing the physical memory chunks on __powerpc64__.
Approved by:	nwhitehorn (mentor)
2011-04-19 07:49:58 +00:00
Andreas Tobler
415a54c8c5 Adjust debugging string to match the actual function.
Approved by: nwhitehorn (mentor)
2011-04-14 19:37:31 +00:00
Andreas Tobler
8fd7d65779 The macro MOEA_PVO_CHECK is empty and not used. It is a left over from the
NetBSD import. Remove the definition and all its occurrences.

Approved by: nwhitehorn (mentor)
2011-04-14 18:26:50 +00:00
Matthew D Fleming
c77715ef6c Mostly revert r219468, as I had misremembered the C standard regarding
the size of an extern array.

Keep one change from strncpy to strlcpy.
2011-03-11 18:56:55 +00:00
Matthew D Fleming
cd67ac41ae Use MAXPATHLEN rather than the size of an extern array when copying the
kernel name.  Also consistenly use strlcpy().

Suggested by:	Warner Losh
2011-03-10 22:56:00 +00:00
Nathan Whitehorn
79c77d726e Turn off default generation of userland dot symbols on powerpc64 now that
we have a binutils that supports it. Kernel dot symbols remain on to assist
DDB.
2011-02-18 21:44:53 +00:00
Dmitry Chagin
a5c1afadeb Add macro to test the sv_flags of any process. Change some places to test
the flags instead of explicit comparing with address of known sysentvec
structures.

MFC after:	1 month
2011-01-26 20:03:58 +00:00
Sergey Kandaurov
4053b05b91 Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize.
Submitted by:	perryh pluto.rain.com (previous version)
Reviewed by:	jhb
Approved by:	kib (mentor)
Tested by:	universe
2011-01-21 10:26:26 +00:00
Konstantin Belousov
55aabb7fd1 For architectures not using direct map , and requiring real KVA page for
sf buf allocation, use wakeup() instead of wakeup_one() to notify sf
buffer waiters about free buffer.

sf_buf_alloc() calls msleep(PCATCH) when SFB_CATCH flag was given,
and for simultaneous wakeup and signal delivery, msleep() returns
EINTR/ERESTART despite the thread was selected for wakeup_one(). As
result, we loose a wakeup, and some other waiter will not be woken up.

Reported and tested by:	az
Reviewed by:	alc, jhb
MFC after:	1 week
2011-01-18 21:57:02 +00:00
Andreas Tobler
49ffb2cf8c Remove unused variables. Spotted by a cppcheck
(devel/cppcheck, http://sourceforge.net/projects/cppcheck) run.

Approved by: nwhitehorn (mentor)
2011-01-15 19:16:05 +00:00
Nathan Whitehorn
ff30eecffe Fix handling of NX pages on capable CPUs. Thanks to kib for prodding me
in the right direction.
2011-01-13 04:37:48 +00:00
Andreas Tobler
7dbe66c157 Remove unused variables. Spotted by a cppcheck
(devel/cppcheck, http://sourceforge.net/projects/cppcheck) run.

Approved by: nwhitehorn (mentor)
2011-01-06 20:19:01 +00:00
Nathan Whitehorn
52a190480f Only keep track of PTE validity statistics for pages not locked in the
table. The 'locked' attribute is used to circumvent the regular page table
locking for some special pages, with the result that including locked pages
here causes races when updating the stats.
2010-12-28 17:02:15 +00:00
Nathan Whitehorn
ed1e1e2a9e Garbage-collect unused variable. 2010-12-19 16:07:53 +00:00
Nathan Whitehorn
41f15bbbd9 Add some isync()s related to the 64-bit MMU scratch page to avoid race
conditions on its invalidation.
2010-12-11 20:29:52 +00:00
Nathan Whitehorn
bef5da7f98 Add an abstraction layer to the 64-bit AIM MMU's page table manipulation
logic to support modifying the page table through a hypervisor. This
uses KOBJ inheritance to provide subclasses of the base 64-bit AIM MMU
class with additional methods for page table manipulation.

Many thanks to Peter Grehan for suggesting this design and implementing
the MMU KOBJ inheritance mechanism.
2010-12-04 02:42:52 +00:00
Dimitry Andric
3e288e6238 After some off-list discussion, revert a number of changes to the
DPCPU_DEFINE and VNET_DEFINE macros, as these cause problems for various
people working on the affected files.  A better long-term solution is
still being considered.  This reversal may give some modules empty
set_pcpu or set_vnet sections, but these are harmless.

Changes reverted:

------------------------------------------------------------------------
r215318 | dim | 2010-11-14 21:40:55 +0100 (Sun, 14 Nov 2010) | 4 lines

Instead of unconditionally emitting .globl's for the __start_set_xxx and
__stop_set_xxx symbols, only emit them when the set_vnet or set_pcpu
sections are actually defined.

------------------------------------------------------------------------
r215317 | dim | 2010-11-14 21:38:11 +0100 (Sun, 14 Nov 2010) | 3 lines

Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.

------------------------------------------------------------------------
r215316 | dim | 2010-11-14 21:23:02 +0100 (Sun, 14 Nov 2010) | 2 lines

Add macros to define static instances of VNET_DEFINE and DPCPU_DEFINE.
2010-11-22 19:32:54 +00:00
Dimitry Andric
31c6a0037e Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.
2010-11-14 20:38:11 +00:00
Nathan Whitehorn
cebdaa5881 Partially revert r215182. There appears to be a silicon bug on the 970
that causes AP bringup to fail if some of the Cell HID-register code
is anywhere in the instruction stream. Pending a better solution, cache
performance on SMP Cell systems running without a hypervisor will be
suboptimal.
2010-11-12 20:26:34 +00:00
Nathan Whitehorn
2971d3bb6e Add CPU support code for the IBM Cell Broadband Engine. 2010-11-12 15:20:10 +00:00
Nathan Whitehorn
fe3b4685c7 Remove use of a separate ofw_pmap on 32-bit CPUs. Many Open Firmware
mappings need to end up in the kernel anyway since the kernel begins
executing in OF context. Separating them adds needless complexity,
especially since the powerpc64 and mmu_oea64 code gave up on it a long
time ago.

As a side effect, the PPC ofw_machdep code is no longer AIM-specific,
so move it to powerpc/ofw.
2010-11-12 05:12:38 +00:00
Nathan Whitehorn
16bfd6f347 Remove or conditionalize some hypervisor-unfriendly instruction sequences. 2010-11-12 04:22:00 +00:00
Nathan Whitehorn
6413b05739 Add some platform KOBJ extensions and continue integrating PowerPC
hypervisor infrastructure support:
- Fix coexistence of multiple platform modules in the same kernel
- Allow platform modules to provide an SMP topology
- PowerPC hypervisors limit the amount of memory accessible in real mode.
  Allow the platform modules to specify the maximum real-mode address,
  and modify the bits of the kernel that need to allocate
  real-mode-accessible buffers to respect this limits.
2010-11-12 04:18:19 +00:00
Nathan Whitehorn
b13c7dec5f Fix an error in r215067. An existing /chosen/mmu but missing translations
property just means we shouldn't add any translations, not that we should
panic.
2010-11-12 04:13:48 +00:00
Nathan Whitehorn
5b7ed13bc8 Centralize CPU idle routines into powerpc/cpu.c and use the same
cpu_idle_hook mechanism that x86 uses for overriding the idle routine.
This is required for supporting ilding the CPU under PowerPC hypervisors.
2010-11-12 03:43:22 +00:00
Rafal Jaworowski
1f87b29431 Fix typo in the comment. 2010-11-11 13:46:28 +00:00
Nathan Whitehorn
5ebee02036 Add support for the IMISS, DLMISS, and DSMISS traps required to run
FreeBSD on a G2 core.

PR:		powerpc/111296
Submitted by:	Andrew Turner
2010-11-11 02:40:00 +00:00
Nathan Whitehorn
2824d5d2f4 Make AIM early-boot code function correctly without Open Firmware. 2010-11-09 23:53:47 +00:00
John Baldwin
0108cce0a4 Adjust the order of operations in spinlock_enter() and spinlock_exit() to
work properly with single-stepping in a kernel debugger.  Specifically,
these routines have always disabled interrupts before increasing the nesting
count and restored the prior state of interrupts after decreasing the nesting
count to avoid problems with a nested interrupt not disabling interrupts
when acquiring a spin lock.  However, trap interrupts for single-stepping
can still occur even when interrupts are disabled.  Now the saved state of
interrupts is not saved in the thread until after interrupts have been
disabled and the nesting count has been increased.  Similarly, the saved
state from the thread cannot be read once the nesting count has been
decreased to zero.  To fix this, use temporary variables to store interrupt
state and shuffle it between the thread's MD area and the appropriate
registers.

In cooperation with:	bde
MFC after:     1 month
2010-11-05 13:42:58 +00:00
Nathan Whitehorn
87acfc2a51 Fix two mistakes on 32-bit systems. The slbmte code in syscall() is 64-bit
only, and should be protected with an ifdef, and the no-execute bit in
32-bit set_user_sr() should be set before the comparison, not after, or
it will never match.
2010-11-03 16:21:47 +00:00
Nathan Whitehorn
e0f88469c7 Clean up the user segment handling code a little more. Now that
set_user_sr() itself caches the user segment VSID, there is no need for
cpu_switch() to do it again. This change also unifies the 32 and 64-bit
code paths for kernel faults on user pages and remaps the user SLB slot
on 64-bit systems when taking a syscall to avoid some unnecessary segment
exception traps.
2010-11-03 15:15:48 +00:00
Alan Cox
e396eb604f Implement pmap_is_prefaultable().
Reviewed by:	nwhitehorn
2010-11-01 02:22:48 +00:00
Nathan Whitehorn
e36e3d8221 Add a security nit to recent copyin/out changes: map the user segment
no-execute in case of exploitable kernel bugs.

MFC after:	1 week
2010-10-31 23:04:15 +00:00
Nathan Whitehorn
ad6b3047a4 Next-to-leading-order perturbation of synchronization operations for
switching the user segment register. All races should now be closed and
a minimum of pipelines flushes be required to close them.
2010-10-31 22:55:51 +00:00
Nathan Whitehorn
c4bcebed17 Add some missing parentheses so that moea_bat_mapped() actually works.
Submitted by:	alc
MFC after:	3 days
2010-10-31 15:07:09 +00:00
Nathan Whitehorn
54c562081f Restructure the way the copyin/copyout segment is stored to prevent a
concurrency bug. Since all SLB/SR entries were invalidated during an
exception, a decrementer exception could cause the user segment to be
invalidated during a copyin()/copyout() without a thread switch that
would cause it to be restored from the PCB, potentially causing the
operation to continue on invalid memory. This is now handled by explicit
restoration of segment 12 from the PCB on 32-bit systems and a check in
the Data Segment Exception handler on 64-bit.

While here, cause copyin()/copyout() to check whether the requested
user segment is already installed, saving some pipeline flushes, and
fix the synchronization primitives around the mtsr and slbmte
instructions to prevent accessing stale segments.

MFC after:	2 weeks
2010-10-30 23:07:30 +00:00
Nathan Whitehorn
2639d62ec2 Handle vector assist traps without a kernel panic, by setting denormalized
values to zero. A correct solution would involve emulating vector
operations on denormalized values, but this has little effect on accuracy
and is much less complicated for now.

MFC after:	2 weeks
2010-10-05 18:08:07 +00:00
Nathan Whitehorn
94363f5311 Follow exactly the steps in architecture manual for correctly invalidating
TLB entries instead of trying to cut corners.
2010-10-04 16:07:48 +00:00
Nathan Whitehorn
cd6a97f065 Fix pmap_page_set_memattr() behavior in the presence of fictitious pages
by just caching the mode for later use by pmap_enter(), following amd64.
While here, correct some mismerges from mmu_oea64 -> mmu_oea and clean
up some dead code found while fixing the fictitious page behavior.
2010-10-01 18:59:30 +00:00
Nathan Whitehorn
c1f4123b05 Add support for memory attributes (pmap_mapdev_attr() and friends) on
PowerPC/AIM. This is currently stubbed out on Book-E, since I have no
idea how to implement it there.
2010-09-30 18:14:12 +00:00
Nathan Whitehorn
6416b9a85d Split the SLB mirror cache into two kinds of object, one for kernel maps
which are similar to the previous ones, and one for user maps, which
are arrays of pointers into the SLB tree. This changes makes user SLB
updates atomic, closing a window for memory corruption. While here,
rearrange the allocation functions to make context switches faster.
2010-09-16 03:46:17 +00:00
Nathan Whitehorn
95fa3335e1 Replace the SLB backing store splay tree used on 64-bit PowerPC AIM
hardware with a lockless sparse tree design. This marginally improves
the performance of PMAP and allows copyin()/copyout() to run without
acquiring locks when used on wired mappings.

Submitted by:	mdf
2010-09-16 00:22:25 +00:00
Peter Grehan
33529b98d5 Introduce inheritance into the PowerPC MMU kobj interface.
include/mmuvar.h - Change the MMU_DEF macro to also create the class
definition as well as define the DATA_SET. Add a macro, MMU_DEF_INHERIT,
which has an extra parameter specifying the MMU class to inherit methods
from. Update the comments at the start of the header file to describe the
new macros.

booke/pmap.c
aim/mmu_oea.c
aim/mmu_oea64.c - Collapse mmu_def_t declaration into updated MMU_DEF macro

The MMU_DEF_INHERIT macro will be used in the PS3 MMU implementation to
allow it to inherit the stock powerpc64 MMU methods.

Reviewed by:	nwhitehorn
2010-09-15 00:17:52 +00:00
Peter Grehan
44633af3c7 Resurrect PSIM support by moving the cacheline size-detection warning
printf outside of the MMU-disabled region. A call into OpenFirmware
with the MMU off resulted in an internal PSIM assert.
2010-09-14 03:18:11 +00:00
Alexander Motin
a157e42516 Refactor timer management code with priority to one-shot operation mode.
The main goal of this is to generate timer interrupts only when there is
some work to do. When CPU is busy interrupts are generating at full rate
of hz + stathz to fullfill scheduler and timekeeping requirements. But
when CPU is idle, only minimum set of interrupts (down to 8 interrupts per
second per CPU now), needed to handle scheduled callouts is executed.
This allows significantly increase idle CPU sleep time, increasing effect
of static power-saving technologies. Also it should reduce host CPU load
on virtualized systems, when guest system is idle.

There is set of tunables, also available as writable sysctls, allowing to
control wanted event timer subsystem behavior:
  kern.eventtimer.timer - allows to choose event timer hardware to use.
On x86 there is up to 4 different kinds of timers. Depending on whether
chosen timer is per-CPU, behavior of other options slightly differs.
  kern.eventtimer.periodic - allows to choose periodic and one-shot
operation mode. In periodic mode, current timer hardware taken as the only
source of time for time events. This mode is quite alike to previous kernel
behavior. One-shot mode instead uses currently selected time counter
hardware to schedule all needed events one by one and program timer to
generate interrupt exactly in specified time. Default value depends of
chosen timer capabilities, but one-shot mode is preferred, until other is
forced by user or hardware.
  kern.eventtimer.singlemul - in periodic mode specifies how much times
higher timer frequency should be, to not strictly alias hardclock() and
statclock() events. Default values are 2 and 4, but could be reduced to 1
if extra interrupts are unwanted.
  kern.eventtimer.idletick - makes each CPU to receive every timer interrupt
independently of whether they busy or not. By default this options is
disabled. If chosen timer is per-CPU and runs in periodic mode, this option
has no effect - all interrupts are generating.

As soon as this patch modifies cpu_idle() on some platforms, I have also
refactored one on x86. Now it makes use of MONITOR/MWAIT instrunctions
(if supported) under high sleep/wakeup rate, as fast alternative to other
methods. It allows SMP scheduler to wake up sleeping CPUs much faster
without using IPI, significantly increasing performance on some highly
task-switching loads.

Tested by:	many (on i386, amd64, sparc64 and powerc)
H/W donated by:	Gheorghe Ardelean
Sponsored by:	iXsystems, Inc.
2010-09-13 07:25:35 +00:00
Alexander Motin
707c2fb950 Update PowerPC event timer code to use new event timers infrastructure.
Reviewed by:	nwitehorn
Tested by:	andreast
H/W donated by:	Gheorghe Ardelean
2010-09-11 04:45:51 +00:00
Andriy Gapon
3d844eddb7 bus_add_child: change type of order parameter to u_int
This reflects actual type used to store and compare child device orders.
Change is mostly done via a Coccinelle (soon to be devel/coccinelle)
semantic patch.
Verified by LINT+modules kernel builds.

Followup to:	r212213
MFC after:	10 days
2010-09-10 11:19:03 +00:00
Nathan Whitehorn
61473c5fd1 Reorder statistics tracking and table lock acquisitions already in place
to avoid race conditions updating the PVO statistics.
2010-09-09 16:06:55 +00:00
Nathan Whitehorn
bcb478eb35 Fix a printf specifier on 64-bit systems. 2010-09-08 19:28:43 +00:00
Nathan Whitehorn
0dfddf6e65 Fix a typo in the original import of this code from NetBSD that caused the
wrong element of the VSID bitmap array to be examined after a collision,
leading to reallocation of in-use VSIDs under some circumstances, with
attendant memory corruption. Also add an assert to check for this kind of
problem in the future.

MFC after:	4 days
2010-09-08 16:58:06 +00:00
Nathan Whitehorn
4982c539ae Fix an error made in r209975 related to context ID allocation for 64-bit
PowerPC CPUs running a 32-bit kernel. This bug could cause in-use VSIDs
to be allocated again to another process, causing memory space overlaps
and corruption.

Reported by:	linimon
2010-09-07 23:31:48 +00:00
Nathan Whitehorn
e9b5f21819 Fix the same race condition on 32-bit AIM CPUs that was fixed for 64-bit
ones in r211967 involving VSID allocation.
2010-09-06 23:07:58 +00:00
Alexander Motin
a3b31d37df Make nexus report name and compat fields as pnpinfo for devices on the
first level of hierarchy, same as done on deeper levels.
2010-09-05 19:57:24 +00:00
Nathan Whitehorn
b2a237be5c Restructure how reset and poweroff are handled on PowerPC systems, since
the existing code was very platform specific, and broken for SMP systems
trying to reboot from KDB.

- Add a new PLATFORM_RESET() method to the platform KOBJ interface, and
  migrate existing reset functions into platform modules.
- Modify the OF_reboot() routine to submit the request by hand to avoid
  the IPIs involved in the regular openfirmware() routine. This fixes
  reboot from KDB on SMP machines.
- Move non-KDB reset and poweroff functions on the Powermac platform
  into the relevant power control drivers (cuda, pmu, smu), instead of
  using them through the Open Firmware backdoor.
- Rename platform_chrp to platform_powermac since it has become
  increasingly Powermac specific. When we gain support for IBM systems,
  we will grow a new platform_chrp.
2010-08-31 15:27:46 +00:00
Nathan Whitehorn
68181d0091 Remove some code made obsolete by the powerpc64 import. 2010-08-31 15:22:09 +00:00
Nathan Whitehorn
1264a5f4c5 Missed one place the SLB lock should be held in r211967. 2010-08-31 02:07:13 +00:00
Nathan Whitehorn
7eeda62ca9 Avoid a race in the allocation of new segment IDs that could result in
memory corruption on heavily loaded SMP systems.

MFC after:	2 weeks
2010-08-29 18:17:38 +00:00
Nathan Whitehorn
50e64c14a2 pmap_mapdev() does not appear to actually need GIANT to be held here,
and asserting that is held breaks drm.

MFC after:	2 weeks
2010-08-27 05:29:59 +00:00
John Baldwin
8c7a92bd4a Remove unused KTRACE includes. 2010-08-19 16:41:27 +00:00
Nathan Whitehorn
3b4b38304e Improve hash coverage for kernel page table entries by modifying the kernel
ESID -> VSID map function. This makes ZFS run stably on PowerPC under
heavy loads (repeated simultaneous SVN checkouts and updates).
2010-07-31 21:35:15 +00:00
Nathan Whitehorn
c3e289e1ce MFppc64:
Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.
2010-07-13 05:32:19 +00:00
Nathan Whitehorn
cc81c44dd8 Unify ABI-related bits of the Book-E and AIM machdep routines
(exec_setregs, etc.) in order to simplify the addition of 64-bit support,
and possible future extension of the Book-E code to handle hard floating
point and Altivec.

MFC after:	1 month
2010-07-12 16:08:07 +00:00
Nathan Whitehorn
932773c882 The number after 2 is 3, not 4.
MFC after:	3 days
2010-07-09 14:04:16 +00:00
Nathan Whitehorn
ce0df83f13 Remove an unnecessary include of opt_psim.h, which is not present on
powerpc64.
2010-07-09 14:02:57 +00:00
Nathan Whitehorn
945c08644e MFppc64:
Minor 64-bit-cleanliness upgrades and support for platform detection on
subtly-broken OF implementations like in the Mambo simulator.
2010-07-09 14:02:24 +00:00
Nathan Whitehorn
f6421f31e3 Replace the existing PowerPC busdma implementation with the one from
amd64 (with slight modifications). This provides support for bounce
buffers, which are required on systems with RAM above 4 GB.
2010-07-08 15:38:55 +00:00
Marcel Moolenaar
8a35d194f2 Remove the unneeded header <machine/intr.h>. 2010-07-02 02:17:39 +00:00
John Baldwin
fc0de8f0b6 Move prototypes for kern_sigtimedwait() and kern_sigprocmask() to
<sys/syscallsubr.h> where all other kern_<syscall> prototypes live.
2010-06-30 18:03:42 +00:00
Nathan Whitehorn
08393b3efa Configure interrupts on SMP systems to be distributed among all online
CPUs by default, and provide a functional version of BUS_BIND_INTR().
While here, fix some potential concurrency problems in the interrupt
handling code.
2010-06-23 22:33:03 +00:00