Commit Graph

1884 Commits

Author SHA1 Message Date
Nathan Whitehorn
bc96dccc69 Fix final bugs in memory barriers on PowerPC:
- Use isync/lwsync unconditionally for acquire/release. Use of isync
  guarantees a complete memory barrier, which is important for serialization
  of bus space accesses with mutexes on multi-processor systems.
- Go back to using sync as the I/O memory barrier, which solves the same
  problem as above with respect to mutex release using lwsync, while not
  penalizing non-I/O operations like a return to sync on the atomic release
  operations would.
- Place an acquisition barrier around thread lock acquisition in
  cpu_switchin().
2012-05-04 16:00:22 +00:00
Dimitry Andric
460378bf13 Add a convenience macro for the returns_twice attribute, and apply it to
the prototypes of the appropriate functions (getcontext, savectx,
setjmp, sigsetjmp and vfork).

MFC after:	2 weeks
2012-04-29 11:04:31 +00:00
Nathan Whitehorn
284ea61312 Fix build on 32-bit systems. 2012-04-28 14:42:49 +00:00
Nathan Whitehorn
50e13823c8 After switching mutexes to use lwsync, they no longer provide sufficient
guarantees on acquire for the tlbie mutex. Conversely, the TLB invalidation
sequence provides guarantees that do not need to be redundantly applied on
release. Roll a small custom lock that is just right. Simultaneously,
convert the SLB tree changes back to lwsync, as changing them to sync
was a misdiagnosis of the tlbie barrier problem this commit actually fixes.
2012-04-28 00:12:23 +00:00
Nathan Whitehorn
de63b4d2d5 Switch the default I/O memory barrier to eieio, as it should be. This
does not appear to cause any problems due to fixes elsewhere.

MFC after:	2 months
2012-04-24 13:37:43 +00:00
Nathan Whitehorn
8387bb0c78 Revert r234581 for this file. The lockless SLB tree code does in fact need
a heavyweight sync instead of a lightweight sync to function properly.
Thanks to mdf for the clarification.
2012-04-24 13:36:41 +00:00
Nathan Whitehorn
51a6f57e4a Fix copy-and-paste error in r230400.
MFC after: 3 days
2012-04-23 20:53:50 +00:00
Nathan Whitehorn
2a134f71d1 Fix missing header for powerpc_iomb().
Pointy hat to:	me
2012-04-23 15:47:07 +00:00
Nathan Whitehorn
a4cbf436e7 Provide a clearer split between read/write and acquire/release barriers.
This should really, actually be correct now.
2012-04-22 22:27:35 +00:00
Nathan Whitehorn
14758466eb Correctly specify assembler constrains for synchronization instructions.
MFC after: 3 days
2012-04-22 21:55:19 +00:00
Nathan Whitehorn
a6349a998d Clarify what we are doing in r234583 a little better: eieio and isync do
not provide general barriers, but only barriers in the context of the
atomic sequences here. As such, make them private and keep the global
*mb() routines using a variant of sync.
2012-04-22 21:11:01 +00:00
Nathan Whitehorn
83ae3d5531 On non-64-bit systems (which generally don't have lwsync), use eieio and
isync to implement read and write barriers, following Appendix B.2 of
Book II of the architecture manual. This provides a 25% speed increase
to fork() on the PowerPC G4.
2012-04-22 20:23:34 +00:00
Nathan Whitehorn
6f26a88999 Use lwsync to provide memory barriers on systems that support it instead
of sync (lwsync is an alternate encoding of sync on systems that do not
support it, providing graceful fallback). This provides more than an order
of magnitude reduction in the time required to acquire or release a mutex.

MFC after:	2 months
2012-04-22 19:00:51 +00:00
Nathan Whitehorn
a1f8f44820 Remove dead code. The routines in atomic.S did not work properly anyway, and
were everywhere unused. If we turn out to need them, they should be
reimplemented.

MFC after:	2 weeks
2012-04-22 18:56:56 +00:00
Nathan Whitehorn
13d47f302f Replace eieio; sync for creating bus-space memory barriers with sync.
sync performs a strict superset of the functions of eieio, so using both
is redundant. While here, expand bus barriers to all bus_space operations,
since many drivers do not correctly use bus_space_barrier().

In principle, we can also replace sync just with eieio, for a significant
performance increase, but it remains to be seen whether any poorly-written
drivers currently depend on the side effects of sync to properly function.

MFC after:	1 week
2012-04-22 18:54:51 +00:00
Nathan Whitehorn
0b852c03eb Avoid a lock order reversal in pmap_extract_and_hold() from relocking
the page. This PMAP requires an additional lock besides the PMAP lock
in pmap_extract_and_hold(), which vm_page_pa_tryrelock() did not release.

Suggested by:	kib
MFC after:	4 days
2012-04-22 17:58:30 +00:00
Nathan Whitehorn
fbd21ea620 Organize some members of ucontext_t in the same order they are in the
trap frame. These are usually not used, and so this changes very little.

MFC after:	5 days
2012-04-21 14:39:47 +00:00
Nathan Whitehorn
c13aac3896 Make sure all pending operations have completed on the existing thread
before (potentially) migrating it to a different CPU.

MFC after:	5 days
2012-04-20 23:01:36 +00:00
Nathan Whitehorn
e3c2930d36 We don't need kcopy() in any of the remaining places it is used, so
remove it.

MFC after:	2 weeks
2012-04-11 22:23:50 +00:00
Nathan Whitehorn
b6aeb1ab97 Only manipulate the PGA_EXECUTABLE flag on managed pages. This is a proxy
for whether the page is physical. On dense phys mem systems (32-bit),
VM_PHYS_TO_PAGE will not return NULL for device memory pages if device
memory is above physical memory even if there is no allocated vm_page.
Attempting to use the returned page could then cause either memory
corruption or a page fault.
2012-04-11 21:56:55 +00:00
Nathan Whitehorn
805bee55eb Fix error in r233949. Synchronizing icaches on uncacheable pages turns out
not to be a good idea, and of course the PV entry list for a page is never
empty after the page has been mapped.
2012-04-11 20:28:05 +00:00
Nathan Whitehorn
88fe385600 Do not restore the register holding the TLS pointer when doing various
usermode context switches (long jumps and ucontext operations). If these
are used across threads, multiple threads can end up with the same TLS base.
Madness will then result.

This makes behavior on PPC match that on x86 systems and on Linux.

MFC after:	10 days
2012-04-11 00:00:40 +00:00
Nathan Whitehorn
b7d0d1fabf Execute an initial ptesync if and only if the PTE is actually being
invalidated, as opposed to a ref/changed bit update.
2012-04-06 22:33:13 +00:00
Nathan Whitehorn
348bc07000 Substantially reduce the scope of the locks held in pmap_enter(), which
improves concurrency slightly.
2012-04-06 18:18:48 +00:00
Nathan Whitehorn
57bd5cce62 Reduce the frequency that the PowerPC/AIM pmaps invalidate instruction
caches, by invalidating kernel icaches only when needed and not flushing
user caches for shared pages.

Suggested by:	kib
MFC after:	2 weeks
2012-04-06 16:03:38 +00:00
Nathan Whitehorn
629e40e45e Give the kernel pmap lock a different name than user pmap locks. It has
(slightly) different semantics and renaming it prevents a (harmless)
WITNESS warning during bootup for 32-bit kernels on 64-bit CPUs.

MFC after:	5 days
2012-04-06 16:00:37 +00:00
John Baldwin
1f22be4547 - Rename VM_MEMATTR_UNCACHED to VM_MEMATTR_WEAK_UNCACHEABLE on x86 to
be less ambiguous and more clearly identify what it means.  This
  attribute is what Intel refers to as UC-, and it's only difference
  relative to normal UC memory is that a WC MTRR will override a UC-
  PAT entry causing the memory to be treated as WC, whereas a UC PAT
  entry will always override the MTRR.
- Remove the VM_MEMATTR_UNCACHED alias from powerpc.
2012-03-29 16:51:22 +00:00
Nathan Whitehorn
13b5e92e01 Allow multiple inclusion of trap.h. This has always been broken, but
until recently never caused problems.
2012-03-29 02:02:14 +00:00
Fabien Thomas
f5f9340b98 Add software PMC support.
New kernel events can be added at various location for sampling or counting.
This will for example allow easy system profiling whatever the processor is
with known tools like pmcstat(8).

Simultaneous usage of software PMC and hardware PMC is possible, for example
looking at the lock acquire failure, page fault while sampling on
instructions.

Sponsored by: NETASQ
MFC after:	1 month
2012-03-28 20:58:30 +00:00
Nathan Whitehorn
7e55df27cb More PMAP performance improvements: skip 256 MB segments entirely if they
are are not mapped during ranged operations and reduce the scope of the
tlbie lock only to the actual tlbie instruction instead of the entire
sequence. There are a few more optimization possibilities here as well.
2012-03-28 17:25:29 +00:00
Nathan Whitehorn
a3e9e259b3 Make sure to call vm_page_dirty() before the pmap lock is released to
prevent a race where another process could conclude the page was clean.

Submitted by:	alc
2012-03-27 01:26:00 +00:00
Nathan Whitehorn
5afcb4c91e More PMAP concurrency improvements: replace the table lock and (almost) all
uses of the page queues mutex with a new rwlock that protects the page
table and the PV lists. This reduces system time during a parallel
buildworld by 35%.

Reviewed by:	alc
2012-03-27 01:24:18 +00:00
Nathan Whitehorn
e71dfa7b84 More PMAP performance improvements: on powerpc64, when TLBIE can be run
with exceptions enabled, leave them enabled and use a regular mutex to
guard TLB invalidations instead of a spinlock.
2012-03-25 06:01:34 +00:00
Nathan Whitehorn
d456d3e31f Only call vm_page_dirty() on pages that are writable in order not to
confuse the VM.
2012-03-24 22:32:19 +00:00
Nathan Whitehorn
8e7c7ea2ea Following suggestions from alc, skip wired mappings in pmap_remove_pages()
and remove moea64_attr_*() in favor of direct calls to vm_page_dirty()
and friends.
2012-03-24 19:59:14 +00:00
Ed Schouten
92396a3174 Remove pty(4) from our kernel configurations.
As of FreeBSD 8, this driver should not be used. Applications that use
posix_openpt(2) and openpty(3) use the pts(4) that is built into the
kernel unconditionally. If it turns out high profile depend on the
pty(4) module anyway, I'd rather get those fixed. So please report any
issues to me.

The pty(4) module is still available as a kernel module of course, so a
simple `kldload pty' can be used to run old-style pseudo-terminals.
2012-03-21 08:38:42 +00:00
Andreas Tobler
43cbf1a696 Provide a fix for certain PowerMacs where the U3 i2c lacks the interrupt
info.

Tested by:	Robert Hish
MFC after:	1 week
2012-03-19 18:03:20 +00:00
Nathan Whitehorn
07b638a98e Remove acquisition of VM page queues lock from pmap_protect(). Any actual
manipulation of the pvo_vlink and pvo_olink entries is already protected
by the table lock, so most remaining instances of the acquisition of the
page queues lock can likely be replaced with the table lock, or removed
if the table lock is already held.

Reviewed by:	alc
2012-03-18 13:22:42 +00:00
Nathan Whitehorn
0d8d9edaaa Make ofw_bus_get_node() consistently return -1 when there is no associated
OF node, instead of a random mixture of 0 and -1. Update all checks for 0
to check for -1 instead.

MFC after:	4 weeks
2012-03-15 22:53:39 +00:00
Nathan Whitehorn
cd907a68aa Implement pmap_remove_pages(). This will be added later to the 32-bit MMU
module.

Suggested by:	alc
2012-03-15 22:50:48 +00:00
Nathan Whitehorn
246e44956e Improve algorithm for deciding whether to loop through all process pages
or look them up individually in pmap_remove() and apply the same logic
in the other ranged operation (pmap_protect). This speeds up make
installworld by a factor of 2 on powerpc64.

MFC after:	1 week
2012-03-15 19:36:52 +00:00
Nathan Whitehorn
cbfa304088 Use LIST_FOREACH_SAFE() instead of LIST_FOREACH() in pmap_remove(), since
the point of this loop is to remove elements. This worked by accident before.

MFC after:	2 days
2012-03-14 20:19:49 +00:00
Dimitry Andric
63d094a7e2 Add casts to __uint16_t to the __bswap16() macros on all arches which
didn't already have them.  This is because the ternary expression will
return int, due to the Usual Arithmetic Conversions.  Such casts are not
needed for the 32 and 64 bit variants.

While here, add additional parentheses around the x86 variant, to
protect against unintended consequences.

MFC after:	2 weeks
2012-03-09 20:34:31 +00:00
Attilio Rao
9c170fd168 Disable the option VFS_ALLOW_NONMPSAFE by default on all the supported
platforms.
This will make every attempt to mount a non-mpsafe filesystem to the
kernel forbidden, unless it is expressely compiled with
VFS_ALLOW_NONMPSAFE option.

This patch is part of the effort of killing non-MPSAFE filesystems
from the tree.

No MFC is expected for this patch.
2012-03-06 20:01:25 +00:00
Andreas Tobler
d59a23dc09 Restore proper dot symbol creation for assembly files in the kernel build case.
Without this patch we were not able to see the assembly function.
Only the function descriptor was visible.

- Distinguish between user-land and kernel when creating the ENTRY() point of
  assembly source.
- Make the ENTRY() macro more readable, replace the .align directive with the
  gas platform independant .p2align directive.
- Create an END()macro for later use to provide traceback tables on powerpc64.
2012-03-04 11:55:28 +00:00
Andreas Tobler
0cb9915474 Add support for PWM controlled fans. I found these fans on my PowerMac9,1.
These fans are not located under the same node as the the RPM controlled ones,
So I had to adapt the current source to parse and fill the properties correctly.
To control the fans we can set the PWM ratio via sysctl between 20 and 100%.

Tested by:	nwhitehorn
MFC after:	3 weeks
2012-03-04 08:43:33 +00:00
John Baldwin
1b1596a3b3 - Add a bus_dma tag to each PCI bus that is a child of a Host-PCI bridge.
The tag enforces a single restriction that all DMA transactions must not
  cross a 4GB boundary.  Note that while this restriction technically only
  applies to PCI-express, this change applies it to all PCI devices as it
  is simpler to implement that way and errs on the side of caution.
- Add a softc structure for PCI bus devices to hold the bus_dma tag and
  a new pci_attach_common() routine that performs actions common to the
  attach phase of all PCI bus drivers.  Right now this only consists of
  a bootverbose printf and the allocate of a bus_dma tag if necessary.
- Adjust all PCI bus drivers to allocate a PCI bus softc and to call
  pci_attach_common() from their attach routines.

MFC after:	2 weeks
2012-03-02 20:38:04 +00:00
John Baldwin
831ce4cb3d - Change contigmalloc() to use the vm_paddr_t type instead of an unsigned
long for specifying a boundary constraint.
- Change bus_dma tags to use bus_addr_t instead of bus_size_t for boundary
  constraints.

These allow boundary constraints to be fully expressed for cases where
sizeof(bus_addr_t) != sizeof(bus_size_t).  Specifically, it allows a
driver to properly specify a 4GB boundary in a PAE kernel.

Note that this cannot be safely MFC'd without a lot of compat shims due
to KBI changes, so I do not intend to merge it.

Reviewed by:	scottl
2012-03-01 19:58:34 +00:00
Justin Hibbits
3cc28bd9bf Add backlight control to ATI-graphics PowerBooks and iBooks.
Approved by:	nwhitehorn (mentor)
MFC after:	1 week
2012-02-26 13:45:25 +00:00
Andreas Tobler
3f07025a76 Enable the new PCI-PCI bridge driver by default.
Tested on 32- and 64-bit PowerMac.
2012-02-19 10:10:12 +00:00