Commit Graph

1290 Commits

Author SHA1 Message Date
Alan Cox
40eab1e944 o Lock page queue accesses by vm_page_try_to_cache(). (The accesses
in kern/vfs_bio.c are already locked.)
 o Assert that the page queues lock is held in vm_page_try_to_cache().
2002-07-20 20:58:46 +00:00
Alan Cox
d82efd2956 o Assert that the page queues lock is held in vm_page_try_to_free(). 2002-07-20 20:12:57 +00:00
Alan Cox
15a5d2108e o Lock page queue accesses by vm_page_cache() in vm_fault() and
vm_pageout_scan().  (The others are already locked.)
 o Assert that the page queues lock is held in vm_page_cache().
2002-07-20 19:34:21 +00:00
Alan Cox
48c0444c98 o Lock accesses to the active page queue in vm_pageout_scan() and
vm_pageout_page_stats().
2002-07-20 18:45:25 +00:00
Alan Cox
bda441aa04 o Lock page queue accesses by vm_page_cache() in vm_contig_launder().
o Micro-optimize the control flow in vm_contig_launder().
2002-07-20 06:11:16 +00:00
Alan Cox
6fd77192b2 o Remove dead and/or unused code. 2002-07-20 05:06:20 +00:00
Peter Wemm
3ebc124838 Infrastructure tweaks to allow having both an Elf32 and an Elf64 executable
handler in the kernel at the same time.  Also, allow for the
exec_new_vmspace() code to build a different sized vmspace depending on
the executable environment.  This is a big help for execing i386 binaries
on ia64.   The ELF exec code grows the ability to map partial pages when
there is a page size difference, eg: emulating 4K pages on 8K or 16K
hardware pages.

Flesh out the i386 emulation support for ia64.  At this point, the only
binary that I know of that fails is cvsup, because the cvsup runtime
tries to execute code in pages not marked executable.

Obtained from:  dfr (mostly, many tweaks from me).
2002-07-20 02:56:12 +00:00
Peter Wemm
16e12eab5a Set P_NOLOAD on the pagezero kthread so that it doesn't artificially skew
the loadav.  This is not real load.  If you have a nice process running in
the background, pagezero may sit in the run queue for ages and add one to
the loadav, and thereby affecting other scheduling decisions.
2002-07-19 21:06:01 +00:00
Alan Cox
eeeaf0fdd1 o Duplicate an odd side-effect of vm_page_wire() in vm_page_allocate()
when VM_ALLOC_WIRED is specified: set the PG_MAPPED bit in flags.
 o In both vm_page_wire() and vm_page_allocate() add a comment saying
   that setting PG_MAPPED does not belong there.
2002-07-19 03:33:04 +00:00
Alan Cox
f23050633f o Remove the acquisition and release of Giant from the idle priority thread
that pre-zeroes free pages.
 o Remove GIANT_REQUIRED from some low-level page queue functions.  (Instead
   assertions on the page queue lock are being added to the higher-level
   functions, like vm_page_wire(), etc.)

In collaboration with:	peter
2002-07-18 17:40:07 +00:00
Mark Murray
f6e34b823a Void functions cannot return values. 2002-07-18 15:53:11 +00:00
Peter Wemm
9e7c1bce60 (VM_MAX_KERNEL_ADDRESS - KERNBASE) / PAGE_SIZE may not fit in an integer.
Use lmin(long, long), not min(u_int, u_int).  This is a problem here on
ia64 which has *way* more than 2^32 pages of KVA.  281474976710655 pages
to be precice.
2002-07-18 10:28:00 +00:00
Alan Cox
827b2fa091 o Introduce an argument, VM_ALLOC_WIRED, that requests vm_page_alloc()
to return a wired page.
 o Use VM_ALLOC_WIRED within Alpha's pmap_growkernel().  Also, because
   Alpha's pmap_growkernel() calls vm_page_alloc() from within a critical
   section, specify VM_ALLOC_INTERRUPT instead of VM_ALLOC_SYSTEM.  (Only
   VM_ALLOC_INTERRUPT is implemented entirely with a spin mutex.)
 o Assert that the page queues mutex is held in vm_page_wire()
   on Alpha, just like the other platforms.
2002-07-18 04:08:10 +00:00
Alan Cox
072e9cbb50 o Use vm_pageq_remove_nowakeup() and vm_pageq_enqueue() in
vm_page_zero_idle() instead of partially duplicated implementations.
   In particular, this change guarantees that the number of free pages
   in the free queue(s) matches the global free page count when Giant
   is released.

Submitted by:	peter (via his p4 "pmap" branch)
2002-07-16 19:39:40 +00:00
Alan Cox
5c8cdc0e2a o Create vm_contig_launder() to replace code that appears twice
in contigmalloc1().
2002-07-15 06:33:31 +00:00
Alan Cox
8b8b8202f9 o Lock page queue accesses by vm_page_wire() that aren't
within a critical section.
 o Assert that the page queues lock is held in vm_page_wire()
   unless an Alpha.
2002-07-14 23:51:55 +00:00
Alan Cox
e16cfdbea4 o Lock page queue accesses by vm_page_wire(). 2002-07-14 19:36:15 +00:00
Alan Cox
eed6f3fd45 o Lock page queue accesses by vm_page_unmanage().
o Assert that the page queues lock is held in vm_page_unmanage().
2002-07-13 23:55:30 +00:00
Alan Cox
1f54526952 o Complete the locking of page queue accesses by vm_page_unwire().
o Assert that the page queues lock is held in vm_page_unwire().
 o Make vm_page_lock_queues() and vm_page_unlock_queues() visible
   to kernel loadable modules.
2002-07-13 20:55:21 +00:00
Alan Cox
2d09a6ad97 o Lock some page queue accesses, in particular, those by vm_page_unwire(). 2002-07-13 19:24:04 +00:00
Alan Cox
93bc4879e6 o Assert GIANT_REQUIRED on system maps in _vm_map_lock(),
_vm_map_lock_read(), and _vm_map_trylock().  Submitted by: tegge
 o Remove GIANT_REQUIRED from kmem_alloc_wait() and kmem_free_wakeup().
   (This clears the way for exec_map accesses to move outside of Giant.
   The exec_map is not a system map.)
 o Remove some premature MPSAFE comments.

Reviewed by:	tegge
2002-07-12 23:20:06 +00:00
Matthew Dillon
fbcf77c2ea Re-enable the idle page-zeroing code. Remove all IPIs from the idle
page-zeroing code as well as from the general page-zeroing code and use a
lazy tlb page invalidation scheme based on a callback made at the end
of mi_switch.

A number of people came up with this idea at the same time so credit
belongs to Peter, John, and Jake as well.

Two-way SMP buildworld -j 5 tests (second run, after stabilization)
    2282.76 real  2515.17 user  704.22 sys	before peter's IPI commit
    2266.69 real  2467.50 user  633.77 sys	after peter's commit
    2232.80 real  2468.99 user  615.89 sys	after this commit

Reviewed by:	peter, jhb
Approved by:	peter
2002-07-12 20:17:06 +00:00
Peter Wemm
a7e9138e37 Avoid a vm_page_lookup() - that uses a spinlock protected hash. We can
just use the object's memq for our nefarious purposes.
2002-07-12 04:38:51 +00:00
Alan Cox
7538e5500d o Lock some (unfortunately, not yet all) accesses to the page queues. 2002-07-12 03:17:22 +00:00
Alan Cox
60e15726af o Lock accesses to the page queues. 2002-07-12 02:55:55 +00:00
Alan Cox
9688f93163 o Add a "needs wakeup" flag to the vm_map for use by kmem_alloc_wait()
and kmem_free_wakeup().  Previously, kmem_free_wakeup() always
   called wakeup().  In general, no one was sleeping.
 o Export vm_map_unlock_and_wait() and vm_map_wakeup() from vm_map.c
   for use in vm_kern.c.
2002-07-11 02:39:24 +00:00
Alan Cox
56030358cb o Lock accesses to the page queues in vm_object_terminate().
o Eliminate some unnecessary 64-bit arithmetic in vm_object_split().
2002-07-09 18:02:03 +00:00
Peter Wemm
5e13bcd6c4 vm_page_queue_free_mtx is a spin mutex, not a normal sleep mutex.
I do not know why this didn't panic my box, but I have most certainly
been using it:
peter@overcee[3:14pm]~src/sys/i386/i386-110> sysctl -a | grep zero
vm.stats.misc.zero_page_count: 2235
vm.stats.misc.cnt_prezero: 638951
vm.idlezero_enable: 1
vm.idlezero_maxrun: 16

Submitted by:	Tor.Egge@cvsup.no.freebsd.org
Approved by:	Tor's patches are never wrong. :-)
2002-07-08 23:12:37 +00:00
Peter Wemm
b428c5fd23 Turn the zeroidle process off for SMP systems, there is still a possible
TLB problem when bouncing from one cpu to another (the original cpu will
not have purged its TLB if the it simply went idle).

Pointed out by:	 Tor.Egge@cvsup.no.freebsd.org
Approved by:	Tor is never wrong. :-)
2002-07-08 23:09:11 +00:00
Peter Wemm
a58b3a6878 Add a special page zero entry point intended to be called via the single
threaded VM pagezero kthread outside of Giant.  For some platforms, this
is really easy since it can just use the direct mapped region.  For others,
IPI sending is involved or there are other issues, so grab Giant when
needed.

We still have preemption issues to deal with, but Alan Cox has an
interesting suggestion on how to minimize the problem on x86.

Use Luigi's hack for preserving the (lack of) priority.

Turn the idle zeroing back on since it can now actually do something useful
outside of Giant in many cases.
2002-07-08 04:24:26 +00:00
Peter Wemm
f59685a4b7 Avoid vm_page_lookup() [grabs a spinlock] and just process the upage
object memq instead.

Suggested by:	alc
2002-07-08 01:11:10 +00:00
Peter Wemm
a136efe9b6 Collect all the (now equivalent) pmap_new_proc/pmap_dispose_proc/
pmap_swapin_proc/pmap_swapout_proc functions from the MD pmap code
and use a single equivalent MI version.  There are other cleanups
needed still.

While here, use the UMA zone hooks to keep a cache of preinitialized
proc structures handy, just like the thread system does.  This eliminates
one dependency on 'struct proc' being persistent even after being freed.
There are some comments about things that can be factored out into
ctor/dtor functions if it is worth it.  For now they are mostly just
doing statistics to get a feel of how it is working.
2002-07-07 23:05:27 +00:00
Alan Cox
25524d3eba o Lock accesses to the free queue(s) in vm_page_zero_idle(). 2002-07-07 19:27:57 +00:00
Alan Cox
c7118ed61b o Traverse the object's memq rather than repeatedly calling vm_page_lookup()
in vm_object_split().
2002-07-07 06:01:25 +00:00
Jeff Roberson
f6b5b182e8 - Hold a lock on the vnode acquired from the file table across the call to
vm_mmap() as well as the GETATTR etc.
 - If the handle is a vnode in vm_mmap() assert that it is locked.
 - Wiggle Giant around a little to account for the extra vnode operation.
2002-07-06 22:14:38 +00:00
Andrew Gallatin
f784043a9f Remove bogus vm_page_wakeup() in vm_page_cowfault() that will cause panics
in the zero-copy send path if a process attempts to write to a page
which is still in flight.

reviewed by: ken
2002-07-05 23:33:27 +00:00
Jeff Roberson
17b9cc4941 Fix a lock order reversal in uma_zdestroy. The uma_mtx needs to be held across
calls to zone_drain().

Noticed by:	scottl
2002-07-05 21:39:52 +00:00
Alan Cox
21f1b5331f o Lock accesses to the free page queues in contigmalloc1(). 2002-07-05 06:43:32 +00:00
Jeff Roberson
f5118d6aaf Remove unnecessary includes. 2002-07-05 05:16:19 +00:00
Alan Cox
70c1763634 o Resurrect vm_page_lock_queues(), vm_page_unlock_queues(), and the free
queue lock (revision 1.33 of vm/vm_page.c removed them).
 o Make the free queue lock a spin lock because it's sometimes acquired
   inside of a critical section.
2002-07-04 22:07:37 +00:00
Julian Elischer
8108a14544 A small cleanup. 2002-07-04 12:37:13 +00:00
Julian Elischer
a30ec8f8b8 Don;t call teh thread setup routines from here..
they are already called when uma calls thread_init()
2002-07-04 12:31:54 +00:00
Alan Cox
22a97b04de o Make the reservation of KVA space for kernel map entries a function
of the KVA space's size in addition to the amount of physical memory
   and reduce it by a factor of two.

Under the old formula, our reservation amounted to one kernel map entry
per virtual page in the KVA space on a 4GB i386.
2002-07-03 19:16:37 +00:00
Jeff Roberson
e221e841b0 Actually use the fini callback.
Pointy hat to:	me :-(
Noticed By:	Julian
2002-07-03 00:30:51 +00:00
Robert Drehmel
47e151dd7a - Use (OFF_TO_IDX(off) - pi) instead of (OFF_TO_IDX(off - IDX_TO_OFF(pi))).
- Reformat a comment.
2002-07-01 14:14:07 +00:00
Alan Cox
c2eda4b565 o Remove some long dead code: from revision 1.41 of vm/vm_pager.c
3+ years ago.
 o Remove some unused prototypes.
2002-07-01 02:38:05 +00:00
Ian Dowse
300b96aca2 Change the type of `tscan' in vm_object_page_clean() to vm_pindex_t,
as it stores an absolute page index that may not fit in a vm_offset_t.
2002-06-29 20:04:38 +00:00
Julian Elischer
e602ba25fd Part 1 of KSE-III
The ability to schedule multiple threads per process
(one one cpu) by making ALL system calls optionally asynchronous.
to come: ia64 and power-pc patches, patches for gdb, test program (in tools)

Reviewed by:	Almost everyone who counts
	(at various times, peter, jhb, matt, alfred, mini, bernd,
	and a cast of thousands)

	NOTE: this is still Beta code, and contains lots of debugging stuff.
	expect slight instability in signals..
2002-06-29 17:26:22 +00:00
Ian Dowse
23f09d50bb Avoid using the 64-bit vm_pindex_t in a few places where 64-bit
types are not required, as the overhead is unnecessary:

 o In the i386 pmap_protect(), `sindex' and `eindex' represent page
   indices within the 32-bit virtual address space.
 o In swp_pager_meta_build() and swp_pager_meta_ctl(), use a temporary
   variable to store the low few bits of a vm_pindex_t that gets used
   as an array index.
 o vm_uiomove() uses `osize' and `idx' for page offsets within a
   map entry.
 o In vm_object_split(), `idx' is a page offset within a map entry.
2002-06-26 20:32:51 +00:00
Ian Dowse
5125fe4f45 Use an explicit cast to avoid relying on sign extension to do the
right thing in code such as `vm_pindex_t x = ~SWAP_META_MASK'.

Reviewed by:	dillon
2002-06-26 19:18:14 +00:00