Commit Graph

82 Commits

Author SHA1 Message Date
Poul-Henning Kamp
f0d45e6aae Fix contigmalloc() and contigmalloc1() arguments. 1997-10-10 18:18:47 +00:00
Peter Wemm
f8ddc1e209 Print correct function name in panics 1997-09-13 15:04:52 +00:00
Bruce Evans
79624e2147 Removed unused #includes. 1997-09-01 03:17:34 +00:00
John Dyson
3075778b63 Get rid of the ad-hoc memory allocator for vm_map_entries, in lieu of
a simple, clean zone type allocator.  This new allocator will also be
used for machine dependent pmap PV entries.
1997-08-05 00:02:08 +00:00
John Dyson
6160099735 Check the correct queue for waking up the pageout daemon. Specifically,
the pageout daemon wasn't always being waken up appropriately when the
(cache + free) queues were depleted.
Submitted by:	David S. Miller <davem@jenolan.rutgers.edu>
1997-05-01 14:36:01 +00:00
John Dyson
c5d593ae63 Fix a significant error in the accounting for pre-zeroed pages. This
is a candidate for RELENG_2_2...
1997-03-23 02:44:54 +00:00
Peter Wemm
6875d25465 Back out part 1 of the MCFH that changed $Id$ to $FreeBSD$. We are not
ready for it yet.
1997-02-22 09:48:43 +00:00
Garrett Wollman
10825343b0 Provide an alternative interface to contigmalloc() which allows a specific
map to be used when allocating the kernel va (e.g., mb_map).  The VM
gurus may want to look this over.
1997-02-13 19:37:40 +00:00
John Dyson
996c772f58 This is the kernel Lite/2 commit. There are some requisite userland
changes, so don't expect to be able to run the kernel as-is (very well)
without the appropriate Lite/2 userland changes.

The system boots and can mount UFS filesystems.

Untested: ext2fs, msdosfs, NFS
Known problems: Incorrect Berkeley ID strings in some files.
		Mount_std mounts will not work until the getfsent
		library routine is changed.

Reviewed by:	various people
Submitted by:	Jeffery Hsu <hsu@freebsd.org>
1997-02-10 02:22:35 +00:00
Jordan K. Hubbard
1130b656e5 Make the long-awaited change from $Id$ to $FreeBSD$
This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore.  This update would have been
insane otherwise.
1997-01-14 07:20:47 +00:00
John Dyson
e0c5a895f1 Make the kernel smaller with at worst a neutral effect on perf by
de-inlining some VM calls.  (Actually, I measured a small improvement.)
1996-11-28 23:15:07 +00:00
John Dyson
5b0a74089d Improve the locality of reference for variables in vm_page and
vm_kern by moving them from .bss to .data.  With this change,
there is a measurable perf improvement in fork/exec.
1996-11-17 02:38:31 +00:00
John Dyson
db2c0faa4c Vastly improved contigmalloc routine. It does not solve the
problem of allocating contiguous buffer memory in general, but
make it much more likely to work at boot-up time.  The best
chance for an LKM-type load of a sound driver is immediately
after the mount of the root filesystem.

This appears to work for a 64K allocation on an 8MB system.
1996-11-05 04:19:08 +00:00
John Dyson
675878e732 Move much of the machine dependent code from vm_glue.c into
pmap.c.  Along with the improved organization, small proc fork
performance is now about 5%-10% faster.
1996-10-15 03:16:45 +00:00
Bruce Evans
8ba0c490a9 Removed __pure's and __pure2's. __pure is a no-op for recent versions
of gcc by definition, and __pure2 is a no-op in effect (presumably the
compiler can see when an inline function has no side effects).
1996-10-12 20:09:48 +00:00
John Dyson
f7d6dab2fd Fix a problem with the page coloring code that the system will not always
be able to use all of the free pages.  This can manifest as a panic
using DIAGNOSTIC, or as a panic on an indirect memory reference.
1996-10-06 18:27:39 +00:00
Bruce Evans
322dfc2bd5 Fixed undeclared variables for the !(PQ_L2_SIZE > 1) case.
Removed redundant #include.
1996-09-28 17:53:18 +00:00
John Dyson
a2f4a84696 Reviewed by:
Submitted by:
Obtained from:
1996-09-28 03:33:40 +00:00
Bruce Evans
c7c34a24a3 Attached vm ddb commands show map', show vmochk', `show object',
`show vmopag', `show page' and `show pageq'.  Moved all vm ddb stuff
to the ends of the vm source files.

Changed printf() to db_printf(), `indent' to db_indent, and iprintf()
to db_iprintf() in ddb commands.  Moved db_indent and db_iprintf()
from vm to ddb.

vm_page.c:
Don't use __pure.  Staticized.

db_output.c:
Reduced page width from 80 to 79 to inhibit double spacing for long
lines (there are still some problems if words are printed across
column 79).
1996-09-14 11:54:59 +00:00
John Dyson
5070c7f8c5 Addition of page coloring support. Various levels of coloring are afforded.
The default level works with minimal overhead, but one can also enable
full, efficient use of a 512K cache.  (Parameters can be generated
to support arbitrary cache sizes also.)
1996-09-08 20:44:49 +00:00
John Dyson
67bf686897 Backed out the recent changes/enhancements to the VM code. The
problem with the 'shell scripts' was found, but there was a 'strange'
problem found with a 486 laptop that we could not find.  This commit
backs the code back to 25-jul, and will be re-entered after the snapshot
in smaller (more easily tested) chunks.
1996-07-30 03:08:57 +00:00
John Dyson
4f4d35edf0 This commit is meant to solve a couple of VM system problems or
performance issues.

	1) The pmap module has had too many inlines, and so the
	   object file is simply bigger than it needs to be.
	   Some common code is also merged into subroutines.
	2) Removal of some *evil* PHYS_TO_VM_PAGE macro calls.
	   Unfortunately, a few have needed to be added also.
	   The removal caused the need for more vm_page_lookups.
	   I added lookup hints to minimize the need for the
	   page table lookup operations.
	3) Removal of some bogus performance improvements, that
	   mostly made the code more complex (tracking individual
	   page table page updates unnecessarily).  Those improvements
	   actually hurt 386 processors perf (not that people who
	   worry about perf use 386 processors anymore :-)).
	4) Changed pv queue manipulations/structures to be TAILQ's.
	5) The pv queue code has had some performance problems since
	   day one.  Some significant scalability issues are resolved
	   by threading the pv entries from the pmap AND the physical
	   address instead of just the physical address.  This makes
	   certain pmap operations run much faster.  This does
	   not affect most micro-benchmarks, but should help loaded system
	   performance *significantly*.  DG helped and came up with most
	   of the solution for this one.
	6) Most if not all pmap bit operations follow the pattern:
		pmap_test_bit();
		pmap_clear_bit();
	   That made for twice the necessary pv list traversal.   The
	   pmap interface now supports only pmap_tc_bit type operations:
	   pmap_[test/clear]_modified, pmap_[test/clear]_referenced.
	   Additionally, the modified routine now takes a vm_page_t arg
	   instead of a phys address.  This eliminates a PHYS_TO_VM_PAGE
	   operation.
	7) Several rewrites of routines that contain redundant code to
	   use common routines, so that there is a greater likelihood of
	   keeping the cache footprint smaller.
1996-07-27 03:24:10 +00:00
John Dyson
38efa82b23 This commit does a couple of things:
Re-enables the RSS limiting, and the routine is now tail-recursive,
	making it much more safe (eliminates the possiblity of kernel stack
	overflow.) Also, the RSS limiting is a little more intelligent about
	finding the likely objects that are pushing the process over the limit.

	Added some sysctls that help with VM system tuning.

New sysctl features:
	1)	Enable/disable lru pageout algorithm.
		vm.pageout_algorithm = 0, default algorithm that works
			well, especially using X windows and heavy
			memory loading.  Can have adverse effects,
			sometimes slowing down program loading.

		vm.pageout_algorithm = 1, close to true LRU.  Works much
			better than clock, etc.  Does not work as well as
			the default algorithm in general.  Certain memory
			"malloc" type benchmarks work a little better with
			this setting.

		Please give me feedback on the performance results
		associated with these.

	2)	Enable/disable swapping.
		vm.swapping_enabled = 1, default.

		vm.swapping_enabled = 0, useful for cases where swapping
			degrades performance.

		The config option "NO_SWAPPING" is still operative, and
		takes precedence over the sysctl.  If "NO_SWAPPING" is
		specified, the sysctl still exists, but "vm.swapping_enabled"
		is hard-wired to "0".

Each of these can be changed "on the fly."
1996-06-26 05:39:27 +00:00
John Dyson
2a4eb04bfd Improve algorithm for page hash queue. It was previously about
as bad as it could be.  This algorithm appears to improve fork
performance (barely) measurably.
1996-06-21 05:39:22 +00:00
John Dyson
ef743ce6ed Several bugfixes/improvements:
1) Make it much less likely to miss a wakeup in vm_page_free_wakeup
	2) Create a new entry point into pmap: pmap_ts_referenced, eliminates
	   the need to scan the pv lists twice in many cases.  Perhaps there
	   is alot more to do here to work on minimizing pv list manipulation
	3) Minor improvements to vm_pageout including the use of pmap_ts_ref.
	4) Major changes and code improvement to pmap.  This code has had
	   several serious bugs in page table page manipulation.  In order
	   to simplify the problem, and hopefully solve it for once and all,
	   page table pages are no longer "managed" with the pv list stuff.
	   Page table pages are only (mapped and held/wired) or
	   (free and unused) now.  Page table pages are never inactive,
	   active or cached.  These changes have probably fixed the
	   hold count problems, but if they haven't, then the code is
	   simpler anyway for future bugfixing.
	5) The pmap code has been sorely in need of re-organization, and I
	   have taken a first (of probably many) steps.  Please tell me
	   if you have any ideas.
1996-06-17 03:35:40 +00:00
John Dyson
b5b40fa62b Various bugfixes/cleanups from me and others:
1) Remove potential race conditions on waking up in vm_page_free_wakeup
   by making sure that it is at splvm().
2) Fix another bug in vm_map_simplify_entry.
3) Be more complete about converting from default to swap pager
   when an object grows to be large enough that there can be
   a problem with data structure allocation under low memory
   conditions.
4) Make some madvise code more efficient.
5) Added some comments.
1996-06-16 20:37:31 +00:00
John Dyson
419702a468 Fix a very significant cnt.v_wire_count leak in vm_page.c, and some
minor leaks in pmap.c.  Bruce Evans made me aware of this problem.
1996-06-12 06:52:12 +00:00
John Dyson
886d3e1150 Adjust the threshold for blocking on movement of pages from the cache
queue in vm_fault.

Move the PG_BUSY in vm_fault to the correct place.

Remove redundant/unnecessary code in pmap.c.

Properly block on rundown of page table pages, if they are busy.

I think that the VM system is in pretty good shape now, and the following
individuals (among others, in no particular order) have helped with this
recent bunch of bugs, thanks!  If I left anyone out, I apologize!

Stephen McKay, Stephen Hocking, Eric J. Chet, Dan O'Brien, James Raynard,
Marc Fournier.
1996-06-08 06:48:35 +00:00
John Dyson
6b6f000870 Keep page-table pages from ever being sensed as dirty. This should fix
some problems with the page-table page management code, since it can't
deal with the notion of page-table pages being paged out or in transit.
Also, clean up some stylistic issues per some suggestions from
Stephen McKay.
1996-06-05 03:31:49 +00:00
John Dyson
f35329ac0f This commit is dual-purpose, to fix more of the pageout daemon
queue corruption problems, and to apply Gary Palmer's code cleanups.
David Greenman helped with these problems also.  There is still
a hang problem using X in small memory machines.
1996-05-31 00:38:04 +00:00
John Dyson
f777ab7b8b Add an assert to vm_page_cache. We should never cache a dirty page. 1996-05-24 05:20:15 +00:00
John Dyson
b18bfc3da7 This set of commits to the VM system does the following, and contain
contributions or ideas from Stephen McKay <syssgm@devetir.qld.gov.au>,
Alan Cox <alc@cs.rice.edu>, David Greenman <davidg@freebsd.org> and me:

	More usage of the TAILQ macros.  Additional minor fix to queue.h.
	Performance enhancements to the pageout daemon.
		Addition of a wait in the case that the pageout daemon
		has to run immediately.
		Slightly modify the pageout algorithm.
	Significant revamp of the pmap/fork code:
		1) PTE's and UPAGES's are NO LONGER in the process's map.
		2) PTE's and UPAGES's reside in their own objects.
		3) TOTAL elimination of recursive page table pagefaults.
		4) The page directory now resides in the PTE object.
		5) Implemented pmap_copy, thereby speeding up fork time.
		6) Changed the pv entries so that the head is a pointer
		   and not an entire entry.
		7) Significant cleanup of pmap_protect, and pmap_remove.
		8) Removed significant amounts of machine dependent
		   fork code from vm_glue.  Pushed much of that code into
		   the machine dependent pmap module.
		9) Support more completely the reuse of already zeroed
		   pages (Page table pages and page directories) as being
		   already zeroed.
	Performance and code cleanups in vm_map:
		1) Improved and simplified allocation of map entries.
		2) Improved vm_map_copy code.
		3) Corrected some minor problems in the simplify code.
	Implemented splvm (combo of splbio and splimp.)  The VM code now
		seldom uses splhigh.
	Improved the speed of and simplified kmem_malloc.
	Minor mod to vm_fault to avoid using pre-zeroed pages in the case
		of objects with backing objects along with the already
		existant condition of having a vnode.  (If there is a backing
		object, there will likely be a COW...  With a COW, it isn't
		necessary to start with a pre-zeroed page.)
	Minor reorg of source to perhaps improve locality of ref.
1996-05-18 03:38:05 +00:00
John Dyson
30dcfc09f2 VM performance improvements, and reorder some operations in VM fault
in anticipation of a fix in pmap that will allow the mlock system call to work
without panicing the system.
1996-03-28 04:53:28 +00:00
John Dyson
9ee58740bc Modify a threshold for waking up the pageout daemon. Also, add a consistancy
check for making sure that held pages aren't freed (DG).
1996-03-09 06:56:39 +00:00
John Dyson
de5f6a7765 1) Eliminate unnecessary bzero of UPAGES.
2) Eliminate unnecessary copying of pages during/after forks.
3) Add user map simplification.
1996-03-02 02:54:24 +00:00
Bruce Evans
324e9ed2a4 Added a `boundary' arg to vm_alloc_page_contig(). Previously the only
way to avoid crossing a 64K DMA boundary was to specify an alignment
greater than the size even when the alignment didn't matter, and for
sizes larger than a page, this reduced the chance of finding enough
contiguous pages.  E.g., allocations of 8K not crossing a 64K boundary
previously had to be allocated on 8K boundaries; now they can be
allocated on any 4K boundary except (64 * n + 60)K.

Fixed bugs in vm_alloc_page_contig():
- the last page wasn't allocated for sizes smaller than a page.
- failures of kmem_alloc_pageable() weren't handled.

Mutated vm_page_alloc_contig() to create a more convenient interface
named contigmalloc().  This is the same as the one in 1.1.5 except
it has `low' and `high' args, and the `alignment' and `boundary'
args are multipliers instead of masks.
1996-01-27 00:13:33 +00:00
John Dyson
bd7e5f992e Eliminated many redundant vm_map_lookup operations for vm_mmap.
Speed up for vfs_bio -- addition of a routine bqrelse to greatly diminish
	overhead for merged cache.
Efficiency improvement for vfs_cluster.  It used to do alot of redundant
	calls to cluster_rbuild.
Correct the ordering for vrele of .text and release of credentials.
Use the selective tlb update for 486/586/P6.
Numerous fixes to the size of objects allocated for files.  Additionally,
	fixes in the various pagers.
Fixes for proper positioning of vnode_pager_setsize in msdosfs and ext2fs.
Fixes in the swap pager for exhausted resources.  The pageout code
	will not as readily thrash.
Change the page queue flags (PG_ACTIVE, PG_INACTIVE, PG_FREE, PG_CACHE) into
	page queue indices (PQ_ACTIVE, PQ_INACTIVE, PQ_FREE, PQ_CACHE),
	thereby improving efficiency of several routines.
Eliminate even more unnecessary vm_page_protect operations.
Significantly speed up process forks.
Make vm_object_page_clean more efficient, thereby eliminating the pause
	that happens every 30seconds.
Make sequential clustered writes B_ASYNC instead of B_DELWRI even in the
	case of filesystems mounted async.
Fix a panic with busy pages when write clustering is done for non-VMIO
	buffers.
1996-01-19 04:00:31 +00:00
Garrett Wollman
0e41ee3037 Convert DDB to new-style option. 1996-01-04 21:13:23 +00:00
Bruce Evans
f2c6b65bab Fixed 1TB filesize changes. Some pindexes had bogus names and types
but worked because vm_pindex_t is indistinuishable from vm_offset_t.
1995-12-17 07:19:58 +00:00
Poul-Henning Kamp
f708ef1b9e Another mega commit to staticize things. 1995-12-14 09:55:16 +00:00
John Dyson
ec07c60c6a Some DIAGNOSTIC code was enabled all of the time in error. The
diagnostic code is now conditional on #ifdef DIAGNOSTIC again.
1995-12-11 15:41:50 +00:00
John Dyson
a316d390bd Changes to support 1Tb filesizes. Pages are now named by an
(object,index) pair instead of (object,offset) pair.
1995-12-11 04:58:34 +00:00
David Greenman
efeaf95a41 Untangled the vm.h include file spaghetti. 1995-12-07 12:48:31 +00:00
Bruce Evans
cac597e4c0 Completed function declarations and/or added prototypes.
Staticized some functions.

__purified some functions.  Some functions were bogusly declared as
returning `const'.  This hasn't done anything since gcc-2.5.  For
later versions of gcc, the equivalent is __attribute__((const)) at
the end of function declarations.
1995-12-03 12:18:39 +00:00
Poul-Henning Kamp
3af768903d Remove unused vars & funcs, make things static, protoize a little bit. 1995-11-20 12:20:02 +00:00
John Dyson
a91c5a7ecd Get rid of machine-dependent NBPG and replace with PAGE_SIZE. 1995-10-23 05:35:48 +00:00
John Dyson
f70f05f238 Machine independent changes to support pre-zeroed free pages. This
significantly improves demand-zero performance.
1995-09-03 20:40:43 +00:00
John Dyson
4589a4b5dc New subroutine "vm_page_set_validclean" for a vfs_bio improvement. 1995-09-03 19:57:25 +00:00
David Greenman
b367ddb191 #if 0'd one of the DIAGNOSTIC checks in vm_page_alloc(). It was too
expensive for "normal" use.
1995-07-20 05:28:07 +00:00
David Greenman
24a1cce34f NOTE: libkvm, w, ps, 'top', and any other utility which depends on struct
proc or any VM system structure will have to be rebuilt!!!

Much needed overhaul of the VM system. Included in this first round of
changes:

1) Improved pager interfaces: init, alloc, dealloc, getpages, putpages,
   haspage, and sync operations are supported. The haspage interface now
   provides information about clusterability. All pager routines now take
   struct vm_object's instead of "pagers".

2) Improved data structures. In the previous paradigm, there is constant
   confusion caused by pagers being both a data structure ("allocate a
   pager") and a collection of routines. The idea of a pager structure has
   escentially been eliminated. Objects now have types, and this type is
   used to index the appropriate pager. In most cases, items in the pager
   structure were duplicated in the object data structure and thus were
   unnecessary. In the few cases that remained, a un_pager structure union
   was created in the object to contain these items.

3) Because of the cleanup of #1 & #2, a lot of unnecessary layering can now
   be removed. For instance, vm_object_enter(), vm_object_lookup(),
   vm_object_remove(), and the associated object hash list were some of the
   things that were removed.

4) simple_lock's removed. Discussion with several people reveals that the
   SMP locking primitives used in the VM system aren't likely the mechanism
   that we'll be adopting. Even if it were, the locking that was in the code
   was very inadequate and would have to be mostly re-done anyway. The
   locking in a uni-processor kernel was a no-op but went a long way toward
   making the code difficult to read and debug.

5) Places that attempted to kludge-up the fact that we don't have kernel
   thread support have been fixed to reflect the reality that we are really
   dealing with processes, not threads. The VM system didn't have complete
   thread support, so the comments and mis-named routines were just wrong.
   We now use tsleep and wakeup directly in the lock routines, for instance.

6) Where appropriate, the pagers have been improved, especially in the
   pager_alloc routines. Most of the pager_allocs have been rewritten and
   are now faster and easier to maintain.

7) The pagedaemon pageout clustering algorithm has been rewritten and
   now tries harder to output an even number of pages before and after
   the requested page. This is sort of the reverse of the ideal pagein
   algorithm and should provide better overall performance.

8) Unnecessary (incorrect) casts to caddr_t in calls to tsleep & wakeup
   have been removed. Some other unnecessary casts have also been removed.

9) Some almost useless debugging code removed.

10) Terminology of shadow objects vs. backing objects straightened out.
    The fact that the vm_object data structure escentially had this
    backwards really confused things. The use of "shadow" and "backing
    object" throughout the code is now internally consistent and correct
    in the Mach terminology.

11) Several minor bug fixes, including one in the vm daemon that caused
    0 RSS objects to not get purged as intended.

12) A "default pager" has now been created which cleans up the transition
    of objects to the "swap" type. The previous checks throughout the code
    for swp->pg_data != NULL were really ugly. This change also provides
    the rudiments for future backing of "anonymous" memory by something
    other than the swap pager (via the vnode pager, for example), and it
    allows the decision about which of these pagers to use to be made
    dynamically (although will need some additional decision code to do
    this, of course).

13) (dyson) MAP_COPY has been deprecated and the corresponding "copy
    object" code has been removed. MAP_COPY was undocumented and non-
    standard. It was furthermore broken in several ways which caused its
    behavior to degrade to MAP_PRIVATE. Binaries that use MAP_COPY will
    continue to work correctly, but via the slightly different semantics
    of MAP_PRIVATE.

14) (dyson) Sharing maps have been removed. It's marginal usefulness in a
    threads design can be worked around in other ways. Both #12 and #13
    were done to simplify the code and improve readability and maintain-
    ability. (As were most all of these changes)

TODO:

1) Rewrite most of the vnode pager to use VOP_GETPAGES/PUTPAGES. Doing
   this will reduce the vnode pager to a mere fraction of its current size.

2) Rewrite vm_fault and the swap/vnode pagers to use the clustering
   information provided by the new haspage pager interface. This will
   substantially reduce the overhead by eliminating a large number of
   VOP_BMAP() calls. The VOP_BMAP() filesystem interface should be
   improved to provide both a "behind" and "ahead" indication of
   contiguousness.

3) Implement the extended features of pager_haspage in swap_pager_haspage().
   It currently just says 0 pages ahead/behind.

4) Re-implement the swap device (swstrategy) in a more elegant way, perhaps
   via a much more general mechanism that could also be used for disk
   striping of regular filesystems.

5) Do something to improve the architecture of vm_object_collapse(). The
   fact that it makes calls into the swap pager and knows too much about
   how the swap pager operates really bothers me. It also doesn't allow
   for collapsing of non-swap pager objects ("unnamed" objects backed by
   other pagers).
1995-07-13 08:48:48 +00:00