Commit Graph

129 Commits

Author SHA1 Message Date
phk
7e26ca1d1a Divorce "dev_t" from the "major|minor" bitmap, which is now called
udev_t in the kernel but still called dev_t in userland.

Provide functions to manipulate both types:
        major()         umajor()
        minor()         uminor()
        makedev()       umakedev()
        dev2udev()      udev2dev()

For now they're functions, they will become in-line functions
after one of the next two steps in this process.

Return major/minor/makedev to macro-hood for userland.

Register a name in cdevsw[] for the "filedescriptor" driver.

In the kernel the udev_t appears in places where we have the
major/minor number combination, (ie: a potential device: we
may not have the driver nor the device), like in inodes, vattr,
cdevsw registration and so on, whereas the dev_t appears where
we carry around a reference to a actual device.

In the future the cdevsw and the aliased-from vnode will be hung
directly from the dev_t, along with up to two softc pointers for
the device driver and a few houskeeping bits.  This will essentially
replace the current "alias" check code (same buck, bigger bang).

A little stunt has been provided to try to catch places where the
wrong type is being used (dev_t vs udev_t), if you see something
not working, #undef DEVT_FASCIST in kern/kern_conf.c and see if
it makes a difference.  If it does, please try to track it down
(many hands make light work) or at least try to reproduce it
as simply as possible, and describe how to do that.

Without DEVT_FASCIST I belive this patch is a no-op.

Stylistic/posixoid comments about the userland view of the <sys/*.h>
files welcome now, from userland they now contain the end result.

Next planned step: make all dev_t's refer to the same devsw[] which
means convert BLK's to CHR's at the perimeter of the vnodes and
other places where they enter the game (bootdev, mknod, sysctl).
1999-05-11 19:55:07 +00:00
phk
f57a01ebfc remove b_proc from struct buf, it's (now) unused.
Reviewed by:	dillon, bde
1999-05-06 20:00:34 +00:00
peter
73556bfee1 Add sufficient braces to keep egcs happy about potentially ambiguous
if/else nesting.
1999-05-06 18:13:11 +00:00
alc
7693bdc245 All directory accesses must be made with NFS_DIRBLKSIZE chunks to avoid
confusing the directory read cookie cache.  The nfs_access implementation
for v2 mounts attempts to read from the directory if root is the user
so that root can't access cached files when the server remaps root
to some other user.

Submitted by:	Doug Rabson <dfr@nlsystems.com>
Reviewed by:	Matthew Dillon <dillon@apollo.backplane.com>
1999-05-03 20:59:14 +00:00
alc
5cb08a2652 The VFS/BIO subsystem contained a number of hacks in order to optimize
piecemeal, middle-of-file writes for NFS.  These hacks have caused no
end of trouble, especially when combined with mmap().  I've removed
them.  Instead, NFS will issue a read-before-write to fully
instantiate the struct buf containing the write.  NFS does, however,
optimize piecemeal appends to files.  For most common file operations,
you will not notice the difference.  The sole remaining fragment in
the VFS/BIO system is b_dirtyoff/end, which NFS uses to avoid cache
coherency issues with read-merge-write style operations.  NFS also
optimizes the write-covers-entire-buffer case by avoiding the
read-before-write.  There is quite a bit of room for further
optimization in these areas.

The VM system marks pages fully-valid (AKA vm_page_t->valid =
VM_PAGE_BITS_ALL) in several places, most noteably in vm_fault.  This
is not correct operation.  The vm_pager_get_pages() code is now
responsible for marking VM pages all-valid.  A number of VM helper
routines have been added to aid in zeroing-out the invalid portions of
a VM page prior to the page being marked all-valid.  This operation is
necessary to properly support mmap().  The zeroing occurs most often
when dealing with file-EOF situations.  Several bugs have been fixed
in the NFS subsystem, including bits handling file and directory EOF
situations and buf->b_flags consistancy issues relating to clearing
B_ERROR & B_INVAL, and handling B_DONE.

getblk() and allocbuf() have been rewritten.  B_CACHE operation is now
formally defined in comments and more straightforward in
implementation.  B_CACHE for VMIO buffers is based on the validity of
the backing store.  B_CACHE for non-VMIO buffers is based simply on
whether the buffer is B_INVAL or not (B_CACHE set if B_INVAL clear,
and vise-versa).  biodone() is now responsible for setting B_CACHE
when a successful read completes.  B_CACHE is also set when a bdwrite()
is initiated and when a bwrite() is initiated.  VFS VOP_BWRITE
routines (there are only two - nfs_bwrite() and bwrite()) are now
expected to set B_CACHE.  This means that bowrite() and bawrite() also
set B_CACHE indirectly.

There are a number of places in the code which were previously using
buf->b_bufsize (which is DEV_BSIZE aligned) when they should have
been using buf->b_bcount.  These have been fixed.  getblk() now clears
B_DONE on return because the rest of the system is so bad about
dealing with B_DONE.

Major fixes to NFS/TCP have been made.  A server-side bug could cause
requests to be lost by the server due to nfs_realign() overwriting
other rpc's in the same TCP mbuf chain.  The server's kernel must be
recompiled to get the benefit of the fixes.

Submitted by:	Matthew Dillon <dillon@apollo.backplane.com>
1999-05-02 23:57:16 +00:00
julian
f27c95753f Reviewed by: Many at differnt times in differnt parts,
including alan, john, me, luoqi, and kirk
Submitted by:	Matt Dillon <dillon@frebsd.org>

This change implements a relatively sophisticated fix to getnewbuf().
There were two problems with getnewbuf(). First, the writerecursion
can lead to a system stack overflow when you have NFS and/or VN
devices in the system. Second, the free/dirty buffer accounting was
completely broken. Not only did the nfs routines blow it trying to
manually account for the buffer state, but the accounting that was
done did not work well with the purpose of their existance: figuring
out when getnewbuf() needs to sleep.

The meat of the change is to kern/vfs_bio.c. The remaining diffs are
all minor except for NFS, which includes both the fixes for bp
interaction AND fixes for a 'biodone(): buffer already done' lockup.
Sys/buf.h also contains a chaining structure which is not used by
this patchset but is used by other patches that are coming soon.
This patch deliniated by tags PRE_MAT_GETBUF and POST_MAT_GETBUF.
(sorry for the missing T matt)
1999-03-12 02:24:58 +00:00
dfr
22ceb237f0 * Change sysctl from using linker_set to construct its tree using SLISTs.
This makes it possible to change the sysctl tree at runtime.

* Change KLD to find and register any sysctl nodes contained in the loaded
  file and to unregister them when the file is unloaded.

Reviewed by: Archie Cobbs <archie@whistle.com>,
	Peter Wemm <peter@netplex.com.au> (well they looked at it anyway)
1999-02-16 10:49:55 +00:00
dillon
717990555d General additional cleanup of VOP API for NFS ops - mainly NFS ignoring
the API for freeing up cnp's.  This cleanup should not effect nominal
    operation one way or the other since NFS VOPs just happen to be called
    with flags that match what it actually does to the NAMEI components it
    gets.  Still, if an NFS error occured, there was probably some memory
    leakage of NAMEI components with certain NFS VOP ops.
1999-02-13 09:47:30 +00:00
dillon
ab62b188ab PR: kern/9970
Remove incorrect vput() in nfs_link()
1999-02-13 08:01:59 +00:00
dillon
42bebed095 Flush delayed-write data out prior to issuing a rename rpc. This appears
to fix the problem w/ NFSV3 whereby a make installworld would get into
    high-network-bandwidth situations continuously trying to retry nfs writes
    that fail with a 'stale file handle' error.
1999-02-06 07:48:56 +00:00
dillon
afb6772e77 Fix nasty bug in nfs_access(). A conditional was if (a = b) instead of
if (a == b).
1999-01-27 22:45:49 +00:00
dillon
72e4bdcf94 Fix warnings in preparation for adding -Wall -Wcast-qual to the
kernel compile
1999-01-27 22:45:13 +00:00
dillon
df24433bbe This is a rather large commit that encompasses the new swapper,
changes to the VM system to support the new swapper, VM bug
    fixes, several VM optimizations, and some additional revamping of the
    VM code.  The specific bug fixes will be documented with additional
    forced commits.  This commit is somewhat rough in regards to code
    cleanup issues.

Reviewed by:	"John S. Dyson" <root@dyson.iquest.net>, "David Greenman" <dg@root.com>
1999-01-21 08:29:12 +00:00
eivind
2f82e08ebd Remove two cases of unused variable sp3. 1999-01-12 12:39:14 +00:00
dfr
1cee4d444d Fix for creating files on a Solaris 7 server with NFSv3 (the request was
slightly garbled but older servers seemed to understand it).

Reviewed by: David O'Brien <obrien@nuxi.ucdavis.edu>
1998-12-25 10:34:27 +00:00
msmith
6461f7d9c1 Reimplement the NFS ACCESS RPC cache as an "accelerator" rather than a true
cache.  If the cached result lets us say "yes", then go with that.  If
we're not sure, or we think the answer might be "no", go to the wire to be
certain.    This avoids all of the possible false veto cases, and allows us
to key the cached value with just the UID for which the cached value holds,
reducing the bloat of the nfsnode structure from 104 bytes to just 12 bytes.

Since the "yes" case is by far the most common, this should still provide
a substantial performance improvement.  Also default the cache to on, with
a conservative timeout (2 seconds).  This improves performance if NFS is
loaded as a KLD module, as there's not (yet) code to parse an option out
of the module arguments to set it, and sysctl doesn't work (yet) for OIDs
in modules.

The 'accelerator' mode was suggested by Bjoern Groenvall (bg@sics.se)

Feedback on this would be appreciated as testing has been necessarily
limited by Comdex, and it would be valuable to have this in 2.2.8.
1998-11-15 20:36:18 +00:00
msmith
5a28cf0283 Avoid a null pointer reference if the target of an NFS rename has been
sillrenamed, or if the source vnode doesn't have an associated nfsnode.

Bug report from Andrew Gallatin <gallatin@cs.duke.edu>
1998-11-13 22:58:48 +00:00
msmith
5ffcaa816e Implement NFS ACCESS RPC result caching.
This yields startling performance increases for NFS clients for many
access profiles, due to the fact that ACCESS results are persistently
cached in the namecache in many cases.

Note that the code is somewhat conservative in that it requires an
exact credential match for a cache hit.  This bloats the nfsnode
structure by sizeof(struct ucred) (96 bytes).  Any less conservative
approach opens the possibility for a false veto in eg. setuid
applications.  Alternative suggestions would be welcomed.

The cache is normally disabled, to activate set the sysctl variable
vfs.nfs.access_cache_timeout to a nonzero value.  This is the time in
seconds that a cached entry will be considered valid; useful values appear
to be 2-10 seconds.  Performance of the cache can be monitored with the
vfs.nfs.access_cache_hits and vfs.nfs.access_cache_hits variables.
1998-11-13 02:39:09 +00:00
peter
4dde2ece8b Remove [apparently] bogus casts to u_long for the vnode_pager_setsize()
second argument.  np_size is a 64 bit int, so is the second arg.  This
might have caused needless 2G/4G file size problems.

I believe it was Bruce who queried this.
1998-11-09 07:00:14 +00:00
peter
8ef35acf90 Use TAILQ macros for clean/dirty block list processing. Set b_xflags
rather than abusing the list next pointer with a magic number.
1998-10-31 15:31:29 +00:00
mckusick
79fbc60c6a In nfs_link(), check for a cross-device mount *before* looking
in the v_data field.
Obtained from: Charles Hannum, via Frank van der Linden <frank@wins.uva.nl>
1998-09-29 23:39:37 +00:00
mckusick
74f40b1c41 Missing vput when cross-device link error is detected in nfs_link. 1998-09-29 23:29:48 +00:00
mckusick
17402e8897 During truncation, have to notify the VM about the new size
of the NFS file *before* doing the nfs_vinvalbuf operation.
Otherwise some invalid data may show up in an mmap.
1998-09-29 23:28:32 +00:00
dfr
cb85cf3e66 Protect all modifications to v_numoutput with splbio(). 1998-08-13 08:09:08 +00:00
julian
4363221ba2 VOP_STRATEGY grows an (struct vnode *) argument
as the value in b_vp is often not really what you want.
(and needs to be frobbed). more cleanups will follow this.
Reviewed by: Bruce Evans <bde@freebsd.org>
1998-07-04 20:45:42 +00:00
peter
4178bd0c41 readlink() returns EINVAL rather than EPERM if called on a non-symlink. 1998-06-01 10:59:23 +00:00
peter
19ad2aa63b For the on-the-wire protocol, u_long -> u_int32_t; long -> int32_t;
int -> int32_t; u_short -> u_int16_t.  Also, use mode_t instead of u_short
for storing modes (mode_t is a u_int16_t).

Obtained from: NetBSD
1998-05-31 20:09:01 +00:00
peter
d9c0dc4a94 xdr encode -1 properly.
Obtained from: NetBSD
1998-05-31 19:29:28 +00:00
peter
2a5188e78c Fully fill in nfsv2 write rpc requests rather than leaving garbage.
Obtained from: NetBSD
1998-05-31 19:28:15 +00:00
peter
da4830ce17 Don't silently fail to set file flags.
Obtained from:  NetBSD
1998-05-31 19:24:19 +00:00
peter
a21fad22e0 Don't pass a second copy of the uid/gid in with the v2/v3 sattr structures,
it just makes more work.  We pass a copy of the uid/gid with the
credentials.  (although, this may need to be revisited if a non AUTHUNIX
authentication method (such as NFSKERB) ever gets implemented).

Obtained from:  NetBSD
1998-05-31 19:00:19 +00:00
peter
1f34203061 Don't let the user try "rmdir ."
Obtained from:  NetBSD
1998-05-31 18:30:42 +00:00
peter
f61450c5b0 Don't let the user try and unlink() a directory on a NFS server.
Obtained from:  NetBSD
1998-05-31 18:28:45 +00:00
peter
c45767477f When a write rpc returns an error, break the loop.
Obtained from: NetBSD
1998-05-31 18:27:07 +00:00
peter
7d869157f3 Don't leak an mbuf when a write rpc returns zero bytes written.
Obtained from: NetBSD
1998-05-31 18:25:32 +00:00
peter
feb54238cc #ifdef a diagnostic printf
Obtained from:  NetBSD
1998-05-31 18:23:24 +00:00
peter
cbeeaf83f2 Some const's
Obtained from: NetBSD
1998-05-31 17:48:07 +00:00
peter
e58631da3c NFS Jumbo commit part 1. Cosmetic and structural changes only. The aim
of this part of commits is to minimize unnecessary differences between
the other NFS's of similar origin.  Yes, there are gratuitous changes here
that the style folks won't like, but it makes the catch-up less difficult.
1998-05-31 17:27:58 +00:00
peter
c8c505e6c0 VOP_ABORTUP() appears to be called with the wrong vnode. The other callers
that I checked (eg: ufs_link()) do the ABORTOP on the directory rather than
the file itself.  After Michael Hancock's patches, the abortop doesn't seem
all that critial now since something else will free the pathname buffer.
1998-05-31 01:03:07 +00:00
peter
aa33f20993 When using NFSv3, use the remote server's idea of the maximum file size
rather than assuming 2^64.  It may not like files that big. :-)
On the nfs server, calculate and report the max file size as the point
that the block numbers in the cache would turn negative.
(ie: 1099511627775 bytes (1TB)).

One of the things I'm worried about however, is that directory offsets
are really cookies on a NFSv3 server and can be rather large, especially
when/if the server generates the opaque directory cookies by using a local
filesystem offset in what comes out as the upper 32 bits of the 64 bit
cookie.  (a server is free to do this, it could save byte swapping
depending on the native 64 bit byte order)

Obtained from:	NetBSD
1998-05-30 16:33:58 +00:00
peter
1777a04b11 Allow control of the attribute cache timeouts at mount time.
We had run out of bits in the nfs mount flags, I have moved the internal
state flags into a seperate variable.  These are no longer visible via
statfs(), but I don't know of anything that looks at them.
1998-05-19 07:11:27 +00:00
bde
3ee93801b9 Get timespecs directly instead of via timevals. 1998-05-16 16:20:50 +00:00
bde
ac772ab5bb Don't abuse `+' to combine flags. 1998-05-16 16:03:10 +00:00
bde
bad475eedf Backed out rev.1.76. It just added style bugs. 1998-05-16 15:21:29 +00:00
peter
ea57a1047f Move the *vpp initialization earlier so that it's set in all error cases.
This should stop the 'panic: leaf should not be empty' nfs panic.

PR: 1856
Submitted by: msaitoh@spa.is.uec.ac.jp
1998-05-13 05:47:09 +00:00
msmith
964ce778b1 In the words of the submitter:
---------
Make callers of namei() responsible for releasing references or locks
instead of having the underlying filesystems do it.  This eliminates
redundancy in all terminal filesystems and makes it possible for stacked
transport layers such as umapfs or nullfs to operate correctly.

Quality testing was done with testvn, and lat_fs from the lmbench suite.

Some NFS client testing courtesy of Patrik Kudo.

vop_mknod and vop_symlink still release the returned vpp.  vop_rename
still releases 4 vnode arguments before it returns.  These remaining cases
will be corrected in the next set of patches.
---------

Submitted by:	Michael Hancock <michaelh@cet.co.jp>
1998-05-07 04:58:58 +00:00
phk
9b703b1455 Eradicate the variable "time" from the kernel, using various measures.
"time" wasn't a atomic variable, so splfoo() protection were needed
around any access to it, unless you just wanted the seconds part.

Most uses of time.tv_sec now uses the new variable time_second instead.

gettime() changed to getmicrotime(0.

Remove a couple of unneeded splfoo() protections, the new getmicrotime()
is atomic, (until Bruce sets a breakpoint in it).

A couple of places needed random data, so use read_random() instead
of mucking about with time which isn't random.

Add a new nfs_curusec() function.

Mark a couple of bogosities involving the now disappeard time variable.

Update ffs_update() to avoid the weird "== &time" checks, by fixing the
one remaining call that passwd &time as args.

Change profiling in ncr.c to use ticks instead of time.  Resolution is
the same.

Add new function "tvtohz()" to avoid the bogus "splfoo(), add time, call
hzto() which subtracts time" sequences.

Reviewed by:	bde
1998-03-30 09:56:58 +00:00
bde
a1015f7749 Don't depend on <sys/mount.h> including <sys/socket.h>. 1998-03-28 12:04:40 +00:00
julian
10c5ccc30a Reviewed by: dyson@freebsd.org (john Dyson), dg@root.com (david greenman)
Submitted by:	Kirk McKusick (mcKusick@mckusick.com)
Obtained from:  WHistle development tree
1998-03-08 09:59:44 +00:00
dyson
8ceb6160f4 This mega-commit is meant to fix numerous interrelated problems. There
has been some bitrot and incorrect assumptions in the vfs_bio code.  These
problems have manifest themselves worse on NFS type filesystems, but can
still affect local filesystems under certain circumstances.  Most of
the problems have involved mmap consistancy, and as a side-effect broke
the vfs.ioopt code.  This code might have been committed seperately, but
almost everything is interrelated.

1)	Allow (pmap_object_init_pt) prefaulting of buffer-busy pages that
	are fully valid.
2)	Rather than deactivating erroneously read initial (header) pages in
	kern_exec, we now free them.
3)	Fix the rundown of non-VMIO buffers that are in an inconsistent
	(missing vp) state.
4)	Fix the disassociation of pages from buffers in brelse.  The previous
	code had rotted and was faulty in a couple of important circumstances.
5)	Remove a gratuitious buffer wakeup in vfs_vmio_release.
6)	Remove a crufty and currently unused cluster mechanism for VBLK
	files in vfs_bio_awrite.  When the code is functional, I'll add back
	a cleaner version.
7)	The page busy count wakeups assocated with the buffer cache usage were
	incorrectly cleaned up in a previous commit by me.  Revert to the
	original, correct version, but with a cleaner implementation.
8)	The cluster read code now tries to keep data associated with buffers
	more aggressively (without breaking the heuristics) when it is presumed
	that the read data (buffers) will be soon needed.
9)	Change to filesystem lockmgr locks so that they use LK_NOPAUSE.  The
	delay loop waiting is not useful for filesystem locks, due to the
	length of the time intervals.
10)	Correct and clean-up spec_getpages.
11)	Implement a fully functional nfs_getpages, nfs_putpages.
12)	Fix nfs_write so that modifications are coherent with the NFS data on
	the server disk (at least as well as NFS seems to allow.)
13)	Properly support MS_INVALIDATE on NFS.
14)	Properly pass down MS_INVALIDATE to lower levels of the VM code from
	vm_map_clean.
15)	Better support the notion of pages being busy but valid, so that
	fewer in-transit waits occur.  (use p->busy more for pageouts instead
	of PG_BUSY.)  Since the page is fully valid, it is still usable for
	reads.
16)	It is possible (in error) for cached pages to be busy.  Make the
	page allocation code handle that case correctly.  (It should probably
	be a printf or panic, but I want the system to handle coding errors
	robustly.  I'll probably add a printf.)
17)	Correct the design and usage of vm_page_sleep.  It didn't handle
	consistancy problems very well, so make the design a little less
	lofty.  After vm_page_sleep, if it ever blocked, it is still important
	to relookup the page (if the object generation count changed), and
	verify it's status (always.)
18)	In vm_pageout.c, vm_pageout_clean had rotted, so clean that up.
19)	Push the page busy for writes and VM_PROT_READ into vm_pageout_flush.
20)	Fix vm_pager_put_pages and it's descendents to support an int flag
	instead of a boolean, so that we can pass down the invalidate bit.
1998-03-07 21:37:31 +00:00