Commit Graph

49 Commits

Author SHA1 Message Date
kib
fa686c638e Implement global and per-uid accounting of the anonymous memory. Add
rlimit RLIMIT_SWAP that limits the amount of swap that may be reserved
for the uid.

The accounting information (charge) is associated with either map entry,
or vm object backing the entry, assuming the object is the first one
in the shadow chain and entry does not require COW. Charge is moved
from entry to object on allocation of the object, e.g. during the mmap,
assuming the object is allocated, or on the first page fault on the
entry. It moves back to the entry on forks due to COW setup.

The per-entry granularity of accounting makes the charge process fair
for processes that change uid during lifetime, and decrements charge
for proper uid when region is unmapped.

The interface of vm_pager_allocate(9) is extended by adding struct ucred *,
that is used to charge appropriate uid when allocation if performed by
kernel, e.g. md(4).

Several syscalls, among them is fork(2), may now return ENOMEM when
global or per-uid limits are enforced.

In collaboration with:	pho
Reviewed by:	alc
Approved by:	re (kensmith)
2009-06-23 20:45:22 +00:00
alc
9ec14df4c8 Eliminate unnecessary variables. 2009-06-13 20:21:08 +00:00
alc
095c1f19b6 Eliminate redundant setting of a page's valid bits and pointless clearing
of the same page's dirty bits.
2009-05-27 18:12:10 +00:00
attilio
1dcb84131b Remove the thread argument from the FSD (File-System Dependent) parts of
the VFS.  Now all the VFS_* functions and relating parts don't want the
context as long as it always refers to curthread.

In some points, in particular when dealing with VOPs and functions living
in the same namespace (eg. vflush) which still need to be converted,
pass curthread explicitly in order to retain the old behaviour.
Such loose ends will be fixed ASAP.

While here fix a bug: now, UFS_EXTATTR can be compiled alone without the
UFS_EXTATTR_AUTOSTART option.

VFS KPI is heavilly changed by this commit so thirdy parts modules needs
to be recompiled.  Bump __FreeBSD_version in order to signal such
situation.
2009-05-11 15:33:26 +00:00
alc
a8c3c977f2 Use uiomove_fromphys() instead of the combination of sf_buf and uiomove().
This is not only shorter; it also eliminates unnecessary thread pinning on
architectures that implement a direct map.

MFC after:	3 weeks
2009-02-22 19:50:09 +00:00
alc
f736b7c6db Simplify the unwiring and activation of pages.
MFC after:	1 week
2009-02-22 18:15:17 +00:00
kib
8571ced24c Lookup up the directory entry for the tmpfs node that are deleted by
both node pointer and name component. This does the right thing for
hardlinks to the same node in the same directory.

Submitted by:	Yoshihiro Ota <ota j email ne jp>
PR:	kern/131356
MFC after:	2 weeks
2009-02-08 19:18:33 +00:00
bz
9fd12161ca Remove unused local variables.
Submitted by:	Christoph Mallon christoph.mallon@gmx.de
Reviewed by:	kib
MFC after:	2 weeks
2009-01-31 17:36:22 +00:00
trasz
0ad8692247 Introduce accmode_t. This is required for NFSv4 ACLs - it will be neccessary
to add more V* constants, and the variables changed by this patch were often
being assigned to mode_t variables, which is 16 bit.

Approved by:	rwatson (mentor)
2008-10-28 13:44:11 +00:00
obrien
d31fa36475 The kernel implemented 'memcmp' is an alias for 'bcmp'. However, memcmp
and bcmp are not the same thing.  'man bcmp' states that the return is
"non-zero" if the two byte strings are not identical.  Where as,
'man memcmp' states that the return is the "difference between the
first two differing bytes (treated as unsigned char values" if the
two byte strings are not identical.

So provide a proper memcmp(9), but it is a C implementation not a tuned
assembly implementation.  Therefore bcmp(9) should be preferred over memcmp(9).
2008-09-23 14:45:10 +00:00
kib
a127656dea fdescfs, devfs, mqueuefs, nfs, portalfs, pseudofs, tmpfs and xfs
initialize the vattr structure in VOP_GETATTR() with VATTR_NULL(),
vattr_null() or by zeroing it. Remove these to allow preinitialization
of fields work in vn_stat(). This is needed to get birthtime initialized
correctly.

Submitted by:   Jaakko Heinonen <jh saunalahti fi>
Discussed on:   freebsd-fs
MFC after:	1 month
2008-09-20 19:50:52 +00:00
kib
ef4f1dc9c7 Initialize va_rdev to NODEV instead of 0 or VNOVAL in VOP_GETATTR().
NODEV is more appropriate when va_rdev doesn't have a meaningful value.

Submitted by:   Jaakko Heinonen <jh saunalahti fi>
Suggested by:   bde
Discussed on:   freebsd-fs
MFC after:	1 month
2008-09-20 19:49:15 +00:00
kib
81d455e702 Initialize va_flags and va_filerev properly in VOP_GETATTR(). Don't
initialize va_vaflags and va_spare because they are not part of the
VOP_GETATTR() API. Also don't initialize birthtime to ctime or zero.

Submitted by:   Jaakko Heinonen <jh saunalahti fi>
Reviewed by:	bde
Discussed on:   freebsd-fs
MFC after:	1 month
2008-09-20 19:46:45 +00:00
delphij
0791911684 Reflect license change of NetBSD code.
Obtained from:	NetBSD
MFC after:	3 days
2008-09-03 18:53:48 +00:00
attilio
dbf35e279f Decontextualize the couplet VOP_GETATTR / VOP_SETATTR as the passed thread
was always curthread and totally unuseful.

Tested by: Giovanni Trematerra <giovanni dot trematerra at gmail dot com>
2008-08-28 15:23:18 +00:00
kib
591c809288 Do not redo the vnode tear-down work already done by insmntque() when
vnode cannot be put on the vnode list for mount.

Reported and tested by:	marck
Guilty party:	me
MFC after:	3 days
2008-06-15 18:40:58 +00:00
kib
52243403eb Move the head of byte-level advisory lock list from the
filesystem-specific vnode data to the struct vnode. Provide the
default implementation for the vop_advlock and vop_advlockasync.
Purge the locks on the vnode reclaim by using the lf_purgelocks().
The default implementation is augmented for the nfs and smbfs.
In the nfs_advlock, push the Giant inside the nfs_dolock.

Before the change, the vop_advlock and vop_advlockasync have taken the
unlocked vnode and dereferenced the fs-private inode data, racing with
with the vnode reclamation due to forced unmount. Now, the vop_getattr
under the shared vnode lock is used to obtain the inode size, and
later, in the lf_advlockasync, after locking the vnode interlock, the
VI_DOOMED flag is checked to prevent an operation on the doomed vnode.

The implementation of the lf_purgelocks() is submitted by dfr.

Reported by:	kris
Tested by:	kris, pho
Discussed with:	jeff, dfr
MFC after:	2 weeks
2008-04-16 11:33:32 +00:00
dfr
79d2dfdaa6 Add the new kernel-mode NFS Lock Manager. To use it instead of the
user-mode lock manager, build a kernel with the NFSLOCKD option and
add '-k' to 'rpc_lockd_flags' in rc.conf.

Highlights include:

* Thread-safe kernel RPC client - many threads can use the same RPC
  client handle safely with replies being de-multiplexed at the socket
  upcall (typically driven directly by the NIC interrupt) and handed
  off to whichever thread matches the reply. For UDP sockets, many RPC
  clients can share the same socket. This allows the use of a single
  privileged UDP port number to talk to an arbitrary number of remote
  hosts.

* Single-threaded kernel RPC server. Adding support for multi-threaded
  server would be relatively straightforward and would follow
  approximately the Solaris KPI. A single thread should be sufficient
  for the NLM since it should rarely block in normal operation.

* Kernel mode NLM server supporting cancel requests and granted
  callbacks. I've tested the NLM server reasonably extensively - it
  passes both my own tests and the NFS Connectathon locking tests
  running on Solaris, Mac OS X and Ubuntu Linux.

* Userland NLM client supported. While the NLM server doesn't have
  support for the local NFS client's locking needs, it does have to
  field async replies and granted callbacks from remote NLMs that the
  local client has contacted. We relay these replies to the userland
  rpc.lockd over a local domain RPC socket.

* Robust deadlock detection for the local lock manager. In particular
  it will detect deadlocks caused by a lock request that covers more
  than one blocking request. As required by the NLM protocol, all
  deadlock detection happens synchronously - a user is guaranteed that
  if a lock request isn't rejected immediately, the lock will
  eventually be granted. The old system allowed for a 'deferred
  deadlock' condition where a blocked lock request could wake up and
  find that some other deadlock-causing lock owner had beaten them to
  the lock.

* Since both local and remote locks are managed by the same kernel
  locking code, local and remote processes can safely use file locks
  for mutual exclusion. Local processes have no fairness advantage
  compared to remote processes when contending to lock a region that
  has just been unlocked - the local lock manager enforces a strict
  first-come first-served model for both local and remote lockers.

Sponsored by:	Isilon Systems
PR:		95247 107555 115524 116679
MFC after:	2 weeks
2008-03-26 15:23:12 +00:00
attilio
4014b55830 Axe the 'thread' argument from VOP_ISLOCKED() and lockstatus() as it is
always curthread.

As KPI gets broken by this patch, manpages and __FreeBSD_version will be
updated by further commits.

Tested by:	Andrea Barberio <insomniac at slackware dot it>
2008-02-25 18:45:57 +00:00
attilio
71b7824213 VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used in
conjuction with 'thread' argument passing which is always curthread.
Remove the unuseful extra-argument and pass explicitly curthread to lower
layer functions, when necessary.

KPI results broken by this change, which should affect several ports, so
version bumping and manpage update will be further committed.

Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>
2008-01-13 14:44:15 +00:00
attilio
18d0a0dd51 vn_lock() is currently only used with the 'curthread' passed as argument.
Remove this argument and pass curthread directly to underlying
VOP_LOCK1() VFS method. This modify makes the code cleaner and in
particular remove an annoying dependence helping next lockmgr() cleanup.
KPI results, obviously, changed.

Manpage and FreeBSD_version will be updated through further commits.

As a side note, would be valuable to say that next commits will address
a similar cleanup about VFS methods, in particular vop_lock1 and
vop_unlock.

Tested by:	Diego Sardina <siarodx at gmail dot com>,
		Andrea Di Pasquale <whyx dot it at gmail dot com>
2008-01-10 01:10:58 +00:00
delphij
e93a39275f Turn MPASS(0) into panic with more obvious reason why the assertion
is failed.
2007-12-07 00:00:21 +00:00
delphij
17d10d92fe size_max should be unsigned, as such, use size_t here. 2007-12-06 23:19:05 +00:00
wkoszek
411cf00f62 Explicitly initialize 'error' to 0 (two places). It lets one to build tmpfs
from the latest source tree with older compiler--gcc3.

Reviewed by:	kib@ (on freebsd-current@)
Approved by:	cognet@ (mentor)
2007-12-04 20:14:15 +00:00
delphij
c5d6580fd4 MFp4: Several fixes to tmpfs which makes it to survive from pho@'s
strees2 suite, to quote his letter, this change:

1. It removes the tn_lookup_dirent stuff. I think this cannot be fixed,
   because nothing protects vnode/tmpfs node between lookup is done, and
   actual operation is performed, in the case the vnode lock is dropped.
   At least, this is the case with the from vnode for rename.

   For now, we do the linear lookup in the parent node. This has its own
   drawbacks. Not mentioning speed (that could be fixed by using hash), the
   real problem is the situation where several hardlinks exist in the dvp.
   But, I think this is fixable.

2. The patch restores the VV_ROOT flag on the root vnode after it became
   reclaimed and allocated again. This fixes MPASS assertion at the start
   of the tmpfs_lookup() reported by many.

Submitted by:	kib
2007-11-18 04:52:40 +00:00
delphij
b0d971c673 MFp4: Fix several style(9) bugs.
Submitted by:	des
2007-11-18 04:40:42 +00:00
delphij
99be157da9 Correct a stack overflow which will trigger panics when
mode= is specified, caused by incorrect format string
specified to vfs_scanopt() and subsequently vsscanf().

Pointed out by:	kib
Submitted by:	des
2007-11-12 18:57:33 +00:00
delphij
0c91cfd26b MFp4: Provide a dummy verb "export" to shut up the message
showed up at start when NFS is enabled.

Reported by:	rafan
Approved by:	re (tmpfs blanket)
2007-10-04 17:11:48 +00:00
delphij
679cdcf0e4 Additional work is still needed before we can claim that tmpfs
is stable enough for production usage.  Warn user upon mount.

Approved by:	re (tmpfs blanket)
2007-10-04 17:08:46 +00:00
delphij
e83de305a6 MFp4: rework tmpfs_readdir() logic in terms of correctness.
Approved by:	re (tmpfs blanket)
Tested with:	fstest, fsx
2007-08-16 11:00:07 +00:00
delphij
5496743409 MFp4:
- LK_RETRY prohibits vget() and vn_lock() to return error.
   Remove associated code. [1]
 - Properly use vhold() and vdrop() instead of their unlocked
   versions, we are guaranteed to have the vnode's interlock
   unheld. [1]
 - Fix a pseudo-infinite loop caused by 64/32-bit arithmetic
   with the same way used in modern NetBSD versions. [2]
 - Reorganize tmpfs_readdir to reduce duplicated code.

Submitted by:	kib [1]
Obtained from:	NetBSD [2]
Approved by:	re (tmpfs blanket)
2007-08-10 11:00:30 +00:00
delphij
1e2d5f7f4a MFp4:
- Respect cnflag and don't lock vnode always as LK_EXCLUSIVE [1]
 - Properly lock around tn_vnode to avoid NULL deference
 - Be more careful handling vnodes (*)

(*) This is a WIP
[1] by pjd via howardsu

Thanks kib@ for his valuable VFS related comments.

Tested with:	fsx, fstest, tmpfs regression test set
Found by:	pho's stress2 suite
Approved by:	re (tmpfs blanket)
2007-08-10 05:24:49 +00:00
delphij
8f39689f22 MFp4 - Refine locking to eliminate some potential race/panics:
- Copy before testing a pointer.  This closes a race window.
 - Use msleep with the node interlock instead of tsleep.
 - Do proper locking around access to tn_vpstate.
 - Assert vnode VOP lock for dir_{atta,de}tach to capture
   inconsistent locking.

Suggested by:	kib
Submitted by:	delphij
Reviewed by:	Howard Su
Approved by:	re (tmpfs blanket)
2007-08-03 06:24:31 +00:00
delphij
41fd4384d1 MFp4: Force 64-bit arithmatic when caculating the maximum file size.
This fixes tmpfs caculations on 32-bit systems equipped with more than
4GB swap.

Reported by:	Craig Boston <craig xfoil gank org>
PR:		kern/114870
Approved by:	re (tmpfs blanket)
2007-07-24 17:14:53 +00:00
delphij
0321a712a7 MFp4: When swapping is not enabled, allow creating files by taking
physical memory pages into account for tm_maxfilesize.

Reported by:	Dominique Goncalves <dominique.goncalves gmail.com>
Submitted by:	Howard Su
Approved by:	re (tmpfs blanket)
2007-07-23 06:54:58 +00:00
delphij
d6ea8c65f3 MFp4: Rework on tmpfs's mapped read/write procedures. This
should finally fix fsx test case.

The printf's added here would be eventually turned into
assertions.

Submitted by:	Mingyan Guo (mostly)
Approved by:	re (tmpfs blanket)
2007-07-19 03:34:50 +00:00
delphij
ff9bf4b792 MFp4: Make use of the kernel unit number allocation facility
for tmpfs nodes.

Submitted by:	Mingyan Guo <guomingyan gmail com>
Approved by:	re (tmpfs blanket)
2007-07-11 14:26:27 +00:00
delphij
b4ff595242 MFp4:
- Plug memory leak.
 - Respect underlying vnode's properties rather than assuming that
   the user want root:wheel + 0755.  Useful for using tmpfs(5) for
   /tmp.
 - Use roundup2 and howmany macros instead of rolling our own version.
 - Try to fix fsx -W -R foo case.
 - Instead of blindly zeroing a page, determine whether we need a pagein
   order to prevent data corruption.
 - Fix several bugs reported by Coverity.

Submitted by:	Mingyan Guo <guomingyan gmail com>, Howard Su, delphij
Coverity ID:	CID 2550, 2551, 2552, 2557
Approved by:	re (tmpfs blanket)
2007-07-08 15:56:12 +00:00
delphij
f5523801a3 MFp4:
- Remove unnecessary NULL checks after M_WAITOK allocations.
 - Use VOP_ACCESS instead of hand-rolled suser_cred()
   calls. [1]
 - Use malloc(9) KPI to allocate memory for string.  The
   optimization taken from NetBSD is not valid for FreeBSD
   because our malloc(9) already act that way. [2]

Requested by:	rwatson [1]
Submitted by:	Howard Su [2]
Approved by:	re (tmpfs blanket)
2007-06-29 05:23:15 +00:00
delphij
66a8b537d6 Space/style cleanups after last set of commits.
Approved by:	re (tmpfs blanket)
2007-06-28 02:39:31 +00:00
delphij
17bb881f75 Staticify most of fifo/vn operations, they should not
be directly exposed outside.

Approved by:	re (tmpfs blanket)
2007-06-28 02:36:41 +00:00
delphij
ee97250932 Use vfs_timestamp instead of nanotime when obtaining
a timestamp for use with timekeeping.

Approved by:	re (tmpfs blanket)
2007-06-28 02:34:32 +00:00
delphij
eedf8bf53d Reorder tf_gen and tf_id in struct tmpfs_fid. This
saves 8 bytes on amd64 architecture.

Obtained from:	NetBSD
Approved by:	re (tmpfs blanket)
2007-06-28 02:32:44 +00:00
delphij
a404044c90 Remove two function prototypes that are no longer used.
Approved by:	re (tmpfs blanket)
2007-06-26 02:08:29 +00:00
delphij
73fffc81bc - Sync with NetBSD's RCSID (HEAD preferred).
- Correct a typo.

Approved by:	re (tmpfs blanket)
2007-06-26 02:07:08 +00:00
delphij
a78e2646a2 MFp4: Several clean-ups and improvements over tmpfs:
- Remove tmpfs_zone_xxx KPI, the uma(9) wrapper, since
   they does not bring any value now.
 - Use |= instead of = when applying VV_ROOT flag.
 - Remove tm_avariable_nodes list.  Use uma to hold the
   released nodes.
 - init/destory interlock mutex of node when init/fini
   instead of ctor/dtor.
 - Change memory computing using u_int to fix negative
   value in 2G mem machine.
 - Remove unnecessary bzero's
 - Rely uma logic to make file id allocation harder to
   guess.
 - Fix some unsigned/signed related things.  Make sure
   we respect -o size=xxxx
 - Use wire instead of hold a page.
 - Pass allocate_zero to obtain zeroed pages upon first
   use.

Submitted by:	Howard Su
Approved by:	re (tmpfs blanket, kensmith)
2007-06-25 18:46:13 +00:00
delphij
1da67b5003 Use vfs_timestamp() instead of nanotime() - make it up to
the user to make decisions about how detail they wanted
timestamps to have.
2007-06-18 14:40:19 +00:00
delphij
d50b261fe6 MFp4: fix two locking problems:
- Hold TMPFS_LOCK while updating tm_pages_used.
 - Hold vm page while doing uiomove.

This will hopefully fix all known panics.

Submitted by:	Howard Su
2007-06-18 01:43:13 +00:00
delphij
ef518e4122 MFp4: Add tmpfs, an efficient memory file system.
Please note that, this is currently considered as an
experimental feature so there could be some rough
edges.  Consult http://wiki.freebsd.org/TMPFS for
more information.

For now, connect tmpfs to build on i386 and amd64
architectures only.  Please let us know if you have
success with other platforms.

This work was developed by Julio M. Merino Vidal
for NetBSD as a SoC project; Rohit Jalan ported it
from NetBSD to FreeBSD.  Howard Su and Glen Leeder
are worked on it to continue this effort.

Obtained from:	NetBSD via p4
Submitted by:	Howard Su (with some minor changes)
Approved by:	re (kensmith)
2007-06-16 01:56:05 +00:00