Commit Graph

1779 Commits

Author SHA1 Message Date
Konstantin Belousov
2b19466836 Properly lock DQREF() with dqhlock. Missed locking caused counter
corruption.

Assert that the dq reference value is sane before decrementing it.

Reported and tested by:	pho
MFC after:	1 week
2012-02-22 20:03:51 +00:00
Konstantin Belousov
526d0bd547 Fix found places where uio_resid is truncated to int.
Add the sysctl debug.iosize_max_clamp, enabled by default. Setting the
sysctl to zero allows to perform the SSIZE_MAX-sized i/o requests from
the usermode.

Discussed with:	bde, das (previous versions)
MFC after:	1 month
2012-02-21 01:05:12 +00:00
Kirk McKusick
e8e848ef8e Missing conditions in checking whether an inode has been written.
Found and tested by: Peter Holm
MFC after:           2 weeks (to 9 only)
2012-02-13 01:33:39 +00:00
Kirk McKusick
5ecba76999 Historically when an application wrote an entire block of a file,
the kernel allocated a buffer but did not zero it as it was about
to be completely filled by a uiomove() from the user's buffer.
However, if the uiomove() failed, the old contents of the buffer
could be exposed especially if the file was being mmap'ed. The
fix was to always zero the buffer when it was allocated.

This change first attempts the uiomove() to the newly allocated
(and dirty) buffer and only zeros it if the uiomove() fails. The
effect is to eliminate the gratuitous zeroing of the buffer in
the usual case where the uiomove() successfully fills it.

Reviewed by:    kib
Tested by:      scottl
MFC after:      2 weeks (to 9 only)
2012-02-09 22:34:16 +00:00
Kirk McKusick
19c87af0fd In the original days of BSD, a sync was issued on every filesystem
every 30 seconds. This spike in I/O caused the system to pause every
30 seconds which was quite annoying. So, the way that sync worked
was changed so that when a vnode was first dirtied, it was put on
a 30-second cleaning queue (see the syncer_workitem_pending queues
in kern/vfs_subr.c). If the file has not been written or deleted
after 30 seconds, the syncer pushes it out. As the syncer runs once
per second, dirty files are trickled out slowly over the 30-second
period instead of all at once by a call to sync(2).

The one drawback to this is that it does not cover the filesystem
metadata. To handle the metadata, vfs_allocate_syncvnode() is called
to create a "filesystem syncer vnode" at mount time which cycles
around the cleaning queue being sync'ed every 30 seconds. In the
original design, the only things it would sync for UFS were the
filesystem metadata: inode blocks, cylinder group bitmaps, and the
superblock (e.g., by VOP_FSYNC'ing devvp, the device vnode from
which the filesystem is mounted).

Somewhere in its path to integration with FreeBSD the flushing of
the filesystem syncer vnode got changed to sync every vnode associated
with the filesystem. The result of this change is to return to the
old filesystem-wide flush every 30-seconds behavior and makes the
whole 30-second delay per vnode useless.

This change goes back to the originally intended trickle out sync
behavior. Key to ensuring that all the intended semantics are
preserved (e.g., that all inode updates get flushed within a bounded
period of time) is that all inode modifications get pushed to their
corresponding inode blocks so that the metadata flush by the
filesystem syncer vnode gets them to the disk in a timely way.
Thanks to Konstantin Belousov (kib@) for doing the audit and commit
-r231122 which ensures that all of these updates are being made.

Reviewed by:    kib
Tested by:      scottl
MFC after:      2 weeks
2012-02-07 20:43:28 +00:00
Konstantin Belousov
36fc415d5d Sprinkle missed calls to asynchronous UFS_UPDATE() in attempt to
guarantee that all UFS inode metadata changes results in the dirtiness
of the inodeblock.  Due to missed inodeblock updates, syncer was
required to fsync each mount point' vnode to guarantee periodic
metadata flush.

Reviewed by:	mckusick
Tested by:	scottl
MFC after:	2 weeks
2012-02-07 09:51:41 +00:00
Konstantin Belousov
752a98b13e Add missing opt_quota.h include to activate #ifdef QUOTA blocks,
apparently a step in unbreaking QUOTA support.

Reported and tested by:	Adam Strohl <adams-freebsd ateamsystems com>
MFC after:	1 week
2012-02-06 17:59:14 +00:00
Konstantin Belousov
b313a71044 JNEWBLK dependency may legitimately appear on the buf dependency
list. If softdep_sync_buf() discovers such dependency, it should do
nothing, which is safe as it is only waiting on the parent buffer to
be written, so it can be removed.

Committed on behalf of:	 jeff
MFC after:   1 week
2012-02-06 11:47:24 +00:00
Konstantin Belousov
c480f781ea Current implementations of sync(2) and syncer vnode fsync() VOP uses
mnt_noasync counter to temporary remove MNTK_ASYNC mount option, which
is needed to guarantee a synchronous completion of the initiated i/o
before syscall or VOP return.  Global removal of MNTK_ASYNC option is
harmful because not only i/o started from corresponding thread becomes
synchronous, but all i/o is synchronous on the filesystem which is
initiated during sync(2) or syncer activity.

Instead of removing MNTK_ASYNC from mnt_kern_flag, provide a local
thread flag to disable async i/o for current thread only. Use the
opportunity to move DOINGASYNC() macro into sys/vnode.h and
consistently use it through places which tested for MNTK_ASYNC.

Some testing demonstrated 60-70% improvements in run time for the
metadata-intensive operations on async-mounted UFS volumes, but still
with great deviation due to other reasons.

Reviewed by:	mckusick
Tested by:	scottl
MFC after:	2 weeks
2012-02-06 11:04:36 +00:00
Kirk McKusick
86b571509a There are several bugs/hangs when trying to take a snapshot on a UFS/FFS
filesystem running with journaled soft updates. Until these problems
have been tracked down, return ENOTSUPP when an attempt is made to
take a snapshot on a filesystem running with journaled soft updates.

MFC after: 2 weeks
2012-01-17 01:14:56 +00:00
Kirk McKusick
cc672d3599 Make sure all intermediate variables holding mount flags (mnt_flag)
and that all internal kernel calls passing mount flags are declared
as uint64_t so that flags in the top 32-bits are not lost.

MFC after: 2 weeks
2012-01-17 01:08:01 +00:00
Ivan Voras
d83064751a Add a bit of verbosity to the comment. 2012-01-16 15:47:42 +00:00
Kirk McKusick
b60ee81e3d Convert FFS mount error messages from kernel printf's to using the
vfs_mount_error error message facility provided by the nmount
interface.

Clean up formatting of mount warnings which still need to use
kernel printf's since they do not return errors.

Requested by: Craig Rodrigues <rodrigc@crodrigues.org>
MFC after: 2 weeks
2012-01-14 07:26:16 +00:00
Konstantin Belousov
3ab0160340 Avoid LOR between vfs_busy() lock and covered vnode lock on quotaon().
The vfs_busy() is after covered vnode lock in the global lock order, but
since quotaon() does recursive VFS call to open quota file, we usually
end up locking covered vnode after mp is busied in sys_quotactl().

Change the interface of VFS_QUOTACTL(), requiring that mp was unbusied
by fs code, and do not try to pick up vfs_busy() reference in ufs quotaon,
esp. if vfs_busy cannot succeed due to unmount being performed.

Reported and tested by:	pho
MFC after:	1 week
2012-01-08 23:06:53 +00:00
Ed Schouten
8f8d30274a Migrate ufs and ext2fs from skpc() to memcchr().
While there, remove a useless check from the code. memcchr() always
returns characters unequal to 0xff in this case, so inosused[i] ^ 0xff
can never be equal to zero. Also, the fact that memcchr() returns a
pointer instead of the number of bytes until the end, makes conversion
to an offset far more easy.
2012-01-01 20:47:33 +00:00
Gleb Kurtsou
58b1333ae5 Use implementation independent inoNN_t scalars for on-disk UFS structures
Approved by:	mdf (mentor)
2011-11-09 07:48:48 +00:00
Ed Schouten
6472ac3d8a Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.
The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.
2011-11-07 15:43:11 +00:00
Ed Schouten
8f80f103b4 Remove MALLOC_DECLAREs of nonexisting malloc-pools.
After careful grepping, it seems none of these pools can be found in our
source tree. They are not in use, nor are they defined.
2011-11-06 20:16:50 +00:00
Peter Holm
b76b6150d5 Fix the wrong commit log message for r226967: "Added missing cache purge
of from argument" and fix the comment.
2011-10-31 20:24:33 +00:00
Peter Holm
6890c3a990 The kern_renameat() looks up the fvp using the DELETE flag, which causes
the removal of the name cache entry for fvp.

Reported by:	Anton Yuzhaninov <citrin citrin ru>
In collaboration with:	kib
MFC after:	1 week
2011-10-31 15:01:47 +00:00
Kirk McKusick
8cd680f8c3 This update eliminates a lock-order reversal warning discovered
whle tracking down the system hang reported in kern/160662 and
corrected in revision 225806. The LOR is not the cause of the system
hang and indeed cannot cause an actual deadlock. However, it can
be easily eliminated by defering the acquisition of a buflock until
after all the vnode locks have been acquired.

Reported by:     Hans Ottevanger
PR:              kern/160662
2011-09-27 17:41:48 +00:00
Kirk McKusick
6b3b8a2109 This update eliminates the system hang reported in kern/160662 when
taking a snapshot on a filesystem running with journaled soft updates.

Reported by:     Hans Ottevanger
Fix verified by: Hans Ottevanger
PR:              kern/160662
2011-09-27 17:34:02 +00:00
Konstantin Belousov
b296414c62 Use nowait sync request for a vnode when doing softdep cleanup. We possibly
own the unrelated vnode lock, doing waiting sync causes deadlocks.

Reported and tested by:	pho
Approved by:	re (bz)
2011-09-20 21:53:26 +00:00
Martin Matuska
82378711f9 Generalize ffs_pages_remove() into vn_pages_remove().
Remove mapped pages for all dataset vnodes in zfs_rezget() using
new vn_pages_remove() to fix mmapped files changed by
zfs rollback or zfs receive -F.

PR:		kern/160035, kern/156933
Reviewed by:	kib, pjd
Approved by:	re (kib)
MFC after:	1 week
2011-08-25 08:17:39 +00:00
Andrey V. Elsukov
5373cf4e34 Fix lock leak.
Reported by:	Alex Lyashkov
Approved by:	re (kib)
MFC after:	1 week
2011-08-23 08:47:27 +00:00
Robert Watson
4b3a6fb933 Fix two cases involving opt_capsicum.h and module builds:
(1) opt_capsicum.h is no longer required in ffs_alloc.c, so remove the
   #include.

(2) portalfs depends on opt_capsicum.h, so have the Makefile generate one
   if required.

These affect only modules built without a kernel (i.e, not buildkernel,
but yes buildworld if the dubious MODULES_WITH_WORLD is used).

Approved by:	re (bz)
Sponsored by:	Google Inc
2011-08-15 07:32:44 +00:00
Robert Watson
a9d2f8d84f Second-to-last commit implementing Capsicum capabilities in the FreeBSD
kernel for FreeBSD 9.0:

Add a new capability mask argument to fget(9) and friends, allowing system
call code to declare what capabilities are required when an integer file
descriptor is converted into an in-kernel struct file *.  With options
CAPABILITIES compiled into the kernel, this enforces capability
protection; without, this change is effectively a no-op.

Some cases require special handling, such as mmap(2), which must preserve
information about the maximum rights at the time of mapping in the memory
map so that they can later be enforced in mprotect(2) -- this is done by
narrowing the rights in the existing max_protection field used for similar
purposes with file permissions.

In namei(9), we assert that the code is not reached from within capability
mode, as we're not yet ready to enforce namespace capabilities there.
This will follow in a later commit.

Update two capability names: CAP_EVENT and CAP_KEVENT become
CAP_POST_KEVENT and CAP_POLL_KEVENT to more accurately indicate what they
represent.

Approved by:	re (bz)
Submitted by:	jonathan
Sponsored by:	Google Inc
2011-08-11 12:30:23 +00:00
Kirk McKusick
fddf7baebe Update to -r224294 to ensure that only one of MNT_SUJ or MNT_SOFTDEP
is set so that mount can revert back to using MNT_NOWAIT when doing
getmntinfo.

Approved by: re (kib)
2011-07-30 00:43:18 +00:00
Kirk McKusick
d716efa9f7 Move the MNTK_SUJ flag in mnt_kern_flag to MNT_SUJ in mnt_flag
so that it is visible to userland programs. This change enables
the `mount' command with no arguments to be able to show if a
filesystem is mounted using journaled soft updates as opposed
to just normal soft updates.

Approved by: re (bz)
2011-07-24 18:27:09 +00:00
Kirk McKusick
2621f2c43f Default debugging error messages to off for journaled soft updates sysctls.
Delete limiting on output of these sysctls.

Approved by: re (kib)
2011-07-22 18:03:33 +00:00
Kirk McKusick
927a12ae16 Add an FFS specific mount option to allow a filesystem checker
(typically fsck_ffs) to register that it wishes to use FFS specific
sysctl's to update the filesystem. This ensures that two checkers
cannot run on a given filesystem at the same time and that no other
process accidentally or maliciously uses the filesystem updating
sysctls inappropriately. This functionality is needed by the
journaling soft-updates recovery code.
2011-07-15 16:20:33 +00:00
Kirk McKusick
b8ea56d7e4 Consistently check mount flag (MNTK_SUJ) rather than superblock
flag (FS_SUJ) when determining whether to do journaling-based
operations. The mount flag is set only when journaling is active
while the superblock flag is set to indicate that journaling is to
be used. For example, when the filesystem is mounted read-only, the
journaling may be present (FS_SUJ) but not active (MNTK_SUJ).
Inappropriate checking of the FS_SUJ flag was causing some
journaling actions to be attempted at inappropriate times.
2011-07-14 18:06:13 +00:00
Kirk McKusick
17ff0cf70f When first creating snapshots, we may free some blocks within it.
These blocks should not have TRIM applied to them.

Submitted by: Kostik Belousov
2011-07-10 05:34:49 +00:00
Kirk McKusick
8795189c98 Allow disk partitions associated with UFS read-only mounted
filesystems to be opened for writing. This functionality used to
be special-cased for just the root filesystem, but with this change
is now available for all UFS filesystems. This change is needed for
journaled soft updates recovery.

Discussed with: Jeff Roberson
2011-07-10 00:41:31 +00:00
Konstantin Belousov
58f9394c50 Use 'curthread_pflags' instead of 'thread_pflags' to signify that only
curthread can be operated upon.

Requested by:	attilio
MFC after:	1 week
2011-07-09 15:16:07 +00:00
Konstantin Belousov
acf5d7101c Use helper functions instead of manually managing TDP_INBDFLUSH.
Sponsored by:	The FreeBSD Foundation
Reviewed by:	alc (previous version)
MFC after:	1 week
2011-07-09 14:42:45 +00:00
Jeff Roberson
e9b4d8327f - Speed up pendingblock processing again. Having too much delay between
ffs_blkfree() and the pending adjustment causes all kinds of
   space related problems.
2011-07-04 22:08:04 +00:00
Jeff Roberson
f2803e61fa - Handle D_JSEGDEP in the softdep_sync_buf() switch. These can now
find themselves on snapshot vnodes.

Reported by:	pho
2011-07-04 21:04:25 +00:00
Jeff Roberson
8e4f5b70b0 - It is impossible to run request_cleanup() while doing a copyonwrite.
This will most likely cause new block allocations which can recurse
   into request cleanup.
 - While here optimize the ufs locking slightly.  We need only acquire and
   drop once.
 - process_removes() and process_truncates() also is only needed once.
 - Attempt to flush each item on the worklist once but do not loop forever
   if some can not be completed.

Discussed with:	mckusick
2011-07-04 20:53:55 +00:00
Jeff Roberson
c0f4e7afa4 - Fix an inode quota leak. We need to decrement the quota once and only
once.

Tested by:	pho
Reviewed by:	mckusick
2011-07-04 20:52:23 +00:00
Kirk McKusick
08af0c8b8d Handle the FREEDEP case in softdep_sync_buf().
This fix failed to get added in -r223325.

Submitted by:	Peter Holm
2011-06-29 22:12:43 +00:00
Alan Cox
6bbee8e28a Add a new option, OBJPR_NOTMAPPED, to vm_object_page_remove(). Passing this
option to vm_object_page_remove() asserts that the specified range of pages
is not mapped, or more precisely that none of these pages have any managed
mappings.  Thus, vm_object_page_remove() need not call pmap_remove_all() on
the pages.

This change not only saves time by eliminating pointless calls to
pmap_remove_all(), but it also eliminates an inconsistency in the use of
pmap_remove_all() versus related functions, like pmap_remove_write().  It
eliminates harmless but pointless calls to pmap_remove_all() that were being
performed on PG_UNMANAGED pages.

Update all of the existing assertions on pmap_remove_all() to reflect this
change.

Reviewed by:	kib
2011-06-29 16:40:41 +00:00
Jeff Roberson
16f7d82285 - Fix directory count rollbacks by passing the mode to the journal dep
earlier.
 - Add rollback/forward code for frag and cluster accounting.
 - Handle the FREEDEP case in softdep_sync_buf().  (submitted by pho)
2011-06-20 03:25:09 +00:00
Kirk McKusick
9957ac07b2 Fixed dereference of a NULL pointer.
Reported by:	Peter Holm
2011-06-18 21:10:03 +00:00
Kirk McKusick
ff13f23f84 Drop the include of <ufs/ffs/ffs_extern.h> from usr.sbin/makefs/ffs/ffs_bswap.c
and usr.sbin/makefs/ffs/ffs_subr.c as they have no need of anything in that
file.  No other programs or libraries include <ufs/ffs/ffs_extern.h> (nor
should they as it is totally in-kernel interfaces). For added protection
I enclosed the entire contents of <ufs/ffs/ffs_extern.h> in ifdef _KERNEL.

Feedback from:	Bruce Evans and Tai-hwa Liang
2011-06-16 23:40:10 +00:00
Tai-hwa Liang
09108f76fa Fixing compilation bustage by introducing another forward declaration. 2011-06-16 05:26:03 +00:00
Kirk McKusick
43a3cc7796 Ensure that filesystem metadata contained within persistent snapshots
is always kept consistent.

Suggested by:	Jeff Roberson
2011-06-15 23:19:09 +00:00
Kirk McKusick
2191e465cc With the restructuring of the block reclaimation code, the notification
messages for a filesystem being out of space need to be moved so that
they do not print out until after a failed cleanup attempt.

Suggested by:	Jeff Roberson
2011-06-15 18:05:08 +00:00
Kirk McKusick
e34a713594 Missing cleanup case after completion of a snapshot vnode write
claiming a released block.

Submitted by:	Jeff Roberson
Tested by:	Peter Holm
2011-06-15 06:13:08 +00:00
Dimitry Andric
222ef43340 Use alternative, less messy solution to avoid breakage after r223020:
put the snapdata structure between #ifdef _KERNEL guards.

Suggested by:	kib
2011-06-13 16:05:41 +00:00