Commit Graph

56 Commits

Author SHA1 Message Date
Kirk McKusick
81fbded23f Revert 248634 and 248643 (e.g., restoring 248625 and 248639).
Build verified by: Glen Barber (gjb@)
2013-03-23 20:00:02 +00:00
Sean Bruno
115f80b8d3 Revert svn r248625
Clang errors around printf could be trivially fixed, but the breakage in
sbin/fsdb were to significant for this type of change.

Submitter of this changeset has been notified and hopefully this can be
restored soon.
2013-03-23 04:26:13 +00:00
Kirk McKusick
776816d32b Speed up fsck by caching the cylinder group maps in pass1 so
that they do not need to be read again in pass5. As this nearly
doubles the memory requirement for fsck, the cache is thrown away
if other memory needs in fsck would otherwise fail. Thus, the
memory footprint of fsck remains unchanged in memory constrained
environments.

This work was inspired by a paper presented at Usenix's FAST '13:
www.usenix.org/conference/fast13/ffsck-fast-file-system-checker

Details of this implementation appears in the April 2013 of ;login:
www.usenix.org/publications/login/april-2013-volume-38-number-2.
A copy of the April 2013 ;login: paper can also be downloaded
from: www.mckusick.com/publications/faster_fsck.pdf.

Reviewed by: kib
Tested by:   Peter Holm
MFC after:   4 weeks
2013-03-22 21:50:43 +00:00
Kirk McKusick
ed75b5a156 When running with the -d option, instrument fsck_ffs to track the number,
data type, and running time of its I/O operations.

No functional changes.
2013-02-24 06:44:29 +00:00
Kirk McKusick
2ec5c91481 Update fsck_ffs buffer cache manager to use TAILQ macros.
No functional changes.
2013-02-15 01:00:48 +00:00
David E. O'Brien
4a83537505 Remove needless (int) casts of write(2)'s 3rd argument.
Also change blwrite() 'size' parameter to a ssize_t to better match
write(2).
2012-09-12 15:36:44 +00:00
Ulrich Spörlein
4b85a12f71 Spelling fixes for sbin/ 2012-01-07 16:09:33 +00:00
Konstantin Belousov
6f100596e5 Change the type of real_dev_bsize variable from long to u_int.
The DIOCGSECTORSIZE takes u_int * as an argument, using long *
causes failures on big-endian targets.

Diagnosed by:	Michiel Boland <boland37 xs4all nl>
PR:	sparc64/163460
Tested by:	pho (x86), flo (sparc64)
MFC after:	1 week
2011-12-20 20:39:00 +00:00
Kirk McKusick
d2404f0470 Break out the pass 5 inode and block map updating into a separate function
so that the function can be used by the journaling soft updates recovery.
2011-07-15 15:43:40 +00:00
Dag-Erling Smørgrav
8d3dfc2691 Add an -E option to mirror newfs's. The idea is that if you have a system
that was built before ffs grew support for TRIM, your filesystem will have
plenty of free blocks that the flash chip doesn't know are free, so it
can't take advantage of them for wear leveling.  Once you've upgraded your
kernel, you enable TRIM on the filesystem (tunefs -t enable), then run
fsck_ffs -E on it before mounting it.

I tested this patch by half-filling an mdconfig'ed filesystem image,
running fsck_ffs -E on it, then verifying that the contents were not
damaged by comparing them to a pristine copy using rsync's checksum
functionality.  There is no reliable way to test it on real hardware.

Many thanks to mckusick@, who provided the tricky parts of this patch and
reviewed the final version.

Reviewed by:	mckusick@
MFC after:	3 weeks
2011-04-29 23:00:23 +00:00
Konstantin Belousov
0947d19a09 In checker, read journal by sectors.
Due to UFS insistence to pretend that device sector size is 512 bytes,
sector size is obtained from ioctl(DIOCGSECTORSIZE) for real devices,
and from the label otherwise. The file images without label have to
be made with 512 sector size.

In collaboration with:	pho
Reviewed by:	jeff
Tested by:	bz, pho
2011-02-12 13:17:14 +00:00
Pawel Jakub Dawidek
d00690aec5 Protect fsck.h from being included twice. 2010-04-24 07:54:49 +00:00
Jeff Roberson
113db2dddb - Merge soft-updates journaling from projects/suj/head into head. This
brings in support for an optional intent log which eliminates the need
   for background fsck on unclean shutdown.

Sponsored by:   iXsystems, Yahoo!, and Juniper.
With help from: McKusick and Peter Holm
2010-04-24 07:05:35 +00:00
Ulf Lilleengen
fd02a3b5c9 - Use volatile for signal variables.
Suggested by:	Jaakko Heinonen <jh -at- saunalahti.fi>
2009-06-02 17:57:24 +00:00
Ulf Lilleengen
a0f163fd83 - Use sig_atomic_t for signal handler variables.
MFC after:	1 week
2009-05-29 20:01:50 +00:00
Kirk McKusick
910b491e7e Update the actions previously attempted by the -D option to make them
robust. With these changes fsck is now able to detect and reliably
rebuild corrupted cylinder group maps. The -D option is no longer
necessary as it has been replaced by a prompt asking whether the
corrupted cylinder group should be rebuilt and doing so when requested.
These actions are only offered and taken when running fsck in manual
mode. Corrupted cylinder groups found during preen mode cause the fsck
to fail.

Add the -r option to free up excess unused inodes. Decreasing the
number of preallocated inodes reduces the running time of future
runs of fsck and frees up space that can allocated to files. The -r
option is ignored when running in preen mode.

Reviewed by: Xin LI <delphij@>
Sponsored by: Rsync.net
2009-02-04 01:02:56 +00:00
David E. O'Brien
111a52201c Add the '-C' "check clean" flag. If the FS is marked clean, skip file
system checking.  However, if the file system is not clean, perform a
full fsck.

Reviewed by:	delphij
Obtained from:	Juniper Networks
2009-01-30 18:33:05 +00:00
Xin LI
7f94ca7233 Rename option 'C' to 'D' (damaged) in order to avoid a conflict with upcoming
Juniper 'C' (clean) flag.

Requested by:	obrien
MFC after:	1 week
2009-01-20 22:49:49 +00:00
Xin LI
14320f1e7f Add a new flag, '-C' which enables a special mode that is intended for
catastrophic recovery.  Currently, this mode only validates whether a
cylindergroup has good signature data, and prompts the user to decide
whether to clear it as a whole.

This mode is useful when there is data damage on a disk and you are
working on copy of the original disk, as fsck_ffs(8) tends to abnormally
exit in such case, as a last resort to recover data from the disk.
2008-04-10 23:49:23 +00:00
Pawel Jakub Dawidek
aef8d2449b Implements gjournal support. If file system has gjournal support enabled
and -p flag was given perform fast file system checking (bascially only
garbage collecting of orphaned objects).

Rename bread() to blread() and bwrite() to blwrite() as we now link to
the libufs library, which also implement functions with that names.

Sponsored by:	home.pl
2006-10-31 22:06:56 +00:00
Xin LI
c0ed8991fb Make background fsck based summary adjustments actually work by
initializing the sysctl mibs data before actually using them.

The original patchset (which is the actual version that is running
on my testboxes) have checked whether all of these sysctls and
refuses to do background fsck if we don't have them.  Kirk has
pointed out that refusing running fsck on old kernels is pointless,
as old kernels will recompute the summary at mount time, so I
have removed these checks.

Unfortunatelly, as the checks will initialize the mib values of
those sysctl's, and which are vital for the runtime summary
adjustment to work, we can not simply remove the check, which
will lead to problem when running background fsck over a dirty
volume.  Add these checks in a different way: give a warning rather
than refusing to work, and complain if the functionality is not
available when adjustments are necessary.

Noticed by:	A power failure at my lab
Pointy hat:	me
MFC After:	3 days
2005-03-07 08:42:49 +00:00
Xin LI
a16baf37b9 The recomputation of file system summary at mount time can be a
very slow process, especially for large file systems that is just
recovered from a crash.

Since the summary is already re-sync'ed every 30 second, we will
not lag behind too much after a crash.  With this consideration
in mind, it is more reasonable to transfer the responsibility to
background fsck, to reduce the delay after a crash.

Add a new sysctl variable, vfs.ffs.compute_summary_at_mount, to
control this behavior.  When set to nonzero, we will get the
"old" behavior, that the summary is computed immediately at mount
time.

Add five new sysctl variables to adjust ndir, nbfree, nifree,
nffree and numclusters respectively.  Teach fsck_ffs about these
API, however, intentionally not to check the existence, since
kernels without these sysctls must have recomputed the summary
and hence no adjustments are necessary.

This change has eliminated the usual tens of minutes of delay of
mounting large dirty volumes.

Reviewed by:	mckusick
MFC After:	1 week
2005-02-20 08:02:15 +00:00
Robert Watson
60c9762920 Explicitly break out NETA license from Berkeley license to clearly
indicate license grant, as well as to indicate that NETA is asserting
only two clauses, not four clauses.

Requested by:	imp
2004-10-20 08:05:02 +00:00
Don Lewis
af6726e657 Eliminate linked list used to track inodes with an initial link
count of zero and instead encode this information in the inode state.
Pass 4 performed a linear search of this list for each inode in
the file system, which performs poorly if the list is long.

Reviewed by:    sam & keramida (an earlier version of the patch), mckusick
MFC after:	1 month
2004-10-08 20:44:47 +00:00
Scott Long
c3b2344b93 Create DIP_SET() and IBLK_SET() macros to fix lvalue warnings.
Inspired by: kan
2004-09-01 05:48:06 +00:00
Mark Murray
4c723140a4 Remove advertising clause from University of California Regent's license,
per letter dated July 22, 1999.

Approved by: core, imp
2004-04-09 19:58:40 +00:00
Scott Long
1660ae8795 In the case of a background fsck, periodically update the process title
with a progress update.
2004-02-28 07:50:42 +00:00
Kirk McKusick
363c185255 Correct lines incorrectly added to the copyright message. Add missing period.
Submitted by:	Bruce Evans <bde@zeta.org.au>
Sponsored by:   DARPA & NAI Labs.
2003-02-14 21:08:14 +00:00
Poul-Henning Kamp
5186921306 Add a source file where EXTATTR checks will happen and hook it in even
if it doesn't do anything yet.

Sponsored by: DARPA & NAI Labs.
2002-09-16 11:27:47 +00:00
Tom Rhodes
ce66ddb763 s/filesystem/file system/g as discussed on -developers 2002-08-21 18:11:48 +00:00
Poul-Henning Kamp
599304a42f Warning cleanup.
Format changes by peter
2002-07-30 13:01:25 +00:00
Kirk McKusick
1c85e6a35d This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by:	Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
Tom Rhodes
3468b317cb more file system > filesystem 2002-05-16 04:10:46 +00:00
Poul-Henning Kamp
381ee4c2e8 UFS2 preparation commit:
Remove support for converting old FFS formats to newer.

Submitted by:	mckusick
Sponspored by: DARPA & NAI Labs.
2002-05-12 23:44:15 +00:00
Warner Losh
b70cd7ee68 o __P removed
o ansi function prototypes
o unifdef -D__STDC__
o __dead2 on usage prototype
o remove now-bogus main prototype
2002-03-20 22:57:10 +00:00
Ian Dowse
bf58d635ba Fix a large number of -Wall, -Wformat and -W compiler warnings.
These were mainly missing casts or wrong format strings in printf
statements, but there were also missing includes, unused variables,
functions and arguments.

The choice of `long' vs `int' still seems almost random in a lot
of places though.
2001-11-17 23:48:21 +00:00
Kris Kennaway
5979df34a6 Silence non-constant format string warnings by marking functions
as __printflike()/__printf0like(), adding const, or adding missing "%s"
format strings, as appropriate.

MFC after:	2 weeks
2001-08-19 08:19:37 +00:00
Kirk McKusick
15fca934f6 Add support for the -F flag which determines whether a specified
filesystem needs foreground checking (usually at boot time) or
can defer to background checking (after the system is up and running).
See the manual page, fsck_ffs(8), for details on the -F and -B options.
These options are primarily intended for use by the fsck front end.

All output is directed to stdout so that the output is coherent
when redirected to a file or a pipe. Unify the code with the fsck
front end that allows either a device or a mount point to be
specified as the argument to be checked.
2001-04-24 22:38:08 +00:00
Ian Dowse
d164d805c7 In fsdb, call sblock_init() which is now necessary to initialise
the global variable dev_bsize. Add a prototype for sblock_init()
to fsck.h, and set the return type correctly.
2001-04-23 21:39:14 +00:00
Kirk McKusick
38375c40b8 Minor background cleanups:
1) Set the FS_NEEDSFSCK flag when unexpected problems are encountered.
2) Clear the FS_NEEDSFSCK flag after a successful foreground cleanup.
3) Refuse to run in background when the FS_NEEDSFSCK flag is set.
4) Avoid taking and removing a snapshot when the filesystem is already clean.
5) Properly implement the force cleaning (-f) flag when in preen mode.

Note that you need to have revision 1.21 (date: 2001/04/14 05:26:28) of
fs.h installed in <ufs/ffs/fs.h> defining FS_NEEDSFSCK for this to compile.
2001-04-16 22:22:21 +00:00
Kirk McKusick
7578c6ab98 Additions to run checks on live filesystems. This change will not
affect current systems until fsck is modified to use these new
facilities. To try out this change, set the fsck passno to zero
in /etc/fstab to cause the filesystem to be mounted without running
fsck, then run `fsck_ffs -p -B <filesystem>' after the system has
been brought up multiuser to run a background cleanup on <filesystem>.
Note that the <filesystem> in question must have soft updates enabled.
2001-03-21 09:48:03 +00:00
Ian Dowse
6db798cae4 Add a simple SIGINFO handler to fsck_ffs. Shortly after receipt of
a SIGINFO (normally via Ctrl-T), a line will be output indicating
the current phase number and progress information relevant to the
current phase.

Approved by:	mckusick
2000-12-15 14:23:55 +00:00
Adrian Chadd
9ea6f4f0be Reviewed by: rwatson, bp
Approved by:	rwatson
Obtained from:	NetBSD-current source tree

The beginnings of the fsck wrappers stuff from NetBSD. This particular commit
brings a newly repo-copied sbin/fsck_ffs/ (from sbin/fsck/) into fsck wrappers
mode.

A quick overview (the code reflects this):

* Documentation changed to reflect fsck_ffs instead of fsck
* Simply acts on a single filesystem, doesn't try to do any multiple filesystem
  magic - this is done by the fsck wrappers now

And then specific to fsck_ffs:

* link to /sbin/fsck_4.2bsd and /sbin/fsck_ufs. This is because right now
  the filesystem is of type ufs not ffs, and that during autodetection the
  labeltype rather than the VFS type is used - this is because when doing
  an autodetection of filesystem type in the fsck wrapper program, it does
  not have any link between label type (4.2bsd, vinum, etc) and VFS string.

Note that this shouldn't break a build since the required buildworld Makefile
magic and import of the fsck wrapper code into src/sbin/fsck/ will happen
in a seperate commit.
2000-10-09 08:26:35 +00:00
Kirk McKusick
142d8d2f40 Teach fsck about snapshot files. These changes should have no
effect on operation of fsck on filesystems without snapshots.
If you get compilation errors, be sure that you have copies of
/usr/include/sys/mount.h (1.94), /usr/include/sys/stat.h (1.21),
and /usr/include/ufs/ffs/fs.h (1.16) as of July 4, 2000 or later.
2000-07-06 02:03:11 +00:00
Kirk McKusick
e50342e665 Yesterday I had to fix a badly broken disk, and found that fsck kept dying:
DIR I=64512 CONNECTED. PARENT WAS I=4032
  fsck: cannot find inode 995904

fsdb found the inodes with no problem:

  fsdb (inum: 64512)> inode 995904
  current inode: directory
  I=995904 MODE=40777 SIZE=512
        MTIME=Feb 14 15:27:07 2000 [0 nsec]
        CTIME=Feb 14 15:27:07 2000 [0 nsec]
        ATIME=Feb 24 10:31:58 2000 [0 nsec]
  OWNER=nobody GRP=nobody LINKCNT=4 FLAGS=0 BLKCNT=2 GEN=38a41386
  Direct blocks: 8094568 0 0 0 0 0 0 0 0 0 0 0
  Indirect blocks:  0 0 0

The problem turns out to be a program logic error in fsck.  It stores
directory inodes internally in hash lists, using the number of
directories to form the hash key:

        inpp = &inphead[inumber % numdirs];

Elsewhere, however, it increments numdirs when it finds unattached
directories.  I've made the following fix, which solved the problem in
the case in hand.

Submitted by:	Greg Lehey <grog@lemis.com>
Reviewed by:	Matthew Dillon <dillon@apollo.backplane.com>
Approved by:	Kirk McKusick <mckusick@mckusick.com>
2000-02-28 20:02:41 +00:00
Peter Wemm
6920536967 - Forgot to nuke hotroot completely.
Submitted by:	Paul Saab <ps@yahoo-inc.com>
2000-01-10 08:18:18 +00:00
Julian Elischer
d33e92f93e Reviewed by: Don Lewis <Don.Lewis@tsc.tdk.com>
Submitted by:	Kirk McKusick <mckusick@McKusick.COM>
Obtained from:	Mckusick, BSDI and a host of others

This exactly matches Kirks sources imported under the
Tag MCKUSICK2. These are as supplied by kirk with one small
change needed to compile under freeBSD.

Some FreeBSD patches will be added back, though many have been
added to Kirk's sources already.
1998-12-03 02:27:35 +00:00
Julian Elischer
b1897c197c Reviewed by: dyson@freebsd.org (john Dyson), dg@root.com (david greenman)
Submitted by:	Kirk McKusick (mcKusick@mckusick.com)
Obtained from:  WHistle development tree
1998-03-08 09:59:44 +00:00
Peter Wemm
780a5c1ec1 Merge from Lite2. Note that Lite2 has it's own filesystem clean check
skipping code that overrides ours sooner.  One should be eliminated,
but for now it works.
1997-03-11 12:20:21 +00:00
Guido van Rooij
47ceb63687 Fix the case where fsck would not see sparse directories and the kernel would
panic. If such a thing is fixed fsck needs a rerun (and bugs the user to do
so).

Reviewed by:	Kirk McKusick
1996-10-08 14:54:53 +00:00