When something was found wrong with an inode the error message
was always "UNKNOWN FILE TYPE". This error is now used only when
the file type field is wrong. Other errors have their own messages:
"BAD FILE SIZE", "NEGATIVE FILE SIZE", "BAD SPECIAL-FILE RDEV",
"INVALID DIRECT BLOCK", and "INVALID INDIRECT BLOCK".
More complete information about the inode is also provided.
Sponsored by: The FreeBSD Foundation
If the root directory exists but has a bad block number Pass1 will
accept it and setup an inoinfo structure for it. When Pass2 runs
and cannot read the root inode's content because of a bad (or
duplicate) block number, it removes the bad root inode and replaces
it. As part of creating the replacement root inode, it creates an
inoinfo entry for it. But Pass2 did delete the inoinfo entry that
Pass1 had set up for the root inode so ended up with two inoinfo
structures for it. The final step of Pass2 checks that all the ".."
entries are correct adding them if they are missing which resulted
in a second ".." entry being added to the root directory which
definitely did not go over well in the kernel name cache!
Reported by: Peter Holm
Sponsored by: The FreeBSD Foundation
Like the detailed diagnostics produced when a bad superblock
is read, provide similar detailed diagnostics when bad
cylinder groups are read.
Reported by: Peter Holm
Tested by: Peter Holm
Sponsored by: The FreeBSD Foundation
The fsck_ffs(8) utility has two subsystems for reading and writing
inodes. The getnextinode() interface is used in Pass 1 (and Pass
1b if needed) to sequentially walk through all the inodes in the
filesystem. The ginode() interface is used to read and write
individual inodes. Pass 1 uses a mix of both interfaces. This
change ensures that ginode() returns a pointer to the inode in the
cache maintained by getnextinode() when that interface holds the
requested inode so that all modifications to the inode are made in
a single place and are all written to the disk together.
Reported by: Peter Holm
Tested by: Peter Holm
Sponsored by: The FreeBSD Foundation
Historically fsck_ffs(8) would only use alternate superblocks when
running in manual mode. When the standard superblock fails, it now
tries to find and use a backup superblocks even when running in `preen'
mode. If an alternate superblock is found and the filesystem is
successfully cleaned up using it, write the alternate superblock
back to the standard superblock so that the filesystem can be
subsequently mounted and used.
Reported by: Peter Holm
Tested by: Peter Holm
Sponsored by: The FreeBSD Foundation
The UFS filesystem expects to find '.' and '..' as the first two entries
in a directory. The kernel's UFS name cache can become quite confused
when these two entries are not present as the first two entries.
Prior to this change, when the fsck_ffs(8) utility detected that
'.' and/or '..' were missing, it would report them, but only offered
to replace them if the space at the beginning of the directory was
available. Otherwise it was left to the system administrator to
move the offending file(s) out of the way and then rerun fsck_ffs(8)
to create the '.' and '..' entries.
With this change, fsck_ffs(8) will always be able to create the '.'
and/or '..' entries. It moves any files in the way elsewhere in the
directory block. If there is no room in the directory block to which
to move them, they are placed in the lost+found directory.
Reported by: Peter Holm
Sponsored by: The FreeBSD Foundation
When either searching for backup UFS superblocks or when explicitly asked
to use one with the -b option, report the reason for failure if it cannot
be used.
Reported by: Peter Holm
Sponsored by: The FreeBSD Foundation
Allocation or I/O failures in fsck_ffs(8) could cause segment
faults because of missing checks or not-yet-initialized data
structures. Correct these issues.
Reported by: Peter Holm
Sponsored by: The FreeBSD Foundation
into ffs_sbsearch() to allow use by other parts of the system.
Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.
This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.
The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.
The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.
The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.
Reviewed by: kib
Reviewed by: Warner Losh
Tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation
The fsck_ffs(8) utility made sanity checks of critical superblock
fields by comparing the values of those fields in the standard
superblock againt the values of those fields in the last alternate
superblock. The code for validating a superblock now cover all the
checked fields as well as many more. Further the checks done are
far more comprehensive. So we now drop the alternate superblock
checks as they no longer provide value. Dropping these checks also
eliminates the need to read the alternate superblock.
Rather than trying to shoehorn flags into the requested superblock
address, create a separate flags parameter to the ffs_sbget()
function in sys/ufs/ffs/ffs_subr.c. The ffs_sbget() function is
used both in the kernel and in user-level utilities through export
to the sbget() function in the libufs(3) library (see sbget(3)
for details). The kernel uses ffs_sbget() when mounting UFS
filesystems, in the glabel(8) and gjournal(8) GEOM utilities,
and in the standalone library used when booting the system
from a UFS root filesystem.
The ffs_sbget() function reads the superblock located at the byte
offset specified by its sblockloc parameter. The value UFS_STDSB
may be specified for sblockloc to request that the standard
location for the superblock be read.
The two existing options are now flags:
UFS_NOHASHFAIL will note if the check hash is wrong but will still
return the superblock. This is used by the bootstrap code to
give the system a chance to come up so that fsck can be run to
correct the problem.
UFS_NOMSG indicates that superblock inconsistency error messages
should not be printed. It is used by programs like fsck that
want to print their own error message and programs like glabel(8)
that just want to know if a UFS filesystem exists on a partition.
One additional flag is added:
UFS_NOCSUM causes only the superblock itself to be returned, but does
not read in any auxiliary data structures like the cylinder group
summary information. It is used by clients like glabel(8) that
just want to check for possible filesystem types. Using UFS_NOCSUM
skips the superblock checks for csum data which allows superblocks
that have corrupted csum data to be read and used.
The validate_sblock() function checks that the superblock has not
been corrupted in a way that can crash or hang the system. Unless
the UFS_NOMSG flag is specified, it will print out any errors that
it finds. Prior to this commit, validate_sblock() returned as soon
as it found an inconsistency so would print at most one message.
It now does all its checks so when UFS_NOMSG has not been specified
will print out everything that it finds inconsistent.
Sponsored by: The FreeBSD Foundation
for missing devices.
The fsck_ffs(8) utility uses its internal function openfilesys()
when opening a disk to be checked. This change avoids the use
of pfatal() in openfilesys() which always exits with failure (exit
value 8) so that the caller can choose the correct exit value.
In the case of a non-existent device it should exit with value 3
which allows the startup system to wait for drives (such as those
attached by USB) to come online.
Reported by: karels
Tested by: karels
PR: 262580
MFC after: 3 days
The cleanup of fsck_ffs(8) in commit c0bfa109b9 broke fsdb(8).
This commit adds the one-line update needed in fsdb(8) to make it
work with the new fsck_ffs(8) structure.
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 3 days
Normally fsck_ffs never does reads or writes that are not aligned
to the size of one of the checked filesystems fragments. The one
exception is when it finds that it needs to write the superblock
recovery information. Here it will write with the alignment reported
by the underlying disk as its sector size as reported by an
ioctl(diskfd, DIOCGSECTORSIZE, &secsize).
Modern disks have a sector size of 4096, but for backward compatibility
with older disks will report that they have a sector size of 512.
When presented with a 512 byte write, they have to read the associated
4096 byte sector, replace the 512 bytes to be written, and write
the updated 4096 byte sector back to the disk. Unfortunately, some
disks report that they have 512 sectors, but fail writes that are not
aligned to 4096 boundaries and are a multiple of 4096 bytes in size.
This commit updates fsck_ffs(8) so that it uses the filesystem fragment
size as the smallest size and alignment for doing writes rather than
the disk's reported sector size.
Reported by: Andriy Gapon
MFC after: 1 week
Part of the problem was that fsck_ffs would read the superblock
multiple times complaining and repairing the superblock check hash
each time and then at the end failing to write out the superblock
with the corrected check hash. This fix reads the superblock just
once and if the check hash is corrected ensures that the fixed
superblock gets written.
Tested by: Peter Holm
PR: 245916
MFC after: 1 week
Sponsored by: Netflix
The ino_blkatoff() and indir_blkatoff() functions failed to release
the buffers holding second and third level indirect blocks. This
commit ensures that these buffers are now properly released.
MFC after: 1 week
Sponsored by: Netflix
When fsck_ffs is running in preen mode and finds a zero-length directory,
it deletes that directory. In doing this operation, it unnecessary set
its internal flag saying that fsck_ffs needed to be rerun. This patch
deletes the rerun request for this case.
Reported by: Mark Johnson
PR: 246962
MFC after: 1 week
Sponsored by: Netflix
Background checks are only allowed for mounted filesystems - don't try
to open the device for writing when performing a background check.
While here, remove a debugging printf that's commented out.
PR: 256746
Fixes: 5cc52631b3
Reviewed by: mckusick
MFC After: 1 week
Differential Revision: https://reviews.freebsd.org/D30880
These were all incorrectly labeled as 2-clause BSD licenses by a
semi-automated process, when in fact they are 3-clause.
Discussed with: pfg, imp
MFC after: 2 weeks
Sponsored by: Axcient
The segfault was being hit in ckfini() (sbin/fsck_ffs/fsutil.c) while
attempting to traverse the buffer cache. The tail queue used for the
buffer cache was not initialized before dropping into gjournal_check().
Initialize the buffer cache before calling gjournal_check().
PR: 245907
Reviewed by: jhb, mckusick
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30537
The segfault was being hit in the rerun of Pass 1 in ginode() when
trying to get an inode that needs to be repaired. When the first run
of fsck_ffs finishes it clears the inode cache, but ginode() was
failing to check properly and tried to access the deallocated cache entry.
Reported by: Peter Holm
Reviewed by: Chuck Silvers
Tested by: Peter Holm and Chuck Silvers
MFC after: 3 days
Sponsored by: Netflix
Only print buffer cache debug message when a cache lookup has been done.
When running `fsck_ffs -d` on a gjournal'ed filesystem, it's possible
that totalreads is greater than zero when no cache lookup has been
done - causing a divide by zero. This commit fixes the following error:
Floating point exception (core dumped)
Reviewed by: mckusick
Differential Revision: https://reviews.freebsd.org/D30370
The segfault was being hit in ckfini() (sbin/fsck_ffs/fsutil.c)
while attempting to traverse the buffer cache to flush dirty buffers.
The tail queue used for the buffer cache was not initialized before
dropping into gjournal_check(). Move the buffer initialization earlier
so that it has been done before calling gjournal_check().
Reported by: crypt47, nvass
Fix by: Robert Wing
Tested by: Robert Wing
PR: 255030
PR: 255979
MFC after: 3 days
Sponsored by: Netflix
Pass 1b of fsck_ffs runs only when Pass 1 has found duplicate blocks.
Pass 1 only knows that a block is duplicate when it finds the second
instance of its use. The role of Pass 1b is to find the first use
of all the duplicate blocks. It makes a pass over the cylinder groups
looking for these blocks. When moving to the next cylinder group,
Pass 1b failed to properly calculate the starting inode number for
the cylinder group resulting in the above error message when it
tried to read the first inode in the cylinder group.
Reported by: Px
Tested by: Px
PR: 255979
MFC after: 3 days
Sponsored by: Netflix
UFS does not allow files to end with a hole; it requires that the
last block of a file be allocated. As fsck_ffs(8) initially scans
each allocated inode, it tracks the last allocated block in the
inode. It then checks that the inode's size falls in the last
allocated block. If the last allocated block falls before the size,
a `file size beyond end of allocated file' warning is issued and
the file is shortened to reference the last allocated block (to avoid
having it reference a hole at its end). If the last allocated block
falls after the size, a `partially truncated file' warning is issued
and all blocks following the block referenced by the size are freed.
Because of an incorrect unsigned comparison, this test was failing
to handle files with no allocated blocks but non-zero size (which
should have had their size reset to zero). Once that was fixed the
test started incorrectly complaining about short symbolic links
that place the link path in the inode rather than in a disk block.
Because these symbolic links have a non-zero size, but no allocated
blocks, fsck_ffs wanted to zero out their size. This patch has to
detect and avoid changing the size of such symbolic links.
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 1 week
Sponsored by: Netflix
When fsck_ffs is creating a lost+found directory it must allocate
an inode and a filesystem block. If it encounters a cylinder group
with a bad check hash, it complains twice: once for the inode and
again for the filesystem block.
This change suppresses the second complaint.
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 1 week
Sponsored by: Netflix
When fsck_ffs is running in interactive mode and finds unlinked files,
it offers to either unlink them or place them in a lost+found directory.
If the lost+found directory option is requested and no lost+found
directory exists, fsck_ffs offers to create one. When creating one,
it must allocate an inode and a filesystem block. It attempts to
allocate them from the first cylinder group. If the first cylinder
group has a bad check hash, it gives up.
This change expands the search into later cylinder groups when the
first one fails with a bad check hash.
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 1 week
Sponsored by: Netflix
Several large data structures are allocated by fsck_ffs to track
resource usage. Most but not all were deallocated at the end of
checking each filesystem. This commit consolidates the freeing
of all data structures in one place and adds one that had previously
been missing.
It is important to clean up these data structures as they can be
large. If the previous allocations have not been freed, fsck_ffs
can run out of address space when many large filesystems are being
checked. An alternative would be to fork a new instance of fsck_ffs
for each filesystem to be checked, but we choose to free the small
set of large structures to save the fork overhead.
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 7 days
Sponsored by: Netflix
This fixes a long-standing but very obscure bug in fsck_ffs when
it is run with the -R (rerun after unexpected errors). It only
occurs if fsck_ffs finds duplicate blocks and they are all contained
in inodes that reside in the first block of inodes (typically among
the first 128 inodes).
Rather than use the usual ginode() interface to walk through the
inodes in pass1, there is a special optimized `getnextinode()'
routine for walking through all the inodes. It has its own private
buffer for reading the inode blocks. If pass 1 finds duplicate
blocks it runs pass 1b to find all the inodes that contain these
duplicate blocks. Pass 1b also uses the `getnextinode()' to search
for the inodes with duplicate blocks. Pass 1b stops when all the
duplicate blocks have been found. If all the duplicate blocks are
found in the first block of inodes, then the getnextinode cache
holds this block of bad inodes. The subsequent cleanup of the inodes
in passes 2-5 is done using ginode() which uses the regular fsck_ffs
cache.
When fsck_ffs restarts, pass1() calls setinodebuf() to point at the
first block of inodes. When it calls getnextinode() to get inode
2, getnextino() sees that its private cache already has the first
set of inodes loaded and starts using them. They are of course the
trashed inodes left over from the previous run of pass1b().
The fix is to always invalidate the getnextinode cache when calling
setinodebuf().
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 3 days
Sponsored by: Netflix
Pass 1b of fsck_ffs runs only when Pass 1 has found duplicate blocks.
When starting up, Pass 1b failed to properly skip over the two unused
inodes at the beginning of the filesystem resulting in the above error
message when it tried to read the filesystem root inode.
Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 3 days
Sponsored by: Netflix
Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.
Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition. Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D28679
A long-standing bug in Pass 1 of fsck_ffs in which it is reading in
blocks of inodes to check their block pointers. It failed to round
up the size of the read to a disk block size. When disks would
accept 512-byte aligned reads, the bug rarely manifested itself.
But many recent disks will no longer accept 512-byte aligned reads
but require 4096-byte aligned reads, so the failure to properly
round-up read sizes to multiples of 4096 bytes makes the error
much more likely to occur.
Reported by: Peter Holm and others
Tested by: Peter Holm and Rozhuk Ivan
MFC after: 3 days
Sponsored by: Netflix
