Commit Graph

113 Commits

Author SHA1 Message Date
Emmanuel Vadot
a7ffc94849 pkgbase: Put ufs related tools and lib in their own package
It's not really useful in a jail or in a mdroot or even if a users
wants to do a full zfs machine.

Reviewed by:	mckusick
Differential Revision:	https://reviews.freebsd.org/D36227
2022-10-26 19:46:34 +02:00
Kirk McKusick
e688661642 Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by:   Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation
2022-08-13 12:43:40 -07:00
Jens Schweikhardt
14e3d3248a Fix wrong capitalization in man page references. 2022-08-13 20:46:59 +02:00
Kirk McKusick
b21582ee03 Add a flags parameter to the ffs_sbget() function that reads UFS superblocks.
Rather than trying to shoehorn flags into the requested superblock
address, create a separate flags parameter to the ffs_sbget()
function in sys/ufs/ffs/ffs_subr.c. The ffs_sbget() function is
used both in the kernel and in user-level utilities through export
to the sbget() function in the libufs(3) library (see sbget(3)
for details). The kernel uses ffs_sbget() when mounting UFS
filesystems, in the glabel(8) and gjournal(8) GEOM utilities,
and in the standalone library used when booting the system
from a UFS root filesystem.

The ffs_sbget() function reads the superblock located at the byte
offset specified by its sblockloc parameter. The value UFS_STDSB
may be specified for sblockloc to request that the standard
location for the superblock be read.

The two existing options are now flags:

UFS_NOHASHFAIL will note if the check hash is wrong but will still
   return the superblock. This is used by the bootstrap code to
   give the system a chance to come up so that fsck can be run to
   correct the problem.

UFS_NOMSG indicates that superblock inconsistency error messages
   should not be printed. It is used by programs like fsck that
   want to print their own error message and programs like glabel(8)
   that just want to know if a UFS filesystem exists on a partition.

One additional flag is added:

UFS_NOCSUM causes only the superblock itself to be returned, but does
   not read in any auxiliary data structures like the cylinder group
   summary information. It is used by clients like glabel(8) that
   just want to check for possible filesystem types. Using UFS_NOCSUM
   skips the superblock checks for csum data which allows superblocks
   that have corrupted csum data to be read and used.

The validate_sblock() function checks that the superblock has not
been corrupted in a way that can crash or hang the system. Unless
the UFS_NOMSG flag is specified, it will print out any errors that
it finds. Prior to this commit, validate_sblock() returned as soon
as it found an inconsistency so would print at most one message.
It now does all its checks so when UFS_NOMSG has not been specified
will print out everything that it finds inconsistent.

Sponsored by: The FreeBSD Foundation
2022-07-30 22:51:38 -07:00
Kirk McKusick
bf24d17fda Have dumpfs(8) ignore superblock check-hash failures.
Set the STDSB_NOHASHFAIL request so that dumpfs(8) can print out
filesystem details even when the superblock checksum is incorrect.
2022-07-24 18:12:24 -07:00
Kirk McKusick
1297a704d9 Output fs_csaddr rather than fs_old_csaddr for UFS1 filesystems.
Since fs_csaddr is what will actually be used, show that value rather
than the historic and possibly incorrect value for UFS1 filesystems.
2022-07-24 18:12:24 -07:00
Robert Wing
dc35484536 dumpfs(8): add option to only print superblock information
Add an option to dumpfs, `-s`, that only prints the super block information.

Reviewed by:	chs, imp
Differential Revision:	https://reviews.freebsd.org/D30881
2021-07-02 14:18:17 -08:00
Chuck Silvers
a9d9cfff01 Print the fs last-mounted time too.
Reviewed by:	mckusick
Approved by:	mckusick (mentor)
Sponsored by:	Netflix
2020-05-02 00:10:25 +00:00
Kirk McKusick
068beacf21 The goal of this change is to prevent accidental foot shooting by
folks running filesystems created on check-hash enabled kernels
(which I will call "new") on a non-check-hash enabled kernels (which
I will call "old). The idea here is to detect when a filesystem is
run on an old kernel and flag the filesystem so that when it gets
moved back to a new kernel, it will not start getting a slew of
check-hash errors.

Back when the UFS version 2 filesystem was created, it added a file
flag FS_INDEXDIRS that was to be set on any filesystem that kept
some sort of on-disk indexing for directories. The idea was precisely
to solve the issue we have today. Specifically that a newer kernel
that supported indexing would be able to tell that the filesystem
had been run on an older non-indexing kernel and that the indexes
should not be used until they had been rebuilt. Since we have never
implemented on-disk directory indicies, the FS_INDEXDIRS flag is
cleared every time any UFS version 2 filesystem ever created is
mounted for writing.

This commit repurposes the FS_INDEXDIRS flag as the FS_METACKHASH
flag. Thus, the FS_METACKHASH is definitively known to have always
been cleared. The FS_INDEXDIRS flag has been moved to a new block
of flags that will always be cleared starting with this commit
(until they get used to implement some future feature which needs
to detect that the filesystem was mounted on a kernel that predates
the new feature).

If a filesystem with check-hashes enabled is mounted on an old
kernel the FS_METACKHASH flag is cleared. When that filesystem is
mounted on a new kernel it will see that the FS_METACKHASH has been
cleared and clears all of the fs_metackhash flags. To get them
re-enabled the user must run fsck (in interactive mode without the
-y flag) which will ask for each supported check hash whether it
should be rebuilt and enabled. When fsck is run in its default preen
mode, it will just ignore the check hashes so they will remain
disabled.

The kernel has always disabled any check hash functions that it
does not support, so as more types of check hashes are added, we
will get a non-surprising result. Specifically if filesystems get
moved to kernels supporting fewer of the check hashes, those that
are not supported will be disabled. If the filesystem is moved back
to a kernel with more of the check-hashes available and fsck is run
interactively to rebuild them, then their checking will resume.
Otherwise just the smaller subset will be checked.

A side effect of this commit is that filesystems running with
cylinder-group check hashes will stop having them checked until
fsck is run to re-enable them (since none of them currently have
the FS_METACKHASH flag set). So, if you want check hashes enabled
on your filesystems after booting a kernel with these changes, you
need to run fsck to enable them. Any newly created filesystems will
have check hashes enabled. If in doubt as to whether you have check
hashes emabled, run dumpfs and look at the list of enabled flags
at the end of the superblock details.
2018-02-08 23:06:58 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Kirk McKusick
75e3597abb Continuing efforts to provide hardening of FFS, this change adds a
check hash to cylinder groups. If a check hash fails when a cylinder
group is read, no further allocations are attempted in that cylinder
group until it has been fixed by fsck. This avoids a class of
filesystem panics related to corrupted cylinder group maps. The
hash is done using crc32c.

Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
used in embedded systems with small memories and low-powered processors
which need as light-weight a filesystem as possible.

Specifics of the changes:

sys/sys/buf.h:
    Add BX_FSPRIV to reserve a set of eight b_xflags that may be used
    by individual filesystems for their own purpose. Their specific
    definitions are found in the header files for each filesystem
    that uses them. Also add fields to struct buf as noted below.

sys/kern/vfs_bio.c:
    It is only necessary to compute a check hash for a cylinder
    group when it is actually read from disk. When calling bread,
    you do not know whether the buffer was found in the cache or
    read. So a new flag (GB_CKHASH) and a pointer to a function to
    perform the hash has been added to breadn_flags to say that the
    function should be called to calculate a hash if the data has
    been read. The check hash is placed in b_ckhash and the B_CKHASH
    flag is set to indicate that a read was done and a check hash
    calculated. Though a rather elaborate mechanism, it should
    also work for check hashing other metadata in the future. A
    kernel internal API change was to change breada into a static
    fucntion and add flags and a function pointer to a check-hash
    function.

sys/ufs/ffs/fs.h:
    Add flags for types of check hashes; stored in a new word in the
    superblock. Define corresponding BX_ flags for the different types
    of check hashes. Add a check hash word in the cylinder group.

sys/ufs/ffs/ffs_alloc.c:
    In ffs_getcg do the dance with breadn_flags to get a check hash and
    if one is provided, check it.

sys/ufs/ffs/ffs_vfsops.c:
    Copy across the BX_FFSTYPES flags in background writes.
    Update the check hash when writing out buffers that need them.

sys/ufs/ffs/ffs_snapshot.c:
    Recompute check hash when updating snapshot cylinder groups.

sys/libkern/crc32.c:
lib/libufs/Makefile:
lib/libufs/libufs.h:
lib/libufs/cgroup.c:
    Include libkern/crc32.c in libufs and use it to compute check
    hashes when updating cylinder groups.

Four utilities are affected:

sbin/newfs/mkfs.c:
    Add the check hashes when building the cylinder groups.

sbin/fsck_ffs/fsck.h:
sbin/fsck_ffs/fsutil.c:
    Verify and update check hashes when checking and writing cylinder groups.

sbin/fsck_ffs/pass5.c:
    Offer to add check hashes to existing filesystems.
    Precompute check hashes when rebuilding cylinder group
    (although this will be done when it is written in fsutil.c
    it is necessary to do it early before comparing with the old
    cylinder group)

sbin/dumpfs/dumpfs.c
    Print out the new check hash flag(s)

sbin/fsdb/Makefile:
    Needs to add libufs now used by pass5.c imported from fsck_ffs.

Reviewed by: kib
Tested by: Peter Holm (pho)
2017-09-22 12:45:15 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Marcelo Araujo
2c953857dd Use MIN macro from sys/param.h.
MFC after:	2 weeks
2016-05-02 01:40:31 +00:00
Glen Barber
406d87b1c3 Explicitly add more files to the 'runtime' package.
Sponsored by:	The FreeBSD Foundation
2016-02-09 20:19:31 +00:00
Edward Tomasz Napierala
2f221c70f9 The dumpfs(8) utility doesn't actually use disktab(5); remove it
from "SEE ALSO".

MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2015-09-03 11:40:47 +00:00
Edward Tomasz Napierala
b8c19fd719 It's 2015, and some people are still trying to use fdisk and then
go asking what debug flags to set for GEOM to make it work.  Advice
them to use gpart(8) instead.

Something similar should probably done with disklabel,
but I need to rewrite the disklabel examples first.

Reviewed by:	wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3315
2015-09-02 14:08:43 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Andrey V. Elsukov
24211a0afc Print leading zeroes of UFS2 fs_id like we do for UFS1.
PR:		156908
MFC after:	1 week
2015-05-26 18:27:38 +00:00
Baptiste Daroussin
13eb765f2d Convert sbin/ to LIBADD
Reduce overlinking
2014-11-25 11:23:12 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Simon J. Gerraty
d1d0158641 Merge from head 2013-09-05 20:18:59 +00:00
Kirk McKusick
9d384d4d3d When running the -m option to generate a newfs(8) command suitable for
recreating the filesystem, check for and output the -i, -k, and -l
options if appropriate.

Note the remaining deficiencies of the -m option in the dumpfs(8)
manual page. Specifically that newfs(8) options -E, -R, -S, and -T
options are not handled and that -p is not useful so is omitted.
Also document that newfs(8) options -n and -r are neither checked
for nor output but should be. The -r flag is needed if the filesystem
uses gjournal(8).

PR:           bin/163992
Reported by:  Dieter <freebsd@sopwith.solgatos.com>
Submitted by: Andy Kosela <akosela@andykosela.com>
MFC after:    1 week
2013-05-16 20:07:08 +00:00
Kirk McKusick
3e8b845956 Clean up trailing whitespace.
Submitted by: Andy Kosela
MFC after:    1 week
2013-05-16 19:39:11 +00:00
Simon J. Gerraty
69e6d7b75e sync from head 2013-04-12 20:48:55 +00:00
Kirk McKusick
baa12a84a7 The purpose of this change to the FFS layout policy is to reduce the
running time for a full fsck. It also reduces the random access time
for large files and speeds the traversal time for directory tree walks.

The key idea is to reserve a small area in each cylinder group
immediately following the inode blocks for the use of metadata,
specifically indirect blocks and directory contents. The new policy
is to preferentially place metadata in the metadata area and
everything else in the blocks that follow the metadata area.

The size of this area can be set when creating a filesystem using
newfs(8) or changed in an existing filesystem using tunefs(8).
Both utilities use the `-k held-for-metadata-blocks' option to
specify the amount of space to be held for metadata blocks in each
cylinder group. By default, newfs(8) sets this area to half of
minfree (typically 4% of the data area).

This work was inspired by a paper presented at Usenix's FAST '13:
www.usenix.org/conference/fast13/ffsck-fast-file-system-checker

Details of this implementation appears in the April 2013 of ;login:
www.usenix.org/publications/login/april-2013-volume-38-number-2.
A copy of the April 2013 ;login: paper can also be downloaded
from: www.mckusick.com/publications/faster_fsck.pdf.

Reviewed by: kib
Tested by:   Peter Holm
MFC after:   4 weeks
2013-03-22 21:45:28 +00:00
Simon J. Gerraty
7cf3a1c6b2 Updated dependencies 2013-03-11 17:21:52 +00:00
Simon J. Gerraty
f5f7c05209 Updated dependencies 2013-02-16 01:23:54 +00:00
Simon J. Gerraty
23090366f7 Sync from head 2012-11-04 02:52:03 +00:00
Edward Tomasz Napierala
549f62fa42 Fix problem with geom_label(4) not recognizing UFS labels on filesystems
extended using growfs(8).  The problem here is that geom_label checks if
the filesystem size recorded in UFS superblock is equal to the provider
(i.e. device) size.  This check cannot be removed due to backward
compatibility.  On the other hand, in most cases growfs(8) cannot set
fs_size in the superblock to match the provider size, because, differently
from newfs(8), it cannot recompute cylinder group sizes.

To fix this problem, add another superblock field, fs_providersize, used
only for this purpose.  The geom_label(4) will attach if either fs_size
(filesystem created with newfs(8)) or fs_providersize (filesystem expanded
using growfs(8)) matches the device size.

PR:		kern/165962
Reviewed by:	mckusick
Sponsored by:	FreeBSD Foundation
2012-10-30 21:32:10 +00:00
Eitan Adler
73ea379e32 Add support for gjournal to dumpfs
PR:		165821
Submitted by:	Andreas Longwitz <longwitz@incore.de>
Approved by:	cperciva
MFC after:	3 days
2012-10-22 03:00:20 +00:00
Marcel Moolenaar
7750ad47a9 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
Christian Brueffer
4ff79fdf3f Add missing -l flag to usage().
PR:		163629
Submitted by:	olgeni
MFC after:	1 week
2011-12-26 16:47:45 +00:00
Ed Schouten
2162cff196 Staticify dumpfs.
This tool only consists of a single C file, so we can simply mark
everything except main() static. This seems to shave off about 8% of the
binary size.
2011-12-13 09:01:44 +00:00
Ed Schouten
1efe3c6b58 Add missing static keywords for global variables to tools in sbin/.
These tools declare global variables without using the static keyword,
even though their use is limited to a single C-file, or without placing
an extern declaration of them in the proper header file.
2011-11-04 13:36:02 +00:00
Xin LI
ef84bb4cd7 Use _PATH_DEV and make the format more consistent with GEOM_LABEL.
Submitted by:	ivoras
2011-07-14 17:43:23 +00:00
Xin LI
e293d08775 Add a -l option to show file system's corresponding /dev/ufsid path.
This is useful for scripts that converts existing system's fstab to
use their /dev/ufsid devices.

MFC after:	2 weeks
2011-07-14 08:06:03 +00:00
Gavin Atkinson
275e1cf79b We now have multiple filesystems (UFS, ZFS, ...), so for tools that only
operate on one type of filesystem, mention this.
While here, capitalise the use of "UFS" in growfs.8 to match other uses of
the term in other man pages.

MFC after:	1 week
2011-05-08 12:34:31 +00:00
Juli Mallett
9775d5ebdf Add support for displaying newfs flags for SU+J and TRIM. 2011-03-23 08:33:12 +00:00
Konstantin Belousov
a738d4cf20 Add support for FS_TRIM to user-mode UFS utilities.
Reviewed by:	mckusick, pjd, pho
Tested by:	pho
MFC after:	1 month
2010-12-29 12:31:18 +00:00
Jeff Roberson
113db2dddb - Merge soft-updates journaling from projects/suj/head into head. This
brings in support for an optional intent log which eliminates the need
   for background fsck on unclean shutdown.

Sponsored by:   iXsystems, Yahoo!, and Juniper.
With help from: McKusick and Peter Holm
2010-04-24 07:05:35 +00:00
Gavin Atkinson
8f70111670 Print leading zeros in the UFS2 FSID.
PR:		bin/142155
Submitted by:	Efstratios Karatzas  gpf.kira gmail.com
Approved by:	ed (mentor)
MFC after:	2 weeks
2010-01-06 14:01:28 +00:00
Edward Tomasz Napierala
9340fc72e6 Implement NFSv4 ACL support for UFS.
Reviewed by:	rwatson
2009-12-21 19:39:10 +00:00
Ruslan Ermilov
106d839190 Switch the default WARNS level for sbin/ to 6.
Submitted by:	Ulrich Spörlein
2009-10-19 16:00:24 +00:00
Ruslan Ermilov
f807cca1a3 Properly re-create "-s size" argument to newfs(8). 2009-10-19 14:04:19 +00:00
Robert Watson
6458691cc4 Print disk offets as %jd rather than %lld; I fixed one before committing
but missed the other, which breaks 64-bit builds.

Reported by:	bf <bf2006a at yahoo dot com>
MFC after:	1 week
2009-01-28 12:08:19 +00:00
Tom Rhodes
4b797ca8d7 Bump doc date for recent change. 2009-01-28 11:31:09 +00:00