freebsd-nq/sbin
Kirk McKusick 75e3597abb Continuing efforts to provide hardening of FFS, this change adds a
check hash to cylinder groups. If a check hash fails when a cylinder
group is read, no further allocations are attempted in that cylinder
group until it has been fixed by fsck. This avoids a class of
filesystem panics related to corrupted cylinder group maps. The
hash is done using crc32c.

Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
used in embedded systems with small memories and low-powered processors
which need as light-weight a filesystem as possible.

Specifics of the changes:

sys/sys/buf.h:
    Add BX_FSPRIV to reserve a set of eight b_xflags that may be used
    by individual filesystems for their own purpose. Their specific
    definitions are found in the header files for each filesystem
    that uses them. Also add fields to struct buf as noted below.

sys/kern/vfs_bio.c:
    It is only necessary to compute a check hash for a cylinder
    group when it is actually read from disk. When calling bread,
    you do not know whether the buffer was found in the cache or
    read. So a new flag (GB_CKHASH) and a pointer to a function to
    perform the hash has been added to breadn_flags to say that the
    function should be called to calculate a hash if the data has
    been read. The check hash is placed in b_ckhash and the B_CKHASH
    flag is set to indicate that a read was done and a check hash
    calculated. Though a rather elaborate mechanism, it should
    also work for check hashing other metadata in the future. A
    kernel internal API change was to change breada into a static
    fucntion and add flags and a function pointer to a check-hash
    function.

sys/ufs/ffs/fs.h:
    Add flags for types of check hashes; stored in a new word in the
    superblock. Define corresponding BX_ flags for the different types
    of check hashes. Add a check hash word in the cylinder group.

sys/ufs/ffs/ffs_alloc.c:
    In ffs_getcg do the dance with breadn_flags to get a check hash and
    if one is provided, check it.

sys/ufs/ffs/ffs_vfsops.c:
    Copy across the BX_FFSTYPES flags in background writes.
    Update the check hash when writing out buffers that need them.

sys/ufs/ffs/ffs_snapshot.c:
    Recompute check hash when updating snapshot cylinder groups.

sys/libkern/crc32.c:
lib/libufs/Makefile:
lib/libufs/libufs.h:
lib/libufs/cgroup.c:
    Include libkern/crc32.c in libufs and use it to compute check
    hashes when updating cylinder groups.

Four utilities are affected:

sbin/newfs/mkfs.c:
    Add the check hashes when building the cylinder groups.

sbin/fsck_ffs/fsck.h:
sbin/fsck_ffs/fsutil.c:
    Verify and update check hashes when checking and writing cylinder groups.

sbin/fsck_ffs/pass5.c:
    Offer to add check hashes to existing filesystems.
    Precompute check hashes when rebuilding cylinder group
    (although this will be done when it is written in fsutil.c
    it is necessary to do it early before comparing with the old
    cylinder group)

sbin/dumpfs/dumpfs.c
    Print out the new check hash flag(s)

sbin/fsdb/Makefile:
    Needs to add libufs now used by pass5.c imported from fsck_ffs.

Reviewed by: kib
Tested by: Peter Holm (pho)
2017-09-22 12:45:15 +00:00
..
adjkerntz Explicitly add more files to the 'runtime' package. 2016-02-09 20:19:31 +00:00
badsect Commit the 64-bit inode project. 2017-05-23 09:29:05 +00:00
bsdlabel sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
camcontrol Fix whitespace bugs 2017-07-10 05:16:07 +00:00
ccdconfig ccdconfig first appeared in NetBSD 1.1 2016-10-02 23:19:05 +00:00
clri Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
comcontrol Explicitly add more files to the 'runtime' package. 2016-02-09 20:19:31 +00:00
conscontrol Explicitly add more files to the 'runtime' package. 2016-02-09 20:19:31 +00:00
ddb Explicitly add more files to the 'runtime' package. 2016-02-09 20:19:31 +00:00
decryptcore DIRDEPS_BUILD: Connect new directories. 2017-05-09 01:48:14 +00:00
devd Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
devfs sbin/devfs: clarify usage 2017-03-08 23:58:10 +00:00
dhclient Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
dmesg Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
dump Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
dumpfs Continuing efforts to provide hardening of FFS, this change adds a 2017-09-22 12:45:15 +00:00
dumpon DIRDEPS_BUILD: Update dependencies. 2017-05-09 01:48:23 +00:00
etherswitchcfg sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
fdisk sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
ffsinfo sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
fsck fsck(8): Don't overrun mount device buffer 2017-04-14 00:22:28 +00:00
fsck_ffs Continuing efforts to provide hardening of FFS, this change adds a 2017-09-22 12:45:15 +00:00
fsck_msdosfs sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
fsdb Continuing efforts to provide hardening of FFS, this change adds a 2017-09-22 12:45:15 +00:00
fsirand prefix UFS symbols with UFS_ to reduce namespace pollution 2017-02-15 19:50:26 +00:00
gbde sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
geom Provide an articulate example of how to properly delete partitions and 2017-09-19 15:08:31 +00:00
ggate sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
growfs Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
gvinum Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP. 2017-06-19 20:47:24 +00:00
hastctl DIRDEPS_BUILD: Update dependencies. 2017-05-09 01:48:23 +00:00
hastd DIRDEPS_BUILD: Update dependencies. 2017-05-09 01:48:23 +00:00
ifconfig Add supporting changes for Add limited sandbox capability to "make check" 2017-08-14 19:21:37 +00:00
init - Use strlcat() instead of strncat(). 2017-07-07 02:48:55 +00:00
ipf Disconnect ipftest and ipresend from the build until it can be verified 2017-03-14 02:10:37 +00:00
ipfw ipfw: Note that bandwidth can take G suffix in the manpage 2017-06-23 17:31:07 +00:00
iscontrol sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
kldconfig Advertise kldxref(8) a little better. 2017-04-27 19:48:00 +00:00
kldload Advertise kldxref(8) a little better. 2017-04-27 19:48:00 +00:00
kldstat Typo. 2017-03-10 19:13:38 +00:00
kldunload Explicitly add more files to the 'runtime' package. 2016-02-09 20:19:31 +00:00
ldconfig sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
md5 Fix date. 2017-03-02 14:50:01 +00:00
mdconfig Add ability to label md(4) devices. 2017-08-28 15:54:07 +00:00
mdmfs Add missing history sections to a number of storage related man pages 2016-09-16 04:28:31 +00:00
mknod Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
mksnap_ffs Fix an issue in the rev.316718 causing variable to be unsed uninitialized. 2017-04-12 19:23:41 +00:00
mount Fix "mount -uw /" when the filesystem type doesn't match. 2017-07-08 11:06:27 +00:00
mount_cd9660 Add a post-humous manpage for cd9660(5), the ISO-9660 file system 2017-03-23 02:57:08 +00:00
mount_fusefs sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
mount_msdosfs Remove msdosfs -o large support. 2017-06-09 12:06:22 +00:00
mount_nfs Fix mount_nfs so that it doesn't create mounttab entries for NFSv4 mounts. 2017-05-08 00:45:05 +00:00
mount_nullfs sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
mount_udf sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
mount_unionfs sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
nandfs Add missing history sections to a number of storage related man pages 2016-09-16 04:28:31 +00:00
natd Add history section to natd(8) 2016-10-05 20:42:35 +00:00
newfs Continuing efforts to provide hardening of FFS, this change adds a 2017-09-22 12:45:15 +00:00
newfs_msdos newfs_msdos: clarify description of -T (timestamp) option 2017-05-16 19:03:26 +00:00
newfs_nandfs newfs_nandfs: Fix unaligned pointer warning 2017-03-04 16:59:55 +00:00
nfsiod Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
nos-tun Add history section to nos-tun(8) 2016-10-05 20:45:21 +00:00
nvmecontrol Print the controller's ID in identify. 2017-08-28 23:54:08 +00:00
pfctl Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
pflogd sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
ping We use a few different ifdef's names to check if we are using Casper or not, 2017-09-21 14:41:41 +00:00
ping6 Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
quotacheck sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
rcorder Add rcorder-visualize.sh, which generates graphviz from rc.d scripts 2016-12-06 15:49:39 +00:00
reboot .Xr mount.conf(5) from boot(8). 2017-05-28 18:13:44 +00:00
recoverdisk sbin: ake use of our rounddown() macro when sys/param.h is available. 2016-05-01 02:24:05 +00:00
resolvconf sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
restore restore(8): Prevent some heap overflows 2017-04-14 00:14:40 +00:00
route Typo, the '-6' option selects inet6. 2017-08-15 19:29:10 +00:00
routed Fix userland tools that don't check the format of routing socket 2017-04-16 19:17:10 +00:00
rtsol sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
savecore Fix style and wrap lines to 80 columns in savecore.c. 2017-07-24 21:51:41 +00:00
sconfig sconfig appeared in FreeBSD 5.2. 2016-10-03 00:40:20 +00:00
setkey Add large replay widow support to setkey(8) and libipsec. 2017-04-13 14:44:17 +00:00
shutdown Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
spppcontrol Add META_MODE support. 2015-06-13 19:20:56 +00:00
sunlabel sbin: normalize paths using SRCTOP-relative paths or :H when possible 2017-03-04 11:33:01 +00:00
swapon Fix late and noauto with geli swap 2017-03-14 12:39:19 +00:00
sysctl kern.osreldate is an integer, not a string 2017-09-16 16:23:00 +00:00
tests Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed 2016-05-04 23:20:53 +00:00
tunefs Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
umount Add a new "-N" option to umount(8), that does a forced dismount of an NFS mount 2017-07-29 20:08:25 +00:00
zfsbootcfg Update the "first appeared in" version in several manual pages. 2017-05-24 17:47:49 +00:00
Makefile Remove SUBDIR ordering/uniquifying in *bin/Makefile 2017-07-06 04:15:30 +00:00
Makefile.amd64
Makefile.arm
Makefile.i386
Makefile.inc Use src.opts.mk in preference to bsd.own.mk except where we need stuff 2014-05-06 04:22:01 +00:00
Makefile.mips
Makefile.sparc64