freebsd-nq/lib
Kirk McKusick 75e3597abb Continuing efforts to provide hardening of FFS, this change adds a
check hash to cylinder groups. If a check hash fails when a cylinder
group is read, no further allocations are attempted in that cylinder
group until it has been fixed by fsck. This avoids a class of
filesystem panics related to corrupted cylinder group maps. The
hash is done using crc32c.

Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
used in embedded systems with small memories and low-powered processors
which need as light-weight a filesystem as possible.

Specifics of the changes:

sys/sys/buf.h:
    Add BX_FSPRIV to reserve a set of eight b_xflags that may be used
    by individual filesystems for their own purpose. Their specific
    definitions are found in the header files for each filesystem
    that uses them. Also add fields to struct buf as noted below.

sys/kern/vfs_bio.c:
    It is only necessary to compute a check hash for a cylinder
    group when it is actually read from disk. When calling bread,
    you do not know whether the buffer was found in the cache or
    read. So a new flag (GB_CKHASH) and a pointer to a function to
    perform the hash has been added to breadn_flags to say that the
    function should be called to calculate a hash if the data has
    been read. The check hash is placed in b_ckhash and the B_CKHASH
    flag is set to indicate that a read was done and a check hash
    calculated. Though a rather elaborate mechanism, it should
    also work for check hashing other metadata in the future. A
    kernel internal API change was to change breada into a static
    fucntion and add flags and a function pointer to a check-hash
    function.

sys/ufs/ffs/fs.h:
    Add flags for types of check hashes; stored in a new word in the
    superblock. Define corresponding BX_ flags for the different types
    of check hashes. Add a check hash word in the cylinder group.

sys/ufs/ffs/ffs_alloc.c:
    In ffs_getcg do the dance with breadn_flags to get a check hash and
    if one is provided, check it.

sys/ufs/ffs/ffs_vfsops.c:
    Copy across the BX_FFSTYPES flags in background writes.
    Update the check hash when writing out buffers that need them.

sys/ufs/ffs/ffs_snapshot.c:
    Recompute check hash when updating snapshot cylinder groups.

sys/libkern/crc32.c:
lib/libufs/Makefile:
lib/libufs/libufs.h:
lib/libufs/cgroup.c:
    Include libkern/crc32.c in libufs and use it to compute check
    hashes when updating cylinder groups.

Four utilities are affected:

sbin/newfs/mkfs.c:
    Add the check hashes when building the cylinder groups.

sbin/fsck_ffs/fsck.h:
sbin/fsck_ffs/fsutil.c:
    Verify and update check hashes when checking and writing cylinder groups.

sbin/fsck_ffs/pass5.c:
    Offer to add check hashes to existing filesystems.
    Precompute check hashes when rebuilding cylinder group
    (although this will be done when it is written in fsutil.c
    it is necessary to do it early before comparing with the old
    cylinder group)

sbin/dumpfs/dumpfs.c
    Print out the new check hash flag(s)

sbin/fsdb/Makefile:
    Needs to add libufs now used by pass5.c imported from fsck_ffs.

Reviewed by: kib
Tested by: Peter Holm (pho)
2017-09-22 12:45:15 +00:00
..
atf Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
clang Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to 2017-09-06 21:21:13 +00:00
csu Don't include GNU object attributes when building with clang. 2017-08-30 19:19:31 +00:00
lib80211 Don't end up manpage titles with a full stop. 2017-05-24 21:02:53 +00:00
libalias Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 03:34:59 +00:00
libarchive Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
libauditd Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 03:55:21 +00:00
libbegemot Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 03:55:43 +00:00
libblacklist DIRDEPS_BUILD: Connect new directories and update dependencies. 2016-06-03 19:25:30 +00:00
libblocksruntime Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 03:56:10 +00:00
libbluetooth Make cached Bluetooth LE host advertise information visible from userland. 2017-04-27 15:03:24 +00:00
libbsdstat Fix off by one error in index limit calculation 2016-05-16 15:42:59 +00:00
libbsm Merge OpenBSM 1.2-alpha5 from vendor branch to FreeBSD -CURRENT: 2017-03-26 21:14:49 +00:00
libbsnmp DIRDEPS_BUILD: Update dependencies. 2017-05-09 01:48:23 +00:00
libbz2 Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 03:58:50 +00:00
libc rename(2): document capability mode errors 2017-09-15 20:12:38 +00:00
libc_nonshared Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:04:25 +00:00
libc++ Merge llvm, clang, lld, lldb, compiler-rt and libc++ r307894, and update 2017-07-13 21:58:45 +00:00
libc++experimental DIRDEPS_BUILD: Connect more libraries. 2017-07-11 00:32:48 +00:00
libcalendar
libcam Add HAS_TESTS to all Makefiles that use the SUBDIR.${MK_TESTS}+= tests idiom 2017-08-02 08:14:06 +00:00
libcapsicum capsicum_helpers: Add FIODTYPE to default ioctls allowed 2017-08-09 18:15:07 +00:00
libcasper Add supporting changes for Add limited sandbox capability to "make check" 2017-08-14 19:21:37 +00:00
libclang_rt Merge ^/head r320573 through r320970. 2017-07-13 22:01:38 +00:00
libcom_err Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:51:36 +00:00
libcompat Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
libcompiler_rt Follow-up to r323001: if the actually selected CPUTYPE is capable of 2017-08-30 07:05:29 +00:00
libcrypt MFhead@r321916 2017-08-02 09:00:59 +00:00
libcuse libcuse: make more use of the howmany() macro when available. 2016-04-26 01:20:16 +00:00
libcxxrt Surround any unmangled C++ names in libcxxrt's version map with 'extern 2017-02-22 18:44:57 +00:00
libdevctl Implement 'devctl clear driver' to undo a previous 'devctl set driver'. 2016-08-29 22:48:36 +00:00
libdevdctl Use .CURDIR:H instead of .CURDIR to simplify pathing in output, etc 2017-01-20 04:55:14 +00:00
libdevinfo MFH 2016-04-04 23:55:32 +00:00
libdevstat
libdl DIRDEPS_BUILD: Connect more libraries. 2017-07-11 00:32:48 +00:00
libdpv
libdwarf Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:54:21 +00:00
libedit libedit: raise the warning level to 3. 2017-09-14 19:50:07 +00:00
libefivar Minor fixes to edge cases in efi_get_next_variable_name 2017-09-13 04:32:23 +00:00
libelf Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:54:09 +00:00
libelftc libelftc: bump version, tracking import in r320343 2017-07-05 02:58:46 +00:00
libevent [private] add libevent1 and sqlite3 include files for our private libraries. 2017-03-27 22:34:43 +00:00
libexecinfo Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:53:40 +00:00
libexpat Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:53:45 +00:00
libfetch In fetch_resolve(), if the port number or service name is included in 2017-08-18 18:20:36 +00:00
libfigpar
libgcc_eh libcc_{s,eh}: build without SSP 2016-11-11 23:28:07 +00:00
libgcc_s Sort entries in libgcc_s Version.map 2017-07-05 13:13:38 +00:00
libgeom libgeom: Remove redundant and duplicated code 2017-09-08 15:44:52 +00:00
libgpio Use GPIOTOGGLE to toggle the pin state instead of read, modify and write. 2016-04-19 15:18:31 +00:00
libgssapi lib: initial use of reallocarray(3). 2017-04-21 19:27:33 +00:00
libiconv_modules lib: initial use of reallocarray(3). 2017-04-21 19:27:33 +00:00
libifconfig DIRDEPS_BUILD: Connect more libraries. 2017-07-11 00:32:48 +00:00
libipsec Add large replay widow support to setkey(8) and libipsec. 2017-04-13 14:44:17 +00:00
libjail libjail: make allocation in jailparam_all() somewhat more robust. 2017-04-16 19:23:10 +00:00
libkiconv Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:50:46 +00:00
libkvm Add HAS_TESTS to all Makefiles that use the SUBDIR.${MK_TESTS}+= tests idiom 2017-08-02 08:14:06 +00:00
libldns Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:50:19 +00:00
liblzma Use SRCTOP-relative paths and .CURDIR with :H instead of ".." specified paths 2017-01-20 04:46:20 +00:00
libmagic MFV r323678: file 5.32 2017-09-17 19:14:38 +00:00
libmd Increase loop unrolling for skein hashes 2017-07-01 21:18:06 +00:00
libmemstat Fix buildworld for powerpc. 2016-11-20 06:10:12 +00:00
libmilter Commit the 64-bit inode project. 2017-05-23 09:29:05 +00:00
libmp Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
libmt Add IBM TS1155 density codes to libmt and the mt(1) man page. 2017-07-14 16:45:46 +00:00
libnandfs
libnetbsd libnetbsd: add emalloc and friends 2017-04-06 14:36:08 +00:00
libnetgraph
libngatm Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:41:53 +00:00
libnv Remove redundant initialization. Don't use variable - just return the value. 2017-09-21 10:00:16 +00:00
libopenbsd
libopie Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:40:10 +00:00
libpam Add options to capture stdout and / or stderr and pass the output on 2017-03-22 13:16:04 +00:00
libpathconv Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
libpcap Stop installing pcap-int.h, which is the internal interface for libpcap. 2017-03-07 16:06:53 +00:00
libpe Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:37:03 +00:00
libpjdlog
libpmc Skylake server core PMC support for hwpmc(4). 2017-09-06 17:19:48 +00:00
libproc Avoid keeping a dangling pointer when the mappings array is resized. 2017-09-06 16:24:34 +00:00
libprocstat libprocstat(3): fix arguments list for procstat_getargv(3) and procstat_getenvv(3) 2017-07-29 22:25:45 +00:00
libradius The NAS-Identifier attribute is a string, not an integer. 2016-05-24 11:44:43 +00:00
librpcsec_gss Fix a potential problem where we might try to shift by more than 31 bits 2017-04-25 10:29:08 +00:00
librpcsvc Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP. 2017-06-19 20:47:24 +00:00
librss DIRDEPS_BUILD: Connect more libraries. 2017-07-11 00:32:48 +00:00
librt Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
librtld_db Avoid double-closing an fd if elf_begin() fails. 2017-03-22 18:14:55 +00:00
libsbuf Add HAS_TESTS to all Makefiles that use the SUBDIR.${MK_TESTS}+= tests idiom 2017-08-02 08:14:06 +00:00
libsdp lib: minor spelling fixes in comments. 2016-05-01 19:37:33 +00:00
libsm Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:35:36 +00:00
libsmb Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:35:00 +00:00
libsmdb Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:35:18 +00:00
libsmutil Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:34:34 +00:00
libsqlite3 [private] add libevent1 and sqlite3 include files for our private libraries. 2017-03-27 22:34:43 +00:00
libstand libstand: tftp_open() can leak pkt on error 2017-09-12 13:51:18 +00:00
libstdbuf
libstdthreads Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:33:45 +00:00
libsysdecode libsysdecode: report invalid cap_rights_t 2017-09-17 14:03:54 +00:00
libtacplus
libtelnet Remove redundant include directories which expand to a noop, 2017-07-31 19:07:45 +00:00
libthr Add HAS_TESTS to all Makefiles that are currently using the 2017-08-02 08:50:42 +00:00
libthread_db libthread_db: unbreak build due to sign/unsigned comparison. 2017-04-20 21:01:59 +00:00
libucl Use SRCTOP to find the sources of libucl 2016-10-15 13:17:27 +00:00
libufs Continuing efforts to provide hardening of FFS, this change adds a 2017-09-22 12:45:15 +00:00
libugidfw
libulog Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:29:23 +00:00
libunbound Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:31:19 +00:00
libusb libusb(3): Expose device caps as libusb_bos_descriptor::dev_capability 2017-08-09 18:06:27 +00:00
libusbhid Sync ^/vendor/NetBSD/tests/dist with upstream 2017-01-12 07:26:39 +00:00
libutil Add caveat to kinfo_getvmmap(3) explaining high CPU utilisation. 2017-08-18 16:42:58 +00:00
libvgl MFH 2016-04-13 16:19:50 +00:00
libvmmapi Capsicum support for bhyve(8). 2017-02-14 13:35:59 +00:00
libwrap Use SRCTOP instead of .CURDIR-relative path in .PATH directive 2017-01-18 18:14:50 +00:00
libxo MFhead@r322023 2017-08-03 18:07:01 +00:00
liby Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
libypclnt Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones 2017-01-20 04:28:41 +00:00
libz Add missing double quote to fix r316635 commit. 2017-04-09 03:50:48 +00:00
libzstd DIRDEPS_BUILD: Connect new directories. 2017-05-09 01:48:14 +00:00
msun lib/msun: add more csqrt unit tests for precision and overflow 2017-08-29 22:37:24 +00:00
ncurses Hide sccsid under #if 0, per example in style(9) 2017-08-12 22:20:08 +00:00
tests Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed 2016-05-04 23:20:53 +00:00
Makefile Provide libdl. 2017-07-10 14:59:21 +00:00
Makefile.inc