Commit Graph

292 Commits

Author SHA1 Message Date
Konstantin Belousov
5942b4b6fd sys/param.h: Add _WANT_P_OSREL
Use it instead of defining IN_RTLD by base sources that want P_OSREL_
defines in userspace, but are not rtld.
This allows to remove abuse of IN_RTLD from userspace.

Reviewed by:	dchagin, markj, imp
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D38585
2023-02-15 02:43:18 +02:00
Kirk McKusick
78f4129876 Enable taking snapshots on UFS/FFS filesystems using journaled soft updates.
All the needed infrastructure updates have been made to allow
snapshots to be taken on UFS/FFS filesystems that are using journaled
soft updates. The most immediate benefit is the ability to use a
snapshot to take a consistent filesystem dump on a live filesystem
using the -L option to dump(8).

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36491
2022-11-12 22:56:03 -08:00
Emmanuel Vadot
a7ffc94849 pkgbase: Put ufs related tools and lib in their own package
It's not really useful in a jail or in a mdroot or even if a users
wants to do a full zfs machine.

Reviewed by:	mckusick
Differential Revision:	https://reviews.freebsd.org/D36227
2022-10-26 19:46:34 +02:00
Kirk McKusick
0929a153fc Add a description of soft updates journaling to newfs(8).
Add a descrition to the newfs(8) -j (journal enablement) flag
that explains what soft updates journaling does, the tradeoffs
to using it, and the limitations that it imposes. Copied from
the description in tunefs(8).

PR:           261944
Sponsored by: The FreeBSD Foundation
2022-10-21 10:57:31 -07:00
Jens Schweikhardt
016aeb7ca0 The fdformat man page is in section 8 (not 1). 2022-08-14 18:14:52 +02:00
Kirk McKusick
f030f1102c Delete UFS2 backup superblock recovery info when building a UFS1 filesystem.
Only the UFS2 filesystem has support for storing information needed
to find alternate superblocks. If that information is inadvertently
left in place when building a UFS1 filesystem, fsck_ffs may stumble
across it and attempt to use it to recover the UFS1 filesystem
which can only end poorly.
2022-07-20 22:52:10 -07:00
Mateusz Piotrowski
fd06117525 *: Do not use the no-op -r flag for bsdlabel(8)
The -r flag is ignored by the FreeBSD implementation of bsdlabel(8)
(also called disklabel(8) in the past). Remove its use from examples
and tests in the tree.

This commit does not touch historical documentation under share/doc/smm
and files under contrib/netbsd-tests.

Reviewed by:	imp
MFC after:	2 weeks
Approved by:	imp (src)
Fixes:		57dfbec57b More axe-work:
Differential Revision:	https://reviews.freebsd.org/D34585
2022-03-17 17:28:07 +01:00
Wuyang Chung
c5f549c1e0 newfs(8): Fix a bug in initialization of sblock.fs_maxbsize .
Fixes:		1c85e6a35d (SVN r98542)
Pull Request:	https://github.com/freebsd/freebsd-src/pull/587
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-03-11 10:17:06 +01:00
Mateusz Piotrowski
5b9b65e92f Explain the newfs naming convention
It might be unclear why newfs and newfs_msdos should cross-reference
each other. Add a note explaining it.

This is a follow-up to 74bd207697.

Reported by:	kib
Reviewed by:    imp, kib, rpokala
MFC after:	3 days
2021-04-17 23:16:17 +02:00
Konstantin Belousov
d485c77f20 Remove #define _KERNEL hacks from libprocstat
Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.

Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition.  Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.

Reviewed by:	mckusick
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D28679
2021-02-21 11:38:21 +02:00
Mateusz Piotrowski
74bd207697 Reference newfs_msdos(8) from the newfs(8) manual
PR:		252484
Reported by:	Graham Perrin <grahamperrin@gmail.com>
MFC after:	3 days
2021-01-07 10:24:07 +01:00
Gordon Bergling
e13534d54a newfs(8): Fix unusual Xr order
- unusual Xr order: gjournal after gpart

MFC after:	3 days
2020-12-28 16:46:28 +01:00
Kirk McKusick
314a6544c5 In the newfs(8) utility, use the more appropriate sbwrite() and cgwrite()
libufs interfaces rather than sbput() and cgput().

No functional change.

MFC after:    7 days
Sponsored by: Netflix
2020-09-13 22:57:50 +00:00
Kirk McKusick
34816cb9ae Move the pointers stored in the superblock into a separate
fs_summary_info structure. This change was originally done
by the CheriBSD project as they need larger pointers that
do not fit in the existing superblock.

This cleanup of the superblock eases the task of the commit
that immediately follows this one.

Suggested by: brooks
Reviewed by:  kib
PR:           246983
Sponsored by: Netflix
2020-06-19 01:02:53 +00:00
Kirk McKusick
47d3e2f83b Correct the location of the first backup superblock in fsck_ffs.8.
Make a note in the newfs.8 manual page to update the first backup
superblock location when changing the default fragment size for
the filesystem.

Reported by:  O. Hartmann
2019-08-07 16:56:00 +00:00
Dmitry Morozovsky
1165591e7f Allow dashes as a valid character in UFS labels.
Reviewed by:	mckusick, imp, 0mp
MFC after:	2 weeks
Differential Revision:	D18991
2019-01-29 10:21:41 +00:00
Kirk McKusick
cd29c58eae Update tunefs and newfs error messages for the -L (volume label) option
to note that underscores are valid.

PR:           235182
Reported by:  Rodney W. Grimes (rgrimes@)
Sponsored by: Netflix
2019-01-26 22:27:12 +00:00
Kirk McKusick
0cde0ab2d3 Allow tunefs to include '_' as a legal character in label names
to make it consistent with newfs. Document the legality of '_'
in label names in both tunefs(8) and newfs(8).

PR:           235182
Submitted by: darius@dons.net.au
Reviewed by:  Conrad Meyer
MFC after:    3 days
Sponsored by: Netflix
2019-01-25 20:07:18 +00:00
Kirk McKusick
8f829a5cf0 Continuing efforts to provide hardening of FFS. This change adds a
check hash to the filesystem inodes. Access attempts to files
associated with an inode with an invalid check hash will fail with
EINVAL (Invalid argument). Access is reestablished after an fsck
is run to find and validate the inodes with invalid check-hashes.
This check avoids a class of filesystem panics related to corrupted
inodes. The hash is done using crc32c.

Note this check-hash is for the inode itself and not any of its
indirect blocks. Check-hash validation may be extended to also
cover indirect block pointers, but that will be a separate (and
more costly) feature.

Check hashes are added only to UFS2 and not to UFS1 as UFS1 is
primarily used in embedded systems with small memories and low-powered
processors which need as light-weight a filesystem as possible.

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-12-11 22:14:37 +00:00
Kirk McKusick
9fc5d538fc In preparation for adding inode check-hashes, clean up and
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getinode / putinode interface.

Convert the utilities that had been using the undocumented
interface to use the new documented interface.

No functional change (as for now the libufs library does not
do inode check-hashes).

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-11-13 21:40:56 +00:00
Kirk McKusick
ec888383cf Continuing efforts to provide hardening of FFS, this change adds a
check hash to the superblock. If a check hash fails when an attempt
is made to mount a filesystem, the mount fails with EINVAL (Invalid
argument). This avoids a class of filesystem panics related to
corrupted superblocks. The hash is done using crc32c.

Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
used in embedded systems with small memories and low-powered processors
which need as light-weight a filesystem as possible.

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-10-23 21:10:06 +00:00
Eitan Adler
4b3dd106e6 newfs: clean up warnings
- remove param: unused since r95357.
- correct definition of usage
- add explicit fallthrough notice. The existing one doesn't work with
our selection of "implicit-fallthrough" strictness.

This results in WARNS=6 building on amd64, but not other arches
2018-06-24 05:40:42 +00:00
Ed Maste
d8ba45e213 Revert r313780 (UFS_ prefix) 2018-03-17 12:59:55 +00:00
Ed Maste
1e2b9afca9 Prefix UFS symbols with UFS_ to reduce namespace pollution
Followup to r313780.  Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by:	kib
Reviewed by:	kib, mckusick
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9623
2018-03-17 01:48:27 +00:00
Kirk McKusick
068beacf21 The goal of this change is to prevent accidental foot shooting by
folks running filesystems created on check-hash enabled kernels
(which I will call "new") on a non-check-hash enabled kernels (which
I will call "old). The idea here is to detect when a filesystem is
run on an old kernel and flag the filesystem so that when it gets
moved back to a new kernel, it will not start getting a slew of
check-hash errors.

Back when the UFS version 2 filesystem was created, it added a file
flag FS_INDEXDIRS that was to be set on any filesystem that kept
some sort of on-disk indexing for directories. The idea was precisely
to solve the issue we have today. Specifically that a newer kernel
that supported indexing would be able to tell that the filesystem
had been run on an older non-indexing kernel and that the indexes
should not be used until they had been rebuilt. Since we have never
implemented on-disk directory indicies, the FS_INDEXDIRS flag is
cleared every time any UFS version 2 filesystem ever created is
mounted for writing.

This commit repurposes the FS_INDEXDIRS flag as the FS_METACKHASH
flag. Thus, the FS_METACKHASH is definitively known to have always
been cleared. The FS_INDEXDIRS flag has been moved to a new block
of flags that will always be cleared starting with this commit
(until they get used to implement some future feature which needs
to detect that the filesystem was mounted on a kernel that predates
the new feature).

If a filesystem with check-hashes enabled is mounted on an old
kernel the FS_METACKHASH flag is cleared. When that filesystem is
mounted on a new kernel it will see that the FS_METACKHASH has been
cleared and clears all of the fs_metackhash flags. To get them
re-enabled the user must run fsck (in interactive mode without the
-y flag) which will ask for each supported check hash whether it
should be rebuilt and enabled. When fsck is run in its default preen
mode, it will just ignore the check hashes so they will remain
disabled.

The kernel has always disabled any check hash functions that it
does not support, so as more types of check hashes are added, we
will get a non-surprising result. Specifically if filesystems get
moved to kernels supporting fewer of the check hashes, those that
are not supported will be disabled. If the filesystem is moved back
to a kernel with more of the check-hashes available and fsck is run
interactively to rebuild them, then their checking will resume.
Otherwise just the smaller subset will be checked.

A side effect of this commit is that filesystems running with
cylinder-group check hashes will stop having them checked until
fsck is run to re-enable them (since none of them currently have
the FS_METACKHASH flag set). So, if you want check hashes enabled
on your filesystems after booting a kernel with these changes, you
need to run fsck to enable them. Any newly created filesystems will
have check hashes enabled. If in doubt as to whether you have check
hashes emabled, run dumpfs and look at the list of enabled flags
at the end of the superblock details.
2018-02-08 23:06:58 +00:00
Kirk McKusick
8bd0b5ce0a Check and report error returns from sbput(3) calls.
Convert to using cgput(3) for writing cylinder groups.
Check and report error returns from cgput(3).

Submitted by: Bruce Evans <bde@freebsd.org>
2018-02-02 23:26:52 +00:00
Kirk McKusick
dffce2150e Refactoring of reading and writing of the UFS/FFS superblock.
Specifically reading is done if ffs_sbget() and writing is done
in ffs_sbput(). These functions are exported to libufs via the
sbget() and sbput() functions which then used in the various
filesystem utilities. This work is in preparation for adding
subperblock check hashes.

No functional change intended.

Reviewed by: kib
2018-01-26 00:58:32 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00
Scott Long
7841fefb62 Rename P_OSREL_CK_CLYGRP to P_OSREL_CK_CYLGRP 2017-11-17 13:12:20 +00:00
Warner Losh
a3c15a4445 Only try to enable CK_CLYGRP if we're running on kernel newer than
1200046, the first version that supports this feature. If we set it,
then use an old kernel, we'll break the 'contract' of having
checksummed cylinder groups this flag signifies. To avoid creating
something with an inconsistent state, don't turn the flag on in these
cases. The first full fsck with a new kernel will turn this on.

Spnsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D13114
2017-11-16 21:28:14 +00:00
Ed Maste
4acb68a8a5 newfs: warn if newer than kernel
Creating a UFS filesystem with a newfs newer than the running kernel,
and then mounting that filesystem, can lead to interesting failures.

Add a safety belt to explicitly warn when newfs is newer than the
running kernel.

Reviewed by:	gjb, jhb, mckusick
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D12765
2017-11-15 18:40:40 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Kirk McKusick
75e3597abb Continuing efforts to provide hardening of FFS, this change adds a
check hash to cylinder groups. If a check hash fails when a cylinder
group is read, no further allocations are attempted in that cylinder
group until it has been fixed by fsck. This avoids a class of
filesystem panics related to corrupted cylinder group maps. The
hash is done using crc32c.

Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
used in embedded systems with small memories and low-powered processors
which need as light-weight a filesystem as possible.

Specifics of the changes:

sys/sys/buf.h:
    Add BX_FSPRIV to reserve a set of eight b_xflags that may be used
    by individual filesystems for their own purpose. Their specific
    definitions are found in the header files for each filesystem
    that uses them. Also add fields to struct buf as noted below.

sys/kern/vfs_bio.c:
    It is only necessary to compute a check hash for a cylinder
    group when it is actually read from disk. When calling bread,
    you do not know whether the buffer was found in the cache or
    read. So a new flag (GB_CKHASH) and a pointer to a function to
    perform the hash has been added to breadn_flags to say that the
    function should be called to calculate a hash if the data has
    been read. The check hash is placed in b_ckhash and the B_CKHASH
    flag is set to indicate that a read was done and a check hash
    calculated. Though a rather elaborate mechanism, it should
    also work for check hashing other metadata in the future. A
    kernel internal API change was to change breada into a static
    fucntion and add flags and a function pointer to a check-hash
    function.

sys/ufs/ffs/fs.h:
    Add flags for types of check hashes; stored in a new word in the
    superblock. Define corresponding BX_ flags for the different types
    of check hashes. Add a check hash word in the cylinder group.

sys/ufs/ffs/ffs_alloc.c:
    In ffs_getcg do the dance with breadn_flags to get a check hash and
    if one is provided, check it.

sys/ufs/ffs/ffs_vfsops.c:
    Copy across the BX_FFSTYPES flags in background writes.
    Update the check hash when writing out buffers that need them.

sys/ufs/ffs/ffs_snapshot.c:
    Recompute check hash when updating snapshot cylinder groups.

sys/libkern/crc32.c:
lib/libufs/Makefile:
lib/libufs/libufs.h:
lib/libufs/cgroup.c:
    Include libkern/crc32.c in libufs and use it to compute check
    hashes when updating cylinder groups.

Four utilities are affected:

sbin/newfs/mkfs.c:
    Add the check hashes when building the cylinder groups.

sbin/fsck_ffs/fsck.h:
sbin/fsck_ffs/fsutil.c:
    Verify and update check hashes when checking and writing cylinder groups.

sbin/fsck_ffs/pass5.c:
    Offer to add check hashes to existing filesystems.
    Precompute check hashes when rebuilding cylinder group
    (although this will be done when it is written in fsutil.c
    it is necessary to do it early before comparing with the old
    cylinder group)

sbin/dumpfs/dumpfs.c
    Print out the new check hash flag(s)

sbin/fsdb/Makefile:
    Needs to add libufs now used by pass5.c imported from fsck_ffs.

Reviewed by: kib
Tested by: Peter Holm (pho)
2017-09-22 12:45:15 +00:00
Kirk McKusick
855662c611 The new fsck recovery information to enable it to find backup
superblocks created in revision 322297 only works on disks
with sector sizes up to 4K. This update allows the recovery
information to be created by newfs and used by fsck on disks
with sector sizes up to 64K. Note that FFS currently limits
filesystem to be mounted from disks with up to 8K sectors.
Expanding this limitation will be the subject of another
commit.

Reported by: Peter Holm
Reviewed with: kib
2017-09-04 20:19:36 +00:00
Kirk McKusick
77b63aa0fc Since the switch to GPT disk labels, fsck for UFS/FFS has been
unable to automatically find alternate superblocks. This checkin
places the information needed to find alternate superblocks to the
end of the area reserved for the boot block.

Filesystems created with a newfs of this vintage or later will
create the recovery information. If you have a filesystem created
prior to this change and wish to have a recovery block created for
your filesystem, you can do so by running fsck in forground mode
(i.e., do not use the -p or -y options). As it starts, fsck will
ask ``SAVE DATA TO FIND ALTERNATE SUPERBLOCKS'' to which you should
answer yes.

Discussed with: kib, imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D11589
2017-08-09 05:17:21 +00:00
Warner Losh
bbbfb2a922 Bump date for today's commit. 2017-07-07 16:58:40 +00:00
Warner Losh
1e001b99a5 Improve wording for -E and -t flags. -E never writes the entire disk,
so don't imply that. Note that if BIO_DELETE isn't supported, the
operation will fail (as opposed to writing the entire disk with
zeros). Thin storage also benefits from trim. List more accurate
reason why trim helps flash-memory.
2017-07-07 16:54:18 +00:00
Kirk McKusick
aeb2785c30 Allow '_' in labels when specifying -L to newfs.
Reported by: Keve Nagy
Reviewed by: kib
PR: 220163
MFC after: 5 days
2017-06-20 21:26:42 +00:00
Enji Cooper
22289a8c3d sbin: normalize paths using SRCTOP-relative paths or :H when possible
This simplifies make logic/output

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 11:33:01 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Ed Maste
1dc349ab95 prefix UFS symbols with UFS_ to reduce namespace pollution
Specifically:
  ROOTINO -> UFS_ROOTINO
  WINO -> UFS_WINO
  NXADDR -> UFS_NXADDR
  NDADDR -> UFS_NDADDR
  NIADDR -> UFS_NIADDR
  MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Also prefix ext2's and nandfs's NDADDR and NIADDR with EXT2_ and NANDFS_

Reviewed by:	kib, mckusick
Obtained from:	NetBSD
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9536
2017-02-15 19:50:26 +00:00
Marcelo Araujo
0193043c46 Use MIN()/MAX() macros from sys/param.h.
Reviewed by:	trasz
MFC after:	2 weeks.
Differential Revision:	https://reviews.freebsd.org/D6118
2016-05-02 00:45:46 +00:00
Marcelo Araujo
edf6b683e8 Use NULL instead of 0 for pointers.
strchr(3) will return NULL if the character does not appear in the
string.

MFC after:	2 weeks.
2016-04-18 14:08:35 +00:00
Edward Tomasz Napierala
b8c19fd719 It's 2015, and some people are still trying to use fdisk and then
go asking what debug flags to set for GEOM to make it work.  Advice
them to use gpart(8) instead.

Something similar should probably done with disklabel,
but I need to rewrite the disklabel examples first.

Reviewed by:	wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3315
2015-09-02 14:08:43 +00:00
Marcelo Araujo
9ae278b077 Code cleanup unused-but-set-variable spotted by gcc.
Approved by:	bapt (mentor)
Differential Revision:	D3475
2015-08-25 15:33:09 +00:00
John-Mark Gurney
57198f08d3 fix the docs, the number of frags per inode (NFPI) changed in r228794
to 2 from 4, but the man page didn't get updated...

other minor changes to make igor happy...

MFC after:	3 days
2015-07-15 21:35:09 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Warner Losh
a91275f72f Remove old ioctl use and support, once and for all. 2015-01-06 05:28:37 +00:00