Commit Graph

98 Commits

Author SHA1 Message Date
Kirk McKusick
baa12a84a7 The purpose of this change to the FFS layout policy is to reduce the
running time for a full fsck. It also reduces the random access time
for large files and speeds the traversal time for directory tree walks.

The key idea is to reserve a small area in each cylinder group
immediately following the inode blocks for the use of metadata,
specifically indirect blocks and directory contents. The new policy
is to preferentially place metadata in the metadata area and
everything else in the blocks that follow the metadata area.

The size of this area can be set when creating a filesystem using
newfs(8) or changed in an existing filesystem using tunefs(8).
Both utilities use the `-k held-for-metadata-blocks' option to
specify the amount of space to be held for metadata blocks in each
cylinder group. By default, newfs(8) sets this area to half of
minfree (typically 4% of the data area).

This work was inspired by a paper presented at Usenix's FAST '13:
www.usenix.org/conference/fast13/ffsck-fast-file-system-checker

Details of this implementation appears in the April 2013 of ;login:
www.usenix.org/publications/login/april-2013-volume-38-number-2.
A copy of the April 2013 ;login: paper can also be downloaded
from: www.mckusick.com/publications/faster_fsck.pdf.

Reviewed by: kib
Tested by:   Peter Holm
MFC after:   4 weeks
2013-03-22 21:45:28 +00:00
Edward Tomasz Napierala
549f62fa42 Fix problem with geom_label(4) not recognizing UFS labels on filesystems
extended using growfs(8).  The problem here is that geom_label checks if
the filesystem size recorded in UFS superblock is equal to the provider
(i.e. device) size.  This check cannot be removed due to backward
compatibility.  On the other hand, in most cases growfs(8) cannot set
fs_size in the superblock to match the provider size, because, differently
from newfs(8), it cannot recompute cylinder group sizes.

To fix this problem, add another superblock field, fs_providersize, used
only for this purpose.  The geom_label(4) will attach if either fs_size
(filesystem created with newfs(8)) or fs_providersize (filesystem expanded
using growfs(8)) matches the device size.

PR:		kern/165962
Reviewed by:	mckusick
Sponsored by:	FreeBSD Foundation
2012-10-30 21:32:10 +00:00
Kirk McKusick
d92f0739ab Add the -j option to enable soft updates journaling when creating
a new file system.

Reviewed by: Kostik Belousov <kostikbel@gmail.com>
2011-02-16 06:00:27 +00:00
Konstantin Belousov
a738d4cf20 Add support for FS_TRIM to user-mode UFS utilities.
Reviewed by:	mckusick, pjd, pho
Tested by:	pho
MFC after:	1 month
2010-12-29 12:31:18 +00:00
Konstantin Belousov
03ee10d8e4 Add the missed 'p' flag to getopt() optstring argument.
MFC after:	1 week
2010-12-15 12:45:28 +00:00
Maxim Sobolev
e0999e592b o bdeficize expand_number_int() function;
o revert most of the recent changes (int -> int64_t conversion) by using
this functon for parsing all options.
2010-03-09 19:31:08 +00:00
Maxim Sobolev
eb8d193be2 Change secrorsize back to int, since that's the data type expected by the
ioctl(DIOCGSECTORSIZE). It creates issues on some architectures.

MFC after:	1 week
Reported by:	Jayachandran C.
2010-03-09 10:31:03 +00:00
Maxim Sobolev
32bdc2b685 Use expand_number(3) from libutil instead of home-grown function to parse
human-friendly power-of-two numbers (i.e. 2k, 5M etc).

Suggested by:	many
MFC after:	1 week
2010-03-03 19:25:28 +00:00
Maxim Sobolev
83b5ab2770 Teach newfs(8) to understand size modifiers for all options taking
size or size-like argument. I.e. "-s 32k" instead of "-s 32768".
Size parsing function has been shamelessly stolen from the truncate(1).
I'm sure many sysadmins out there will appreciate this small
improvement.

MFC after:	1 week
2010-03-03 02:05:09 +00:00
Kirk McKusick
cb464c69c0 Ensure that newfs will never create a filesystem with more than 2^32
inodes by cutting back on the number of inodes per cylinder group if
necessary to stay under the limit. For a default (16K block) file
system, this limit begins to take effect for file systems above 32Tb.

This fix is in addition to -r203763 which corrected a problem in the
kernel that treated large inode numbers as negative rather than unsigned.
For a default (16K block) file system, this bug began to show up at a
file system size above about 16Tb.

Reported by: Scott Burns, John Kilburg, Bruce Evans
Followup by: Jeff Roberson
PR:          133980
MFC after:   2 weeks
2010-02-10 20:17:46 +00:00
Xin LI
e475c594e4 Correct two typos.
Reported by:	Brandon Falk <falkman gamozo org>
MFC after:	1 week
2010-02-06 00:25:46 +00:00
Olivier Houchard
02dda28606 Don't add a bwrite() symbol, it breaks the build when building newfs
statically.
Instead, bring in a stripped down version of sbwrite(), and add the offset
to every bwrite() calls.
2009-02-12 15:28:15 +00:00
Luigi Rizzo
59cf6deb68 Move the check for the ending char in the partition name where
it was before -- the check is only made when getdisklabel()
returns valid info.
On passing, use MAXPARTITIONS to identify the max partition number,
instead of the hardwired 'h'

MFC after:	4 weeks
2008-12-12 15:56:38 +00:00
Luigi Rizzo
64c8fef580 Enable operation of newfs on plain files, which is useful when you
want to prepare disk images for emulators (though 'makefs' in port
can do something similar).

This relies on:
+ minor changes to pass the consistency checks even when working on a file;

+ an additional option, '-p partition' , to specify the disk partition to
  initialize;

+ some changes on the I/O routines to deal with partition offsets.

The latter was a bit tricky to implement, see the details in newfs.h:
in newfs, I/O is done through libufs which assumes that the file
descriptor refers to the whole partition. Introducing support for
the offset in libufs would require a non-backward compatible change
in the library, to be dealt with a version bump or with symbol
versioning.

I felt both approaches to be overkill for this specific application,
especially because there might be other changes to libufs that might
become necessary in the near future.

So I used the following trick:
- read access is always done by calling bread() directly, so we just add
  the offset in the (few) places that call bread();
- write access is done through bwrite() and sbwrite(), which in turn
  calls bwrite(). To avoid rewriting sbwrite(), we supply our own version
  of bwrite() here, which takes precedence over the version in libufs.

MFC after:	4 weeks
2008-12-03 18:36:59 +00:00
Poul-Henning Kamp
9a6378d803 Rename the undocumented -E option to -X.
Implement -E option which will erase the filesystem sectors before
making the new filesystem.  Reserved space in front of the superblock
(bootcode) is not erased.

NB: Erasing can take as long time as writing every sector sequentially.

This is relevant for all flash based disks which use wearlevelling.
2007-12-16 19:41:31 +00:00
Yaroslav Tykhiy
3249f70d0f - Pay attention to the fact that ioctl(2) is only known to
return -1 on error while any other return value from it can
indicate success.  (See RETURN VALUE in our ioctl(2) manpage
and the POSIX spec.)

- Avoid assumptions about the state of the data buffer after
ioctl(2) failure.
2007-11-28 07:54:42 +00:00
Yaroslav Tykhiy
35956d32df MFp4:
Add a new option to newfs(8), -r, to specify reserved space at the
end of the device.  It can be useful, e.g., when the device is to
become a member of a gmirror array later w/o losing the file system
on it.

Document the new option in the manpage.

While I'm here, improve error handling for -s option, which is
syntactically similar to -r; and document the fact that -s0 selects
the default fs size explicitly, which can be useful, e.g., in a
menu-based wrapper around newfs(8) requiring some value be entered
for the fs size.

Also fix a small typo in the help line for -s (missing space).

Idea and initial implementation by:	marck
Discussed on:				-fs
Critical review by:			bde
Tested with:				cmp(1)
2007-11-28 07:29:10 +00:00
Pawel Jakub Dawidek
b8f6a34f3e Document -J in usage.
Submitted by:	Eric Anderson <anderson@freebsd.org>
2007-03-02 20:07:59 +00:00
Pawel Jakub Dawidek
868c68ed1d Add -J flag to both newfs(8) and tunefs(8) which allows to enable gjournal
support.
I left -j flag for UFS journal implementation which we may gain at some
point.

Sponsored by:	home.pl
2006-10-31 21:52:28 +00:00
Ruslan Ermilov
d591eb90f7 Document -l and -n options in usage(). 2005-01-22 14:37:57 +00:00
Wes Peters
34b59b6bf2 Add an option to suppress the creation of the .snap directory in
the new filesystem.  This is intended for memory and vnode filesystems
that will never be fsck'ed or dumped.

Obtained from:	St. Bernard Software RAPID
MFC after:	2 weeks
2005-01-21 22:20:25 +00:00
Pawel Jakub Dawidek
e075f345b6 Cast to intmax_t when using %jd format.
MFC after:	3 days
2005-01-08 17:19:56 +00:00
Pawel Jakub Dawidek
f4d2631187 Fix '-s' option for large disks and fix printing maximum file system size. 2004-09-19 10:01:51 +00:00
Mark Murray
4c723140a4 Remove advertising clause from University of California Regent's license,
per letter dated July 22, 1999.

Approved by: core, imp
2004-04-09 19:58:40 +00:00
Robert Watson
ce20d788fa Add a "-l" flag to newfs, which sets the FS_MULTILABEL flag. This
permits users of newfs to set the multilabel flag on UFS1 and UFS2
file systems from inception without using tunefs.

Obtained from:	TrustedBSD Project
Sponsored by:	DARPA, McAfee Research
2004-02-26 01:14:27 +00:00
Wes Peters
0af4e34b2e Add the -E command line option to force error conditions for testing.
Sponsord by:	St. Bernard Software
2003-11-16 07:17:30 +00:00
David E. O'Brien
c69284ca08 Use __FBSDID() to quiet GCC 3.3 warnings. 2003-05-03 18:41:59 +00:00
Robert Watson
b459937e0c Throw the switch--change to UFS2 as our default file system format for
FreeBSD 5.1-RELEASE and later:

- newfs(8) will now create UFS2 file systems unless UFS1 is specifically
  requested (-O1).  To do this, I just twiddled the Oflag default.

- sysinstall(8) will now select UFS2 as the default layout for new
  file systems unless specifically requested (use '1' and '2' to change
  the file system layout in the disk labeler).  To do this, I inverted
  the ufs2 flag into a ufs1 flag, since ufs2 is now the default and
  ufs1 is the edge case.  There's a slight semantic change in the
  key behavior: '2' no longer toggles, it changes the selection to UFS2.

This is very similar to a patch David O'Brien sent me at one point, and
that I couldn't find.

Approved by:	re (telecon)
Reviewed by:	mckusick, phk, bmah
2003-04-20 14:08:05 +00:00
Kirk McKusick
363c185255 Correct lines incorrectly added to the copyright message. Add missing period.
Submitted by:	Bruce Evans <bde@zeta.org.au>
Sponsored by:   DARPA & NAI Labs.
2003-02-14 21:08:14 +00:00
Juli Mallett
fc903aa525 Convert newfs to libufs (really). Solves one real issue with previous
version of such.  Differences in filesystems generated were found to be
from 1) sbwrite with the "all" parameter 2) removal of writecache.  The
sbwrite call was made to perform as the original version, and otherwise
this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
Gordon Tetlow
c715b047bc Bring in support for volume labels to the filesystem utilities.
Reviewed by:	mckusick
2003-02-01 04:17:10 +00:00
Juli Mallett
5a29754e3f Back out conversion to libufs, for now. It seems to cause problems.
Reported by: phk
2003-01-29 22:52:27 +00:00
Juli Mallett
9d492cddac Convert newfs to use libufs. I've tested this on md filesystems, as has
keramida, and all seems well.
2003-01-27 07:24:32 +00:00
Kirk McKusick
41e20344a2 Add some more checks to newfs so that it will not build filesystems
that the kernel will refuse to mount. Specifically it now enforces
the MAXBSIZE blocksize limit. This update also fixes a problem where
newfs could segment fault if the selected fragment size was too large.

PR:		bin/30959
Submitted by:	Ceri Davies <setantae@submonkey.net>
Sponsored by:   DARPA & NAI Labs.
2002-11-30 18:28:26 +00:00
Poul-Henning Kamp
3f8322d6b8 Remove a comma trailing an if clause.
According to Kirk: "Luckily, the statement is usually true".

Spotted by:	FlexeLint
2002-10-01 17:31:28 +00:00
Poul-Henning Kamp
1851342297 Failure to rewrite the disklabel should not be fatal.
Sponsored by:	DARPA & NAI Labs.
2002-09-22 09:41:41 +00:00
Tom Rhodes
ce66ddb763 s/filesystem/file system/g as discussed on -developers 2002-08-21 18:11:48 +00:00
Kirk McKusick
1c85e6a35d This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by:	Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
Tom Rhodes
3468b317cb more file system > filesystem 2002-05-16 04:10:46 +00:00
Poul-Henning Kamp
4ab9c1d126 Remove the -v option, it is now default behaviour.
Sponsored by:	DARPA & NAI Labs
2002-04-24 12:27:03 +00:00
Poul-Henning Kamp
9aba332745 Continue the cleanup preparations for UFS2 (& GEOM):
Use only one filedescriptor.  Open in R/O or R/W based in the '-N' option.
Make the filedescriptor a global variable instead of passing it around
as semi-global variable(s).

Remove the undocumented ability to specify type without '-T' option.

Replace fatal() with straight err(3)/errx(3).  Save calls to strerror()
where applicable.  Loose the progname variable.

Get the sense of the cpgflag test correct so we only issue warnings if
people specify cpg and can't get that.  It can be argued that this
should be an error.

Remove the check to see if the disk is mounted:  Open for writing
would fail if it were mounted.

Attempt to get the sectorsize and mediasize with the generic disk
ioctls, fall back to disklabel and /etc/disktab as we can.

Notice that on-disk labels still take precedence over /etc/disktab,
this is probably wrong, but not as wrong as the entire concept of
/etc/disktab is.

Sponsored by:   DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
Poul-Henning Kamp
ebdb43a2f8 bbsize and sbsize cannot ever be trusted from the disklabel, in
particular as there may not be one.  Remove #if 0'ed code which might
mislead people to think otherwise.

unifdef -ULOSTDIR, fsck can make lost+found on the fly.

Sponsored by:	DARPA & NAI Labs
2002-04-07 14:57:57 +00:00
Bruce Evans
73c36e3672 Fixed some style bugs in axings. Whitespace before __P was not axed when
__P was axed.  The ordering of several things was bogotified by axing
ifdefs.
2002-04-04 09:56:51 +00:00
Poul-Henning Kamp
fa8d75e0a1 Unifdef -DCOMPAT 2002-04-03 19:53:09 +00:00
Poul-Henning Kamp
5dccd5c649 Swing the axe and remove some archaic features from newfs which modern
diskdrives do neither need nor want:

	-O create a 4.3BSD format filesystem
	-d rotational delay between contiguous blocks
	-k sector 0 skew, per track
	-l hardware sector interleave
	-n number of distinguished rotational positions
	-p spare sectors per track
	-r revolutions/minute
	-t tracks/cylinder
	-x spare sectors per cylinder

No change in the produced filesystem image unless one or more of
these options were used.

Approved by:	mckusick
2002-03-20 07:16:15 +00:00
Poul-Henning Kamp
89fb8ee796 Add the undocumented -R option to disable randomness for regression-testing.
Add a couple of simple regression tests accessible with "make test", they
depend on the md(4) driver.

FYI I have also tried running the test against a week old newfs and it
passed.
2002-03-19 21:05:29 +00:00
Poul-Henning Kamp
8409849dd0 Further cleanups. 2002-03-19 20:01:38 +00:00
Ian Dowse
bf57cced53 Complete the ANSIfication of newfs by converting function declarations
to C89 style.
2002-03-19 17:20:02 +00:00
Ian Dowse
9710700cb1 Remove the ancient STANDALONE code.
Approved by:	phk
2002-03-19 16:47:20 +00:00
Ian Dowse
af53d6d86e Remove yet more vestiges of mount_mfs. 2002-03-18 15:31:44 +00:00