Commit Graph

286 Commits

Author SHA1 Message Date
Xin LI
3a6ab3de8d Explicitly say which gid do we use as a fallback, when operator
is not found.

Suggested by:	kensmith
2006-09-27 05:49:21 +00:00
Ian Dowse
9405aea2e2 Don't treat failure to find the operator GID as a fatal error; this
made it impossible to use newfs (and mdmfs) when /etc/group is
missing and /etc is read-only.
2005-08-14 17:07:04 +00:00
Xin LI
3ae329b8d2 When creating a new FFS file system, the block size will indirectly
affect the largest file size that is allowed by the file system.
On the other hand, when creating a snapshot, the snapshot file will
appear as it is as big as the file system itself.  Hence we will not
be able to create a file system on large file systems with small
block sizes.

Add a warning about this, and gives some hints to correct the issue.

Reviewed by:	mckusick
MFC After:	1 week
2005-02-20 06:33:18 +00:00
Ruslan Ermilov
d591eb90f7 Document -l and -n options in usage(). 2005-01-22 14:37:57 +00:00
Ruslan Ermilov
a1761aecf5 Polish previous revision:
- Bump document date.
- Spell "file system" properly.
- Add missing markup bits.
2005-01-22 14:36:51 +00:00
Wes Peters
34b59b6bf2 Add an option to suppress the creation of the .snap directory in
the new filesystem.  This is intended for memory and vnode filesystems
that will never be fsck'ed or dumped.

Obtained from:	St. Bernard Software RAPID
MFC after:	2 weeks
2005-01-21 22:20:25 +00:00
Pawel Jakub Dawidek
e075f345b6 Cast to intmax_t when using %jd format.
MFC after:	3 days
2005-01-08 17:19:56 +00:00
Pawel Jakub Dawidek
f4d2631187 Fix '-s' option for large disks and fix printing maximum file system size. 2004-09-19 10:01:51 +00:00
John Baldwin
b72ea57f3b Generalize the UFS bad magic value used to determine when a filesystem
has only been partly initialized via newfs(8) so that it applies to both
UFS1 and UFS2.

Submitted by:	"Xin LI" delphij at frontfree dot net
MFC:		maybe?
2004-08-19 11:09:13 +00:00
Ruslan Ermilov
d04b5dfe6c Assorted markup, grammar, and spelling fixes. 2004-05-17 08:35:43 +00:00
Mark Murray
4c723140a4 Remove advertising clause from University of California Regent's license,
per letter dated July 22, 1999.

Approved by: core, imp
2004-04-09 19:58:40 +00:00
Robert Watson
ce20d788fa Add a "-l" flag to newfs, which sets the FS_MULTILABEL flag. This
permits users of newfs to set the multilabel flag on UFS1 and UFS2
file systems from inception without using tunefs.

Obtained from:	TrustedBSD Project
Sponsored by:	DARPA, McAfee Research
2004-02-26 01:14:27 +00:00
Wes Peters
96982f9bfd Fix whitespace error in previous commit.
Approved by:	RE@ (Robert Watson)
2003-11-27 01:19:23 +00:00
Wes Peters
f44ec7f89e Don't use UFS2_BAD_MAGIC on UFS (v1) filesystems; it is Not Ready
for Prime Time there.

Submitted by:	Xin LI <delphij@frontfree.net>
Approved by:	RE@ (John, Scott)
2003-11-23 08:29:01 +00:00
Wes Peters
0af4e34b2e Add the -E command line option to force error conditions for testing.
Sponsord by:	St. Bernard Software
2003-11-16 07:17:30 +00:00
Wes Peters
ec52df8eb9 Write the UFS2 superblock with a 'BAD' magic number at the beginning
of newfs, to signify the newfs operation has not yet completed.  Re-
write the superblock with the correct magic number once all of the
cylinder groups have been created to show the operation has finished.

Sponsored by:	St. Bernard Software
2003-11-16 07:08:27 +00:00
Kirk McKusick
524ee1107f Create a .snap directory mode 770 group operator in the root of
a new filesystem. Dump and fsck will create snapshots in this
directory rather than in the root for two reasons:

1) For terabyte-sized filesystems, the snapshot may require many
   minutes to build. Although the filesystem will not be suspended
   during most of the snapshot build, the snapshot file itself is
   locked during the entire snapshot build period. Thus, if it is
   accessed during the period that it is being built, the process
   trying to access it will block holding its containing directory
   locked. If the snapshot is in the root, the root will lock and
   the system will come to a halt until the snapshot finishes. By
   putting the snapshot in a subdirectory, it is out of the likely
   path of any process traversing through the root and hence much
   less likely to cause a lock race to the root.

2) The dump program is usually run by a non-root user running with
   operator group privilege. Such a user is typically not permitted
   to create files in the root of a filesystem. By having a directory
   in group operator with group write access available, such a user
   will be able to create a snapshot there. Having the dump program
   create its snapshot in a subdirectory below the root will benefit
   from point (1) as well.

Sponsored by:   DARPA & NAI Labs.
2003-11-04 07:34:32 +00:00
Marc Fonvieille
8b23842d38 s/disklabel/bsdlabel where needed. 2003-10-11 08:24:07 +00:00
Ceri Davies
2918f54c0a Remove an unneccessary comma. 2003-09-14 20:35:22 +00:00
Ruslan Ermilov
fe08efe680 mdoc(7): Use the new feature of the .In macro. 2003-09-08 19:57:22 +00:00
Yaroslav Tykhiy
244fca1ffa Exit with a non-zero status upon a block allocation failure.
The old way of just returning could result in a file system
extremely likely to panic the kernel.  The warning printed
wouldn't help much since tools invoking newfs(8), e.g., mdmfs(8),
couldn't detect the error.

PR:		bin/55078
MFC after:	1 week
2003-08-05 13:35:17 +00:00
Doug Barton
a32bb1b53a When newfs'ing a partition with UFS2 that had previously been newfs'ed
with UFS1, the UFS1 superblocks were not deleted. This allowed any
RELENG_4 (or other non-UFS2-aware) fsck to think it knew how to "fix"
the file system, resulting in severe data scrambling.

This patch is a more advanced version than the one originally submitted.
Lukas improved it based on feedback from Kirk, and testing by me. It
blanks all UFS1 superblocks (if any) during a UFS2 newfs, thereby causing
fsck's that are not UFS2 aware to generate the "SEARCH FOR ALTERNATE
SUPER-BLOCK FAILED" message, and exit without damaging the fs.

PR:		bin/51619
Submitted by:	Lukas Ertl <l.ertl@univie.ac.at>
Reviewed by:	kirk
Approved by:	re (scottl)
2003-05-22 18:38:54 +00:00
Ian Dowse
7bdf1805b1 Put back the error checking in wtfs() that was lost when newfs was
changed to use libufs in revision 1.71. Without this, any write
failures in newfs were silently ignored.

Note that this will display a meaningless errno string in the case
of a short write as opposed to a write error, since bwrite()'s
return value does not allow the caller to determine if errno is
valid.

Reported by:	Lukas Ertl <l.ertl@univie.ac.at>
Reviewed by:	jmallett
Approved by:	re (bmah)
2003-05-10 18:58:17 +00:00
David E. O'Brien
c69284ca08 Use __FBSDID() to quiet GCC 3.3 warnings. 2003-05-03 18:41:59 +00:00
Christian Brueffer
d4cb71fc53 Remove reference to diskpart(8)
PR:		51193
Submitted by:	Yonatan@xpert.com
2003-04-20 19:16:21 +00:00
Robert Watson
b459937e0c Throw the switch--change to UFS2 as our default file system format for
FreeBSD 5.1-RELEASE and later:

- newfs(8) will now create UFS2 file systems unless UFS1 is specifically
  requested (-O1).  To do this, I just twiddled the Oflag default.

- sysinstall(8) will now select UFS2 as the default layout for new
  file systems unless specifically requested (use '1' and '2' to change
  the file system layout in the disk labeler).  To do this, I inverted
  the ufs2 flag into a ufs1 flag, since ufs2 is now the default and
  ufs1 is the edge case.  There's a slight semantic change in the
  key behavior: '2' no longer toggles, it changes the selection to UFS2.

This is very similar to a patch David O'Brien sent me at one point, and
that I couldn't find.

Approved by:	re (telecon)
Reviewed by:	mckusick, phk, bmah
2003-04-20 14:08:05 +00:00
Ruslan Ermilov
522ccf3f35 mdoc(7) police: markup laundry. 2003-02-23 01:47:49 +00:00
Kirk McKusick
e27c9f46ad Fix the -R flag so that it provides sequential "random" numbers
so that the regression test will succeed.

Sponsored by:   DARPA & NAI Labs.
2003-02-22 23:26:11 +00:00
John W. De Boskey
66227a4409 Our first keyword hit for apropos ufs2. 2003-02-19 02:41:29 +00:00
Kirk McKusick
aca3e4974f Replace use of random() with arc4random() to provide less guessable
values for the initial inode generation numbers in newfs and for
newly allocated inode generation numbers in the kernel.

Submitted by:	Theo de Raadt <deraadt@cvs.openbsd.org>
Sponsored by:   DARPA & NAI Labs.
2003-02-14 21:31:58 +00:00
Kirk McKusick
363c185255 Correct lines incorrectly added to the copyright message. Add missing period.
Submitted by:	Bruce Evans <bde@zeta.org.au>
Sponsored by:   DARPA & NAI Labs.
2003-02-14 21:08:14 +00:00
Juli Mallett
fc903aa525 Convert newfs to libufs (really). Solves one real issue with previous
version of such.  Differences in filesystems generated were found to be
from 1) sbwrite with the "all" parameter 2) removal of writecache.  The
sbwrite call was made to perform as the original version, and otherwise
this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
Gordon Tetlow
c715b047bc Bring in support for volume labels to the filesystem utilities.
Reviewed by:	mckusick
2003-02-01 04:17:10 +00:00
Juli Mallett
5a29754e3f Back out conversion to libufs, for now. It seems to cause problems.
Reported by: phk
2003-01-29 22:52:27 +00:00
Juli Mallett
9d492cddac Convert newfs to use libufs. I've tested this on md filesystems, as has
keramida, and all seems well.
2003-01-27 07:24:32 +00:00
Kirk McKusick
33493b1820 Correctly calculate the initial number of fragments in a filesystem
so that fsck does not complain with `SUMMARY BLK COUNT(S) WRONG IN
SUPERBLK' the first time it is run on a new filesystem.

Reported by:	Poul-Henning Kamp <phk@freebsd.org>
Sponsored by:   DARPA & NAI Labs.
2002-12-02 19:31:53 +00:00
Kirk McKusick
41e20344a2 Add some more checks to newfs so that it will not build filesystems
that the kernel will refuse to mount. Specifically it now enforces
the MAXBSIZE blocksize limit. This update also fixes a problem where
newfs could segment fault if the selected fragment size was too large.

PR:		bin/30959
Submitted by:	Ceri Davies <setantae@submonkey.net>
Sponsored by:   DARPA & NAI Labs.
2002-11-30 18:28:26 +00:00
Kirk McKusick
ada981b228 Create a new 32-bit fs_flags word in the superblock. Add code to move
the old 8-bit fs_old_flags to the new location the first time that the
filesystem is mounted by a new kernel. One of the unused flags in
fs_old_flags is used to indicate that the flags have been moved.
Leave the fs_old_flags word intact so that it will work properly if
used on an old kernel.

Change the fs_sblockloc superblock location field to be in units
of bytes instead of in units of filesystem fragments. The old units
did not work properly when the fragment size exceeeded the superblock
size (8192). Update old fs_sblockloc values at the same time that
the flags are moved.

Suggested by:	BOUWSMA Barry <freebsd-misuser@netscum.dyndns.dk>
Sponsored by:   DARPA & NAI Labs.
2002-11-27 02:18:58 +00:00
Kirk McKusick
59a825617f Properly calculate the initial number of fragments in a large filesystem.
Sponsored by:   DARPA & NAI Labs.
2002-11-15 23:50:14 +00:00
Kirk McKusick
ecfc865a4b Bound the size of the superblock to SBLOCKSIZE.
Submitted by:	BOUWSMA Beery <freebsd-misuser@netscum.dyndns.dk>
Sponsored by:	DARPA & NAI Labs.
2002-10-18 23:17:30 +00:00
Poul-Henning Kamp
3f8322d6b8 Remove a comma trailing an if clause.
According to Kirk: "Luckily, the statement is usually true".

Spotted by:	FlexeLint
2002-10-01 17:31:28 +00:00
Mike Barcroft
89fdc4e117 Use the standardized CHAR_BIT constant instead of NBBY in userland. 2002-09-25 04:06:37 +00:00
Poul-Henning Kamp
1851342297 Failure to rewrite the disklabel should not be fatal.
Sponsored by:	DARPA & NAI Labs.
2002-09-22 09:41:41 +00:00
Tom Rhodes
5af4935a22 Fix some 'SYNOPSIS' and 'usage' messages. 2002-08-27 00:49:27 +00:00
Tom Rhodes
ce66ddb763 s/filesystem/file system/g as discussed on -developers 2002-08-21 18:11:48 +00:00
Ruslan Ermilov
3e40554f06 mdoc(7) police: Consistently name options' arguments in the SYNOPSIS and DESCRIPTION sections.. 2002-08-13 13:13:20 +00:00
Ollivier Robert
752c7d0862 di_createtime -> di_birthtime.
Submitted by:	Udo Schweigert <Udo.Schweigert@siemens.com>
2002-07-17 10:31:38 +00:00
Bruce Evans
ffcaf36b2c Fixed 4 printf format errors that were fatal on alphas. %qd is not even
suitable for printing quad_t's since it is equivalent to %lld but quad_t
is unsigned long on alphas.  quad_t shouldn't be used anyway.
2002-07-11 17:49:41 +00:00
Philippe Charnier
e1205e80e5 The .Nm utility 2002-07-06 19:34:18 +00:00
Kirk McKusick
5e5d87ff36 Get rid of paranoia that zeros the boot block area as this has
bad effect on existing bootstraps.

Submitted by:	Jake Burkholder <jake@locore.ca>
Sponsored by:	DARPA & NAI Labs.
2002-06-22 22:44:09 +00:00
Kirk McKusick
1c85e6a35d This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by:	Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
Tom Rhodes
3468b317cb more file system > filesystem 2002-05-16 04:10:46 +00:00
Poul-Henning Kamp
4ab9c1d126 Remove the -v option, it is now default behaviour.
Sponsored by:	DARPA & NAI Labs
2002-04-24 12:27:03 +00:00
Poul-Henning Kamp
9aba332745 Continue the cleanup preparations for UFS2 (& GEOM):
Use only one filedescriptor.  Open in R/O or R/W based in the '-N' option.
Make the filedescriptor a global variable instead of passing it around
as semi-global variable(s).

Remove the undocumented ability to specify type without '-T' option.

Replace fatal() with straight err(3)/errx(3).  Save calls to strerror()
where applicable.  Loose the progname variable.

Get the sense of the cpgflag test correct so we only issue warnings if
people specify cpg and can't get that.  It can be argued that this
should be an error.

Remove the check to see if the disk is mounted:  Open for writing
would fail if it were mounted.

Attempt to get the sectorsize and mediasize with the generic disk
ioctls, fall back to disklabel and /etc/disktab as we can.

Notice that on-disk labels still take precedence over /etc/disktab,
this is probably wrong, but not as wrong as the entire concept of
/etc/disktab is.

Sponsored by:   DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
Poul-Henning Kamp
ebdb43a2f8 bbsize and sbsize cannot ever be trusted from the disklabel, in
particular as there may not be one.  Remove #if 0'ed code which might
mislead people to think otherwise.

unifdef -ULOSTDIR, fsck can make lost+found on the fly.

Sponsored by:	DARPA & NAI Labs
2002-04-07 14:57:57 +00:00
Bruce Evans
73c36e3672 Fixed some style bugs in axings. Whitespace before __P was not axed when
__P was axed.  The ordering of several things was bogotified by axing
ifdefs.
2002-04-04 09:56:51 +00:00
Bruce Evans
a2f4e30c8c Fixed some English errors in previous commit.
Fixed some style bugs in the removal of __P(()).  Whitespace before
"__P((" was not removed.
2002-04-04 09:45:11 +00:00
Poul-Henning Kamp
1f35193bdc Add more DWIM/autoadjustment and less evil style(9) banned exit(2) codes.
Add some missing statics.

Sponsored by: DARPA & NAI Labs.
2002-04-03 20:48:05 +00:00
Poul-Henning Kamp
fa8d75e0a1 Unifdef -DCOMPAT 2002-04-03 19:53:09 +00:00
Poul-Henning Kamp
5dccd5c649 Swing the axe and remove some archaic features from newfs which modern
diskdrives do neither need nor want:

	-O create a 4.3BSD format filesystem
	-d rotational delay between contiguous blocks
	-k sector 0 skew, per track
	-l hardware sector interleave
	-n number of distinguished rotational positions
	-p spare sectors per track
	-r revolutions/minute
	-t tracks/cylinder
	-x spare sectors per cylinder

No change in the produced filesystem image unless one or more of
these options were used.

Approved by:	mckusick
2002-03-20 07:16:15 +00:00
Poul-Henning Kamp
89fb8ee796 Add the undocumented -R option to disable randomness for regression-testing.
Add a couple of simple regression tests accessible with "make test", they
depend on the md(4) driver.

FYI I have also tried running the test against a week old newfs and it
passed.
2002-03-19 21:05:29 +00:00
Poul-Henning Kamp
8409849dd0 Further cleanups. 2002-03-19 20:01:38 +00:00
Ian Dowse
475df34ac2 Replace a number of similar for' loops with a new ilog2()' function
that computes the base-2 log of a power of 2.
2002-03-19 17:39:01 +00:00
Ian Dowse
bf57cced53 Complete the ANSIfication of newfs by converting function declarations
to C89 style.
2002-03-19 17:20:02 +00:00
Ian Dowse
f7b48c89c8 The FSIRAND code is always compiled in, and it is unlikely that
anyone needs a newfs without it. Remove the #ifdef's from around
the code and the -DFSIRAND from the Makefile. Also remove redundant
declarations of random() and srandomdev().
2002-03-19 17:03:14 +00:00
Ian Dowse
9710700cb1 Remove the ancient STANDALONE code.
Approved by:	phk
2002-03-19 16:47:20 +00:00
Ian Dowse
af53d6d86e Remove yet more vestiges of mount_mfs. 2002-03-18 15:31:44 +00:00
Bruce Evans
0c08079e26 Fixed some style bugs (mainly ones not fixed or made worse by rev.1.44).
Don't use ISO string concatentation to obfuscate long single-line
messages...
2002-03-18 03:04:58 +00:00
Bruce Evans
63dab85cea Fixed some style bugs (mainly ones not fixed or made worse by rev.1.41).
Old code obfuscates long (but single-line) messages by printing them in
pieces using %s.  Rev.1.41 obfuscated some new long messages using ISO
string concatenation.  This commit only fixes the new obfuscations.
2002-03-18 02:43:14 +00:00
Bruce Evans
3ac7f11229 Removed vestiges of mount_mfs. Sorted the Makefile a bit. 2002-03-18 02:23:43 +00:00
Bruce Evans
3f305db0a2 Fixed 2 layers of breakage of WARNS. Setting WARNS unconditionally to
0 was bad and setting it unconditionally to 2 was worse.
2002-03-18 02:13:38 +00:00
Poul-Henning Kamp
345b78a301 Remove __P() and register.
Set WARNS=2

This is the beginning of a pre-UFS2 cleanup of newfs.

Sponsored by:	DARPA, NAI Labs
2002-03-17 09:01:41 +00:00
Sheldon Hearn
3626f83327 Update the default newfs block and fragment sizes from 8192/1024 to
16384/2048.

Following recent discussions on the -arch mailing list, involving dillon
and mckusick, this change parallels the one made over a decade ago when
the default was bumped up from 4096/512.

This should provide significant performance improvements for most
folks, less significant performance losses for a few folks and
wasted space lost to large fragments for many folks.

For discussion, please see the following thread in the -arch archive:

Subject: Using a larger block size on large filesystems

The discussion ceases to be relevant when the issue of partitioning
schemes is raised.
2001-12-11 16:21:40 +00:00
Sheldon Hearn
be42c56328 Fix typo: 'fragement' -> 'fragment' 2001-12-07 13:18:28 +00:00
David E. O'Brien
2d68bf45bf Default to WARNS=2.
Binary builds that cannot handle this must explicitly set WARNS=0.

Reviewed by:	mike
2001-12-04 02:19:58 +00:00
Sheldon Hearn
629c25d4a9 Fix the example of suggested default settings. It stated that settings
were only of benefit to large filesystems, which recent research
suggests is not the case, and which the original author of the text
no longer endorses.
2001-11-27 19:39:07 +00:00
Sheldon Hearn
2441e154ef Correct the example introduced in rev 1.29, which suggested a block:frag
size ratio other than 8:1.  Currently, we only recommend an 8:1
ratio, because the impact of others ratios has not been adequately
investigated.

Also, do not recommend the use of the -c option in the example, since
newfs now automatically calculates the best cyl:cylgrp ratio.

This change was discussed with the author of rev 1.29.
2001-11-27 17:01:17 +00:00
Peter Wemm
1165817956 Remove support for FreeBSD/tahoe
Submitted by:	phk
2001-11-03 08:35:11 +00:00
Poul-Henning Kamp
bfd1f63d44 style(9) cleanup.
Submitted by:	j mckitrick <jcm@freebsd-uk.eu.org>
Reviewed by:	phk, /sbin/md5
2001-11-02 09:23:34 +00:00
Ollivier Robert
4f63c70a2b Fix diskless clients by removing the code for calculating the minimum
value for cpg. The change was bogus.

Submitted by:	bde
MFC after:	2 days
2001-10-18 09:48:28 +00:00
Doug Barton
eb9d1d2716 Document the optimal block:fragment ratio, per discussion
on -arch and cvs-all.

Reviewed by:	dillon
2001-10-15 22:47:55 +00:00
Ollivier Robert
08870345f5 Following the discussion in -arch and the submission of a patch by bde, here
it is. I added the manpage change.

Submitted by:	bde
MFC after:	1 week
2001-10-04 12:15:50 +00:00
Brian Somers
9cfe90fe1f Handle snprintf() returning < 0 (not just -1)
MFC after:	2 weeks
2001-08-20 14:53:05 +00:00
Brian Somers
327e849ae1 Handle snprintf() returning -1.
MFC after:	2 weeks
2001-08-20 12:56:45 +00:00
Kris Kennaway
5979df34a6 Silence non-constant format string warnings by marking functions
as __printflike()/__printf0like(), adding const, or adding missing "%s"
format strings, as appropriate.

MFC after:	2 weeks
2001-08-19 08:19:37 +00:00
Dima Dorfman
7ebcc426ef Remove whitespace at EOL. 2001-07-15 07:53:42 +00:00
Ruslan Ermilov
9fe48c6e8d mdoc(7) police: removed HISTORY info from the .Os call. 2001-07-10 11:04:34 +00:00
Dima Dorfman
70d51341bf mdoc(7) police: remove extraneous .Pp before and/or after .Sh. 2001-07-09 09:54:33 +00:00
Bruce Evans
e38c0bd643 Don't link ${BINDIR}/newfs to nowhere.
Don't clutter this Makefile (not to mention the error output) with
$(BDECFLAGS}.
2001-05-30 09:31:24 +00:00
Dima Dorfman
f628a4b10e Remove -DMFS from CFLAGS. 2001-05-29 23:57:23 +00:00
Dima Dorfman
7ccb741f6a Remove all references to MFS. 2001-05-29 23:55:43 +00:00
Poul-Henning Kamp
80f86e526b A more complete removal of MFS related code.
XXX: This program badly needs a style(9) + BDECFLAGS treatment.
2001-05-29 19:40:39 +00:00
Poul-Henning Kamp
a383ba34c2 Initial cleanout of MFS from newfs. More complete wash needed. 2001-05-29 18:52:39 +00:00
Kris Kennaway
1fef4cc97d sprintf() -> snprintf()
Partially submitted by:	"Andrew R. Reiter" <arr@watson.org>
Obtained from:	OpenBSD
2001-04-24 10:26:00 +00:00
Kris Kennaway
c3b1df1293 Add a missing argument to an error message format string. 2001-04-17 07:21:48 +00:00
Nik Clayton
044479f5ad Add information about the new options to newfs and tunefs which set the
expected average file size and number of files per directory.  Could do
with some fleshing out.
2001-04-10 10:36:44 +00:00
Kirk McKusick
a61ab64ac4 Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>.
His description of the problem and solution follow. My own tests show
speedups on typical filesystem intensive workloads of 5% to 12% which
is very impressive considering the small amount of code change involved.

------

  One day I noticed that some file operations run much faster on
small file systems then on big ones. I've looked at the ffs
algorithms, thought about them, and redesigned the dirpref algorithm.

  First I want to describe the results of my tests. These results are old
and I have improved the algorithm after these tests were done. Nevertheless
they show how big the perfomance speedup may be. I have done two file/directory
intensive tests on a two OpenBSD systems with old and new dirpref algorithm.
The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports".
The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release.
It contains 6596 directories and 13868 files. The test systems are:

1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for
   test is at wd1. Size of test file system is 8 Gb, number of cg=991,
   size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current
   from Dec 2000 with BUFCACHEPERCENT=35

2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system
   at wd0, file system for test is at wd1. Size of test file system is 40 Gb,
   number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k
   OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50

You can get more info about the test systems and methods at:
http://www.ptci.ru/gluk/dirpref/old/dirpref.html

                              Test Results

             tar -xzf ports.tar.gz               rm -rf ports
  mode  old dirpref new dirpref speedup old dirprefnew dirpref speedup
                             First system
 normal     667         472      1.41       477        331       1.44
 async      285         144      1.98       130         14       9.29
 sync       768         616      1.25       477        334       1.43
 softdep    413         252      1.64       241         38       6.34
                             Second system
 normal     329         81       4.06       263.5       93.5     2.81
 async      302         25.7    11.75       112          2.26   49.56
 sync       281         57.0     4.93       263         90.5     2.9
 softdep    341         40.6     8.4        284          4.76   59.66

"old dirpref" and "new dirpref" columns give a test time in seconds.
speedup - speed increasement in times, ie. old dirpref / new dirpref.

------

Algorithm description

The old dirpref algorithm is described in comments:

/*
 * Find a cylinder to place a directory.
 *
 * The policy implemented by this algorithm is to select from
 * among those cylinder groups with above the average number of
 * free inodes, the one with the smallest number of directories.
 */

A new directory is allocated in a different cylinder groups than its
parent directory resulting in a directory tree that is spreaded across
all the cylinder groups. This spreading out results in a non-optimal
access to the directories and files. When we have a small filesystem
it is not a problem but when the filesystem is big then perfomance
degradation becomes very apparent.

What I mean by a big file system ?

  1. A big filesystem is a filesystem which occupy 20-30 or more percent
     of total drive space, i.e. first and last cylinder are physically
     located relatively far from each other.
  2. It has a relatively large number of cylinder groups, for example
     more cylinder groups than 50% of the buffers in the buffer cache.

The first results in long access times, while the second results in
many buffers being used by metadata operations. Such operations use
cylinder group blocks and on-disk inode blocks. The cylinder group
block (fs->fs_cblkno) contains struct cg, inode and block bit maps.
It is 2k in size for the default filesystem parameters. If new and
parent directories are located in different cylinder groups then the
system performs more input/output operations and uses more buffers.
On filesystems with many cylinder groups, lots of cache buffers are
used for metadata operations.

My solution for this problem is very simple. I allocate many directories
in one cylinder group. I also do some things, so that the new allocation
method does not cause excessive fragmentation and all directory inodes
will not be located at a location far from its file's inodes and data.
The algorithm is:
/*
 * Find a cylinder group to place a directory.
 *
 * The policy implemented by this algorithm is to allocate a
 * directory inode in the same cylinder group as its parent
 * directory, but also to reserve space for its files inodes
 * and data. Restrict the number of directories which may be
 * allocated one after another in the same cylinder group
 * without intervening allocation of files.
 *
 * If we allocate a first level directory then force allocation
 * in another cylinder group.
 */

  My early versions of dirpref give me a good results for a wide range of
file operations and different filesystem capacities except one case:
those applications that create their entire directory structure first
and only later fill this structure with files.

  My solution for such and similar cases is to limit a number of
directories which may be created one after another in the same cylinder
group without intervening file creations. For this purpose, I allocate
an array of counters at mount time. This array is linked to the superblock
fs->fs_contigdirs[cg]. Each time a directory is created the counter
increases and each time a file is created the counter decreases. A 60Gb
filesystem with 8mb/cg requires 10kb of memory for the counters array.

  The maxcontigdirs is a maximum number of directories which may be created
without an intervening file creation. I found in my tests that the best
performance occurs when I restrict the number of directories in one cylinder
group such that all its files may be located in the same cylinder group.
There may be some deterioration in performance if all the file inodes
are in the same cylinder group as its containing directory, but their
data partially resides in a different cylinder group. The maxcontigdirs
value is calculated to try to prevent this condition. Since there is
no way to know how many files and directories will be allocated later
I added two optimization parameters in superblock/tunefs. They are:

        int32_t  fs_avgfilesize;   /* expected average file size */
        int32_t  fs_avgfpdir;      /* expected # of files per directory */

These parameters have reasonable defaults but may be tweeked for special
uses of a filesystem. They are only necessary in rare cases like better
tuning a filesystem being used to store a squid cache.

I have been using this algorithm for about 3 months. I have done
a lot of testing on filesystems with different capacities, average
filesize, average number of files per directory, and so on. I think
this algorithm has no negative impact on filesystem perfomance. It
works better than the default one in all cases. The new dirpref
will greatly improve untarring/removing/coping of big directories,
decrease load on cvs servers and much more. The new dirpref doesn't
speedup a compilation process, but also doesn't slow it down.

Obtained from:	Grigoriy Orlov <gluk@ptci.ru>
2001-04-10 08:38:59 +00:00
Bruce Evans
5f98b5af89 Fixed style bugs in previous commit. 2001-04-03 09:35:36 +00:00
David E. O'Brien
8af4736afe Document the newfs.c rev 1.33 changing the default c/g from 16 to 22. 2001-04-02 22:48:54 +00:00
David E. O'Brien
991bf32193 Fix patch merge braino. 2001-04-02 22:46:02 +00:00
David E. O'Brien
b2cd1ce8ee Allow enabling soft updates (with -U) on a new filesystem.
[I first added this functionality, and thought to check prior art.  Seeing
OpenBSD had already done this, I changed my addition to reduce the diffs
between the two and went with their option letter.]
Obtained from:	OpenBSD
2001-04-02 01:25:55 +00:00
David E. O'Brien
094ab93715 The common wisdom is to use the largest number of cylinders per group.
So bump the default from `16' to `22', which is the largest value allowed
with the current default block size.  This change increases the the
group size from 32MB/g to 44MB/g on a 4GB SCSI disk.
2001-03-27 01:34:58 +00:00
Ruslan Ermilov
0a5779d45b - Backout botched attempt to introduce MANSECT feature.
- MAN[1-9] -> MAN.
2001-03-26 14:33:27 +00:00
Ruslan Ermilov
fe655281c5 Set the default manual section for sbin/ to 8. 2001-03-20 18:13:31 +00:00
Poul-Henning Kamp
7cf109c1b4 Make mount_mfs annoy users for 15 seconds and point them at mdconfig(8). 2001-01-30 10:21:20 +00:00
Ruslan Ermilov
9884911506 mdoc(7) police: fixed broken references. 2001-01-16 11:52:00 +00:00
Ian Dowse
f55ff3f3ef The ffs superblock includes a 128-byte region for use by temporary
in-core pointers to summary information. An array in this region
(fs_csp) could overflow on filesystems with a very large number of
cylinder groups (~16000 on i386 with 8k blocks). When this happens,
other fields in the superblock get corrupted, and fsck refuses to
check the filesystem.

Solve this problem by replacing the fs_csp array in 'struct fs'
with a single pointer, and add padding to keep the length of the
128-byte region fixed. Update the kernel and userland utilities
to use just this single pointer.

With this change, the kernel no longer makes use of the superblock
fields 'fs_csshift' and 'fs_csmask'. Add a comment to newfs/mkfs.c
to indicate that these fields must be calculated for compatibility
with older kernels.

Reviewed by:	mckusick
2001-01-15 18:30:40 +00:00
Eivind Eklund
10185bdc57 Replace reference to replacing mkfs(8) with a paragraph actually
describing what newfs *does*.
2001-01-15 03:13:26 +00:00
Ruslan Ermilov
d90d7015f9 Prepare for mdoc(7)NG. 2000-12-27 14:40:52 +00:00
Warner Losh
64dee60f8c o Add an example for a large file system.
o Remove bug about boot blocks hating non-8k file systems.  This hasn't been
  the case for a long time.

Not Objected to by: hackers, doc
2000-12-19 21:55:07 +00:00
Ruslan Ermilov
7c7fb079b9 mdoc(7) police: use the new features of the Nm macro. 2000-11-20 16:52:27 +00:00
Ruslan Ermilov
726b61ab5f Avoid use of direct troff requests in mdoc(7) manual pages. 2000-11-10 17:46:15 +00:00
John W. De Boskey
929f494bc7 Cast block number to off_t to avoid possible overflow bugs.
Pointed out by: Bruce Evans <bde@zeta.org.au>
2000-10-24 03:28:59 +00:00
John W. De Boskey
45c29d5cda The write combining code in revision 1.30 needs a few additional
touch ups.  The cache needs to be flushed against block
reads, and a final flush at process termination to force the
backup superblocks to disk.

I believe this will allow 'make release' to complete.

Submitted by:	Tor.Egge@fast.no
2000-10-24 00:08:30 +00:00
Peter Wemm
3927beeda1 Implement simple write combining for newfs - this is particularly useful
for large scsi disks with WCE = 0.  This yields around a 7 times speedup
on elapsed newfs time on test disks here.  64k clusters seems to be the
sweet spot for scsi disks using our present drivers.
2000-10-17 00:41:36 +00:00
Poul-Henning Kamp
fc87418be0 Turn dkcksum() into an __inline function.
Change its type to u_int_16_t.
2000-09-16 13:43:00 +00:00
Mike Smith
6801d96c70 Don't try to do anything with the /dev/rXXX device. 2000-05-31 01:00:51 +00:00
Sheldon Hearn
ef8f7ac935 Remove single-space hard sentence breaks. These degrade the quality
of the typeset output, tend to make diffs harder to read and provide
bad examples for new-comers to mdoc.
2000-03-01 11:27:47 +00:00
Kris Kennaway
d9d67bbaf8 Add Xref to camcontrol(8) (replacing previously-removed scsiformat(8)).
Submitted by:	joerg
2000-01-30 20:58:33 +00:00
Mike Pritchard
f050f700ad Fix various man pages to stop abusing the .Bx macro to generate
the strings "FreeBSD" and "NetBSD".  Use the .Fx or .Nx macro
instead.
2000-01-23 01:30:05 +00:00
Kris Kennaway
3c7fbe12d1 Remove dead xref to scsiformat(8)
Obtained from: OpenBSD (kind of)
1999-11-15 02:56:34 +00:00
Peter Wemm
7f3dea244c $Id$ -> $FreeBSD$ 1999-08-28 00:22:10 +00:00
Bill Fumerola
5682c39f91 Don't print a "," after the last superblock.
Submitted by:	adrian
1999-08-21 22:07:27 +00:00
Nick Hibma
de4a801355 Add again the ':' after the x option in th eargument list to getopt.
It disappeared in rev. 1.23 newfs.c

PR: 12292
Submitted by: Cy Schubert <cy@cschuber.net.gov.bc.ca>
1999-06-19 13:32:27 +00:00
Greg Lehey
63a134bc15 Describe the default values for useful options.
Clarify which options are no longer useful.
1999-03-10 21:59:02 +00:00
Matthew Dillon
cb84cdb1c4 Fix bug in mount_mfs whereby mount_mfs would sometimes return before
the mount is completely active, causing the next few commands attempting
    to manipulate data on the mount to fail.  mount_mfs's parent now tries
    to wait for the mount point st_dev to change before returning, indicating
    that the mount has gone active.
1999-02-09 17:19:19 +00:00
Bruce Evans
e14589db82 Straightened the terminology straightening in 1.17-1.18. Fixed hard
line breaks in rev.1.16-1.18.
1998-11-29 13:09:01 +00:00
Robert Nordier
c276ec5a07 Refer to "da" rather than "sd" device. 1998-11-28 10:02:52 +00:00
Bruce Evans
8116455e9d Backed out previous commit. It broke fsck again. See rev.1.22 and the
references there, and rev.1.38 of sys/ufs/ufs/ufs_disksubr.c.
1998-10-17 08:03:52 +00:00
Jordan K. Hubbard
7b955a8e5d Don't rewrite the disk label. The type field is already set correctly
and we don't use the frags info, so why bother?  More to the point, it
seems to result in an EXDEV error when the label is written out and we
lose because of it (don't know why though).  This is a work-around and
is marked as such.
1998-10-17 04:19:29 +00:00
Greg Lehey
e34129e8ba Correct source file corruption in last checkin
Observed by:  jkh
1998-09-30 07:53:52 +00:00
Greg Lehey
11ee80a991 Don't require an argument for -v flag
Correct checks for null special file names
Add Usage entry for -v flag
Get terminology straight in man page
Reviewed by:	bde
1998-09-29 23:20:04 +00:00
Greg Lehey
f7fb3ee05b Reviewed by: bde (again)
Correct terminology (partitions are in slices, not the other way around)
1998-09-11 07:08:49 +00:00
Greg Lehey
58343e4cc6 Reviewed by: bde,jkh
Add -v flag to newfs:

     -v      Specify that the partition does not contain any slices, and that
             newfs should treat the whole partition as the file system.  This
             option is useful for synthetic disks such as ccd and vinum.
1998-09-11 06:26:08 +00:00
Doug Rabson
48145dcdb4 Use explicitly sized types when formatting cylinder groups. 1998-08-27 07:38:33 +00:00
Philippe Charnier
e791a73343 Forgot to remove a ';' in my previous commit. 1998-08-12 06:07:43 +00:00
Bruce Evans
519e76df7c Backed out rev.1.9 (except don't bring back the vax code deleted
in rev.1.9).  fsck uses the per-partition ffs-related information
in the label to find alternate superblocks when the main superblock
is hosed.  Rev.1.9 broke this by deleting the code that wrote the
label.

PR:		2537
xref:		fsck/setup.c rev.1.8
1998-07-20 12:04:42 +00:00
Philippe Charnier
16e80a4182 Make it compile again in the !__STDC__ case.
Found by: Bruce.
1998-07-16 12:04:52 +00:00
Philippe Charnier
27750b35a5 Add prototypes. Check malloc() return value. Use err(). Remove unused #includes
Do not \n nor dot terminate syslog()/err() messages. -Wall.
1998-07-15 06:28:05 +00:00
Bruce Evans
ca46ad5f48 Fixed printf format errors. 1998-06-28 20:11:23 +00:00
Bruce Evans
5a3e77d851 Fixed overflow in the calculation of the number of inodes per group
for filesystems with almost the maximum number of sectors.  The maxiumum
is 2^31, but overflow is common for that size, and overflow normally
occurred here at size (2^31 - 4096).
1998-05-31 12:21:50 +00:00
Poul-Henning Kamp
e72f5a72fd Add warning about root-fs blocksize expectations.
PR:		4485
Reviewed by:	phk
Submitted by:	Kees Jan Koster <kjk1@ukc.ac.uk>
1998-04-26 17:44:23 +00:00
Bruce Evans
87f01287d2 Removed definition of _NEW_VFSCONF. The new vfsconf interface is now
the default.
1998-01-20 10:40:18 +00:00
Bruce Evans
90e05a748a Don't create superblocks with size larger than SBSIZE (8192). The
size was rounded up to a multiple of the fragment size, but this
gave invalid file systems when the fragment size was > SBSIZE (fsck
aborts early on them).  Now a fragment size of 32768 seems to work
(too-simple tests with fsck and iozone worked).
1998-01-19 16:55:26 +00:00
Bruce Evans
6e1a705dd4 Fixed some spelling errors. 1998-01-16 06:31:23 +00:00
David E. O'Brien
e83b2033a7 Fix syntax error for the mount_mfs example.
Also update to a more realistic example.
1997-11-24 03:28:54 +00:00
Peter Wemm
079709dc3a Some tweaks to get this to cope with ELF where the address space starts
higher up in memory (0x0800000 upwards) rather than near zero (0x1000
for our qmagic a.out format).  The method that mount_mfs uses to allocate
the memory within data size rlimits for the ram disk is entirely too much
of a kludge for my liking.  I mean, if it's run as root, surely it makes
sense to just raise the resource limits to infinity or something, and if
it's a non-root user mount (do these work? with mfs?) it could just fail
if it's outside limits.
1997-09-13 11:41:50 +00:00
Bruce Evans
28596d2346 Removed "hack to prevent overflow of a 32bit block number". Lite2 has a
better hack in ffs_vfsops.c.  The hack here restricted the maximum file
size to 2^39 bytes (512GB).  fs_bsize * 2^31 - 1 (16TB for the default
blocksize of 8K) would have been better.  There is no good way to remove
this limit on old BSD4.4 file systems.
1997-07-13 15:13:07 +00:00
Tor Egge
dc6f8d1bd5 Allow use of the name "swap" instead of an actual swap device.
This makes configuration of mfs /tmp on diskless clients more intuitive
for people like me, that have used this feature on NetBSD and SunOS.
Using the -T option and /dev/null, while already supported,
is neither intuitive nor documented in the handbook.
Obtained from: NetBSD
1997-06-23 22:44:51 +00:00
Andrey A. Chernov
545cda7db5 Remove srandomdev fallback 1997-06-14 00:17:53 +00:00