d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1bb11493f3
freebsd-skq/sbin/newfs/newfs.c

435 lines
13 KiB
C
Raw Normal View History

BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
* Copyright (c) 2002 Networks Associates Technology, Inc.
* All rights reserved.
*
* This software was developed for the FreeBSD Project by Marshall
* Kirk McKusick and Network Associates Laboratories, the Security
* Research Division of Network Associates, Inc. under DARPA/SPAWAR
* contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS
Correct lines incorrectly added to the copyright message. Add missing period. Submitted by: Bruce Evans <bde@zeta.org.au> Sponsored by: DARPA & NAI Labs.
2003-02-14 21:08:14 +00:00
* research program.
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
*
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
* Copyright (c) 1983, 1989, 1993, 1994
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
Use __FBSDID() to quiet GCC 3.3 warnings.
2003-05-03 18:41:59 +00:00
#if 0
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#ifndef lint
Add prototypes. Check malloc() return value. Use err(). Remove unused #includes Do not \n nor dot terminate syslog()/err() messages. -Wall.
1998-07-15 06:28:05 +00:00
static const char copyright[] =
"@(#) Copyright (c) 1983, 1989, 1993, 1994\n\
The Regents of the University of California. All rights reserved.\n";
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#endif /* not lint */
#ifndef lint
Add prototypes. Check malloc() return value. Use err(). Remove unused #includes Do not \n nor dot terminate syslog()/err() messages. -Wall.
1998-07-15 06:28:05 +00:00
static char sccsid[] = "@(#)newfs.c 8.13 (Berkeley) 5/1/95";
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#endif /* not lint */
Use __FBSDID() to quiet GCC 3.3 warnings.
2003-05-03 18:41:59 +00:00
#endif
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
* newfs: friendly front end to mkfs
*/
#include <sys/param.h>
#include <sys/stat.h>
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
#include <sys/disk.h>
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#include <sys/disklabel.h>
#include <sys/file.h>
#include <sys/mount.h>
Make it compile again in the !__STDC__ case. Found by: Bruce.
1998-07-16 12:04:52 +00:00
#include <ufs/ufs/dir.h>
#include <ufs/ufs/dinode.h>
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#include <ufs/ffs/fs.h>
Merge from Lite2: - use new getvfsbyname() and mount(2) interface (mount_mfs) - use new fs include files - updated inode / cg layout calculations (?)
1997-03-11 12:48:17 +00:00
#include <ufs/ufs/ufsmount.h>
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#include <ctype.h>
Add prototypes. Check malloc() return value. Use err(). Remove unused #includes Do not \n nor dot terminate syslog()/err() messages. -Wall.
1998-07-15 06:28:05 +00:00
#include <err.h>
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#include <errno.h>
#include <paths.h>
Fixed some style bugs in axings. Whitespace before __P was not axed when __P was axed. The ordering of several things was bogotified by axing ifdefs.
2002-04-04 09:56:51 +00:00
#include <stdarg.h>
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <syslog.h>
#include <unistd.h>
Further cleanups.
2002-03-19 20:01:38 +00:00
#include "newfs.h"
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
* The following two constants set the default block and fragment sizes.
* Both constants must be a power of 2 and meet the following constraints:
* MINBSIZE <= DESBLKSIZE <= MAXBSIZE
* sectorsize <= DESFRAGSIZE <= DESBLKSIZE
* DESBLKSIZE / DESFRAGSIZE <= 8
*/
Update the default newfs block and fragment sizes from 8192/1024 to 16384/2048. Following recent discussions on the -arch mailing list, involving dillon and mckusick, this change parallels the one made over a decade ago when the default was bumped up from 4096/512. This should provide significant performance improvements for most folks, less significant performance losses for a few folks and wasted space lost to large fragments for many folks. For discussion, please see the following thread in the -arch archive: Subject: Using a larger block size on large filesystems The discussion ceases to be relevant when the issue of partitioning schemes is raised.
2001-12-11 16:21:40 +00:00
#define DFL_FRAGSIZE 2048
#define DFL_BLKSIZE 16384
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
* Cylinder groups may have up to MAXBLKSPERCG blocks. The actual
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
* number used depends upon how much information can be stored
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
* in a cylinder group map which must fit in a single file system
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
* block. The default is to use as many as possible blocks per group.
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
*/
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
#define MAXBLKSPERCG 0x7fffffff /* desired fs_fpg ("infinity") */
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
* MAXBLKPG determines the maximum number of data blocks which are
* placed in a single cylinder group. The default is one indirect
* block worth of data blocks.
*/
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
#define MAXBLKPG(bsize) ((bsize) / sizeof(ufs2_daddr_t))
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
* Each file system has a number of inodes statically allocated.
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
* We allocate one inode slot per NFPI fragments, expecting this
* to be far more than we will ever need.
*/
#define NFPI 4
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
int Lflag; /* add a volume label */
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
int Nflag; /* run without writing file system */
Throw the switch--change to UFS2 as our default file system format for FreeBSD 5.1-RELEASE and later: - newfs(8) will now create UFS2 file systems unless UFS1 is specifically requested (-O1). To do this, I just twiddled the Oflag default. - sysinstall(8) will now select UFS2 as the default layout for new file systems unless specifically requested (use '1' and '2' to change the file system layout in the disk labeler). To do this, I inverted the ufs2 flag into a ufs1 flag, since ufs2 is now the default and ufs1 is the edge case. There's a slight semantic change in the key behavior: '2' no longer toggles, it changes the selection to UFS2. This is very similar to a patch David O'Brien sent me at one point, and that I couldn't find. Approved by: re (telecon) Reviewed by: mckusick, phk, bmah
2003-04-20 14:08:05 +00:00
int Oflag = 2; /* file system format (1 => UFS1, 2 => UFS2) */
Add the undocumented -R option to disable randomness for regression-testing. Add a couple of simple regression tests accessible with "make test", they depend on the md(4) driver. FYI I have also tried running the test against a week old newfs and it passed.
2002-03-19 21:05:29 +00:00
int Rflag; /* regression test */
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
int Uflag; /* enable soft updates for file system */
Add the -E command line option to force error conditions for testing. Sponsord by: St. Bernard Software
2003-11-16 07:17:30 +00:00
int Eflag = 0; /* exit in middle of newfs for testing */
Add a "-l" flag to newfs, which sets the FS_MULTILABEL flag. This permits users of newfs to set the multilabel flag on UFS1 and UFS2 file systems from inception without using tunefs. Obtained from: TrustedBSD Project Sponsored by: DARPA, McAfee Research
2004-02-26 01:14:27 +00:00
int lflag; /* enable multilabel for file system */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
quad_t fssize; /* file system size */
int sectorsize; /* bytes/sector */
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
int realsectorsize; /* bytes/sector in hardware */
int fsize = 0; /* fragment size */
int bsize = 0; /* block size */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
int maxbsize = 0; /* maximum clustering */
int maxblkspercg = MAXBLKSPERCG; /* maximum blocks per cylinder group */
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
int minfree = MINFREE; /* free space threshold */
int opt = DEFAULTOPT; /* optimization preference (space or time) */
int density; /* number of bytes per inode */
int maxcontig = 0; /* max contiguous blocks to allocate */
int maxbpg; /* maximum blocks per file in a cyl group */
Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>. His description of the problem and solution follow. My own tests show speedups on typical filesystem intensive workloads of 5% to 12% which is very impressive considering the small amount of code change involved. ------ One day I noticed that some file operations run much faster on small file systems then on big ones. I've looked at the ffs algorithms, thought about them, and redesigned the dirpref algorithm. First I want to describe the results of my tests. These results are old and I have improved the algorithm after these tests were done. Nevertheless they show how big the perfomance speedup may be. I have done two file/directory intensive tests on a two OpenBSD systems with old and new dirpref algorithm. The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports". The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release. It contains 6596 directories and 13868 files. The test systems are: 1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for test is at wd1. Size of test file system is 8 Gb, number of cg=991, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=35 2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system at wd0, file system for test is at wd1. Size of test file system is 40 Gb, number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50 You can get more info about the test systems and methods at: http://www.ptci.ru/gluk/dirpref/old/dirpref.html Test Results tar -xzf ports.tar.gz rm -rf ports mode old dirpref new dirpref speedup old dirprefnew dirpref speedup First system normal 667 472 1.41 477 331 1.44 async 285 144 1.98 130 14 9.29 sync 768 616 1.25 477 334 1.43 softdep 413 252 1.64 241 38 6.34 Second system normal 329 81 4.06 263.5 93.5 2.81 async 302 25.7 11.75 112 2.26 49.56 sync 281 57.0 4.93 263 90.5 2.9 softdep 341 40.6 8.4 284 4.76 59.66 "old dirpref" and "new dirpref" columns give a test time in seconds. speedup - speed increasement in times, ie. old dirpref / new dirpref. ------ Algorithm description The old dirpref algorithm is described in comments: /* * Find a cylinder to place a directory. * * The policy implemented by this algorithm is to select from * among those cylinder groups with above the average number of * free inodes, the one with the smallest number of directories. */ A new directory is allocated in a different cylinder groups than its parent directory resulting in a directory tree that is spreaded across all the cylinder groups. This spreading out results in a non-optimal access to the directories and files. When we have a small filesystem it is not a problem but when the filesystem is big then perfomance degradation becomes very apparent. What I mean by a big file system ? 1. A big filesystem is a filesystem which occupy 20-30 or more percent of total drive space, i.e. first and last cylinder are physically located relatively far from each other. 2. It has a relatively large number of cylinder groups, for example more cylinder groups than 50% of the buffers in the buffer cache. The first results in long access times, while the second results in many buffers being used by metadata operations. Such operations use cylinder group blocks and on-disk inode blocks. The cylinder group block (fs->fs_cblkno) contains struct cg, inode and block bit maps. It is 2k in size for the default filesystem parameters. If new and parent directories are located in different cylinder groups then the system performs more input/output operations and uses more buffers. On filesystems with many cylinder groups, lots of cache buffers are used for metadata operations. My solution for this problem is very simple. I allocate many directories in one cylinder group. I also do some things, so that the new allocation method does not cause excessive fragmentation and all directory inodes will not be located at a location far from its file's inodes and data. The algorithm is: /* * Find a cylinder group to place a directory. * * The policy implemented by this algorithm is to allocate a * directory inode in the same cylinder group as its parent * directory, but also to reserve space for its files inodes * and data. Restrict the number of directories which may be * allocated one after another in the same cylinder group * without intervening allocation of files. * * If we allocate a first level directory then force allocation * in another cylinder group. */ My early versions of dirpref give me a good results for a wide range of file operations and different filesystem capacities except one case: those applications that create their entire directory structure first and only later fill this structure with files. My solution for such and similar cases is to limit a number of directories which may be created one after another in the same cylinder group without intervening file creations. For this purpose, I allocate an array of counters at mount time. This array is linked to the superblock fs->fs_contigdirs[cg]. Each time a directory is created the counter increases and each time a file is created the counter decreases. A 60Gb filesystem with 8mb/cg requires 10kb of memory for the counters array. The maxcontigdirs is a maximum number of directories which may be created without an intervening file creation. I found in my tests that the best performance occurs when I restrict the number of directories in one cylinder group such that all its files may be located in the same cylinder group. There may be some deterioration in performance if all the file inodes are in the same cylinder group as its containing directory, but their data partially resides in a different cylinder group. The maxcontigdirs value is calculated to try to prevent this condition. Since there is no way to know how many files and directories will be allocated later I added two optimization parameters in superblock/tunefs. They are: int32_t fs_avgfilesize; /* expected average file size */ int32_t fs_avgfpdir; /* expected # of files per directory */ These parameters have reasonable defaults but may be tweeked for special uses of a filesystem. They are only necessary in rare cases like better tuning a filesystem being used to store a squid cache. I have been using this algorithm for about 3 months. I have done a lot of testing on filesystems with different capacities, average filesize, average number of files per directory, and so on. I think this algorithm has no negative impact on filesystem perfomance. It works better than the default one in all cases. The new dirpref will greatly improve untarring/removing/coping of big directories, decrease load on cvs servers and much more. The new dirpref doesn't speedup a compilation process, but also doesn't slow it down. Obtained from: Grigoriy Orlov <gluk@ptci.ru>
2001-04-10 08:38:59 +00:00
int avgfilesize = AVFILESIZ;/* expected average file size */
int avgfilesperdir = AFPDIR;/* expected number of files per directory */
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
u_char *volumelabel = NULL; /* volume label for filesystem */
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
struct uufsd disk; /* libufs disk structure */
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
Swing the axe and remove some archaic features from newfs which modern diskdrives do neither need nor want: -O create a 4.3BSD format filesystem -d rotational delay between contiguous blocks -k sector 0 skew, per track -l hardware sector interleave -n number of distinguished rotational positions -p spare sectors per track -r revolutions/minute -t tracks/cylinder -x spare sectors per cylinder No change in the produced filesystem image unless one or more of these options were used. Approved by: mckusick
2002-03-20 07:16:15 +00:00
static char device[MAXPATHLEN];
Fixed some style bugs in axings. Whitespace before __P was not axed when __P was axed. The ordering of several things was bogotified by axing ifdefs.
2002-04-04 09:56:51 +00:00
static char *disktype;
static int unlabeled;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
static struct disklabel *getdisklabel(char *s);
static void rewritelabel(char *s, struct disklabel *lp);
Fixed some style bugs in axings. Whitespace before __P was not axed when __P was axed. The ordering of several things was bogotified by axing ifdefs.
2002-04-04 09:56:51 +00:00
static void usage(void);
Add prototypes. Check malloc() return value. Use err(). Remove unused #includes Do not \n nor dot terminate syslog()/err() messages. -Wall.
1998-07-15 06:28:05 +00:00
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
int
Complete the ANSIfication of newfs by converting function declarations to C89 style.
2002-03-19 17:20:02 +00:00
main(int argc, char *argv[])
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
{
Remove __P() and register. Set WARNS=2 This is the beginning of a pre-UFS2 cleanup of newfs. Sponsored by: DARPA, NAI Labs
2002-03-17 09:01:41 +00:00
struct partition *pp;
struct disklabel *lp;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
struct partition oldpartition;
struct stat st;
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
char *cp, *special;
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
int ch, i;
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
off_t mediasize;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
Further cleanups.
2002-03-19 20:01:38 +00:00
while ((ch = getopt(argc, argv,
Add a "-l" flag to newfs, which sets the FS_MULTILABEL flag. This permits users of newfs to set the multilabel flag on UFS1 and UFS2 file systems from inception without using tunefs. Obtained from: TrustedBSD Project Sponsored by: DARPA, McAfee Research
2004-02-26 01:14:27 +00:00
"EL:NO:RS:T:Ua:b:c:d:e:f:g:h:i:lm:o:s:")) != -1)
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
switch (ch) {
Add the -E command line option to force error conditions for testing. Sponsord by: St. Bernard Software
2003-11-16 07:17:30 +00:00
case 'E':
Eflag++;
break;
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
case 'L':
volumelabel = optarg;
i = -1;
while (isalnum(volumelabel[++i]));
if (volumelabel[i] != '\0') {
errx(1, "bad volume label. Valid characters are alphanumerics.");
}
if (strlen(volumelabel) >= MAXVOLLEN) {
errx(1, "bad volume label. Length is longer than %d.",
MAXVOLLEN);
}
Lflag = 1;
break;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
case 'N':
Nflag = 1;
break;
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
case 'O':
if ((Oflag = atoi(optarg)) < 1 || Oflag > 2)
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
errx(1, "%s: bad file system format value",
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
optarg);
break;
Add the undocumented -R option to disable randomness for regression-testing. Add a couple of simple regression tests accessible with "make test", they depend on the md(4) driver. FYI I have also tried running the test against a week old newfs and it passed.
2002-03-19 21:05:29 +00:00
case 'R':
Rflag = 1;
break;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
case 'S':
if ((sectorsize = atoi(optarg)) <= 0)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: bad sector size", optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
case 'T':
disktype = optarg;
break;
Allow enabling soft updates (with -U) on a new filesystem. [I first added this functionality, and thought to check prior art. Seeing OpenBSD had already done this, I changed my addition to reduce the diffs between the two and went with their option letter.] Obtained from: OpenBSD
2001-04-02 01:25:55 +00:00
case 'U':
Uflag = 1;
break;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
case 'a':
if ((maxcontig = atoi(optarg)) <= 0)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: bad maximum contiguous blocks",
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
optarg);
break;
case 'b':
if ((bsize = atoi(optarg)) < MINBSIZE)
Add some more checks to newfs so that it will not build filesystems that the kernel will refuse to mount. Specifically it now enforces the MAXBSIZE blocksize limit. This update also fixes a problem where newfs could segment fault if the selected fragment size was too large. PR: bin/30959 Submitted by: Ceri Davies <setantae@submonkey.net> Sponsored by: DARPA & NAI Labs.
2002-11-30 18:28:26 +00:00
errx(1, "%s: block size too small, min is %d",
optarg, MINBSIZE);
if (bsize > MAXBSIZE)
errx(1, "%s: block size too large, max is %d",
optarg, MAXBSIZE);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
case 'c':
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
if ((maxblkspercg = atoi(optarg)) <= 0)
errx(1, "%s: bad blocks per cylinder group",
optarg);
break;
case 'd':
if ((maxbsize = atoi(optarg)) < MINBSIZE)
errx(1, "%s: bad extent block size", optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
case 'e':
if ((maxbpg = atoi(optarg)) <= 0)
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
errx(1, "%s: bad blocks per file in a cylinder group",
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
optarg);
break;
case 'f':
if ((fsize = atoi(optarg)) <= 0)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: bad fragment size", optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>. His description of the problem and solution follow. My own tests show speedups on typical filesystem intensive workloads of 5% to 12% which is very impressive considering the small amount of code change involved. ------ One day I noticed that some file operations run much faster on small file systems then on big ones. I've looked at the ffs algorithms, thought about them, and redesigned the dirpref algorithm. First I want to describe the results of my tests. These results are old and I have improved the algorithm after these tests were done. Nevertheless they show how big the perfomance speedup may be. I have done two file/directory intensive tests on a two OpenBSD systems with old and new dirpref algorithm. The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports". The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release. It contains 6596 directories and 13868 files. The test systems are: 1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for test is at wd1. Size of test file system is 8 Gb, number of cg=991, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=35 2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system at wd0, file system for test is at wd1. Size of test file system is 40 Gb, number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50 You can get more info about the test systems and methods at: http://www.ptci.ru/gluk/dirpref/old/dirpref.html Test Results tar -xzf ports.tar.gz rm -rf ports mode old dirpref new dirpref speedup old dirprefnew dirpref speedup First system normal 667 472 1.41 477 331 1.44 async 285 144 1.98 130 14 9.29 sync 768 616 1.25 477 334 1.43 softdep 413 252 1.64 241 38 6.34 Second system normal 329 81 4.06 263.5 93.5 2.81 async 302 25.7 11.75 112 2.26 49.56 sync 281 57.0 4.93 263 90.5 2.9 softdep 341 40.6 8.4 284 4.76 59.66 "old dirpref" and "new dirpref" columns give a test time in seconds. speedup - speed increasement in times, ie. old dirpref / new dirpref. ------ Algorithm description The old dirpref algorithm is described in comments: /* * Find a cylinder to place a directory. * * The policy implemented by this algorithm is to select from * among those cylinder groups with above the average number of * free inodes, the one with the smallest number of directories. */ A new directory is allocated in a different cylinder groups than its parent directory resulting in a directory tree that is spreaded across all the cylinder groups. This spreading out results in a non-optimal access to the directories and files. When we have a small filesystem it is not a problem but when the filesystem is big then perfomance degradation becomes very apparent. What I mean by a big file system ? 1. A big filesystem is a filesystem which occupy 20-30 or more percent of total drive space, i.e. first and last cylinder are physically located relatively far from each other. 2. It has a relatively large number of cylinder groups, for example more cylinder groups than 50% of the buffers in the buffer cache. The first results in long access times, while the second results in many buffers being used by metadata operations. Such operations use cylinder group blocks and on-disk inode blocks. The cylinder group block (fs->fs_cblkno) contains struct cg, inode and block bit maps. It is 2k in size for the default filesystem parameters. If new and parent directories are located in different cylinder groups then the system performs more input/output operations and uses more buffers. On filesystems with many cylinder groups, lots of cache buffers are used for metadata operations. My solution for this problem is very simple. I allocate many directories in one cylinder group. I also do some things, so that the new allocation method does not cause excessive fragmentation and all directory inodes will not be located at a location far from its file's inodes and data. The algorithm is: /* * Find a cylinder group to place a directory. * * The policy implemented by this algorithm is to allocate a * directory inode in the same cylinder group as its parent * directory, but also to reserve space for its files inodes * and data. Restrict the number of directories which may be * allocated one after another in the same cylinder group * without intervening allocation of files. * * If we allocate a first level directory then force allocation * in another cylinder group. */ My early versions of dirpref give me a good results for a wide range of file operations and different filesystem capacities except one case: those applications that create their entire directory structure first and only later fill this structure with files. My solution for such and similar cases is to limit a number of directories which may be created one after another in the same cylinder group without intervening file creations. For this purpose, I allocate an array of counters at mount time. This array is linked to the superblock fs->fs_contigdirs[cg]. Each time a directory is created the counter increases and each time a file is created the counter decreases. A 60Gb filesystem with 8mb/cg requires 10kb of memory for the counters array. The maxcontigdirs is a maximum number of directories which may be created without an intervening file creation. I found in my tests that the best performance occurs when I restrict the number of directories in one cylinder group such that all its files may be located in the same cylinder group. There may be some deterioration in performance if all the file inodes are in the same cylinder group as its containing directory, but their data partially resides in a different cylinder group. The maxcontigdirs value is calculated to try to prevent this condition. Since there is no way to know how many files and directories will be allocated later I added two optimization parameters in superblock/tunefs. They are: int32_t fs_avgfilesize; /* expected average file size */ int32_t fs_avgfpdir; /* expected # of files per directory */ These parameters have reasonable defaults but may be tweeked for special uses of a filesystem. They are only necessary in rare cases like better tuning a filesystem being used to store a squid cache. I have been using this algorithm for about 3 months. I have done a lot of testing on filesystems with different capacities, average filesize, average number of files per directory, and so on. I think this algorithm has no negative impact on filesystem perfomance. It works better than the default one in all cases. The new dirpref will greatly improve untarring/removing/coping of big directories, decrease load on cvs servers and much more. The new dirpref doesn't speedup a compilation process, but also doesn't slow it down. Obtained from: Grigoriy Orlov <gluk@ptci.ru>
2001-04-10 08:38:59 +00:00
case 'g':
if ((avgfilesize = atoi(optarg)) <= 0)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: bad average file size", optarg);
Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>. His description of the problem and solution follow. My own tests show speedups on typical filesystem intensive workloads of 5% to 12% which is very impressive considering the small amount of code change involved. ------ One day I noticed that some file operations run much faster on small file systems then on big ones. I've looked at the ffs algorithms, thought about them, and redesigned the dirpref algorithm. First I want to describe the results of my tests. These results are old and I have improved the algorithm after these tests were done. Nevertheless they show how big the perfomance speedup may be. I have done two file/directory intensive tests on a two OpenBSD systems with old and new dirpref algorithm. The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports". The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release. It contains 6596 directories and 13868 files. The test systems are: 1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for test is at wd1. Size of test file system is 8 Gb, number of cg=991, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=35 2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system at wd0, file system for test is at wd1. Size of test file system is 40 Gb, number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50 You can get more info about the test systems and methods at: http://www.ptci.ru/gluk/dirpref/old/dirpref.html Test Results tar -xzf ports.tar.gz rm -rf ports mode old dirpref new dirpref speedup old dirprefnew dirpref speedup First system normal 667 472 1.41 477 331 1.44 async 285 144 1.98 130 14 9.29 sync 768 616 1.25 477 334 1.43 softdep 413 252 1.64 241 38 6.34 Second system normal 329 81 4.06 263.5 93.5 2.81 async 302 25.7 11.75 112 2.26 49.56 sync 281 57.0 4.93 263 90.5 2.9 softdep 341 40.6 8.4 284 4.76 59.66 "old dirpref" and "new dirpref" columns give a test time in seconds. speedup - speed increasement in times, ie. old dirpref / new dirpref. ------ Algorithm description The old dirpref algorithm is described in comments: /* * Find a cylinder to place a directory. * * The policy implemented by this algorithm is to select from * among those cylinder groups with above the average number of * free inodes, the one with the smallest number of directories. */ A new directory is allocated in a different cylinder groups than its parent directory resulting in a directory tree that is spreaded across all the cylinder groups. This spreading out results in a non-optimal access to the directories and files. When we have a small filesystem it is not a problem but when the filesystem is big then perfomance degradation becomes very apparent. What I mean by a big file system ? 1. A big filesystem is a filesystem which occupy 20-30 or more percent of total drive space, i.e. first and last cylinder are physically located relatively far from each other. 2. It has a relatively large number of cylinder groups, for example more cylinder groups than 50% of the buffers in the buffer cache. The first results in long access times, while the second results in many buffers being used by metadata operations. Such operations use cylinder group blocks and on-disk inode blocks. The cylinder group block (fs->fs_cblkno) contains struct cg, inode and block bit maps. It is 2k in size for the default filesystem parameters. If new and parent directories are located in different cylinder groups then the system performs more input/output operations and uses more buffers. On filesystems with many cylinder groups, lots of cache buffers are used for metadata operations. My solution for this problem is very simple. I allocate many directories in one cylinder group. I also do some things, so that the new allocation method does not cause excessive fragmentation and all directory inodes will not be located at a location far from its file's inodes and data. The algorithm is: /* * Find a cylinder group to place a directory. * * The policy implemented by this algorithm is to allocate a * directory inode in the same cylinder group as its parent * directory, but also to reserve space for its files inodes * and data. Restrict the number of directories which may be * allocated one after another in the same cylinder group * without intervening allocation of files. * * If we allocate a first level directory then force allocation * in another cylinder group. */ My early versions of dirpref give me a good results for a wide range of file operations and different filesystem capacities except one case: those applications that create their entire directory structure first and only later fill this structure with files. My solution for such and similar cases is to limit a number of directories which may be created one after another in the same cylinder group without intervening file creations. For this purpose, I allocate an array of counters at mount time. This array is linked to the superblock fs->fs_contigdirs[cg]. Each time a directory is created the counter increases and each time a file is created the counter decreases. A 60Gb filesystem with 8mb/cg requires 10kb of memory for the counters array. The maxcontigdirs is a maximum number of directories which may be created without an intervening file creation. I found in my tests that the best performance occurs when I restrict the number of directories in one cylinder group such that all its files may be located in the same cylinder group. There may be some deterioration in performance if all the file inodes are in the same cylinder group as its containing directory, but their data partially resides in a different cylinder group. The maxcontigdirs value is calculated to try to prevent this condition. Since there is no way to know how many files and directories will be allocated later I added two optimization parameters in superblock/tunefs. They are: int32_t fs_avgfilesize; /* expected average file size */ int32_t fs_avgfpdir; /* expected # of files per directory */ These parameters have reasonable defaults but may be tweeked for special uses of a filesystem. They are only necessary in rare cases like better tuning a filesystem being used to store a squid cache. I have been using this algorithm for about 3 months. I have done a lot of testing on filesystems with different capacities, average filesize, average number of files per directory, and so on. I think this algorithm has no negative impact on filesystem perfomance. It works better than the default one in all cases. The new dirpref will greatly improve untarring/removing/coping of big directories, decrease load on cvs servers and much more. The new dirpref doesn't speedup a compilation process, but also doesn't slow it down. Obtained from: Grigoriy Orlov <gluk@ptci.ru>
2001-04-10 08:38:59 +00:00
break;
case 'h':
if ((avgfilesperdir = atoi(optarg)) <= 0)
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
errx(1, "%s: bad average files per dir", optarg);
Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>. His description of the problem and solution follow. My own tests show speedups on typical filesystem intensive workloads of 5% to 12% which is very impressive considering the small amount of code change involved. ------ One day I noticed that some file operations run much faster on small file systems then on big ones. I've looked at the ffs algorithms, thought about them, and redesigned the dirpref algorithm. First I want to describe the results of my tests. These results are old and I have improved the algorithm after these tests were done. Nevertheless they show how big the perfomance speedup may be. I have done two file/directory intensive tests on a two OpenBSD systems with old and new dirpref algorithm. The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports". The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release. It contains 6596 directories and 13868 files. The test systems are: 1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for test is at wd1. Size of test file system is 8 Gb, number of cg=991, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=35 2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system at wd0, file system for test is at wd1. Size of test file system is 40 Gb, number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50 You can get more info about the test systems and methods at: http://www.ptci.ru/gluk/dirpref/old/dirpref.html Test Results tar -xzf ports.tar.gz rm -rf ports mode old dirpref new dirpref speedup old dirprefnew dirpref speedup First system normal 667 472 1.41 477 331 1.44 async 285 144 1.98 130 14 9.29 sync 768 616 1.25 477 334 1.43 softdep 413 252 1.64 241 38 6.34 Second system normal 329 81 4.06 263.5 93.5 2.81 async 302 25.7 11.75 112 2.26 49.56 sync 281 57.0 4.93 263 90.5 2.9 softdep 341 40.6 8.4 284 4.76 59.66 "old dirpref" and "new dirpref" columns give a test time in seconds. speedup - speed increasement in times, ie. old dirpref / new dirpref. ------ Algorithm description The old dirpref algorithm is described in comments: /* * Find a cylinder to place a directory. * * The policy implemented by this algorithm is to select from * among those cylinder groups with above the average number of * free inodes, the one with the smallest number of directories. */ A new directory is allocated in a different cylinder groups than its parent directory resulting in a directory tree that is spreaded across all the cylinder groups. This spreading out results in a non-optimal access to the directories and files. When we have a small filesystem it is not a problem but when the filesystem is big then perfomance degradation becomes very apparent. What I mean by a big file system ? 1. A big filesystem is a filesystem which occupy 20-30 or more percent of total drive space, i.e. first and last cylinder are physically located relatively far from each other. 2. It has a relatively large number of cylinder groups, for example more cylinder groups than 50% of the buffers in the buffer cache. The first results in long access times, while the second results in many buffers being used by metadata operations. Such operations use cylinder group blocks and on-disk inode blocks. The cylinder group block (fs->fs_cblkno) contains struct cg, inode and block bit maps. It is 2k in size for the default filesystem parameters. If new and parent directories are located in different cylinder groups then the system performs more input/output operations and uses more buffers. On filesystems with many cylinder groups, lots of cache buffers are used for metadata operations. My solution for this problem is very simple. I allocate many directories in one cylinder group. I also do some things, so that the new allocation method does not cause excessive fragmentation and all directory inodes will not be located at a location far from its file's inodes and data. The algorithm is: /* * Find a cylinder group to place a directory. * * The policy implemented by this algorithm is to allocate a * directory inode in the same cylinder group as its parent * directory, but also to reserve space for its files inodes * and data. Restrict the number of directories which may be * allocated one after another in the same cylinder group * without intervening allocation of files. * * If we allocate a first level directory then force allocation * in another cylinder group. */ My early versions of dirpref give me a good results for a wide range of file operations and different filesystem capacities except one case: those applications that create their entire directory structure first and only later fill this structure with files. My solution for such and similar cases is to limit a number of directories which may be created one after another in the same cylinder group without intervening file creations. For this purpose, I allocate an array of counters at mount time. This array is linked to the superblock fs->fs_contigdirs[cg]. Each time a directory is created the counter increases and each time a file is created the counter decreases. A 60Gb filesystem with 8mb/cg requires 10kb of memory for the counters array. The maxcontigdirs is a maximum number of directories which may be created without an intervening file creation. I found in my tests that the best performance occurs when I restrict the number of directories in one cylinder group such that all its files may be located in the same cylinder group. There may be some deterioration in performance if all the file inodes are in the same cylinder group as its containing directory, but their data partially resides in a different cylinder group. The maxcontigdirs value is calculated to try to prevent this condition. Since there is no way to know how many files and directories will be allocated later I added two optimization parameters in superblock/tunefs. They are: int32_t fs_avgfilesize; /* expected average file size */ int32_t fs_avgfpdir; /* expected # of files per directory */ These parameters have reasonable defaults but may be tweeked for special uses of a filesystem. They are only necessary in rare cases like better tuning a filesystem being used to store a squid cache. I have been using this algorithm for about 3 months. I have done a lot of testing on filesystems with different capacities, average filesize, average number of files per directory, and so on. I think this algorithm has no negative impact on filesystem perfomance. It works better than the default one in all cases. The new dirpref will greatly improve untarring/removing/coping of big directories, decrease load on cvs servers and much more. The new dirpref doesn't speedup a compilation process, but also doesn't slow it down. Obtained from: Grigoriy Orlov <gluk@ptci.ru>
2001-04-10 08:38:59 +00:00
break;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
case 'i':
if ((density = atoi(optarg)) <= 0)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: bad bytes per inode", optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
Add a "-l" flag to newfs, which sets the FS_MULTILABEL flag. This permits users of newfs to set the multilabel flag on UFS1 and UFS2 file systems from inception without using tunefs. Obtained from: TrustedBSD Project Sponsored by: DARPA, McAfee Research
2004-02-26 01:14:27 +00:00
case 'l':
lflag = 1;
break;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
case 'm':
if ((minfree = atoi(optarg)) < 0 || minfree > 99)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: bad free space %%", optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
case 'o':
A more complete removal of MFS related code. XXX: This program badly needs a style(9) + BDECFLAGS treatment.
2001-05-29 19:40:39 +00:00
if (strcmp(optarg, "space") == 0)
opt = FS_OPTSPACE;
else if (strcmp(optarg, "time") == 0)
opt = FS_OPTTIME;
else
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1,
Fixed some style bugs (mainly ones not fixed or made worse by rev.1.44). Don't use ISO string concatentation to obfuscate long single-line messages...
2002-03-18 03:04:58 +00:00
"%s: unknown optimization preference: use `space' or `time'",
optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
case 's':
if ((fssize = atoi(optarg)) <= 0)
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
errx(1, "%s: bad file system size", optarg);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
break;
case '?':
default:
usage();
}
argc -= optind;
argv += optind;
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
if (argc != 1)
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
usage();
special = argv[0];
Merge from Lite2: - use new getvfsbyname() and mount(2) interface (mount_mfs) - use new fs include files - updated inode / cg layout calculations (?)
1997-03-11 12:48:17 +00:00
cp = strrchr(special, '/');
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
if (cp == 0) {
/*
Fixed some style bugs (mainly ones not fixed or made worse by rev.1.44). Don't use ISO string concatentation to obfuscate long single-line messages...
2002-03-18 03:04:58 +00:00
* No path prefix; try prefixing _PATH_DEV.
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
*/
Fixed some style bugs (mainly ones not fixed or made worse by rev.1.44). Don't use ISO string concatentation to obfuscate long single-line messages...
2002-03-18 03:04:58 +00:00
snprintf(device, sizeof(device), "%s%s", _PATH_DEV, special);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
special = device;
}
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
if (ufs_disk_fillout_blank(&disk, special) == -1 ||
(!Nflag && ufs_disk_write(&disk) == -1)) {
if (disk.d_error != NULL)
errx(1, "%s: %s", special, disk.d_error);
else
err(1, "%s", special);
}
if (fstat(disk.d_fd, &st) < 0)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
err(1, "%s", special);
A more complete removal of MFS related code. XXX: This program badly needs a style(9) + BDECFLAGS treatment.
2001-05-29 19:40:39 +00:00
if ((st.st_mode & S_IFMT) != S_IFCHR)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
errx(1, "%s: not a character-special device", special);
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
if (sectorsize == 0)
ioctl(disk.d_fd, DIOCGSECTORSIZE, &sectorsize);
if (sectorsize && !ioctl(disk.d_fd, DIOCGMEDIASIZE, &mediasize)) {
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
if (fssize == 0)
fssize = mediasize / sectorsize;
else if (fssize > mediasize / sectorsize)
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
errx(1, "%s: maximum file system size is %u",
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
special, (u_int)(mediasize / sectorsize));
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
}
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
pp = NULL;
lp = getdisklabel(special);
if (lp != NULL) {
cp = strchr(special, '\0');
cp--;
Remove the -v option, it is now default behaviour. Sponsored by: DARPA & NAI Labs
2002-04-24 12:27:03 +00:00
if ((*cp < 'a' || *cp > 'h') && !isdigit(*cp))
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
errx(1, "%s: can't figure out file system partition",
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
special);
Remove the -v option, it is now default behaviour. Sponsored by: DARPA & NAI Labs
2002-04-24 12:27:03 +00:00
if (isdigit(*cp))
pp = &lp->d_partitions[RAW_PART];
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
else
pp = &lp->d_partitions[*cp - 'a'];
oldpartition = *pp;
if (pp->p_size == 0)
errx(1, "%s: `%c' partition is unavailable",
special, *cp);
if (pp->p_fstype == FS_BOOT)
errx(1, "%s: `%c' partition overlaps boot program",
special, *cp);
if (fssize == 0)
fssize = pp->p_size;
if (fssize > pp->p_size)
errx(1,
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
"%s: maximum file system size %d", special, pp->p_size);
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
if (sectorsize == 0)
sectorsize = lp->d_secsize;
if (fsize == 0)
fsize = pp->p_fsize;
if (bsize == 0)
bsize = pp->p_frag * pp->p_fsize;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
}
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
if (sectorsize <= 0)
errx(1, "%s: no default sector size", special);
if (fsize <= 0)
fsize = MAX(DFL_FRAGSIZE, sectorsize);
if (bsize <= 0)
bsize = MIN(DFL_BLKSIZE, 8 * fsize);
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
if (maxbsize == 0)
maxbsize = bsize;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
/*
* Maxcontig sets the default for the maximum number of blocks
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
* that may be allocated sequentially. With file system clustering
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
* it is possible to allocate contiguous blocks up to the maximum
* transfer size permitted by the controller or buffering.
*/
if (maxcontig == 0)
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
maxcontig = MAX(1, MAXPHYS / bsize);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
if (density == 0)
density = NFPI * fsize;
if (minfree < MINFREE && opt != FS_OPTSPACE) {
fprintf(stderr, "Warning: changing optimization to space ");
fprintf(stderr, "because minfree is less than %d%%\n", MINFREE);
opt = FS_OPTSPACE;
}
if (maxbpg == 0)
maxbpg = MAXBLKPG(bsize);
This update adds the support for != 512 byte sector SCSI devices to the sd & od drivers. There is also slight changes to fdisk & newfs in order to comply with different sectorsizes. Currently sectors of size 512, 1024 & 2048 are supported, the only restriction beeing in fdisk, which hunts for the sectorsize of the device. This is based on patches to od.c and the other system files by John Gumb & Barry Scott, minor changes and the sd.c patches by me. There also exist some patches for the msdos filesys code, but I havn't been able to test those (yet). John Gumb (john@talisker.demon.co.uk) Barry Scott (barry@scottb.demon.co.uk)
1996-12-01 11:25:38 +00:00
realsectorsize = sectorsize;
if (sectorsize != DEV_BSIZE) { /* XXX */
int secperblk = sectorsize / DEV_BSIZE;
sectorsize = DEV_BSIZE;
fssize *= secperblk;
Remove a comma trailing an if clause. According to Kirk: "Luckily, the statement is usually true". Spotted by: FlexeLint
2002-10-01 17:31:28 +00:00
if (pp != NULL)
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
pp->p_size *= secperblk;
}
mkfs(pp, special);
if (!unlabeled) {
if (realsectorsize != DEV_BSIZE)
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
pp->p_size /= realsectorsize / DEV_BSIZE;
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
if (!Nflag && bcmp(pp, &oldpartition, sizeof(oldpartition)))
rewritelabel(special, lp);
This update adds the support for != 512 byte sector SCSI devices to the sd & od drivers. There is also slight changes to fdisk & newfs in order to comply with different sectorsizes. Currently sectors of size 512, 1024 & 2048 are supported, the only restriction beeing in fdisk, which hunts for the sectorsize of the device. This is based on patches to od.c and the other system files by John Gumb & Barry Scott, minor changes and the sd.c patches by me. There also exist some patches for the msdos filesys code, but I havn't been able to test those (yet). John Gumb (john@talisker.demon.co.uk) Barry Scott (barry@scottb.demon.co.uk)
1996-12-01 11:25:38 +00:00
}
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
ufs_disk_close(&disk);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
exit(0);
}
struct disklabel *
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
getdisklabel(char *s)
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
{
static struct disklabel lab;
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
struct disklabel *lp;
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
if (!ioctl(disk.d_fd, DIOCGDINFO, (char *)&lab))
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
return (&lab);
unlabeled++;
if (disktype) {
lp = getdiskbyname(disktype);
if (lp != NULL)
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
return (lp);
}
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
return (NULL);
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
}
A more complete removal of MFS related code. XXX: This program badly needs a style(9) + BDECFLAGS treatment.
2001-05-29 19:40:39 +00:00
void
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
rewritelabel(char *s, struct disklabel *lp)
Backed out rev.1.9 (except don't bring back the vax code deleted in rev.1.9). fsck uses the per-partition ffs-related information in the label to find alternate superblocks when the main superblock is hosed. Rev.1.9 broke this by deleting the code that wrote the label. PR: 2537 xref: fsck/setup.c rev.1.8
1998-07-20 12:04:42 +00:00
{
if (unlabeled)
return;
lp->d_checksum = 0;
lp->d_checksum = dkcksum(lp);
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
if (ioctl(disk.d_fd, DIOCWDINFO, (char *)lp) < 0)
Failure to rewrite the disklabel should not be fatal. Sponsored by: DARPA & NAI Labs.
2002-09-22 09:41:41 +00:00
warn("ioctl (WDINFO): %s: can't rewrite disk label", s);
Backed out rev.1.9 (except don't bring back the vax code deleted in rev.1.9). fsck uses the per-partition ffs-related information in the label to find alternate superblocks when the main superblock is hosed. Rev.1.9 broke this by deleting the code that wrote the label. PR: 2537 xref: fsck/setup.c rev.1.8
1998-07-20 12:04:42 +00:00
}
Add prototypes. Check malloc() return value. Use err(). Remove unused #includes Do not \n nor dot terminate syslog()/err() messages. -Wall.
1998-07-15 06:28:05 +00:00
static void
Back out conversion to libufs, for now. It seems to cause problems. Reported by: phk
2003-01-29 22:52:27 +00:00
usage()
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
{
A more complete removal of MFS related code. XXX: This program badly needs a style(9) + BDECFLAGS treatment.
2001-05-29 19:40:39 +00:00
fprintf(stderr,
"usage: %s [ -fsoptions ] special-device%s\n",
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 11:44:02 +00:00
getprogname(),
A more complete removal of MFS related code. XXX: This program badly needs a style(9) + BDECFLAGS treatment.
2001-05-29 19:40:39 +00:00
" [device-type]");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
fprintf(stderr, "where fsoptions are:\n");
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
fprintf(stderr, "\t-L volume label to add to superblock\n");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
fprintf(stderr,
s/filesystem/file system/g as discussed on -developers
2002-08-21 18:11:48 +00:00
"\t-N do not create file system, just print out parameters\n");
fprintf(stderr, "\t-O file system format: 1 => UFS1, 2 => UFS2\n");
Add the undocumented -R option to disable randomness for regression-testing. Add a couple of simple regression tests accessible with "make test", they depend on the md(4) driver. FYI I have also tried running the test against a week old newfs and it passed.
2002-03-19 21:05:29 +00:00
fprintf(stderr, "\t-R regression test, supress random factors\n");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
fprintf(stderr, "\t-S sector size\n");
fprintf(stderr, "\t-T disktype\n");
Allow enabling soft updates (with -U) on a new filesystem. [I first added this functionality, and thought to check prior art. Seeing OpenBSD had already done this, I changed my addition to reduce the diffs between the two and went with their option letter.] Obtained from: OpenBSD
2001-04-02 01:25:55 +00:00
fprintf(stderr, "\t-U enable soft updates\n");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
fprintf(stderr, "\t-a maximum contiguous blocks\n");
fprintf(stderr, "\t-b block size\n");
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
fprintf(stderr, "\t-c blocks per cylinders group\n");
fprintf(stderr, "\t-d maximum extent size\n");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
fprintf(stderr, "\t-e maximum blocks per file in a cylinder group\n");
fprintf(stderr, "\t-f frag size\n");
Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>. His description of the problem and solution follow. My own tests show speedups on typical filesystem intensive workloads of 5% to 12% which is very impressive considering the small amount of code change involved. ------ One day I noticed that some file operations run much faster on small file systems then on big ones. I've looked at the ffs algorithms, thought about them, and redesigned the dirpref algorithm. First I want to describe the results of my tests. These results are old and I have improved the algorithm after these tests were done. Nevertheless they show how big the perfomance speedup may be. I have done two file/directory intensive tests on a two OpenBSD systems with old and new dirpref algorithm. The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports". The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release. It contains 6596 directories and 13868 files. The test systems are: 1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for test is at wd1. Size of test file system is 8 Gb, number of cg=991, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=35 2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system at wd0, file system for test is at wd1. Size of test file system is 40 Gb, number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50 You can get more info about the test systems and methods at: http://www.ptci.ru/gluk/dirpref/old/dirpref.html Test Results tar -xzf ports.tar.gz rm -rf ports mode old dirpref new dirpref speedup old dirprefnew dirpref speedup First system normal 667 472 1.41 477 331 1.44 async 285 144 1.98 130 14 9.29 sync 768 616 1.25 477 334 1.43 softdep 413 252 1.64 241 38 6.34 Second system normal 329 81 4.06 263.5 93.5 2.81 async 302 25.7 11.75 112 2.26 49.56 sync 281 57.0 4.93 263 90.5 2.9 softdep 341 40.6 8.4 284 4.76 59.66 "old dirpref" and "new dirpref" columns give a test time in seconds. speedup - speed increasement in times, ie. old dirpref / new dirpref. ------ Algorithm description The old dirpref algorithm is described in comments: /* * Find a cylinder to place a directory. * * The policy implemented by this algorithm is to select from * among those cylinder groups with above the average number of * free inodes, the one with the smallest number of directories. */ A new directory is allocated in a different cylinder groups than its parent directory resulting in a directory tree that is spreaded across all the cylinder groups. This spreading out results in a non-optimal access to the directories and files. When we have a small filesystem it is not a problem but when the filesystem is big then perfomance degradation becomes very apparent. What I mean by a big file system ? 1. A big filesystem is a filesystem which occupy 20-30 or more percent of total drive space, i.e. first and last cylinder are physically located relatively far from each other. 2. It has a relatively large number of cylinder groups, for example more cylinder groups than 50% of the buffers in the buffer cache. The first results in long access times, while the second results in many buffers being used by metadata operations. Such operations use cylinder group blocks and on-disk inode blocks. The cylinder group block (fs->fs_cblkno) contains struct cg, inode and block bit maps. It is 2k in size for the default filesystem parameters. If new and parent directories are located in different cylinder groups then the system performs more input/output operations and uses more buffers. On filesystems with many cylinder groups, lots of cache buffers are used for metadata operations. My solution for this problem is very simple. I allocate many directories in one cylinder group. I also do some things, so that the new allocation method does not cause excessive fragmentation and all directory inodes will not be located at a location far from its file's inodes and data. The algorithm is: /* * Find a cylinder group to place a directory. * * The policy implemented by this algorithm is to allocate a * directory inode in the same cylinder group as its parent * directory, but also to reserve space for its files inodes * and data. Restrict the number of directories which may be * allocated one after another in the same cylinder group * without intervening allocation of files. * * If we allocate a first level directory then force allocation * in another cylinder group. */ My early versions of dirpref give me a good results for a wide range of file operations and different filesystem capacities except one case: those applications that create their entire directory structure first and only later fill this structure with files. My solution for such and similar cases is to limit a number of directories which may be created one after another in the same cylinder group without intervening file creations. For this purpose, I allocate an array of counters at mount time. This array is linked to the superblock fs->fs_contigdirs[cg]. Each time a directory is created the counter increases and each time a file is created the counter decreases. A 60Gb filesystem with 8mb/cg requires 10kb of memory for the counters array. The maxcontigdirs is a maximum number of directories which may be created without an intervening file creation. I found in my tests that the best performance occurs when I restrict the number of directories in one cylinder group such that all its files may be located in the same cylinder group. There may be some deterioration in performance if all the file inodes are in the same cylinder group as its containing directory, but their data partially resides in a different cylinder group. The maxcontigdirs value is calculated to try to prevent this condition. Since there is no way to know how many files and directories will be allocated later I added two optimization parameters in superblock/tunefs. They are: int32_t fs_avgfilesize; /* expected average file size */ int32_t fs_avgfpdir; /* expected # of files per directory */ These parameters have reasonable defaults but may be tweeked for special uses of a filesystem. They are only necessary in rare cases like better tuning a filesystem being used to store a squid cache. I have been using this algorithm for about 3 months. I have done a lot of testing on filesystems with different capacities, average filesize, average number of files per directory, and so on. I think this algorithm has no negative impact on filesystem perfomance. It works better than the default one in all cases. The new dirpref will greatly improve untarring/removing/coping of big directories, decrease load on cvs servers and much more. The new dirpref doesn't speedup a compilation process, but also doesn't slow it down. Obtained from: Grigoriy Orlov <gluk@ptci.ru>
2001-04-10 08:38:59 +00:00
fprintf(stderr, "\t-g average file size\n");
fprintf(stderr, "\t-h average files per directory\n");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
fprintf(stderr, "\t-i number of bytes per inode\n");
fprintf(stderr, "\t-m minimum free space %%\n");
fprintf(stderr, "\t-o optimization preference (`space' or `time')\n");
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
fprintf(stderr, "\t-s file systemsize (sectors)\n");
BSD 4.4 Lite sbin Sources Note: XNSrouted and routed NOT imported here, they shall be imported with usr.sbin.
1994-05-26 06:35:07 +00:00
exit(1);
}
Reference in New Issue Copy Permalink