freebsd-dev/sbin/fsck_ffs/fsck.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

545 lines
21 KiB
C
Raw Normal View History

/*-
* SPDX-License-Identifier: BSD-3-Clause and BSD-2-Clause
*
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
* Copyright (c) 2002 Networks Associates Technology, Inc.
* All rights reserved.
*
* This software was developed for the FreeBSD Project by Marshall
* Kirk McKusick and Network Associates Laboratories, the Security
* Research Division of Network Associates, Inc. under DARPA/SPAWAR
* contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS
* research program.
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* Copyright (c) 1980, 1986, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* @(#)fsck.h 8.4 (Berkeley) 5/9/95
* $FreeBSD$
*/
#ifndef _FSCK_H_
#define _FSCK_H_
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/queue.h>
#define MAXDUP 10 /* limit on dup blks (per inode) */
#define MAXBAD 10 /* limit on bad blks (per inode) */
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
#define MINBUFS 100 /* minimum number of buffers required */
#define INOBUFSIZE 64*1024 /* size of buffer to read inodes in pass1 */
#define ZEROBUFSIZE (dev_bsize * 128) /* size of zero buffer used by -Z */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
union dinode {
struct ufs1_dinode dp1;
struct ufs2_dinode dp2;
};
#define DIP(dp, field) \
((sblock.fs_magic == FS_UFS1_MAGIC) ? \
(dp)->dp1.field : (dp)->dp2.field)
#define DIP_SET(dp, field, val) do { \
if (sblock.fs_magic == FS_UFS1_MAGIC) \
(dp)->dp1.field = (val); \
else \
(dp)->dp2.field = (val); \
} while (0)
/*
* Each inode on the file system is described by the following structure.
* The linkcnt is initially set to the value in the inode. Each time it
* is found during the descent in passes 2, 3, and 4 the count is
* decremented. Any inodes whose count is non-zero after pass 4 needs to
* have its link count adjusted by the value remaining in ino_linkcnt.
*/
struct inostat {
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
u_char ino_state; /* state of inode, see below */
u_char ino_type:4; /* type of inode */
u_char ino_idtype:4; /* idesc id_type, SNAP or ADDR */
u_short ino_linkcnt; /* number of links not found */
};
/*
* Inode states.
*/
#define USTATE 0x1 /* inode not allocated */
#define FSTATE 0x2 /* inode is file */
#define FZLINK 0x3 /* inode is file with a link count of zero */
#define DSTATE 0x4 /* inode is directory */
#define DZLINK 0x5 /* inode is directory with a zero link count */
#define DFOUND 0x6 /* directory found during descent */
/* 0x7 UNUSED - see S_IS_DVALID() definition */
#define DCLEAR 0x8 /* directory is to be cleared */
#define FCLEAR 0x9 /* file is to be cleared */
/* DUNFOUND === (state == DSTATE || state == DZLINK) */
#define S_IS_DUNFOUND(state) (((state) & ~0x1) == DSTATE)
/* DVALID === (state == DSTATE || state == DZLINK || state == DFOUND) */
#define S_IS_DVALID(state) (((state) & ~0x3) == DSTATE)
#define INO_IS_DUNFOUND(ino) S_IS_DUNFOUND(inoinfo(ino)->ino_state)
#define INO_IS_DVALID(ino) S_IS_DVALID(inoinfo(ino)->ino_state)
/*
* Inode state information is contained on per cylinder group lists
* which are described by the following structure.
*/
extern struct inostatlist {
long il_numalloced; /* number of inodes allocated in this cg */
struct inostat *il_stat;/* inostat info for this cylinder group */
} *inostathead;
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
/*
* Structure to reference a dinode.
*/
struct inode {
struct bufarea *i_bp; /* buffer containing the dinode */
union dinode *i_dp; /* pointer to dinode in buffer */
ino_t i_number; /* inode number */
};
/*
* Size of hash tables
*/
#define HASHSIZE 2048
#define HASH(x) ((x * 2654435761) & (HASHSIZE - 1))
/*
* buffer cache structure.
*/
struct bufarea {
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
TAILQ_ENTRY(bufarea) b_list; /* LRU buffer queue */
LIST_ENTRY(bufarea) b_hash; /* hash list */
ufs2_daddr_t b_bno; /* disk block number */
int b_size; /* size of I/O */
int b_errs; /* I/O error */
int b_flags; /* B_ flags below */
int b_type; /* BT_ type below */
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
int b_refcnt; /* ref count of users */
int b_index; /* for BT_LEVEL, ptr index */
/* for BT_INODES, first inum */
union {
char *b_buf; /* buffer space */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
ufs1_daddr_t *b_indir1; /* UFS1 indirect block */
ufs2_daddr_t *b_indir2; /* UFS2 indirect block */
struct fs *b_fs; /* super block */
struct cg *b_cg; /* cylinder group */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
struct ufs1_dinode *b_dinode1; /* UFS1 inode block */
struct ufs2_dinode *b_dinode2; /* UFS2 inode block */
} b_un;
};
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
#define IBLK(bp, i) \
((sblock.fs_magic == FS_UFS1_MAGIC) ? \
(bp)->b_un.b_indir1[i] : (bp)->b_un.b_indir2[i])
#define IBLK_SET(bp, i, val) do { \
if (sblock.fs_magic == FS_UFS1_MAGIC) \
(bp)->b_un.b_indir1[i] = (val); \
else \
(bp)->b_un.b_indir2[i] = (val); \
} while (0)
/*
* Buffer flags
*/
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
#define B_DIRTY 0x00000001 /* Buffer is dirty */
/*
* Type of data in buffer
*/
#define BT_UNKNOWN 0 /* Buffer type is unknown */
#define BT_SUPERBLK 1 /* Buffer holds a superblock */
#define BT_CYLGRP 2 /* Buffer holds a cylinder group map */
#define BT_LEVEL1 3 /* Buffer holds single level indirect */
#define BT_LEVEL2 4 /* Buffer holds double level indirect */
#define BT_LEVEL3 5 /* Buffer holds triple level indirect */
#define BT_EXTATTR 6 /* Buffer holds external attribute data */
#define BT_INODES 7 /* Buffer holds inodes */
#define BT_DIRDATA 8 /* Buffer holds directory data */
#define BT_DATA 9 /* Buffer holds user data */
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
#define BT_NUMBUFTYPES 10
#define BT_NAMES { \
"unknown", \
"Superblock", \
"Cylinder Group", \
"Single Level Indirect", \
"Double Level Indirect", \
"Triple Level Indirect", \
"External Attribute", \
"Inode Block", \
"Directory Contents", \
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
"User Data" }
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
extern char *buftype[];
#define BT_BUFTYPE(type) \
type < BT_NUMBUFTYPES ? buftype[type] : buftype[BT_UNKNOWN]
extern long readcnt[BT_NUMBUFTYPES];
extern long totalreadcnt[BT_NUMBUFTYPES];
extern struct timespec readtime[BT_NUMBUFTYPES];
extern struct timespec totalreadtime[BT_NUMBUFTYPES];
extern struct timespec startprog;
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
extern struct bufarea *icachebp; /* inode cache buffer */
extern struct bufarea sblk; /* file system superblock */
extern struct bufarea *pdirbp; /* current directory contents */
#define dirty(bp) do { \
if (fswritefd < 0) \
pfatal("SETTING DIRTY FLAG IN READ_ONLY MODE\n"); \
else \
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
(bp)->b_flags |= B_DIRTY; \
} while (0)
#define initbarea(bp, type) do { \
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
(bp)->b_bno = (ufs2_daddr_t)-4; \
(bp)->b_size = 0; \
(bp)->b_errs = 0; \
(bp)->b_flags = 0; \
(bp)->b_type = type; \
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
(bp)->b_refcnt = 0; \
(bp)->b_index = 0; \
} while (0)
#define sbdirty() dirty(&sblk)
#define sblock (*sblk.b_un.b_fs)
enum fixstate {DONTKNOW, NOFIX, FIX, IGNORE};
extern ino_t cursnapshot;
struct inodesc {
enum fixstate id_fix; /* policy on fixing errors */
int (*id_func)(struct inodesc *);
/* function to be applied to blocks of inode */
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
struct bufarea *id_bp; /* ckinode: buffer with indirect pointers */
union dinode *id_dp; /* ckinode: dinode being traversed */
ino_t id_number; /* inode number described */
ino_t id_parent; /* for DATA nodes, their parent */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
ufs_lbn_t id_lbn; /* logical block number of current block */
ufs2_daddr_t id_blkno; /* current block number being examined */
int id_level; /* level of indirection of this block */
int id_numfrags; /* number of frags contained in block */
ufs_lbn_t id_lballoc; /* pass1: last LBN that is allocated */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
off_t id_filesize; /* for DATA nodes, the size of the directory */
ufs2_daddr_t id_entryno;/* for DATA nodes, current entry number */
int id_loc; /* for DATA nodes, current location in dir */
struct direct *id_dirp; /* for DATA nodes, ptr to current entry */
char *id_name; /* for DATA nodes, name to find or enter */
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
char id_type; /* type of descriptor, DATA, ADDR, or SNAP */
};
/* file types */
#define DATA 1 /* a directory */
#define SNAP 2 /* a snapshot */
#define ADDR 3 /* anything but a directory or a snapshot */
/*
* Linked list of duplicate blocks.
1995-05-30 06:12:45 +00:00
*
* The list is composed of two parts. The first part of the
* list (from duplist through the node pointed to by muldup)
1995-05-30 06:12:45 +00:00
* contains a single copy of each duplicate block that has been
* found. The second part of the list (from muldup to the end)
* contains duplicate blocks that have been found more than once.
* To check if a block has been found as a duplicate it is only
1995-05-30 06:12:45 +00:00
* necessary to search from duplist through muldup. To find the
* total number of times that a block has been found as a duplicate
2012-01-07 16:09:33 +00:00
* the entire list must be searched for occurrences of the block
* in question. The following diagram shows a sample list where
* w (found twice), x (found once), y (found three times), and z
* (found once) are duplicate block numbers:
*
* w -> y -> x -> z -> y -> w -> y
* ^ ^
* | |
* duplist muldup
*/
struct dups {
struct dups *next;
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
ufs2_daddr_t dup;
};
extern struct dups *duplist; /* head of dup list */
extern struct dups *muldup; /* end of unique duplicate dup block numbers */
/*
* Inode cache data structures.
*/
struct inoinfo {
SLIST_ENTRY(inoinfo) i_hash; /* hash list */
ino_t i_number; /* inode number of this entry */
ino_t i_parent; /* inode number of parent */
ino_t i_dotdot; /* inode number of `..' */
size_t i_isize; /* size of inode */
Improvement in UFS/FFS directory placement when doing mkdir(2). The algorithm for laying out new directories was devised in the 1980s and markedly improved the performance of the filesystem. In those days large disks had at most 100 cylinder groups and often as few as 10-20. Modern multi-terrabyte disks have thousands of cylinder groups. The original algorithm does not handle these large sizes well. This change attempts to expand the scope of the original algorithm to work well with these much larger disks while still retaining the properties of the original algorithm for small disks. The filesystem implementation is divided into policy routines and implementation routines. The policy routines can be changed in any way desired without risk of corrupting the filesystem. The policy requests are handled by the implementation layer. If the policy asks for an available resource, it is granted. But if it asks for an already in-use resource, then the implementation will provide an available one nearby the request. Thus it is impossible for a policy to double allocate. This change is limited to the policy implementation. This change updates the ffs_dirpref() routine which is responsible for selecting the cylinder group into which a new directory should be placed. If we are near the root of the filesystem we aim to spread them out as much as possible. As we descend deeper from the root we cluster them closer together around their parent as we expect them to be more closely interactive. Higher-level directories like usr/src/sys and usr/src/bin should be separated while the directories in these areas are more likely to be accessed together so should be closer. And directories within commands or kernel subsystems should be closer still. We pick a range of cylinder groups around the cylinder group of the directory in which we are being created. The size of the range for our search is based on our depth from the root of our filesystem. We then probe that range based on how many directories are already present. The first new directory is at 1/2 (middle) of the range; the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8, 5/8, 7/8, 1/16, 3/16, 5/16, etc. It is desirable to store the depth of a directory in its on-disk inode so that it is available when we need it. We add a new field di_dirdepth to track the depth of each directory. Because there are few spare fields left in the inode, we choose to share an existing field in the inode rather than having one of our own. Specifically we create a union with the di_freelink field. The di_freelink field is used to track inodes that have been unlinked but remain referenced. It is not needed until a rmdir(2) operation has been done on a directory. At that point, the directory has no contents and even if it is kept active as a current directory is no longer able to have any new directories or files created in it. Thus the use of di_dirdepth and di_freelink will never coincide. Reported by: Timo Voelker Reviewed by: kib Tested by: Peter Holm MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D39246
2023-03-30 04:09:39 +00:00
u_int i_depth; /* depth of directory from root */
u_int i_flags; /* flags, see below */
u_int i_numblks; /* size of block array in bytes */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
ufs2_daddr_t i_blks[1]; /* actually longer */
};
extern SLIST_HEAD(inohash, inoinfo) *inphash;
extern struct inoinfo **inpsort;
/*
* flags for struct inoinfo
*/
#define INFO_NEW 0x0000001 /* replaced broken directory */
extern long dirhash, inplast;
extern unsigned long numdirs, listmax;
extern long countdirs; /* number of directories we actually found */
#define MIBSIZE 3 /* size of fsck sysctl MIBs */
extern int adjblkcnt[MIBSIZE]; /* MIB cmd to adjust inode block count */
extern int adjrefcnt[MIBSIZE]; /* MIB cmd to adjust inode reference count */
extern int adjndir[MIBSIZE]; /* MIB cmd to adjust number of directories */
extern int adjnbfree[MIBSIZE]; /* MIB cmd to adjust number of free blocks */
extern int adjnifree[MIBSIZE]; /* MIB cmd to adjust number of free inodes */
extern int adjnffree[MIBSIZE]; /* MIB cmd to adjust number of free frags */
extern int adjnumclusters[MIBSIZE]; /* MIB cmd adjust number of free clusters */
extern int adjdepth[MIBSIZE]; /* MIB cmd to adjust directory depth count */
extern int freefiles[MIBSIZE]; /* MIB cmd to free a set of files */
extern int freedirs[MIBSIZE]; /* MIB cmd to free a set of directories */
extern int freeblks[MIBSIZE]; /* MIB cmd to free a set of data blocks */
extern int setsize[MIBSIZE]; /* MIB cmd to set inode size */
extern struct fsck_cmd cmd; /* sysctl file system update commands */
extern int bkgrdcheck; /* determine if background check is possible */
extern int bkgrdsumadj; /* whether the kernel has the ability to adjust
the superblock summary fields */
extern off_t bflag; /* location of alternate super block */
extern int bkgrdflag; /* use a snapshot to run on an active system */
extern char *blockmap; /* ptr to primary blk allocation map */
extern char *cdevname; /* name of device being checked */
extern int cgheader_corrupt; /* one or more CG headers are corrupt */
extern char ckclean; /* only do work if not cleanly unmounted */
extern int ckhashadd; /* check hashes to be added */
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
extern char *copybuf; /* buffer to copy snapshot blocks */
extern int cvtlevel; /* convert to newer file system format */
extern long dev_bsize; /* computed value of DEV_BSIZE */
extern u_int real_dev_bsize; /* actual disk sector size, not overridden */
extern int debug; /* output debugging info */
extern int Eflag; /* delete empty data blocks */
extern int fsmodified; /* 1 => write done to file system */
extern int fsreadfd; /* file descriptor for reading file system */
extern int fswritefd; /* file descriptor for writing file system */
extern char havesb; /* superblock has been read */
extern int inoopt; /* trim out unused inodes */
extern ino_t lfdir; /* lost & found directory inode number */
extern int lfmode; /* lost & found directory creation mode */
extern const char *lfname; /* lost & found directory name */
extern ufs2_daddr_t maxfsblock; /* number of blocks in the file system */
extern ino_t maxino; /* number of inodes in file system */
extern ufs2_daddr_t n_blks; /* number of blocks in use */
extern ino_t n_files; /* number of files in use */
extern char nflag; /* assume a no response */
extern char preen; /* just fix normal inconsistencies */
extern char rerun; /* rerun fsck. Only used in non-preen mode */
extern char resolved; /* cleared if unresolved changes => not clean */
extern int returntosingle; /* 1 => return to single user mode on exit */
extern long secsize; /* actual disk sector size */
extern char skipclean; /* skip clean file systems if preening */
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
extern int snapcnt; /* number of active snapshots */
extern struct inode snaplist[FSMAXSNAP + 1]; /* list of active snapshots */
extern char snapname[BUFSIZ]; /* when doing snapshots, the name of the file */
extern int sujrecovery; /* 1 => doing check using the journal */
extern int surrender; /* Give up if reads fail */
extern char usedsoftdep; /* just fix soft dependency inconsistencies */
extern int wantrestart; /* Restart fsck on early termination */
extern char yflag; /* assume a yes response */
extern int zflag; /* zero unused directory space */
extern int Zflag; /* zero empty data blocks */
extern volatile sig_atomic_t got_siginfo; /* received a SIGINFO */
extern volatile sig_atomic_t got_sigalarm; /* received a SIGALRM */
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
#define clearinode(dp) \
if (sblock.fs_magic == FS_UFS1_MAGIC) { \
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
(dp)->dp1 = zino.dp1; \
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
} else { \
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
(dp)->dp2 = zino.dp2; \
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
}
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
extern union dinode zino;
#define setbmap(blkno) setbit(blockmap, blkno)
#define testbmap(blkno) isset(blockmap, blkno)
#define clrbmap(blkno) clrbit(blockmap, blkno)
#define STOP 0x01
#define SKIP 0x02
#define KEEPON 0x04
#define ALTERED 0x08
#define FOUND 0x10
#define EEXIT 8 /* Standard error exit. */
#define ERERUN 16 /* fsck needs to be re-run. */
#define ERESTART -1
int flushentry(void);
/*
* Wrapper for malloc() that flushes the cylinder group cache to try
* to get space.
*/
static inline void*
Malloc(size_t size)
{
void *retval;
while ((retval = malloc(size)) == NULL)
if (flushentry() == 0)
break;
return (retval);
}
/*
* Wrapper for calloc() that flushes the cylinder group cache to try
* to get space.
*/
static inline void*
Calloc(size_t cnt, size_t size)
{
void *retval;
while ((retval = calloc(cnt, size)) == NULL)
if (flushentry() == 0)
break;
return (retval);
}
struct fstab;
void adjust(struct inodesc *, int lcnt);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void alarmhandler(int sig);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
ufs2_daddr_t allocblk(long cg, long frags, ufs2_daddr_t (*checkblkavail)
(ufs2_daddr_t blkno, long frags));
ino_t allocdir(ino_t parent, ino_t request, int mode);
ino_t allocino(ino_t request, int type);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
void binval(struct bufarea *);
void blkerror(ino_t ino, const char *type, ufs2_daddr_t blk);
char *blockcheck(char *name);
int blread(int fd, char *buf, ufs2_daddr_t blk, long size);
void bufinit(void);
void blwrite(int fd, char *buf, ufs2_daddr_t blk, ssize_t size);
void blerase(int fd, ufs2_daddr_t blk, long size);
void blzero(int fd, ufs2_daddr_t blk, long size);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void brelse(struct bufarea *);
struct inoinfo *cacheino(union dinode *dp, ino_t inumber);
void catch(int);
void catchquit(int);
void cgdirty(struct bufarea *);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
struct bufarea *cglookup(int cg);
Improvement in UFS/FFS directory placement when doing mkdir(2). The algorithm for laying out new directories was devised in the 1980s and markedly improved the performance of the filesystem. In those days large disks had at most 100 cylinder groups and often as few as 10-20. Modern multi-terrabyte disks have thousands of cylinder groups. The original algorithm does not handle these large sizes well. This change attempts to expand the scope of the original algorithm to work well with these much larger disks while still retaining the properties of the original algorithm for small disks. The filesystem implementation is divided into policy routines and implementation routines. The policy routines can be changed in any way desired without risk of corrupting the filesystem. The policy requests are handled by the implementation layer. If the policy asks for an available resource, it is granted. But if it asks for an already in-use resource, then the implementation will provide an available one nearby the request. Thus it is impossible for a policy to double allocate. This change is limited to the policy implementation. This change updates the ffs_dirpref() routine which is responsible for selecting the cylinder group into which a new directory should be placed. If we are near the root of the filesystem we aim to spread them out as much as possible. As we descend deeper from the root we cluster them closer together around their parent as we expect them to be more closely interactive. Higher-level directories like usr/src/sys and usr/src/bin should be separated while the directories in these areas are more likely to be accessed together so should be closer. And directories within commands or kernel subsystems should be closer still. We pick a range of cylinder groups around the cylinder group of the directory in which we are being created. The size of the range for our search is based on our depth from the root of our filesystem. We then probe that range based on how many directories are already present. The first new directory is at 1/2 (middle) of the range; the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8, 5/8, 7/8, 1/16, 3/16, 5/16, etc. It is desirable to store the depth of a directory in its on-disk inode so that it is available when we need it. We add a new field di_dirdepth to track the depth of each directory. Because there are few spare fields left in the inode, we choose to share an existing field in the inode rather than having one of our own. Specifically we create a union with the di_freelink field. The di_freelink field is used to track inodes that have been unlinked but remain referenced. It is not needed until a rmdir(2) operation has been done on a directory. At that point, the directory has no contents and even if it is kept active as a current directory is no longer able to have any new directories or files created in it. Thus the use of di_dirdepth and di_freelink will never coincide. Reported by: Timo Voelker Reviewed by: kib Tested by: Peter Holm MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D39246
2023-03-30 04:09:39 +00:00
int changeino(ino_t dir, const char *name, ino_t newnum, int depth);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
void check_blkcnt(struct inode *ip);
int check_cgmagic(int cg, struct bufarea *cgbp);
void rebuild_cg(int cg, struct bufarea *cgbp);
Improvement in UFS/FFS directory placement when doing mkdir(2). The algorithm for laying out new directories was devised in the 1980s and markedly improved the performance of the filesystem. In those days large disks had at most 100 cylinder groups and often as few as 10-20. Modern multi-terrabyte disks have thousands of cylinder groups. The original algorithm does not handle these large sizes well. This change attempts to expand the scope of the original algorithm to work well with these much larger disks while still retaining the properties of the original algorithm for small disks. The filesystem implementation is divided into policy routines and implementation routines. The policy routines can be changed in any way desired without risk of corrupting the filesystem. The policy requests are handled by the implementation layer. If the policy asks for an available resource, it is granted. But if it asks for an already in-use resource, then the implementation will provide an available one nearby the request. Thus it is impossible for a policy to double allocate. This change is limited to the policy implementation. This change updates the ffs_dirpref() routine which is responsible for selecting the cylinder group into which a new directory should be placed. If we are near the root of the filesystem we aim to spread them out as much as possible. As we descend deeper from the root we cluster them closer together around their parent as we expect them to be more closely interactive. Higher-level directories like usr/src/sys and usr/src/bin should be separated while the directories in these areas are more likely to be accessed together so should be closer. And directories within commands or kernel subsystems should be closer still. We pick a range of cylinder groups around the cylinder group of the directory in which we are being created. The size of the range for our search is based on our depth from the root of our filesystem. We then probe that range based on how many directories are already present. The first new directory is at 1/2 (middle) of the range; the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8, 5/8, 7/8, 1/16, 3/16, 5/16, etc. It is desirable to store the depth of a directory in its on-disk inode so that it is available when we need it. We add a new field di_dirdepth to track the depth of each directory. Because there are few spare fields left in the inode, we choose to share an existing field in the inode rather than having one of our own. Specifically we create a union with the di_freelink field. The di_freelink field is used to track inodes that have been unlinked but remain referenced. It is not needed until a rmdir(2) operation has been done on a directory. At that point, the directory has no contents and even if it is kept active as a current directory is no longer able to have any new directories or files created in it. Thus the use of di_dirdepth and di_freelink will never coincide. Reported by: Timo Voelker Reviewed by: kib Tested by: Peter Holm MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D39246
2023-03-30 04:09:39 +00:00
void check_dirdepth(struct inoinfo *inp);
int chkfilesize(mode_t mode, u_int64_t filesize);
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
int chkrange(ufs2_daddr_t blk, int cnt);
void ckfini(int markclean);
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
int ckinode(union dinode *dp, struct inodesc *);
void clri(struct inodesc *, const char *type, int flag);
int clearentry(struct inodesc *);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
void copyonwrite(struct fs *, struct bufarea *,
ufs2_daddr_t (*checkblkavail)(ufs2_daddr_t, long));
void direrror(ino_t ino, const char *errmesg);
int dirscan(struct inodesc *);
int dofix(struct inodesc *, const char *msg);
int eascan(struct inodesc *, struct ufs2_dinode *dp);
void fileerror(ino_t cwd, ino_t ino, const char *errmesg);
void finalIOstats(void);
int findino(struct inodesc *);
int findname(struct inodesc *);
void flush(int fd, struct bufarea *bp);
int freeblock(struct inodesc *);
void freedirino(ino_t ino, ino_t parent);
void freeino(ino_t ino);
void freeinodebuf(void);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
void fsckinit(void);
void fsutilinit(void);
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 06:18:05 +00:00
int ftypeok(union dinode *dp);
void getblk(struct bufarea *bp, ufs2_daddr_t blk, long size);
struct bufarea *getdatablk(ufs2_daddr_t blkno, long size, int type);
struct inoinfo *getinoinfo(ino_t inumber);
union dinode *getnextinode(ino_t inumber, int rebuiltcg);
void getpathname(char *namebuf, ino_t curdir, ino_t ino);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void ginode(ino_t, struct inode *);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
void gjournal_check(const char *filesys);
void infohandler(int sig);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void irelse(struct inode *);
ufs2_daddr_t ino_blkatoff(union dinode *, ino_t, ufs_lbn_t, int *,
struct bufarea **);
void inocleanup(void);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void inodirty(struct inode *);
struct inostat *inoinfo(ino_t inum);
void IOstats(char *what);
int linkup(ino_t orphan, ino_t parentdir, char *name);
int makeentry(ino_t parent, ino_t ino, const char *name);
int openfilesys(char *dev);
void panic(const char *fmt, ...) __printflike(1, 2);
void pass1(void);
void pass1b(void);
int pass1check(struct inodesc *);
void pass2(void);
void pass3(void);
void pass4(void);
void pass5(void);
void pfatal(const char *fmt, ...) __printflike(1, 2);
void propagate(void);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
void prtbuf(struct bufarea *, const char *, ...) __printflike(2, 3);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void prtinode(struct inode *);
void pwarn(const char *fmt, ...) __printflike(1, 2);
Move the ability to search for alternate UFS superblocks from fsck_ffs(8) into ffs_sbsearch() to allow use by other parts of the system. Historically only fsck_ffs(8), the UFS filesystem checker, had code to track down and use alternate UFS superblocks. Since fsdb(8) used much of the fsck_ffs(8) implementation it had some ability to track down alternate superblocks. This change extracts the code to track down alternate superblocks from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found in ffs_subr.c, these functions can be used directly by the kernel subsystems. Additionally they are exported to the UFS library, libufs(8) so that they can be used by user-level programs. The new functions added to libufs(8) are sbfind(3) that is an alternative to sbread(3) and sbsearch(3) that is an alternative to sbget(3). See their manual pages for further details. The utilities that have been changed to search for superblocks are dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8) tool found in tools/diag/prtblknos searches for superblocks. The UFS specific mount code uses the superblock search interface when mounting the root filesystem and when the administrator doing a mount(8) command specifies the force flag (-f). The standalone UFS boot code (found in stand/libsa/ufs.c) uses the superblock search code in the hope of being able to get the system up and running so that fsck_ffs(8) can be used to get the filesystem cleaned up. The following utilities have not been changed to search for superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8), dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and glabel(8). When these utilities fail, they do report the cause of the failure. The one exception is the tasting code used to try and figure what a given disk contains. The tasting code will remain silent so as not to put out a slew of messages as it trying to taste every new mass storage device that shows up. Reviewed by: kib Reviewed by: Warner Losh Tested by: Peter Holm Differential Revision: https://reviews.freebsd.org/D36053 Sponsored by: The FreeBSD Foundation
2022-08-13 19:41:53 +00:00
int readsb(void);
int removecachedino(ino_t);
int reply(const char *question);
void rwerror(const char *mesg, ufs2_daddr_t blk);
void sblock_init(void);
Rewrite the disk I/O management system in fsck_ffs(8). Other than making fsck_ffs(8) run faster, there should be no functional change. The original fsck_ffs(8) had its own disk I/O management system. When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8) to do the necessary gjournal rollback. Rather than use the existing fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly when journalled soft updates were added in FreeBSD 9, code was added to fsck_ffs(8) to do the necessary journal rollback. And once again, rather than using either of the existing fsck_ffs(8) disk I/O systems, it wrote its own from scratch. Lastly the fsdb(8) utility uses the fsck_ffs(8) disk I/O management system. In preparation for making the changes necessary to enable snapshots to be taken when using journalled soft updates, it was necessary to have a single disk I/O system used by all the various subsystems in fsck_ffs(8). This commit merges the functionality required by all the different subsystems into a single disk I/O system that supports all of their needs. In so doing it picks up optimizations from each of them with the results that each of the subsystems does fewer reads and writes than it did with its own customized I/O system. It also greatly simplifies making changes to fsck_ffs(8) since everything goes through a single place. For example the ginode() function fetches an inode from the disk. When inode check hashes were added, they previously had to be checked in the code implementing inode fetch in each of the three different disk I/O systems. Now they need only be checked in ginode(). Tested by: Peter Holm Sponsored by: Netflix
2021-01-07 01:37:08 +00:00
void setinodebuf(int, ino_t);
int setup(char *dev);
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
int snapblkfree(struct fs *, ufs2_daddr_t, long, ino_t,
ufs2_daddr_t (*)(ufs2_daddr_t, long));
void snapremove(ino_t);
void snapflush(ufs2_daddr_t (*checkblkavail)(ufs2_daddr_t, long));
Add support for managing UFS/FFS snapshots to fsck_ffs(8). The kernel handles the managment of UFS/FFS snapshots. Since UFS/FFS updates filesystem data (rather than always writing changes to new locations like ZFS), the kernel must check every filesystem write to see if the block being written is part of a snapshot. If it is part of a snapshot, then the kernel must make a copy of the old block value into a newly allocated block for the snapshot before allowing the write to be done. Similarly, if a block is being freed, the kernel must check to see if it is part of a snapshot and let the snapshot claim the block rather than freeing it for future use. When a snapshot is freed, its blocks need to be offered to older snapshots and freed only if no older snapshots wish to claim them. When snapshots were added to UFS/FFS they were integrated into soft updates and just a small part of the management of snapshots needed to be added to fsck_ffs(8) as soft updates minimized the set of snapshot changes that might need correction. When journaling was added to soft updates a much more complete knowledge of snapshots needed to be added to fsck_ffs(8) for it to be able to properly handle the filesystem changes that a journal rollback needs to do (specifically the freeing and allocation of blocks). Since this functionality was unavailable, the use of snapshots was disabled when running with journaled soft updates. This set of changes imports the kernel code for the management of snapshots to fsck_ffs(8). With this code in place it will become possible to enable snapshots when running with journalled soft updates. The most immediate benefit will be the ability to use snapshots to take consistent filesystem dumps on live filesystems. Future work will be done to update fsck_ffs(8) to be able to use snapshots to run in background on live filesystems running with journaled soft updates. Reviewed by: kib Tested by: Peter Holm Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36491
2022-11-09 18:44:03 +00:00
ufs2_daddr_t std_checkblkavail(ufs2_daddr_t blkno, long frags);
ufs2_daddr_t suj_checkblkavail(ufs2_daddr_t, long);
int suj_check(const char *filesys);
void update_maps(struct cg *, struct cg*, int);
#endif /* !_FSCK_H_ */