d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
e7b433ef19
freebsd-dev/sys/ufs/ffs/ffs_softdep.c

4808 lines
144 KiB
C
Raw Normal View History

Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
Update to new copyright.
2000-06-22 00:29:53 +00:00
* Copyright 1998, 2000 Marshall Kirk McKusick. All Rights Reserved.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
* The soft updates code is derived from the appendix of a University
* of Michigan technical report (Gregory R. Ganger and Yale N. Patt,
* "Soft Updates: A Solution to the Metadata Update Problem in File
* Systems", CSE-TR-254-95, August 1995).
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*
Update to new copyright.
2000-06-22 00:29:53 +00:00
* Further information about soft updates can be obtained from:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*
Update to new copyright.
2000-06-22 00:29:53 +00:00
* Marshall Kirk McKusick http://www.mckusick.com/softdep/
* 1614 Oxford Street mckusick@mckusick.com
* Berkeley, CA 94709-1608 +1-510-843-9542
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
* USA
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY MARSHALL KIRK MCKUSICK ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL MARSHALL KIRK MCKUSICK BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
Update to new copyright.
2000-06-22 00:29:53 +00:00
* from: @(#)ffs_softdep.c 9.59 (McKusick) 6/21/00
$Id$ -> $FreeBSD$
1999-08-28 02:16:32 +00:00
* $FreeBSD$
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
*/
/*
* For now we want the safety net that the DIAGNOSTIC and DEBUG flags provide.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
#ifndef DIAGNOSTIC
#define DIAGNOSTIC
#endif
#ifndef DEBUG
#define DEBUG
#endif
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#include <sys/param.h>
#include <sys/kernel.h>
Keep the inlines for <sys/buf.h> happy..
1999-06-27 13:26:23 +00:00
#include <sys/systm.h>
Separate the struct bio related stuff out of <sys/buf.h> into <sys/bio.h>. <sys/bio.h> is now a prerequisite for <sys/buf.h> but it shall not be made a nested include according to bdes teachings on the subject of nested includes. Diskdrivers and similar stuff below specfs::strategy() should no longer need to include <sys/buf.> unless they need caching of data. Still a few bogus uses of struct buf to track down. Repocopy by: peter
2000-05-05 09:59:14 +00:00
#include <sys/bio.h>
Keep the inlines for <sys/buf.h> happy..
1999-06-27 13:26:23 +00:00
#include <sys/buf.h>
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#include <sys/malloc.h>
#include <sys/mount.h>
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
#include <sys/proc.h>
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#include <sys/syslog.h>
#include <sys/vnode.h>
Decommision miscfs/specfs/specdev.h. Most of it goes into <sys/conf.h>, a few lines into <sys/vnode.h>. Add a few fields to struct specinfo, paving the way for the fun part.
1999-08-08 18:43:05 +00:00
#include <sys/conf.h>
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#include <ufs/ufs/dir.h>
Introduce extended attribute support for FFS, allowing arbitrary (name, value) pairs to be associated with inodes. This support is used for ACLs, MAC labels, and Capabilities in the TrustedBSD security extensions, which are currently under development. In this implementation, attributes are backed to data vnodes in the style of the quota support in FFS. Support for FFS extended attributes may be enabled using the FFS_EXTATTR kernel option (disabled by default). Userland utilities and man pages will be committed in the next batch. VFS interfaces and man pages have been in the repo since 4.0-RELEASE and are unchanged. o ufs/ufs/extattr.h: UFS-specific extattr defines o ufs/ufs/ufs_extattr.c: bulk of support routines o ufs/{ufs,ffs,mfs}/*.[ch]: hooks and extattr.h includes o contrib/softupdates/ffs_softdep.c: extattr.h includes o conf/options, conf/files, i386/conf/LINT: added FFS_EXTATTR o coda/coda_vfsops.c: XXX required extattr.h due to ufsmount.h (This should not be the case, and will be fixed in a future commit) Currently attributes are not supported in MFS. This will be fixed. Reviewed by: adrian, bp, freebsd-fs, other unthanked souls Obtained from: TrustedBSD Project
2000-04-15 03:34:27 +00:00
#include <ufs/ufs/extattr.h>
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#include <ufs/ufs/quota.h>
#include <ufs/ufs/inode.h>
#include <ufs/ufs/ufsmount.h>
#include <ufs/ffs/fs.h>
#include <ufs/ffs/softdep.h>
#include <ufs/ffs/ffs_extern.h>
#include <ufs/ufs/ufs_extern.h>
/*
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
* These definitions need to be adapted to the system to which
* this file is being ported.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
/*
* malloc types defined for the softdep system.
*/
Staticize some malloc M_ instances.
2000-12-08 20:09:00 +00:00
static MALLOC_DEFINE(M_PAGEDEP, "pagedep","File page dependencies");
static MALLOC_DEFINE(M_INODEDEP, "inodedep","Inode dependencies");
static MALLOC_DEFINE(M_NEWBLK, "newblk","New block allocation");
static MALLOC_DEFINE(M_BMSAFEMAP, "bmsafemap","Block or frag allocated from cyl group map");
static MALLOC_DEFINE(M_ALLOCDIRECT, "allocdirect","Block or frag dependency for an inode");
static MALLOC_DEFINE(M_INDIRDEP, "indirdep","Indirect block dependencies");
static MALLOC_DEFINE(M_ALLOCINDIR, "allocindir","Block dependency for an indirect block");
static MALLOC_DEFINE(M_FREEFRAG, "freefrag","Previously used frag for an inode");
static MALLOC_DEFINE(M_FREEBLKS, "freeblks","Blocks freed from an inode");
static MALLOC_DEFINE(M_FREEFILE, "freefile","Inode deallocated");
static MALLOC_DEFINE(M_DIRADD, "diradd","New directory entry");
static MALLOC_DEFINE(M_MKDIR, "mkdir","New directory");
static MALLOC_DEFINE(M_DIRREM, "dirrem","Directory entry deleted");
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
#define M_SOFTDEP_FLAGS (M_WAITOK | M_USE_RESERVE)
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
#define D_PAGEDEP 0
#define D_INODEDEP 1
#define D_NEWBLK 2
#define D_BMSAFEMAP 3
#define D_ALLOCDIRECT 4
#define D_INDIRDEP 5
#define D_ALLOCINDIR 6
#define D_FREEFRAG 7
#define D_FREEBLKS 8
#define D_FREEFILE 9
#define D_DIRADD 10
#define D_MKDIR 11
#define D_DIRREM 12
#define D_LAST D_DIRREM
/*
* translate from workitem type to memory type
* MUST match the defines above, such that memtype[D_XXX] == M_XXX
*/
static struct malloc_type *memtype[] = {
M_PAGEDEP,
M_INODEDEP,
M_NEWBLK,
M_BMSAFEMAP,
M_ALLOCDIRECT,
M_INDIRDEP,
M_ALLOCINDIR,
M_FREEFRAG,
M_FREEBLKS,
M_FREEFILE,
M_DIRADD,
M_MKDIR,
M_DIRREM
};
#define DtoM(type) (memtype[type])
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Names of malloc types.
*/
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
#define TYPENAME(type) \
((unsigned)(type) < D_LAST ? memtype[type]->ks_shortdesc : "???")
/*
* End system adaptaion definitions.
*/
/*
* Internal function prototypes.
*/
static void softdep_error __P((char *, int));
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
static void drain_output __P((struct vnode *, int));
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
static int getdirtybuf __P((struct buf **, int));
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
static void clear_remove __P((struct proc *));
static void clear_inodedeps __P((struct proc *));
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
static int flush_pagedep_deps __P((struct vnode *, struct mount *,
struct diraddhd *));
static int flush_inodedep_deps __P((struct fs *, ino_t));
static int handle_written_filepage __P((struct pagedep *, struct buf *));
static void diradd_inode_written __P((struct diradd *, struct inodedep *));
static int handle_written_inodeblock __P((struct inodedep *, struct buf *));
static void handle_allocdirect_partdone __P((struct allocdirect *));
static void handle_allocindir_partdone __P((struct allocindir *));
static void initiate_write_filepage __P((struct pagedep *, struct buf *));
static void handle_written_mkdir __P((struct mkdir *, int));
static void initiate_write_inodeblock __P((struct inodedep *, struct buf *));
static void handle_workitem_freefile __P((struct freefile *));
static void handle_workitem_remove __P((struct dirrem *));
static struct dirrem *newdirrem __P((struct buf *, struct inode *,
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
struct inode *, int, struct dirrem **));
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
static void free_diradd __P((struct diradd *));
static void free_allocindir __P((struct allocindir *, struct inodedep *));
static int indir_trunc __P((struct inode *, ufs_daddr_t, int, ufs_lbn_t,
long *));
static void deallocate_dependencies __P((struct buf *, struct inodedep *));
static void free_allocdirect __P((struct allocdirectlst *,
struct allocdirect *, int));
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
static int check_inode_unwritten __P((struct inodedep *));
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
static int free_inodedep __P((struct inodedep *));
static void handle_workitem_freeblocks __P((struct freeblks *));
static void merge_inode_lists __P((struct inodedep *));
static void setup_allocindir_phase2 __P((struct buf *, struct inode *,
struct allocindir *));
static struct allocindir *newallocindir __P((struct inode *, int, ufs_daddr_t,
ufs_daddr_t));
static void handle_workitem_freefrag __P((struct freefrag *));
static struct freefrag *newfreefrag __P((struct inode *, ufs_daddr_t, long));
static void allocdirect_merge __P((struct allocdirectlst *,
struct allocdirect *, struct allocdirect *));
static struct bmsafemap *bmsafemap_lookup __P((struct buf *));
static int newblk_lookup __P((struct fs *, ufs_daddr_t, int,
struct newblk **));
static int inodedep_lookup __P((struct fs *, ino_t, int, struct inodedep **));
static int pagedep_lookup __P((struct inode *, ufs_lbn_t, int,
struct pagedep **));
static void pause_timer __P((void *));
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
static int request_cleanup __P((int, int));
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
static int process_worklist_item __P((struct mount *, int));
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
static void add_to_worklist __P((struct worklist *));
/*
* Exported softdep operations.
*/
Make static non-exported functions from soft updates.
2000-01-09 22:40:09 +00:00
static void softdep_disk_io_initiation __P((struct buf *));
static void softdep_disk_write_complete __P((struct buf *));
static void softdep_deallocate_dependencies __P((struct buf *));
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
static void softdep_move_dependencies __P((struct buf *, struct buf *));
static int softdep_count_dependencies __P((struct buf *bp, int));
Make static non-exported functions from soft updates.
2000-01-09 22:40:09 +00:00
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
struct bio_ops bioops = {
softdep_disk_io_initiation, /* io_start */
softdep_disk_write_complete, /* io_complete */
softdep_deallocate_dependencies, /* io_deallocate */
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
softdep_move_dependencies, /* io_movedeps */
softdep_count_dependencies, /* io_countdeps */
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
};
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Locking primitives.
*
* For a uniprocessor, all we need to do is protect against disk
* interrupts. For a multiprocessor, this lock would have to be
* a mutex. A single mutex is used throughout this file, though
* finer grain locking could be used if contention warranted it.
*
* For a multiprocessor, the sleep call would accept a lock and
* release it after the sleep processing was complete. In a uniprocessor
* implementation there is no such interlock, so we simple mark
* the places where it needs to be done with the `interlocked' form
* of the lock calls. Since the uniprocessor sleep already interlocks
* the spl, there is nothing that really needs to be done.
*/
#ifndef /* NOT */ DEBUG
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
static struct lockit {
int lkt_spl;
} lk = { 0 };
#define ACQUIRE_LOCK(lk) (lk)->lkt_spl = splbio()
#define FREE_LOCK(lk) splx((lk)->lkt_spl)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#define ACQUIRE_LOCK_INTERLOCKED(lk)
#define FREE_LOCK_INTERLOCKED(lk)
#else /* DEBUG */
static struct lockit {
int lkt_spl;
pid_t lkt_held;
} lk = { 0, -1 };
static int lockcnt;
static void acquire_lock __P((struct lockit *));
static void free_lock __P((struct lockit *));
static void acquire_lock_interlocked __P((struct lockit *));
static void free_lock_interlocked __P((struct lockit *));
#define ACQUIRE_LOCK(lk) acquire_lock(lk)
#define FREE_LOCK(lk) free_lock(lk)
#define ACQUIRE_LOCK_INTERLOCKED(lk) acquire_lock_interlocked(lk)
#define FREE_LOCK_INTERLOCKED(lk) free_lock_interlocked(lk)
static void
acquire_lock(lk)
struct lockit *lk;
{
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
if (lk->lkt_held != -1) {
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
if (lk->lkt_held == CURPROC->p_pid)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("softdep_lock: locking against myself");
else
panic("softdep_lock: lock held by %d", lk->lkt_held);
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
lk->lkt_spl = splbio();
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
lk->lkt_held = CURPROC->p_pid;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
lockcnt++;
}
static void
free_lock(lk)
struct lockit *lk;
{
if (lk->lkt_held == -1)
panic("softdep_unlock: lock not held");
lk->lkt_held = -1;
splx(lk->lkt_spl);
}
static void
acquire_lock_interlocked(lk)
struct lockit *lk;
{
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
if (lk->lkt_held != -1) {
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
if (lk->lkt_held == CURPROC->p_pid)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("softdep_lock_interlocked: locking against self");
else
panic("softdep_lock_interlocked: lock held by %d",
lk->lkt_held);
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
}
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
lk->lkt_held = CURPROC->p_pid;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
lockcnt++;
}
static void
free_lock_interlocked(lk)
struct lockit *lk;
{
if (lk->lkt_held == -1)
panic("softdep_unlock_interlocked: lock not held");
lk->lkt_held = -1;
}
#endif /* DEBUG */
/*
* Place holder for real semaphores.
*/
struct sema {
int value;
pid_t holder;
char *name;
int prio;
int timo;
};
static void sema_init __P((struct sema *, char *, int, int));
static int sema_get __P((struct sema *, struct lockit *));
static void sema_release __P((struct sema *));
static void
sema_init(semap, name, prio, timo)
struct sema *semap;
char *name;
int prio, timo;
{
semap->holder = -1;
semap->value = 0;
semap->name = name;
semap->prio = prio;
semap->timo = timo;
}
static int
sema_get(semap, interlock)
struct sema *semap;
struct lockit *interlock;
{
if (semap->value++ > 0) {
if (interlock != NULL)
FREE_LOCK_INTERLOCKED(interlock);
tsleep((caddr_t)semap, semap->prio, semap->name, semap->timo);
if (interlock != NULL) {
ACQUIRE_LOCK_INTERLOCKED(interlock);
FREE_LOCK(interlock);
}
return (0);
}
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
semap->holder = CURPROC->p_pid;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (interlock != NULL)
FREE_LOCK(interlock);
return (1);
}
static void
sema_release(semap)
struct sema *semap;
{
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
if (semap->value <= 0 || semap->holder != CURPROC->p_pid)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("sema_release: not held");
if (--semap->value > 0) {
semap->value = 0;
wakeup(semap);
}
semap->holder = -1;
}
/*
* Worklist queue management.
* These routines require that the lock be held.
*/
#ifndef /* NOT */ DEBUG
#define WORKLIST_INSERT(head, item) do { \
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
(item)->wk_state |= ONWORKLIST; \
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_INSERT_HEAD(head, item, wk_list); \
} while (0)
#define WORKLIST_REMOVE(item) do { \
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
(item)->wk_state &= ~ONWORKLIST; \
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_REMOVE(item, wk_list); \
} while (0)
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
#define WORKITEM_FREE(item, type) FREE(item, DtoM(type))
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#else /* DEBUG */
static void worklist_insert __P((struct workhead *, struct worklist *));
static void worklist_remove __P((struct worklist *));
static void workitem_free __P((struct worklist *, int));
#define WORKLIST_INSERT(head, item) worklist_insert(head, item)
#define WORKLIST_REMOVE(item) worklist_remove(item)
#define WORKITEM_FREE(item, type) workitem_free((struct worklist *)item, type)
static void
worklist_insert(head, item)
struct workhead *head;
struct worklist *item;
{
if (lk.lkt_held == -1)
panic("worklist_insert: lock not held");
if (item->wk_state & ONWORKLIST)
panic("worklist_insert: already on list");
item->wk_state |= ONWORKLIST;
LIST_INSERT_HEAD(head, item, wk_list);
}
static void
worklist_remove(item)
struct worklist *item;
{
if (lk.lkt_held == -1)
panic("worklist_remove: lock not held");
if ((item->wk_state & ONWORKLIST) == 0)
panic("worklist_remove: not on list");
item->wk_state &= ~ONWORKLIST;
LIST_REMOVE(item, wk_list);
}
static void
workitem_free(item, type)
struct worklist *item;
int type;
{
if (item->wk_state & ONWORKLIST)
panic("workitem_free: still on list");
if (item->wk_type != type)
panic("workitem_free: type mismatch");
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
FREE(item, DtoM(type));
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
#endif /* DEBUG */
/*
* Workitem queue management
*/
static struct workhead softdep_workitem_pending;
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
static int num_on_worklist; /* number of worklist items to be processed */
static int softdep_worklist_busy; /* 1 => trying to do unmount */
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
static int softdep_worklist_req; /* serialized waiters */
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
static int max_softdeps; /* maximum number of structs before slowdown */
static int tickdelay = 2; /* number of ticks to pause during slowdown */
static int proc_waiting; /* tracks whether we have a timeout posted */
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
static int *stat_countp; /* statistic to count in proc_waiting timeout */
static struct callout_handle handle; /* handle on posted proc_waiting timeout */
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
static struct proc *filesys_syncer; /* proc of filesystem syncer process */
static int req_clear_inodedeps; /* syncer process flush some inodedeps */
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
#define FLUSH_INODES 1
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
static int req_clear_remove; /* syncer process flush some freeblks */
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
#define FLUSH_REMOVE 2
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
/*
* runtime statistics
*/
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
static int stat_worklist_push; /* number of worklist cleanups */
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
static int stat_blk_limit_push; /* number of times block limit neared */
static int stat_ino_limit_push; /* number of times inode limit neared */
static int stat_blk_limit_hit; /* number of times block slowdown imposed */
static int stat_ino_limit_hit; /* number of times inode slowdown imposed */
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
static int stat_sync_limit_hit; /* number of synchronous slowdowns imposed */
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
static int stat_indir_blk_ptrs; /* bufs redirtied as indir ptrs not written */
static int stat_inode_bitmap; /* bufs redirtied as inode bitmap not written */
static int stat_direct_blk_ptrs;/* bufs redirtied as direct ptrs not written */
static int stat_dir_entry; /* bufs redirtied as dir entry cannot write */
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
#ifdef DEBUG
#include <vm/vm.h>
#include <sys/sysctl.h>
SYSCTL_INT(_debug, OID_AUTO, max_softdeps, CTLFLAG_RW, &max_softdeps, 0, "");
SYSCTL_INT(_debug, OID_AUTO, tickdelay, CTLFLAG_RW, &tickdelay, 0, "");
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
SYSCTL_INT(_debug, OID_AUTO, worklist_push, CTLFLAG_RW, &stat_worklist_push, 0,"");
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
SYSCTL_INT(_debug, OID_AUTO, blk_limit_push, CTLFLAG_RW, &stat_blk_limit_push, 0,"");
SYSCTL_INT(_debug, OID_AUTO, ino_limit_push, CTLFLAG_RW, &stat_ino_limit_push, 0,"");
SYSCTL_INT(_debug, OID_AUTO, blk_limit_hit, CTLFLAG_RW, &stat_blk_limit_hit, 0, "");
SYSCTL_INT(_debug, OID_AUTO, ino_limit_hit, CTLFLAG_RW, &stat_ino_limit_hit, 0, "");
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
SYSCTL_INT(_debug, OID_AUTO, sync_limit_hit, CTLFLAG_RW, &stat_sync_limit_hit, 0, "");
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
SYSCTL_INT(_debug, OID_AUTO, indir_blk_ptrs, CTLFLAG_RW, &stat_indir_blk_ptrs, 0, "");
SYSCTL_INT(_debug, OID_AUTO, inode_bitmap, CTLFLAG_RW, &stat_inode_bitmap, 0, "");
SYSCTL_INT(_debug, OID_AUTO, direct_blk_ptrs, CTLFLAG_RW, &stat_direct_blk_ptrs, 0, "");
SYSCTL_INT(_debug, OID_AUTO, dir_entry, CTLFLAG_RW, &stat_dir_entry, 0, "");
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
#endif /* DEBUG */
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Add an item to the end of the work queue.
* This routine requires that the lock be held.
* This is the only routine that adds items to the list.
* The following routine is the only one that removes items
* and does so in order from first to last.
*/
static void
add_to_worklist(wk)
struct worklist *wk;
{
static struct worklist *worklist_tail;
if (wk->wk_state & ONWORKLIST)
panic("add_to_worklist: already on list");
wk->wk_state |= ONWORKLIST;
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
if (LIST_FIRST(&softdep_workitem_pending) == NULL)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_INSERT_HEAD(&softdep_workitem_pending, wk, wk_list);
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
else
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_INSERT_AFTER(worklist_tail, wk, wk_list);
worklist_tail = wk;
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
num_on_worklist += 1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Process that runs once per second to handle items in the background queue.
*
* Note that we ensure that everything is done in the order in which they
* appear in the queue. The code below depends on this property to ensure
* that blocks of a file are freed before the inode itself is freed. This
* ordering ensures that no new <vfsid, inum, lbn> triples will be generated
* until all the old ones have been purged from the dependency lists.
*/
ARGH! I have too many source trees :-( Fix prototype errors in last commit.
2000-06-16 13:00:33 +00:00
int
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_process_worklist(matchmnt)
struct mount *matchmnt;
{
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
struct proc *p = CURPROC;
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
int matchcnt, loopcount;
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
long starttime;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
/*
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
* Record the process identifier of our caller so that we can give
* this process preferential treatment in request_cleanup below.
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
*/
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
filesys_syncer = p;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
matchcnt = 0;
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* There is no danger of having multiple processes run this
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
* code, but we have to single-thread it when softdep_flushfiles()
* is in operation to get an accurate count of the number of items
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
* related to its mount point that are in the list.
*/
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
if (matchmnt == NULL) {
if (softdep_worklist_busy < 0)
return(-1);
softdep_worklist_busy += 1;
}
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
/*
* If requested, try removing inode or removal dependencies.
*/
if (req_clear_inodedeps) {
clear_inodedeps(p);
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
req_clear_inodedeps -= 1;
wakeup_one(&proc_waiting);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
}
if (req_clear_remove) {
clear_remove(p);
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
req_clear_remove -= 1;
wakeup_one(&proc_waiting);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
}
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
loopcount = 1;
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
starttime = time_second;
while (num_on_worklist > 0) {
matchcnt += process_worklist_item(matchmnt, 0);
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
/*
* If a umount operation wants to run the worklist
* accurately, abort.
*/
if (softdep_worklist_req && matchmnt == NULL) {
matchcnt = -1;
break;
}
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
/*
* If requested, try removing inode or removal dependencies.
*/
if (req_clear_inodedeps) {
clear_inodedeps(p);
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
req_clear_inodedeps -= 1;
wakeup_one(&proc_waiting);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
}
if (req_clear_remove) {
clear_remove(p);
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
req_clear_remove -= 1;
wakeup_one(&proc_waiting);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
}
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
/*
* We do not generally want to stop for buffer space, but if
* we are really being a buffer hog, we will stop and wait.
*/
if (loopcount++ % 128 == 0)
bwillwrite();
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
/*
* Never allow processing to run for more than one
* second. Otherwise the other syncer tasks may get
* excessively backlogged.
*/
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
if (starttime != time_second && matchmnt == NULL) {
matchcnt = -1;
break;
}
}
if (matchmnt == NULL) {
softdep_worklist_busy -= 1;
if (softdep_worklist_req && softdep_worklist_busy == 0)
wakeup(&softdep_worklist_req);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
return (matchcnt);
}
/*
* Process one item on the worklist.
*/
static int
process_worklist_item(matchmnt, flags)
struct mount *matchmnt;
int flags;
{
struct worklist *wk;
struct dirrem *dirrem;
struct mount *mp;
struct vnode *vp;
int matchcnt = 0;
ACQUIRE_LOCK(&lk);
/*
* Normally we just process each item on the worklist in order.
* However, if we are in a situation where we cannot lock any
* inodes, we have to skip over any dirrem requests whose
* vnodes are resident and locked.
*/
LIST_FOREACH(wk, &softdep_workitem_pending, wk_list) {
if ((flags & LK_NOWAIT) == 0 || wk->wk_type != D_DIRREM)
break;
dirrem = WK_DIRREM(wk);
vp = ufs_ihashlookup(VFSTOUFS(dirrem->dm_mnt)->um_dev,
dirrem->dm_oldinum);
if (vp == NULL || !VOP_ISLOCKED(vp, CURPROC))
break;
}
if (wk == 0)
return (0);
WORKLIST_REMOVE(wk);
num_on_worklist -= 1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
switch (wk->wk_type) {
case D_DIRREM:
/* removal of a directory entry */
mp = WK_DIRREM(wk)->dm_mnt;
if (vn_write_suspend_wait(NULL, mp, V_NOWAIT))
panic("%s: dirrem on suspended filesystem",
"process_worklist_item");
if (mp == matchmnt)
matchcnt += 1;
handle_workitem_remove(WK_DIRREM(wk));
break;
case D_FREEBLKS:
/* releasing blocks and/or fragments from a file */
mp = WK_FREEBLKS(wk)->fb_mnt;
if (vn_write_suspend_wait(NULL, mp, V_NOWAIT))
panic("%s: freeblks on suspended filesystem",
"process_worklist_item");
if (mp == matchmnt)
matchcnt += 1;
handle_workitem_freeblocks(WK_FREEBLKS(wk));
break;
case D_FREEFRAG:
/* releasing a fragment when replaced as a file grows */
mp = WK_FREEFRAG(wk)->ff_mnt;
if (vn_write_suspend_wait(NULL, mp, V_NOWAIT))
panic("%s: freefrag on suspended filesystem",
"process_worklist_item");
if (mp == matchmnt)
matchcnt += 1;
handle_workitem_freefrag(WK_FREEFRAG(wk));
break;
case D_FREEFILE:
/* releasing an inode when its link count drops to 0 */
mp = WK_FREEFILE(wk)->fx_mnt;
if (vn_write_suspend_wait(NULL, mp, V_NOWAIT))
panic("%s: freefile on suspended filesystem",
"process_worklist_item");
if (mp == matchmnt)
matchcnt += 1;
handle_workitem_freefile(WK_FREEFILE(wk));
break;
default:
panic("%s_process_worklist: Unknown type %s",
"softdep", TYPENAME(wk->wk_type));
/* NOTREACHED */
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (matchcnt);
}
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
/*
* Move dependencies from one buffer to another.
*/
static void
softdep_move_dependencies(oldbp, newbp)
struct buf *oldbp;
struct buf *newbp;
{
struct worklist *wk, *wktail;
if (LIST_FIRST(&newbp->b_dep) != NULL)
panic("softdep_move_dependencies: need merge code");
wktail = 0;
ACQUIRE_LOCK(&lk);
Non-operational change, fix compiler warning. Reviewed by: mckusick
2000-01-14 04:39:28 +00:00
while ((wk = LIST_FIRST(&oldbp->b_dep)) != NULL) {
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
LIST_REMOVE(wk, wk_list);
if (wktail == 0)
LIST_INSERT_HEAD(&newbp->b_dep, wk, wk_list);
else
LIST_INSERT_AFTER(wktail, wk, wk_list);
wktail = wk;
}
FREE_LOCK(&lk);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Purge the work list of all items associated with a particular mount point.
*/
int
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
softdep_flushworklist(oldmnt, countp, p)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct mount *oldmnt;
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
int *countp;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct proc *p;
{
struct vnode *devvp;
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
int count, error = 0;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
* Await our turn to clear out the queue, then serialize access.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
while (softdep_worklist_busy) {
softdep_worklist_req += 1;
tsleep(&softdep_worklist_req, PRIBIO, "softflush", 0);
softdep_worklist_req -= 1;
}
softdep_worklist_busy = -1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Alternately flush the block device associated with the mount
* point and process any dependencies that the flushing
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
* creates. We continue until no more worklist dependencies
* are found.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
*countp = 0;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
devvp = VFSTOUFS(oldmnt)->um_devvp;
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
while ((count = softdep_process_worklist(oldmnt)) > 0) {
*countp += count;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
vn_lock(devvp, LK_EXCLUSIVE | LK_RETRY, p);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
error = VOP_FSYNC(devvp, p->p_ucred, MNT_WAIT, p);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
VOP_UNLOCK(devvp, 0, p);
if (error)
break;
}
softdep_worklist_busy = 0;
Fix a race between the syncer and umount. When you umount a softupdates filesystem softdep_process_worklist() is called in a loop until it indicates that no dependancies remain, but the determination of that fact depends on there only being one softdep_process_worklist() instance running. It was possible for the syncer to also be running softdep_process_worklist() and the pre-existing checks in the code to prevent this were not sufficient to prevent the race. This patch solves the problem. Approved-by: mckusick
2001-01-30 06:31:59 +00:00
if (softdep_worklist_req)
wakeup(&softdep_worklist_req);
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
return (error);
}
/*
* Flush all vnodes and worklist items associated with a specified mount point.
*/
int
softdep_flushfiles(oldmnt, flags, p)
struct mount *oldmnt;
int flags;
struct proc *p;
{
int error, count, loopcnt;
/*
* Alternately flush the vnodes associated with the mount
* point and process any dependencies that the flushing
* creates. In theory, this loop can happen at most twice,
* but we give it a few extra just to be sure.
*/
for (loopcnt = 10; loopcnt > 0; loopcnt--) {
/*
* Do another flush in case any vnodes were brought in
* as part of the cleanup operations.
*/
if ((error = ffs_flushfiles(oldmnt, flags, p)) != 0)
break;
if ((error = softdep_flushworklist(oldmnt, &count, p)) != 0 ||
count == 0)
break;
}
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
/*
* If we are unmounting then it is an error to fail. If we
* are simply trying to downgrade to read-only, then filesystem
* activity can keep us busy forever, so we just fail with EBUSY.
*/
if (loopcnt == 0) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if (oldmnt->mnt_kern_flag & MNTK_UNMOUNT)
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
panic("softdep_flushfiles: looping");
error = EBUSY;
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (error);
}
/*
* Structure hashing.
*
* There are three types of structures that can be looked up:
* 1) pagedep structures identified by mount point, inode number,
* and logical block.
* 2) inodedep structures identified by mount point and inode number.
* 3) newblk structures identified by mount point and
* physical block number.
*
* The "pagedep" and "inodedep" dependency structures are hashed
* separately from the file blocks and inodes to which they correspond.
* This separation helps when the in-memory copy of an inode or
* file block must be replaced. It also obviates the need to access
* an inode or file page when simply updating (or de-allocating)
* dependency structures. Lookup of newblk structures is needed to
* find newly allocated blocks when trying to associate them with
* their allocdirect or allocindir structure.
*
* The lookup routines optionally create and hash a new instance when
* an existing entry is not found.
*/
#define DEPALLOC 0x0001 /* allocate structure if lookup fails */
/*
* Structures and routines associated with pagedep caching.
*/
Back out the previous change to the queue(3) interface. It was not discussed and should probably not happen. Requested by: msmith and others
2000-05-26 02:09:24 +00:00
LIST_HEAD(pagedep_hashhead, pagedep) *pagedep_hashtbl;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
u_long pagedep_hash; /* size of hash table - 1 */
#define PAGEDEP_HASH(mp, inum, lbn) \
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
(&pagedep_hashtbl[((((register_t)(mp)) >> 13) + (inum) + (lbn)) & \
pagedep_hash])
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
static struct sema pagedep_in_progress;
/*
* Look up a pagedep. Return 1 if found, 0 if not found.
* If not found, allocate if DEPALLOC flag is passed.
* Found or allocated entry is returned in pagedeppp.
* This routine must be called with splbio interrupts blocked.
*/
static int
pagedep_lookup(ip, lbn, flags, pagedeppp)
struct inode *ip;
ufs_lbn_t lbn;
int flags;
struct pagedep **pagedeppp;
{
struct pagedep *pagedep;
struct pagedep_hashhead *pagedephd;
struct mount *mp;
int i;
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("pagedep_lookup: lock not held");
#endif
mp = ITOV(ip)->v_mount;
pagedephd = PAGEDEP_HASH(mp, ip->i_number, lbn);
top:
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(pagedep, pagedephd, pd_hash)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (ip->i_number == pagedep->pd_ino &&
lbn == pagedep->pd_lbn &&
mp == pagedep->pd_mnt)
break;
if (pagedep) {
*pagedeppp = pagedep;
return (1);
}
if ((flags & DEPALLOC) == 0) {
*pagedeppp = NULL;
return (0);
}
if (sema_get(&pagedep_in_progress, &lk) == 0) {
ACQUIRE_LOCK(&lk);
goto top;
}
MALLOC(pagedep, struct pagedep *, sizeof(struct pagedep), M_PAGEDEP,
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
pagedep->pd_list.wk_type = D_PAGEDEP;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
pagedep->pd_mnt = mp;
pagedep->pd_ino = ip->i_number;
pagedep->pd_lbn = lbn;
LIST_INIT(&pagedep->pd_dirremhd);
LIST_INIT(&pagedep->pd_pendinghd);
for (i = 0; i < DAHASHSZ; i++)
LIST_INIT(&pagedep->pd_diraddhd[i]);
ACQUIRE_LOCK(&lk);
LIST_INSERT_HEAD(pagedephd, pagedep, pd_hash);
sema_release(&pagedep_in_progress);
*pagedeppp = pagedep;
return (0);
}
/*
* Structures and routines associated with inodedep caching.
*/
Back out the previous change to the queue(3) interface. It was not discussed and should probably not happen. Requested by: msmith and others
2000-05-26 02:09:24 +00:00
LIST_HEAD(inodedep_hashhead, inodedep) *inodedep_hashtbl;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
static u_long inodedep_hash; /* size of hash table - 1 */
static long num_inodedep; /* number of inodedep allocated */
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#define INODEDEP_HASH(fs, inum) \
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
(&inodedep_hashtbl[((((register_t)(fs)) >> 13) + (inum)) & inodedep_hash])
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
static struct sema inodedep_in_progress;
/*
* Look up a inodedep. Return 1 if found, 0 if not found.
* If not found, allocate if DEPALLOC flag is passed.
* Found or allocated entry is returned in inodedeppp.
* This routine must be called with splbio interrupts blocked.
*/
static int
inodedep_lookup(fs, inum, flags, inodedeppp)
struct fs *fs;
ino_t inum;
int flags;
struct inodedep **inodedeppp;
{
struct inodedep *inodedep;
struct inodedep_hashhead *inodedephd;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
int firsttry;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("inodedep_lookup: lock not held");
#endif
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
firsttry = 1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
inodedephd = INODEDEP_HASH(fs, inum);
top:
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(inodedep, inodedephd, id_hash)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (inum == inodedep->id_ino && fs == inodedep->id_fs)
break;
if (inodedep) {
*inodedeppp = inodedep;
return (1);
}
if ((flags & DEPALLOC) == 0) {
*inodedeppp = NULL;
return (0);
}
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
/*
* If we are over our limit, try to improve the situation.
*/
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
if (num_inodedep > max_softdeps && firsttry &&
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
request_cleanup(FLUSH_INODES, 1)) {
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
firsttry = 0;
goto top;
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (sema_get(&inodedep_in_progress, &lk) == 0) {
ACQUIRE_LOCK(&lk);
goto top;
}
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
num_inodedep += 1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
MALLOC(inodedep, struct inodedep *, sizeof(struct inodedep),
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_INODEDEP, M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
inodedep->id_list.wk_type = D_INODEDEP;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
inodedep->id_fs = fs;
inodedep->id_ino = inum;
inodedep->id_state = ALLCOMPLETE;
inodedep->id_nlinkdelta = 0;
inodedep->id_savedino = NULL;
inodedep->id_savedsize = -1;
inodedep->id_buf = NULL;
LIST_INIT(&inodedep->id_pendinghd);
LIST_INIT(&inodedep->id_inowait);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
LIST_INIT(&inodedep->id_bufwait);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
TAILQ_INIT(&inodedep->id_inoupdt);
TAILQ_INIT(&inodedep->id_newinoupdt);
ACQUIRE_LOCK(&lk);
LIST_INSERT_HEAD(inodedephd, inodedep, id_hash);
sema_release(&inodedep_in_progress);
*inodedeppp = inodedep;
return (0);
}
/*
* Structures and routines associated with newblk caching.
*/
Back out the previous change to the queue(3) interface. It was not discussed and should probably not happen. Requested by: msmith and others
2000-05-26 02:09:24 +00:00
LIST_HEAD(newblk_hashhead, newblk) *newblk_hashtbl;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
u_long newblk_hash; /* size of hash table - 1 */
#define NEWBLK_HASH(fs, inum) \
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
(&newblk_hashtbl[((((register_t)(fs)) >> 13) + (inum)) & newblk_hash])
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
static struct sema newblk_in_progress;
/*
* Look up a newblk. Return 1 if found, 0 if not found.
* If not found, allocate if DEPALLOC flag is passed.
* Found or allocated entry is returned in newblkpp.
*/
static int
newblk_lookup(fs, newblkno, flags, newblkpp)
struct fs *fs;
ufs_daddr_t newblkno;
int flags;
struct newblk **newblkpp;
{
struct newblk *newblk;
struct newblk_hashhead *newblkhd;
newblkhd = NEWBLK_HASH(fs, newblkno);
top:
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(newblk, newblkhd, nb_hash)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (newblkno == newblk->nb_newblkno && fs == newblk->nb_fs)
break;
if (newblk) {
*newblkpp = newblk;
return (1);
}
if ((flags & DEPALLOC) == 0) {
*newblkpp = NULL;
return (0);
}
if (sema_get(&newblk_in_progress, 0) == 0)
goto top;
MALLOC(newblk, struct newblk *, sizeof(struct newblk),
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_NEWBLK, M_SOFTDEP_FLAGS);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
newblk->nb_state = 0;
newblk->nb_fs = fs;
newblk->nb_newblkno = newblkno;
LIST_INSERT_HEAD(newblkhd, newblk, nb_hash);
sema_release(&newblk_in_progress);
*newblkpp = newblk;
return (0);
}
/*
* Executed during filesystem system initialization before
* mounting any file systems.
*/
void
softdep_initialize()
{
LIST_INIT(&mkdirlisthd);
LIST_INIT(&softdep_workitem_pending);
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
max_softdeps = min(desiredvnodes * 8,
M_INODEDEP->ks_limit / (2 * sizeof(struct inodedep)));
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
pagedep_hashtbl = hashinit(desiredvnodes / 5, M_PAGEDEP,
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
&pagedep_hash);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
sema_init(&pagedep_in_progress, "pagedep", PRIBIO, 0);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
inodedep_hashtbl = hashinit(desiredvnodes, M_INODEDEP, &inodedep_hash);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
sema_init(&inodedep_in_progress, "inodedep", PRIBIO, 0);
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
newblk_hashtbl = hashinit(64, M_NEWBLK, &newblk_hash);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
sema_init(&newblk_in_progress, "newblk", PRIBIO, 0);
}
/*
* Called at mount time to notify the dependency code that a
* filesystem wishes to use it.
*/
int
softdep_mount(devvp, mp, fs, cred)
struct vnode *devvp;
struct mount *mp;
struct fs *fs;
struct ucred *cred;
{
struct csum cstotal;
struct cg *cgp;
struct buf *bp;
int error, cyl;
Bring up-to-date with Whistle's current version Includes some debugging code.
1998-05-19 23:07:25 +00:00
mp->mnt_flag &= ~MNT_ASYNC;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
mp->mnt_flag |= MNT_SOFTDEP;
/*
* When doing soft updates, the counters in the
* superblock may have gotten out of sync, so we have
* to scan the cylinder groups and recalculate them.
*/
if (fs->fs_clean != 0)
return (0);
bzero(&cstotal, sizeof cstotal);
for (cyl = 0; cyl < fs->fs_ncg; cyl++) {
if ((error = bread(devvp, fsbtodb(fs, cgtod(fs, cyl)),
fs->fs_cgsize, cred, &bp)) != 0) {
brelse(bp);
return (error);
}
cgp = (struct cg *)bp->b_data;
cstotal.cs_nffree += cgp->cg_cs.cs_nffree;
cstotal.cs_nbfree += cgp->cg_cs.cs_nbfree;
cstotal.cs_nifree += cgp->cg_cs.cs_nifree;
cstotal.cs_ndir += cgp->cg_cs.cs_ndir;
fs->fs_cs(fs, cyl) = cgp->cg_cs;
brelse(bp);
}
#ifdef DEBUG
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
if (bcmp(&cstotal, &fs->fs_cstotal, sizeof cstotal))
Clarify a rather ambiguous debugging message.
1998-10-28 10:37:54 +00:00
printf("ffs_mountfs: superblock updated for soft updates\n");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#endif
bcopy(&cstotal, &fs->fs_cstotal, sizeof cstotal);
return (0);
}
/*
* Protecting the freemaps (or bitmaps).
*
* To eliminate the need to execute fsck before mounting a file system
* after a power failure, one must (conservatively) guarantee that the
* on-disk copy of the bitmaps never indicate that a live inode or block is
* free. So, when a block or inode is allocated, the bitmap should be
* updated (on disk) before any new pointers. When a block or inode is
* freed, the bitmap should not be updated until all pointers have been
* reset. The latter dependency is handled by the delayed de-allocation
* approach described below for block and inode de-allocation. The former
* dependency is handled by calling the following procedure when a block or
* inode is allocated. When an inode is allocated an "inodedep" is created
* with its DEPCOMPLETE flag cleared until its bitmap is written to disk.
* Each "inodedep" is also inserted into the hash indexing structure so
* that any additional link additions can be made dependent on the inode
* allocation.
*
* The ufs file system maintains a number of free block counts (e.g., per
* cylinder group, per cylinder and per <cylinder, rotational position> pair)
* in addition to the bitmaps. These counts are used to improve efficiency
* during allocation and therefore must be consistent with the bitmaps.
* There is no convenient way to guarantee post-crash consistency of these
* counts with simple update ordering, for two main reasons: (1) The counts
* and bitmaps for a single cylinder group block are not in the same disk
* sector. If a disk write is interrupted (e.g., by power failure), one may
* be written and the other not. (2) Some of the counts are located in the
* superblock rather than the cylinder group block. So, we focus our soft
* updates implementation on protecting the bitmaps. When mounting a
* filesystem, we recompute the auxiliary counts from the bitmaps.
*/
/*
* Called just after updating the cylinder group block to allocate an inode.
*/
void
softdep_setup_inomapdep(bp, ip, newinum)
struct buf *bp; /* buffer for cylgroup block with inode map */
struct inode *ip; /* inode related to allocation */
ino_t newinum; /* new inode number being allocated */
{
struct inodedep *inodedep;
struct bmsafemap *bmsafemap;
/*
* Create a dependency for the newly allocated inode.
* Panic if it already exists as something is seriously wrong.
* Otherwise add it to the dependency list for the buffer holding
* the cylinder group map from which it was allocated.
*/
ACQUIRE_LOCK(&lk);
if (inodedep_lookup(ip->i_fs, newinum, DEPALLOC, &inodedep) != 0)
panic("softdep_setup_inomapdep: found inode");
inodedep->id_buf = bp;
inodedep->id_state &= ~DEPCOMPLETE;
bmsafemap = bmsafemap_lookup(bp);
LIST_INSERT_HEAD(&bmsafemap->sm_inodedephd, inodedep, id_deps);
FREE_LOCK(&lk);
}
/*
* Called just after updating the cylinder group block to
* allocate block or fragment.
*/
void
softdep_setup_blkmapdep(bp, fs, newblkno)
struct buf *bp; /* buffer for cylgroup block with block map */
struct fs *fs; /* filesystem doing allocation */
ufs_daddr_t newblkno; /* number of newly allocated block */
{
struct newblk *newblk;
struct bmsafemap *bmsafemap;
/*
* Create a dependency for the newly allocated block.
* Add it to the dependency list for the buffer holding
* the cylinder group map from which it was allocated.
*/
if (newblk_lookup(fs, newblkno, DEPALLOC, &newblk) != 0)
panic("softdep_setup_blkmapdep: found block");
ACQUIRE_LOCK(&lk);
newblk->nb_bmsafemap = bmsafemap = bmsafemap_lookup(bp);
LIST_INSERT_HEAD(&bmsafemap->sm_newblkhd, newblk, nb_deps);
FREE_LOCK(&lk);
}
/*
* Find the bmsafemap associated with a cylinder group buffer.
* If none exists, create one. The buffer must be locked when
* this routine is called and this routine must be called with
* splbio interrupts blocked.
*/
static struct bmsafemap *
bmsafemap_lookup(bp)
struct buf *bp;
{
struct bmsafemap *bmsafemap;
struct worklist *wk;
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("bmsafemap_lookup: lock not held");
#endif
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
LIST_FOREACH(wk, &bp->b_dep, wk_list)
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if (wk->wk_type == D_BMSAFEMAP)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (WK_BMSAFEMAP(wk));
FREE_LOCK(&lk);
MALLOC(bmsafemap, struct bmsafemap *, sizeof(struct bmsafemap),
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_BMSAFEMAP, M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
bmsafemap->sm_list.wk_type = D_BMSAFEMAP;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bmsafemap->sm_list.wk_state = 0;
bmsafemap->sm_buf = bp;
LIST_INIT(&bmsafemap->sm_allocdirecthd);
LIST_INIT(&bmsafemap->sm_allocindirhd);
LIST_INIT(&bmsafemap->sm_inodedephd);
LIST_INIT(&bmsafemap->sm_newblkhd);
ACQUIRE_LOCK(&lk);
WORKLIST_INSERT(&bp->b_dep, &bmsafemap->sm_list);
return (bmsafemap);
}
/*
* Direct block allocation dependencies.
*
* When a new block is allocated, the corresponding disk locations must be
* initialized (with zeros or new data) before the on-disk inode points to
* them. Also, the freemap from which the block was allocated must be
* updated (on disk) before the inode's pointer. These two dependencies are
* independent of each other and are needed for all file blocks and indirect
* blocks that are pointed to directly by the inode. Just before the
* "in-core" version of the inode is updated with a newly allocated block
* number, a procedure (below) is called to setup allocation dependency
* structures. These structures are removed when the corresponding
* dependencies are satisfied or when the block allocation becomes obsolete
* (i.e., the file is deleted, the block is de-allocated, or the block is a
* fragment that gets upgraded). All of these cases are handled in
* procedures described later.
*
* When a file extension causes a fragment to be upgraded, either to a larger
* fragment or to a full block, the on-disk location may change (if the
* previous fragment could not simply be extended). In this case, the old
* fragment must be de-allocated, but not until after the inode's pointer has
* been updated. In most cases, this is handled by later procedures, which
* will construct a "freefrag" structure to be added to the workitem queue
* when the inode update is complete (or obsolete). The main exception to
* this is when an allocation occurs while a pending allocation dependency
* (for the same block pointer) remains. This case is handled in the main
* allocation dependency setup procedure by immediately freeing the
* unreferenced fragments.
*/
void
softdep_setup_allocdirect(ip, lbn, newblkno, oldblkno, newsize, oldsize, bp)
struct inode *ip; /* inode to which block is being added */
ufs_lbn_t lbn; /* block pointer within inode */
ufs_daddr_t newblkno; /* disk block number being added */
ufs_daddr_t oldblkno; /* previous block number, 0 unless frag */
long newsize; /* size of new block */
long oldsize; /* size of new block */
struct buf *bp; /* bp for allocated block */
{
struct allocdirect *adp, *oldadp;
struct allocdirectlst *adphead;
struct bmsafemap *bmsafemap;
struct inodedep *inodedep;
struct pagedep *pagedep;
struct newblk *newblk;
MALLOC(adp, struct allocdirect *, sizeof(struct allocdirect),
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
M_ALLOCDIRECT, M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
adp->ad_list.wk_type = D_ALLOCDIRECT;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
adp->ad_lbn = lbn;
adp->ad_newblkno = newblkno;
adp->ad_oldblkno = oldblkno;
adp->ad_newsize = newsize;
adp->ad_oldsize = oldsize;
adp->ad_state = ATTACHED;
if (newblkno == oldblkno)
adp->ad_freefrag = NULL;
else
adp->ad_freefrag = newfreefrag(ip, oldblkno, oldsize);
if (newblk_lookup(ip->i_fs, newblkno, 0, &newblk) == 0)
panic("softdep_setup_allocdirect: lost block");
ACQUIRE_LOCK(&lk);
(void) inodedep_lookup(ip->i_fs, ip->i_number, DEPALLOC, &inodedep);
adp->ad_inodedep = inodedep;
if (newblk->nb_state == DEPCOMPLETE) {
adp->ad_state |= DEPCOMPLETE;
adp->ad_buf = NULL;
} else {
bmsafemap = newblk->nb_bmsafemap;
adp->ad_buf = bmsafemap->sm_buf;
LIST_REMOVE(newblk, nb_deps);
LIST_INSERT_HEAD(&bmsafemap->sm_allocdirecthd, adp, ad_deps);
}
LIST_REMOVE(newblk, nb_hash);
FREE(newblk, M_NEWBLK);
WORKLIST_INSERT(&bp->b_dep, &adp->ad_list);
if (lbn >= NDADDR) {
/* allocating an indirect block */
if (oldblkno != 0)
panic("softdep_setup_allocdirect: non-zero indir");
} else {
/*
* Allocating a direct block.
*
* If we are allocating a directory block, then we must
* allocate an associated pagedep to track additions and
* deletions.
*/
if ((ip->i_mode & IFMT) == IFDIR &&
pagedep_lookup(ip, lbn, DEPALLOC, &pagedep) == 0)
WORKLIST_INSERT(&bp->b_dep, &pagedep->pd_list);
}
/*
* The list of allocdirects must be kept in sorted and ascending
* order so that the rollback routines can quickly determine the
* first uncommitted block (the size of the file stored on disk
* ends at the end of the lowest committed fragment, or if there
* are no fragments, at the end of the highest committed block).
* Since files generally grow, the typical case is that the new
* block is to be added at the end of the list. We speed this
* special case by checking against the last allocdirect in the
* list before laboriously traversing the list looking for the
* insertion point.
*/
adphead = &inodedep->id_newinoupdt;
oldadp = TAILQ_LAST(adphead, allocdirectlst);
if (oldadp == NULL || oldadp->ad_lbn <= lbn) {
/* insert at end of list */
TAILQ_INSERT_TAIL(adphead, adp, ad_next);
if (oldadp != NULL && oldadp->ad_lbn == lbn)
allocdirect_merge(adphead, adp, oldadp);
FREE_LOCK(&lk);
return;
}
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
TAILQ_FOREACH(oldadp, adphead, ad_next) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (oldadp->ad_lbn >= lbn)
break;
}
if (oldadp == NULL)
panic("softdep_setup_allocdirect: lost entry");
/* insert in middle of list */
TAILQ_INSERT_BEFORE(oldadp, adp, ad_next);
if (oldadp->ad_lbn == lbn)
allocdirect_merge(adphead, adp, oldadp);
FREE_LOCK(&lk);
}
/*
* Replace an old allocdirect dependency with a newer one.
* This routine must be called with splbio interrupts blocked.
*/
static void
allocdirect_merge(adphead, newadp, oldadp)
struct allocdirectlst *adphead; /* head of list holding allocdirects */
struct allocdirect *newadp; /* allocdirect being added */
struct allocdirect *oldadp; /* existing allocdirect being checked */
{
struct freefrag *freefrag;
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("allocdirect_merge: lock not held");
#endif
if (newadp->ad_oldblkno != oldadp->ad_newblkno ||
newadp->ad_oldsize != oldadp->ad_newsize ||
newadp->ad_lbn >= NDADDR)
Remove some compiler warnings.
1998-12-10 20:11:47 +00:00
panic("allocdirect_check: old %d != new %d || lbn %ld >= %d",
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
newadp->ad_oldblkno, oldadp->ad_newblkno, newadp->ad_lbn,
NDADDR);
newadp->ad_oldblkno = oldadp->ad_oldblkno;
newadp->ad_oldsize = oldadp->ad_oldsize;
/*
* If the old dependency had a fragment to free or had never
* previously had a block allocated, then the new dependency
* can immediately post its freefrag and adopt the old freefrag.
* This action is done by swapping the freefrag dependencies.
* The new dependency gains the old one's freefrag, and the
* old one gets the new one and then immediately puts it on
* the worklist when it is freed by free_allocdirect. It is
* not possible to do this swap when the old dependency had a
* non-zero size but no previous fragment to free. This condition
* arises when the new block is an extension of the old block.
* Here, the first part of the fragment allocated to the new
* dependency is part of the block currently claimed on disk by
* the old dependency, so cannot legitimately be freed until the
* conditions for the new dependency are fulfilled.
*/
if (oldadp->ad_freefrag != NULL || oldadp->ad_oldblkno == 0) {
freefrag = newadp->ad_freefrag;
newadp->ad_freefrag = oldadp->ad_freefrag;
oldadp->ad_freefrag = freefrag;
}
free_allocdirect(adphead, oldadp, 0);
}
/*
* Allocate a new freefrag structure if needed.
*/
static struct freefrag *
newfreefrag(ip, blkno, size)
struct inode *ip;
ufs_daddr_t blkno;
long size;
{
struct freefrag *freefrag;
struct fs *fs;
if (blkno == 0)
return (NULL);
fs = ip->i_fs;
if (fragnum(fs, blkno) + numfrags(fs, size) > fs->fs_frag)
panic("newfreefrag: frag size");
MALLOC(freefrag, struct freefrag *, sizeof(struct freefrag),
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_FREEFRAG, M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
freefrag->ff_list.wk_type = D_FREEFRAG;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
freefrag->ff_state = ip->i_uid & ~ONWORKLIST; /* XXX - used below */
freefrag->ff_inum = ip->i_number;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
freefrag->ff_mnt = ITOV(ip)->v_mount;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
freefrag->ff_devvp = ip->i_devvp;
freefrag->ff_blkno = blkno;
freefrag->ff_fragsize = size;
return (freefrag);
}
/*
* This workitem de-allocates fragments that were replaced during
* file block allocation.
*/
static void
handle_workitem_freefrag(freefrag)
struct freefrag *freefrag;
{
struct inode tip;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
tip.i_vnode = NULL;
tip.i_fs = VFSTOUFS(freefrag->ff_mnt)->um_fs;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
tip.i_devvp = freefrag->ff_devvp;
tip.i_dev = freefrag->ff_devvp->v_rdev;
tip.i_number = freefrag->ff_inum;
tip.i_uid = freefrag->ff_state & ~ONWORKLIST; /* XXX - set above */
ffs_blkfree(&tip, freefrag->ff_blkno, freefrag->ff_fragsize);
FREE(freefrag, M_FREEFRAG);
}
/*
* Indirect block allocation dependencies.
*
* The same dependencies that exist for a direct block also exist when
* a new block is allocated and pointed to by an entry in a block of
* indirect pointers. The undo/redo states described above are also
* used here. Because an indirect block contains many pointers that
* may have dependencies, a second copy of the entire in-memory indirect
* block is kept. The buffer cache copy is always completely up-to-date.
* The second copy, which is used only as a source for disk writes,
* contains only the safe pointers (i.e., those that have no remaining
* update dependencies). The second copy is freed when all pointers
* are safe. The cache is not allowed to replace indirect blocks with
* pending update dependencies. If a buffer containing an indirect
* block with dependencies is written, these routines will mark it
* dirty again. It can only be successfully written once all the
* dependencies are removed. The ffs_fsync routine in conjunction with
* softdep_sync_metadata work together to get all the dependencies
* removed so that a file can be successfully written to disk. Three
* procedures are used when setting up indirect block pointer
* dependencies. The division is necessary because of the organization
* of the "balloc" routine and because of the distinction between file
* pages and file metadata blocks.
*/
/*
* Allocate a new allocindir structure.
*/
static struct allocindir *
newallocindir(ip, ptrno, newblkno, oldblkno)
struct inode *ip; /* inode for file being extended */
int ptrno; /* offset of pointer in indirect block */
ufs_daddr_t newblkno; /* disk block number being added */
ufs_daddr_t oldblkno; /* previous block number, 0 if none */
{
struct allocindir *aip;
MALLOC(aip, struct allocindir *, sizeof(struct allocindir),
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
M_ALLOCINDIR, M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
aip->ai_list.wk_type = D_ALLOCINDIR;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
aip->ai_state = ATTACHED;
aip->ai_offset = ptrno;
aip->ai_newblkno = newblkno;
aip->ai_oldblkno = oldblkno;
aip->ai_freefrag = newfreefrag(ip, oldblkno, ip->i_fs->fs_bsize);
return (aip);
}
/*
* Called just before setting an indirect block pointer
* to a newly allocated file page.
*/
void
softdep_setup_allocindir_page(ip, lbn, bp, ptrno, newblkno, oldblkno, nbp)
struct inode *ip; /* inode for file being extended */
ufs_lbn_t lbn; /* allocated block number within file */
struct buf *bp; /* buffer with indirect blk referencing page */
int ptrno; /* offset of pointer in indirect block */
ufs_daddr_t newblkno; /* disk block number being added */
ufs_daddr_t oldblkno; /* previous block number, 0 if none */
struct buf *nbp; /* buffer holding allocated page */
{
struct allocindir *aip;
struct pagedep *pagedep;
aip = newallocindir(ip, ptrno, newblkno, oldblkno);
ACQUIRE_LOCK(&lk);
/*
* If we are allocating a directory page, then we must
* allocate an associated pagedep to track additions and
* deletions.
*/
if ((ip->i_mode & IFMT) == IFDIR &&
pagedep_lookup(ip, lbn, DEPALLOC, &pagedep) == 0)
WORKLIST_INSERT(&nbp->b_dep, &pagedep->pd_list);
WORKLIST_INSERT(&nbp->b_dep, &aip->ai_list);
FREE_LOCK(&lk);
setup_allocindir_phase2(bp, ip, aip);
}
/*
* Called just before setting an indirect block pointer to a
* newly allocated indirect block.
*/
void
softdep_setup_allocindir_meta(nbp, ip, bp, ptrno, newblkno)
struct buf *nbp; /* newly allocated indirect block */
struct inode *ip; /* inode for file being extended */
struct buf *bp; /* indirect block referencing allocated block */
int ptrno; /* offset of pointer in indirect block */
ufs_daddr_t newblkno; /* disk block number being added */
{
struct allocindir *aip;
aip = newallocindir(ip, ptrno, newblkno, 0);
ACQUIRE_LOCK(&lk);
WORKLIST_INSERT(&nbp->b_dep, &aip->ai_list);
FREE_LOCK(&lk);
setup_allocindir_phase2(bp, ip, aip);
}
/*
* Called to finish the allocation of the "aip" allocated
* by one of the two routines above.
*/
static void
setup_allocindir_phase2(bp, ip, aip)
struct buf *bp; /* in-memory copy of the indirect block */
struct inode *ip; /* inode for file being extended */
struct allocindir *aip; /* allocindir allocated by the above routines */
{
struct worklist *wk;
struct indirdep *indirdep, *newindirdep;
struct bmsafemap *bmsafemap;
struct allocindir *oldaip;
struct freefrag *freefrag;
struct newblk *newblk;
if (bp->b_lblkno >= 0)
panic("setup_allocindir_phase2: not indir blk");
for (indirdep = NULL, newindirdep = NULL; ; ) {
ACQUIRE_LOCK(&lk);
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(wk, &bp->b_dep, wk_list) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if (wk->wk_type != D_INDIRDEP)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
continue;
indirdep = WK_INDIRDEP(wk);
break;
}
if (indirdep == NULL && newindirdep) {
indirdep = newindirdep;
WORKLIST_INSERT(&bp->b_dep, &indirdep->ir_list);
newindirdep = NULL;
}
FREE_LOCK(&lk);
if (indirdep) {
if (newblk_lookup(ip->i_fs, aip->ai_newblkno, 0,
&newblk) == 0)
panic("setup_allocindir: lost block");
ACQUIRE_LOCK(&lk);
if (newblk->nb_state == DEPCOMPLETE) {
aip->ai_state |= DEPCOMPLETE;
aip->ai_buf = NULL;
} else {
bmsafemap = newblk->nb_bmsafemap;
aip->ai_buf = bmsafemap->sm_buf;
LIST_REMOVE(newblk, nb_deps);
LIST_INSERT_HEAD(&bmsafemap->sm_allocindirhd,
aip, ai_deps);
}
LIST_REMOVE(newblk, nb_hash);
FREE(newblk, M_NEWBLK);
aip->ai_indirdep = indirdep;
/*
* Check to see if there is an existing dependency
* for this block. If there is, merge the old
* dependency into the new one.
*/
if (aip->ai_oldblkno == 0)
oldaip = NULL;
else
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(oldaip, &indirdep->ir_deplisthd, ai_next)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (oldaip->ai_offset == aip->ai_offset)
break;
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
freefrag = NULL;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (oldaip != NULL) {
if (oldaip->ai_newblkno != aip->ai_oldblkno)
panic("setup_allocindir_phase2: blkno");
aip->ai_oldblkno = oldaip->ai_oldblkno;
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
freefrag = aip->ai_freefrag;
aip->ai_freefrag = oldaip->ai_freefrag;
oldaip->ai_freefrag = NULL;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
free_allocindir(oldaip, NULL);
}
LIST_INSERT_HEAD(&indirdep->ir_deplisthd, aip, ai_next);
((ufs_daddr_t *)indirdep->ir_savebp->b_data)
[aip->ai_offset] = aip->ai_oldblkno;
FREE_LOCK(&lk);
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
if (freefrag != NULL)
handle_workitem_freefrag(freefrag);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
if (newindirdep) {
if (indirdep->ir_savebp != NULL)
brelse(newindirdep->ir_savebp);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE((caddr_t)newindirdep, D_INDIRDEP);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
if (indirdep)
break;
MALLOC(newindirdep, struct indirdep *, sizeof(struct indirdep),
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_INDIRDEP, M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
newindirdep->ir_list.wk_type = D_INDIRDEP;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
newindirdep->ir_state = ATTACHED;
LIST_INIT(&newindirdep->ir_deplisthd);
LIST_INIT(&newindirdep->ir_donehd);
Bring up-to-date with Whistle's current version Includes some debugging code.
1998-05-19 23:07:25 +00:00
if (bp->b_blkno == bp->b_lblkno) {
VOP_BMAP(bp->b_vp, bp->b_lblkno, NULL, &bp->b_blkno,
NULL, NULL);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
newindirdep->ir_savebp =
getblk(ip->i_devvp, bp->b_blkno, bp->b_bcount, 0, 0);
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
BUF_KERNPROC(newindirdep->ir_savebp);
Bring up-to-date with Whistle's current version Includes some debugging code.
1998-05-19 23:07:25 +00:00
bcopy(bp->b_data, newindirdep->ir_savebp->b_data, bp->b_bcount);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
}
/*
* Block de-allocation dependencies.
*
* When blocks are de-allocated, the on-disk pointers must be nullified before
* the blocks are made available for use by other files. (The true
* requirement is that old pointers must be nullified before new on-disk
* pointers are set. We chose this slightly more stringent requirement to
* reduce complexity.) Our implementation handles this dependency by updating
* the inode (or indirect block) appropriately but delaying the actual block
* de-allocation (i.e., freemap and free space count manipulation) until
* after the updated versions reach stable storage. After the disk is
* updated, the blocks can be safely de-allocated whenever it is convenient.
* This implementation handles only the common case of reducing a file's
* length to zero. Other cases are handled by the conventional synchronous
* write approach.
*
* The ffs implementation with which we worked double-checks
* the state of the block pointers and file size as it reduces
* a file's length. Some of this code is replicated here in our
* soft updates implementation. The freeblks->fb_chkcnt field is
* used to transfer a part of this information to the procedure
* that eventually de-allocates the blocks.
*
* This routine should be called from the routine that shortens
* a file's length, before the inode's size or block pointers
* are modified. It will save the block pointer information for
* later release and zero the inode so that the calling routine
* can release it.
*/
void
softdep_setup_freeblocks(ip, length)
struct inode *ip; /* The inode whose length is to be reduced */
off_t length; /* The new length for the file */
{
struct freeblks *freeblks;
struct inodedep *inodedep;
struct allocdirect *adp;
struct vnode *vp;
struct buf *bp;
struct fs *fs;
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
int i, delay, error;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
fs = ip->i_fs;
if (length != 0)
panic("softde_setup_freeblocks: non-zero length");
MALLOC(freeblks, struct freeblks *, sizeof(struct freeblks),
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
M_FREEBLKS, M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
freeblks->fb_list.wk_type = D_FREEBLKS;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
freeblks->fb_uid = ip->i_uid;
freeblks->fb_previousinum = ip->i_number;
freeblks->fb_devvp = ip->i_devvp;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
freeblks->fb_mnt = ITOV(ip)->v_mount;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
freeblks->fb_oldsize = ip->i_size;
freeblks->fb_newsize = length;
freeblks->fb_chkcnt = ip->i_blocks;
for (i = 0; i < NDADDR; i++) {
freeblks->fb_dblks[i] = ip->i_db[i];
ip->i_db[i] = 0;
}
for (i = 0; i < NIADDR; i++) {
freeblks->fb_iblks[i] = ip->i_ib[i];
ip->i_ib[i] = 0;
}
ip->i_blocks = 0;
ip->i_size = 0;
/*
* Push the zero'ed inode to to its disk buffer so that we are free
* to delete its dependencies below. Once the dependencies are gone
* the buffer can be safely released.
*/
if ((error = bread(ip->i_devvp,
fsbtodb(fs, ino_to_fsba(fs, ip->i_number)),
(int)fs->fs_bsize, NOCRED, &bp)) != 0)
softdep_error("softdep_setup_freeblocks", error);
*((struct dinode *)bp->b_data + ino_to_fsbo(fs, ip->i_number)) =
ip->i_din;
/*
* Find and eliminate any inode dependencies.
*/
ACQUIRE_LOCK(&lk);
(void) inodedep_lookup(fs, ip->i_number, DEPALLOC, &inodedep);
if ((inodedep->id_state & IOSTARTED) != 0)
panic("softdep_setup_freeblocks: inode busy");
When deleting a file, the ordering of events imposed by soft updates is to first write the deleted directory entry to disk, second write the zero'ed inode to disk, and finally to release the freed blocks and the inode back to the cylinder-group map. As this ordering requires two disk writes to occur which are normally spaced about 30 seconds apart (except when memory is under duress), it takes about a minute from the time that a file is deleted until its inode and data blocks show up in the cylinder-group map for reallocation. If a file has had only a brief lifetime (less than 30 seconds from creation to deletion), neither its inode nor its directory entry may have been written to disk. If its directory entry has not been written to disk, then we need not wait for that directory block to be written as the on-disk directory block does not reference the inode. Similarly, if the allocated inode has never been written to disk, we do not have to wait for it to be written back either as its on-disk representation is still zero'ed out. Thus, in the case of a short lived file, we can simply release the blocks and inode to the cylinder-group map immediately. As the inode and its blocks are released immediately, they are immediately available for other uses. If they are not released for a minute, then other inodes and blocks must be allocated for short lived files, cluttering up the vnode and buffer caches. The previous code was a bit too aggressive in trying to release the blocks and inode back to the cylinder-group map resulting in their being made available when in fact the inode on disk had not yet been zero'ed. This patch takes a more conservative approach to doing the release which avoids doing the release prematurely.
2000-11-14 09:00:25 +00:00
/*
* Add the freeblks structure to the list of operations that
* must await the zero'ed inode being written to disk. If we
* still have a bitmap dependency (delay == 0), then the inode
* has never been written to disk, so we can process the
* freeblks below once we have deleted the dependencies.
*/
delay = (inodedep->id_state & DEPCOMPLETE);
if (delay)
WORKLIST_INSERT(&inodedep->id_bufwait, &freeblks->fb_list);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Because the file length has been truncated to zero, any
* pending block allocation dependency structures associated
* with this inode are obsolete and can simply be de-allocated.
* We must first merge the two dependency lists to get rid of
* any duplicate freefrag structures, then purge the merged list.
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
* If we still have a bitmap dependency, then the inode has never
* been written to disk, so we can free any fragments without delay.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
merge_inode_lists(inodedep);
while ((adp = TAILQ_FIRST(&inodedep->id_inoupdt)) != 0)
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
free_allocdirect(&inodedep->id_inoupdt, adp, delay);
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
FREE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bdwrite(bp);
/*
* We must wait for any I/O in progress to finish so that
* all potential buffers on the dirty list will be visible.
* Once they are all there, walk the list and get rid of
* any dependencies.
*/
vp = ITOV(ip);
Reorganize locking to avoid holding the lock during calls to bdwrite and brelse (which may sleep in some systems). Obtained from: Matthew Dillon <dillon@apollo.backplane.com>
1999-03-02 06:38:07 +00:00
ACQUIRE_LOCK(&lk);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
drain_output(vp, 1);
Change dirty block list handling to use TAILQ macros.
1998-10-31 15:33:32 +00:00
while (getdirtybuf(&TAILQ_FIRST(&vp->v_dirtyblkhd), MNT_WAIT)) {
bp = TAILQ_FIRST(&vp->v_dirtyblkhd);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
(void) inodedep_lookup(fs, ip->i_number, 0, &inodedep);
deallocate_dependencies(bp, inodedep);
Add B_NOCACHE to several cases where BSD4.4 only required a B_INVAL. Change worked out by john and kirk in consort.
1998-06-11 17:44:32 +00:00
bp->b_flags |= B_INVAL | B_NOCACHE;
Reorganize locking to avoid holding the lock during calls to bdwrite and brelse (which may sleep in some systems). Obtained from: Matthew Dillon <dillon@apollo.backplane.com>
1999-03-02 06:38:07 +00:00
FREE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
brelse(bp);
Reorganize locking to avoid holding the lock during calls to bdwrite and brelse (which may sleep in some systems). Obtained from: Matthew Dillon <dillon@apollo.backplane.com>
1999-03-02 06:38:07 +00:00
ACQUIRE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
When deleting a file, the ordering of events imposed by soft updates is to first write the deleted directory entry to disk, second write the zero'ed inode to disk, and finally to release the freed blocks and the inode back to the cylinder-group map. As this ordering requires two disk writes to occur which are normally spaced about 30 seconds apart (except when memory is under duress), it takes about a minute from the time that a file is deleted until its inode and data blocks show up in the cylinder-group map for reallocation. If a file has had only a brief lifetime (less than 30 seconds from creation to deletion), neither its inode nor its directory entry may have been written to disk. If its directory entry has not been written to disk, then we need not wait for that directory block to be written as the on-disk directory block does not reference the inode. Similarly, if the allocated inode has never been written to disk, we do not have to wait for it to be written back either as its on-disk representation is still zero'ed out. Thus, in the case of a short lived file, we can simply release the blocks and inode to the cylinder-group map immediately. As the inode and its blocks are released immediately, they are immediately available for other uses. If they are not released for a minute, then other inodes and blocks must be allocated for short lived files, cluttering up the vnode and buffer caches. The previous code was a bit too aggressive in trying to release the blocks and inode back to the cylinder-group map resulting in their being made available when in fact the inode on disk had not yet been zero'ed. This patch takes a more conservative approach to doing the release which avoids doing the release prematurely.
2000-11-14 09:00:25 +00:00
if (inodedep_lookup(fs, ip->i_number, 0, &inodedep) != 0)
(void) free_inodedep(inodedep);
FREE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
When deleting a file, the ordering of events imposed by soft updates is to first write the deleted directory entry to disk, second write the zero'ed inode to disk, and finally to release the freed blocks and the inode back to the cylinder-group map. As this ordering requires two disk writes to occur which are normally spaced about 30 seconds apart (except when memory is under duress), it takes about a minute from the time that a file is deleted until its inode and data blocks show up in the cylinder-group map for reallocation. If a file has had only a brief lifetime (less than 30 seconds from creation to deletion), neither its inode nor its directory entry may have been written to disk. If its directory entry has not been written to disk, then we need not wait for that directory block to be written as the on-disk directory block does not reference the inode. Similarly, if the allocated inode has never been written to disk, we do not have to wait for it to be written back either as its on-disk representation is still zero'ed out. Thus, in the case of a short lived file, we can simply release the blocks and inode to the cylinder-group map immediately. As the inode and its blocks are released immediately, they are immediately available for other uses. If they are not released for a minute, then other inodes and blocks must be allocated for short lived files, cluttering up the vnode and buffer caches. The previous code was a bit too aggressive in trying to release the blocks and inode back to the cylinder-group map resulting in their being made available when in fact the inode on disk had not yet been zero'ed. This patch takes a more conservative approach to doing the release which avoids doing the release prematurely.
2000-11-14 09:00:25 +00:00
* If the inode has never been written to disk (delay == 0),
* then we can process the freeblks now that we have deleted
* the dependencies.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
When deleting a file, the ordering of events imposed by soft updates is to first write the deleted directory entry to disk, second write the zero'ed inode to disk, and finally to release the freed blocks and the inode back to the cylinder-group map. As this ordering requires two disk writes to occur which are normally spaced about 30 seconds apart (except when memory is under duress), it takes about a minute from the time that a file is deleted until its inode and data blocks show up in the cylinder-group map for reallocation. If a file has had only a brief lifetime (less than 30 seconds from creation to deletion), neither its inode nor its directory entry may have been written to disk. If its directory entry has not been written to disk, then we need not wait for that directory block to be written as the on-disk directory block does not reference the inode. Similarly, if the allocated inode has never been written to disk, we do not have to wait for it to be written back either as its on-disk representation is still zero'ed out. Thus, in the case of a short lived file, we can simply release the blocks and inode to the cylinder-group map immediately. As the inode and its blocks are released immediately, they are immediately available for other uses. If they are not released for a minute, then other inodes and blocks must be allocated for short lived files, cluttering up the vnode and buffer caches. The previous code was a bit too aggressive in trying to release the blocks and inode back to the cylinder-group map resulting in their being made available when in fact the inode on disk had not yet been zero'ed. This patch takes a more conservative approach to doing the release which avoids doing the release prematurely.
2000-11-14 09:00:25 +00:00
if (!delay)
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
handle_workitem_freeblocks(freeblks);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Reclaim any dependency structures from a buffer that is about to
* be reallocated to a new vnode. The buffer must be locked, thus,
* no I/O completion operations can occur while we are manipulating
* its associated dependencies. The mutex is held so that other I/O's
* associated with related dependencies do not occur.
*/
static void
deallocate_dependencies(bp, inodedep)
struct buf *bp;
struct inodedep *inodedep;
{
struct worklist *wk;
struct indirdep *indirdep;
struct allocindir *aip;
struct pagedep *pagedep;
struct dirrem *dirrem;
struct diradd *dap;
int i;
while ((wk = LIST_FIRST(&bp->b_dep)) != NULL) {
switch (wk->wk_type) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INDIRDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
indirdep = WK_INDIRDEP(wk);
/*
* None of the indirect pointers will ever be visible,
* so they can simply be tossed. GOINGAWAY ensures
* that allocated pointers will be saved in the buffer
* cache until they are freed. Note that they will
* only be able to be found by their physical address
* since the inode mapping the logical address will
* be gone. The save buffer used for the safe copy
* was allocated in setup_allocindir_phase2 using
* the physical address so it could be used for this
* purpose. Hence we swap the safe copy with the real
* copy, allowing the safe copy to be freed and holding
* on to the real copy for later use in indir_trunc.
*/
if (indirdep->ir_state & GOINGAWAY)
panic("deallocate_dependencies: already gone");
indirdep->ir_state |= GOINGAWAY;
while ((aip = LIST_FIRST(&indirdep->ir_deplisthd)) != 0)
free_allocindir(aip, inodedep);
if (bp->b_lblkno >= 0 ||
bp->b_blkno != indirdep->ir_savebp->b_lblkno)
panic("deallocate_dependencies: not indir");
Change to stop a silly panic. This should be understood better. Change a buffer swizzle trick to a bcopy. It would be nice if the efficient trick could be used in the future.
1998-05-19 20:50:41 +00:00
bcopy(bp->b_data, indirdep->ir_savebp->b_data,
bp->b_bcount);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
WORKLIST_REMOVE(wk);
WORKLIST_INSERT(&indirdep->ir_savebp->b_dep, wk);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_PAGEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
pagedep = WK_PAGEDEP(wk);
/*
* None of the directory additions will ever be
* visible, so they can simply be tossed.
*/
for (i = 0; i < DAHASHSZ; i++)
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
while ((dap =
LIST_FIRST(&pagedep->pd_diraddhd[i])))
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
free_diradd(dap);
while ((dap = LIST_FIRST(&pagedep->pd_pendinghd)) != 0)
free_diradd(dap);
/*
* Copy any directory remove dependencies to the list
* to be processed after the zero'ed inode is written.
* If the inode has already been written, then they
* can be dumped directly onto the work list.
*/
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(dirrem, &pagedep->pd_dirremhd, dm_next) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_REMOVE(dirrem, dm_next);
dirrem->dm_dirinum = pagedep->pd_ino;
Better bounding on softdep_flushfiles; other minor tweeks to checks.
2000-01-17 06:35:11 +00:00
if (inodedep == NULL ||
(inodedep->id_state & ALLCOMPLETE) ==
ALLCOMPLETE)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
add_to_worklist(&dirrem->dm_list);
else
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
WORKLIST_INSERT(&inodedep->id_bufwait,
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
&dirrem->dm_list);
}
WORKLIST_REMOVE(&pagedep->pd_list);
LIST_REMOVE(pagedep, pd_hash);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(pagedep, D_PAGEDEP);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_ALLOCINDIR:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
free_allocindir(WK_ALLOCINDIR(wk), inodedep);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_ALLOCDIRECT:
case D_INODEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("deallocate_dependencies: Unexpected type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
default:
panic("deallocate_dependencies: Unknown type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
}
}
}
/*
* Free an allocdirect. Generate a new freefrag work request if appropriate.
* This routine must be called with splbio interrupts blocked.
*/
static void
free_allocdirect(adphead, adp, delay)
struct allocdirectlst *adphead;
struct allocdirect *adp;
int delay;
{
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("free_allocdirect: lock not held");
#endif
if ((adp->ad_state & DEPCOMPLETE) == 0)
LIST_REMOVE(adp, ad_deps);
TAILQ_REMOVE(adphead, adp, ad_next);
if ((adp->ad_state & COMPLETE) == 0)
WORKLIST_REMOVE(&adp->ad_list);
if (adp->ad_freefrag != NULL) {
if (delay)
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
WORKLIST_INSERT(&adp->ad_inodedep->id_bufwait,
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
&adp->ad_freefrag->ff_list);
else
add_to_worklist(&adp->ad_freefrag->ff_list);
}
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(adp, D_ALLOCDIRECT);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Prepare an inode to be freed. The actual free operation is not
* done until the zero'ed inode has been written to disk.
*/
void
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
softdep_freefile(pvp, ino, mode)
struct vnode *pvp;
ino_t ino;
int mode;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
{
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
struct inode *ip = VTOI(pvp);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct inodedep *inodedep;
struct freefile *freefile;
/*
* This sets up the inode de-allocation dependency.
*/
MALLOC(freefile, struct freefile *, sizeof(struct freefile),
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_FREEFILE, M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
freefile->fx_list.wk_type = D_FREEFILE;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
freefile->fx_list.wk_state = 0;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
freefile->fx_mode = mode;
freefile->fx_oldinum = ino;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
freefile->fx_devvp = ip->i_devvp;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
freefile->fx_mnt = ITOV(ip)->v_mount;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* If the inodedep does not exist, then the zero'ed inode has
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
* been written to disk. If the allocated inode has never been
* written to disk, then the on-disk inode is zero'ed. In either
* case we can free the file immediately.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
ACQUIRE_LOCK(&lk);
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
if (inodedep_lookup(ip->i_fs, ino, 0, &inodedep) == 0 ||
check_inode_unwritten(inodedep)) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
handle_workitem_freefile(freefile);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return;
}
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
WORKLIST_INSERT(&inodedep->id_inowait, &freefile->fx_list);
FREE_LOCK(&lk);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
/*
* Check to see if an inode has never been written to disk. If
* so free the inodedep and return success, otherwise return failure.
* This routine must be called with splbio interrupts blocked.
*
* If we still have a bitmap dependency, then the inode has never
* been written to disk. Drop the dependency as it is no longer
* necessary since the inode is being deallocated. We set the
* ALLCOMPLETE flags since the bitmap now properly shows that the
* inode is not allocated. Even if the inode is actively being
* written, it has been rolled back to its zero'ed state, so we
* are ensured that a zero inode is what is on the disk. For short
* lived files, this change will usually result in removing all the
* dependencies from the inode so that it can be freed immediately.
*/
static int
check_inode_unwritten(inodedep)
struct inodedep *inodedep;
{
if ((inodedep->id_state & DEPCOMPLETE) != 0 ||
LIST_FIRST(&inodedep->id_pendinghd) != NULL ||
LIST_FIRST(&inodedep->id_bufwait) != NULL ||
LIST_FIRST(&inodedep->id_inowait) != NULL ||
TAILQ_FIRST(&inodedep->id_inoupdt) != NULL ||
TAILQ_FIRST(&inodedep->id_newinoupdt) != NULL ||
inodedep->id_nlinkdelta != 0)
return (0);
inodedep->id_state |= ALLCOMPLETE;
LIST_REMOVE(inodedep, id_deps);
inodedep->id_buf = NULL;
When running with quotas enabled on a filesystem using soft updates, the system would panic when a user's inode quota was exceeded (see PR 18959 for details). This fixes that problem. PR: 18959 Submitted by: Jason Godsey <jason@unixguy.fidalgo.net>
2000-06-18 22:14:28 +00:00
if (inodedep->id_state & ONWORKLIST)
WORKLIST_REMOVE(&inodedep->id_list);
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
if (inodedep->id_savedino != NULL) {
FREE(inodedep->id_savedino, M_INODEDEP);
inodedep->id_savedino = NULL;
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
}
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
if (free_inodedep(inodedep) == 0)
panic("check_inode_unwritten: busy inode");
return (1);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Try to free an inodedep structure. Return 1 if it could be freed.
*/
static int
free_inodedep(inodedep)
struct inodedep *inodedep;
{
if ((inodedep->id_state & ONWORKLIST) != 0 ||
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
(inodedep->id_state & ALLCOMPLETE) != ALLCOMPLETE ||
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_FIRST(&inodedep->id_pendinghd) != NULL ||
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
LIST_FIRST(&inodedep->id_bufwait) != NULL ||
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_FIRST(&inodedep->id_inowait) != NULL ||
TAILQ_FIRST(&inodedep->id_inoupdt) != NULL ||
TAILQ_FIRST(&inodedep->id_newinoupdt) != NULL ||
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
inodedep->id_nlinkdelta != 0 || inodedep->id_savedino != NULL)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (0);
LIST_REMOVE(inodedep, id_hash);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(inodedep, D_INODEDEP);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
num_inodedep -= 1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (1);
}
/*
* This workitem routine performs the block de-allocation.
* The workitem is added to the pending list after the updated
* inode block has been written to disk. As mentioned above,
* checks regarding the number of blocks de-allocated (compared
* to the number of blocks allocated for the file) are also
* performed in this function.
*/
static void
handle_workitem_freeblocks(freeblks)
struct freeblks *freeblks;
{
struct inode tip;
ufs_daddr_t bn;
struct fs *fs;
int i, level, bsize;
long nblocks, blocksreleased = 0;
int error, allerror = 0;
ufs_lbn_t baselbns[NIADDR], tmpval;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
tip.i_fs = fs = VFSTOUFS(freeblks->fb_mnt)->um_fs;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
tip.i_number = freeblks->fb_previousinum;
tip.i_devvp = freeblks->fb_devvp;
tip.i_dev = freeblks->fb_devvp->v_rdev;
tip.i_size = freeblks->fb_oldsize;
tip.i_uid = freeblks->fb_uid;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
tip.i_vnode = NULL;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
tmpval = 1;
baselbns[0] = NDADDR;
for (i = 1; i < NIADDR; i++) {
tmpval *= NINDIR(fs);
baselbns[i] = baselbns[i - 1] + tmpval;
}
nblocks = btodb(fs->fs_bsize);
blocksreleased = 0;
/*
* Indirect blocks first.
*/
for (level = (NIADDR - 1); level >= 0; level--) {
if ((bn = freeblks->fb_iblks[level]) == 0)
continue;
if ((error = indir_trunc(&tip, fsbtodb(fs, bn), level,
baselbns[level], &blocksreleased)) == 0)
allerror = error;
ffs_blkfree(&tip, bn, fs->fs_bsize);
blocksreleased += nblocks;
}
/*
* All direct blocks or frags.
*/
for (i = (NDADDR - 1); i >= 0; i--) {
if ((bn = freeblks->fb_dblks[i]) == 0)
continue;
bsize = blksize(fs, &tip, i);
ffs_blkfree(&tip, bn, bsize);
blocksreleased += btodb(bsize);
}
#ifdef DIAGNOSTIC
if (freeblks->fb_chkcnt != blocksreleased)
When deleting a file, the ordering of events imposed by soft updates is to first write the deleted directory entry to disk, second write the zero'ed inode to disk, and finally to release the freed blocks and the inode back to the cylinder-group map. As this ordering requires two disk writes to occur which are normally spaced about 30 seconds apart (except when memory is under duress), it takes about a minute from the time that a file is deleted until its inode and data blocks show up in the cylinder-group map for reallocation. If a file has had only a brief lifetime (less than 30 seconds from creation to deletion), neither its inode nor its directory entry may have been written to disk. If its directory entry has not been written to disk, then we need not wait for that directory block to be written as the on-disk directory block does not reference the inode. Similarly, if the allocated inode has never been written to disk, we do not have to wait for it to be written back either as its on-disk representation is still zero'ed out. Thus, in the case of a short lived file, we can simply release the blocks and inode to the cylinder-group map immediately. As the inode and its blocks are released immediately, they are immediately available for other uses. If they are not released for a minute, then other inodes and blocks must be allocated for short lived files, cluttering up the vnode and buffer caches. The previous code was a bit too aggressive in trying to release the blocks and inode back to the cylinder-group map resulting in their being made available when in fact the inode on disk had not yet been zero'ed. This patch takes a more conservative approach to doing the release which avoids doing the release prematurely.
2000-11-14 09:00:25 +00:00
printf("handle_workitem_freeblocks: block count");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (allerror)
softdep_error("handle_workitem_freeblks", allerror);
#endif /* DIAGNOSTIC */
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(freeblks, D_FREEBLKS);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Release blocks associated with the inode ip and stored in the indirect
* block dbn. If level is greater than SINGLE, the block is an indirect block
* and recursive calls to indirtrunc must be used to cleanse other indirect
* blocks.
*/
static int
indir_trunc(ip, dbn, level, lbn, countp)
struct inode *ip;
ufs_daddr_t dbn;
int level;
ufs_lbn_t lbn;
long *countp;
{
struct buf *bp;
ufs_daddr_t *bap;
ufs_daddr_t nb;
struct fs *fs;
struct worklist *wk;
struct indirdep *indirdep;
int i, lbnadd, nblocks;
int error, allerror = 0;
fs = ip->i_fs;
lbnadd = 1;
for (i = level; i > 0; i--)
lbnadd *= NINDIR(fs);
/*
* Get buffer of block pointers to be freed. This routine is not
* called until the zero'ed inode has been written, so it is safe
* to free blocks as they are encountered. Because the inode has
* been zero'ed, calls to bmap on these blocks will fail. So, we
* have to use the on-disk address and the block device for the
* filesystem to look them up. If the file was deleted before its
* indirect blocks were all written to disk, the routine that set
* us up (deallocate_dependencies) will have arranged to leave
* a complete copy of the indirect block in memory for our use.
* Otherwise we have to read the blocks in from the disk.
*/
ACQUIRE_LOCK(&lk);
if ((bp = incore(ip->i_devvp, dbn)) != NULL &&
(wk = LIST_FIRST(&bp->b_dep)) != NULL) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if (wk->wk_type != D_INDIRDEP ||
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
(indirdep = WK_INDIRDEP(wk))->ir_savebp != bp ||
(indirdep->ir_state & GOINGAWAY) == 0)
panic("indir_trunc: lost indirdep");
WORKLIST_REMOVE(wk);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(indirdep, D_INDIRDEP);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (LIST_FIRST(&bp->b_dep) != NULL)
panic("indir_trunc: dangling dep");
FREE_LOCK(&lk);
} else {
FREE_LOCK(&lk);
error = bread(ip->i_devvp, dbn, (int)fs->fs_bsize, NOCRED, &bp);
if (error)
return (error);
}
/*
* Recursively free indirect blocks.
*/
bap = (ufs_daddr_t *)bp->b_data;
nblocks = btodb(fs->fs_bsize);
for (i = NINDIR(fs) - 1; i >= 0; i--) {
if ((nb = bap[i]) == 0)
continue;
if (level != 0) {
if ((error = indir_trunc(ip, fsbtodb(fs, nb),
level - 1, lbn + (i * lbnadd), countp)) != 0)
allerror = error;
}
ffs_blkfree(ip, nb, fs->fs_bsize);
*countp += nblocks;
}
Add B_NOCACHE to several cases where BSD4.4 only required a B_INVAL. Change worked out by john and kirk in consort.
1998-06-11 17:44:32 +00:00
bp->b_flags |= B_INVAL | B_NOCACHE;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
brelse(bp);
return (allerror);
}
/*
* Free an allocindir.
* This routine must be called with splbio interrupts blocked.
*/
static void
free_allocindir(aip, inodedep)
struct allocindir *aip;
struct inodedep *inodedep;
{
struct freefrag *freefrag;
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("free_allocindir: lock not held");
#endif
if ((aip->ai_state & DEPCOMPLETE) == 0)
LIST_REMOVE(aip, ai_deps);
if (aip->ai_state & ONWORKLIST)
WORKLIST_REMOVE(&aip->ai_list);
LIST_REMOVE(aip, ai_next);
if ((freefrag = aip->ai_freefrag) != NULL) {
if (inodedep == NULL)
add_to_worklist(&freefrag->ff_list);
else
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
WORKLIST_INSERT(&inodedep->id_bufwait,
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
&freefrag->ff_list);
}
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(aip, D_ALLOCINDIR);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Directory entry addition dependencies.
*
* When adding a new directory entry, the inode (with its incremented link
* count) must be written to disk before the directory entry's pointer to it.
* Also, if the inode is newly allocated, the corresponding freemap must be
* updated (on disk) before the directory entry's pointer. These requirements
* are met via undo/redo on the directory entry's pointer, which consists
* simply of the inode number.
*
* As directory entries are added and deleted, the free space within a
* directory block can become fragmented. The ufs file system will compact
* a fragmented directory block to make space for a new entry. When this
* occurs, the offsets of previously added entries change. Any "diradd"
* dependency structures corresponding to these entries must be updated with
* the new offsets.
*/
/*
* This routine is called after the in-memory inode's link
* count has been incremented, but before the directory entry's
* pointer to the inode has been set.
*/
void
softdep_setup_directory_add(bp, dp, diroffset, newinum, newdirbp)
struct buf *bp; /* buffer containing directory block */
struct inode *dp; /* inode for directory */
off_t diroffset; /* offset of new entry in directory */
long newinum; /* inode referenced by new directory entry */
struct buf *newdirbp; /* non-NULL => contents of new mkdir */
{
int offset; /* offset of new entry within directory block */
ufs_lbn_t lbn; /* block in directory containing new entry */
struct fs *fs;
struct diradd *dap;
struct pagedep *pagedep;
struct inodedep *inodedep;
struct mkdir *mkdir1, *mkdir2;
/*
* Whiteouts have no dependencies.
*/
if (newinum == WINO) {
if (newdirbp != NULL)
bdwrite(newdirbp);
return;
}
fs = dp->i_fs;
lbn = lblkno(fs, diroffset);
offset = blkoff(fs, diroffset);
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
MALLOC(dap, struct diradd *, sizeof(struct diradd), M_DIRADD,
M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
dap->da_list.wk_type = D_DIRADD;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
dap->da_offset = offset;
dap->da_newinum = newinum;
dap->da_state = ATTACHED;
if (newdirbp == NULL) {
dap->da_state |= DEPCOMPLETE;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
ACQUIRE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
} else {
dap->da_state |= MKDIR_BODY | MKDIR_PARENT;
MALLOC(mkdir1, struct mkdir *, sizeof(struct mkdir), M_MKDIR,
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
mkdir1->md_list.wk_type = D_MKDIR;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
mkdir1->md_state = MKDIR_BODY;
mkdir1->md_diradd = dap;
MALLOC(mkdir2, struct mkdir *, sizeof(struct mkdir), M_MKDIR,
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_SOFTDEP_FLAGS);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
mkdir2->md_list.wk_type = D_MKDIR;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
mkdir2->md_state = MKDIR_PARENT;
mkdir2->md_diradd = dap;
/*
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
* Dependency on "." and ".." being written to disk.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Ensure that softdep_sync_metadata can handle bmsafemap and mkdir entries if they ever arise (which should not happen as softdep_sync_metadata is currently used).
1999-03-02 00:19:47 +00:00
mkdir1->md_buf = newdirbp;
Reorganize locking to avoid holding the lock during calls to bdwrite and brelse (which may sleep in some systems). Obtained from: Matthew Dillon <dillon@apollo.backplane.com>
1999-03-02 06:38:07 +00:00
ACQUIRE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
LIST_INSERT_HEAD(&mkdirlisthd, mkdir1, md_mkdirs);
WORKLIST_INSERT(&newdirbp->b_dep, &mkdir1->md_list);
Reorganize locking to avoid holding the lock during calls to bdwrite and brelse (which may sleep in some systems). Obtained from: Matthew Dillon <dillon@apollo.backplane.com>
1999-03-02 06:38:07 +00:00
FREE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bdwrite(newdirbp);
/*
* Dependency on link count increase for parent directory
*/
Reorganize locking to avoid holding the lock during calls to bdwrite and brelse (which may sleep in some systems). Obtained from: Matthew Dillon <dillon@apollo.backplane.com>
1999-03-02 06:38:07 +00:00
ACQUIRE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (inodedep_lookup(dp->i_fs, dp->i_number, 0, &inodedep) == 0
|| (inodedep->id_state & ALLCOMPLETE) == ALLCOMPLETE) {
dap->da_state &= ~MKDIR_PARENT;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(mkdir2, D_MKDIR);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
} else {
LIST_INSERT_HEAD(&mkdirlisthd, mkdir2, md_mkdirs);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
WORKLIST_INSERT(&inodedep->id_bufwait,&mkdir2->md_list);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
}
/*
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
* Link into parent directory pagedep to await its being written.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
if (pagedep_lookup(dp, lbn, DEPALLOC, &pagedep) == 0)
WORKLIST_INSERT(&bp->b_dep, &pagedep->pd_list);
dap->da_pagedep = pagedep;
LIST_INSERT_HEAD(&pagedep->pd_diraddhd[DIRADDHASH(offset)], dap,
da_pdlist);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
/*
* Link into its inodedep. Put it on the id_bufwait list if the inode
* is not yet written. If it is written, do the post-inode write
* processing to put it on the id_pendinghd list.
*/
(void) inodedep_lookup(fs, newinum, DEPALLOC, &inodedep);
if ((inodedep->id_state & ALLCOMPLETE) == ALLCOMPLETE)
diradd_inode_written(dap, inodedep);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
else
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
WORKLIST_INSERT(&inodedep->id_bufwait, &dap->da_list);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
}
/*
* This procedure is called to change the offset of a directory
* entry when compacting a directory block which must be owned
* exclusively by the caller. Note that the actual entry movement
* must be done in this procedure to ensure that no I/O completions
* occur while the move is in progress.
*/
void
softdep_change_directoryentry_offset(dp, base, oldloc, newloc, entrysize)
struct inode *dp; /* inode for directory */
caddr_t base; /* address of dp->i_offset */
caddr_t oldloc; /* address of old directory location */
caddr_t newloc; /* address of new directory location */
int entrysize; /* size of directory entry */
{
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
int offset, oldoffset, newoffset;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct pagedep *pagedep;
struct diradd *dap;
ufs_lbn_t lbn;
ACQUIRE_LOCK(&lk);
lbn = lblkno(dp->i_fs, dp->i_offset);
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
offset = blkoff(dp->i_fs, dp->i_offset);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (pagedep_lookup(dp, lbn, 0, &pagedep) == 0)
goto done;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
oldoffset = offset + (oldloc - base);
newoffset = offset + (newloc - base);
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(dap, &pagedep->pd_diraddhd[DIRADDHASH(oldoffset)], da_pdlist) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (dap->da_offset != oldoffset)
continue;
dap->da_offset = newoffset;
if (DIRADDHASH(newoffset) == DIRADDHASH(oldoffset))
break;
LIST_REMOVE(dap, da_pdlist);
LIST_INSERT_HEAD(&pagedep->pd_diraddhd[DIRADDHASH(newoffset)],
dap, da_pdlist);
break;
}
fix double LIST_REMOVE; other cosmetic changes to match version 9.32. Obtained from: Jeffrey Hsu <hsu@FreeBSD.ORG>
1999-02-17 20:01:20 +00:00
if (dap == NULL) {
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(dap, &pagedep->pd_pendinghd, da_pdlist) {
fix double LIST_REMOVE; other cosmetic changes to match version 9.32. Obtained from: Jeffrey Hsu <hsu@FreeBSD.ORG>
1999-02-17 20:01:20 +00:00
if (dap->da_offset == oldoffset) {
dap->da_offset = newoffset;
break;
}
}
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
done:
bcopy(oldloc, newloc, entrysize);
FREE_LOCK(&lk);
}
/*
* Free a diradd dependency structure. This routine must be called
* with splbio interrupts blocked.
*/
static void
free_diradd(dap)
struct diradd *dap;
{
struct dirrem *dirrem;
struct pagedep *pagedep;
struct inodedep *inodedep;
struct mkdir *mkdir, *nextmd;
#ifdef DEBUG
if (lk.lkt_held == -1)
panic("free_diradd: lock not held");
#endif
WORKLIST_REMOVE(&dap->da_list);
LIST_REMOVE(dap, da_pdlist);
if ((dap->da_state & DIRCHG) == 0) {
pagedep = dap->da_pagedep;
} else {
dirrem = dap->da_previous;
pagedep = dirrem->dm_pagedep;
Handle the case of moving a directory onto the top of a sibling's child of the same name. Submitted by: Kirk Mckusick with fixes from luoqi Chen Obtained from: Whistle test tree.
1998-08-12 20:46:47 +00:00
dirrem->dm_dirinum = pagedep->pd_ino;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
add_to_worklist(&dirrem->dm_list);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
if (inodedep_lookup(VFSTOUFS(pagedep->pd_mnt)->um_fs, dap->da_newinum,
0, &inodedep) != 0)
(void) free_inodedep(inodedep);
if ((dap->da_state & (MKDIR_PARENT | MKDIR_BODY)) != 0) {
for (mkdir = LIST_FIRST(&mkdirlisthd); mkdir; mkdir = nextmd) {
nextmd = LIST_NEXT(mkdir, md_mkdirs);
if (mkdir->md_diradd != dap)
continue;
dap->da_state &= ~mkdir->md_state;
WORKLIST_REMOVE(&mkdir->md_list);
LIST_REMOVE(mkdir, md_mkdirs);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(mkdir, D_MKDIR);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
if ((dap->da_state & (MKDIR_PARENT | MKDIR_BODY)) != 0)
panic("free_diradd: unfound ref");
}
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(dap, D_DIRADD);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Directory entry removal dependencies.
*
* When removing a directory entry, the entry's inode pointer must be
* zero'ed on disk before the corresponding inode's link count is decremented
* (possibly freeing the inode for re-use). This dependency is handled by
* updating the directory entry but delaying the inode count reduction until
* after the directory block has been written to disk. After this point, the
* inode count can be decremented whenever it is convenient.
*/
/*
* This routine should be called immediately after removing
* a directory entry. The inode's link count should not be
* decremented by the calling procedure -- the soft updates
* code will do this task when it is safe.
*/
void
softdep_setup_remove(bp, dp, ip, isrmdir)
struct buf *bp; /* buffer containing directory block */
struct inode *dp; /* inode for the directory being modified */
struct inode *ip; /* inode for directory entry being removed */
int isrmdir; /* indicates if doing RMDIR */
{
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
struct dirrem *dirrem, *prevdirrem;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Allocate a new dirrem if appropriate and ACQUIRE_LOCK.
*/
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
dirrem = newdirrem(bp, dp, ip, isrmdir, &prevdirrem);
/*
* If the COMPLETE flag is clear, then there were no active
* entries and we want to roll back to a zeroed entry until
* the new inode is committed to disk. If the COMPLETE flag is
* set then we have deleted an entry that never made it to
* disk. If the entry we deleted resulted from a name change,
* then the old name still resides on disk. We cannot delete
* its inode (returned to us in prevdirrem) until the zeroed
* directory entry gets to disk. The new inode has never been
* referenced on the disk, so can be deleted immediately.
*/
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if ((dirrem->dm_state & COMPLETE) == 0) {
LIST_INSERT_HEAD(&dirrem->dm_pagedep->pd_dirremhd, dirrem,
dm_next);
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
FREE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
} else {
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
if (prevdirrem != NULL)
LIST_INSERT_HEAD(&dirrem->dm_pagedep->pd_dirremhd,
prevdirrem, dm_next);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
dirrem->dm_dirinum = dirrem->dm_pagedep->pd_ino;
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
FREE_LOCK(&lk);
handle_workitem_remove(dirrem);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
}
/*
* Allocate a new dirrem if appropriate and return it along with
* its associated pagedep. Called without a lock, returns with lock.
*/
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
static long num_dirrem; /* number of dirrem allocated */
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
static struct dirrem *
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
newdirrem(bp, dp, ip, isrmdir, prevdirremp)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct buf *bp; /* buffer containing directory block */
struct inode *dp; /* inode for the directory being modified */
struct inode *ip; /* inode for directory entry being removed */
int isrmdir; /* indicates if doing RMDIR */
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
struct dirrem **prevdirremp; /* previously referenced inode, if any */
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
{
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
int offset;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ufs_lbn_t lbn;
struct diradd *dap;
struct dirrem *dirrem;
struct pagedep *pagedep;
/*
* Whiteouts have no deletion dependencies.
*/
if (ip == NULL)
panic("newdirrem: whiteout");
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
/*
* If we are over our limit, try to improve the situation.
* Limiting the number of dirrem structures will also limit
* the number of freefile and freeblks structures.
*/
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
if (num_dirrem > max_softdeps / 2)
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
(void) request_cleanup(FLUSH_REMOVE, 0);
num_dirrem += 1;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
MALLOC(dirrem, struct dirrem *, sizeof(struct dirrem),
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
M_DIRREM, M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
dirrem->dm_list.wk_type = D_DIRREM;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
dirrem->dm_state = isrmdir ? RMDIR : 0;
dirrem->dm_mnt = ITOV(ip)->v_mount;
dirrem->dm_oldinum = ip->i_number;
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
*prevdirremp = NULL;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ACQUIRE_LOCK(&lk);
lbn = lblkno(dp->i_fs, dp->i_offset);
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
offset = blkoff(dp->i_fs, dp->i_offset);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (pagedep_lookup(dp, lbn, DEPALLOC, &pagedep) == 0)
WORKLIST_INSERT(&bp->b_dep, &pagedep->pd_list);
dirrem->dm_pagedep = pagedep;
Fix for "live inode" panic. Submitted by: Kirk McKusick <mckusick@McKusick.COM> Reviewed by: yeah right...
1998-06-10 20:45:46 +00:00
/*
* Check for a diradd dependency for the same directory entry.
* If present, then both dependencies become obsolete and can
* be de-allocated. Check for an entry on both the pd_dirraddhd
* list and the pd_pendinghd list.
*/
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(dap, &pagedep->pd_diraddhd[DIRADDHASH(offset)], da_pdlist)
Fix for "live inode" panic. Submitted by: Kirk McKusick <mckusick@McKusick.COM> Reviewed by: yeah right...
1998-06-10 20:45:46 +00:00
if (dap->da_offset == offset)
break;
if (dap == NULL) {
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(dap, &pagedep->pd_pendinghd, da_pdlist)
Fix for "live inode" panic. Submitted by: Kirk McKusick <mckusick@McKusick.COM> Reviewed by: yeah right...
1998-06-10 20:45:46 +00:00
if (dap->da_offset == offset)
break;
if (dap == NULL)
return (dirrem);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
Fix for "live inode" panic. Submitted by: Kirk McKusick <mckusick@McKusick.COM> Reviewed by: yeah right...
1998-06-10 20:45:46 +00:00
/*
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
* Must be ATTACHED at this point.
Fix for "live inode" panic. Submitted by: Kirk McKusick <mckusick@McKusick.COM> Reviewed by: yeah right...
1998-06-10 20:45:46 +00:00
*/
if ((dap->da_state & ATTACHED) == 0)
panic("newdirrem: not ATTACHED");
if (dap->da_newinum != ip->i_number)
panic("newdirrem: inum %d should be %d",
ip->i_number, dap->da_newinum);
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
/*
* If we are deleting a changed name that never made it to disk,
* then return the dirrem describing the previous inode (which
* represents the inode currently referenced from this entry on disk).
*/
if ((dap->da_state & DIRCHG) != 0) {
*prevdirremp = dap->da_previous;
dap->da_state &= ~DIRCHG;
dap->da_pagedep = pagedep;
}
/*
* We are deleting an entry that never made it to disk.
* Mark it COMPLETE so we can delete its inode immediately.
*/
Fix for "live inode" panic. Submitted by: Kirk McKusick <mckusick@McKusick.COM> Reviewed by: yeah right...
1998-06-10 20:45:46 +00:00
dirrem->dm_state |= COMPLETE;
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
free_diradd(dap);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (dirrem);
}
/*
* Directory entry change dependencies.
*
* Changing an existing directory entry requires that an add operation
* be completed first followed by a deletion. The semantics for the addition
* are identical to the description of adding a new entry above except
* that the rollback is to the old inode number rather than zero. Once
* the addition dependency is completed, the removal is done as described
* in the removal routine above.
*/
/*
* This routine should be called immediately after changing
* a directory entry. The inode's link count should not be
* decremented by the calling procedure -- the soft updates
* code will perform this task when it is safe.
*/
void
softdep_setup_directory_change(bp, dp, ip, newinum, isrmdir)
struct buf *bp; /* buffer containing directory block */
struct inode *dp; /* inode for the directory being modified */
struct inode *ip; /* inode for directory entry being removed */
long newinum; /* new inode number for changed entry */
int isrmdir; /* indicates if doing RMDIR */
{
int offset;
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
struct diradd *dap = NULL;
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
struct dirrem *dirrem, *prevdirrem;
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
struct pagedep *pagedep;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct inodedep *inodedep;
offset = blkoff(dp->i_fs, dp->i_offset);
/*
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
* Whiteouts do not need diradd dependencies.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
if (newinum != WINO) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
MALLOC(dap, struct diradd *, sizeof(struct diradd),
Convert more malloc+bzero to malloc+M_ZERO. Submitted by: josh@zipperup.org Submitted by: Robert Drehmel <robd@gmx.net>
2000-12-08 21:51:06 +00:00
M_DIRADD, M_SOFTDEP_FLAGS|M_ZERO);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
dap->da_list.wk_type = D_DIRADD;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
dap->da_state = DIRCHG | ATTACHED | DEPCOMPLETE;
dap->da_offset = offset;
dap->da_newinum = newinum;
}
/*
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
* Allocate a new dirrem and ACQUIRE_LOCK.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
dirrem = newdirrem(bp, dp, ip, isrmdir, &prevdirrem);
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
pagedep = dirrem->dm_pagedep;
Handle the case of moving a directory onto the top of a sibling's child of the same name. Submitted by: Kirk Mckusick with fixes from luoqi Chen Obtained from: Whistle test tree.
1998-08-12 20:46:47 +00:00
/*
* The possible values for isrmdir:
* 0 - non-directory file rename
* 1 - directory rename within same directory
* inum - directory rename to new directory of given inode number
* When renaming to a new directory, we are both deleting and
* creating a new directory entry, so the link count on the new
* directory should not change. Thus we do not need the followup
* dirrem which is usually done in handle_workitem_remove. We set
* the DIRCHG flag to tell handle_workitem_remove to skip the
* followup dirrem.
*/
if (isrmdir > 1)
dirrem->dm_state |= DIRCHG;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
* Whiteouts have no additional dependencies,
* so just put the dirrem on the correct list.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
if (newinum == WINO) {
if ((dirrem->dm_state & COMPLETE) == 0) {
LIST_INSERT_HEAD(&pagedep->pd_dirremhd, dirrem,
dm_next);
} else {
dirrem->dm_dirinum = pagedep->pd_ino;
add_to_worklist(&dirrem->dm_list);
}
FREE_LOCK(&lk);
return;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
Must track multiple uncommitted renames until one ultimately gets committed to disk or is removed.
2000-01-17 06:28:18 +00:00
/*
* If the COMPLETE flag is clear, then there were no active
* entries and we want to roll back to the previous inode until
* the new inode is committed to disk. If the COMPLETE flag is
* set, then we have deleted an entry that never made it to disk.
* If the entry we deleted resulted from a name change, then the old
* inode reference still resides on disk. Any rollback that we do
* needs to be to that old inode (returned to us in prevdirrem). If
* the entry we deleted resulted from a create, then there is
* no entry on the disk, so we want to roll back to zero rather
* than the uncommitted inode. In either of the COMPLETE cases we
* want to immediately free the unwritten and unreferenced inode.
*/
if ((dirrem->dm_state & COMPLETE) == 0) {
dap->da_previous = dirrem;
} else {
if (prevdirrem != NULL) {
dap->da_previous = prevdirrem;
} else {
dap->da_state &= ~DIRCHG;
dap->da_pagedep = pagedep;
}
dirrem->dm_dirinum = pagedep->pd_ino;
add_to_worklist(&dirrem->dm_list);
}
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
/*
* Link into its inodedep. Put it on the id_bufwait list if the inode
* is not yet written. If it is written, do the post-inode write
* processing to put it on the id_pendinghd list.
*/
if (inodedep_lookup(dp->i_fs, newinum, DEPALLOC, &inodedep) == 0 ||
(inodedep->id_state & ALLCOMPLETE) == ALLCOMPLETE) {
dap->da_state |= COMPLETE;
LIST_INSERT_HEAD(&pagedep->pd_pendinghd, dap, da_pdlist);
WORKLIST_INSERT(&inodedep->id_pendinghd, &dap->da_list);
} else {
LIST_INSERT_HEAD(&pagedep->pd_diraddhd[DIRADDHASH(offset)],
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
dap, da_pdlist);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
WORKLIST_INSERT(&inodedep->id_bufwait, &dap->da_list);
Fix the case when renaming to a file that you've just created and deleted, that had an inode that has not yet been written to disk, when the inode of the new file is also not yet written to disk, and your old directory entry is not yet on disk but you need to remove it and the new name exists in memory but has been deleted but the transaction to write the deleted name to disk exists and has not yet been cancelled by the request to delete the non existant name. I don't know how kirk could have missed such a glaring problem for so long. :-) Especially since the inconsitency survived on the disk for a whole 4 second on average before being fixed by other code. This was not a crashing bug but just led to filesystem inconsitencies if you crashed. Submitted by: Kirk McKusick (mckusick@mckusick.com)
1998-06-12 20:48:30 +00:00
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
}
/*
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
* Called whenever the link count on an inode is changed.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
* It creates an inode dependency so that the new reference(s)
* to the inode cannot be committed to disk until the updated
* inode has been written.
*/
void
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
softdep_change_linkcnt(ip)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct inode *ip; /* the inode with the increased link count */
{
struct inodedep *inodedep;
ACQUIRE_LOCK(&lk);
(void) inodedep_lookup(ip->i_fs, ip->i_number, DEPALLOC, &inodedep);
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
if (ip->i_nlink < ip->i_effnlink)
panic("softdep_change_linkcnt: bad delta");
inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
}
/*
* This workitem decrements the inode's link count.
* If the link count reaches zero, the file is removed.
*/
static void
handle_workitem_remove(dirrem)
struct dirrem *dirrem;
{
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
struct proc *p = CURPROC; /* XXX */
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct inodedep *inodedep;
struct vnode *vp;
struct inode *ip;
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
ino_t oldinum;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
int error;
if ((error = VFS_VGET(dirrem->dm_mnt, dirrem->dm_oldinum, &vp)) != 0) {
softdep_error("handle_workitem_remove: vget", error);
return;
}
ip = VTOI(vp);
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
ACQUIRE_LOCK(&lk);
if ((inodedep_lookup(ip->i_fs, dirrem->dm_oldinum, 0, &inodedep)) == 0)
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
panic("handle_workitem_remove: lost inodedep");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Normal file deletion.
*/
if ((dirrem->dm_state & RMDIR) == 0) {
ip->i_nlink--;
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
ip->i_flag |= IN_CHANGE;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
if (ip->i_nlink < ip->i_effnlink)
panic("handle_workitem_remove: bad file delta");
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink;
FREE_LOCK(&lk);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
vput(vp);
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
num_dirrem -= 1;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(dirrem, D_DIRREM);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return;
}
/*
* Directory deletion. Decrement reference count for both the
* just deleted parent directory entry and the reference for ".".
* Next truncate the directory to length zero. When the
* truncation completes, arrange to have the reference count on
* the parent decremented to account for the loss of "..".
*/
ip->i_nlink -= 2;
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
ip->i_flag |= IN_CHANGE;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (ip->i_nlink < ip->i_effnlink)
panic("handle_workitem_remove: bad dir delta");
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink;
FREE_LOCK(&lk);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if ((error = UFS_TRUNCATE(vp, (off_t)0, 0, p->p_ucred, p)) != 0)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_error("handle_workitem_remove: truncate", error);
Handle the case of moving a directory onto the top of a sibling's child of the same name. Submitted by: Kirk Mckusick with fixes from luoqi Chen Obtained from: Whistle test tree.
1998-08-12 20:46:47 +00:00
/*
* Rename a directory to a new parent. Since, we are both deleting
* and creating a new directory entry, the link count on the new
* directory should not change. Thus we skip the followup dirrem.
*/
if (dirrem->dm_state & DIRCHG) {
vput(vp);
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
num_dirrem -= 1;
Handle the case of moving a directory onto the top of a sibling's child of the same name. Submitted by: Kirk Mckusick with fixes from luoqi Chen Obtained from: Whistle test tree.
1998-08-12 20:46:47 +00:00
WORKITEM_FREE(dirrem, D_DIRREM);
return;
}
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
/*
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
* If the inodedep does not exist, then the zero'ed inode has
* been written to disk. If the allocated inode has never been
* written to disk, then the on-disk inode is zero'ed. In either
* case we can remove the file immediately.
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
*/
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ACQUIRE_LOCK(&lk);
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
dirrem->dm_state = 0;
oldinum = dirrem->dm_oldinum;
dirrem->dm_oldinum = dirrem->dm_dirinum;
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
if (inodedep_lookup(ip->i_fs, oldinum, 0, &inodedep) == 0 ||
check_inode_unwritten(inodedep)) {
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
FREE_LOCK(&lk);
vput(vp);
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
handle_workitem_remove(dirrem);
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
return;
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
}
During fastpath processing for removal of a short-lived inode, the set of restrictions for cancelling an inode dependency (inodedep) is somewhat stronger than originally coded. Since this check appears in two places, we codify it into the function check_inode_unwritten which we then call from the two sites, one freeing blocks and the other freeing directory entries. Submitted by: Steinar Haug via Matthew Dillon
2000-01-18 01:33:05 +00:00
WORKLIST_INSERT(&inodedep->id_inowait, &dirrem->dm_list);
We cannot proceed to free the blocks of the file until the dependencies have been cleaned up by deallocte_dependencies(). Once that is done, it is safe to post the request to free the blocks. A similar change is also needed for the freefile case.
2000-01-11 06:52:35 +00:00
FREE_LOCK(&lk);
vput(vp);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Inode de-allocation dependencies.
*
* When an inode's link count is reduced to zero, it can be de-allocated. We
* found it convenient to postpone de-allocation until after the inode is
* written to disk with its new link count (zero). At this point, all of the
* on-disk inode's block pointers are nullified and, with careful dependency
* list ordering, all dependencies related to the inode will be satisfied and
* the corresponding dependency structures de-allocated. So, if/when the
* inode is reused, there will be no mixing of old dependencies with new
* ones. This artificial dependency is set up by the block de-allocation
* procedure above (softdep_setup_freeblocks) and completed by the
* following procedure.
*/
static void
handle_workitem_freefile(freefile)
struct freefile *freefile;
{
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
struct fs *fs;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct vnode vp;
struct inode tip;
struct inodedep *idp;
int error;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
fs = VFSTOUFS(freefile->fx_mnt)->um_fs;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
#ifdef DEBUG
ACQUIRE_LOCK(&lk);
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
if (inodedep_lookup(fs, freefile->fx_oldinum, 0, &idp))
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("handle_workitem_freefile: inodedep survived");
FREE_LOCK(&lk);
#endif
tip.i_devvp = freefile->fx_devvp;
tip.i_dev = freefile->fx_devvp->v_rdev;
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
tip.i_fs = fs;
tip.i_vnode = &vp;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
vp.v_data = &tip;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if ((error = ffs_freefile(&vp, freefile->fx_oldinum, freefile->fx_mode)) != 0)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_error("handle_workitem_freefile", error);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(freefile, D_FREEFILE);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Disk writes.
*
* The dependency structures constructed above are most actively used when file
* system blocks are written to disk. No constraints are placed on when a
* block can be written, but unsatisfied update dependencies are made safe by
* modifying (or replacing) the source memory for the duration of the disk
* write. When the disk write completes, the memory block is again brought
* up-to-date.
*
* In-core inode structure reclamation.
*
* Because there are a finite number of "in-core" inode structures, they are
* reused regularly. By transferring all inode-related dependencies to the
* in-memory inode block and indexing them separately (via "inodedep"s), we
* can allow "in-core" inode structures to be reused at any time and avoid
* any increase in contention.
*
* Called just before entering the device driver to initiate a new disk I/O.
* The buffer must be locked, thus, no I/O completion operations can occur
* while we are manipulating its associated dependencies.
*/
Make static non-exported functions from soft updates.
2000-01-09 22:40:09 +00:00
static void
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_disk_io_initiation(bp)
struct buf *bp; /* structure describing disk write to occur */
{
struct worklist *wk, *nextwk;
struct indirdep *indirdep;
/*
* We only care about write operations. There should never
* be dependencies for reads.
*/
Remove B_READ, B_WRITE and B_FREEBUF and replace them with a new field in struct buf: b_iocmd. The b_iocmd is enforced to have exactly one bit set. B_WRITE was bogusly defined as zero giving rise to obvious coding mistakes. Also eliminate the redundant struct buf flag B_CALL, it can just as efficiently be done by comparing b_iodone to NULL. Should you get a panic or drop into the debugger, complaining about "b_iocmd", don't continue. It is likely to write on your disk where it should have been reading. This change is a step in the direction towards a stackable BIO capability. A lot of this patch were machine generated (Thanks to style(9) compliance!) Vinum users: Greg has not had time to test this yet, be careful.
2000-03-20 10:44:49 +00:00
if (bp->b_iocmd == BIO_READ)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("softdep_disk_io_initiation: read");
/*
* Do any necessary pre-I/O processing.
*/
for (wk = LIST_FIRST(&bp->b_dep); wk; wk = nextwk) {
nextwk = LIST_NEXT(wk, wk_list);
switch (wk->wk_type) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_PAGEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
initiate_write_filepage(WK_PAGEDEP(wk), bp);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INODEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
initiate_write_inodeblock(WK_INODEDEP(wk), bp);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INDIRDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
indirdep = WK_INDIRDEP(wk);
if (indirdep->ir_state & GOINGAWAY)
panic("disk_io_initiation: indirdep gone");
/*
* If there are no remaining dependencies, this
* will be writing the real pointers, so the
* dependency can be freed.
*/
if (LIST_FIRST(&indirdep->ir_deplisthd) == NULL) {
Add B_NOCACHE to several cases where BSD4.4 only required a B_INVAL. Change worked out by john and kirk in consort.
1998-06-11 17:44:32 +00:00
indirdep->ir_savebp->b_flags |= B_INVAL | B_NOCACHE;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
brelse(indirdep->ir_savebp);
/* inline expand WORKLIST_REMOVE(wk); */
wk->wk_state &= ~ONWORKLIST;
LIST_REMOVE(wk, wk_list);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(indirdep, D_INDIRDEP);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
continue;
}
/*
* Replace up-to-date version with safe version.
*/
Cannot do MALLOC with M_WAITOK while holding ACQUIRE_LOCK Obtained from: Ethan Solomita <ethan@geocast.com>
2000-09-07 23:02:55 +00:00
MALLOC(indirdep->ir_saveddata, caddr_t, bp->b_bcount,
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
M_INDIRDEP, M_SOFTDEP_FLAGS);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ACQUIRE_LOCK(&lk);
indirdep->ir_state &= ~ATTACHED;
indirdep->ir_state |= UNDONE;
Bring up-to-date with Whistle's current version Includes some debugging code.
1998-05-19 23:07:25 +00:00
bcopy(bp->b_data, indirdep->ir_saveddata, bp->b_bcount);
bcopy(indirdep->ir_savebp->b_data, bp->b_data,
bp->b_bcount);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_MKDIR:
case D_BMSAFEMAP:
case D_ALLOCDIRECT:
case D_ALLOCINDIR:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
continue;
default:
panic("handle_disk_io_initiation: Unexpected type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
}
}
}
/*
* Called from within the procedure above to deal with unsatisfied
* allocation dependencies in a directory. The buffer must be locked,
* thus, no I/O completion operations can occur while we are
* manipulating its associated dependencies.
*/
static void
initiate_write_filepage(pagedep, bp)
struct pagedep *pagedep;
struct buf *bp;
{
struct diradd *dap;
struct direct *ep;
int i;
if (pagedep->pd_state & IOSTARTED) {
/*
* This can only happen if there is a driver that does not
* understand chaining. Here biodone will reissue the call
* to strategy for the incomplete buffers.
*/
printf("initiate_write_filepage: already started\n");
return;
}
pagedep->pd_state |= IOSTARTED;
ACQUIRE_LOCK(&lk);
for (i = 0; i < DAHASHSZ; i++) {
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(dap, &pagedep->pd_diraddhd[i], da_pdlist) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ep = (struct direct *)
((char *)bp->b_data + dap->da_offset);
if (ep->d_ino != dap->da_newinum)
panic("%s: dir inum %d != new %d",
"initiate_write_filepage",
ep->d_ino, dap->da_newinum);
if (dap->da_state & DIRCHG)
ep->d_ino = dap->da_previous->dm_oldinum;
else
ep->d_ino = 0;
dap->da_state &= ~ATTACHED;
dap->da_state |= UNDONE;
}
}
FREE_LOCK(&lk);
}
/*
* Called from within the procedure above to deal with unsatisfied
* allocation dependencies in an inodeblock. The buffer must be
* locked, thus, no I/O completion operations can occur while we
* are manipulating its associated dependencies.
*/
static void
initiate_write_inodeblock(inodedep, bp)
struct inodedep *inodedep;
struct buf *bp; /* The inode block */
{
struct allocdirect *adp, *lastadp;
struct dinode *dp;
struct fs *fs;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
ufs_lbn_t prevlbn = 0;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
int i, deplist;
if (inodedep->id_state & IOSTARTED)
panic("initiate_write_inodeblock: already started");
inodedep->id_state |= IOSTARTED;
fs = inodedep->id_fs;
dp = (struct dinode *)bp->b_data +
ino_to_fsbo(fs, inodedep->id_ino);
/*
* If the bitmap is not yet written, then the allocated
* inode cannot be written to disk.
*/
if ((inodedep->id_state & DEPCOMPLETE) == 0) {
if (inodedep->id_savedino != NULL)
panic("initiate_write_inodeblock: already doing I/O");
MALLOC(inodedep->id_savedino, struct dinode *,
Implement a low-memory deadlock solution. Removed most of the hacks that were trying to deal with low-memory situations prior to now. The new code is based on the concept that I/O must be able to function in a low memory situation. All major modules related to I/O (except networking) have been adjusted to allow allocation out of the system reserve memory pool. These modules now detect a low memory situation but rather then block they instead continue to operate, then return resources to the memory pool instead of cache them or leave them wired. Code has been added to stall in a low-memory situation prior to a vnode being locked. Thus situations where a process blocks in a low-memory condition while holding a locked vnode have been reduced to near nothing. Not only will I/O continue to operate, but many prior deadlock conditions simply no longer exist. Implement a number of VFS/BIO fixes (found by Ian): in biodone(), bogus-page replacement code, the loop was not properly incrementing loop variables prior to a continue statement. We do not believe this code can be hit anyway but we aren't taking any chances. We'll turn the whole section into a panic (as it already is in brelse()) after the release is rolled. In biodone(), the foff calculation was incorrectly clamped to the iosize, causing the wrong foff to be calculated for pages in the case of an I/O error or biodone() called without initiating I/O. The problem always caused a panic before. Now it doesn't. The problem is mainly an issue with NFS. Fixed casts for ~PAGE_MASK. This code worked properly before only because the calculations use signed arithmatic. Better to properly extend PAGE_MASK first before inverting it for the 64 bit masking op. In brelse(), the bogus_page fixup code was improperly throwing away the original contents of 'm' when it did the j-loop to fix the bogus pages. The result was that it would potentially invalidate parts of the *WRONG* page(!), leading to corruption. There may still be cases where a background bitmap write is being duplicated, causing potential corruption. We have identified a potentially serious bug related to this but the fix is still TBD. So instead this patch contains a KASSERT to detect the problem and panic the machine rather then continue to corrupt the filesystem. The problem does not occur very often.. it is very hard to reproduce, and it may or may not be the cause of the corruption people have reported. Review by: (VFS/BIO: mckusick, Ian Dowse <iedowse@maths.tcd.ie>) Testing by: (VM/Deadlock) Paul Saab <ps@yahoo-inc.com>
2000-11-18 23:06:26 +00:00
sizeof(struct dinode), M_INODEDEP, M_SOFTDEP_FLAGS);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*inodedep->id_savedino = *dp;
bzero((caddr_t)dp, sizeof(struct dinode));
return;
}
/*
* If no dependencies, then there is nothing to roll back.
*/
inodedep->id_savedsize = dp->di_size;
if (TAILQ_FIRST(&inodedep->id_inoupdt) == NULL)
return;
/*
* Set the dependencies to busy.
*/
ACQUIRE_LOCK(&lk);
for (deplist = 0, adp = TAILQ_FIRST(&inodedep->id_inoupdt); adp;
adp = TAILQ_NEXT(adp, ad_next)) {
#ifdef DIAGNOSTIC
if (deplist != 0 && prevlbn >= adp->ad_lbn)
panic("softdep_write_inodeblock: lbn order");
prevlbn = adp->ad_lbn;
if (adp->ad_lbn < NDADDR &&
dp->di_db[adp->ad_lbn] != adp->ad_newblkno)
Remove some compiler warnings.
1998-12-10 20:11:47 +00:00
panic("%s: direct pointer #%ld mismatch %d != %d",
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
"softdep_write_inodeblock", adp->ad_lbn,
dp->di_db[adp->ad_lbn], adp->ad_newblkno);
if (adp->ad_lbn >= NDADDR &&
dp->di_ib[adp->ad_lbn - NDADDR] != adp->ad_newblkno)
Remove some compiler warnings.
1998-12-10 20:11:47 +00:00
panic("%s: indirect pointer #%ld mismatch %d != %d",
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
"softdep_write_inodeblock", adp->ad_lbn - NDADDR,
dp->di_ib[adp->ad_lbn - NDADDR], adp->ad_newblkno);
deplist |= 1 << adp->ad_lbn;
if ((adp->ad_state & ATTACHED) == 0)
panic("softdep_write_inodeblock: Unknown state 0x%x",
adp->ad_state);
#endif /* DIAGNOSTIC */
adp->ad_state &= ~ATTACHED;
adp->ad_state |= UNDONE;
}
/*
* The on-disk inode cannot claim to be any larger than the last
* fragment that has been written. Otherwise, the on-disk inode
* might have fragments that were not the last block in the file
* which would corrupt the filesystem.
*/
for (lastadp = NULL, adp = TAILQ_FIRST(&inodedep->id_inoupdt); adp;
lastadp = adp, adp = TAILQ_NEXT(adp, ad_next)) {
if (adp->ad_lbn >= NDADDR)
break;
dp->di_db[adp->ad_lbn] = adp->ad_oldblkno;
/* keep going until hitting a rollback to a frag */
if (adp->ad_oldsize == 0 || adp->ad_oldsize == fs->fs_bsize)
continue;
dp->di_size = fs->fs_bsize * adp->ad_lbn + adp->ad_oldsize;
for (i = adp->ad_lbn + 1; i < NDADDR; i++) {
#ifdef DIAGNOSTIC
if (dp->di_db[i] != 0 && (deplist & (1 << i)) == 0)
panic("softdep_write_inodeblock: lost dep1");
#endif /* DIAGNOSTIC */
dp->di_db[i] = 0;
}
for (i = 0; i < NIADDR; i++) {
#ifdef DIAGNOSTIC
if (dp->di_ib[i] != 0 &&
(deplist & ((1 << NDADDR) << i)) == 0)
panic("softdep_write_inodeblock: lost dep2");
#endif /* DIAGNOSTIC */
dp->di_ib[i] = 0;
}
FREE_LOCK(&lk);
return;
}
/*
* If we have zero'ed out the last allocated block of the file,
* roll back the size to the last currently allocated block.
* We know that this last allocated block is a full-sized as
* we already checked for fragments in the loop above.
*/
if (lastadp != NULL &&
dp->di_size <= (lastadp->ad_lbn + 1) * fs->fs_bsize) {
for (i = lastadp->ad_lbn; i >= 0; i--)
if (dp->di_db[i] != 0)
break;
dp->di_size = (i + 1) * fs->fs_bsize;
}
/*
* The only dependencies are for indirect blocks.
*
* The file size for indirect block additions is not guaranteed.
* Such a guarantee would be non-trivial to achieve. The conventional
* synchronous write implementation also does not make this guarantee.
* Fsck should catch and fix discrepancies. Arguably, the file size
* can be over-estimated without destroying integrity when the file
* moves into the indirect blocks (i.e., is large). If we want to
* postpone fsck, we are stuck with this argument.
*/
for (; adp; adp = TAILQ_NEXT(adp, ad_next))
dp->di_ib[adp->ad_lbn - NDADDR] = 0;
FREE_LOCK(&lk);
}
/*
* This routine is called during the completion interrupt
* service routine for a disk write (from the procedure called
* by the device driver to inform the file system caches of
* a request completion). It should be called early in this
* procedure, before the block is made available to other
* processes or other routines are called.
*/
Make static non-exported functions from soft updates.
2000-01-09 22:40:09 +00:00
static void
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_disk_write_complete(bp)
struct buf *bp; /* describes the completed disk write */
{
struct worklist *wk;
struct workhead reattach;
struct newblk *newblk;
struct allocindir *aip;
struct allocdirect *adp;
struct indirdep *indirdep;
struct inodedep *inodedep;
struct bmsafemap *bmsafemap;
#ifdef DEBUG
if (lk.lkt_held != -1)
panic("softdep_disk_write_complete: lock is held");
lk.lkt_held = -2;
#endif
LIST_INIT(&reattach);
while ((wk = LIST_FIRST(&bp->b_dep)) != NULL) {
WORKLIST_REMOVE(wk);
switch (wk->wk_type) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_PAGEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (handle_written_filepage(WK_PAGEDEP(wk), bp))
WORKLIST_INSERT(&reattach, wk);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INODEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (handle_written_inodeblock(WK_INODEDEP(wk), bp))
WORKLIST_INSERT(&reattach, wk);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_BMSAFEMAP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bmsafemap = WK_BMSAFEMAP(wk);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
while ((newblk = LIST_FIRST(&bmsafemap->sm_newblkhd))) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
newblk->nb_state |= DEPCOMPLETE;
newblk->nb_bmsafemap = NULL;
LIST_REMOVE(newblk, nb_deps);
}
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
while ((adp =
LIST_FIRST(&bmsafemap->sm_allocdirecthd))) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
adp->ad_state |= DEPCOMPLETE;
adp->ad_buf = NULL;
LIST_REMOVE(adp, ad_deps);
handle_allocdirect_partdone(adp);
}
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
while ((aip =
LIST_FIRST(&bmsafemap->sm_allocindirhd))) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
aip->ai_state |= DEPCOMPLETE;
aip->ai_buf = NULL;
LIST_REMOVE(aip, ai_deps);
handle_allocindir_partdone(aip);
}
while ((inodedep =
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
LIST_FIRST(&bmsafemap->sm_inodedephd)) != NULL) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
inodedep->id_state |= DEPCOMPLETE;
LIST_REMOVE(inodedep, id_deps);
inodedep->id_buf = NULL;
}
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(bmsafemap, D_BMSAFEMAP);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_MKDIR:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
handle_written_mkdir(WK_MKDIR(wk), MKDIR_BODY);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_ALLOCDIRECT:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
adp = WK_ALLOCDIRECT(wk);
adp->ad_state |= COMPLETE;
handle_allocdirect_partdone(adp);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_ALLOCINDIR:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
aip = WK_ALLOCINDIR(wk);
aip->ai_state |= COMPLETE;
handle_allocindir_partdone(aip);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INDIRDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
indirdep = WK_INDIRDEP(wk);
if (indirdep->ir_state & GOINGAWAY)
panic("disk_write_complete: indirdep gone");
Bring up-to-date with Whistle's current version Includes some debugging code.
1998-05-19 23:07:25 +00:00
bcopy(indirdep->ir_saveddata, bp->b_data, bp->b_bcount);
FREE(indirdep->ir_saveddata, M_INDIRDEP);
indirdep->ir_saveddata = 0;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
indirdep->ir_state &= ~UNDONE;
indirdep->ir_state |= ATTACHED;
while ((aip = LIST_FIRST(&indirdep->ir_donehd)) != 0) {
handle_allocindir_partdone(aip);
fix double LIST_REMOVE; other cosmetic changes to match version 9.32. Obtained from: Jeffrey Hsu <hsu@FreeBSD.ORG>
1999-02-17 20:01:20 +00:00
if (aip == LIST_FIRST(&indirdep->ir_donehd))
panic("disk_write_complete: not gone");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
WORKLIST_INSERT(&reattach, wk);
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
if ((bp->b_flags & B_DELWRI) == 0)
stat_indir_blk_ptrs++;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bdirty(bp);
continue;
default:
panic("handle_disk_write_complete: Unknown type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
}
}
/*
* Reattach any requests that must be redone.
*/
while ((wk = LIST_FIRST(&reattach)) != NULL) {
WORKLIST_REMOVE(wk);
WORKLIST_INSERT(&bp->b_dep, wk);
}
#ifdef DEBUG
if (lk.lkt_held != -2)
panic("softdep_disk_write_complete: lock lost");
lk.lkt_held = -1;
#endif
}
/*
* Called from within softdep_disk_write_complete above. Note that
* this routine is always called from interrupt level with further
* splbio interrupts blocked.
*/
static void
handle_allocdirect_partdone(adp)
struct allocdirect *adp; /* the completed allocdirect */
{
struct allocdirect *listadp;
struct inodedep *inodedep;
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
long bsize, delay;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if ((adp->ad_state & ALLCOMPLETE) != ALLCOMPLETE)
return;
if (adp->ad_buf != NULL)
panic("handle_allocdirect_partdone: dangling dep");
/*
* The on-disk inode cannot claim to be any larger than the last
* fragment that has been written. Otherwise, the on-disk inode
* might have fragments that were not the last block in the file
* which would corrupt the filesystem. Thus, we cannot free any
* allocdirects after one whose ad_oldblkno claims a fragment as
* these blocks must be rolled back to zero before writing the inode.
* We check the currently active set of allocdirects in id_inoupdt.
*/
inodedep = adp->ad_inodedep;
bsize = inodedep->id_fs->fs_bsize;
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
TAILQ_FOREACH(listadp, &inodedep->id_inoupdt, ad_next) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/* found our block */
if (listadp == adp)
break;
/* continue if ad_oldlbn is not a fragment */
if (listadp->ad_oldsize == 0 ||
listadp->ad_oldsize == bsize)
continue;
/* hit a fragment */
return;
}
/*
* If we have reached the end of the current list without
* finding the just finished dependency, then it must be
* on the future dependency list. Future dependencies cannot
* be freed until they are moved to the current list.
*/
if (listadp == NULL) {
#ifdef DEBUG
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
TAILQ_FOREACH(listadp, &inodedep->id_newinoupdt, ad_next)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/* found our block */
if (listadp == adp)
break;
if (listadp == NULL)
panic("handle_allocdirect_partdone: lost dep");
#endif /* DEBUG */
return;
}
/*
* If we have found the just finished dependency, then free
* it along with anything that follows it that is complete.
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
* If the inode still has a bitmap dependency, then it has
* never been written to disk, hence the on-disk inode cannot
* reference the old fragment so we can free it without delay.
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
*/
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
delay = (inodedep->id_state & DEPCOMPLETE);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
for (; adp; adp = listadp) {
listadp = TAILQ_NEXT(adp, ad_next);
if ((adp->ad_state & ALLCOMPLETE) != ALLCOMPLETE)
return;
Some additional performance improvements. When freeing an inode check to see if it has been committed to disk. If it has never been written, it can be freed immediately. For short lived files this change allows the same inode to be reused repeatedly. Similarly, when upgrading a fragment to a larger size, if it has never been claimed by an inode on disk, it too can be freed immediately making it available for reuse often in the next slowly growing block of the same file.
2000-06-18 22:05:57 +00:00
free_allocdirect(&inodedep->id_inoupdt, adp, delay);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
}
/*
* Called from within softdep_disk_write_complete above. Note that
* this routine is always called from interrupt level with further
* splbio interrupts blocked.
*/
static void
handle_allocindir_partdone(aip)
struct allocindir *aip; /* the completed allocindir */
{
struct indirdep *indirdep;
if ((aip->ai_state & ALLCOMPLETE) != ALLCOMPLETE)
return;
if (aip->ai_buf != NULL)
panic("handle_allocindir_partdone: dangling dependency");
indirdep = aip->ai_indirdep;
if (indirdep->ir_state & UNDONE) {
LIST_REMOVE(aip, ai_next);
LIST_INSERT_HEAD(&indirdep->ir_donehd, aip, ai_next);
return;
}
((ufs_daddr_t *)indirdep->ir_savebp->b_data)[aip->ai_offset] =
aip->ai_newblkno;
LIST_REMOVE(aip, ai_next);
if (aip->ai_freefrag != NULL)
add_to_worklist(&aip->ai_freefrag->ff_list);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(aip, D_ALLOCINDIR);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Called from within softdep_disk_write_complete above to restore
* in-memory inode block contents to their most up-to-date state. Note
* that this routine is always called from interrupt level with further
* splbio interrupts blocked.
*/
static int
handle_written_inodeblock(inodedep, bp)
struct inodedep *inodedep;
struct buf *bp; /* buffer containing the inode block */
{
struct worklist *wk, *filefree;
struct allocdirect *adp, *nextadp;
struct dinode *dp;
int hadchanges;
if ((inodedep->id_state & IOSTARTED) == 0)
panic("handle_written_inodeblock: not started");
inodedep->id_state &= ~IOSTARTED;
inodedep->id_state |= COMPLETE;
dp = (struct dinode *)bp->b_data +
ino_to_fsbo(inodedep->id_fs, inodedep->id_ino);
/*
* If we had to rollback the inode allocation because of
* bitmaps being incomplete, then simply restore it.
* Keep the block dirty so that it will not be reclaimed until
* all associated dependencies have been cleared and the
* corresponding updates written to disk.
*/
if (inodedep->id_savedino != NULL) {
*dp = *inodedep->id_savedino;
FREE(inodedep->id_savedino, M_INODEDEP);
inodedep->id_savedino = NULL;
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
if ((bp->b_flags & B_DELWRI) == 0)
stat_inode_bitmap++;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bdirty(bp);
return (1);
}
/*
* Roll forward anything that had to be rolled back before
* the inode could be updated.
*/
hadchanges = 0;
for (adp = TAILQ_FIRST(&inodedep->id_inoupdt); adp; adp = nextadp) {
nextadp = TAILQ_NEXT(adp, ad_next);
if (adp->ad_state & ATTACHED)
panic("handle_written_inodeblock: new entry");
if (adp->ad_lbn < NDADDR) {
if (dp->di_db[adp->ad_lbn] != adp->ad_oldblkno)
Remove some compiler warnings.
1998-12-10 20:11:47 +00:00
panic("%s: %s #%ld mismatch %d != %d",
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
"handle_written_inodeblock",
"direct pointer", adp->ad_lbn,
dp->di_db[adp->ad_lbn], adp->ad_oldblkno);
dp->di_db[adp->ad_lbn] = adp->ad_newblkno;
} else {
if (dp->di_ib[adp->ad_lbn - NDADDR] != 0)
Remove some compiler warnings.
1998-12-10 20:11:47 +00:00
panic("%s: %s #%ld allocated as %d",
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
"handle_written_inodeblock",
"indirect pointer", adp->ad_lbn - NDADDR,
dp->di_ib[adp->ad_lbn - NDADDR]);
dp->di_ib[adp->ad_lbn - NDADDR] = adp->ad_newblkno;
}
adp->ad_state &= ~UNDONE;
adp->ad_state |= ATTACHED;
hadchanges = 1;
}
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
if (hadchanges && (bp->b_flags & B_DELWRI) == 0)
stat_direct_blk_ptrs++;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Reset the file size to its most up-to-date value.
*/
if (inodedep->id_savedsize == -1)
panic("handle_written_inodeblock: bad size");
if (dp->di_size != inodedep->id_savedsize) {
dp->di_size = inodedep->id_savedsize;
hadchanges = 1;
}
inodedep->id_savedsize = -1;
/*
* If there were any rollbacks in the inode block, then it must be
* marked dirty so that its will eventually get written back in
* its correct form.
*/
if (hadchanges)
bdirty(bp);
/*
* Process any allocdirects that completed during the update.
*/
if ((adp = TAILQ_FIRST(&inodedep->id_inoupdt)) != NULL)
handle_allocdirect_partdone(adp);
/*
* Process deallocations that were held pending until the
* inode had been written to disk. Freeing of the inode
* is delayed until after all blocks have been freed to
* avoid creation of new <vfsid, inum, lbn> triples
* before the old ones have been deleted.
*/
filefree = NULL;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
while ((wk = LIST_FIRST(&inodedep->id_bufwait)) != NULL) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
WORKLIST_REMOVE(wk);
switch (wk->wk_type) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_FREEFILE:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* We defer adding filefree to the worklist until
* all other additions have been made to ensure
* that it will be done after all the old blocks
* have been freed.
*/
if (filefree != NULL)
panic("handle_written_inodeblock: filefree");
filefree = wk;
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_MKDIR:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
handle_written_mkdir(WK_MKDIR(wk), MKDIR_PARENT);
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_DIRADD:
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
diradd_inode_written(WK_DIRADD(wk), inodedep);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
continue;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_FREEBLKS:
case D_FREEFRAG:
case D_DIRREM:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
add_to_worklist(wk);
continue;
default:
panic("handle_written_inodeblock: Unknown type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
}
}
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
if (filefree != NULL) {
if (free_inodedep(inodedep) == 0)
panic("handle_written_inodeblock: live inodedep");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
add_to_worklist(filefree);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
return (0);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* If no outstanding dependencies, free it.
*/
if (free_inodedep(inodedep) || TAILQ_FIRST(&inodedep->id_inoupdt) == 0)
return (0);
return (hadchanges);
}
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
/*
* Process a diradd entry after its dependent inode has been written.
* This routine must be called with splbio interrupts blocked.
*/
static void
diradd_inode_written(dap, inodedep)
struct diradd *dap;
struct inodedep *inodedep;
{
struct pagedep *pagedep;
dap->da_state |= COMPLETE;
if ((dap->da_state & ALLCOMPLETE) == ALLCOMPLETE) {
if (dap->da_state & DIRCHG)
pagedep = dap->da_previous->dm_pagedep;
else
pagedep = dap->da_pagedep;
LIST_REMOVE(dap, da_pdlist);
LIST_INSERT_HEAD(&pagedep->pd_pendinghd, dap, da_pdlist);
}
WORKLIST_INSERT(&inodedep->id_pendinghd, &dap->da_list);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Handle the completion of a mkdir dependency.
*/
static void
handle_written_mkdir(mkdir, type)
struct mkdir *mkdir;
int type;
{
struct diradd *dap;
struct pagedep *pagedep;
if (mkdir->md_state != type)
panic("handle_written_mkdir: bad type");
dap = mkdir->md_diradd;
dap->da_state &= ~type;
if ((dap->da_state & (MKDIR_PARENT | MKDIR_BODY)) == 0)
dap->da_state |= DEPCOMPLETE;
if ((dap->da_state & ALLCOMPLETE) == ALLCOMPLETE) {
if (dap->da_state & DIRCHG)
pagedep = dap->da_previous->dm_pagedep;
else
pagedep = dap->da_pagedep;
LIST_REMOVE(dap, da_pdlist);
LIST_INSERT_HEAD(&pagedep->pd_pendinghd, dap, da_pdlist);
}
LIST_REMOVE(mkdir, md_mkdirs);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(mkdir, D_MKDIR);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Called from within softdep_disk_write_complete above.
* A write operation was just completed. Removed inodes can
* now be freed and associated block pointers may be committed.
* Note that this routine is always called from interrupt level
* with further splbio interrupts blocked.
*/
static int
handle_written_filepage(pagedep, bp)
struct pagedep *pagedep;
struct buf *bp; /* buffer containing the written page */
{
struct dirrem *dirrem;
struct diradd *dap, *nextdap;
struct direct *ep;
int i, chgs;
if ((pagedep->pd_state & IOSTARTED) == 0)
panic("handle_written_filepage: not started");
pagedep->pd_state &= ~IOSTARTED;
/*
* Process any directory removals that have been committed.
*/
while ((dirrem = LIST_FIRST(&pagedep->pd_dirremhd)) != NULL) {
LIST_REMOVE(dirrem, dm_next);
dirrem->dm_dirinum = pagedep->pd_ino;
add_to_worklist(&dirrem->dm_list);
}
/*
* Free any directory additions that have been committed.
*/
while ((dap = LIST_FIRST(&pagedep->pd_pendinghd)) != NULL)
free_diradd(dap);
/*
* Uncommitted directory entries must be restored.
*/
for (chgs = 0, i = 0; i < DAHASHSZ; i++) {
for (dap = LIST_FIRST(&pagedep->pd_diraddhd[i]); dap;
dap = nextdap) {
nextdap = LIST_NEXT(dap, da_pdlist);
if (dap->da_state & ATTACHED)
panic("handle_written_filepage: attached");
ep = (struct direct *)
((char *)bp->b_data + dap->da_offset);
ep->d_ino = dap->da_newinum;
dap->da_state &= ~UNDONE;
dap->da_state |= ATTACHED;
chgs = 1;
/*
* If the inode referenced by the directory has
* been written out, then the dependency can be
* moved to the pending list.
*/
if ((dap->da_state & ALLCOMPLETE) == ALLCOMPLETE) {
LIST_REMOVE(dap, da_pdlist);
LIST_INSERT_HEAD(&pagedep->pd_pendinghd, dap,
da_pdlist);
}
}
}
/*
* If there were any rollbacks in the directory, then it must be
* marked dirty so that its will eventually get written back in
* its correct form.
*/
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
if (chgs) {
if ((bp->b_flags & B_DELWRI) == 0)
stat_dir_entry++;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bdirty(bp);
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* If no dependencies remain, the pagedep will be freed.
* Otherwise it will remain to update the page before it
* is written back to disk.
*/
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
if (LIST_FIRST(&pagedep->pd_pendinghd) == 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
for (i = 0; i < DAHASHSZ; i++)
if (LIST_FIRST(&pagedep->pd_diraddhd[i]) != NULL)
break;
if (i == DAHASHSZ) {
LIST_REMOVE(pagedep, pd_hash);
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
WORKITEM_FREE(pagedep, D_PAGEDEP);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (0);
}
}
return (1);
}
/*
* Writing back in-core inode structures.
*
* The file system only accesses an inode's contents when it occupies an
* "in-core" inode structure. These "in-core" structures are separate from
* the page frames used to cache inode blocks. Only the latter are
* transferred to/from the disk. So, when the updated contents of the
* "in-core" inode structure are copied to the corresponding in-memory inode
* block, the dependencies are also transferred. The following procedure is
* called when copying a dirty "in-core" inode to a cached inode block.
*/
/*
* Called when an inode is loaded from disk. If the effective link count
* differed from the actual link count when it was last flushed, then we
* need to ensure that the correct effective link count is put back.
*/
void
softdep_load_inodeblock(ip)
struct inode *ip; /* the "in_core" copy of the inode */
{
struct inodedep *inodedep;
/*
* Check for alternate nlink count.
*/
ip->i_effnlink = ip->i_nlink;
ACQUIRE_LOCK(&lk);
if (inodedep_lookup(ip->i_fs, ip->i_number, 0, &inodedep) == 0) {
FREE_LOCK(&lk);
return;
}
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
ip->i_effnlink -= inodedep->id_nlinkdelta;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
}
/*
* This routine is called just before the "in-core" inode
* information is to be copied to the in-memory inode block.
* Recall that an inode block contains several inodes. If
* the force flag is set, then the dependencies will be
* cleared so that the update can always be made. Note that
* the buffer is locked when this routine is called, so we
* will never be in the middle of writing the inode block
* to disk.
*/
void
softdep_update_inodeblock(ip, bp, waitfor)
struct inode *ip; /* the "in_core" copy of the inode */
struct buf *bp; /* the buffer containing the inode block */
UFS_UPDATE() takes a boolean `waitfor' arg, so don't pass it the value MNT_WAIT when we mean boolean `true' or check for that value not being passed. There was no problem in practice because MNT_WAIT had the magic value of 1.
1999-01-06 18:18:06 +00:00
int waitfor; /* nonzero => update must be allowed */
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
{
struct inodedep *inodedep;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
struct worklist *wk;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
int error, gotit;
/*
* If the effective link count is not equal to the actual link
* count, then we must track the difference in an inodedep while
* the inode is (potentially) tossed out of the cache. Otherwise,
* if there is no existing inodedep, then there are no dependencies
* to track.
*/
ACQUIRE_LOCK(&lk);
Better bounding on softdep_flushfiles; other minor tweeks to checks.
2000-01-17 06:35:11 +00:00
if (inodedep_lookup(ip->i_fs, ip->i_number, 0, &inodedep) == 0) {
if (ip->i_effnlink != ip->i_nlink)
panic("softdep_update_inodeblock: bad link count");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
return;
}
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
if (inodedep->id_nlinkdelta != ip->i_nlink - ip->i_effnlink)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
panic("softdep_update_inodeblock: bad delta");
/*
* Changes have been initiated. Anything depending on these
* changes cannot occur until this inode has been written.
*/
inodedep->id_state &= ~COMPLETE;
if ((inodedep->id_state & ONWORKLIST) == 0)
WORKLIST_INSERT(&bp->b_dep, &inodedep->id_list);
/*
* Any new dependencies associated with the incore inode must
* now be moved to the list associated with the buffer holding
* the in-memory copy of the inode. Once merged process any
* allocdirects that are completed by the merger.
*/
merge_inode_lists(inodedep);
if (TAILQ_FIRST(&inodedep->id_inoupdt) != NULL)
handle_allocdirect_partdone(TAILQ_FIRST(&inodedep->id_inoupdt));
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
/*
* Now that the inode has been pushed into the buffer, the
* operations dependent on the inode being written to disk
* can be moved to the id_bufwait so that they will be
* processed when the buffer I/O completes.
*/
while ((wk = LIST_FIRST(&inodedep->id_inowait)) != NULL) {
WORKLIST_REMOVE(wk);
WORKLIST_INSERT(&inodedep->id_bufwait, wk);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Newly allocated inodes cannot be written until the bitmap
* that allocates them have been written (indicated by
* DEPCOMPLETE being set in id_state). If we are doing a
* forced sync (e.g., an fsync on a file), we force the bitmap
* to be written so that the update can be done.
*/
if ((inodedep->id_state & DEPCOMPLETE) != 0 || waitfor == 0) {
FREE_LOCK(&lk);
return;
}
gotit = getdirtybuf(&inodedep->id_buf, MNT_WAIT);
FREE_LOCK(&lk);
Add a vnode argument to VOP_BWRITE to get rid of the last vnode operator special case. Delete special case code from vnode_if.sh, vnode_if.src, umap_vnops.c, and null_vnops.c.
1999-06-16 23:27:55 +00:00
if (gotit &&
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
(error = BUF_WRITE(inodedep->id_buf)) != 0)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_error("softdep_update_inodeblock: bwrite", error);
if ((inodedep->id_state & DEPCOMPLETE) == 0)
panic("softdep_update_inodeblock: update failed");
}
/*
* Merge the new inode dependency list (id_newinoupdt) into the old
* inode dependency list (id_inoupdt). This routine must be called
* with splbio interrupts blocked.
*/
static void
merge_inode_lists(inodedep)
struct inodedep *inodedep;
{
struct allocdirect *listadp, *newadp;
newadp = TAILQ_FIRST(&inodedep->id_newinoupdt);
for (listadp = TAILQ_FIRST(&inodedep->id_inoupdt); listadp && newadp;) {
if (listadp->ad_lbn < newadp->ad_lbn) {
listadp = TAILQ_NEXT(listadp, ad_next);
continue;
}
TAILQ_REMOVE(&inodedep->id_newinoupdt, newadp, ad_next);
TAILQ_INSERT_BEFORE(listadp, newadp, ad_next);
if (listadp->ad_lbn == newadp->ad_lbn) {
allocdirect_merge(&inodedep->id_inoupdt, newadp,
listadp);
listadp = newadp;
}
newadp = TAILQ_FIRST(&inodedep->id_newinoupdt);
}
while ((newadp = TAILQ_FIRST(&inodedep->id_newinoupdt)) != NULL) {
TAILQ_REMOVE(&inodedep->id_newinoupdt, newadp, ad_next);
TAILQ_INSERT_TAIL(&inodedep->id_inoupdt, newadp, ad_next);
}
}
/*
* If we are doing an fsync, then we must ensure that any directory
* entries for the inode have been written after the inode gets to disk.
*/
ARGH! I have too many source trees :-( Fix prototype errors in last commit.
2000-06-16 13:00:33 +00:00
int
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_fsync(vp)
struct vnode *vp; /* the "in_core" copy of the inode */
{
struct inodedep *inodedep;
struct pagedep *pagedep;
struct worklist *wk;
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
struct diradd *dap;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct mount *mnt;
struct vnode *pvp;
struct inode *ip;
struct buf *bp;
struct fs *fs;
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
struct proc *p = CURPROC; /* XXX */
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
int error, flushparent;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ino_t parentino;
ufs_lbn_t lbn;
ip = VTOI(vp);
fs = ip->i_fs;
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
ACQUIRE_LOCK(&lk);
if (inodedep_lookup(fs, ip->i_number, 0, &inodedep) == 0) {
FREE_LOCK(&lk);
return (0);
}
if (LIST_FIRST(&inodedep->id_inowait) != NULL ||
LIST_FIRST(&inodedep->id_bufwait) != NULL ||
TAILQ_FIRST(&inodedep->id_inoupdt) != NULL ||
TAILQ_FIRST(&inodedep->id_newinoupdt) != NULL)
panic("softdep_fsync: pending ops");
for (error = 0, flushparent = 0; ; ) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if ((wk = LIST_FIRST(&inodedep->id_pendinghd)) == NULL)
break;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
if (wk->wk_type != D_DIRADD)
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
panic("softdep_fsync: Unexpected type %s",
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
TYPENAME(wk->wk_type));
dap = WK_DIRADD(wk);
/*
* Flush our parent if this directory entry
* has a MKDIR_PARENT dependency.
*/
if (dap->da_state & DIRCHG)
pagedep = dap->da_previous->dm_pagedep;
else
pagedep = dap->da_pagedep;
mnt = pagedep->pd_mnt;
parentino = pagedep->pd_ino;
lbn = pagedep->pd_lbn;
if ((dap->da_state & (MKDIR_BODY | COMPLETE)) != COMPLETE)
panic("softdep_fsync: dirty");
flushparent = dap->da_state & MKDIR_PARENT;
/*
* If we are being fsync'ed as part of vgone'ing this vnode,
* then we will not be able to release and recover the
* vnode below, so we just have to give up on writing its
* directory entry out. It will eventually be written, just
* not now, but then the user was not asking to have it
* written, so we are not breaking any promises.
*/
if (vp->v_flag & VXLOCK)
break;
/*
* We prevent deadlock by always fetching inodes from the
* root, moving down the directory tree. Thus, when fetching
* our parent directory, we must unlock ourselves before
* requesting the lock on our parent. See the comment in
* ufs_lookup for details on possible races.
*/
FREE_LOCK(&lk);
VOP_UNLOCK(vp, 0, p);
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
error = VFS_VGET(mnt, parentino, &pvp);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
vn_lock(vp, LK_EXCLUSIVE | LK_RETRY, p);
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
if (error != 0)
return (error);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (flushparent) {
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
if ((error = UFS_UPDATE(pvp, 1)) != 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
vput(pvp);
return (error);
}
}
/*
* Flush directory page containing the inode's name.
*/
error = bread(pvp, lbn, blksize(fs, VTOI(pvp), lbn), p->p_ucred,
&bp);
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
if (error == 0)
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
error = BUF_WRITE(bp);
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
vput(pvp);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (error != 0)
return (error);
Reorganize softdep_fsync so that it only does the inode-is-flushed check before the inode is unlocked while grabbing its parent directory. Once it is unlocked, other operations may slip in that could make the inode-is-flushed check fail. Allowing other writes to the inode before returning from fsync does not break the semantics of fsync since we have flushed everything that was dirty at the time of the fsync call.
2000-01-09 23:14:57 +00:00
ACQUIRE_LOCK(&lk);
if (inodedep_lookup(fs, ip->i_number, 0, &inodedep) == 0)
break;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
FREE_LOCK(&lk);
return (0);
}
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
/*
* Flush all the dirty bitmaps associated with the block device
* before flushing the rest of the dirty blocks so as to reduce
* the number of dependencies that will have to be rolled back.
*/
void
softdep_fsync_mountdev(vp)
struct vnode *vp;
{
struct buf *bp, *nbp;
struct worklist *wk;
Give vn_isdisk() a second argument where it can return a suitable errno. Suggested by: bde
2000-01-10 12:04:27 +00:00
if (!vn_isdisk(vp, NULL))
Convert various pieces of code to use vn_isdisk() rather than checking for vp->v_type == VBLK. In ccd: we don't need to call VOP_GETATTR to find the type of a vnode. Reviewed by: sos
1999-11-22 10:33:55 +00:00
panic("softdep_fsync_mountdev: vnode not a disk");
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
ACQUIRE_LOCK(&lk);
for (bp = TAILQ_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
nbp = TAILQ_NEXT(bp, b_vnbufs);
/*
* If it is already scheduled, skip to the next buffer.
*/
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_NOWAIT))
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
continue;
if ((bp->b_flags & B_DELWRI) == 0)
panic("softdep_fsync_mountdev: not dirty");
/*
* We are only interested in bitmaps with outstanding
* dependencies.
*/
if ((wk = LIST_FIRST(&bp->b_dep)) == NULL ||
When writing out bitmap buffers, need to skip over ones that already have a write in progress. Otherwise one can get in an infinite loop trying to get them all flushed. Submitted by: Matthew Dillon <dillon@apollo.backplane.com>
2000-01-30 20:32:59 +00:00
wk->wk_type != D_BMSAFEMAP ||
(bp->b_xflags & BX_BKGRDINPROG)) {
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
BUF_UNLOCK(bp);
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
continue;
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
}
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
bremfree(bp);
FREE_LOCK(&lk);
(void) bawrite(bp);
ACQUIRE_LOCK(&lk);
/*
* Since we may have slept during the I/O, we need
* to start from a known point.
*/
nbp = TAILQ_FIRST(&vp->v_dirtyblkhd);
}
drain_output(vp, 1);
FREE_LOCK(&lk);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* This routine is called when we are trying to synchronously flush a
* file. This routine must eliminate any filesystem metadata dependencies
* so that the syncing routine can succeed by pushing the dirty blocks
* associated with the file. If any I/O errors occur, they are returned.
*/
int
softdep_sync_metadata(ap)
struct vop_fsync_args /* {
struct vnode *a_vp;
struct ucred *a_cred;
int a_waitfor;
struct proc *a_p;
} */ *ap;
{
struct vnode *vp = ap->a_vp;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
struct pagedep *pagedep;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct allocdirect *adp;
struct allocindir *aip;
struct buf *bp, *nbp;
struct worklist *wk;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
int i, error, waitfor;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Check whether this vnode is involved in a filesystem
* that is doing soft dependency processing.
*/
Give vn_isdisk() a second argument where it can return a suitable errno. Suggested by: bde
2000-01-10 12:04:27 +00:00
if (!vn_isdisk(vp, NULL)) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (!DOINGSOFTDEP(vp))
return (0);
} else
Blow away the v_specmountpoint define, replacing it with what it was defined as (rdev->si_mountpoint)
2000-10-09 17:31:39 +00:00
if (vp->v_rdev->si_mountpoint == NULL ||
(vp->v_rdev->si_mountpoint->mnt_flag & MNT_SOFTDEP) == 0)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (0);
/*
* Ensure that any direct block dependencies have been cleared.
*/
ACQUIRE_LOCK(&lk);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
if ((error = flush_inodedep_deps(VTOI(vp)->i_fs, VTOI(vp)->i_number))) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
return (error);
}
/*
* For most files, the only metadata dependencies are the
* cylinder group maps that allocate their inode or blocks.
* The block allocation dependencies can be found by traversing
* the dependency lists for any buffers that remain on their
* dirty buffer list. The inode allocation dependency will
* be resolved when the inode is updated with MNT_WAIT.
* This work is done in two passes. The first pass grabs most
* of the buffers and begins asynchronously writing them. The
* only way to wait for these asynchronous writes is to sleep
* on the filesystem vnode which may stay busy for a long time
* if the filesystem is active. So, instead, we make a second
* pass over the dependencies blocking on each write. In the
* usual case we will be blocking against a write that we
* initiated, so when it is done the dependency will have been
* resolved. Thus the second pass is expected to end quickly.
*/
waitfor = MNT_NOWAIT;
top:
Change dirty block list handling to use TAILQ macros.
1998-10-31 15:33:32 +00:00
if (getdirtybuf(&TAILQ_FIRST(&vp->v_dirtyblkhd), MNT_WAIT) == 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
return (0);
}
Change dirty block list handling to use TAILQ macros.
1998-10-31 15:33:32 +00:00
bp = TAILQ_FIRST(&vp->v_dirtyblkhd);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
loop:
/*
* As we hold the buffer locked, none of its dependencies
* will disappear.
*/
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(wk, &bp->b_dep, wk_list) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
switch (wk->wk_type) {
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_ALLOCDIRECT:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
adp = WK_ALLOCDIRECT(wk);
if (adp->ad_state & DEPCOMPLETE)
break;
nbp = adp->ad_buf;
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
if (getdirtybuf(&nbp, waitfor) == 0)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
break;
FREE_LOCK(&lk);
if (waitfor == MNT_NOWAIT) {
bawrite(nbp);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
} else if ((error = BUF_WRITE(nbp)) != 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bawrite(bp);
return (error);
}
ACQUIRE_LOCK(&lk);
break;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_ALLOCINDIR:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
aip = WK_ALLOCINDIR(wk);
if (aip->ai_state & DEPCOMPLETE)
break;
nbp = aip->ai_buf;
if (getdirtybuf(&nbp, waitfor) == 0)
break;
FREE_LOCK(&lk);
if (waitfor == MNT_NOWAIT) {
bawrite(nbp);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
} else if ((error = BUF_WRITE(nbp)) != 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bawrite(bp);
return (error);
}
ACQUIRE_LOCK(&lk);
break;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INDIRDEP:
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
restart:
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(aip, &WK_INDIRDEP(wk)->ir_deplisthd, ai_next) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (aip->ai_state & DEPCOMPLETE)
continue;
nbp = aip->ai_buf;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
if (getdirtybuf(&nbp, MNT_WAIT) == 0)
goto restart;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
if ((error = BUF_WRITE(nbp)) != 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bawrite(bp);
return (error);
}
ACQUIRE_LOCK(&lk);
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
goto restart;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
break;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_INODEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if ((error = flush_inodedep_deps(WK_INODEDEP(wk)->id_fs,
WK_INODEDEP(wk)->id_ino)) != 0) {
FREE_LOCK(&lk);
bawrite(bp);
return (error);
}
break;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
case D_PAGEDEP:
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* We are trying to sync a directory that may
* have dependencies on both its own metadata
* and/or dependencies on the inodes of any
* recently allocated files. We walk its diradd
* lists pushing out the associated inode.
*/
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
pagedep = WK_PAGEDEP(wk);
for (i = 0; i < DAHASHSZ; i++) {
if (LIST_FIRST(&pagedep->pd_diraddhd[i]) == 0)
continue;
Cosmetic changes to make it compile without errors in gcc -Wall
1999-05-22 04:43:04 +00:00
if ((error =
flush_pagedep_deps(vp, pagedep->pd_mnt,
&pagedep->pd_diraddhd[i]))) {
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
FREE_LOCK(&lk);
bawrite(bp);
return (error);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
break;
Ensure that softdep_sync_metadata can handle bmsafemap and mkdir entries if they ever arise (which should not happen as softdep_sync_metadata is currently used).
1999-03-02 00:19:47 +00:00
case D_MKDIR:
/*
* This case should never happen if the vnode has
* been properly sync'ed. However, if this function
* is used at a place where the vnode has not yet
* been sync'ed, this dependency can show up. So,
* rather than panic, just flush it.
*/
nbp = WK_MKDIR(wk)->md_buf;
if (getdirtybuf(&nbp, waitfor) == 0)
break;
FREE_LOCK(&lk);
if (waitfor == MNT_NOWAIT) {
bawrite(nbp);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
} else if ((error = BUF_WRITE(nbp)) != 0) {
Ensure that softdep_sync_metadata can handle bmsafemap and mkdir entries if they ever arise (which should not happen as softdep_sync_metadata is currently used).
1999-03-02 00:19:47 +00:00
bawrite(bp);
return (error);
}
ACQUIRE_LOCK(&lk);
break;
case D_BMSAFEMAP:
/*
* This case should never happen if the vnode has
* been properly sync'ed. However, if this function
* is used at a place where the vnode has not yet
* been sync'ed, this dependency can show up. So,
* rather than panic, just flush it.
*/
nbp = WK_BMSAFEMAP(wk)->sm_buf;
if (getdirtybuf(&nbp, waitfor) == 0)
break;
FREE_LOCK(&lk);
if (waitfor == MNT_NOWAIT) {
bawrite(nbp);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
} else if ((error = BUF_WRITE(nbp)) != 0) {
Ensure that softdep_sync_metadata can handle bmsafemap and mkdir entries if they ever arise (which should not happen as softdep_sync_metadata is currently used).
1999-03-02 00:19:47 +00:00
bawrite(bp);
return (error);
}
ACQUIRE_LOCK(&lk);
break;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
default:
panic("softdep_sync_metadata: Unknown type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
}
}
Change dirty block list handling to use TAILQ macros.
1998-10-31 15:33:32 +00:00
(void) getdirtybuf(&TAILQ_NEXT(bp, b_vnbufs), MNT_WAIT);
nbp = TAILQ_NEXT(bp, b_vnbufs);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
bawrite(bp);
ACQUIRE_LOCK(&lk);
if (nbp != NULL) {
bp = nbp;
goto loop;
}
/*
* We must wait for any I/O in progress to finish so that
* all potential buffers on the dirty list will be visible.
* Once they are all there, proceed with the second pass
* which will wait for the I/O as per above.
*/
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
drain_output(vp, 1);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* The brief unlock is to allow any pent up dependency
* processing to be done.
*/
if (waitfor == MNT_NOWAIT) {
waitfor = MNT_WAIT;
FREE_LOCK(&lk);
ACQUIRE_LOCK(&lk);
goto top;
}
/*
* If we have managed to get rid of all the dirty buffers,
* then we are done. For certain directories and block
* devices, we may need to do further work.
*/
Change dirty block list handling to use TAILQ macros.
1998-10-31 15:33:32 +00:00
if (TAILQ_FIRST(&vp->v_dirtyblkhd) == NULL) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
return (0);
}
FREE_LOCK(&lk);
/*
* If we are trying to sync a block device, some of its buffers may
* contain metadata that cannot be written until the contents of some
* partially written files have been written to disk. The only easy
* way to accomplish this is to sync the entire filesystem (luckily
* this happens rarely).
*/
Give vn_isdisk() a second argument where it can return a suitable errno. Suggested by: bde
2000-01-10 12:04:27 +00:00
if (vn_isdisk(vp, NULL) &&
Blow away the v_specmountpoint define, replacing it with what it was defined as (rdev->si_mountpoint)
2000-10-09 17:31:39 +00:00
vp->v_rdev->si_mountpoint && !VOP_ISLOCKED(vp, NULL) &&
(error = VFS_SYNC(vp->v_rdev->si_mountpoint, MNT_WAIT, ap->a_cred,
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ap->a_p)) != 0)
return (error);
return (0);
}
/*
* Flush the dependencies associated with an inodedep.
* Called with splbio blocked.
*/
static int
flush_inodedep_deps(fs, ino)
struct fs *fs;
ino_t ino;
{
struct inodedep *inodedep;
struct allocdirect *adp;
int error, waitfor;
struct buf *bp;
/*
* This work is done in two passes. The first pass grabs most
* of the buffers and begins asynchronously writing them. The
* only way to wait for these asynchronous writes is to sleep
* on the filesystem vnode which may stay busy for a long time
* if the filesystem is active. So, instead, we make a second
* pass over the dependencies blocking on each write. In the
* usual case we will be blocking against a write that we
* initiated, so when it is done the dependency will have been
* resolved. Thus the second pass is expected to end quickly.
* We give a brief window at the top of the loop to allow
* any pending I/O to complete.
*/
for (waitfor = MNT_NOWAIT; ; ) {
FREE_LOCK(&lk);
ACQUIRE_LOCK(&lk);
if (inodedep_lookup(fs, ino, 0, &inodedep) == 0)
return (0);
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
TAILQ_FOREACH(adp, &inodedep->id_inoupdt, ad_next) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (adp->ad_state & DEPCOMPLETE)
continue;
bp = adp->ad_buf;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
if (getdirtybuf(&bp, waitfor) == 0) {
if (waitfor == MNT_NOWAIT)
continue;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
break;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
if (waitfor == MNT_NOWAIT) {
bawrite(bp);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
} else if ((error = BUF_WRITE(bp)) != 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ACQUIRE_LOCK(&lk);
return (error);
}
ACQUIRE_LOCK(&lk);
break;
}
if (adp != NULL)
continue;
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
TAILQ_FOREACH(adp, &inodedep->id_newinoupdt, ad_next) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (adp->ad_state & DEPCOMPLETE)
continue;
bp = adp->ad_buf;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
if (getdirtybuf(&bp, waitfor) == 0) {
if (waitfor == MNT_NOWAIT)
continue;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
break;
Merge in Kirk's changes to stop softupdates from hogging all of memory.
1998-05-19 21:45:53 +00:00
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
if (waitfor == MNT_NOWAIT) {
bawrite(bp);
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
} else if ((error = BUF_WRITE(bp)) != 0) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ACQUIRE_LOCK(&lk);
return (error);
}
ACQUIRE_LOCK(&lk);
break;
}
if (adp != NULL)
continue;
/*
* If pass2, we are done, otherwise do pass 2.
*/
if (waitfor == MNT_WAIT)
break;
waitfor = MNT_WAIT;
}
/*
* Try freeing inodedep in case all dependencies have been removed.
*/
if (inodedep_lookup(fs, ino, 0, &inodedep) != 0)
(void) free_inodedep(inodedep);
return (0);
}
/*
* Eliminate a pagedep dependency by flushing out all its diradd dependencies.
* Called with splbio blocked.
*/
static int
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
flush_pagedep_deps(pvp, mp, diraddhdp)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct vnode *pvp;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
struct mount *mp;
struct diraddhd *diraddhdp;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
{
Merge with Kirk's version as of Feb 20 His version 9.23 == our version 1.5 of ffs_softdep.c His version 9.5 == our version 1.4 of softdep.c
1998-05-19 22:54:53 +00:00
struct proc *p = CURPROC; /* XXX */
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
struct inodedep *inodedep;
struct ufsmount *ump;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
struct diradd *dap;
struct vnode *vp;
First published FreeBSD version of soft updates Feb 5.
1998-05-19 20:18:42 +00:00
int gotit, error = 0;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
struct buf *bp;
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ino_t inum;
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
ump = VFSTOUFS(mp);
while ((dap = LIST_FIRST(diraddhdp)) != NULL) {
/*
* Flush ourselves if this directory entry
* has a MKDIR_PARENT dependency.
*/
if (dap->da_state & MKDIR_PARENT) {
FREE_LOCK(&lk);
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
if ((error = UFS_UPDATE(pvp, 1)) != 0)
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
break;
ACQUIRE_LOCK(&lk);
/*
* If that cleared dependencies, go on to next.
*/
if (dap != LIST_FIRST(diraddhdp))
continue;
if (dap->da_state & MKDIR_PARENT)
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
panic("flush_pagedep_deps: MKDIR_PARENT");
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
}
/*
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
* A newly allocated directory must have its "." and
* ".." entries written out before its name can be
* committed in its parent. We do not want or need
* the full semantics of a synchronous VOP_FSYNC as
* that may end up here again, once for each directory
* level in the filesystem. Instead, we push the blocks
* and wait for them to clear. We have to fsync twice
* because the first call may choose to defer blocks
* that still have dependencies, but deferral will
* happen at most once.
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
*/
inum = dap->da_newinum;
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
if (dap->da_state & MKDIR_BODY) {
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
FREE_LOCK(&lk);
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
if ((error = VFS_VGET(mp, inum, &vp)) != 0)
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
break;
A panic occurs during an fsync when a dirty block associated with a vnode has not been written (which would clear certain of its dependencies). The problems arises because fsync with MNT_NOWAIT no longer pushes all the dirty blocks associated with a vnode. It skips those that require rollbacks, since they will just get instantly dirty again. Such skipped blocks are marked so that they will not be skipped a second time (otherwise circular dependencies would never clear). So, we fsync twice to ensure that everything will be written at least once.
2000-01-13 07:17:39 +00:00
if ((error=VOP_FSYNC(vp, p->p_ucred, MNT_NOWAIT, p)) ||
(error=VOP_FSYNC(vp, p->p_ucred, MNT_NOWAIT, p))) {
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
vput(vp);
break;
}
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
drain_output(vp, 0);
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
vput(vp);
ACQUIRE_LOCK(&lk);
/*
* If that cleared dependencies, go on to next.
*/
if (dap != LIST_FIRST(diraddhdp))
continue;
if (dap->da_state & MKDIR_BODY)
panic("flush_pagedep_deps: MKDIR_BODY");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
/*
* Flush the inode on which the directory entry depends.
* Having accounted for MKDIR_PARENT and MKDIR_BODY above,
* the only remaining dependency is that the updated inode
* count must get pushed to disk. The inode has already
* been pushed into its inode buffer (via VOP_UPDATE) at
* the time of the reference count change. So we need only
* locate that buffer, ensure that there will be no rollback
* caused by a bitmap dependency, then write the inode buffer.
*/
if (inodedep_lookup(ump->um_fs, inum, 0, &inodedep) == 0)
panic("flush_pagedep_deps: lost inode");
/*
* If the inode still has bitmap dependencies,
* push them to disk.
*/
if ((inodedep->id_state & DEPCOMPLETE) == 0) {
gotit = getdirtybuf(&inodedep->id_buf, MNT_WAIT);
FREE_LOCK(&lk);
if (gotit &&
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
(error = BUF_WRITE(inodedep->id_buf)) != 0)
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
break;
ACQUIRE_LOCK(&lk);
if (dap != LIST_FIRST(diraddhdp))
continue;
}
/*
* If the inode is still sitting in a buffer waiting
* to be written, push it to disk.
*/
FREE_LOCK(&lk);
if ((error = bread(ump->um_devvp,
fsbtodb(ump->um_fs, ino_to_fsba(ump->um_fs, inum)),
(int)ump->um_fs->fs_bsize, NOCRED, &bp)) != 0)
break;
Rename the existing BUF_STRATEGY() to DEV_STRATEGY() substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
2000-03-20 11:29:10 +00:00
if ((error = BUF_WRITE(bp)) != 0)
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
break;
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
ACQUIRE_LOCK(&lk);
Import the next version received from kirk after some FreeBSD feedback.
1998-05-19 20:03:29 +00:00
/*
* If we have failed to get rid of all the dependencies
* then something is seriously wrong.
*/
if (dap == LIST_FIRST(diraddhdp))
Need to reorganize the flushing of directory entry (pagedep) dependencies so that they never try to lock an inode corresponding to ".." as this can lead to deadlock. We observe that any inode with an updated link count is always pushed into its buffer at the time of the link count change, so we do not need to do a VOP_UPDATE, but merely find its buffer and write it. The only time we need to get the inode itself is from the result of a mkdir whose name will never be ".." and hence locking such an inode will never request a lock above us in the filesystem tree. Thanks to Brian Fundakowski Feldman for providing the test program that tickled soft updates into hanging in "inode" sleep. Submitted by: Brian Fundakowski Feldman <green@FreeBSD.org>
2000-01-18 01:30:03 +00:00
panic("flush_pagedep_deps: flush failed");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
if (error)
ACQUIRE_LOCK(&lk);
return (error);
}
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
/*
* A large burst of file addition or deletion activity can drive the
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
* memory load excessively high. First attempt to slow things down
* using the techniques below. If that fails, this routine requests
* the offending operations to fall back to running synchronously
* until the memory load returns to a reasonable level.
*/
int
softdep_slowdown(vp)
struct vnode *vp;
{
int max_softdeps_hard;
max_softdeps_hard = max_softdeps * 11 / 10;
if (num_dirrem < max_softdeps_hard / 2 &&
num_inodedep < max_softdeps_hard)
return (0);
stat_sync_limit_hit += 1;
return (1);
}
/*
* If memory utilization has gotten too high, deliberately slow things
* down and speed up the I/O processing.
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
*/
static int
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
request_cleanup(resource, islocked)
int resource;
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
int islocked;
{
struct proc *p = CURPROC;
/*
* We never hold up the filesystem syncer process.
*/
if (p == filesys_syncer)
return (0);
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
/*
* First check to see if the work list has gotten backlogged.
* If it has, co-opt this process to help clean up two entries.
* Because this process may hold inodes locked, we cannot
* handle any remove requests that might block on a locked
* inode as that could lead to deadlock.
*/
if (num_on_worklist > max_softdeps / 10) {
process_worklist_item(NULL, LK_NOWAIT);
process_worklist_item(NULL, LK_NOWAIT);
stat_worklist_push += 2;
return(0);
}
/*
* Next, we attempt to speed up the syncer process. If that
* is successful, then we allow the process to continue.
*/
if (speedup_syncer())
return(0);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
/*
* If we are resource constrained on inode dependencies, try
* flushing some dirty inodes. Otherwise, we are constrained
* by file deletions, so try accelerating flushes of directories
* with removal dependencies. We would like to do the cleanup
* here, but we probably hold an inode locked at this point and
* that might deadlock against one that we try to clean. So,
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
* the best that we can do is request the syncer daemon to do
* the cleanup for us.
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
*/
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
switch (resource) {
case FLUSH_INODES:
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
stat_ino_limit_push += 1;
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
req_clear_inodedeps += 1;
stat_countp = &stat_ino_limit_hit;
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
break;
case FLUSH_REMOVE:
Add a hook to ffs_fsync to allow soft updates to get first chance at doing a sync on the block device for the filesystem. That allows it to push the bitmap blocks before the inode blocks which greatly reduces the number of inode rollbacks that need to be done.
1999-05-14 01:26:46 +00:00
stat_blk_limit_push += 1;
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
req_clear_remove += 1;
stat_countp = &stat_blk_limit_hit;
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
break;
default:
panic("request_cleanup: unknown type");
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
}
/*
* Hopefully the syncer daemon will catch up and awaken us.
* We wait at most tickdelay before proceeding in any case.
*/
if (islocked == 0)
ACQUIRE_LOCK(&lk);
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
proc_waiting += 1;
if (handle.callout == NULL)
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
handle = timeout(pause_timer, 0, tickdelay > 2 ? tickdelay : 2);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
FREE_LOCK_INTERLOCKED(&lk);
The function request_cleanup() had a tsleep() with PCATCH. It is quite dangerous, since the process may hold locks at the point, and if it is stopped in that tsleep the machine may hang. Because the sleep is so short, the PCATCH is not required here, so it has been removed. For the future, the FreeBSD team needs to decide whether it is still reasonable to stop a process in tsleep, as that may affect any other code that uses PCATCH while holding kernel locks. Submitted by: Dmitrij Tejblum <tejblum@arc.hq.cti.ru> Reviewed by: Kirk McKusick <mckusick@mckusick.com>
1999-12-16 22:02:09 +00:00
(void) tsleep((caddr_t)&proc_waiting, PPAUSE, "softupdate", 0);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
ACQUIRE_LOCK_INTERLOCKED(&lk);
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
proc_waiting -= 1;
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
if (islocked == 0)
FREE_LOCK(&lk);
return (1);
}
/*
Get rid of the global variable rushjob and replace it with a function in kern/vfs_subr.c named speedup_syncer() which handles the speedup request. Change the various clients of rushjob to use the new function.
1999-06-15 23:37:29 +00:00
* Awaken processes pausing in request_cleanup and clear proc_waiting
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
* to indicate that there is no longer a timer running.
*/
void
pause_timer(arg)
void *arg;
{
More aggressively rate limit the growth of soft dependency structures in the face of multiple processes doing massive numbers of filesystem operations. While this patch will work in nearly all situations, there are still some perverse workloads that can overwhelm the system. Detecting and handling these perverse workloads will be the subject of another patch. Reviewed by: Paul Saab <ps@yahoo-inc.com> Obtained from: Ethan Solomita <ethan@geocast.com>
2000-11-20 06:22:39 +00:00
*stat_countp += 1;
wakeup_one(&proc_waiting);
Preventing runaway kernel soft updates memory, take three. Previously, the syncer process was the only process in the system that could process the soft updates background work list. If enough other processes were adding requests to that list, it would eventually grow without bound. Because some of the work list requests require vnodes to be locked, it was not generally safe to let random processes process the work list while they already held vnodes locked. By adding a flag to the work list queue processing function to indicate whether the calling process could safely lock vnodes, it becomes possible to co-opt other processes into helping out with the work list. Now when the worklist gets too large, other processes can safely help out by picking off those work requests that can be handled without locking a vnode, leaving only the small number of requests requiring a vnode lock for the syncer process. With this change, it appears possible to keep even the nastiest workloads under control. Submitted by: Paul Saab <ps@yahoo-inc.com>
2000-12-13 08:30:35 +00:00
if (proc_waiting > 0)
handle = timeout(pause_timer, 0, tickdelay > 2 ? tickdelay : 2);
else
handle.callout = NULL;
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
}
/*
Keep tighter control of removal dependencies by limiting the number of dirrem structure rather than the collaterally created freeblks and freefile structures. Limit the rate of buffer dirtying by the syncer process during periods of intense file removal.
2000-01-09 23:35:38 +00:00
* Flush out a directory with at least one removal dependency in an effort to
* reduce the number of dirrem, freefile, and freeblks dependency structures.
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
*/
static void
clear_remove(p)
struct proc *p;
{
struct pagedep_hashhead *pagedephd;
struct pagedep *pagedep;
static int next = 0;
struct mount *mp;
struct vnode *vp;
int error, cnt;
ino_t ino;
ACQUIRE_LOCK(&lk);
for (cnt = 0; cnt < pagedep_hash; cnt++) {
pagedephd = &pagedep_hashtbl[next++];
if (next >= pagedep_hash)
next = 0;
Another round of the <sys/queue.h> FOREACH transmogriffer. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 16:08:18 +00:00
LIST_FOREACH(pagedep, pagedephd, pd_hash) {
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
if (LIST_FIRST(&pagedep->pd_dirremhd) == NULL)
continue;
mp = pagedep->pd_mnt;
ino = pagedep->pd_ino;
FREE_LOCK(&lk);
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
if (vn_start_write(NULL, &mp, V_NOWAIT) != 0)
continue;
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
if ((error = VFS_VGET(mp, ino, &vp)) != 0) {
softdep_error("clear_remove: vget", error);
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
vn_finished_write(mp);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
return;
}
if ((error = VOP_FSYNC(vp, p->p_ucred, MNT_NOWAIT, p)))
softdep_error("clear_remove: fsync", error);
drain_output(vp, 0);
vput(vp);
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
vn_finished_write(mp);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
return;
}
}
FREE_LOCK(&lk);
}
/*
* Clear out a block of dirty inodes in an effort to reduce
* the number of inodedep dependency structures.
*/
static void
clear_inodedeps(p)
struct proc *p;
{
struct inodedep_hashhead *inodedephd;
struct inodedep *inodedep;
static int next = 0;
struct mount *mp;
struct vnode *vp;
struct fs *fs;
int error, cnt;
ino_t firstino, lastino, ino;
ACQUIRE_LOCK(&lk);
/*
* Pick a random inode dependency to be cleared.
* We will then gather up all the inodes in its block
* that have dependencies and flush them out.
*/
for (cnt = 0; cnt < inodedep_hash; cnt++) {
inodedephd = &inodedep_hashtbl[next++];
if (next >= inodedep_hash)
next = 0;
if ((inodedep = LIST_FIRST(inodedephd)) != NULL)
break;
}
/*
* Ugly code to find mount point given pointer to superblock.
*/
fs = inodedep->id_fs;
struct mountlist and struct mount.mnt_list have no business being a CIRCLEQ. Change them to TAILQ_HEAD and TAILQ_ENTRY respectively. This removes ugly mp != (void*)&mountlist comparisons. Requested by: phk Submitted by: Jake Burkholder jake@checker.org PR: 14967
1999-11-20 10:00:46 +00:00
TAILQ_FOREACH(mp, &mountlist, mnt_list)
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
if ((mp->mnt_flag & MNT_SOFTDEP) && fs == VFSTOUFS(mp)->um_fs)
break;
/*
* Find the last inode in the block with dependencies.
*/
firstino = inodedep->id_ino & ~(INOPB(fs) - 1);
for (lastino = firstino + INOPB(fs) - 1; lastino > firstino; lastino--)
if (inodedep_lookup(fs, lastino, 0, &inodedep) != 0)
break;
/*
* Asynchronously push all but the last inode with dependencies.
* Synchronously push the last inode with dependencies to ensure
* that the inode block gets written to free up the inodedeps.
*/
for (ino = firstino; ino <= lastino; ino++) {
if (inodedep_lookup(fs, ino, 0, &inodedep) == 0)
continue;
FREE_LOCK(&lk);
This patch corrects the first round of panics and hangs reported with the new snapshot code. Update addaliasu to correctly implement the semantics of the old checkalias function. When a device vnode first comes into existence, check to see if an anonymous vnode for the same device was created at boot time by bdevvp(). If so, adopt the bdevvp vnode rather than creating a new vnode for the device. This corrects a problem which caused the kernel to panic when taking a snapshot of the root filesystem. Change the calling convention of vn_write_suspend_wait() to be the same as vn_start_write(). Split out softdep_flushworklist() from softdep_flushfiles() so that it can be used to clear the work queue when suspending filesystem operations. Access to buffers becomes recursive so that snapshots can recursively traverse their indirect blocks using ffs_copyonwrite() when checking for the need for copy on write when flushing one of their own indirect blocks. This eliminates a deadlock between the syncer daemon and a process taking a snapshot. Ensure that softdep_process_worklist() can never block because of a snapshot being taken. This eliminates a problem with buffer starvation. Cleanup change in ffs_sync() which did not synchronously wait when MNT_WAIT was specified. The result was an unclean filesystem panic when doing forcible unmount with heavy filesystem I/O in progress. Return a zero'ed block when reading a block that was not in use at the time that a snapshot was taken. Normally, these blocks should never be read. However, the readahead code will occationally read them which can cause unexpected behavior. Clean up the debugging code that ensures that no blocks be written on a filesystem while it is suspended. Snapshots must explicitly label the blocks that they are writing during the suspension so that they do not cause a `write on suspended filesystem' panic. Reorganize ffs_copyonwrite() to eliminate a deadlock and also to prevent a race condition that would permit the same block to be copied twice. This change eliminates an unexpected soft updates inconsistency in fsck caused by the double allocation. Use bqrelse rather than brelse for buffers that will be needed soon again by the snapshot code. This improves snapshot performance.
2000-07-24 05:28:33 +00:00
if (vn_start_write(NULL, &mp, V_NOWAIT) != 0)
continue;
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
if ((error = VFS_VGET(mp, ino, &vp)) != 0) {
softdep_error("clear_inodedeps: vget", error);
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
vn_finished_write(mp);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
return;
}
if (ino == lastino) {
if ((error = VOP_FSYNC(vp, p->p_ucred, MNT_WAIT, p)))
softdep_error("clear_inodedeps: fsync1", error);
} else {
if ((error = VOP_FSYNC(vp, p->p_ucred, MNT_NOWAIT, p)))
softdep_error("clear_inodedeps: fsync2", error);
drain_output(vp, 0);
}
vput(vp);
Add snapshots to the fast filesystem. Most of the changes support the gating of system calls that cause modifications to the underlying filesystem. The gating can be enabled by any filesystem that needs to consistently suspend operations by adding the vop_stdgetwritemount to their set of vnops. Once gating is enabled, the function vfs_write_suspend stops all new write operations to a filesystem, allows any filesystem modifying system calls already in progress to complete, then sync's the filesystem to disk and returns. The function vfs_write_resume allows the suspended write operations to begin again. Gating is not added by default for all filesystems as for SMP systems it adds two extra locks to such critical kernel paths as the write system call. Thus, gating should only be added as needed. Details on the use and current status of snapshots in FFS can be found in /sys/ufs/ffs/README.snapshot so for brevity and timelyness is not included here. Unless and until you create a snapshot file, these changes should have no effect on your system (famous last words).
2000-07-11 22:07:57 +00:00
vn_finished_write(mp);
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
ACQUIRE_LOCK(&lk);
}
FREE_LOCK(&lk);
}
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
/*
* Function to determine if the buffer has outstanding dependencies
* that will cause a roll-back if the buffer is written. If wantcount
* is set, return number of dependencies, otherwise just yes or no.
*/
static int
softdep_count_dependencies(bp, wantcount)
struct buf *bp;
int wantcount;
{
struct worklist *wk;
struct inodedep *inodedep;
struct indirdep *indirdep;
struct allocindir *aip;
struct pagedep *pagedep;
struct diradd *dap;
int i, retval;
retval = 0;
ACQUIRE_LOCK(&lk);
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
LIST_FOREACH(wk, &bp->b_dep, wk_list) {
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
switch (wk->wk_type) {
case D_INODEDEP:
inodedep = WK_INODEDEP(wk);
if ((inodedep->id_state & DEPCOMPLETE) == 0) {
/* bitmap allocation dependency */
retval += 1;
if (!wantcount)
goto out;
}
if (TAILQ_FIRST(&inodedep->id_inoupdt)) {
/* direct block pointer dependency */
retval += 1;
if (!wantcount)
goto out;
}
continue;
case D_INDIRDEP:
indirdep = WK_INDIRDEP(wk);
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(aip, &indirdep->ir_deplisthd, ai_next) {
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
/* indirect block pointer dependency */
retval += 1;
if (!wantcount)
goto out;
}
continue;
case D_PAGEDEP:
pagedep = WK_PAGEDEP(wk);
for (i = 0; i < DAHASHSZ; i++) {
Use <sys/queue.h> macro API.
2001-02-04 12:37:48 +00:00
LIST_FOREACH(dap, &pagedep->pd_diraddhd[i], da_pdlist) {
Several performance improvements for soft updates have been added: 1) Fastpath deletions. When a file is being deleted, check to see if it was so recently created that its inode has not yet been written to disk. If so, the delete can proceed to immediately free the inode. 2) Background writes: No file or block allocations can be done while the bitmap is being written to disk. To avoid these stalls, the bitmap is copied to another buffer which is written thus leaving the original available for futher allocations. 3) Link count tracking. Constantly track the difference in i_effnlink and i_nlink so that inodes that have had no change other than i_effnlink need not be written. 4) Identify buffers with rollback dependencies so that the buffer flushing daemon can choose to skip over them.
2000-01-10 00:24:24 +00:00
/* directory entry dependency */
retval += 1;
if (!wantcount)
goto out;
}
}
continue;
case D_BMSAFEMAP:
case D_ALLOCDIRECT:
case D_ALLOCINDIR:
case D_MKDIR:
/* never a dependency on these blocks */
continue;
default:
panic("softdep_check_for_rollback: Unexpected type %s",
TYPENAME(wk->wk_type));
/* NOTREACHED */
}
}
out:
FREE_LOCK(&lk);
return retval;
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Acquire exclusive access to a buffer.
* Must be called with splbio blocked.
* Return 1 if buffer was acquired.
*/
static int
getdirtybuf(bpp, waitfor)
struct buf **bpp;
int waitfor;
{
struct buf *bp;
for (;;) {
if ((bp = *bpp) == NULL)
return (0);
Because cylinder group blocks are now written in background, it is no longer sufficient to get a lock on a buffer to know that its write has been completed. We have to first get the lock on the buffer, then check to see if it is doing a background write. If it is doing background write, we have to wait for the background write to finish, then check to see if that fullfilled our dependency, and if not to start another write. Luckily the explanation is longer than the fix.
2000-01-13 07:20:01 +00:00
if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_NOWAIT) == 0) {
if ((bp->b_xflags & BX_BKGRDINPROG) == 0)
break;
BUF_UNLOCK(bp);
if (waitfor != MNT_WAIT)
return (0);
bp->b_xflags |= BX_BKGRDWAIT;
Free the global softupdates lock prior to tsleep() in getdirtybuf(). This seems to be responsible for a bunch of panics where the process sleeps and something else finds softupdates "locked" when it shouldn't be. This commit is unreviewed, but has been a big help here. Previously my boxes would panic pretty much on the first fsync() that wrote something to disk.
2000-01-13 18:48:12 +00:00
FREE_LOCK_INTERLOCKED(&lk);
Because cylinder group blocks are now written in background, it is no longer sufficient to get a lock on a buffer to know that its write has been completed. We have to first get the lock on the buffer, then check to see if it is doing a background write. If it is doing background write, we have to wait for the background write to finish, then check to see if that fullfilled our dependency, and if not to start another write. Luckily the explanation is longer than the fix.
2000-01-13 07:20:01 +00:00
tsleep(&bp->b_xflags, PRIBIO, "getbuf", 0);
Free the global softupdates lock prior to tsleep() in getdirtybuf(). This seems to be responsible for a bunch of panics where the process sleeps and something else finds softupdates "locked" when it shouldn't be. This commit is unreviewed, but has been a big help here. Previously my boxes would panic pretty much on the first fsync() that wrote something to disk.
2000-01-13 18:48:12 +00:00
ACQUIRE_LOCK_INTERLOCKED(&lk);
Because cylinder group blocks are now written in background, it is no longer sufficient to get a lock on a buffer to know that its write has been completed. We have to first get the lock on the buffer, then check to see if it is doing a background write. If it is doing background write, we have to wait for the background write to finish, then check to see if that fullfilled our dependency, and if not to start another write. Luckily the explanation is longer than the fix.
2000-01-13 07:20:01 +00:00
continue;
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
if (waitfor != MNT_WAIT)
return (0);
FREE_LOCK_INTERLOCKED(&lk);
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_SLEEPFAIL) != ENOLCK)
panic("getdirtybuf: inconsistent lock");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
ACQUIRE_LOCK_INTERLOCKED(&lk);
}
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
if ((bp->b_flags & B_DELWRI) == 0) {
BUF_UNLOCK(bp);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
return (0);
Convert buffer locking from using the B_BUSY and B_WANTED flags to using lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
1999-06-26 02:47:16 +00:00
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
bremfree(bp);
return (1);
}
Severe slowdowns have been reported when creating or removing many files at once on a filesystem running soft updates. The root of the problem is that soft updates limits the amount of memory that may be allocated to dependency structures so as to avoid hogging kernel memory. The original algorithm just waited for the disk I/O to catch up and reduce the number of dependencies. This new code takes a much more aggressive approach. Basically there are two resources that routinely hit the limit. Inode dependencies during periods with a high file creation rate and file and block removal dependencies during periods with a high file removal rate. I have attacked these problems from two fronts. When the inode dependency limits are reached, I pick a random inode dependency, UFS_UPDATE it together with all the other dirty inodes contained within its disk block and then write that disk block. This trick usually clears 5-50 inode dependencies in a single disk I/O. For block and file removal dependencies, I pick a random directory page that has at least one remove pending and VOP_FSYNC its directory. That releases all its removal dependencies to the work queue. To further hasten things along, I also immediately start the work queue process rather than waiting for its next one second scheduled run.
1999-05-07 02:26:47 +00:00
/*
* Wait for pending output on a vnode to complete.
* Must be called with vnode locked.
*/
static void
drain_output(vp, islocked)
struct vnode *vp;
int islocked;
{
if (!islocked)
ACQUIRE_LOCK(&lk);
while (vp->v_numoutput) {
vp->v_flag |= VBWAIT;
FREE_LOCK_INTERLOCKED(&lk);
tsleep((caddr_t)&vp->v_numoutput, PRIBIO + 1, "drainvp", 0);
ACQUIRE_LOCK_INTERLOCKED(&lk);
}
if (!islocked)
FREE_LOCK(&lk);
}
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/*
* Called whenever a buffer that is being invalidated or reallocated
* contains dependencies. This should only happen if an I/O error has
* occurred. The routine is called with the buffer locked.
*/
Make static non-exported functions from soft updates.
2000-01-09 22:40:09 +00:00
static void
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_deallocate_dependencies(bp)
struct buf *bp;
{
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
Move B_ERROR flag to b_ioflags and call it BIO_ERROR. (Much of this done by script) Move B_ORDERED flag to b_ioflags and call it BIO_ORDERED. Move b_pblkno and b_iodone_chain to struct bio while we transition, they will be obsoleted once bio structs chain/stack. Add bio_queue field for struct bio aware disksort. Address a lot of stylistic issues brought up by bde.
2000-04-02 15:24:56 +00:00
if ((bp->b_ioflags & BIO_ERROR) == 0)
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
panic("softdep_deallocate_dependencies: dangling deps");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
softdep_error(bp->b_vp->v_mount->mnt_stat.f_mntonname, bp->b_error);
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
panic("softdep_deallocate_dependencies: unrecovered I/O error");
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
/*
* Function to handle asynchronous write errors in the filesystem.
*/
void
softdep_error(func, error)
char *func;
int error;
{
Get rid of random debugging cruft; sync up with latest version.
1999-05-07 05:11:31 +00:00
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
/* XXX should do something better! */
Gutted softdep_deallocate_dependencies and replaced it with a panic. It turns out to not be useful to unwind the dependencies and continue in the face of a fatal error. Also changed the log() to a printf() in softdep_error() so that it will be output in the case of a impending panic. Submitted by: Kirk McKusick <mckusick@mckusick.com>
1999-01-22 09:07:32 +00:00
printf("%s: got error %d while accessing filesystem\n", func, error);
Import the earliest version of the soft update code that I have.
1998-05-19 19:47:22 +00:00
}
Reference in New Issue Copy Permalink