freebsd-nq/sys/sys
Kirk McKusick 3a7053cb60 Prevent large files from monopolizing the system buffers. Keep
track of the number of dirty buffers held by a vnode. When a
bdwrite is done on a buffer, check the existing number of dirty
buffers associated with its vnode. If the number rises above
vfs.dirtybufthresh (currently 90% of vfs.hidirtybuffers), one
of the other (hopefully older) dirty buffers associated with
the vnode is written (using bawrite). In the event that this
approach fails to curb the growth in it the vnode's number of
dirty buffers (due to soft updates rollback dependencies),
the more drastic approach of doing a VOP_FSYNC on the vnode
is used. This code primarily affects very large and actively
written files such as snapshots. This change should eliminate
hanging when taking snapshots or doing background fsck on
very large filesystems.

Hopefully, one day it will be possible to cache filesystem
metadata in the VM cache as is done with file data. As it
stands, only the buffer cache can be used which limits total
metadata storage to about 20Mb no matter how much memory is
available on the system. This rather small memory gets badly
thrashed causing a lot of extra I/O. For example, taking a
snapshot of a 1Tb filesystem minimally requires about 35,000
write operations, but because of the cache thrashing (we only
have about 350 buffers at our disposal) ends up doing about
237,540 I/O's thus taking twenty-five minutes instead of four
if it could run entirely in the cache.

Reported by:	Attila Nagy <bra@fsn.hu>
Sponsored by:   DARPA & NAI Labs.
2003-02-25 06:44:42 +00:00
..
_iovec.h It is bad style to define the same structure in multiple header 2002-12-28 22:17:29 +00:00
_label.h Update licenses and wording: NAI has authorized the removal of clause three 2002-11-04 02:35:46 +00:00
_lock.h Renamed the idempotency identifier to match the file name. 2002-06-07 14:37:09 +00:00
_mutex.h Save 16 bytes per mutex if MUTEX_PROFILING is not defined. 2002-12-29 11:14:41 +00:00
_semaphore.h Add the rest of the kernel support for the sem_ API in kern/uipc_sem.c. 2002-09-19 00:43:32 +00:00
_sigset.h Split 4.x and 5.x signal handling so that we can keep 4.x signal 2002-10-25 19:10:58 +00:00
_timespec.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
_timeval.h Move struct timeval to its own header so that it can be shared between 2002-12-31 04:08:41 +00:00
_types.h Add suseconds_t and useconds_t for storing microseconds. 2002-12-22 22:02:41 +00:00
aac_ioctl.h
acct.h
acl.h Provide prototypes for new ACL system calls to manipulate ACLs "by 2002-12-29 20:30:00 +00:00
agpio.h Document a few acronyms. 2003-02-08 20:42:26 +00:00
aio.h
alq.h - Export the alq daemon thread pointer. 2002-09-26 07:38:56 +00:00
assym.h
ata.h First round off updates/fixes to the ATA driver. 2003-02-20 19:55:45 +00:00
bio.h Implement a bio-taskqueue to reduce number of context switches in 2003-02-11 22:30:26 +00:00
bitstring.h
blist.h Remove all use of the LOG2() macro/inline, undoing some non-optimal cruft 2003-01-11 01:09:51 +00:00
buf.h - Add an interlock argument to BUF_LOCK and BUF_TIMELOCK. 2003-02-25 03:37:48 +00:00
bus_dma.h Implement bus_dmamem_alloc_size() and bus_dmamem_free_size() as 2003-01-29 07:25:27 +00:00
bus.h Bump pnpinfo and location size to 128 bytes. Since these are computed 2003-02-17 19:01:33 +00:00
callout.h
ccdvar.h NO_GEOM cleanup: Convert CCD(4) to be use "struct disk*" instead of "dev_t" 2003-02-21 23:25:43 +00:00
cdefs.h Add the ability to specify a strict C99 environment with the 2002-10-21 20:50:30 +00:00
cdio.h
cdrio.h * Add CDRIOC{READ,WRITE}SPEED ioctls to cd(4). Units are in KB/sec. 2002-10-18 22:03:39 +00:00
chio.h
clist.h
clock.h
condvar.h Part 1 of KSE-III 2002-06-29 17:26:22 +00:00
conf.h Cleanup of the d_mmap_t interface. 2003-02-25 03:21:22 +00:00
cons.h Change the console interface to pass a "struct consdev *" instead of a 2003-02-20 20:54:45 +00:00
consio.h 1. Allow information about current history size be retrieved using ioctl(2); 2002-08-19 16:32:09 +00:00
copyright.h We've had something like this file since 1992, so therefore this file 2003-01-07 04:37:40 +00:00
ctype.h Resolve conflicts arising from the ACPI CA 20021118 import. 2002-11-27 18:09:20 +00:00
dataacq.h
device_port.h
devicestat.h Un-inline the non-trivial "trivial" bio* functions. 2002-09-14 19:34:11 +00:00
digiio.h
dir.h
dirent.h Fix namespace issues by using the relatively new visibility 2002-09-10 18:12:16 +00:00
disk.h NO_GEOM cleanup: 2003-02-21 15:13:26 +00:00
disklabel.h NO_GEOM cleanup: unifdef 2003-01-30 13:12:31 +00:00
diskmbr.h Add NEXTDOSPART (N-Extended-Dos-Partitions) as replacement for the 2003-01-20 11:06:24 +00:00
diskpc98.h Remove NEXTDOSPART. 2003-01-21 14:02:11 +00:00
diskslice.h NO_GEOM cleanup: unifdef 2003-01-30 13:12:31 +00:00
dkstat.h This file is now empty and should no longer be used anywhere. 2003-02-16 14:15:03 +00:00
domain.h
dvdio.h
elf32.h Move the definition of ElfN_Hashelt to common headers. The only platform 2002-05-30 08:32:18 +00:00
elf64.h Move the definition of ElfN_Hashelt to common headers. The only platform 2002-05-30 08:32:18 +00:00
elf_common.h - Call a SHT_REL 'relocation section' rather than 2002-08-11 21:07:50 +00:00
elf_generic.h Infrastructure tweaks to allow having both an Elf32 and an Elf64 executable 2002-07-20 02:56:12 +00:00
endian.h Make this header self-reliant with regard to the types it uses. 2002-04-29 16:58:54 +00:00
errno.h Copyin and copyout are only possible from a process-native thread, 2002-10-07 06:25:26 +00:00
event.h Tweak the definition of the EV_SET macro so that it evaluates each 2003-02-02 19:39:51 +00:00
eventhandler.h - Add an event that is triggered when the system is low on memory. This is 2002-11-21 09:17:56 +00:00
eventvar.h
exec.h Use __CONCAT and __XSTRING when concatenating or stringifying arguments to 2003-01-04 08:17:27 +00:00
extattr.h Prototypes for new link extended attribute calls. Sync to MAC tree. 2002-10-09 21:48:57 +00:00
fbio.h
fcntl.h o Add typedefs for mode_t, off_t, pid_t rather than including 2002-09-17 22:22:50 +00:00
fdcio.h
file.h Do not allow kqueues to be passed via unix domain sockets. 2003-02-15 06:04:55 +00:00
filedesc.h Fix LOR with PROC/filedesc. Introduce fdesc_mtx that will be used as a 2003-02-15 05:52:56 +00:00
filio.h
fnv_hash.h
gmon.h
gpt.h o Include <sys/uuid.h>. This avoids that applications such as 2002-11-10 20:13:58 +00:00
iconv.h Convert GNU variadic macros to the ISO 9X variety. 2002-07-15 13:34:50 +00:00
imgact_aout.h Forward declare struct vnode so that <sys/vnode.h> or some other header 2002-09-05 07:54:03 +00:00
imgact_elf.h - Provide backwards compatibility for kern.fallback_elf_brand. 2003-01-05 03:48:14 +00:00
imgact.h Bring in two sets of changes: 2002-11-05 17:51:56 +00:00
inflate.h
interrupt.h
ioccom.h
ioctl_bt848.h
ioctl_compat.h
ioctl_meteor.h
ioctl.h
ipc.h It is possible for an active aio to prevent shared memory from being 2003-01-13 23:04:32 +00:00
jail.h Include <sys/queue.h> so that this file provides its own namespace 2002-05-06 03:13:08 +00:00
joystick.h Fix typo in the BSD copyright: s/withough/without/ 2002-06-02 20:05:59 +00:00
jumbo.h Remove the advertising clause from the Duke BSD copyright on the 2002-07-06 02:44:15 +00:00
kbio.h
kenv.h
kernel.h Split the global timezone structure into two integer fields to 2003-02-03 19:49:35 +00:00
kerneldump.h Add kernel dump support, based on the ia64 version (which was committed 2002-10-20 17:03:15 +00:00
kobj.h Convert hit and miss counters to unsigned values. Surely negative values 2002-06-10 22:40:26 +00:00
kse.h Add a timeout parameter to kse_release. 2003-02-20 08:18:15 +00:00
kthread.h Some kernel threads try to do significant work, and the default KSTACK_PAGES 2002-10-02 07:44:29 +00:00
ktr.h - Add support for logging KTR via ALQ. This is optional and enabled by the 2002-09-22 07:13:45 +00:00
ktrace.h Overhaul the ktrace subsystem a bit. For the most part, the actual vnode 2002-06-07 05:32:59 +00:00
libkern.h Add an implementation of strdup() to libkern. Allocated memory is of 2003-02-23 14:30:17 +00:00
linedisc.h Cleanup of the d_mmap_t interface. 2003-02-25 03:21:22 +00:00
link_aout.h Repo copy link.h to sys/link_elf.h and sys/link_aout.h since they are 2002-08-22 20:35:23 +00:00
link_elf.h Implement dlinfo() function. 2003-02-13 17:47:44 +00:00
linker_set.h Add a __section(x) macro as well. Use this in linker_set.h. ie: 2002-09-23 06:11:29 +00:00
linker.h Resolve relative relocations in klds before trying to parse the module's 2003-01-21 02:42:44 +00:00
lock.h Forward declare struct thread so that this header doesn't depend on its 2002-08-28 05:14:55 +00:00
lockf.h
lockmgr.h - Add an interlock argument to BUF_LOCK and BUF_TIMELOCK. 2003-02-25 03:37:48 +00:00
mac_policy.h Remove dm_root entry from struct devfs_mount. It's never set, and is 2002-12-09 03:44:28 +00:00
mac.h Remove dm_root entry from struct devfs_mount. It's never set, and is 2002-12-09 03:44:28 +00:00
malloc.h Add an implementation of strdup() to libkern. Allocated memory is of 2003-02-23 14:30:17 +00:00
mbuf.h o Allow "buckets" in mb_alloc to be differently sized (according to 2003-02-20 04:26:58 +00:00
mchain.h Some style fixes. 2003-02-21 16:24:49 +00:00
md4.h
md5.h Oops, this should have been part of my previous commit. 2002-06-24 14:18:39 +00:00
mdioctl.h Add an ioctl to return the unit numbers of the configured md devices. 2003-01-27 07:54:59 +00:00
memrange.h Some BIOSs are using MTRR values that are only documented under NDA 2002-09-15 15:07:55 +00:00
mman.h mlockall() and munlockall() are unimplemented; remove their prototypes. 2002-09-13 18:29:15 +00:00
module.h
mount.h Re-add MNT_ACLS to the list of "updateable" mount flags, per our 2002-12-27 18:20:16 +00:00
mouse.h
msg.h Add const qualifier to data argument for msgsnd. 2003-01-26 20:09:34 +00:00
msgbuf.h Make the msg_size, msg_bufx and msg_bufr memebers of struct msgbuf 2002-11-14 16:11:12 +00:00
mtio.h Make the mtio data structures explicitly sized. 2002-05-14 07:30:13 +00:00
mutex.h Allow lint-like tools to override DROP_GIANT and friends. 2002-12-28 22:29:10 +00:00
namei.h Remove four members of struct nameidata which have been commented 2003-01-13 08:49:36 +00:00
nlist_aout.h Pad after "char *n_name;" in the !_AOUT_INCLUDE_ case so that struct nlist 2002-09-05 08:03:02 +00:00
param.h Bump __FreeBSD_version to mark the fact that <sys/dkstat.h> should no longer 2003-02-19 06:32:52 +00:00
pciio.h
pcpu.h Unbreak the KSE code. Keep track of zobie threads using the Per-CPU storage 2002-12-10 02:33:45 +00:00
pioctl.h Kernel modifications necessary to allow to follow fork()ed children. 2002-08-04 01:07:02 +00:00
pipe.h Move to a nested include of _label.h instead of mac.h in sys/sys/*.h 2002-08-14 01:04:43 +00:00
poll.h Reconnect a comment with its code. 2002-07-10 04:47:25 +00:00
posix4.h Back out M_* changes, per decision of the TRB. 2003-02-19 05:47:46 +00:00
power.h Very minor warns fix. Add a declaration for an unused parameter. 2002-07-15 14:06:03 +00:00
priority.h
proc.h Eliminate unused KSE symbols. 2003-02-19 06:59:00 +00:00
procfs.h
protosw.h Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup, 2003-01-01 18:49:04 +00:00
ptio.h
ptrace.h
queue.h Add a macro for SLIST traversal 'SLIST_FOREACH_PREVPTR', 2002-08-05 05:18:43 +00:00
random.h add RANDOM_PURE for use by crypto drivers that harvest data from h/w RNG's 2002-10-04 20:34:08 +00:00
reboot.h
regression.h
resource.h Move cp_time[CPUSTATES] and related #defines to <sys/resource.h> 2003-02-16 13:30:29 +00:00
resourcevar.h Remove the PL_SHAREMOD flag from struct plimit, which could have been 2003-02-20 04:18:42 +00:00
rman.h Implement rman_get_device 2003-02-12 07:00:59 +00:00
rtprio.h
runq.h Make the run queue parameters machine dependent. Optimize 64 bit 2002-05-25 01:12:23 +00:00
sbuf.h Add the new function "sbuf_done()" which returns non-zero if the sbuf is 2002-10-04 09:58:17 +00:00
sched.h - Add the new sched_pctcpu() function to the sched_* api. 2002-11-21 09:30:55 +00:00
select.h Add complete struct timeval by including <sys/_timeval.h>. 2002-12-31 04:13:50 +00:00
selinfo.h
sem.h Bring semop() closer the the opengroup standards. 2003-01-25 21:27:37 +00:00
sema.h
semaphore.h o Adjust the SEM_VALUE_MAX macro so that <machine/limits.h> isn't 2002-10-04 21:31:33 +00:00
shm.h Bring shm functions closer the the opengroup standards. 2003-01-25 21:33:05 +00:00
sigio.h When compiling the kernel do not implicitly include filedesc.h from proc.h, 2003-01-01 01:56:19 +00:00
signal.h Fix drift of the comment about sa_sigaction away from its code. 2002-12-29 19:34:37 +00:00
signalvar.h Split 4.x and 5.x signal handling so that we can keep 4.x signal 2002-10-25 19:10:58 +00:00
smp.h Forward declare struct thread. 2002-05-20 16:11:38 +00:00
snoop.h Declare the snp ioctl()s to work on udev_t, since that is what they 2002-11-11 10:45:31 +00:00
socket.h It is bad style to define the same structure in multiple header 2002-12-28 22:17:29 +00:00
socketvar.h Back out M_* changes, per decision of the TRB. 2003-02-19 05:47:46 +00:00
sockio.h
soundcard.h Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup, 2003-01-01 18:49:04 +00:00
stat.h Use unsigned long' instead of u_long' so that this header compiles 2002-10-11 15:52:14 +00:00
statvfs.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
stddef.h Add the sys/stddef.h header, so that we can have ptrdiff_t 2002-11-13 15:14:57 +00:00
stdint.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
sun_disklabel.h Duh. #ifdef CTASSERT so we don't break user-land software. 2002-10-29 06:43:57 +00:00
sx.h
syscall.h Add a timeout parameter to kse_release. 2003-02-20 08:18:15 +00:00
syscall.mk Add a timeout parameter to kse_release. 2003-02-20 08:18:15 +00:00
syscallsubr.h Break out the bind and connect syscalls to intend to make calling 2003-02-03 17:36:52 +00:00
sysctl.h Introduce the ability to flag a sysctl for operation at secure level 2 or 3 2003-01-14 19:35:33 +00:00
sysent.h SCARGS removal take II. 2002-12-14 01:56:26 +00:00
syslimits.h Add the POSIX sanctioned "LOGIN_NAME_MAX" -- Maximum length of a login name. 2003-01-06 04:33:47 +00:00
syslog.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
sysproto.h Add a timeout parameter to kse_release. 2003-02-20 08:18:15 +00:00
systm.h Add vsnrprintf() which is just like vsnprintf() but takes a "radix" 2003-02-04 10:00:34 +00:00
taskqueue.h
termios.h
tiio.h At long last, commit the zero copy sockets code. 2002-06-26 03:37:47 +00:00
time.h Implement CLOCK_MONOTONIC. 2003-02-23 10:18:31 +00:00
timeb.h
timepps.h Brucifixion ? Yes, out that door, row on the left, one patch each. 2002-04-30 19:48:45 +00:00
timers.h
times.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
timespec.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
timetc.h Move timecounters notion of frequency to 64 bits. 2003-01-29 11:29:22 +00:00
timex.h Removed unused forward struct declaration. 2002-04-28 09:51:45 +00:00
tree.h Import OpenBSD's <sys/tree.h>, needed by OpenSSH. 2002-06-23 14:38:51 +00:00
tty.h Declare tty related counters in a tty related .h file. 2003-02-16 13:23:29 +00:00
ttychars.h
ttycom.h
ttydefaults.h
ttydev.h
types.h Style: keep most typedefs in the same place. 2003-01-29 19:36:08 +00:00
ucontext.h Add getcontext, setcontext, and swapcontext as system calls. 2002-11-16 06:35:53 +00:00
ucred.h Allow one to grab the definition of struct ucred by defining _WANT_UCRED 2002-08-28 20:39:48 +00:00
uio.h It is bad style to define the same structure in multiple header 2002-12-28 22:17:29 +00:00
un.h o Merge <machine/ansi.h> and <machine/types.h> into a new header 2002-08-21 16:20:02 +00:00
unistd.h Update limits and configuration parameters for 1003.1/TC1/D6. 2002-10-27 18:03:02 +00:00
unpcb.h Remove vestiges of no longer needed unp_rvnode field. 2003-02-06 01:34:43 +00:00
user.h The a.out md_coredump stuff isn't referenced anywhere anymore, and 2002-10-15 00:02:50 +00:00
utsname.h
uuid.h Wrap function prototype declarations in __BEGIN_DECLS to do the right thing 2002-11-05 10:57:53 +00:00
vmmeter.h - It's more accurate to say that vm_paging_needed() returns TRUE 2003-02-02 07:16:40 +00:00
vnode.h Prevent large files from monopolizing the system buffers. Keep 2003-02-25 06:44:42 +00:00
wait.h Remove the deprecated 4.2/4.3BSD wait union. 2002-06-05 02:21:01 +00:00
xrpuio.h