831 lines
29 KiB
Groff
831 lines
29 KiB
Groff
.\" Copyright (c) 2001, Matthew Dillon. Terms and conditions are those of
|
|
.\" the BSD Copyright as specified in the file "/usr/src/COPYRIGHT" in
|
|
.\" the source tree.
|
|
.\"
|
|
.\" $FreeBSD$
|
|
.\"
|
|
.Dd June 25, 2002
|
|
.Dt TUNING 7
|
|
.Os
|
|
.Sh NAME
|
|
.Nm tuning
|
|
.Nd performance tuning under FreeBSD
|
|
.Sh SYSTEM SETUP - DISKLABEL, NEWFS, TUNEFS, SWAP
|
|
When using
|
|
.Xr disklabel 8
|
|
or
|
|
.Xr sysinstall 8
|
|
to lay out your filesystems on a hard disk it is important to remember
|
|
that hard drives can transfer data much more quickly from outer tracks
|
|
than they can from inner tracks.
|
|
To take advantage of this you should
|
|
try to pack your smaller filesystems and swap closer to the outer tracks,
|
|
follow with the larger filesystems, and end with the largest filesystems.
|
|
It is also important to size system standard filesystems such that you
|
|
will not be forced to resize them later as you scale the machine up.
|
|
I usually create, in order, a 128M root, 1G swap, 128M
|
|
.Pa /var ,
|
|
128M
|
|
.Pa /var/tmp ,
|
|
3G
|
|
.Pa /usr ,
|
|
and use any remaining space for
|
|
.Pa /home .
|
|
.Pp
|
|
You should typically size your swap space to approximately 2x main memory.
|
|
If you do not have a lot of RAM, though, you will generally want a lot
|
|
more swap.
|
|
It is not recommended that you configure any less than
|
|
256M of swap on a system and you should keep in mind future memory
|
|
expansion when sizing the swap partition.
|
|
The kernel's VM paging algorithms are tuned to perform best when there is
|
|
at least 2x swap versus main memory.
|
|
Configuring too little swap can lead
|
|
to inefficiencies in the VM page scanning code as well as create issues
|
|
later on if you add more memory to your machine.
|
|
Finally, on larger systems
|
|
with multiple SCSI disks (or multiple IDE disks operating on different
|
|
controllers), we strongly recommend that you configure swap on each drive
|
|
(up to four drives).
|
|
The swap partitions on the drives should be approximately the same size.
|
|
The kernel can handle arbitrary sizes but
|
|
internal data structures scale to 4 times the largest swap partition.
|
|
Keeping
|
|
the swap partitions near the same size will allow the kernel to optimally
|
|
stripe swap space across the N disks.
|
|
Do not worry about overdoing it a
|
|
little, swap space is the saving grace of
|
|
.Ux
|
|
and even if you do not normally use much swap, it can give you more time to
|
|
recover from a runaway program before being forced to reboot.
|
|
.Pp
|
|
How you size your
|
|
.Pa /var
|
|
partition depends heavily on what you intend to use the machine for.
|
|
This
|
|
partition is primarily used to hold mailboxes, the print spool, and log
|
|
files.
|
|
Some people even make
|
|
.Pa /var/log
|
|
its own partition (but except for extreme cases it is not worth the waste
|
|
of a partition ID).
|
|
If your machine is intended to act as a mail
|
|
or print server,
|
|
or you are running a heavily visited web server, you should consider
|
|
creating a much larger partition \(en perhaps a gig or more.
|
|
It is very easy
|
|
to underestimate log file storage requirements.
|
|
.Pp
|
|
Sizing
|
|
.Pa /var/tmp
|
|
depends on the kind of temporary file usage you think you will need.
|
|
128M is
|
|
the minimum we recommend.
|
|
Also note that sysinstall will create a
|
|
.Pa /tmp
|
|
directory.
|
|
Dedicating a partition for temporary file storage is important for
|
|
two reasons: first, it reduces the possibility of filesystem corruption
|
|
in a crash, and second it reduces the chance of a runaway process that
|
|
fills up
|
|
.Oo Pa /var Oc Ns Pa /tmp
|
|
from blowing up more critical subsystems (mail,
|
|
logging, etc).
|
|
Filling up
|
|
.Oo Pa /var Oc Ns Pa /tmp
|
|
is a very common problem to have.
|
|
.Pp
|
|
In the old days there were differences between
|
|
.Pa /tmp
|
|
and
|
|
.Pa /var/tmp ,
|
|
but the introduction of
|
|
.Pa /var
|
|
(and
|
|
.Pa /var/tmp )
|
|
led to massive confusion
|
|
by program writers so today programs haphazardly use one or the
|
|
other and thus no real distinction can be made between the two.
|
|
So it makes sense to have just one temporary directory.
|
|
However you handle
|
|
.Pa /tmp ,
|
|
the one thing you do not want to do is leave it sitting
|
|
on the root partition where it might cause root to fill up or possibly
|
|
corrupt root in a crash/reboot situation.
|
|
.Pp
|
|
The
|
|
.Pa /usr
|
|
partition holds the bulk of the files required to support the system and
|
|
a subdirectory within it called
|
|
.Pa /usr/local
|
|
holds the bulk of the files installed from the
|
|
.Xr ports 7
|
|
hierarchy.
|
|
If you do not use ports all that much and do not intend to keep
|
|
system source
|
|
.Pq Pa /usr/src
|
|
on the machine, you can get away with
|
|
a 1 gigabyte
|
|
.Pa /usr
|
|
partition.
|
|
However, if you install a lot of ports
|
|
(especially window managers and Linux-emulated binaries), we recommend
|
|
at least a 2 gigabyte
|
|
.Pa /usr
|
|
and if you also intend to keep system source
|
|
on the machine, we recommend a 3 gigabyte
|
|
.Pa /usr .
|
|
Do not underestimate the
|
|
amount of space you will need in this partition, it can creep up and
|
|
surprise you!
|
|
.Pp
|
|
The
|
|
.Pa /home
|
|
partition is typically used to hold user-specific data.
|
|
I usually size it to the remainder of the disk.
|
|
.Pp
|
|
Why partition at all?
|
|
Why not create one big
|
|
.Pa /
|
|
partition and be done with it?
|
|
Then I do not have to worry about undersizing things!
|
|
Well, there are several reasons this is not a good idea.
|
|
First,
|
|
each partition has different operational characteristics and separating them
|
|
allows the filesystem to tune itself to those characteristics.
|
|
For example,
|
|
the root and
|
|
.Pa /usr
|
|
partitions are read-mostly, with very little writing, while
|
|
a lot of reading and writing could occur in
|
|
.Pa /var
|
|
and
|
|
.Pa /var/tmp .
|
|
By properly
|
|
partitioning your system fragmentation introduced in the smaller more
|
|
heavily write-loaded partitions will not bleed over into the mostly-read
|
|
partitions.
|
|
Additionally, keeping the write-loaded partitions closer to
|
|
the edge of the disk (i.e. before the really big partitions instead of after
|
|
in the partition table) will increase I/O performance in the partitions
|
|
where you need it the most.
|
|
Now it is true that you might also need I/O
|
|
performance in the larger partitions, but they are so large that shifting
|
|
them more towards the edge of the disk will not lead to a significant
|
|
performance improvement whereas moving
|
|
.Pa /var
|
|
to the edge can have a huge impact.
|
|
Finally, there are safety concerns.
|
|
Having a small neat root partition that
|
|
is essentially read-only gives it a greater chance of surviving a bad crash
|
|
intact.
|
|
.Pp
|
|
Properly partitioning your system also allows you to tune
|
|
.Xr newfs 8 ,
|
|
and
|
|
.Xr tunefs 8
|
|
parameters.
|
|
Tuning
|
|
.Xr newfs 8
|
|
requires more experience but can lead to significant improvements in
|
|
performance.
|
|
There are three parameters that are relatively safe to tune:
|
|
.Em blocksize , bytes/i-node ,
|
|
and
|
|
.Em cylinders/group .
|
|
.Pp
|
|
.Fx
|
|
performs best when using 8K or 16K filesystem block sizes.
|
|
The default filesystem block size is 16K,
|
|
which provides best performance for most applications,
|
|
with the exception of those that perform random access on large files
|
|
(such as database server software).
|
|
Such applications tend to perform better with a smaller block size,
|
|
although modern disk characteristics are such that the performance
|
|
gain from using a smaller block size may not be worth consideration.
|
|
Using a block size larger than 16K
|
|
can cause fragmentation of the buffer cache and
|
|
lead to lower performance.
|
|
.Pp
|
|
The defaults may be unsuitable
|
|
for a filesystem that requires a very large number of i-nodes
|
|
or is intended to hold a large number of very small files.
|
|
Such a filesystem should be created with an 8K or 4K block size.
|
|
This also requires you to specify a smaller
|
|
fragment size.
|
|
We recommend always using a fragment size that is 1/8
|
|
the block size (less testing has been done on other fragment size factors).
|
|
The
|
|
.Xr newfs 8
|
|
options for this would be
|
|
.Dq Li "newfs -f 1024 -b 8192 ..." .
|
|
.Pp
|
|
If a large partition is intended to be used to hold fewer, larger files, such
|
|
as a database files, you can increase the
|
|
.Em bytes/i-node
|
|
ratio which reduces the number of i-nodes (maximum number of files and
|
|
directories that can be created) for that partition.
|
|
Decreasing the number
|
|
of i-nodes in a filesystem can greatly reduce
|
|
.Xr fsck 8
|
|
recovery times after a crash.
|
|
Do not use this option
|
|
unless you are actually storing large files on the partition, because if you
|
|
overcompensate you can wind up with a filesystem that has lots of free
|
|
space remaining but cannot accommodate any more files.
|
|
Using 32768, 65536, or 262144 bytes/i-node is recommended.
|
|
You can go higher but
|
|
it will have only incremental effects on
|
|
.Xr fsck 8
|
|
recovery times.
|
|
For example,
|
|
.Dq Li "newfs -i 32768 ..." .
|
|
.Pp
|
|
.Xr tunefs 8
|
|
may be used to further tune a filesystem.
|
|
This command can be run in
|
|
single-user mode without having to reformat the filesystem.
|
|
However, this is possibly the most abused program in the system.
|
|
Many people attempt to
|
|
increase available filesystem space by setting the min-free percentage to 0.
|
|
This can lead to severe filesystem fragmentation and we do not recommend
|
|
that you do this.
|
|
Really the only
|
|
.Xr tunefs 8
|
|
option worthwhile here is turning on
|
|
.Em softupdates
|
|
with
|
|
.Dq Li "tunefs -n enable /filesystem" .
|
|
(Note: in
|
|
.Fx 4.5
|
|
and later, softupdates can be turned on using the
|
|
.Fl U
|
|
option to
|
|
.Xr newfs 8 ,
|
|
and
|
|
.Xr sysinstall 8
|
|
will typically enable softupdates automatically for non-root filesystems).
|
|
Softupdates drastically improves meta-data performance, mainly file
|
|
creation and deletion.
|
|
We recommend enabling softupdates on most filesystems; however, there
|
|
are two limitations to softupdates that you should be aware of when
|
|
determining whether to use it on a filesystem.
|
|
First, softupdates guarantees filesystem consistency in the
|
|
case of a crash but could very easily be several seconds (even a minute!)
|
|
behind updating the physical disk.
|
|
If you crash you may lose more work
|
|
than otherwise.
|
|
Secondly, softupdates delays the freeing of filesystem
|
|
blocks.
|
|
If you have a filesystem (such as the root filesystem) which is
|
|
close to full, doing a major update of it, e.g.\&
|
|
.Dq Li "make installworld" ,
|
|
can run it out of space and cause the update to fail.
|
|
.Pp
|
|
A number of run-time
|
|
.Xr mount 8
|
|
options exist that can help you tune the system.
|
|
For this reason, softupdates will not be enabled on the root filesystem
|
|
during a typical install.
|
|
The most obvious and most dangerous one is
|
|
.Cm async .
|
|
Do not ever use it, it is far too dangerous.
|
|
A less dangerous and more
|
|
useful
|
|
.Xr mount 8
|
|
option is called
|
|
.Cm noatime .
|
|
.Ux
|
|
filesystems normally update the last-accessed time of a file or
|
|
directory whenever it is accessed.
|
|
This operation is handled in
|
|
.Fx
|
|
with a delayed write and normally does not create a burden on the system.
|
|
However, if your system is accessing a huge number of files on a continuing
|
|
basis the buffer cache can wind up getting polluted with atime updates,
|
|
creating a burden on the system.
|
|
For example, if you are running a heavily
|
|
loaded web site, or a news server with lots of readers, you might want to
|
|
consider turning off atime updates on your larger partitions with this
|
|
.Xr mount 8
|
|
option.
|
|
However, you should not gratuitously turn off atime
|
|
updates everywhere.
|
|
For example, the
|
|
.Pa /var
|
|
filesystem customarily
|
|
holds mailboxes, and atime (in combination with mtime) is used to
|
|
determine whether a mailbox has new mail.
|
|
You might as well leave
|
|
atime turned on for mostly read-only partitions such as
|
|
.Pa /
|
|
and
|
|
.Pa /usr
|
|
as well.
|
|
This is especially useful for
|
|
.Pa /
|
|
since some system utilities
|
|
use the atime field for reporting.
|
|
.Sh STRIPING DISKS
|
|
In larger systems you can stripe partitions from several drives together
|
|
to create a much larger overall partition.
|
|
Striping can also improve
|
|
the performance of a filesystem by splitting I/O operations across two
|
|
or more disks.
|
|
The
|
|
.Xr vinum 8
|
|
and
|
|
.Xr ccdconfig 8
|
|
utilities may be used to create simple striped filesystems.
|
|
Generally
|
|
speaking, striping smaller partitions such as the root and
|
|
.Pa /var/tmp ,
|
|
or essentially read-only partitions such as
|
|
.Pa /usr
|
|
is a complete waste of time.
|
|
You should only stripe partitions that require serious I/O performance,
|
|
typically
|
|
.Pa /var , /home ,
|
|
or custom partitions used to hold databases and web pages.
|
|
Choosing the proper stripe size is also
|
|
important.
|
|
Filesystems tend to store meta-data on power-of-2 boundaries
|
|
and you usually want to reduce seeking rather than increase seeking.
|
|
This
|
|
means you want to use a large off-center stripe size such as 1152 sectors
|
|
so sequential I/O does not seek both disks and so meta-data is distributed
|
|
across both disks rather than concentrated on a single disk.
|
|
If
|
|
you really need to get sophisticated, we recommend using a real hardware
|
|
RAID controller from the list of
|
|
.Fx
|
|
supported controllers.
|
|
.Sh SYSCTL TUNING
|
|
.Xr sysctl 8
|
|
variables permit system behavior to be monitored and controlled at
|
|
run-time.
|
|
Some sysctls simply report on the behavior of the system; others allow
|
|
the system behavior to be modified;
|
|
some may be set at boot time using
|
|
.Xr rc.conf 5 ,
|
|
but most will be set via
|
|
.Xr sysctl.conf 5 .
|
|
There are several hundred sysctls in the system, including many that appear
|
|
to be candidates for tuning but actually are not.
|
|
In this document we will only cover the ones that have the greatest effect
|
|
on the system.
|
|
.Pp
|
|
The
|
|
.Va kern.ipc.shm_use_phys
|
|
sysctl defaults to 0 (off) and may be set to 0 (off) or 1 (on).
|
|
Setting
|
|
this parameter to 1 will cause all System V shared memory segments to be
|
|
mapped to unpageable physical RAM.
|
|
This feature only has an effect if you
|
|
are either (A) mapping small amounts of shared memory across many (hundreds)
|
|
of processes, or (B) mapping large amounts of shared memory across any
|
|
number of processes.
|
|
This feature allows the kernel to remove a great deal
|
|
of internal memory management page-tracking overhead at the cost of wiring
|
|
the shared memory into core, making it unswappable.
|
|
.Pp
|
|
The
|
|
.Va vfs.vmiodirenable
|
|
sysctl defaults to 1 (on).
|
|
This parameter controls how directories are cached
|
|
by the system.
|
|
Most directories are small and use but a single fragment
|
|
(typically 1K) in the filesystem and even less (typically 512 bytes) in
|
|
the buffer cache.
|
|
However, when operating in the default mode the buffer
|
|
cache will only cache a fixed number of directories even if you have a huge
|
|
amount of memory.
|
|
Turning on this sysctl allows the buffer cache to use
|
|
the VM Page Cache to cache the directories.
|
|
The advantage is that all of
|
|
memory is now available for caching directories.
|
|
The disadvantage is that
|
|
the minimum in-core memory used to cache a directory is the physical page
|
|
size (typically 4K) rather than 512 bytes.
|
|
We recommend turning this option off in memory-constrained environments;
|
|
however, when on, it will substantially improve the performance of services
|
|
that manipulate a large number of files.
|
|
Such services can include web caches, large mail systems, and news systems.
|
|
Turning on this option will generally not reduce performance even with the
|
|
wasted memory but you should experiment to find out.
|
|
.Pp
|
|
The
|
|
.Va vfs.write_behind
|
|
sysctl defaults to 1 (on).
|
|
This tells the filesystem to issue media
|
|
writes as full clusters are collected, which typically occurs when writing
|
|
large sequential files.
|
|
The idea is to avoid saturating the buffer
|
|
cache with dirty buffers when it would not benefit I/O performance.
|
|
However,
|
|
this may stall processes and under certain circumstances you may wish to turn
|
|
it off.
|
|
.Pp
|
|
The
|
|
.Va vfs.hirunningspace
|
|
sysctl determines how much outstanding write I/O may be queued to
|
|
disk controllers system-wide at any given instance.
|
|
The default is
|
|
usually sufficient but on machines with lots of disks you may want to bump
|
|
it up to four or five megabytes.
|
|
Note that setting too high a value
|
|
(exceeding the buffer cache's write threshold) can lead to extremely
|
|
bad clustering performance.
|
|
Do not set this value arbitrarily high!
|
|
Also,
|
|
higher write queueing values may add latency to reads occuring at the same
|
|
time.
|
|
.Pp
|
|
There are various other buffer-cache and VM page cache related sysctls.
|
|
We do not recommend modifying these values.
|
|
As of
|
|
.Fx 4.3 ,
|
|
the VM system does an extremely good job tuning itself.
|
|
.Pp
|
|
The
|
|
.Va net.inet.tcp.sendspace
|
|
and
|
|
.Va net.inet.tcp.recvspace
|
|
sysctls are of particular interest if you are running network intensive
|
|
applications.
|
|
This controls the amount of send and receive buffer space
|
|
allowed for any given TCP connection.
|
|
The default sending buffer is 32K; the default receiving buffer
|
|
is 64K.
|
|
You can often
|
|
improve bandwidth utilization by increasing the default at the cost of
|
|
eating up more kernel memory for each connection.
|
|
We do not recommend
|
|
increasing the defaults if you are serving hundreds or thousands of
|
|
simultaneous connections because it is possible to quickly run the system
|
|
out of memory due to stalled connections building up.
|
|
But if you need
|
|
high bandwidth over a fewer number of connections, especially if you have
|
|
gigabit Ethernet, increasing these defaults can make a huge difference.
|
|
You can adjust the buffer size for incoming and outgoing data separately.
|
|
For example, if your machine is primarily doing web serving you may want
|
|
to decrease the recvspace in order to be able to increase the
|
|
sendspace without eating too much kernel memory.
|
|
Note that the routing table (see
|
|
.Xr route 8 )
|
|
can be used to introduce route-specific send and receive buffer size
|
|
defaults.
|
|
.Pp
|
|
As an additional management tool you can use pipes in your
|
|
firewall rules (see
|
|
.Xr ipfw 8 )
|
|
to limit the bandwidth going to or from particular IP blocks or ports.
|
|
For example, if you have a T1 you might want to limit your web traffic
|
|
to 70% of the T1's bandwidth in order to leave the remainder available
|
|
for mail and interactive use.
|
|
Normally a heavily loaded web server
|
|
will not introduce significant latencies into other services even if
|
|
the network link is maxed out, but enforcing a limit can smooth things
|
|
out and lead to longer term stability.
|
|
Many people also enforce artificial
|
|
bandwidth limitations in order to ensure that they are not charged for
|
|
using too much bandwidth.
|
|
.Pp
|
|
Setting the send or receive TCP buffer to values larger then 65535 will result
|
|
in a marginal performance improvement unless both hosts support the window
|
|
scaling extension of the TCP protocol, which is controlled by the
|
|
.Va net.inet.tcp.rfc1323
|
|
sysctl.
|
|
These extensions should be enabled and the TCP buffer size should be set
|
|
to a value larger than 65536 in order to obtain good performance out of
|
|
certain types of network links; specifically, gigabit WAN links and
|
|
high-latency satellite links.
|
|
RFC1323 support is enabled by default.
|
|
.Pp
|
|
The
|
|
.Va net.inet.tcp.always_keepalive
|
|
sysctl determines whether or not the TCP implementation should attempt
|
|
to detect dead TCP connections by intermittently delivering
|
|
.Dq keepalives
|
|
on the connection.
|
|
By default, this is enabled for all applications; by setting this
|
|
sysctl to 0, only applications that specifically request keepalives
|
|
will use them.
|
|
In most environments, TCP keepalives will improve the management of
|
|
system state by expiring dead TCP connections, particularly for
|
|
systems serving dialup users who may not always terminate individual
|
|
TCP connections before disconnecting from the network.
|
|
However, in some environments, temporary network outages may be
|
|
incorrectly identified as dead sessions, resulting in unexpectedly
|
|
terminated TCP connections.
|
|
In such environments, setting the sysctl to 0 may reduce the occurrence of
|
|
TCP session disconnections.
|
|
.Pp
|
|
The
|
|
.Va kern.ipc.somaxconn
|
|
sysctl limits the size of the listen queue for accepting new TCP connections.
|
|
The default value of 128 is typically too low for robust handling of new
|
|
connections in a heavily loaded web server environment.
|
|
For such environments,
|
|
we recommend increasing this value to 1024 or higher.
|
|
The service daemon
|
|
may itself limit the listen queue size (e.g.\&
|
|
.Xr sendmail 8 ,
|
|
apache) but will
|
|
often have a directive in its configuration file to adjust the queue size up.
|
|
Larger listen queues also do a better job of fending off denial of service
|
|
attacks.
|
|
.Pp
|
|
The
|
|
.Va kern.maxfiles
|
|
sysctl determines how many open files the system supports.
|
|
The default is
|
|
typically a few thousand but you may need to bump this up to ten or twenty
|
|
thousand if you are running databases or large descriptor-heavy daemons.
|
|
The read-only
|
|
.Va kern.openfiles
|
|
sysctl may be interrogated to determine the current number of open files
|
|
on the system.
|
|
.Pp
|
|
The
|
|
.Va vm.swap_idle_enabled
|
|
sysctl is useful in large multi-user systems where you have lots of users
|
|
entering and leaving the system and lots of idle processes.
|
|
Such systems
|
|
tend to generate a great deal of continuous pressure on free memory reserves.
|
|
Turning this feature on and adjusting the swapout hysteresis (in idle
|
|
seconds) via
|
|
.Va vm.swap_idle_threshold1
|
|
and
|
|
.Va vm.swap_idle_threshold2
|
|
allows you to depress the priority of pages associated with idle processes
|
|
more quickly then the normal pageout algorithm.
|
|
This gives a helping hand
|
|
to the pageout daemon.
|
|
Do not turn this option on unless you need it,
|
|
because the tradeoff you are making is to essentially pre-page memory sooner
|
|
rather then later, eating more swap and disk bandwidth.
|
|
In a small system
|
|
this option will have a detrimental effect but in a large system that is
|
|
already doing moderate paging this option allows the VM system to stage
|
|
whole processes into and out of memory more easily.
|
|
.Sh LOADER TUNABLES
|
|
Some aspects of the system behavior may not be tunable at runtime because
|
|
memory allocations they perform must occur early in the boot process.
|
|
To change loader tunables, you must set their values in
|
|
.Xr loader.conf 5
|
|
and reboot the system.
|
|
.Pp
|
|
.Va kern.maxusers
|
|
controls the scaling of a number of static system tables, including defaults
|
|
for the maximum number of open files, sizing of network memory resources, etc.
|
|
As of
|
|
.Fx 4.5 ,
|
|
.Va kern.maxusers
|
|
is automatically sized at boot based on the amount of memory available in
|
|
the system, and may be determined at run-time by inspecting the value of the
|
|
read-only
|
|
.Va kern.maxusers
|
|
sysctl.
|
|
Some sites will require larger or smaller values of
|
|
.Va kern.maxusers
|
|
and may set it as a loader tunable; values of 64, 128, and 256 are not
|
|
uncommon.
|
|
We do not recommend going above 256 unless you need a huge number
|
|
of file descriptors; many of the tunable values set to their defaults by
|
|
.Va kern.maxusers
|
|
may be individually overridden at boot-time or run-time as described
|
|
elsewhere in this document.
|
|
Systems older than
|
|
.Fx 4.4
|
|
must set this value via the kernel
|
|
.Xr config 8
|
|
option
|
|
.Cd maxusers
|
|
instead.
|
|
.Pp
|
|
.Va kern.ipc.nmbclusters
|
|
may be adjusted to increase the number of network mbufs the system is
|
|
willing to allocate.
|
|
Each cluster represents approximately 2K of memory,
|
|
so a value of 1024 represents 2M of kernel memory reserved for network
|
|
buffers.
|
|
You can do a simple calculation to figure out how many you need.
|
|
If you have a web server which maxes out at 1000 simultaneous connections,
|
|
and each connection eats a 16K receive and 16K send buffer, you need
|
|
approximate 32MB worth of network buffers to deal with it.
|
|
A good rule of
|
|
thumb is to multiply by 2, so 32MBx2 = 64MB/2K = 32768.
|
|
So for this case
|
|
you would want to set
|
|
.Va kern.ipc.nmbclusters
|
|
to 32768.
|
|
We recommend values between
|
|
1024 and 4096 for machines with moderates amount of memory, and between 4096
|
|
and 32768 for machines with greater amounts of memory.
|
|
Under no circumstances
|
|
should you specify an arbitrarily high value for this parameter, it could
|
|
lead to a boot-time crash.
|
|
The
|
|
.Fl m
|
|
option to
|
|
.Xr netstat 1
|
|
may be used to observe network cluster use.
|
|
Older versions of
|
|
.Fx
|
|
do not have this tunable and require that the
|
|
kernel
|
|
.Xr config 8
|
|
option
|
|
.Dv NMBCLUSTERS
|
|
be set instead.
|
|
.Pp
|
|
More and more programs are using the
|
|
.Xr sendfile 2
|
|
system call to transmit files over the network.
|
|
The
|
|
.Va kern.ipc.nsfbufs
|
|
sysctl controls the number of filesystem buffers
|
|
.Xr sendfile 2
|
|
is allowed to use to perform its work.
|
|
This parameter nominally scales
|
|
with
|
|
.Va kern.maxusers
|
|
so you should not need to modify this parameter except under extreme
|
|
circumstances.
|
|
.Sh KERNEL CONFIG TUNING
|
|
There are a number of kernel options that you may have to fiddle with in
|
|
a large scale system.
|
|
In order to change these options you need to be
|
|
able to compile a new kernel from source.
|
|
The
|
|
.Xr config 8
|
|
manual page and the handbook are good starting points for learning how to
|
|
do this.
|
|
Generally the first thing you do when creating your own custom
|
|
kernel is to strip out all the drivers and services you do not use.
|
|
Removing things like
|
|
.Dv INET6
|
|
and drivers you do not have will reduce the size of your kernel, sometimes
|
|
by a megabyte or more, leaving more memory available for applications.
|
|
.Pp
|
|
.Dv SCSI_DELAY
|
|
and
|
|
.Dv IDE_DELAY
|
|
may be used to reduce system boot times.
|
|
The defaults are fairly high and
|
|
can be responsible for 15+ seconds of delay in the boot process.
|
|
Reducing
|
|
.Dv SCSI_DELAY
|
|
to 5 seconds usually works (especially with modern drives).
|
|
Reducing
|
|
.Dv IDE_DELAY
|
|
also works but you have to be a little more careful.
|
|
.Pp
|
|
There are a number of
|
|
.Dv *_CPU
|
|
options that can be commented out.
|
|
If you only want the kernel to run
|
|
on a Pentium class CPU, you can easily remove
|
|
.Dv I386_CPU
|
|
and
|
|
.Dv I486_CPU ,
|
|
but only remove
|
|
.Dv I586_CPU
|
|
if you are sure your CPU is being recognized as a Pentium II or better.
|
|
Some clones may be recognized as a Pentium or even a 486 and not be able
|
|
to boot without those options.
|
|
If it works, great!
|
|
The operating system
|
|
will be able to better-use higher-end CPU features for MMU, task switching,
|
|
timebase, and even device operations.
|
|
Additionally, higher-end CPUs support
|
|
4MB MMU pages which the kernel uses to map the kernel itself into memory,
|
|
which increases its efficiency under heavy syscall loads.
|
|
.Sh IDE WRITE CACHING
|
|
.Fx 4.3
|
|
flirted with turning off IDE write caching.
|
|
This reduced write bandwidth
|
|
to IDE disks but was considered necessary due to serious data consistency
|
|
issues introduced by hard drive vendors.
|
|
Basically the problem is that
|
|
IDE drives lie about when a write completes.
|
|
With IDE write caching turned
|
|
on, IDE hard drives will not only write data to disk out of order, they
|
|
will sometimes delay some of the blocks indefinitely when under heavy disk
|
|
loads.
|
|
A crash or power failure can result in serious filesystem
|
|
corruption.
|
|
So our default was changed to be safe.
|
|
Unfortunately, the
|
|
result was such a huge loss in performance that we caved in and changed the
|
|
default back to on after the release.
|
|
You should check the default on
|
|
your system by observing the
|
|
.Va hw.ata.wc
|
|
sysctl variable.
|
|
If IDE write caching is turned off, you can turn it back
|
|
on by setting the
|
|
.Va hw.ata.wc
|
|
loader tunable to 1.
|
|
More information on tuning the ATA driver system may be found in
|
|
.Xr ata 4 .
|
|
.Pp
|
|
There is a new experimental feature for IDE hard drives called
|
|
.Va hw.ata.tags
|
|
(you also set this in the boot loader) which allows write caching to be safely
|
|
turned on.
|
|
This brings SCSI tagging features to IDE drives.
|
|
As of this
|
|
writing only IBM DPTA and DTLA drives support the feature.
|
|
Warning!
|
|
These
|
|
drives apparently have quality control problems and I do not recommend
|
|
purchasing them at this time.
|
|
If you need performance, go with SCSI.
|
|
.Sh CPU, MEMORY, DISK, NETWORK
|
|
The type of tuning you do depends heavily on where your system begins to
|
|
bottleneck as load increases.
|
|
If your system runs out of CPU (idle times
|
|
are perpetually 0%) then you need to consider upgrading the CPU or moving to
|
|
an SMP motherboard (multiple CPU's), or perhaps you need to revisit the
|
|
programs that are causing the load and try to optimize them.
|
|
If your system
|
|
is paging to swap a lot you need to consider adding more memory.
|
|
If your
|
|
system is saturating the disk you typically see high CPU idle times and
|
|
total disk saturation.
|
|
.Xr systat 1
|
|
can be used to monitor this.
|
|
There are many solutions to saturated disks:
|
|
increasing memory for caching, mirroring disks, distributing operations across
|
|
several machines, and so forth.
|
|
If disk performance is an issue and you
|
|
are using IDE drives, switching to SCSI can help a great deal.
|
|
While modern
|
|
IDE drives compare with SCSI in raw sequential bandwidth, the moment you
|
|
start seeking around the disk SCSI drives usually win.
|
|
.Pp
|
|
Finally, you might run out of network suds.
|
|
The first line of defense for
|
|
improving network performance is to make sure you are using switches instead
|
|
of hubs, especially these days where switches are almost as cheap.
|
|
Hubs
|
|
have severe problems under heavy loads due to collision backoff and one bad
|
|
host can severely degrade the entire LAN.
|
|
Second, optimize the network path
|
|
as much as possible.
|
|
For example, in
|
|
.Xr firewall 7
|
|
we describe a firewall protecting internal hosts with a topology where
|
|
the externally visible hosts are not routed through it.
|
|
Use 100BaseT rather
|
|
than 10BaseT, or use 1000BaseT rather then 100BaseT, depending on your needs.
|
|
Most bottlenecks occur at the WAN link (e.g.\&
|
|
modem, T1, DSL, whatever).
|
|
If expanding the link is not an option it may be possible to use
|
|
.Xr dummynet 4
|
|
feature to implement peak shaving or other forms of traffic shaping to
|
|
prevent the overloaded service (such as web services) from affecting other
|
|
services (such as email), or vice versa.
|
|
In home installations this could
|
|
be used to give interactive traffic (your browser,
|
|
.Xr ssh 1
|
|
logins) priority
|
|
over services you export from your box (web services, email).
|
|
.Sh SEE ALSO
|
|
.Xr netstat 1 ,
|
|
.Xr systat 1 ,
|
|
.Xr ata 4 ,
|
|
.Xr dummynet 4 ,
|
|
.Xr login.conf 5 ,
|
|
.Xr rc.conf 5 ,
|
|
.Xr sysctl.conf 5 ,
|
|
.Xr firewall 7 ,
|
|
.Xr hier 7 ,
|
|
.Xr ports 7 ,
|
|
.Xr boot 8 ,
|
|
.Xr ccdconfig 8 ,
|
|
.Xr config 8 ,
|
|
.Xr disklabel 8 ,
|
|
.Xr fsck 8 ,
|
|
.Xr ifconfig 8 ,
|
|
.Xr ipfw 8 ,
|
|
.Xr loader 8 ,
|
|
.Xr mount 8 ,
|
|
.Xr newfs 8 ,
|
|
.Xr route 8 ,
|
|
.Xr sysctl 8 ,
|
|
.Xr sysinstall 8 ,
|
|
.Xr tunefs 8 ,
|
|
.Xr vinum 8
|
|
.Sh HISTORY
|
|
The
|
|
.Nm
|
|
manual page was originally written by
|
|
.An Matthew Dillon
|
|
and first appeared
|
|
in
|
|
.Fx 4.3 ,
|
|
May 2001.
|