366 lines
11 KiB
Groff
366 lines
11 KiB
Groff
.\" Copyright (c) 2007 Matthew Jacob
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
|
|
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
|
|
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
.\" SUCH DAMAGE.
|
|
.\"
|
|
.\" $FreeBSD$
|
|
.\"
|
|
.Dd June 11, 2015
|
|
.Dt GMULTIPATH 8
|
|
.Os
|
|
.Sh NAME
|
|
.Nm gmultipath
|
|
.Nd "disk multipath control utility"
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Cm create
|
|
.Op Fl ARv
|
|
.Ar name
|
|
.Ar prov ...
|
|
.Nm
|
|
.Cm label
|
|
.Op Fl ARv
|
|
.Ar name
|
|
.Ar prov ...
|
|
.Nm
|
|
.Cm configure
|
|
.Op Fl APRv
|
|
.Ar name
|
|
.Nm
|
|
.Cm add
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm remove
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm fail
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm restore
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm rotate
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm prefer
|
|
.Op Fl v
|
|
.Ar name
|
|
.Ar prov
|
|
.Nm
|
|
.Cm getactive
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm destroy
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm stop
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm clear
|
|
.Op Fl v
|
|
.Ar prov ...
|
|
.Nm
|
|
.Cm list
|
|
.Nm
|
|
.Cm status
|
|
.Nm
|
|
.Cm load
|
|
.Nm
|
|
.Cm unload
|
|
.Sh DESCRIPTION
|
|
The
|
|
.Nm
|
|
utility is used for device multipath configuration.
|
|
.Pp
|
|
The multipath device can be configured using two different methods:
|
|
.Dq manual
|
|
or
|
|
.Dq automatic .
|
|
When using the
|
|
.Dq manual
|
|
method, no metadata are stored on the devices, so the multipath
|
|
device has to be configured by hand every time it is needed.
|
|
Additional device paths also won't be detected automatically.
|
|
The
|
|
.Dq automatic
|
|
method uses on-disk metadata to detect device and all it's paths.
|
|
Metadata use the last sector of the underlying disk device and
|
|
include device name and UUID.
|
|
The UUID guarantees uniqueness in a shared storage environment
|
|
but is in general too cumbersome to use.
|
|
The name is what is exported via the device interface.
|
|
.Pp
|
|
The first argument to
|
|
.Nm
|
|
indicates an action to be performed:
|
|
.Bl -tag -width ".Cm destroy"
|
|
.It Cm create
|
|
Create multipath device with
|
|
.Dq manual
|
|
method without writing any on-disk metadata.
|
|
It is up to administrator, how to properly identify device paths.
|
|
Kernel will only check that all given providers have same media and
|
|
sector sizes.
|
|
.Pp
|
|
.Fl A
|
|
option enables Active/Active mode,
|
|
.Fl R
|
|
option enables Active/Read mode, otherwise Active/Passive mode is used
|
|
by default.
|
|
.It Cm label
|
|
Create multipath device with
|
|
.Dq automatic
|
|
method.
|
|
Label the first given provider with on-disk metadata using the specified
|
|
.Ar name .
|
|
The rest of given providers will be retasted to detect these metadata.
|
|
It reliably protects against specifying unrelated providers.
|
|
Providers with no matching metadata detected will not be added to the device.
|
|
.Pp
|
|
.Fl A
|
|
option enables Active/Active mode,
|
|
.Fl R
|
|
option enables Active/Read mode, otherwise Active/Passive mode is used
|
|
by default.
|
|
.It Cm configure
|
|
Configure the given multipath device.
|
|
.Pp
|
|
.Fl A
|
|
option enables Active/Active mode,
|
|
.Fl P
|
|
option enables Active/Passive mode,
|
|
.Fl R
|
|
option enables Active/Read mode.
|
|
.It Cm add
|
|
Add the given provider as a path to the given multipath device.
|
|
Should normally be used only for devices created with
|
|
.Dq manual
|
|
method, unless you know what you are doing (you are sure that it is another
|
|
device path, but tasting its metadata in regular
|
|
.Dq automatic
|
|
way is not possible).
|
|
.It Cm remove
|
|
Remove the given provider as a path from the given multipath device.
|
|
If the last path removed, the multipath device will be destroyed.
|
|
.It Cm fail
|
|
Mark specified provider as a path of the specified multipath device as failed.
|
|
If there are other paths present, new requests will be forwarded there.
|
|
.It Cm restore
|
|
Mark specified provider as a path of the specified multipath device as
|
|
operational, allowing it to handle requests.
|
|
.It Cm rotate
|
|
Change the active provider/path to the next available provider in Active/Passive mode.
|
|
.It Cm prefer
|
|
Change the active provider/path to the specified provider in Active/Passive mode.
|
|
.It Cm getactive
|
|
Get the currently active provider(s)/path(s).
|
|
.It Cm destroy
|
|
Destroy the given multipath device clearing metadata.
|
|
.It Cm stop
|
|
Stop the given multipath device without clearing metadata.
|
|
.It Cm clear
|
|
Clear metadata on the given provider.
|
|
.It Cm list
|
|
See
|
|
.Xr geom 8 .
|
|
.It Cm status
|
|
See
|
|
.Xr geom 8 .
|
|
.It Cm load
|
|
See
|
|
.Xr geom 8 .
|
|
.It Cm unload
|
|
See
|
|
.Xr geom 8 .
|
|
.El
|
|
.Sh SYSCTL VARIABLES
|
|
The following
|
|
.Xr sysctl 8
|
|
variable can be used to control the behavior of the
|
|
.Nm MULTIPATH
|
|
GEOM class.
|
|
.Bl -tag -width indent
|
|
.It Va kern.geom.multipath.debug : No 0
|
|
Debug level of the
|
|
.Nm MULTIPATH
|
|
GEOM class.
|
|
This can be set to 0 (default) or 1 to disable or enable various
|
|
forms of chattiness.
|
|
.It Va kern.geom.multipath.exclusive : No 1
|
|
Open underlying providers exclusively, preventing individual paths access.
|
|
.El
|
|
.Sh EXIT STATUS
|
|
Exit status is 0 on success, and 1 if the command fails.
|
|
.Sh MULTIPATH ARCHITECTURE
|
|
This is a multiple path architecture with no device knowledge or
|
|
presumptions other than size matching built in.
|
|
Therefore the user must exercise some care
|
|
in selecting providers that do indeed represent multiple paths to the
|
|
same underlying disk device.
|
|
The reason for this is that there are several
|
|
criteria across multiple underlying transport types that can
|
|
.Ar indicate
|
|
identity, but in all respects such identity can rarely be considered
|
|
.Ar definitive .
|
|
.Pp
|
|
For example, if you use the World Word Port Name of a Fibre Channel
|
|
disk object you might believe that two disks that have the same WWPN
|
|
on different paths (or even disjoint fabrics) might be considered
|
|
the same disk.
|
|
Nearly always this would be a safe assumption, until
|
|
you realize that a WWPN, like an Ethernet MAC address, is a soft
|
|
programmable entity, and that a misconfigured Director Class switch
|
|
could lead you to believe incorrectly that you have found multiple
|
|
paths to the same device.
|
|
This is an extreme and theoretical case, but
|
|
it is possible enough to indicate that the policy for deciding which
|
|
of multiple pathnames refer to the same device should be left to the
|
|
system operator who will use tools and knowledge of their own storage
|
|
subsystem to make the correct configuration selection.
|
|
.Pp
|
|
There are Active/Passive, Active/Read and Active/Active operation modes
|
|
supported.
|
|
In Active/Passive mode only one path has I/O moving on it
|
|
at any point in time.
|
|
This I/O continues until an I/O is returned with
|
|
a generic I/O error or a "Nonexistent Device" error.
|
|
When this occurs, that path is marked FAIL, the next path
|
|
in a list is selected as active and the failed I/O reissued.
|
|
In Active/Active mode all paths not marked FAIL may handle I/O at the same time.
|
|
Requests are distributed between paths to equalize load.
|
|
For capable devices it allows to utilize the bandwidth of all paths.
|
|
In Active/Read mode all paths not marked FAIL may handle reads at the same time,
|
|
but unlike in Active/Active mode only one path handles write requests at any
|
|
point in time.
|
|
It allows to closer follow the original write request order if the layer above
|
|
needs it for data consistency (not waiting for requisite write completion
|
|
before sending dependent write).
|
|
.Pp
|
|
When new devices are added to the system the
|
|
.Nm MULTIPATH
|
|
GEOM class is given an opportunity to taste these new devices.
|
|
If a new
|
|
device has a
|
|
.Nm MULTIPATH
|
|
on-disk metadata label, the device is either used to create a new
|
|
.Nm MULTIPATH
|
|
GEOM, or added to the list of paths for an existing
|
|
.Nm MULTIPATH
|
|
GEOM.
|
|
.Pp
|
|
It is this mechanism that works reasonably with
|
|
.Xr isp 4
|
|
and
|
|
.Xr mpt 4
|
|
based Fibre Channel disk devices.
|
|
For these devices, when a device disappears
|
|
(due to e.g., a cable pull or power failure to a switch), the device is
|
|
proactively marked as gone and I/O to it failed.
|
|
This causes the
|
|
.Nm MULTIPATH
|
|
failure event just described.
|
|
.Pp
|
|
When Fibre Channel events inform either
|
|
.Xr isp 4
|
|
or
|
|
.Xr mpt 4
|
|
host bus adapters that new devices may have arrived (e.g., the arrival
|
|
of an RSCN event from the Fabric Domain Controller), they can cause
|
|
a rescan to occur and cause the attachment and configuration of any
|
|
(now) new devices to occur, causing the taste event described above.
|
|
.Pp
|
|
This means that this multipath architecture is not a one-shot path
|
|
failover, but can be considered to be steady state as long as failed
|
|
paths are repaired (automatically or otherwise).
|
|
.Pp
|
|
Automatic rescanning is not a requirement.
|
|
Nor is Fibre Channel.
|
|
The
|
|
same failover mechanisms work equally well for traditional "Parallel"
|
|
SCSI but may require manual intervention with
|
|
.Xr camcontrol 8
|
|
to cause the reattachment of repaired device links.
|
|
.Sh EXAMPLES
|
|
The following example shows how to use
|
|
.Xr camcontrol 8
|
|
to find possible multiple path devices and to create a
|
|
.Nm MULTIPATH
|
|
GEOM class for them.
|
|
.Bd -literal -offset indent
|
|
mysys# camcontrol devlist
|
|
<ECNCTX @WESTVILLE > at scbus0 target 0 lun 0 (da0,pass0)
|
|
<ECNCTX @WESTVILLE > at scbus0 target 0 lun 1 (da1,pass1)
|
|
<ECNCTX @WESTVILLE > at scbus1 target 0 lun 0 (da2,pass2)
|
|
<ECNCTX @WESTVILLE > at scbus1 target 0 lun 1 (da3,pass3)
|
|
mysys# camcontrol inquiry da0 -S
|
|
ECNTX0LUN000000SER10ac0d01
|
|
mysys# camcontrol inquiry da2 -S
|
|
ECNTX0LUN000000SER10ac0d01
|
|
.Ed
|
|
.Pp
|
|
Now that you have used the Serial Number to compare two disk paths
|
|
it is not entirely unreasonable to conclude that these are multiple
|
|
paths to the same device.
|
|
However, only the user who is familiar
|
|
with their storage is qualified to make this judgement.
|
|
.Pp
|
|
You can then use the
|
|
.Nm
|
|
command to label and create a
|
|
.Nm MULTIPATH
|
|
GEOM provider named
|
|
.Ar FRED .
|
|
.Bd -literal -offset indent
|
|
gmultipath label -v FRED /dev/da0 /dev/da2
|
|
disklabel -Brw /dev/multipath/FRED auto
|
|
newfs /dev/multipath/FREDa
|
|
mount /dev/multipath/FREDa /mnt....
|
|
.Ed
|
|
.Pp
|
|
The resultant console output looks something like:
|
|
.Bd -literal -offset indent
|
|
GEOM_MULTIPATH: da0 added to FRED
|
|
GEOM_MULTIPATH: da0 is now active path in FRED
|
|
GEOM_MULTIPATH: da2 added to FRED
|
|
.Ed
|
|
.Sh SEE ALSO
|
|
.Xr geom 4 ,
|
|
.Xr isp 4 ,
|
|
.Xr mpt 4 ,
|
|
.Xr loader.conf 5 ,
|
|
.Xr camcontrol 8 ,
|
|
.Xr geom 8 ,
|
|
.Xr mount 8 ,
|
|
.Xr newfs 8 ,
|
|
.Xr sysctl 8
|
|
.Sh AUTHORS
|
|
.An Matthew Jacob Aq Mt mjacob@FreeBSD.org
|
|
.An Alexander Motin Aq Mt mav@FreeBSD.org
|