fd06117525
The -r flag is ignored by the FreeBSD implementation of bsdlabel(8) (also called disklabel(8) in the past). Remove its use from examples and tests in the tree. This commit does not touch historical documentation under share/doc/smm and files under contrib/netbsd-tests. Reviewed by: imp MFC after: 2 weeks Approved by: imp (src) Fixes: 57dfbec57b6d More axe-work: Differential Revision: https://reviews.freebsd.org/D34585
378 lines
11 KiB
Groff
378 lines
11 KiB
Groff
.\" Copyright (c) 2007 Matthew Jacob
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
|
|
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
|
|
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
.\" SUCH DAMAGE.
|
|
.\"
|
|
.\" $FreeBSD$
|
|
.\"
|
|
.Dd March 17, 2022
|
|
.Dt GMULTIPATH 8
|
|
.Os
|
|
.Sh NAME
|
|
.Nm gmultipath
|
|
.Nd "disk multipath control utility"
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Cm create
|
|
.Op Fl ARv
|
|
.Ar name
|
|
.Ar prov ...
|
|
.Nm
|
|
.Cm label
|
|
.Op Fl ARv
|
|
.Ar name
|
|
.Ar prov ...
|
|
.Nm
|
|
.Cm configure
|
|
.Op Fl APRv
|
|
.Ar name
|
|
.Nm
|
|
.Cm add
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm remove
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm fail
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm restore
|
|
.Op Fl v
|
|
.Ar name prov
|
|
.Nm
|
|
.Cm rotate
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm prefer
|
|
.Op Fl v
|
|
.Ar name
|
|
.Ar prov
|
|
.Nm
|
|
.Cm getactive
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm destroy
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm stop
|
|
.Op Fl v
|
|
.Ar name
|
|
.Nm
|
|
.Cm clear
|
|
.Op Fl v
|
|
.Ar prov ...
|
|
.Nm
|
|
.Cm list
|
|
.Nm
|
|
.Cm status
|
|
.Nm
|
|
.Cm load
|
|
.Nm
|
|
.Cm unload
|
|
.Sh DESCRIPTION
|
|
The
|
|
.Nm
|
|
utility is used for device multipath configuration.
|
|
.Pp
|
|
The multipath device can be configured using two different methods:
|
|
.Dq manual
|
|
or
|
|
.Dq automatic .
|
|
When using the
|
|
.Dq manual
|
|
method, no metadata are stored on the devices, so the multipath
|
|
device has to be configured by hand every time it is needed.
|
|
Additional device paths also will not be detected automatically.
|
|
The
|
|
.Dq automatic
|
|
method uses on-disk metadata to detect device and all its paths.
|
|
Metadata use the last sector of the underlying disk device and
|
|
include device name and UUID.
|
|
The UUID guarantees uniqueness in a shared storage environment
|
|
but is in general too cumbersome to use.
|
|
The name is what is exported via the device interface.
|
|
.Pp
|
|
The first argument to
|
|
.Nm
|
|
indicates an action to be performed:
|
|
.Bl -tag -width ".Cm destroy"
|
|
.It Cm create
|
|
Create multipath device with
|
|
.Dq manual
|
|
method without writing any on-disk metadata.
|
|
It is up to administrator, how to properly identify device paths.
|
|
Kernel will only check that all given providers have same media and
|
|
sector sizes.
|
|
.Pp
|
|
.Fl A
|
|
option enables Active/Active mode,
|
|
.Fl R
|
|
option enables Active/Read mode, otherwise Active/Passive mode is used
|
|
by default.
|
|
.It Cm label
|
|
Create multipath device with
|
|
.Dq automatic
|
|
method.
|
|
Label the first given provider with on-disk metadata using the specified
|
|
.Ar name .
|
|
The rest of given providers will be retasted to detect these metadata.
|
|
It reliably protects against specifying unrelated providers.
|
|
Providers with no matching metadata detected will not be added to the device.
|
|
.Pp
|
|
.Fl A
|
|
option enables Active/Active mode,
|
|
.Fl R
|
|
option enables Active/Read mode, otherwise Active/Passive mode is used
|
|
by default.
|
|
.It Cm configure
|
|
Configure the given multipath device.
|
|
.Pp
|
|
.Fl A
|
|
option enables Active/Active mode,
|
|
.Fl P
|
|
option enables Active/Passive mode,
|
|
.Fl R
|
|
option enables Active/Read mode.
|
|
.It Cm add
|
|
Add the given provider as a path to the given multipath device.
|
|
Should normally be used only for devices created with
|
|
.Dq manual
|
|
method, unless you know what you are doing (you are sure that it is another
|
|
device path, but tasting its metadata in regular
|
|
.Dq automatic
|
|
way is not possible).
|
|
.It Cm remove
|
|
Remove the given provider as a path from the given multipath device.
|
|
If the last path removed, the multipath device will be destroyed.
|
|
.It Cm fail
|
|
Mark specified provider as a path of the specified multipath device as failed.
|
|
If there are other paths present, new requests will be forwarded there.
|
|
.It Cm restore
|
|
Mark specified provider as a path of the specified multipath device as
|
|
operational, allowing it to handle requests.
|
|
.It Cm rotate
|
|
Change the active provider/path to the next available provider in Active/Passive mode.
|
|
.It Cm prefer
|
|
Change the active provider/path to the specified provider in Active/Passive mode.
|
|
.It Cm getactive
|
|
Get the currently active provider(s)/path(s).
|
|
.It Cm destroy
|
|
Destroy the given multipath device clearing metadata.
|
|
.It Cm stop
|
|
Stop the given multipath device without clearing metadata.
|
|
.It Cm clear
|
|
Clear metadata on the given provider.
|
|
.It Cm list
|
|
See
|
|
.Xr geom 8 .
|
|
.It Cm status
|
|
See
|
|
.Xr geom 8 .
|
|
.It Cm load
|
|
See
|
|
.Xr geom 8 .
|
|
.It Cm unload
|
|
See
|
|
.Xr geom 8 .
|
|
.El
|
|
.Sh SYSCTL VARIABLES
|
|
The following
|
|
.Xr sysctl 8
|
|
variable can be used to control the behavior of the
|
|
.Nm MULTIPATH
|
|
GEOM class.
|
|
.Bl -tag -width indent
|
|
.It Va kern.geom.multipath.debug : No 0
|
|
Debug level of the
|
|
.Nm MULTIPATH
|
|
GEOM class.
|
|
This can be set to 0 (default) or 1 to disable or enable various
|
|
forms of chattiness.
|
|
.It Va kern.geom.multipath.exclusive : No 1
|
|
Open underlying providers exclusively, preventing individual paths access.
|
|
.El
|
|
.Sh EXIT STATUS
|
|
Exit status is 0 on success, and 1 if the command fails.
|
|
.Sh MULTIPATH ARCHITECTURE
|
|
This is a multiple path architecture with no device knowledge or
|
|
presumptions other than size matching built in.
|
|
Therefore the user must exercise some care
|
|
in selecting providers that do indeed represent multiple paths to the
|
|
same underlying disk device.
|
|
The reason for this is that there are several
|
|
criteria across multiple underlying transport types that can
|
|
.Ar indicate
|
|
identity, but in all respects such identity can rarely be considered
|
|
.Ar definitive .
|
|
.Pp
|
|
For example, if you use the World Wide Port Name of a Fibre Channel
|
|
disk object you might believe that two disks that have the same WWPN
|
|
on different paths (or even disjoint fabrics) might be considered
|
|
the same disk.
|
|
Nearly always this would be a safe assumption, until
|
|
you realize that a WWPN, like an Ethernet MAC address, is a soft
|
|
programmable entity, and that a misconfigured Director Class switch
|
|
could lead you to believe incorrectly that you have found multiple
|
|
paths to the same device.
|
|
This is an extreme and theoretical case, but
|
|
it is possible enough to indicate that the policy for deciding which
|
|
of multiple pathnames refer to the same device should be left to the
|
|
system operator who will use tools and knowledge of their own storage
|
|
subsystem to make the correct configuration selection.
|
|
.Pp
|
|
There are Active/Passive, Active/Read and Active/Active operation modes
|
|
supported.
|
|
In Active/Passive mode only one path has I/O moving on it
|
|
at any point in time.
|
|
This I/O continues until an I/O is returned with
|
|
a generic I/O error or a "Nonexistent Device" error.
|
|
When this occurs, that path is marked FAIL, the next path
|
|
in a list is selected as active and the failed I/O reissued.
|
|
In Active/Active mode all paths not marked FAIL may handle I/O at the same time.
|
|
Requests are distributed between paths to equalize load.
|
|
For capable devices it allows the utilisation of the bandwidth available on all paths.
|
|
In Active/Read mode all paths not marked FAIL may handle reads at the same time,
|
|
but unlike in Active/Active mode only one path handles write requests at any
|
|
point in time; closely following the original write request order if the layer
|
|
above needs it for data consistency (not waiting for requisite write completion
|
|
before sending dependent write).
|
|
.Pp
|
|
When new devices are added to the system the
|
|
.Nm MULTIPATH
|
|
GEOM class is given an opportunity to taste these new devices.
|
|
If a new
|
|
device has a
|
|
.Nm MULTIPATH
|
|
on-disk metadata label, the device is either used to create a new
|
|
.Nm MULTIPATH
|
|
GEOM, or added to the list of paths for an existing
|
|
.Nm MULTIPATH
|
|
GEOM.
|
|
.Pp
|
|
It is this mechanism that works reasonably with
|
|
.Xr isp 4
|
|
and
|
|
.Xr mpt 4
|
|
based Fibre Channel disk devices.
|
|
For these devices, when a device disappears
|
|
(due to e.g., a cable pull or power failure to a switch), the device is
|
|
proactively marked as gone and I/O to it failed.
|
|
This causes the
|
|
.Nm MULTIPATH
|
|
failure event just described.
|
|
.Pp
|
|
When Fibre Channel events inform either
|
|
.Xr isp 4
|
|
or
|
|
.Xr mpt 4
|
|
host bus adapters that new devices may have arrived (e.g., the arrival
|
|
of an RSCN event from the Fabric Domain Controller), they can cause
|
|
a rescan to occur and cause the attachment and configuration of any
|
|
(now) new devices to occur, causing the taste event described above.
|
|
.Pp
|
|
This means that this multipath architecture is not a one-shot path
|
|
failover, but can be considered to be steady state as long as failed
|
|
paths are repaired (automatically or otherwise).
|
|
.Pp
|
|
Automatic rescanning is not a requirement.
|
|
Nor is Fibre Channel.
|
|
The
|
|
same failover mechanisms work equally well for traditional "Parallel"
|
|
SCSI but may require manual intervention with
|
|
.Xr camcontrol 8
|
|
to cause the reattachment of repaired device links.
|
|
.Sh EXAMPLES
|
|
The following example shows how to use
|
|
.Xr camcontrol 8
|
|
to find possible multiple path devices and to create a
|
|
.Nm MULTIPATH
|
|
GEOM class for them.
|
|
.Bd -literal -offset indent
|
|
mysys# camcontrol devlist
|
|
<ECNCTX @WESTVILLE > at scbus0 target 0 lun 0 (da0,pass0)
|
|
<ECNCTX @WESTVILLE > at scbus0 target 0 lun 1 (da1,pass1)
|
|
<ECNCTX @WESTVILLE > at scbus1 target 0 lun 0 (da2,pass2)
|
|
<ECNCTX @WESTVILLE > at scbus1 target 0 lun 1 (da3,pass3)
|
|
mysys# camcontrol inquiry da0 -S
|
|
ECNTX0LUN000000SER10ac0d01
|
|
mysys# camcontrol inquiry da2 -S
|
|
ECNTX0LUN000000SER10ac0d01
|
|
.Ed
|
|
.Pp
|
|
Now that you have used the Serial Number to compare two disk paths
|
|
it is not entirely unreasonable to conclude that these are multiple
|
|
paths to the same device.
|
|
However, only the user who is familiar
|
|
with their storage is qualified to make this judgement.
|
|
.Pp
|
|
You can then use the
|
|
.Nm
|
|
command to label and create a
|
|
.Nm MULTIPATH
|
|
GEOM provider named
|
|
.Ar FRED .
|
|
.Bd -literal -offset indent
|
|
gmultipath label -v FRED /dev/da0 /dev/da2
|
|
disklabel -Bw /dev/multipath/FRED auto
|
|
newfs /dev/multipath/FREDa
|
|
mount /dev/multipath/FREDa /mnt....
|
|
.Ed
|
|
.Pp
|
|
The resultant console output looks something like:
|
|
.Bd -literal -offset indent
|
|
GEOM_MULTIPATH: da0 added to FRED
|
|
GEOM_MULTIPATH: da0 is now active path in FRED
|
|
GEOM_MULTIPATH: da2 added to FRED
|
|
.Ed
|
|
.Pp
|
|
To load the
|
|
.Nm
|
|
module at boot time, add this entry to
|
|
.Pa /boot/loader.conf :
|
|
.Bd -literal -offset ident
|
|
geom_multipath_load="YES"
|
|
.Ed
|
|
.Sh SEE ALSO
|
|
.Xr geom 4 ,
|
|
.Xr isp 4 ,
|
|
.Xr mpt 4 ,
|
|
.Xr loader.conf 5 ,
|
|
.Xr camcontrol 8 ,
|
|
.Xr geom 8 ,
|
|
.Xr mount 8 ,
|
|
.Xr newfs 8 ,
|
|
.Xr sysctl 8
|
|
.Sh HISTORY
|
|
The
|
|
.Nm
|
|
utility first appeared in
|
|
.Fx 7.0
|
|
.Sh AUTHORS
|
|
.An Matthew Jacob Aq Mt mjacob@FreeBSD.org
|
|
.An Alexander Motin Aq Mt mav@FreeBSD.org
|