d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
40ea77a036
freebsd-dev/sys/geom/multipath/g_multipath.c

1427 lines
35 KiB
C
Raw Normal View History

First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
/*-
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
* Copyright (c) 2011 Alexander Motin <mav@FreeBSD.org>
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
* Copyright (c) 2006-2007 Matthew Jacob <mjacob@FreeBSD.org>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
/*
* Based upon work by Pawel Jakub Dawidek <pjd@FreeBSD.org> for all of the
* fine geom examples, and by Poul Henning Kamp <phk@FreeBSD.org> for GEOM
* itself, all of which is most gratefully acknowledged.
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/systm.h>
#include <sys/kernel.h>
#include <sys/module.h>
#include <sys/lock.h>
#include <sys/mutex.h>
#include <sys/bio.h>
Include sys/sbuf.h directly. Reviewed by: pjd
2011-07-11 05:22:31 +00:00
#include <sys/sbuf.h>
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
#include <sys/sysctl.h>
#include <sys/kthread.h>
#include <sys/malloc.h>
#include <geom/geom.h>
#include <geom/multipath/g_multipath.h>
Add some FEATURE macros for various GEOM classes. No FreeBSD version bump, the userland application to query the features will be committed last and can serve as an indication of the availablility if needed. Sponsored by: Google Summer of Code 2010 Submitted by: kibab Reviewed by: silence on geom@ during 2 weeks X-MFC after: to be determined in last commit with code from this project
2011-02-25 10:24:35 +00:00
FEATURE(geom_multipath, "GEOM multipath support");
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
SYSCTL_DECL(_kern_geom);
Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs. The SYSCTL_NODE macro defines a list that stores all child-elements of that node. If there's no SYSCTL_DECL macro anywhere else, there's no reason why it shouldn't be static.
2011-11-07 15:43:11 +00:00
static SYSCTL_NODE(_kern_geom, OID_AUTO, multipath, CTLFLAG_RW, 0,
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
"GEOM_MULTIPATH tunables");
static u_int g_multipath_debug = 0;
SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, debug, CTLFLAG_RW,
&g_multipath_debug, 0, "Debug level");
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
static u_int g_multipath_exclusive = 1;
SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, exclusive, CTLFLAG_RW,
&g_multipath_exclusive, 0, "Exclusively open providers");
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static enum {
GKT_NIL,
GKT_RUN,
GKT_DIE
} g_multipath_kt_state;
static struct bio_queue_head gmtbq;
static struct mtx gmtbq_mtx;
static void g_multipath_orphan(struct g_consumer *);
static void g_multipath_start(struct bio *);
static void g_multipath_done(struct bio *);
static void g_multipath_done_error(struct bio *);
static void g_multipath_kt(void *);
static int g_multipath_destroy(struct g_geom *);
static int
g_multipath_destroy_geom(struct gctl_req *, struct g_class *, struct g_geom *);
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
static struct g_geom *g_multipath_find_geom(struct g_class *, const char *);
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
static int g_multipath_rotate(struct g_geom *);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static g_taste_t g_multipath_taste;
static g_ctl_req_t g_multipath_config;
static g_init_t g_multipath_init;
static g_fini_t g_multipath_fini;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
static g_dumpconf_t g_multipath_dumpconf;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
struct g_class g_multipath_class = {
.name = G_MULTIPATH_CLASS_NAME,
.version = G_VERSION,
.ctlreq = g_multipath_config,
.taste = g_multipath_taste,
.destroy_geom = g_multipath_destroy_geom,
.init = g_multipath_init,
.fini = g_multipath_fini
};
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
#define MP_FAIL 0x00000001
#define MP_LOST 0x00000002
#define MP_NEW 0x00000004
#define MP_POSTED 0x00000008
#define MP_BAD (MP_FAIL | MP_LOST | MP_NEW)
#define MP_IDLE 0x00000010
#define MP_IDLE_MASK 0xfffffff0
static int
g_multipath_good(struct g_geom *gp)
{
struct g_consumer *cp;
int n = 0;
LIST_FOREACH(cp, &gp->consumer, consumer) {
if ((cp->index & MP_BAD) == 0)
n++;
}
return (n);
}
static void
g_multipath_fault(struct g_consumer *cp, int cause)
{
struct g_multipath_softc *sc;
struct g_consumer *lcp;
struct g_geom *gp;
gp = cp->geom;
sc = gp->softc;
cp->index |= cause;
if (g_multipath_good(gp) == 0 && sc->sc_ndisks > 0) {
LIST_FOREACH(lcp, &gp->consumer, consumer) {
if (lcp->provider == NULL ||
(lcp->index & (MP_LOST | MP_NEW)))
continue;
if (sc->sc_ndisks > 1 && lcp == cp)
continue;
printf("GEOM_MULTIPATH: "
"all paths in %s were marked FAIL, restore %s\n",
sc->sc_name, lcp->provider->name);
lcp->index &= ~MP_FAIL;
}
}
if (cp != sc->sc_active)
return;
sc->sc_active = NULL;
LIST_FOREACH(lcp, &gp->consumer, consumer) {
if ((lcp->index & MP_BAD) == 0) {
sc->sc_active = lcp;
break;
}
}
if (sc->sc_active == NULL) {
printf("GEOM_MULTIPATH: out of providers for %s\n",
sc->sc_name);
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
} else if (sc->sc_active_active != 1) {
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
printf("GEOM_MULTIPATH: %s is now active path in %s\n",
sc->sc_active->provider->name, sc->sc_name);
}
}
static struct g_consumer *
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
g_multipath_choose(struct g_geom *gp, struct bio *bp)
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
{
struct g_multipath_softc *sc;
struct g_consumer *best, *cp;
sc = gp->softc;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if (sc->sc_active_active == 0 ||
(sc->sc_active_active == 2 && bp->bio_cmd != BIO_READ))
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
return (sc->sc_active);
best = NULL;
LIST_FOREACH(cp, &gp->consumer, consumer) {
if (cp->index & MP_BAD)
continue;
cp->index += MP_IDLE;
if (best == NULL || cp->private < best->private ||
(cp->private == best->private && cp->index > best->index))
best = cp;
}
if (best != NULL)
best->index &= ~MP_IDLE_MASK;
return (best);
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static void
g_mpd(void *arg, int flags __unused)
{
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_geom *gp;
struct g_multipath_softc *sc;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
struct g_consumer *cp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
int w;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_topology_assert();
cp = arg;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
gp = cp->geom;
if (cp->acr > 0 || cp->acw > 0 || cp->ace > 0) {
w = cp->acw;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_access(cp, -cp->acr, -cp->acw, -cp->ace);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (w > 0 && cp->provider != NULL &&
(cp->provider->geom->flags & G_GEOM_WITHER) == 0) {
g_post_event(g_mpd, cp, M_WAITOK, NULL);
return;
}
}
sc = gp->softc;
mtx_lock(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
if (cp->provider) {
printf("GEOM_MULTIPATH: %s removed from %s\n",
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
cp->provider->name, gp->name);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_detach(cp);
}
g_destroy_consumer(cp);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
mtx_unlock(&sc->sc_mtx);
if (LIST_EMPTY(&gp->consumer))
g_multipath_destroy(gp);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
static void
g_multipath_orphan(struct g_consumer *cp)
{
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
uintptr_t *cnt;
g_topology_assert();
printf("GEOM_MULTIPATH: %s in %s was disconnected\n",
cp->provider->name, cp->geom->name);
sc = cp->geom->softc;
cnt = (uintptr_t *)&cp->private;
mtx_lock(&sc->sc_mtx);
sc->sc_ndisks--;
g_multipath_fault(cp, MP_LOST);
if (*cnt == 0 && (cp->index & MP_POSTED) == 0) {
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
cp->index |= MP_POSTED;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
mtx_unlock(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_mpd(cp, 0);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
} else
mtx_unlock(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
static void
g_multipath_start(struct bio *bp)
{
struct g_multipath_softc *sc;
struct g_geom *gp;
struct g_consumer *cp;
struct bio *cbp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
uintptr_t *cnt;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
gp = bp->bio_to->geom;
sc = gp->softc;
KASSERT(sc != NULL, ("NULL sc"));
cbp = g_clone_bio(bp);
if (cbp == NULL) {
g_io_deliver(bp, ENOMEM);
return;
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
mtx_lock(&sc->sc_mtx);
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
cp = g_multipath_choose(gp, bp);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (cp == NULL) {
mtx_unlock(&sc->sc_mtx);
g_destroy_bio(cbp);
g_io_deliver(bp, ENXIO);
return;
}
if ((uintptr_t)bp->bio_driver1 < sc->sc_ndisks)
bp->bio_driver1 = (void *)(uintptr_t)sc->sc_ndisks;
cnt = (uintptr_t *)&cp->private;
(*cnt)++;
mtx_unlock(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
cbp->bio_done = g_multipath_done;
g_io_request(cbp, cp);
}
static void
g_multipath_done(struct bio *bp)
{
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
struct g_consumer *cp;
uintptr_t *cnt;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
if (bp->bio_error == ENXIO || bp->bio_error == EIO) {
mtx_lock(&gmtbq_mtx);
bioq_insert_tail(&gmtbq, bp);
mtx_unlock(&gmtbq_mtx);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
wakeup(&g_multipath_kt_state);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
} else {
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
cp = bp->bio_from;
sc = cp->geom->softc;
cnt = (uintptr_t *)&cp->private;
mtx_lock(&sc->sc_mtx);
(*cnt)--;
if (*cnt == 0 && (cp->index & MP_LOST)) {
cp->index |= MP_POSTED;
mtx_unlock(&sc->sc_mtx);
g_post_event(g_mpd, cp, M_WAITOK, NULL);
} else
mtx_unlock(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_std_done(bp);
}
}
static void
g_multipath_done_error(struct bio *bp)
{
struct bio *pbp;
struct g_geom *gp;
struct g_multipath_softc *sc;
struct g_consumer *cp;
struct g_provider *pp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
uintptr_t *cnt;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
/*
* If we had a failure, we have to check first to see
* whether the consumer it failed on was the currently
* active consumer (i.e., this is the first in perhaps
* a number of failures). If so, we then switch consumers
* to the next available consumer.
*/
pbp = bp->bio_parent;
gp = pbp->bio_to->geom;
sc = gp->softc;
cp = bp->bio_from;
pp = cp->provider;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
cnt = (uintptr_t *)&cp->private;
mtx_lock(&sc->sc_mtx);
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if ((cp->index & MP_FAIL) == 0) {
printf("GEOM_MULTIPATH: Error %d, %s in %s marked FAIL\n",
bp->bio_error, pp->name, sc->sc_name);
g_multipath_fault(cp, MP_FAIL);
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
(*cnt)--;
if (*cnt == 0 && (cp->index & (MP_LOST | MP_POSTED)) == MP_LOST) {
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
cp->index |= MP_POSTED;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
mtx_unlock(&sc->sc_mtx);
g_post_event(g_mpd, cp, M_WAITOK, NULL);
} else
mtx_unlock(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
/*
* If we can fruitfully restart the I/O, do so.
*/
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (pbp->bio_children < (uintptr_t)pbp->bio_driver1) {
pbp->bio_inbed++;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_destroy_bio(bp);
g_multipath_start(pbp);
} else {
g_std_done(bp);
}
}
static void
g_multipath_kt(void *arg)
{
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_multipath_kt_state = GKT_RUN;
mtx_lock(&gmtbq_mtx);
while (g_multipath_kt_state == GKT_RUN) {
for (;;) {
struct bio *bp;
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
bp = bioq_takefirst(&gmtbq);
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (bp == NULL)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
break;
mtx_unlock(&gmtbq_mtx);
g_multipath_done_error(bp);
mtx_lock(&gmtbq_mtx);
}
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if (g_multipath_kt_state != GKT_RUN)
break;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
msleep(&g_multipath_kt_state, &gmtbq_mtx, PRIBIO,
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
"gkt:wait", 0);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
mtx_unlock(&gmtbq_mtx);
wakeup(&g_multipath_kt_state);
Rename the kthread_xxx (e.g. kthread_create()) calls to kproc_xxx as they actually make whole processes. Thos makes way for us to add REAL kthread_create() and friends that actually make theads. it turns out that most of these calls actually end up being moved back to the thread version when it's added. but we need to make this cosmetic change first. I'd LOVE to do this rename in 7.0 so that we can eventually MFC the new kthread_xxx() calls.
2007-10-20 23:23:23 +00:00
kproc_exit(0);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
static int
g_multipath_access(struct g_provider *pp, int dr, int dw, int de)
{
struct g_geom *gp;
struct g_consumer *cp, *badcp = NULL;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
int error;
gp = pp->geom;
LIST_FOREACH(cp, &gp->consumer, consumer) {
error = g_access(cp, dr, dw, de);
if (error) {
badcp = cp;
goto fail;
}
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc = gp->softc;
sc->sc_opened += dr + dw + de;
if (sc->sc_stopping && sc->sc_opened == 0)
g_multipath_destroy(gp);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (0);
fail:
LIST_FOREACH(cp, &gp->consumer, consumer) {
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (cp == badcp)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
break;
(void) g_access(cp, -dr, -dw, -de);
}
return (error);
}
static struct g_geom *
g_multipath_create(struct g_class *mp, struct g_multipath_metadata *md)
{
struct g_multipath_softc *sc;
struct g_geom *gp;
struct g_provider *pp;
g_topology_assert();
LIST_FOREACH(gp, &mp->geom, geom) {
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc = gp->softc;
if (sc == NULL || sc->sc_stopping)
continue;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
if (strcmp(gp->name, md->md_name) == 0) {
printf("GEOM_MULTIPATH: name %s already exists\n",
md->md_name);
return (NULL);
}
}
- Don't pass geom and provider names as format strings. - Add __printflike() attributes. - Remove an extra argument for the g_new_geomf() call in swapongeom_ev(). Reviewed by: pjd
2012-11-20 12:32:18 +00:00
gp = g_new_geomf(mp, "%s", md->md_name);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
sc = g_malloc(sizeof(*sc), M_WAITOK | M_ZERO);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
mtx_init(&sc->sc_mtx, "multipath", NULL, MTX_DEF);
memcpy(sc->sc_uuid, md->md_uuid, sizeof (sc->sc_uuid));
memcpy(sc->sc_name, md->md_name, sizeof (sc->sc_name));
sc->sc_active_active = md->md_active_active;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
gp->softc = sc;
gp->start = g_multipath_start;
gp->orphan = g_multipath_orphan;
gp->access = g_multipath_access;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
gp->dumpconf = g_multipath_dumpconf;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
pp = g_new_providerf(gp, "multipath/%s", md->md_name);
Merge GEOM direct dispatch changes from the projects/camlock branch. When safety requirements are met, it allows to avoid passing I/O requests to GEOM g_up/g_down thread, executing them directly in the caller context. That allows to avoid CPU bottlenecks in g_up/g_down threads, plus avoid several context switches per I/O. The defined now safety requirements are: - caller should not hold any locks and should be reenterable; - callee should not depend on GEOM dual-threaded concurency semantics; - on the way down, if request is unmapped while callee doesn't support it, the context should be sleepable; - kernel thread stack usage should be below 50%. To keep compatibility with GEOM classes not meeting above requirements new provider and consumer flags added: - G_CF_DIRECT_SEND -- consumer code meets caller requirements (request); - G_CF_DIRECT_RECEIVE -- consumer code meets callee requirements (done); - G_PF_DIRECT_SEND -- provider code meets caller requirements (done); - G_PF_DIRECT_RECEIVE -- provider code meets callee requirements (request). Capable GEOM class can set them, allowing direct dispatch in cases where it is safe. If any of requirements are not met, request is queued to g_up or g_down thread same as before. Such GEOM classes were reviewed and updated to support direct dispatch: CONCAT, DEV, DISK, GATE, MD, MIRROR, MULTIPATH, NOP, PART, RAID, STRIPE, VFS, ZERO, ZFS::VDEV, ZFS::ZVOL, all classes based on g_slice KPI (LABEL, MAP, FLASHMAP, etc). To declare direct completion capability disk(9) KPI got new flag equivalent to G_PF_DIRECT_SEND -- DISKFLAG_DIRECT_COMPLETION. da(4) and ada(4) disk drivers got it set now thanks to earlier CAM locking work. This change more then twice increases peak block storage performance on systems with manu CPUs, together with earlier CAM locking changes reaching more then 1 million IOPS (512 byte raw reads from 16 SATA SSDs on 4 HBAs to 256 user-level threads). Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-22 08:22:19 +00:00
pp->flags |= G_PF_DIRECT_SEND | G_PF_DIRECT_RECEIVE;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (md->md_size != 0) {
pp->mediasize = md->md_size -
((md->md_uuid[0] != 0) ? md->md_sectorsize : 0);
pp->sectorsize = md->md_sectorsize;
}
sc->sc_pp = pp;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_error_provider(pp, 0);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
printf("GEOM_MULTIPATH: %s created\n", gp->name);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (gp);
}
static int
g_multipath_add_disk(struct g_geom *gp, struct g_provider *pp)
{
struct g_multipath_softc *sc;
struct g_consumer *cp, *nxtcp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
int error, acr, acw, ace;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_topology_assert();
sc = gp->softc;
KASSERT(sc, ("no softc"));
/*
* Make sure that the passed provider isn't already attached
*/
LIST_FOREACH(cp, &gp->consumer, consumer) {
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (cp->provider == pp)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
break;
}
if (cp) {
printf("GEOM_MULTIPATH: provider %s already attached to %s\n",
pp->name, gp->name);
return (EEXIST);
}
nxtcp = LIST_FIRST(&gp->consumer);
cp = g_new_consumer(gp);
Merge GEOM direct dispatch changes from the projects/camlock branch. When safety requirements are met, it allows to avoid passing I/O requests to GEOM g_up/g_down thread, executing them directly in the caller context. That allows to avoid CPU bottlenecks in g_up/g_down threads, plus avoid several context switches per I/O. The defined now safety requirements are: - caller should not hold any locks and should be reenterable; - callee should not depend on GEOM dual-threaded concurency semantics; - on the way down, if request is unmapped while callee doesn't support it, the context should be sleepable; - kernel thread stack usage should be below 50%. To keep compatibility with GEOM classes not meeting above requirements new provider and consumer flags added: - G_CF_DIRECT_SEND -- consumer code meets caller requirements (request); - G_CF_DIRECT_RECEIVE -- consumer code meets callee requirements (done); - G_PF_DIRECT_SEND -- provider code meets caller requirements (done); - G_PF_DIRECT_RECEIVE -- provider code meets callee requirements (request). Capable GEOM class can set them, allowing direct dispatch in cases where it is safe. If any of requirements are not met, request is queued to g_up or g_down thread same as before. Such GEOM classes were reviewed and updated to support direct dispatch: CONCAT, DEV, DISK, GATE, MD, MIRROR, MULTIPATH, NOP, PART, RAID, STRIPE, VFS, ZERO, ZFS::VDEV, ZFS::ZVOL, all classes based on g_slice KPI (LABEL, MAP, FLASHMAP, etc). To declare direct completion capability disk(9) KPI got new flag equivalent to G_PF_DIRECT_SEND -- DISKFLAG_DIRECT_COMPLETION. da(4) and ada(4) disk drivers got it set now thanks to earlier CAM locking work. This change more then twice increases peak block storage performance on systems with manu CPUs, together with earlier CAM locking changes reaching more then 1 million IOPS (512 byte raw reads from 16 SATA SSDs on 4 HBAs to 256 user-level threads). Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-22 08:22:19 +00:00
cp->flags |= G_CF_DIRECT_SEND | G_CF_DIRECT_RECEIVE;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
cp->private = NULL;
cp->index = MP_NEW;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
error = g_attach(cp, pp);
if (error != 0) {
printf("GEOM_MULTIPATH: cannot attach %s to %s",
pp->name, sc->sc_name);
g_destroy_consumer(cp);
return (error);
}
/*
* Set access permissions on new consumer to match other consumers
*/
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (sc->sc_pp) {
acr = sc->sc_pp->acr;
acw = sc->sc_pp->acw;
ace = sc->sc_pp->ace;
} else
acr = acw = ace = 0;
if (g_multipath_exclusive) {
acr++;
acw++;
ace++;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
error = g_access(cp, acr, acw, ace);
if (error) {
printf("GEOM_MULTIPATH: cannot set access in "
"attaching %s to %s (%d)\n",
pp->name, sc->sc_name, error);
g_detach(cp);
g_destroy_consumer(cp);
return (error);
}
if (sc->sc_pp != NULL && sc->sc_pp->mediasize == 0) {
sc->sc_pp->mediasize = pp->mediasize -
((sc->sc_uuid[0] != 0) ? pp->sectorsize : 0);
sc->sc_pp->sectorsize = pp->sectorsize;
}
if (sc->sc_pp != NULL &&
sc->sc_pp->stripesize == 0 && sc->sc_pp->stripeoffset == 0) {
sc->sc_pp->stripesize = pp->stripesize;
sc->sc_pp->stripeoffset = pp->stripeoffset;
}
Make GEOM MULTIPATH to report unmapped bio support if underling path report it. GEOM MULTIPATH itself never touches the data and so transparent.
2013-03-25 07:24:58 +00:00
if (sc->sc_pp != NULL)
sc->sc_pp->flags |= pp->flags & G_PF_ACCEPT_UNMAPPED;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
mtx_lock(&sc->sc_mtx);
cp->index = 0;
sc->sc_ndisks++;
mtx_unlock(&sc->sc_mtx);
printf("GEOM_MULTIPATH: %s added to %s\n",
pp->name, sc->sc_name);
if (sc->sc_active == NULL) {
sc->sc_active = cp;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if (sc->sc_active_active != 1)
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
printf("GEOM_MULTIPATH: %s is now active path in %s\n",
pp->name, sc->sc_name);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
return (0);
}
static int
g_multipath_destroy(struct g_geom *gp)
{
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
struct g_consumer *cp, *cp1;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_topology_assert();
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (gp->softc == NULL)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (ENXIO);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc = gp->softc;
if (!sc->sc_stopping) {
printf("GEOM_MULTIPATH: destroying %s\n", gp->name);
sc->sc_stopping = 1;
}
if (sc->sc_opened != 0) {
if (sc->sc_pp != NULL) {
g_wither_provider(sc->sc_pp, ENXIO);
sc->sc_pp = NULL;
}
return (EINPROGRESS);
}
LIST_FOREACH_SAFE(cp, &gp->consumer, consumer, cp1) {
mtx_lock(&sc->sc_mtx);
if ((cp->index & MP_POSTED) == 0) {
cp->index |= MP_POSTED;
mtx_unlock(&sc->sc_mtx);
g_mpd(cp, 0);
if (cp1 == NULL)
return(0); /* Recursion happened. */
} else
mtx_unlock(&sc->sc_mtx);
}
if (!LIST_EMPTY(&gp->consumer))
return (EINPROGRESS);
mtx_destroy(&sc->sc_mtx);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_free(gp->softc);
gp->softc = NULL;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
printf("GEOM_MULTIPATH: %s destroyed\n", gp->name);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_wither_geom(gp, ENXIO);
return (0);
}
static int
g_multipath_destroy_geom(struct gctl_req *req, struct g_class *mp,
struct g_geom *gp)
{
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (g_multipath_destroy(gp));
}
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
static int
g_multipath_rotate(struct g_geom *gp)
{
(g_multipath_rotate): Fix algorithm so that it does rotate over all good providers, not just the last two. PR: kern/170379 Reviewed by: mav MFC after: 2 weeks
2012-08-25 10:36:31 +00:00
struct g_consumer *lcp, *first_good_cp = NULL;
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
struct g_multipath_softc *sc = gp->softc;
(g_multipath_rotate): Fix algorithm so that it does rotate over all good providers, not just the last two. PR: kern/170379 Reviewed by: mav MFC after: 2 weeks
2012-08-25 10:36:31 +00:00
int active_cp_seen = 0;
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
g_topology_assert();
if (sc == NULL)
return (ENXIO);
LIST_FOREACH(lcp, &gp->consumer, consumer) {
if ((lcp->index & MP_BAD) == 0) {
(g_multipath_rotate): Fix algorithm so that it does rotate over all good providers, not just the last two. PR: kern/170379 Reviewed by: mav MFC after: 2 weeks
2012-08-25 10:36:31 +00:00
if (first_good_cp == NULL)
first_good_cp = lcp;
if (active_cp_seen)
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
break;
}
(g_multipath_rotate): Fix algorithm so that it does rotate over all good providers, not just the last two. PR: kern/170379 Reviewed by: mav MFC after: 2 weeks
2012-08-25 10:36:31 +00:00
if (sc->sc_active == lcp)
active_cp_seen = 1;
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
}
(g_multipath_rotate): Fix algorithm so that it does rotate over all good providers, not just the last two. PR: kern/170379 Reviewed by: mav MFC after: 2 weeks
2012-08-25 10:36:31 +00:00
if (lcp == NULL)
lcp = first_good_cp;
if (lcp && lcp != sc->sc_active) {
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc->sc_active = lcp;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if (sc->sc_active_active != 1)
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
printf("GEOM_MULTIPATH: %s is now active path in %s\n",
lcp->provider->name, sc->sc_name);
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
}
return (0);
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static void
g_multipath_init(struct g_class *mp)
{
bioq_init(&gmtbq);
mtx_init(&gmtbq_mtx, "gmtbq", NULL, MTX_DEF);
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
kproc_create(g_multipath_kt, mp, NULL, 0, 0, "g_mp_kt");
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
static void
g_multipath_fini(struct g_class *mp)
{
if (g_multipath_kt_state == GKT_RUN) {
mtx_lock(&gmtbq_mtx);
g_multipath_kt_state = GKT_DIE;
wakeup(&g_multipath_kt_state);
msleep(&g_multipath_kt_state, &gmtbq_mtx, PRIBIO,
"gmp:fini", 0);
mtx_unlock(&gmtbq_mtx);
}
}
static int
g_multipath_read_metadata(struct g_consumer *cp,
struct g_multipath_metadata *md)
{
struct g_provider *pp;
u_char *buf;
int error;
g_topology_assert();
error = g_access(cp, 1, 0, 0);
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (error != 0)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (error);
pp = cp->provider;
g_topology_unlock();
buf = g_read_data(cp, pp->mediasize - pp->sectorsize,
pp->sectorsize, &error);
g_topology_lock();
g_access(cp, -1, 0, 0);
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (buf == NULL)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (error);
multipath_metadata_decode(buf, md);
g_free(buf);
return (0);
}
static struct g_geom *
g_multipath_taste(struct g_class *mp, struct g_provider *pp, int flags __unused)
{
struct g_multipath_metadata md;
struct g_multipath_softc *sc;
struct g_consumer *cp;
struct g_geom *gp, *gp1;
int error, isnew;
g_topology_assert();
gp = g_new_geomf(mp, "multipath:taste");
gp->start = g_multipath_start;
gp->access = g_multipath_access;
gp->orphan = g_multipath_orphan;
cp = g_new_consumer(gp);
g_attach(cp, pp);
error = g_multipath_read_metadata(cp, &md);
g_detach(cp);
g_destroy_consumer(cp);
g_destroy_geom(gp);
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (error != 0)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (NULL);
gp = NULL;
if (strcmp(md.md_magic, G_MULTIPATH_MAGIC) != 0) {
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (g_multipath_debug)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
printf("%s is not MULTIPATH\n", pp->name);
return (NULL);
}
if (md.md_version != G_MULTIPATH_VERSION) {
printf("%s has version %d multipath id- this module is version "
" %d: rejecting\n", pp->name, md.md_version,
G_MULTIPATH_VERSION);
return (NULL);
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (md.md_size != 0 && md.md_size != pp->mediasize)
return (NULL);
if (md.md_sectorsize != 0 && md.md_sectorsize != pp->sectorsize)
return (NULL);
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (g_multipath_debug)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
printf("MULTIPATH: %s/%s\n", md.md_name, md.md_uuid);
/*
* Let's check if such a device already is present. We check against
* uuid alone first because that's the true distinguishor. If that
* passes, then we check for name conflicts. If there are conflicts,
* modify the name.
*
* The whole purpose of this is to solve the problem that people don't
* pick good unique names, but good unique names (like uuids) are a
* pain to use. So, we allow people to build GEOMs with friendly names
* and uuids, and modify the names in case there's a collision.
*/
sc = NULL;
LIST_FOREACH(gp, &mp->geom, geom) {
sc = gp->softc;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (sc == NULL || sc->sc_stopping)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
continue;
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (strncmp(md.md_uuid, sc->sc_uuid, sizeof(md.md_uuid)) == 0)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
break;
}
LIST_FOREACH(gp1, &mp->geom, geom) {
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (gp1 == gp)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
continue;
sc = gp1->softc;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (sc == NULL || sc->sc_stopping)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
continue;
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (strncmp(md.md_name, sc->sc_name, sizeof(md.md_name)) == 0)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
break;
}
/*
* If gp is NULL, we had no extant MULTIPATH geom with this uuid.
*
* If gp1 is *not* NULL, that means we have a MULTIPATH geom extant
* with the same name (but a different UUID).
*
* If gp is NULL, then modify the name with a random number and
* complain, but allow the creation of the geom to continue.
*
* If gp is *not* NULL, just use the geom's name as we're attaching
* this disk to the (previously generated) name.
*/
if (gp1) {
sc = gp1->softc;
if (gp == NULL) {
char buf[16];
u_long rand = random();
snprintf(buf, sizeof (buf), "%s-%lu", md.md_name, rand);
printf("GEOM_MULTIPATH: geom %s/%s exists already\n",
sc->sc_name, sc->sc_uuid);
printf("GEOM_MULTIPATH: %s will be (temporarily) %s\n",
md.md_uuid, buf);
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
strlcpy(md.md_name, buf, sizeof(md.md_name));
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
} else {
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
strlcpy(md.md_name, sc->sc_name, sizeof(md.md_name));
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
}
if (gp == NULL) {
gp = g_multipath_create(mp, &md);
if (gp == NULL) {
printf("GEOM_MULTIPATH: cannot create geom %s/%s\n",
md.md_name, md.md_uuid);
return (NULL);
}
isnew = 1;
} else {
isnew = 0;
}
sc = gp->softc;
KASSERT(sc != NULL, ("sc is NULL"));
error = g_multipath_add_disk(gp, pp);
if (error != 0) {
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (isnew)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
g_multipath_destroy(gp);
return (NULL);
}
return (gp);
}
static void
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
g_multipath_ctl_add_name(struct gctl_req *req, struct g_class *mp,
const char *name)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
{
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
struct g_geom *gp;
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
struct g_consumer *cp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_provider *pp;
const char *mpname;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static const char devpf[6] = "/dev/";
g_topology_assert();
mpname = gctl_get_asciiparam(req, "arg0");
if (mpname == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
gp = g_multipath_find_geom(mp, mpname);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", mpname);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return;
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc = gp->softc;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
- Style fixes. - Prefer strlcpy() over strncpy().
2010-02-18 22:29:35 +00:00
if (strncmp(name, devpf, 5) == 0)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
name += 5;
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
pp = g_provider_by_name(name);
if (pp == NULL) {
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
gctl_error(req, "Provider %s is invalid", name);
return;
}
/*
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
* Check to make sure parameters match.
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
*/
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
LIST_FOREACH(cp, &gp->consumer, consumer) {
if (cp->provider == pp) {
gctl_error(req, "provider %s is already there",
pp->name);
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
return;
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (sc->sc_pp != NULL && sc->sc_pp->mediasize != 0 &&
sc->sc_pp->mediasize + (sc->sc_uuid[0] != 0 ? pp->sectorsize : 0)
!= pp->mediasize) {
gctl_error(req, "Providers size mismatch %jd != %jd",
(intmax_t) sc->sc_pp->mediasize +
(sc->sc_uuid[0] != 0 ? pp->sectorsize : 0),
(intmax_t) pp->mediasize);
return;
}
if (sc->sc_pp != NULL && sc->sc_pp->sectorsize != 0 &&
sc->sc_pp->sectorsize != pp->sectorsize) {
gctl_error(req, "Providers sectorsize mismatch %u != %u",
sc->sc_pp->sectorsize, pp->sectorsize);
return;
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
/*
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
* Now add....
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
*/
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
(void) g_multipath_add_disk(gp, pp);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
}
New command "gmultipath prefer" to force selection of a specified provider in an Active/Passive configuration. Reviewed by: mav MFC after: 4 weeks
2012-08-03 14:55:35 +00:00
static void
g_multipath_ctl_prefer(struct gctl_req *req, struct g_class *mp)
{
struct g_geom *gp;
struct g_multipath_softc *sc;
struct g_consumer *cp;
const char *name, *mpname;
static const char devpf[6] = "/dev/";
int *nargs;
g_topology_assert();
mpname = gctl_get_asciiparam(req, "arg0");
if (mpname == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, mpname);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", mpname);
return;
}
sc = gp->softc;
nargs = gctl_get_paraml(req, "nargs", sizeof(*nargs));
if (nargs == NULL) {
gctl_error(req, "No 'nargs' argument");
return;
}
if (*nargs != 2) {
gctl_error(req, "missing device");
return;
}
name = gctl_get_asciiparam(req, "arg1");
if (name == NULL) {
gctl_error(req, "No 'arg1' argument");
return;
}
if (strncmp(name, devpf, 5) == 0) {
name += 5;
}
LIST_FOREACH(cp, &gp->consumer, consumer) {
if (cp->provider != NULL
&& strcmp(cp->provider->name, name) == 0)
break;
}
if (cp == NULL) {
gctl_error(req, "Provider %s not found", name);
return;
}
mtx_lock(&sc->sc_mtx);
if (cp->index & MP_BAD) {
gctl_error(req, "Consumer %s is invalid", name);
mtx_unlock(&sc->sc_mtx);
return;
}
/* Here when the consumer is present and in good shape */
sc->sc_active = cp;
if (!sc->sc_active_active)
printf("GEOM_MULTIPATH: %s now active path in %s\n",
sc->sc_active->provider->name, sc->sc_name);
mtx_unlock(&sc->sc_mtx);
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
static void
g_multipath_ctl_add(struct gctl_req *req, struct g_class *mp)
{
struct g_multipath_softc *sc;
struct g_geom *gp;
const char *mpname, *name;
mpname = gctl_get_asciiparam(req, "arg0");
if (mpname == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, mpname);
if (gp == NULL) {
gctl_error(req, "Device %s not found", mpname);
return;
}
sc = gp->softc;
name = gctl_get_asciiparam(req, "arg1");
if (name == NULL) {
gctl_error(req, "No 'arg1' argument");
return;
}
g_multipath_ctl_add_name(req, mp, name);
}
static void
g_multipath_ctl_create(struct gctl_req *req, struct g_class *mp)
{
struct g_multipath_metadata md;
struct g_multipath_softc *sc;
struct g_geom *gp;
const char *mpname, *name;
char param[16];
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
int *nargs, i, *val;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
g_topology_assert();
nargs = gctl_get_paraml(req, "nargs", sizeof(*nargs));
if (*nargs < 2) {
gctl_error(req, "wrong number of arguments.");
return;
}
mpname = gctl_get_asciiparam(req, "arg0");
if (mpname == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, mpname);
if (gp != NULL) {
gctl_error(req, "Device %s already exist", mpname);
return;
}
sc = gp->softc;
memset(&md, 0, sizeof(md));
strlcpy(md.md_magic, G_MULTIPATH_MAGIC, sizeof(md.md_magic));
md.md_version = G_MULTIPATH_VERSION;
strlcpy(md.md_name, mpname, sizeof(md.md_name));
md.md_size = 0;
md.md_sectorsize = 0;
md.md_uuid[0] = 0;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
md.md_active_active = 0;
val = gctl_get_paraml(req, "active_active", sizeof(*val));
if (val != NULL && *val != 0)
md.md_active_active = 1;
val = gctl_get_paraml(req, "active_read", sizeof(*val));
if (val != NULL && *val != 0)
md.md_active_active = 2;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
gp = g_multipath_create(mp, &md);
if (gp == NULL) {
gctl_error(req, "GEOM_MULTIPATH: cannot create geom %s/%s\n",
md.md_name, md.md_uuid);
return;
}
sc = gp->softc;
for (i = 1; i < *nargs; i++) {
snprintf(param, sizeof(param), "arg%d", i);
name = gctl_get_asciiparam(req, param);
g_multipath_ctl_add_name(req, mp, name);
}
if (sc->sc_ndisks != (*nargs - 1))
g_multipath_destroy(gp);
}
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
static void
g_multipath_ctl_configure(struct gctl_req *req, struct g_class *mp)
{
struct g_multipath_softc *sc;
struct g_geom *gp;
struct g_consumer *cp;
struct g_provider *pp;
Fix `gmultipath configure` for big-endian machines. MFC after: 1 week
2012-05-06 05:49:23 +00:00
struct g_multipath_metadata md;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
const char *name;
int error, *val;
void *buf;
g_topology_assert();
name = gctl_get_asciiparam(req, "arg0");
if (name == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, name);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", name);
return;
}
sc = gp->softc;
val = gctl_get_paraml(req, "active_active", sizeof(*val));
if (val != NULL && *val != 0)
sc->sc_active_active = 1;
val = gctl_get_paraml(req, "active_read", sizeof(*val));
if (val != NULL && *val != 0)
sc->sc_active_active = 2;
val = gctl_get_paraml(req, "active_passive", sizeof(*val));
if (val != NULL && *val != 0)
sc->sc_active_active = 0;
if (sc->sc_uuid[0] != 0 && sc->sc_active != NULL) {
cp = sc->sc_active;
pp = cp->provider;
error = g_access(cp, 1, 1, 1);
if (error != 0) {
gctl_error(req, "Can't open %s (%d)", pp->name, error);
return;
}
g_topology_unlock();
Fix `gmultipath configure` for big-endian machines. MFC after: 1 week
2012-05-06 05:49:23 +00:00
buf = g_malloc(pp->sectorsize, M_WAITOK | M_ZERO);
strlcpy(md.md_magic, G_MULTIPATH_MAGIC, sizeof(md.md_magic));
memcpy(md.md_uuid, sc->sc_uuid, sizeof (sc->sc_uuid));
strlcpy(md.md_name, name, sizeof(md.md_name));
md.md_version = G_MULTIPATH_VERSION;
md.md_size = pp->mediasize;
md.md_sectorsize = pp->sectorsize;
md.md_active_active = sc->sc_active_active;
multipath_metadata_encode(&md, buf);
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
error = g_write_data(cp, pp->mediasize - pp->sectorsize,
buf, pp->sectorsize);
g_topology_lock();
g_access(cp, -1, -1, -1);
if (error != 0)
gctl_error(req, "Can't update metadata on %s (%d)",
pp->name, error);
}
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
static void
g_multipath_ctl_fail(struct gctl_req *req, struct g_class *mp, int fail)
{
struct g_multipath_softc *sc;
struct g_geom *gp;
struct g_consumer *cp;
const char *mpname, *name;
int found;
mpname = gctl_get_asciiparam(req, "arg0");
if (mpname == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, mpname);
if (gp == NULL) {
gctl_error(req, "Device %s not found", mpname);
return;
}
sc = gp->softc;
name = gctl_get_asciiparam(req, "arg1");
if (name == NULL) {
gctl_error(req, "No 'arg1' argument");
return;
}
found = 0;
mtx_lock(&sc->sc_mtx);
LIST_FOREACH(cp, &gp->consumer, consumer) {
if (cp->provider != NULL &&
strcmp(cp->provider->name, name) == 0 &&
(cp->index & MP_LOST) == 0) {
found = 1;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if (!fail == !(cp->index & MP_FAIL))
continue;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
printf("GEOM_MULTIPATH: %s in %s is marked %s.\n",
name, sc->sc_name, fail ? "FAIL" : "OK");
if (fail) {
g_multipath_fault(cp, MP_FAIL);
} else {
cp->index &= ~MP_FAIL;
}
}
}
mtx_unlock(&sc->sc_mtx);
if (found == 0)
gctl_error(req, "Provider %s not found", name);
}
static void
g_multipath_ctl_remove(struct gctl_req *req, struct g_class *mp)
{
struct g_multipath_softc *sc;
struct g_geom *gp;
struct g_consumer *cp, *cp1;
const char *mpname, *name;
uintptr_t *cnt;
int found;
mpname = gctl_get_asciiparam(req, "arg0");
if (mpname == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, mpname);
if (gp == NULL) {
gctl_error(req, "Device %s not found", mpname);
return;
}
sc = gp->softc;
name = gctl_get_asciiparam(req, "arg1");
if (name == NULL) {
gctl_error(req, "No 'arg1' argument");
return;
}
found = 0;
mtx_lock(&sc->sc_mtx);
LIST_FOREACH_SAFE(cp, &gp->consumer, consumer, cp1) {
if (cp->provider != NULL &&
strcmp(cp->provider->name, name) == 0 &&
(cp->index & MP_LOST) == 0) {
found = 1;
printf("GEOM_MULTIPATH: removing %s from %s\n",
cp->provider->name, cp->geom->name);
sc->sc_ndisks--;
g_multipath_fault(cp, MP_LOST);
cnt = (uintptr_t *)&cp->private;
if (*cnt == 0 && (cp->index & MP_POSTED) == 0) {
cp->index |= MP_POSTED;
mtx_unlock(&sc->sc_mtx);
g_mpd(cp, 0);
if (cp1 == NULL)
return; /* Recursion happened. */
mtx_lock(&sc->sc_mtx);
}
}
}
mtx_unlock(&sc->sc_mtx);
if (found == 0)
gctl_error(req, "Provider %s not found", name);
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static struct g_geom *
g_multipath_find_geom(struct g_class *mp, const char *name)
{
struct g_geom *gp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
LIST_FOREACH(gp, &mp->geom, geom) {
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc = gp->softc;
if (sc == NULL || sc->sc_stopping)
continue;
if (strcmp(gp->name, name) == 0)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
return (gp);
}
return (NULL);
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
static void
g_multipath_ctl_stop(struct gctl_req *req, struct g_class *mp)
{
struct g_geom *gp;
const char *name;
int error;
g_topology_assert();
name = gctl_get_asciiparam(req, "arg0");
if (name == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, name);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", name);
return;
}
error = g_multipath_destroy(gp);
if (error != 0 && error != EINPROGRESS)
gctl_error(req, "failed to stop %s (err=%d)", name, error);
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static void
g_multipath_ctl_destroy(struct gctl_req *req, struct g_class *mp)
{
struct g_geom *gp;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_multipath_softc *sc;
struct g_consumer *cp;
struct g_provider *pp;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
const char *name;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
uint8_t *buf;
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
int error;
g_topology_assert();
name = gctl_get_asciiparam(req, "arg0");
if (name == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, name);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", name);
return;
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc = gp->softc;
if (sc->sc_uuid[0] != 0 && sc->sc_active != NULL) {
cp = sc->sc_active;
pp = cp->provider;
error = g_access(cp, 1, 1, 1);
if (error != 0) {
gctl_error(req, "Can't open %s (%d)", pp->name, error);
goto destroy;
}
g_topology_unlock();
buf = g_malloc(pp->sectorsize, M_WAITOK | M_ZERO);
error = g_write_data(cp, pp->mediasize - pp->sectorsize,
buf, pp->sectorsize);
g_topology_lock();
g_access(cp, -1, -1, -1);
if (error != 0)
gctl_error(req, "Can't erase metadata on %s (%d)",
pp->name, error);
}
destroy:
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
error = g_multipath_destroy(gp);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
if (error != 0 && error != EINPROGRESS)
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
gctl_error(req, "failed to destroy %s (err=%d)", name, error);
}
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
static void
g_multipath_ctl_rotate(struct gctl_req *req, struct g_class *mp)
{
struct g_geom *gp;
const char *name;
int error;
g_topology_assert();
name = gctl_get_asciiparam(req, "arg0");
if (name == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, name);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", name);
return;
}
error = g_multipath_rotate(gp);
if (error != 0) {
gctl_error(req, "failed to rotate %s (err=%d)", name, error);
}
}
static void
g_multipath_ctl_getactive(struct gctl_req *req, struct g_class *mp)
{
struct sbuf *sb;
struct g_geom *gp;
struct g_multipath_softc *sc;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
struct g_consumer *cp;
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
const char *name;
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
int empty;
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
sb = sbuf_new_auto();
g_topology_assert();
name = gctl_get_asciiparam(req, "arg0");
if (name == NULL) {
gctl_error(req, "No 'arg0' argument");
return;
}
gp = g_multipath_find_geom(mp, name);
if (gp == NULL) {
gctl_error(req, "Device %s is invalid", name);
return;
}
sc = gp->softc;
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
if (sc->sc_active_active == 1) {
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
empty = 1;
LIST_FOREACH(cp, &gp->consumer, consumer) {
if (cp->index & MP_BAD)
continue;
if (!empty)
sbuf_cat(sb, " ");
sbuf_cat(sb, cp->provider->name);
empty = 0;
}
if (empty)
sbuf_cat(sb, "none");
sbuf_cat(sb, "\n");
} else if (sc->sc_active && sc->sc_active->provider) {
sbuf_printf(sb, "%s\n", sc->sc_active->provider->name);
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
} else {
sbuf_printf(sb, "none\n");
}
sbuf_finish(sb);
gctl_set_param_err(req, "output", sbuf_data(sb), sbuf_len(sb) + 1);
sbuf_delete(sb);
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
static void
g_multipath_config(struct gctl_req *req, struct g_class *mp, const char *verb)
{
uint32_t *version;
g_topology_assert();
version = gctl_get_paraml(req, "version", sizeof(*version));
if (version == NULL) {
gctl_error(req, "No 'version' argument");
} else if (*version != G_MULTIPATH_VERSION) {
gctl_error(req, "Userland and kernel parts are out of sync");
Change how multipath labels are created and managed. This makes it easier to support various storage boxes which really aren't active-active. We only write the label on the *first* provider. For all other providers we just "add" the disk. This also allows for an "add" verb. A usage implication is that you should specificy the currently active storage path as the first provider. Note that this does not add RDAC-like functionality, but better allows for autovolumefailover configurations (additional checkins elsewhere will support this). Sponsored by: Panasas MFC after: 1 month
2010-03-29 18:04:06 +00:00
} else if (strcmp(verb, "add") == 0) {
g_multipath_ctl_add(req, mp);
New command "gmultipath prefer" to force selection of a specified provider in an Active/Passive configuration. Reviewed by: mav MFC after: 4 weeks
2012-08-03 14:55:35 +00:00
} else if (strcmp(verb, "prefer") == 0) {
g_multipath_ctl_prefer(req, mp);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
} else if (strcmp(verb, "create") == 0) {
g_multipath_ctl_create(req, mp);
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
} else if (strcmp(verb, "configure") == 0) {
g_multipath_ctl_configure(req, mp);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
} else if (strcmp(verb, "stop") == 0) {
g_multipath_ctl_stop(req, mp);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
} else if (strcmp(verb, "destroy") == 0) {
g_multipath_ctl_destroy(req, mp);
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
} else if (strcmp(verb, "fail") == 0) {
g_multipath_ctl_fail(req, mp, 1);
} else if (strcmp(verb, "restore") == 0) {
g_multipath_ctl_fail(req, mp, 0);
} else if (strcmp(verb, "remove") == 0) {
g_multipath_ctl_remove(req, mp);
Add 'rotate' and 'getactive' verbs to provide some control and information about what the currently active path is. Sponsored by: Panasas MFC after: 1 month
2010-03-21 15:02:47 +00:00
} else if (strcmp(verb, "rotate") == 0) {
g_multipath_ctl_rotate(req, mp);
} else if (strcmp(verb, "getactive") == 0) {
g_multipath_ctl_getactive(req, mp);
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
} else {
gctl_error(req, "Unknown verb %s", verb);
}
}
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
static void
g_multipath_dumpconf(struct sbuf *sb, const char *indent, struct g_geom *gp,
struct g_consumer *cp, struct g_provider *pp)
{
struct g_multipath_softc *sc;
int good;
g_topology_assert();
sc = gp->softc;
if (sc == NULL)
return;
if (cp != NULL) {
Add missing newlines into XML output. MFC after: 3 days Sponsored by: iXsystems, Inc.
2012-06-05 16:46:34 +00:00
sbuf_printf(sb, "%s<State>%s</State>\n", indent,
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
(cp->index & MP_NEW) ? "NEW" :
(cp->index & MP_LOST) ? "LOST" :
(cp->index & MP_FAIL) ? "FAIL" :
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
(sc->sc_active_active == 1 || sc->sc_active == cp) ?
"ACTIVE" :
sc->sc_active_active == 2 ? "READ" : "PASSIVE");
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
} else {
good = g_multipath_good(gp);
Add missing newlines into XML output. MFC after: 3 days Sponsored by: iXsystems, Inc.
2012-06-05 16:46:34 +00:00
sbuf_printf(sb, "%s<State>%s</State>\n", indent,
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
good == 0 ? "BROKEN" :
(good != sc->sc_ndisks || sc->sc_ndisks == 1) ?
"DEGRADED" : "OPTIMAL");
}
if (cp == NULL && pp == NULL) {
Add missing newlines into XML output. MFC after: 3 days Sponsored by: iXsystems, Inc.
2012-06-05 16:46:34 +00:00
sbuf_printf(sb, "%s<UUID>%s</UUID>\n", indent, sc->sc_uuid);
sbuf_printf(sb, "%s<Mode>Active/%s</Mode>\n", indent,
Some improvements to GEOM MULTIPATH: - Implement "configure" command to allow switching operation mode of running device on-fly without destroying and recreation. - Implement Active/Read mode as hybrid of Active/Active and Active/Passive. In this mode all paths not marked FAIL may handle reads same time, but unlike Active/Active only one path handles write requests at any point in time. It allows to closer follow original write request order if above layers need it for data consistency (not waiting for requisite write completion before sending dependent write). - Hide duplicate messages about device status change. - Remove periodic thread wake up with 10Hz rate. MFC after: 2 weeks Sponsored by: iXsystems, Inc.
2012-04-18 09:42:14 +00:00
sc->sc_active_active == 2 ? "Read" :
sc->sc_active_active == 1 ? "Active" : "Passive");
Add missing newlines into XML output. MFC after: 3 days Sponsored by: iXsystems, Inc.
2012-06-05 16:46:34 +00:00
sbuf_printf(sb, "%s<Type>%s</Type>\n", indent,
Major GEOM MULTIPATH class rewrite: - Improved locking and destruction process to fix crashes. - Improved "automatic" configuration method to make it consistent and safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of ordering conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to allow manage paths manually. - Failed paths are no longer dropped from geom, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - geom is now destroyed on last path disconnection. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by the device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Sponsored by: iXsystems, inc. MFC after: 1 month
2011-11-12 09:52:27 +00:00
sc->sc_uuid[0] == 0 ? "MANUAL" : "AUTOMATIC");
}
}
First cut at GEOM based multipath. This is an active/passive{/passive...} arrangement that has no intrinsic internal knowledge of whether devices it is given are truly multipath devices. As such, this is a simplistic approach, but still a useful one. The basic approach is to (at present- this will change soon) use camcontrol to find likely identical devices and and label the trailing sector of the first one. This label contains both a full UUID and a name. The name is what is presented in /dev/multipath, but the UUID is used as a true distinguishor at g_taste time, thus making sure we don't have chaos on a shared SAN where everyone names their data multipath as "Fred". The first of N identical devices (and N *may* be 1!) becomes the active path until a BIO request is failed with EIO or ENXIO. When this occurs, the active disk is ripped away and the next in a list is picked to (retry and) continue with. During g_taste events new disks that meet the match criteria for existing multipath geoms get added to the tail end of the list. Thus, this active/passive setup actually does work for devices which go away and come back, as do (now) mpt(4) and isp(4) SAN based disks. There is still a lot to do to improve this- like about 5 of the 12 recommendations I've received about it, but it's been functional enough for a while that it deserves a broader test base. Reviewed by: pjd Sponsored by: IronPort Systems MFC: 2 months
2007-02-27 04:01:58 +00:00
DECLARE_GEOM_CLASS(g_multipath_class, g_multipath);
Reference in New Issue Copy Permalink