d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Import the gvinum work that have been done during and after Summer of Code 2007.

Browse Source 
The work have been under testing and fixing since then, and it is mature enough
to be put into HEAD for further testing.

A lot have changed in this time, and here are the most important:
- Gvinum now uses one single workerthread instead of one thread for each
  volume and each plex. The reason for this is that the previous scheme was
  very complex, and was the cause of many of the bugs discovered in gvinum.
  Instead, gvinum now uses one worker thread with an event queue, quite
  similar to what used in gmirror.
- The rebuild/grow/initialize/parity check routines no longer runs in
  separate threads, but are run as regular I/O requests with special flags.
  This made it easier to support mounted growing and parity rebuild.
- Support for growing striped and raid5-plexes, meaning that one can extend the
  volumes for these plex types in addition to the concat type. Also works while
  the volume is mounted.
- Implementation of many of the missing commands from the old vinum:
  attach/detach, start (was partially implemented), stop (was partially
  implemented), concat, mirror, stripe, raid5 (shortcuts for creating volumes
  with one plex of these organizations).
- The parity check and rebuild no longer goes between userland/kernel, meaning
  that the gvinum command will not stay and wait forever for the rebuild to
  finish. You can instead watch the status with the list command.
- Many problems with gvinum have been reported since 5.x, and some has been hard
  to fix due to the complicated architecture. Hopefully, it should be more
  stable and better handle edge cases that previously made gvinum crash.
- Failed drives no longer disappears entirely, but now leave behind a dummy
  drive that makes sure the original state is not forgotten in case the system
  is rebooted between drive failures/swaps.
- Update manpage to reflect new commands and extend it with some examples.

Sponsored by:   Google Summer of Code 2007
Mentored by:    le
Tested by:      Rick C. Petty <rick-freebsd2008 -at- kiwi-computer.com>
This commit is contained in:
lulf 2009-03-28 17:20:08 +00:00
parent 1552563928
commit 14e3eb7296
20 changed files with 4106 additions and 3569 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

177
sbin/gvinum/gvinum.8
View File

@ -40,6 +40,13 @@
.Op Fl options
.Sh COMMANDS
.Bl -tag -width indent
.It Ic attach Ar plex volume Op Cm rename
.It Ic attach Ar subdisk plex Oo Ar offset Oc Op Cm rename
Attach a plex to a volume, or a subdisk to a plex.
If offset is specified, the subdisk will be attached to the given offset within
the plex.
If rename is specified, the subdisk or plex will change name according to the
object it attaches to.
.It Ic checkparity Oo Fl f Oc Ar plex
Check the parity blocks of a RAID-5 plex.
The parity check will start at the
@ -49,7 +56,10 @@ flag is specified, or otherwise at the location of the parity check pointer,
the first location at which plex's parity is incorrect.
All subdisks in the
plex must be up for a parity check.
.It Ic create Op Ar description-file
.It Ic concat Oo Fl fv Oc Oo Fl n Ar name Oc Ar drives
Create a concatenated volume from the specified drives.
If no name is specified, a unique name will be set by gvinum.
.It Ic create Oo Fl f Oc Op Ar description-file
Create a volume as described in
.Ar description-file .
If no
@ -57,6 +67,18 @@ If no
provided, opens an editor and provides the current
.Nm
configuration for editing.
The
.Fl f
flag will make gvinum ignore any errors regarding creating objects that already
exists.
However, in contrast to vinum, objects that are not properly named in the
.Ar description-file
will not be created when the
.Fl f
flag is given.
.It Ic detach Oo Fl f Oc Op Ar plex | subdisk
Detach a plex or subdisk from the volume or plex to which it is
attached.
.It Ic help
Provides a synopsis of
.Nm
@ -76,6 +98,14 @@ The
and
.Fl V
flags provide progressively more detailed output.
.It Ic mirror Oo Fl fsv Oc Oo Fl n Ar name Oc Ar drives
Create a mirrored volume from the specified drives.
It requires at least a multiple of 2 drives.
If no name is specified, a unique name will be set by gvinum.
If the
.Fl s
flag is specified, a striped mirror will be created, and thus requires a
multiple of 4 drives.
.It Ic move | mv Fl f Ar drive subdisk Op Ar ...
Move the subdisk(s) to the specified drive.
The
@ -85,12 +115,19 @@ part of the move.
This can currently only be done when the subdisk is
not being accessed.
.Pp
If the subdisk(s) form part of a RAID-5 plex, the disk(s) will need to be set
to the
If a single subdisk is moved, and it forms a part of a RAID-5 plex, the moved
subdisks will need to be set to the
.Dq stale
state, and the plex will require a
.Ic start
command.
If multiple subdisk(s) is moved, and form part of a RAID-5 plex, the
moved disk(s) will need to be set to the
.Dq up
state and the plex will require a
.Ic rebuildparity
command; if the subdisk(s) form part of a plex that is mirrored with other
command.
If the subdisk(s) form part of a plex that is mirrored with other
plexes, the plex will require restarting and will sync once restarted.
Moving
more than one subdisk in a RAID-5 plex or subdisks from both sides of a
@ -105,6 +142,11 @@ Exit
when running in interactive mode.
Normally this would be done by entering the
EOF character.
.It Ic raid5 Oo Fl fv Oc Oo Fl s Ar stripesize Oc Oo Fl n Ar name Oc Ar drives
Create a RAID-5 volume from the specified drives.
If no name is specified,a unique name will be set by
.Ic gvinum.
This organization requires at least three drives.
.It Ic rename Oo Fl r Oc Ar drive | subdisk | plex | volume newname
Change the name of the specified object.
The
@ -143,9 +185,21 @@ flag forces state changes regardless of whether they are legal.
Read configuration from all vinum drives.
.It Ic start Oo Fl S Ar size Oc Ar volume | plex | subdisk
Allow the system to access the objects.
If necessary, plexes will be synced and rebuilt.
If a subdisk was added to a running RAID-5 or striped plex, gvinum will
expand into this subdisk and grow the whole RAID-5 array.
This can be done without unmounting your filesystem.
The
.Fl S
flag is currently ignored.
.It Ic stop Oo Fl f Oc Op Ar volume | plex | subdisk
Terminate access to the objects, or stop
.Nm
if no parameters are specified.
.It Ic stripe Oo Fl fv Oc Oo Fl n Ar name Oc Ar drives
Create a striped volume from the specified drives. If no name is specified,
a unique name will be set by Ic gvinum. This organization requires at least two
drives.
.El
.Sh DESCRIPTION
The
@ -217,15 +271,90 @@ is invoked.
directory with device nodes for
.Nm
objects
.It Pa /dev/gvinum/plex
directory containing device nodes for
.Nm
plexes
.It Pa /dev/gvinum/sd
directory containing device nodes for
.Nm
subdisks
.El
.Sh EXAMPLES
To create a mirror on disks /dev/ad1 and /dev/ad2, create a filesystem, mount,
unmount and then stop Ic gvinum:
.Pp
.Dl "gvinum mirror /dev/ad1 /dev/ad2"
.Dl "newfs /dev/gvinum/gvinumvolume0"
.Dl "mount /dev/gvinum/gvinumvolume0 /mnt"
.Dl "..."
.Dl "unmount /mnt"
.Dl "gvinum stop"
.Pp
To create a striped mirror on disks /dev/ad1 /dev/ad2 /dev/ad3 and /dev/ad4
named "data" and create a filesystem:
.Pp
.Dl "gvinum mirror -s -n data /dev/ad1 /dev/ad2 /dev/ad3 /dev/ad4"
.Dl "newfs /dev/gvinum/data"
.Pp
To create a raid5 array on disks /dev/ad1 /dev/ad2 and /dev/ad3, with stripesize
493k you can use the raid5 command:
.Pp
.Dl "gvinum raid5 -s 493k /dev/ad1 /dev/ad2 /dev/ad3"
.Pp
Then the volume will be created automatically.
Afterwards, you have to initialize the volume:
.Pp
.Dl "gvinum start myraid5vol"
.Pp
The initialization will start, and the states will be updated when it's
finished.
The list command will give you information about its progress.
.Pp
Imagine that one of the drives fails, and the output of 'printconfig' looks
something like this:
.Pp
.Dl "drive gvinumdrive1 device /dev/ad2"
.Dl "drive gvinumdrive2 device /dev/???"
.Dl "drive gvinumdrive0 device /dev/ad1"
.Dl "volume myraid5vol"
.Dl "plex name myraid5vol.p0 org raid5 986s vol myraid5vol"
.Dl "sd name myraid5vol.p0.s2 drive gvinumdrive2 len 32538s driveoffset 265s"
.Dl "plex myraid5vol.p0 plexoffset 1972s"
.Dl "sd name myraid5vol.p0.s1 drive gvinumdrive1 len 32538s driveoffset 265s"
.Dl "plex myraid5vol.p0 plexoffset 986s"
.Dl "sd name myraid5vol.p0.s0 drive gvinumdrive0 len 32538s driveoffset 265s"
.Dl "plex myraid5vol.p0 plexoffset 0s"
.Pp
Create a new drive with this configuration:
.Pp
.Dl "drive gdrive4 device /dev/ad4"
.Pp
Then move the stale subdisk to the new drive:
.Pp
.Dl "gvinum move gdrive4 myraid5vol.p0.s2"
.Pp
Then, initiate the rebuild:
.Pp
.Dl "gvinum start myraid5vol.p0"
.Pp
The plex will go up form degraded mode after the rebuild is finished.
The plex can still be used while the rebuild is in progress, although requests
might be delayed.
For a more advanced usage and detailed explanation of gvinum, the
handbook is recommended.
.Pp
Given the configuration as in the previous example, growing a RAID-5 or STRIPED
array is accomplished by adding a new subdisk to the plex with a
.Ar description-file
similar to this:
.Pp
.Dl "drive newdrive device /dev/ad4"
.Dl "sd drive newdrive plex myraid5vol.p0"
.Pp
If everything went ok, the plex state should now be set to growable.
You can then start the growing with the
.Ic start
command:
.Pp
.Dl "gvinum start myraid5vol.p0"
.Pp
As with rebuilding, you can watch the progress using the
.Ic list
command.
.Pp
.Sh SEE ALSO
.Xr geom 4 ,
.Xr geom 8
@ -255,9 +384,13 @@ documentation were added by
.An "Chris Jones"
through the 2005 Google Summer
of Code program.
.Ic a partial rewrite of gvinum was done by "Lukas Ertl" and "Ulf Lilleengen"
through the 2007 Google Summer of Code program.
The documentation have been updated to reflect the new functionality.
.Sh AUTHORS
.An Lukas Ertl Aq le@FreeBSD.org
.An Chris Jones Aq soc-cjones@FreeBSD.org
.An Ulf Lilleengen Aq lulf@FreeBSD.org
.Sh BUGS
Currently,
.Nm
@ -271,10 +404,6 @@ initsize flag to
.Ic start
is ignored.
.Pp
The
.Ic stop
command does not work.
.Pp
Moving subdisks that are not part of a mirrored or RAID-5 volume will
destroy data.
It is perhaps a bug to permit this.
@ -291,18 +420,10 @@ Specifically, the following commands from
.Xr vinum 4
are not supported:
.Bl -tag -width indent
.It Ic attach Ar plex volume Op Cm rename
.It Ic attach Ar subdisk plex Oo Ar offset Oc Op Cm rename
Attach a plex to a volume, or a subdisk to a plex.
.It Ic concat Oo Fl fv Oc Oo Fl n Ar name Oc Ar drives
Create a concatenated volume from the specified drives.
.It Ic debug
Cause the volume manager to enter the kernel debugger.
.It Ic debug Ar flags
Set debugging flags.
.It Ic detach Oo Fl f Oc Op Ar plex | subdisk
Detach a plex or subdisk from the volume or plex to which it is
attached.
.It Ic dumpconfig Op Ar drive ...
List the configuration information stored on the specified drives, or all
drives in the system if no drive names are specified.
@ -310,17 +431,9 @@ drives in the system if no drive names are specified.
List information about volume manager state.
.It Ic label Ar volume
Create a volume label.
.It Ic mirror Oo Fl fsv Oc Oo Fl n Ar name Oc Ar drives
Create a mirrored volume from the specified drives.
.It Ic resetstats Oo Fl r Oc Op Ar volume | plex | subdisk
Reset statistics counters for the specified objects, or for all objects if none
are specified.
.It Ic setdaemon Op Ar value
Set daemon configuration.
.It Ic stop Oo Fl f Oc Op Ar volume | plex | subdisk
Terminate access to the objects, or stop
.Nm
if no parameters are specified.
.It Ic stripe Oo Fl fv Oc Oo Fl n Ar name Oc Ar drives
Create a striped volume from the specified drives.
.El

531
sbin/gvinum/gvinum.c
View File

@ -1,5 +1,7 @@
/*
* Copyright (c) 2004 Lukas Ertl, 2005 Chris Jones
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2005 Chris Jones
* Copyright (c) 2007 Ulf Lilleengen
* All rights reserved.
*
* Portions of this software were developed for the FreeBSD Project
@ -43,6 +45,7 @@
#include <ctype.h>
#include <err.h>
#include <errno.h>
#include <libgeom.h>
#include <stdint.h>
#include <stdio.h>
@ -54,12 +57,17 @@
#include "gvinum.h"
void gvinum_attach(int, char **);
void gvinum_concat(int, char **);
void gvinum_create(int, char **);
void gvinum_detach(int, char **);
void gvinum_help(void);
void gvinum_list(int, char **);
void gvinum_move(int, char **);
void gvinum_mirror(int, char **);
void gvinum_parityop(int, char **, int);
void gvinum_printconfig(int, char **);
void gvinum_raid5(int, char **);
void gvinum_rename(int, char **);
void gvinum_resetconfig(void);
void gvinum_rm(int, char **);
@ -67,9 +75,15 @@ void gvinum_saveconfig(void);
void gvinum_setstate(int, char **);
void gvinum_start(int, char **);
void gvinum_stop(int, char **);
void gvinum_stripe(int, char **);
void parseline(int, char **);
void printconfig(FILE *, char *);
char *create_drive(char *);
void create_volume(int, char **, char *);
char *find_name(const char *, int, int);
char *find_pattern(char *, char *);
int
main(int argc, char **argv)
{
@ -111,6 +125,44 @@ main(int argc, char **argv)
exit(0);
}
/* Attach a plex to a volume or a subdisk to a plex. */
void
gvinum_attach(int argc, char **argv)
{
struct gctl_req *req;
const char *errstr;
int rename;
off_t offset;
rename = 0;
offset = -1;
if (argc < 3) {
warnx("usage:\tattach <subdisk> <plex> [rename] "
"[<plexoffset>]\n"
"\tattach <plex> <volume> [rename]");
return;
}
if (argc > 3) {
if (!strcmp(argv[3], "rename")) {
rename = 1;
if (argc == 5)
offset = strtol(argv[4], NULL, 0);
} else
offset = strtol(argv[3], NULL, 0);
}
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, "attach");
gctl_ro_param(req, "child", -1, argv[1]);
gctl_ro_param(req, "parent", -1, argv[2]);
gctl_ro_param(req, "offset", sizeof(off_t), &offset);
gctl_ro_param(req, "rename", sizeof(int), &rename);
errstr = gctl_issue(req);
if (errstr != NULL)
warnx("attach failed: %s", errstr);
gctl_free(req);
}
void
gvinum_create(int argc, char **argv)
{
@ -120,19 +172,30 @@ gvinum_create(int argc, char **argv)
struct gv_sd *s;
struct gv_volume *v;
FILE *tmp;
int drives, errors, fd, line, plexes, plex_in_volume;
int sd_in_plex, status, subdisks, tokens, volumes;
int drives, errors, fd, flags, i, line, plexes, plex_in_volume;
int sd_in_plex, status, subdisks, tokens, undeffd, volumes;
const char *errstr;
char buf[BUFSIZ], buf1[BUFSIZ], commandline[BUFSIZ], *ed;
char buf[BUFSIZ], buf1[BUFSIZ], commandline[BUFSIZ], *ed, *sdname;
char original[BUFSIZ], tmpfile[20], *token[GV_MAXARGS];
char plex[GV_MAXPLEXNAME], volume[GV_MAXVOLNAME];
if (argc == 2) {
if ((tmp = fopen(argv[1], "r")) == NULL) {
warn("can't open '%s' for reading", argv[1]);
return;
}
} else {
tmp = NULL;
flags = 0;
for (i = 1; i < argc; i++) {
/* Force flag used to ignore already created drives. */
if (!strcmp(argv[i], "-f")) {
flags |= GV_FLAG_F;
/* Else it must be a file. */
} else {
if ((tmp = fopen(argv[1], "r")) == NULL) {
warn("can't open '%s' for reading", argv[1]);
return;
}
}
}
/* We didn't get a file. */
if (tmp == NULL) {
snprintf(tmpfile, sizeof(tmpfile), "/tmp/gvinum.XXXXXX");
if ((fd = mkstemp(tmpfile)) == -1) {
@ -167,9 +230,11 @@ gvinum_create(int argc, char **argv)
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, "create");
gctl_ro_param(req, "flags", sizeof(int), &flags);
drives = volumes = plexes = subdisks = 0;
plex_in_volume = sd_in_plex = 0;
plex_in_volume = sd_in_plex = undeffd = 0;
plex[0] = '\0';
errors = 0;
line = 1;
while ((fgets(buf, BUFSIZ, tmp)) != NULL) {
@ -187,7 +252,7 @@ gvinum_create(int argc, char **argv)
* Copy the original input line in case we need it for error
* output.
*/
strncpy(original, buf, sizeof(buf));
strlcpy(original, buf, sizeof(original));
tokens = gv_tokenize(buf, token, GV_MAXARGS);
if (tokens <= 0) {
@ -214,7 +279,7 @@ gvinum_create(int argc, char **argv)
* Set default volume name for following plex
* definitions.
*/
strncpy(volume, v->name, sizeof(volume));
strlcpy(volume, v->name, sizeof(volume));
snprintf(buf1, sizeof(buf1), "volume%d", volumes);
gctl_ro_param(req, buf1, sizeof(*v), v);
@ -236,13 +301,13 @@ gvinum_create(int argc, char **argv)
/* Default name. */
if (strlen(p->name) == 0) {
snprintf(p->name, GV_MAXPLEXNAME, "%s.p%d",
snprintf(p->name, sizeof(p->name), "%s.p%d",
volume, plex_in_volume++);
}
/* Default volume. */
if (strlen(p->volume) == 0) {
snprintf(p->volume, GV_MAXVOLNAME, "%s",
snprintf(p->volume, sizeof(p->volume), "%s",
volume);
}
@ -250,7 +315,7 @@ gvinum_create(int argc, char **argv)
* Set default plex name for following subdisk
* definitions.
*/
strncpy(plex, p->name, GV_MAXPLEXNAME);
strlcpy(plex, p->name, sizeof(plex));
snprintf(buf1, sizeof(buf1), "plex%d", plexes);
gctl_ro_param(req, buf1, sizeof(*p), p);
@ -270,13 +335,21 @@ gvinum_create(int argc, char **argv)
/* Default name. */
if (strlen(s->name) == 0) {
snprintf(s->name, GV_MAXSDNAME, "%s.s%d",
plex, sd_in_plex++);
if (strlen(plex) == 0) {
sdname = find_name("gvinumsubdisk.p",
GV_TYPE_SD, GV_MAXSDNAME);
snprintf(s->name, sizeof(s->name),
"%s.s%d", sdname, undeffd++);
free(sdname);
} else {
snprintf(s->name, sizeof(s->name),
"%s.s%d",plex, sd_in_plex++);
}
}
/* Default plex. */
if (strlen(s->plex) == 0)
snprintf(s->plex, GV_MAXPLEXNAME, "%s", plex);
snprintf(s->plex, sizeof(s->plex), "%s", plex);
snprintf(buf1, sizeof(buf1), "sd%d", subdisks);
gctl_ro_param(req, buf1, sizeof(*s), s);
@ -320,7 +393,279 @@ gvinum_create(int argc, char **argv)
warnx("create failed: %s", errstr);
}
gctl_free(req);
gvinum_list(0, NULL);
}
/* Create a concatenated volume. */
void
gvinum_concat(int argc, char **argv)
{
if (argc < 2) {
warnx("usage:\tconcat [-fv] [-n name] drives\n");
return;
}
create_volume(argc, argv, "concat");
}
/* Create a drive quick and dirty. */
char *
create_drive(char *device)
{
struct gv_drive *d;
struct gctl_req *req;
const char *errstr;
char *drivename, *dname;
int drives, i, flags, volumes, subdisks, plexes;
flags = plexes = subdisks = volumes = 0;
drives = 1;
dname = NULL;
/* Strip away eventual /dev/ in front. */
if (strncmp(device, "/dev/", 5) == 0)
device += 5;
drivename = find_name("gvinumdrive", GV_TYPE_DRIVE, GV_MAXDRIVENAME);
if (drivename == NULL)
return (NULL);
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, "create");
d = malloc(sizeof(struct gv_drive));
if (d == NULL)
err(1, "unable to allocate for gv_drive object");
memset(d, 0, sizeof(struct gv_drive));
strlcpy(d->name, drivename, sizeof(d->name));
strlcpy(d->device, device, sizeof(d->device));
gctl_ro_param(req, "drive0", sizeof(*d), d);
gctl_ro_param(req, "flags", sizeof(int), &flags);
gctl_ro_param(req, "drives", sizeof(int), &drives);
gctl_ro_param(req, "volumes", sizeof(int), &volumes);
gctl_ro_param(req, "plexes", sizeof(int), &plexes);
gctl_ro_param(req, "subdisks", sizeof(int), &subdisks);
errstr = gctl_issue(req);
if (errstr != NULL) {
warnx("error creating drive: %s", errstr);
gctl_free(req);
return (NULL);
} else {
gctl_free(req);
/* XXX: This is needed because we have to make sure the drives
* are created before we return. */
/* Loop until it's in the config. */
for (i = 0; i < 100000; i++) {
dname = find_name("gvinumdrive", GV_TYPE_DRIVE,
GV_MAXDRIVENAME);
/* If we got a different name, quit. */
if (dname == NULL)
continue;
if (strcmp(dname, drivename)) {
free(dname);
return (drivename);
}
free(dname);
dname = NULL;
usleep(100000); /* Sleep for 0.1s */
}
}
gctl_free(req);
return (drivename);
}
/*
* General routine for creating a volume. Mainly for use by concat, mirror,
* raid5 and stripe commands.
*/
void
create_volume(int argc, char **argv, char *verb)
{
struct gctl_req *req;
const char *errstr;
char buf[BUFSIZ], *drivename, *volname;
int drives, flags, i;
off_t stripesize;
flags = 0;
drives = 0;
volname = NULL;
stripesize = 262144;
/* XXX: Should we check for argument length? */
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
for (i = 1; i < argc; i++) {
if (!strcmp(argv[i], "-f")) {
flags |= GV_FLAG_F;
} else if (!strcmp(argv[i], "-n")) {
volname = argv[++i];
} else if (!strcmp(argv[i], "-v")) {
flags |= GV_FLAG_V;
} else if (!strcmp(argv[i], "-s")) {
flags |= GV_FLAG_S;
if (!strcmp(verb, "raid5"))
stripesize = gv_sizespec(argv[++i]);
} else {
/* Assume it's a drive. */
snprintf(buf, sizeof(buf), "drive%d", drives++);
/* First we create the drive. */
drivename = create_drive(argv[i]);
if (drivename == NULL)
goto bad;
/* Then we add it to the request. */
gctl_ro_param(req, buf, -1, drivename);
}
}
gctl_ro_param(req, "stripesize", sizeof(off_t), &stripesize);
/* Find a free volume name. */
if (volname == NULL)
volname = find_name("gvinumvolume", GV_TYPE_VOL, GV_MAXVOLNAME);
/* Then we send a request to actually create the volumes. */
gctl_ro_param(req, "verb", -1, verb);
gctl_ro_param(req, "flags", sizeof(int), &flags);
gctl_ro_param(req, "drives", sizeof(int), &drives);
gctl_ro_param(req, "name", -1, volname);
errstr = gctl_issue(req);
if (errstr != NULL)
warnx("creating %s volume failed: %s", verb, errstr);
bad:
gctl_free(req);
}
/* Parse a line of the config, return the word after <pattern>. */
char *
find_pattern(char *line, char *pattern)
{
char *ptr;
ptr = strsep(&line, " ");
while (ptr != NULL) {
if (!strcmp(ptr, pattern)) {
/* Return the next. */
ptr = strsep(&line, " ");
return (ptr);
}
ptr = strsep(&line, " ");
}
return (NULL);
}
/* Find a free name for an object given a a prefix. */
char *
find_name(const char *prefix, int type, int namelen)
{
struct gctl_req *req;
char comment[1], buf[GV_CFG_LEN - 1], *name, *sname, *ptr;
const char *errstr;
int i, n, begin, len, conflict;
char line[1024];
comment[0] = '\0';
/* Find a name. Fetch out configuration first. */
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, "getconfig");
gctl_ro_param(req, "comment", -1, comment);
gctl_rw_param(req, "config", sizeof(buf), buf);
errstr = gctl_issue(req);
if (errstr != NULL) {
warnx("can't get configuration: %s", errstr);
return (NULL);
}
gctl_free(req);
begin = 0;
len = strlen(buf);
i = 0;
sname = malloc(namelen + 1);
/* XXX: Max object setting? */
for (n = 0; n < 10000; n++) {
snprintf(sname, namelen, "%s%d", prefix, n);
conflict = 0;
begin = 0;
/* Loop through the configuration line by line. */
for (i = 0; i < len; i++) {
if (buf[i] == '\n' || buf[i] == '\0') {
ptr = buf + begin;
strlcpy(line, ptr, (i - begin) + 1);
begin = i + 1;
switch (type) {
case GV_TYPE_DRIVE:
name = find_pattern(line, "drive");
break;
case GV_TYPE_VOL:
name = find_pattern(line, "volume");
break;
case GV_TYPE_PLEX:
case GV_TYPE_SD:
name = find_pattern(line, "name");
break;
default:
printf("Invalid type given\n");
continue;
}
if (name == NULL)
continue;
if (!strcmp(sname, name)) {
conflict = 1;
/* XXX: Could quit the loop earlier. */
}
}
}
if (!conflict)
return (sname);
}
free(sname);
return (NULL);
}
/* Detach a plex or subdisk from its parent. */
void
gvinum_detach(int argc, char **argv)
{
const char *errstr;
struct gctl_req *req;
int flags, i;
optreset = 1;
optind = 1;
while ((i = getopt(argc, argv, "f")) != -1) {
switch(i) {
case 'f':
flags |= GV_FLAG_F;
break;
default:
warn("invalid flag: %c", i);
return;
}
}
argc -= optind;
argv += optind;
if (argc != 1) {
warnx("usage: detach [-f] <subdisk> | <plex>");
return;
}
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, "detach");
gctl_ro_param(req, "object", -1, argv[0]);
gctl_ro_param(req, "flags", sizeof(int), &flags);
errstr = gctl_issue(req);
if (errstr != NULL)
warnx("detach failed: %s", errstr);
gctl_free(req);
}
void
@ -329,8 +674,16 @@ gvinum_help(void)
printf("COMMANDS\n"
"checkparity [-f] plex\n"
" Check the parity blocks of a RAID-5 plex.\n"
"create description-file\n"
"create [-f] description-file\n"
" Create as per description-file or open editor.\n"
"attach plex volume [rename]\n"
"attach subdisk plex [offset] [rename]\n"
" Attach a plex to a volume, or a subdisk to a plex\n"
"concat [-fv] [-n name] drives\n"
" Create a concatenated volume from the specified drives.\n"
"detach [-f] [plex | subdisk]\n"
" Detach a plex or a subdisk from the volume or plex to\n"
" which it is attached.\n"
"l | list [-r] [-v] [-V] [volume | plex | subdisk]\n"
" List information about specified objects.\n"
"ld [-r] [-v] [-V] [volume]\n"
@ -341,18 +694,22 @@ gvinum_help(void)
" List information about plexes.\n"
"lv [-r] [-v] [-V] [volume]\n"
" List information about volumes.\n"
"mirror [-fsv] [-n name] drives\n"
" Create a mirrored volume from the specified drives.\n"
"move | mv -f drive object ...\n"
" Move the object(s) to the specified drive.\n"
"quit Exit the vinum program when running in interactive mode."
" Nor-\n"
" mally this would be done by entering the EOF character.\n"
"raid5 [-fv] [-s stripesize] [-n name] drives\n"
" Create a RAID-5 volume from the specified drives.\n"
"rename [-r] [drive | subdisk | plex | volume] newname\n"
" Change the name of the specified object.\n"
"rebuildparity plex [-f]\n"
" Rebuild the parity blocks of a RAID-5 plex.\n"
"resetconfig\n"
" Reset the complete gvinum configuration\n"
"rm [-r] volume | plex | subdisk | drive\n"
"rm [-r] [-f] volume | plex | subdisk | drive\n"
" Remove an object.\n"
"saveconfig\n"
" Save vinum configuration to disk after configuration"
@ -363,6 +720,8 @@ gvinum_help(void)
" poses only.\n"
"start [-S size] volume | plex | subdisk\n"
" Allow the system to access the objects.\n"
"stripe [-fv] [-n name] drives\n"
" Create a striped volume from the specified drives.\n"
);
return;
@ -488,6 +847,18 @@ gvinum_list(int argc, char **argv)
return;
}
/* Create a mirrored volume. */
void
gvinum_mirror(int argc, char **argv)
{
if (argc < 2) {
warnx("usage\tmirror [-fsv] [-n name] drives\n");
return;
}
create_volume(argc, argv, "mirror");
}
/* Note that move is currently of form '[-r] target object [...]' */
void
gvinum_move(int argc, char **argv)
@ -553,8 +924,7 @@ void
gvinum_parityop(int argc, char **argv, int rebuild)
{
struct gctl_req *req;
int flags, i, rv;
off_t offset;
int flags, i;
const char *errstr;
char *op, *msg;
@ -591,47 +961,32 @@ gvinum_parityop(int argc, char **argv, int rebuild)
return;
}
do {
rv = 0;
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, "parityop");
gctl_ro_param(req, "flags", sizeof(int), &flags);
gctl_ro_param(req, "rebuild", sizeof(int), &rebuild);
gctl_rw_param(req, "rv", sizeof(int), &rv);
gctl_rw_param(req, "offset", sizeof(off_t), &offset);
gctl_ro_param(req, "plex", -1, argv[0]);
errstr = gctl_issue(req);
if (errstr) {
warnx("%s\n", errstr);
gctl_free(req);
break;
}
gctl_free(req);
if (flags & GV_FLAG_V) {
printf("\r%s at %s ... ", msg,
gv_roughlength(offset, 1));
}
if (rv == 1) {
printf("Parity incorrect at offset 0x%jx\n",
(intmax_t)offset);
if (!rebuild)
break;
}
fflush(stdout);
req = gctl_get_handle();
gctl_ro_param(req, "class", -1, "VINUM");
gctl_ro_param(req, "verb", -1, op);
gctl_ro_param(req, "rebuild", sizeof(int), &rebuild);
gctl_ro_param(req, "flags", sizeof(int), &flags);
gctl_ro_param(req, "plex", -1, argv[0]);
/* Clear the -f flag. */
flags &= ~GV_FLAG_F;
} while (rv >= 0);
if ((rv == 2) && (flags & GV_FLAG_V)) {
if (rebuild)
printf("Rebuilt parity on %s\n", argv[0]);
else
printf("%s has correct parity\n", argv[0]);
}
errstr = gctl_issue(req);
if (errstr)
warnx("%s\n", errstr);
gctl_free(req);
}
/* Create a RAID-5 volume. */
void
gvinum_raid5(int argc, char **argv)
{
if (argc < 2) {
warnx("usage:\traid5 [-fv] [-s stripesize] [-n name] drives\n");
return;
}
create_volume(argc, argv, "raid5");
}
void
gvinum_rename(int argc, char **argv)
{
@ -697,8 +1052,11 @@ gvinum_rm(int argc, char **argv)
flags = 0;
optreset = 1;
optind = 1;
while ((j = getopt(argc, argv, "r")) != -1) {
while ((j = getopt(argc, argv, "rf")) != -1) {
switch (j) {
case 'f':
flags |= GV_FLAG_F;
break;
case 'r':
flags |= GV_FLAG_R;
break;
@ -728,7 +1086,6 @@ gvinum_rm(int argc, char **argv)
return;
}
gctl_free(req);
gvinum_list(0, NULL);
}
void
@ -763,7 +1120,6 @@ gvinum_resetconfig(void)
return;
}
gctl_free(req);
gvinum_list(0, NULL);
printf("gvinum configuration obliterated\n");
}
@ -833,28 +1189,53 @@ gvinum_start(int argc, char **argv)
}
gctl_free(req);
gvinum_list(0, NULL);
}
void
gvinum_stop(int argc, char **argv)
{
int fileid;
int err, fileid;
fileid = kldfind(GVINUMMOD);
if (fileid == -1) {
warn("cannot find " GVINUMMOD);
return;
}
if (kldunload(fileid) != 0) {
/*
* This little hack prevents that we end up in an infinite loop in
* g_unload_class(). gv_unload() will return EAGAIN so that the GEOM
* event thread will be free for the g_wither_geom() call from
* gv_unload(). It's silly, but it works.
*/
printf("unloading " GVINUMMOD " kernel module... ");
fflush(stdout);
if ((err = kldunload(fileid)) != 0 && (errno == EAGAIN)) {
sleep(1);
err = kldunload(fileid);
}
if (err != 0) {
printf(" failed!\n");
warn("cannot unload " GVINUMMOD);
return;
}
warnx(GVINUMMOD " unloaded");
printf("done\n");
exit(0);
}
/* Create a striped volume. */
void
gvinum_stripe(int argc, char **argv)
{
if (argc < 2) {
warnx("usage:\tstripe [-fv] [-n name] drives\n");
return;
}
create_volume(argc, argv, "stripe");
}
void
parseline(int argc, char **argv)
{
@ -865,6 +1246,12 @@ parseline(int argc, char **argv)
gvinum_create(argc, argv);
else if (!strcmp(argv[0], "exit") || !strcmp(argv[0], "quit"))
exit(0);
else if (!strcmp(argv[0], "attach"))
gvinum_attach(argc, argv);
else if (!strcmp(argv[0], "detach"))
gvinum_detach(argc, argv);
else if (!strcmp(argv[0], "concat"))
gvinum_concat(argc, argv);
else if (!strcmp(argv[0], "help"))
gvinum_help();
else if (!strcmp(argv[0], "list") || !strcmp(argv[0], "l"))
@ -877,12 +1264,16 @@ parseline(int argc, char **argv)
gvinum_list(argc, argv);
else if (!strcmp(argv[0], "lv"))
gvinum_list(argc, argv);
else if (!strcmp(argv[0], "mirror"))
gvinum_mirror(argc, argv);
else if (!strcmp(argv[0], "move"))
gvinum_move(argc, argv);
else if (!strcmp(argv[0], "mv"))
gvinum_move(argc, argv);
else if (!strcmp(argv[0], "printconfig"))
gvinum_printconfig(argc, argv);
else if (!strcmp(argv[0], "raid5"))
gvinum_raid5(argc, argv);
else if (!strcmp(argv[0], "rename"))
gvinum_rename(argc, argv);
else if (!strcmp(argv[0], "resetconfig"))
@ -897,6 +1288,8 @@ parseline(int argc, char **argv)
gvinum_start(argc, argv);
else if (!strcmp(argv[0], "stop"))
gvinum_stop(argc, argv);
else if (!strcmp(argv[0], "stripe"))
gvinum_stripe(argc, argv);
else if (!strcmp(argv[0], "checkparity"))
gvinum_parityop(argc, argv, 0);
else if (!strcmp(argv[0], "rebuildparity"))

1016
sys/geom/vinum/geom_vinum.c
View File

File diff suppressed because it is too large Load Diff

120
sys/geom/vinum/geom_vinum.h
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -29,20 +29,25 @@
#ifndef _GEOM_VINUM_H_
#define _GEOM_VINUM_H_
#define ERRBUFSIZ 1024
/* geom_vinum_create.c */
void gv_concat(struct g_geom *gp, struct gctl_req *);
void gv_mirror(struct g_geom *gp, struct gctl_req *);
void gv_stripe(struct g_geom *gp, struct gctl_req *);
void gv_raid5(struct g_geom *gp, struct gctl_req *);
int gv_create_drive(struct gv_softc *, struct gv_drive *);
int gv_create_volume(struct gv_softc *, struct gv_volume *);
int gv_create_plex(struct gv_softc *, struct gv_plex *);
int gv_create_sd(struct gv_softc *, struct gv_sd *);
/* geom_vinum_drive.c */
void gv_config_new_drive(struct gv_drive *);
void gv_drive_modify(struct gv_drive *);
void gv_save_config(struct gv_softc *);
int gv_read_header(struct g_consumer *, struct gv_hdr *);
void gv_save_config_all(struct gv_softc *);
void gv_save_config(struct g_consumer *, struct gv_drive *,
struct gv_softc *);
int gv_write_header(struct g_consumer *, struct gv_hdr *);
/* geom_vinum_init.c */
void gv_parityop(struct g_geom *, struct gctl_req *);
void gv_start_obj(struct g_geom *, struct gctl_req *);
int gv_start_plex(struct gv_plex *);
int gv_start_vol(struct gv_volume *);
/* geom_vinum_list.c */
void gv_ld(struct g_geom *, struct gctl_req *, struct sbuf *);
@ -53,48 +58,97 @@ void gv_list(struct g_geom *, struct gctl_req *);
/* geom_vinum_move.c */
void gv_move(struct g_geom *, struct gctl_req *);
int gv_move_sd(struct gv_softc *, struct gv_sd *, struct gv_drive *, int);
/* geom_vinum_rename.c */
void gv_rename(struct g_geom *, struct gctl_req *);
int gv_rename_drive(struct gv_softc *, struct gv_drive *, char *, int);
int gv_rename_plex(struct gv_softc *, struct gv_plex *, char *, int);
int gv_rename_sd(struct gv_softc *, struct gv_sd *, char *, int);
int gv_rename_vol(struct gv_softc *, struct gv_volume *, char *, int);
/* geom_vinum_rm.c */
void gv_remove(struct g_geom *, struct gctl_req *);
int gv_resetconfig(struct g_geom *, struct gctl_req *);
int gv_rm_sd(struct gv_softc *sc, struct gctl_req *req,
struct gv_sd *s, int flags);
int gv_resetconfig(struct gv_softc *);
void gv_rm_sd(struct gv_softc *sc, struct gv_sd *s);
void gv_rm_drive(struct gv_softc *, struct gv_drive *, int);
void gv_rm_plex(struct gv_softc *, struct gv_plex *);
void gv_rm_vol(struct gv_softc *, struct gv_volume *);
/* geom_vinum_state.c */
int gv_sdstatemap(struct gv_plex *);
void gv_setstate(struct g_geom *, struct gctl_req *);
int gv_set_drive_state(struct gv_drive *, int, int);
int gv_set_sd_state(struct gv_sd *, int, int);
int gv_set_vol_state(struct gv_volume *, int, int);
int gv_set_plex_state(struct gv_plex *, int, int);
void gv_update_sd_state(struct gv_sd *);
void gv_update_plex_state(struct gv_plex *);
void gv_update_vol_state(struct gv_volume *);
/* geom_vinum_subr.c */
void gv_adjust_freespace(struct gv_sd *, off_t);
void gv_free_sd(struct gv_sd *);
struct g_geom *find_vinum_geom(void);
struct gv_drive *gv_find_drive(struct gv_softc *, char *);
struct gv_plex *gv_find_plex(struct gv_softc *, char *);
struct gv_sd *gv_find_sd(struct gv_softc *, char *);
struct gv_volume *gv_find_vol(struct gv_softc *, char *);
void gv_format_config(struct gv_softc *, struct sbuf *, int, char *);
int gv_is_striped(struct gv_plex *);
int gv_is_open(struct g_geom *);
void gv_kill_drive_thread(struct gv_drive *);
void gv_kill_plex_thread(struct gv_plex *);
void gv_kill_vol_thread(struct gv_volume *);
int gv_object_type(struct gv_softc *, char *);
void gv_parse_config(struct gv_softc *, u_char *, int);
int gv_sd_to_drive(struct gv_softc *, struct gv_drive *, struct gv_sd *,
char *, int);
int gv_sd_to_plex(struct gv_plex *, struct gv_sd *, int);
void gv_update_plex_config(struct gv_plex *);
void gv_update_vol_size(struct gv_volume *, off_t);
off_t gv_vol_size(struct gv_volume *);
off_t gv_plex_size(struct gv_plex *);
void gv_adjust_freespace(struct gv_sd *, off_t);
void gv_free_sd(struct gv_sd *);
struct gv_drive *gv_find_drive(struct gv_softc *, char *);
struct gv_drive *gv_find_drive_device(struct gv_softc *, char *);
struct gv_plex *gv_find_plex(struct gv_softc *, char *);
struct gv_sd *gv_find_sd(struct gv_softc *, char *);
struct gv_volume *gv_find_vol(struct gv_softc *, char *);
void gv_format_config(struct gv_softc *, struct sbuf *, int,
char *);
int gv_is_striped(struct gv_plex *);
int gv_consumer_is_open(struct g_consumer *);
int gv_provider_is_open(struct g_provider *);
int gv_object_type(struct gv_softc *, char *);
void gv_parse_config(struct gv_softc *, char *,
struct gv_drive *);
int gv_sd_to_drive(struct gv_sd *, struct gv_drive *);
int gv_sd_to_plex(struct gv_sd *, struct gv_plex *);
int gv_sdcount(struct gv_plex *, int);
void gv_update_plex_config(struct gv_plex *);
void gv_update_vol_size(struct gv_volume *, off_t);
off_t gv_vol_size(struct gv_volume *);
off_t gv_plex_size(struct gv_plex *);
int gv_plexdown(struct gv_volume *);
int gv_attach_plex(struct gv_plex *, struct gv_volume *,
int);
int gv_attach_sd(struct gv_sd *, struct gv_plex *, off_t,
int);
int gv_detach_plex(struct gv_plex *, int);
int gv_detach_sd(struct gv_sd *, int);
/* geom_vinum.c */
void gv_worker(void *);
void gv_post_event(struct gv_softc *, int, void *, void *, intmax_t,
intmax_t);
void gv_drive_tasted(struct gv_softc *, struct g_provider *);
void gv_drive_lost(struct gv_softc *, struct gv_drive *);
void gv_setup_objects(struct gv_softc *);
void gv_start(struct bio *);
int gv_access(struct g_provider *, int, int, int);
void gv_cleanup(struct gv_softc *);
/* geom_vinum_volume.c */
void gv_done(struct bio *);
void gv_volume_start(struct gv_softc *, struct bio *);
void gv_volume_flush(struct gv_volume *);
void gv_bio_done(struct gv_softc *, struct bio *);
/* geom_vinum_plex.c */
void gv_plex_start(struct gv_plex *, struct bio *);
void gv_plex_raid5_done(struct gv_plex *, struct bio *);
void gv_plex_normal_done(struct gv_plex *, struct bio *);
int gv_grow_request(struct gv_plex *, off_t, off_t, int, caddr_t);
void gv_grow_complete(struct gv_plex *, struct bio *);
void gv_init_request(struct gv_sd *, off_t, caddr_t, off_t);
void gv_init_complete(struct gv_plex *, struct bio *);
void gv_parity_request(struct gv_plex *, int, off_t);
void gv_parity_complete(struct gv_plex *, struct bio *);
void gv_rebuild_complete(struct gv_plex *, struct bio *);
int gv_sync_request(struct gv_plex *, struct gv_plex *, off_t, off_t, int,
caddr_t);
int gv_sync_complete(struct gv_plex *, struct bio *);
extern u_int g_vinum_debug;

665
sys/geom/vinum/geom_vinum_drive.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004, 2005 Lukas Ertl
* Copyright (c) 2004, 2005, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -27,35 +27,20 @@
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/bio.h>
#include <sys/errno.h>
#include <sys/endian.h>
#include <sys/conf.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/libkern.h>
#include <sys/lock.h>
#include <sys/malloc.h>
#include <sys/module.h>
#include <sys/mutex.h>
#include <sys/sbuf.h>
#include <sys/systm.h>
#include <sys/time.h>
#include <sys/vimage.h>
#include <geom/geom.h>
#include <geom/vinum/geom_vinum_var.h>
#include <geom/vinum/geom_vinum.h>
#include <geom/vinum/geom_vinum_share.h>
#define GV_LEGACY_I386 0
#define GV_LEGACY_AMD64 1
#define GV_LEGACY_SPARC64 2
#define GV_LEGACY_POWERPC 3
static void gv_drive_dead(void *, int);
static void gv_drive_worker(void *);
static int gv_legacy_header_type(uint8_t *, int);
/*
@ -267,620 +252,96 @@ gv_write_header(struct g_consumer *cp, struct gv_hdr *m_hdr)
return (ret);
}
/* Save the vinum configuration back to each involved disk. */
void
gv_config_new_drive(struct gv_drive *d)
{
struct gv_hdr *vhdr;
struct gv_freelist *fl;
KASSERT(d != NULL, ("config_new_drive: NULL d"));
vhdr = g_malloc(sizeof(*vhdr), M_WAITOK | M_ZERO);
vhdr->magic = GV_MAGIC;
vhdr->config_length = GV_CFG_LEN;
mtx_lock(&hostname_mtx);
bcopy(G_hostname, vhdr->label.sysname, GV_HOSTNAME_LEN);
mtx_unlock(&hostname_mtx);
strncpy(vhdr->label.name, d->name, GV_MAXDRIVENAME);
microtime(&vhdr->label.date_of_birth);
d->hdr = vhdr;
LIST_INIT(&d->subdisks);
LIST_INIT(&d->freelist);
fl = g_malloc(sizeof(struct gv_freelist), M_WAITOK | M_ZERO);
fl->offset = GV_DATA_START;
fl->size = d->avail;
LIST_INSERT_HEAD(&d->freelist, fl, freelist);
d->freelist_entries = 1;
d->bqueue = g_malloc(sizeof(struct bio_queue_head), M_WAITOK | M_ZERO);
bioq_init(d->bqueue);
mtx_init(&d->bqueue_mtx, "gv_drive", NULL, MTX_DEF);
kproc_create(gv_drive_worker, d, NULL, 0, 0, "gv_d %s", d->name);
d->flags |= GV_DRIVE_THREAD_ACTIVE;
}
void
gv_save_config_all(struct gv_softc *sc)
gv_save_config(struct gv_softc *sc)
{
struct g_consumer *cp;
struct gv_drive *d;
g_topology_assert();
LIST_FOREACH(d, &sc->drives, drive) {
if (d->geom == NULL)
continue;
gv_save_config(NULL, d, sc);
}
}
/* Save the vinum configuration back to disk. */
void
gv_save_config(struct g_consumer *cp, struct gv_drive *d, struct gv_softc *sc)
{
struct g_geom *gp;
struct g_consumer *cp2;
struct gv_hdr *vhdr, *hdr;
struct sbuf *sb;
struct timeval last_update;
int error;
g_topology_assert();
KASSERT(d != NULL, ("gv_save_config: null d"));
KASSERT(sc != NULL, ("gv_save_config: null sc"));
/*
* We can't save the config on a drive that isn't up, but drives that
* were just created aren't officially up yet, so we check a special
* flag.
*/
if ((d->state != GV_DRIVE_UP) && !(d->flags && GV_DRIVE_NEWBORN))
return;
if (cp == NULL) {
gp = d->geom;
KASSERT(gp != NULL, ("gv_save_config: null gp"));
cp2 = LIST_FIRST(&gp->consumer);
KASSERT(cp2 != NULL, ("gv_save_config: null cp2"));
} else
cp2 = cp;
vhdr = g_malloc(GV_HDR_LEN, M_WAITOK | M_ZERO);
vhdr->magic = GV_MAGIC;
vhdr->config_length = GV_CFG_LEN;
hdr = d->hdr;
if (hdr == NULL) {
G_VINUM_DEBUG(0, "drive %s has NULL hdr", d->name);
g_free(vhdr);
return;
}
microtime(&hdr->label.last_update);
bcopy(&hdr->label, &vhdr->label, sizeof(struct gv_label));
microtime(&last_update);
sb = sbuf_new(NULL, NULL, GV_CFG_LEN, SBUF_FIXEDLEN);
gv_format_config(sc, sb, 1, NULL);
sbuf_finish(sb);
error = g_access(cp2, 0, 1, 0);
if (error) {
G_VINUM_DEBUG(0, "g_access failed on drive %s, errno %d",
d->name, error);
sbuf_delete(sb);
g_free(vhdr);
return;
}
g_topology_unlock();
LIST_FOREACH(d, &sc->drives, drive) {
/*
* We can't save the config on a drive that isn't up, but
* drives that were just created aren't officially up yet, so
* we check a special flag.
*/
if (d->state != GV_DRIVE_UP)
continue;
do {
error = gv_write_header(cp2, vhdr);
cp = d->consumer;
if (cp == NULL) {
G_VINUM_DEBUG(0, "drive '%s' has no consumer!",
d->name);
continue;
}
hdr = d->hdr;
if (hdr == NULL) {
G_VINUM_DEBUG(0, "drive '%s' has no header",
d->name);
g_free(vhdr);
continue;
}
bcopy(&last_update, &hdr->label.last_update,
sizeof(struct timeval));
bcopy(&hdr->label, &vhdr->label, sizeof(struct gv_label));
g_topology_lock();
error = g_access(cp, 0, 1, 0);
if (error) {
G_VINUM_DEBUG(0, "g_access failed on "
"drive %s, errno %d", d->name, error);
g_topology_unlock();
continue;
}
g_topology_unlock();
error = gv_write_header(cp, vhdr);
if (error) {
G_VINUM_DEBUG(0, "writing vhdr failed on drive %s, "
"errno %d", d->name, error);
break;
g_topology_lock();
g_access(cp, 0, -1, 0);
g_topology_unlock();
continue;
}
error = g_write_data(cp2, GV_CFG_OFFSET, sbuf_data(sb),
/* First config copy. */
error = g_write_data(cp, GV_CFG_OFFSET, sbuf_data(sb),
GV_CFG_LEN);
if (error) {
G_VINUM_DEBUG(0, "writing first config copy failed "
"on drive %s, errno %d", d->name, error);
break;
G_VINUM_DEBUG(0, "writing first config copy failed on "
"drive %s, errno %d", d->name, error);
g_topology_lock();
g_access(cp, 0, -1, 0);
g_topology_unlock();
continue;
}
error = g_write_data(cp2, GV_CFG_OFFSET + GV_CFG_LEN,
/* Second config copy. */
error = g_write_data(cp, GV_CFG_OFFSET + GV_CFG_LEN,
sbuf_data(sb), GV_CFG_LEN);
if (error)
G_VINUM_DEBUG(0, "writing second config copy failed "
"on drive %s, errno %d", d->name, error);
} while (0);
G_VINUM_DEBUG(0, "writing second config copy failed on "
"drive %s, errno %d", d->name, error);
g_topology_lock();
g_access(cp, 0, -1, 0);
g_topology_unlock();
}
g_topology_lock();
g_access(cp2, 0, -1, 0);
sbuf_delete(sb);
g_free(vhdr);
if (d->geom != NULL)
gv_drive_modify(d);
}
/* This resembles g_slice_access(). */
static int
gv_drive_access(struct g_provider *pp, int dr, int dw, int de)
{
struct g_geom *gp;
struct g_consumer *cp;
struct g_provider *pp2;
struct gv_drive *d;
struct gv_sd *s, *s2;
int error;
gp = pp->geom;
cp = LIST_FIRST(&gp->consumer);
if (cp == NULL)
return (0);
d = gp->softc;
if (d == NULL)
return (0);
s = pp->private;
KASSERT(s != NULL, ("gv_drive_access: NULL s"));
LIST_FOREACH(s2, &d->subdisks, from_drive) {
if (s == s2)
continue;
if (s->drive_offset + s->size <= s2->drive_offset)
continue;
if (s2->drive_offset + s2->size <= s->drive_offset)
continue;
/* Overlap. */
pp2 = s2->provider;
KASSERT(s2 != NULL, ("gv_drive_access: NULL s2"));
if ((pp->acw + dw) > 0 && pp2->ace > 0)
return (EPERM);
if ((pp->ace + de) > 0 && pp2->acw > 0)
return (EPERM);
}
error = g_access(cp, dr, dw, de);
return (error);
}
static void
gv_drive_done(struct bio *bp)
{
struct gv_drive *d;
/* Put the BIO on the worker queue again. */
d = bp->bio_from->geom->softc;
bp->bio_cflags |= GV_BIO_DONE;
mtx_lock(&d->bqueue_mtx);
bioq_insert_tail(d->bqueue, bp);
wakeup(d);
mtx_unlock(&d->bqueue_mtx);
}
static void
gv_drive_start(struct bio *bp)
{
struct gv_drive *d;
struct gv_sd *s;
switch (bp->bio_cmd) {
case BIO_READ:
case BIO_WRITE:
case BIO_DELETE:
break;
case BIO_GETATTR:
default:
g_io_deliver(bp, EOPNOTSUPP);
return;
}
s = bp->bio_to->private;
if ((s->state == GV_SD_DOWN) || (s->state == GV_SD_STALE)) {
g_io_deliver(bp, ENXIO);
return;
}
d = bp->bio_to->geom->softc;
/*
* Put the BIO on the worker queue, where the worker thread will pick
* it up.
*/
mtx_lock(&d->bqueue_mtx);
bioq_disksort(d->bqueue, bp);
wakeup(d);
mtx_unlock(&d->bqueue_mtx);
}
static void
gv_drive_worker(void *arg)
{
struct bio *bp, *cbp;
struct g_geom *gp;
struct g_provider *pp;
struct gv_drive *d;
struct gv_sd *s;
int error;
d = arg;
mtx_lock(&d->bqueue_mtx);
for (;;) {
/* We were signaled to exit. */
if (d->flags & GV_DRIVE_THREAD_DIE)
break;
/* Take the first BIO from out queue. */
bp = bioq_takefirst(d->bqueue);
if (bp == NULL) {
msleep(d, &d->bqueue_mtx, PRIBIO, "-", hz/10);
continue;
}
mtx_unlock(&d->bqueue_mtx);
pp = bp->bio_to;
gp = pp->geom;
/* Completed request. */
if (bp->bio_cflags & GV_BIO_DONE) {
error = bp->bio_error;
/* Deliver the original request. */
g_std_done(bp);
/* The request had an error, we need to clean up. */
if (error != 0) {
g_topology_lock();
gv_set_drive_state(d, GV_DRIVE_DOWN,
GV_SETSTATE_FORCE | GV_SETSTATE_CONFIG);
g_topology_unlock();
g_post_event(gv_drive_dead, d, M_WAITOK, d,
NULL);
}
/* New request, needs to be sent downwards. */
} else {
s = pp->private;
if ((s->state == GV_SD_DOWN) ||
(s->state == GV_SD_STALE)) {
g_io_deliver(bp, ENXIO);
mtx_lock(&d->bqueue_mtx);
continue;
}
if (bp->bio_offset > s->size) {
g_io_deliver(bp, EINVAL);
mtx_lock(&d->bqueue_mtx);
continue;
}
cbp = g_clone_bio(bp);
if (cbp == NULL) {
g_io_deliver(bp, ENOMEM);
mtx_lock(&d->bqueue_mtx);
continue;
}
if (cbp->bio_offset + cbp->bio_length > s->size)
cbp->bio_length = s->size -
cbp->bio_offset;
cbp->bio_done = gv_drive_done;
cbp->bio_offset += s->drive_offset;
g_io_request(cbp, LIST_FIRST(&gp->consumer));
}
mtx_lock(&d->bqueue_mtx);
}
while ((bp = bioq_takefirst(d->bqueue)) != NULL) {
mtx_unlock(&d->bqueue_mtx);
if (bp->bio_cflags & GV_BIO_DONE)
g_std_done(bp);
else
g_io_deliver(bp, ENXIO);
mtx_lock(&d->bqueue_mtx);
}
mtx_unlock(&d->bqueue_mtx);
d->flags |= GV_DRIVE_THREAD_DEAD;
kproc_exit(ENXIO);
}
static void
gv_drive_orphan(struct g_consumer *cp)
{
struct g_geom *gp;
struct gv_drive *d;
g_topology_assert();
gp = cp->geom;
g_trace(G_T_TOPOLOGY, "gv_drive_orphan(%s)", gp->name);
d = gp->softc;
if (d != NULL) {
gv_set_drive_state(d, GV_DRIVE_DOWN,
GV_SETSTATE_FORCE | GV_SETSTATE_CONFIG);
g_post_event(gv_drive_dead, d, M_WAITOK, d, NULL);
} else
g_wither_geom(gp, ENXIO);
}
static struct g_geom *
gv_drive_taste(struct g_class *mp, struct g_provider *pp, int flags __unused)
{
struct g_geom *gp, *gp2;
struct g_consumer *cp;
struct gv_drive *d;
struct gv_sd *s;
struct gv_softc *sc;
struct gv_freelist *fl;
struct gv_hdr *vhdr;
int error;
char *buf, errstr[ERRBUFSIZ];
vhdr = NULL;
d = NULL;
g_trace(G_T_TOPOLOGY, "gv_drive_taste(%s, %s)", mp->name, pp->name);
g_topology_assert();
/* Find the VINUM class and its associated geom. */
gp2 = find_vinum_geom();
if (gp2 == NULL)
return (NULL);
sc = gp2->softc;
gp = g_new_geomf(mp, "%s.vinumdrive", pp->name);
gp->start = gv_drive_start;
gp->orphan = gv_drive_orphan;
gp->access = gv_drive_access;
gp->start = gv_drive_start;
cp = g_new_consumer(gp);
g_attach(cp, pp);
error = g_access(cp, 1, 0, 0);
if (error) {
g_detach(cp);
g_destroy_consumer(cp);
g_destroy_geom(gp);
return (NULL);
}
g_topology_unlock();
/* Now check if the provided slice is a valid vinum drive. */
do {
vhdr = g_malloc(GV_HDR_LEN, M_WAITOK | M_ZERO);
error = gv_read_header(cp, vhdr);
if (error) {
g_free(vhdr);
break;
}
/* A valid vinum drive, let's parse the on-disk information. */
buf = g_read_data(cp, GV_CFG_OFFSET, GV_CFG_LEN, NULL);
if (buf == NULL) {
g_free(vhdr);
break;
}
g_topology_lock();
gv_parse_config(sc, buf, 1);
g_free(buf);
/*
* Let's see if this drive is already known in the
* configuration.
*/
d = gv_find_drive(sc, vhdr->label.name);
/* We already know about this drive. */
if (d != NULL) {
/* Check if this drive already has a geom. */
if (d->geom != NULL) {
g_topology_unlock();
g_free(vhdr);
break;
}
bcopy(vhdr, d->hdr, sizeof(*vhdr));
g_free(vhdr);
/* This is a new drive. */
} else {
d = g_malloc(sizeof(*d), M_WAITOK | M_ZERO);
/* Initialize all needed variables. */
d->size = pp->mediasize - GV_DATA_START;
d->avail = d->size;
d->hdr = vhdr;
strncpy(d->name, vhdr->label.name, GV_MAXDRIVENAME);
LIST_INIT(&d->subdisks);
LIST_INIT(&d->freelist);
/* We also need a freelist entry. */
fl = g_malloc(sizeof(*fl), M_WAITOK | M_ZERO);
fl->offset = GV_DATA_START;
fl->size = d->avail;
LIST_INSERT_HEAD(&d->freelist, fl, freelist);
d->freelist_entries = 1;
/* Save it into the main configuration. */
LIST_INSERT_HEAD(&sc->drives, d, drive);
}
/*
* Create bio queue, queue mutex and a worker thread, if
* necessary.
*/
if (d->bqueue == NULL) {
d->bqueue = g_malloc(sizeof(struct bio_queue_head),
M_WAITOK | M_ZERO);
bioq_init(d->bqueue);
}
if (mtx_initialized(&d->bqueue_mtx) == 0)
mtx_init(&d->bqueue_mtx, "gv_drive", NULL, MTX_DEF);
if (!(d->flags & GV_DRIVE_THREAD_ACTIVE)) {
kproc_create(gv_drive_worker, d, NULL, 0, 0,
"gv_d %s", d->name);
d->flags |= GV_DRIVE_THREAD_ACTIVE;
}
g_access(cp, -1, 0, 0);
gp->softc = d;
d->geom = gp;
d->vinumconf = sc;
strncpy(d->device, pp->name, GV_MAXDRIVENAME);
/*
* Find out which subdisks belong to this drive and crosslink
* them.
*/
LIST_FOREACH(s, &sc->subdisks, sd) {
if (!strncmp(s->drive, d->name, GV_MAXDRIVENAME))
/* XXX: errors ignored */
gv_sd_to_drive(sc, d, s, errstr,
sizeof(errstr));
}
/* This drive is now up for sure. */
gv_set_drive_state(d, GV_DRIVE_UP, 0);
/*
* If there are subdisks on this drive, we need to create
* providers for them.
*/
if (d->sdcount)
gv_drive_modify(d);
return (gp);
} while (0);
g_topology_lock();
g_access(cp, -1, 0, 0);
g_detach(cp);
g_destroy_consumer(cp);
g_destroy_geom(gp);
return (NULL);
}
/*
* Modify the providers for the given drive 'd'. It is assumed that the
* subdisk list of 'd' is already correctly set up.
*/
void
gv_drive_modify(struct gv_drive *d)
{
struct g_geom *gp;
struct g_consumer *cp;
struct g_provider *pp, *pp2;
struct gv_sd *s;
KASSERT(d != NULL, ("gv_drive_modify: null d"));
gp = d->geom;
KASSERT(gp != NULL, ("gv_drive_modify: null gp"));
cp = LIST_FIRST(&gp->consumer);
KASSERT(cp != NULL, ("gv_drive_modify: null cp"));
pp = cp->provider;
KASSERT(pp != NULL, ("gv_drive_modify: null pp"));
g_topology_assert();
LIST_FOREACH(s, &d->subdisks, from_drive) {
/* This subdisk already has a provider. */
if (s->provider != NULL)
continue;
pp2 = g_new_providerf(gp, "gvinum/sd/%s", s->name);
pp2->mediasize = s->size;
pp2->sectorsize = pp->sectorsize;
g_error_provider(pp2, 0);
s->provider = pp2;
pp2->private = s;
}
}
static void
gv_drive_dead(void *arg, int flag)
{
struct g_geom *gp;
struct g_consumer *cp;
struct gv_drive *d;
struct gv_sd *s;
g_topology_assert();
KASSERT(arg != NULL, ("gv_drive_dead: NULL arg"));
if (flag == EV_CANCEL)
return;
d = arg;
if (d->state != GV_DRIVE_DOWN)
return;
g_trace(G_T_TOPOLOGY, "gv_drive_dead(%s)", d->name);
gp = d->geom;
if (gp == NULL)
return;
LIST_FOREACH(cp, &gp->consumer, consumer) {
if (cp->nstart != cp->nend) {
G_VINUM_DEBUG(0, "dead drive '%s' still has "
"active requests, cannot detach consumer",
d->name);
g_post_event(gv_drive_dead, d, M_WAITOK, d,
NULL);
return;
}
if (cp->acr != 0 || cp->acw != 0 || cp->ace != 0)
g_access(cp, -cp->acr, -cp->acw, -cp->ace);
}
G_VINUM_DEBUG(1, "lost drive '%s'", d->name);
d->geom = NULL;
LIST_FOREACH(s, &d->subdisks, from_drive) {
s->provider = NULL;
s->consumer = NULL;
}
gv_kill_drive_thread(d);
gp->softc = NULL;
g_wither_geom(gp, ENXIO);
}
static int
gv_drive_destroy_geom(struct gctl_req *req, struct g_class *mp,
struct g_geom *gp)
{
struct gv_drive *d;
g_trace(G_T_TOPOLOGY, "gv_drive_destroy_geom: %s", gp->name);
g_topology_assert();
d = gp->softc;
gv_kill_drive_thread(d);
g_wither_geom(gp, ENXIO);
return (0);
}
#define VINUMDRIVE_CLASS_NAME "VINUMDRIVE"
static struct g_class g_vinum_drive_class = {
.name = VINUMDRIVE_CLASS_NAME,
.version = G_VERSION,
.taste = gv_drive_taste,
.destroy_geom = gv_drive_destroy_geom
};
DECLARE_GEOM_CLASS(g_vinum_drive_class, g_vinum_drive);

737
sys/geom/vinum/geom_vinum_init.c
View File

@ -1,5 +1,6 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* Copyright (c) 2007, 2009 Ulf Lilleengen
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -26,158 +27,21 @@
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/bio.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/libkern.h>
#include <sys/malloc.h>
#include <sys/queue.h>
#include <geom/geom.h>
#include <geom/vinum/geom_vinum_var.h>
#include <geom/vinum/geom_vinum.h>
#include <geom/vinum/geom_vinum_share.h>
static int gv_init_plex(struct gv_plex *);
void gv_init_td(void *);
static int gv_rebuild_plex(struct gv_plex *);
void gv_rebuild_td(void *);
static int gv_start_plex(struct gv_plex *);
static int gv_start_vol(struct gv_volume *);
static int gv_sync(struct gv_volume *);
void gv_sync_td(void *);
struct gv_sync_args {
struct gv_volume *v;
struct gv_plex *from;
struct gv_plex *to;
off_t syncsize;
};
void
gv_parityop(struct g_geom *gp, struct gctl_req *req)
{
struct gv_softc *sc;
struct gv_plex *p;
struct bio *bp;
struct g_consumer *cp;
int error, *flags, type, *rebuild, rv;
char *plex;
rv = -1;
plex = gctl_get_param(req, "plex", NULL);
if (plex == NULL) {
gctl_error(req, "no plex given");
goto out;
}
flags = gctl_get_paraml(req, "flags", sizeof(*flags));
if (flags == NULL) {
gctl_error(req, "no flags given");
goto out;
}
rebuild = gctl_get_paraml(req, "rebuild", sizeof(*rebuild));
if (rebuild == NULL) {
gctl_error(req, "no rebuild op given");
goto out;
}
sc = gp->softc;
type = gv_object_type(sc, plex);
switch (type) {
case GV_TYPE_PLEX:
break;
case GV_TYPE_VOL:
case GV_TYPE_SD:
case GV_TYPE_DRIVE:
default:
gctl_error(req, "'%s' is not a plex", plex);
goto out;
}
p = gv_find_plex(sc, plex);
if (p->state != GV_PLEX_UP) {
gctl_error(req, "plex %s is not completely accessible",
p->name);
goto out;
}
if (p->org != GV_PLEX_RAID5) {
gctl_error(req, "plex %s is not a RAID5 plex", p->name);
goto out;
}
cp = p->consumer;
error = g_access(cp, 1, 1, 0);
if (error) {
gctl_error(req, "cannot access consumer");
goto out;
}
g_topology_unlock();
/* Reset the check pointer when using -f. */
if (*flags & GV_FLAG_F)
p->synced = 0;
bp = g_new_bio();
if (bp == NULL) {
gctl_error(req, "cannot create BIO - out of memory");
g_topology_lock();
error = g_access(cp, -1, -1, 0);
goto out;
}
bp->bio_cmd = BIO_WRITE;
bp->bio_done = NULL;
bp->bio_data = g_malloc(p->stripesize, M_WAITOK | M_ZERO);
bp->bio_cflags |= GV_BIO_CHECK;
if (*rebuild)
bp->bio_cflags |= GV_BIO_PARITY;
bp->bio_offset = p->synced;
bp->bio_length = p->stripesize;
/* Schedule it down ... */
g_io_request(bp, cp);
/* ... and wait for the result. */
error = biowait(bp, "gwrite");
g_free(bp->bio_data);
g_destroy_bio(bp);
if (error) {
/* Incorrect parity. */
if (error == EAGAIN)
rv = 1;
/* Some other error happened. */
else
gctl_error(req, "Parity check failed at offset 0x%jx, "
"errno %d", (intmax_t)p->synced, error);
/* Correct parity. */
} else
rv = 0;
gctl_set_param(req, "offset", &p->synced, sizeof(p->synced));
/* Advance the checkpointer if there was no error. */
if (rv == 0)
p->synced += p->stripesize;
/* End of plex; reset the check pointer and signal it to the caller. */
if (p->synced >= p->size) {
p->synced = 0;
rv = -2;
}
g_topology_lock();
error = g_access(cp, -1, -1, 0);
out:
gctl_set_param(req, "rv", &rv, sizeof(rv));
}
static int gv_sync(struct gv_volume *);
static int gv_rebuild_plex(struct gv_plex *);
static int gv_init_plex(struct gv_plex *);
static int gv_grow_plex(struct gv_plex *);
static int gv_sync_plex(struct gv_plex *, struct gv_plex *);
static struct gv_plex *gv_find_good_plex(struct gv_volume *);
void
gv_start_obj(struct g_geom *gp, struct gctl_req *req)
@ -187,7 +51,7 @@ gv_start_obj(struct g_geom *gp, struct gctl_req *req)
struct gv_plex *p;
int *argc, *initsize;
char *argv, buf[20];
int err, i, type;
int i, type;
argc = gctl_get_paraml(req, "argc", sizeof(*argc));
initsize = gctl_get_paraml(req, "initsize", sizeof(*initsize));
@ -208,37 +72,21 @@ gv_start_obj(struct g_geom *gp, struct gctl_req *req)
switch (type) {
case GV_TYPE_VOL:
v = gv_find_vol(sc, argv);
err = gv_start_vol(v);
if (err) {
if (err == EINPROGRESS) {
gctl_error(req, "cannot start volume "
"'%s': already in progress", argv);
} else {
gctl_error(req, "cannot start volume "
"'%s'; errno: %d", argv, err);
}
return;
}
if (v != NULL)
gv_post_event(sc, GV_EVENT_START_VOLUME, v,
NULL, *initsize, 0);
break;
case GV_TYPE_PLEX:
p = gv_find_plex(sc, argv);
err = gv_start_plex(p);
if (err) {
if (err == EINPROGRESS) {
gctl_error(req, "cannot start plex "
"'%s': already in progress", argv);
} else {
gctl_error(req, "cannot start plex "
"'%s'; errno: %d", argv, err);
}
return;
}
if (p != NULL)
gv_post_event(sc, GV_EVENT_START_PLEX, p, NULL,
*initsize, 0);
break;
case GV_TYPE_SD:
case GV_TYPE_DRIVE:
/* XXX not yet */
/* XXX Not implemented, but what is the use? */
gctl_error(req, "cannot start '%s' - not yet supported",
argv);
return;
@ -249,36 +97,73 @@ gv_start_obj(struct g_geom *gp, struct gctl_req *req)
}
}
static int
int
gv_start_plex(struct gv_plex *p)
{
struct gv_volume *v;
struct gv_plex *up;
struct gv_sd *s;
int error;
KASSERT(p != NULL, ("gv_start_plex: NULL p"));
if (p->state == GV_PLEX_UP)
return (0);
error = 0;
v = p->vol_sc;
if ((v != NULL) && (v->plexcount > 1))
error = gv_sync(v);
else if (p->org == GV_PLEX_RAID5) {
if (p->state == GV_PLEX_DEGRADED)
error = gv_rebuild_plex(p);
else
error = gv_init_plex(p);
}
/* RAID5 plexes can either be init, rebuilt or grown. */
if (p->org == GV_PLEX_RAID5) {
if (p->state > GV_PLEX_DEGRADED) {
LIST_FOREACH(s, &p->subdisks, in_plex) {
if (s->flags & GV_SD_GROW) {
error = gv_grow_plex(p);
return (error);
}
}
} else if (p->state == GV_PLEX_DEGRADED) {
error = gv_rebuild_plex(p);
} else
error = gv_init_plex(p);
} else {
/* We want to sync from the other plex if we're down. */
if (p->state == GV_PLEX_DOWN && v->plexcount > 1) {
up = gv_find_good_plex(v);
if (up == NULL) {
G_VINUM_DEBUG(1, "unable to find a good plex");
return (ENXIO);
}
g_topology_lock();
error = gv_access(v->provider, 1, 1, 0);
if (error) {
g_topology_unlock();
G_VINUM_DEBUG(0, "sync from '%s' failed to "
"access volume: %d", up->name, error);
return (error);
}
g_topology_unlock();
error = gv_sync_plex(p, up);
if (error)
return (error);
/*
* In case we have a stripe that is up, check whether it can be
* grown.
*/
} else if (p->org == GV_PLEX_STRIPED &&
p->state != GV_PLEX_DOWN) {
LIST_FOREACH(s, &p->subdisks, in_plex) {
if (s->flags & GV_SD_GROW) {
error = gv_grow_plex(p);
break;
}
}
}
}
return (error);
}
static int
int
gv_start_vol(struct gv_volume *v)
{
struct gv_plex *p;
struct gv_sd *s;
int error;
KASSERT(v != NULL, ("gv_start_vol: NULL v"));
@ -291,39 +176,46 @@ gv_start_vol(struct gv_volume *v)
else if (v->plexcount == 1) {
p = LIST_FIRST(&v->plexes);
KASSERT(p != NULL, ("gv_start_vol: NULL p on %s", v->name));
if (p->org == GV_PLEX_RAID5) {
switch (p->state) {
case GV_PLEX_DOWN:
error = gv_init_plex(p);
break;
case GV_PLEX_DEGRADED:
error = gv_rebuild_plex(p);
break;
default:
return (0);
}
} else {
LIST_FOREACH(s, &p->subdisks, in_plex) {
gv_set_sd_state(s, GV_SD_UP,
GV_SETSTATE_CONFIG);
}
}
error = gv_start_plex(p);
} else
error = gv_sync(v);
return (error);
}
/* Sync a plex p from the plex up. */
static int
gv_sync(struct gv_volume *v)
gv_sync_plex(struct gv_plex *p, struct gv_plex *up)
{
struct gv_softc *sc;
struct gv_plex *p, *up;
struct gv_sync_args *sync;
int error;
KASSERT(v != NULL, ("gv_sync: NULL v"));
sc = v->vinumconf;
KASSERT(sc != NULL, ("gv_sync: NULL sc on %s", v->name));
KASSERT(p != NULL, ("%s: NULL p", __func__));
KASSERT(up != NULL, ("%s: NULL up", __func__));
if ((p == up) || (p->state == GV_PLEX_UP))
return (0);
if (p->flags & GV_PLEX_SYNCING ||
p->flags & GV_PLEX_REBUILDING ||
p->flags & GV_PLEX_GROWING) {
return (EINPROGRESS);
}
p->synced = 0;
p->flags |= GV_PLEX_SYNCING;
G_VINUM_DEBUG(1, "starting sync of plex %s", p->name);
error = gv_sync_request(up, p, p->synced,
MIN(GV_DFLT_SYNCSIZE, up->size - p->synced),
BIO_READ, NULL);
if (error) {
G_VINUM_DEBUG(0, "error syncing plex %s", p->name);
return (error);
}
return (0);
}
/* Return a good plex from volume v. */
static struct gv_plex *
gv_find_good_plex(struct gv_volume *v)
{
struct gv_plex *up;
/* Find the plex that's up. */
up = NULL;
@ -331,48 +223,125 @@ gv_sync(struct gv_volume *v)
if (up->state == GV_PLEX_UP)
break;
}
/* Didn't find a good plex. */
return (up);
}
static int
gv_sync(struct gv_volume *v)
{
struct gv_softc *sc;
struct gv_plex *p, *up;
int error;
KASSERT(v != NULL, ("gv_sync: NULL v"));
sc = v->vinumconf;
KASSERT(sc != NULL, ("gv_sync: NULL sc on %s", v->name));
up = gv_find_good_plex(v);
if (up == NULL)
return (ENXIO);
LIST_FOREACH(p, &v->plexes, in_volume) {
if ((p == up) || (p->state == GV_PLEX_UP))
continue;
if (p->flags & GV_PLEX_SYNCING) {
return (EINPROGRESS);
}
p->flags |= GV_PLEX_SYNCING;
sync = g_malloc(sizeof(*sync), M_WAITOK | M_ZERO);
sync->v = v;
sync->from = up;
sync->to = p;
sync->syncsize = GV_DFLT_SYNCSIZE;
kproc_create(gv_sync_td, sync, NULL, 0, 0, "gv_sync '%s'",
p->name);
g_topology_lock();
error = gv_access(v->provider, 1, 1, 0);
if (error) {
g_topology_unlock();
G_VINUM_DEBUG(0, "sync from '%s' failed to access volume: %d",
up->name, error);
return (error);
}
g_topology_unlock();
/* Go through the good plex, and issue BIO's to all other plexes. */
LIST_FOREACH(p, &v->plexes, in_volume) {
error = gv_sync_plex(p, up);
if (error)
break;
}
return (0);
}
static int
gv_rebuild_plex(struct gv_plex *p)
{
struct gv_sync_args *sync;
struct gv_drive *d;
struct gv_sd *s;
int error;
if (gv_is_open(p->geom))
return (EBUSY);
if (p->flags & GV_PLEX_SYNCING)
if (p->flags & GV_PLEX_SYNCING ||
p->flags & GV_PLEX_REBUILDING ||
p->flags & GV_PLEX_GROWING)
return (EINPROGRESS);
p->flags |= GV_PLEX_SYNCING;
/*
* Make sure that all subdisks have consumers. We won't allow a rebuild
* unless every subdisk have one.
*/
LIST_FOREACH(s, &p->subdisks, in_plex) {
d = s->drive_sc;
if (d == NULL || (d->flags & GV_DRIVE_REFERENCED)) {
G_VINUM_DEBUG(0, "can't rebuild %s, subdisk(s) have no "
"drives", p->name);
return (ENXIO);
}
}
p->flags |= GV_PLEX_REBUILDING;
p->synced = 0;
sync = g_malloc(sizeof(*sync), M_WAITOK | M_ZERO);
sync->to = p;
sync->syncsize = GV_DFLT_SYNCSIZE;
g_topology_assert_not();
g_topology_lock();
error = gv_access(p->vol_sc->provider, 1, 1, 0);
if (error) {
G_VINUM_DEBUG(0, "unable to access provider");
return (0);
}
g_topology_unlock();
kproc_create(gv_rebuild_td, sync, NULL, 0, 0, "gv_rebuild %s",
p->name);
gv_parity_request(p, GV_BIO_REBUILD, 0);
return (0);
}
static int
gv_grow_plex(struct gv_plex *p)
{
struct gv_volume *v;
struct gv_sd *s;
off_t origsize, origlength;
int error, sdcount;
KASSERT(p != NULL, ("gv_grow_plex: NULL p"));
v = p->vol_sc;
KASSERT(v != NULL, ("gv_grow_plex: NULL v"));
if (p->flags & GV_PLEX_GROWING ||
p->flags & GV_PLEX_SYNCING ||
p->flags & GV_PLEX_REBUILDING)
return (EINPROGRESS);
g_topology_lock();
error = gv_access(v->provider, 1, 1, 0);
g_topology_unlock();
if (error) {
G_VINUM_DEBUG(0, "unable to access provider");
return (error);
}
/* XXX: This routine with finding origsize is used two other places as
* well, so we should create a function for it. */
sdcount = p->sdcount;
LIST_FOREACH(s, &p->subdisks, in_plex) {
if (s->flags & GV_SD_GROW)
sdcount--;
}
s = LIST_FIRST(&p->subdisks);
if (s == NULL) {
G_VINUM_DEBUG(0, "error growing plex without subdisks");
return (GV_ERR_NOTFOUND);
}
p->flags |= GV_PLEX_GROWING;
origsize = (sdcount - 1) * s->size;
origlength = (sdcount - 1) * p->stripesize;
p->synced = 0;
G_VINUM_DEBUG(1, "starting growing of plex %s", p->name);
gv_grow_request(p, 0, MIN(origlength, origsize), BIO_READ, NULL);
return (0);
}
@ -380,7 +349,11 @@ gv_rebuild_plex(struct gv_plex *p)
static int
gv_init_plex(struct gv_plex *p)
{
struct gv_drive *d;
struct gv_sd *s;
int error;
off_t start;
caddr_t data;
KASSERT(p != NULL, ("gv_init_plex: NULL p"));
@ -389,283 +362,27 @@ gv_init_plex(struct gv_plex *p)
return (EINPROGRESS);
gv_set_sd_state(s, GV_SD_INITIALIZING, GV_SETSTATE_FORCE);
s->init_size = GV_DFLT_SYNCSIZE;
kproc_create(gv_init_td, s, NULL, 0, 0, "gv_init %s",
s->name);
start = s->drive_offset + s->initialized;
d = s->drive_sc;
if (d == NULL) {
G_VINUM_DEBUG(0, "subdisk %s has no drive yet", s->name);
break;
}
/*
* Take the lock here since we need to avoid a race in
* gv_init_request if the BIO is completed before the lock is
* released.
*/
g_topology_lock();
error = g_access(d->consumer, 0, 1, 0);
g_topology_unlock();
if (error) {
G_VINUM_DEBUG(0, "error accessing consumer when "
"initializing %s", s->name);
break;
}
data = g_malloc(s->init_size, M_WAITOK | M_ZERO);
gv_init_request(s, start, data, s->init_size);
}
return (0);
}
/* This thread is responsible for rebuilding a degraded RAID5 plex. */
void
gv_rebuild_td(void *arg)
{
struct bio *bp;
struct gv_plex *p;
struct g_consumer *cp;
struct gv_sync_args *sync;
u_char *buf;
off_t i;
int error;
buf = NULL;
bp = NULL;
sync = arg;
p = sync->to;
p->synced = 0;
cp = p->consumer;
g_topology_lock();
error = g_access(cp, 1, 1, 0);
if (error) {
g_topology_unlock();
G_VINUM_DEBUG(0, "rebuild of %s failed to access consumer: "
"%d", p->name, error);
kproc_exit(error);
}
g_topology_unlock();
buf = g_malloc(sync->syncsize, M_WAITOK);
G_VINUM_DEBUG(1, "rebuild of %s started", p->name);
i = 0;
for (i = 0; i < p->size; i += (p->stripesize * (p->sdcount - 1))) {
/*
if (i + sync->syncsize > p->size)
sync->syncsize = p->size - i;
*/
bp = g_new_bio();
if (bp == NULL) {
G_VINUM_DEBUG(0, "rebuild of %s failed creating bio: "
"out of memory", p->name);
break;
}
bp->bio_cmd = BIO_WRITE;
bp->bio_done = NULL;
bp->bio_data = buf;
bp->bio_cflags |= GV_BIO_REBUILD;
bp->bio_offset = i;
bp->bio_length = p->stripesize;
/* Schedule it down ... */
g_io_request(bp, cp);
/* ... and wait for the result. */
error = biowait(bp, "gwrite");
if (error) {
G_VINUM_DEBUG(0, "rebuild of %s failed at offset %jd "
"errno: %d", p->name, i, error);
break;
}
g_destroy_bio(bp);
bp = NULL;
}
if (bp != NULL)
g_destroy_bio(bp);
if (buf != NULL)
g_free(buf);
g_topology_lock();
g_access(cp, -1, -1, 0);
gv_save_config_all(p->vinumconf);
g_topology_unlock();
p->flags &= ~GV_PLEX_SYNCING;
p->synced = 0;
/* Successful initialization. */
if (!error)
G_VINUM_DEBUG(1, "rebuild of %s finished", p->name);
g_free(sync);
kproc_exit(error);
}
void
gv_sync_td(void *arg)
{
struct bio *bp;
struct gv_plex *p;
struct g_consumer *from, *to;
struct gv_sync_args *sync;
u_char *buf;
off_t i;
int error;
sync = arg;
from = sync->from->consumer;
to = sync->to->consumer;
p = sync->to;
p->synced = 0;
error = 0;
g_topology_lock();
error = g_access(from, 1, 0, 0);
if (error) {
g_topology_unlock();
G_VINUM_DEBUG(0, "sync from '%s' failed to access "
"consumer: %d", sync->from->name, error);
g_free(sync);
kproc_exit(error);
}
error = g_access(to, 0, 1, 0);
if (error) {
g_access(from, -1, 0, 0);
g_topology_unlock();
G_VINUM_DEBUG(0, "sync to '%s' failed to access "
"consumer: %d", p->name, error);
g_free(sync);
kproc_exit(error);
}
g_topology_unlock();
G_VINUM_DEBUG(1, "plex sync %s -> %s started", sync->from->name,
sync->to->name);
for (i = 0; i < p->size; i+= sync->syncsize) {
/* Read some bits from the good plex. */
buf = g_read_data(from, i, sync->syncsize, &error);
if (buf == NULL) {
G_VINUM_DEBUG(0, "sync read from '%s' failed at "
"offset %jd; errno: %d", sync->from->name, i,
error);
break;
}
/*
* Create a bio and schedule it down on the 'bad' plex. We
* cannot simply use g_write_data() because we have to let the
* lower parts know that we are an initialization process and
* not a 'normal' request.
*/
bp = g_new_bio();
if (bp == NULL) {
G_VINUM_DEBUG(0, "sync write to '%s' failed at "
"offset %jd; out of memory", p->name, i);
g_free(buf);
break;
}
bp->bio_cmd = BIO_WRITE;
bp->bio_offset = i;
bp->bio_length = sync->syncsize;
bp->bio_data = buf;
bp->bio_done = NULL;
/*
* This hack declare this bio as part of an initialization
* process, so that the lower levels allow it to get through.
*/
bp->bio_cflags |= GV_BIO_SYNCREQ;
/* Schedule it down ... */
g_io_request(bp, to);
/* ... and wait for the result. */
error = biowait(bp, "gwrite");
g_destroy_bio(bp);
g_free(buf);
if (error) {
G_VINUM_DEBUG(0, "sync write to '%s' failed at "
"offset %jd; errno: %d\n", p->name, i, error);
break;
}
/* Note that we have synced a little bit more. */
p->synced += sync->syncsize;
}
g_topology_lock();
g_access(from, -1, 0, 0);
g_access(to, 0, -1, 0);
gv_save_config_all(p->vinumconf);
g_topology_unlock();
/* Successful initialization. */
if (!error)
G_VINUM_DEBUG(1, "plex sync %s -> %s finished",
sync->from->name, sync->to->name);
p->flags &= ~GV_PLEX_SYNCING;
p->synced = 0;
g_free(sync);
kproc_exit(error);
}
void
gv_init_td(void *arg)
{
struct gv_sd *s;
struct gv_drive *d;
struct g_geom *gp;
struct g_consumer *cp;
int error;
off_t i, init_size, start, offset, length;
u_char *buf;
s = arg;
KASSERT(s != NULL, ("gv_init_td: NULL s"));
d = s->drive_sc;
KASSERT(d != NULL, ("gv_init_td: NULL d"));
gp = d->geom;
KASSERT(gp != NULL, ("gv_init_td: NULL gp"));
cp = LIST_FIRST(&gp->consumer);
KASSERT(cp != NULL, ("gv_init_td: NULL cp"));
s->init_error = 0;
init_size = s->init_size;
start = s->drive_offset + s->initialized;
offset = s->drive_offset;
length = s->size;
buf = g_malloc(s->init_size, M_WAITOK | M_ZERO);
g_topology_lock();
error = g_access(cp, 0, 1, 0);
if (error) {
s->init_error = error;
g_topology_unlock();
G_VINUM_DEBUG(0, "subdisk '%s' init: failed to access "
"consumer; error: %d", s->name, error);
kproc_exit(error);
}
g_topology_unlock();
for (i = start; i < offset + length; i += init_size) {
error = g_write_data(cp, i, buf, init_size);
if (error) {
G_VINUM_DEBUG(0, "subdisk '%s' init: write failed"
" at offset %jd (drive offset %jd); error %d",
s->name, (intmax_t)s->initialized, (intmax_t)i,
error);
break;
}
s->initialized += init_size;
}
g_free(buf);
g_topology_lock();
g_access(cp, 0, -1, 0);
g_topology_unlock();
if (error) {
s->init_error = error;
g_topology_lock();
gv_set_sd_state(s, GV_SD_STALE,
GV_SETSTATE_FORCE | GV_SETSTATE_CONFIG);
g_topology_unlock();
} else {
g_topology_lock();
gv_set_sd_state(s, GV_SD_UP, GV_SETSTATE_CONFIG);
g_topology_unlock();
s->initialized = 0;
G_VINUM_DEBUG(1, "subdisk '%s' init: finished successfully",
s->name);
}
kproc_exit(error);
}

35
sys/geom/vinum/geom_vinum_list.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -28,7 +28,6 @@
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/libkern.h>
#include <sys/malloc.h>
@ -302,19 +301,37 @@ gv_lpi(struct gv_plex *p, struct sbuf *sb, int flags)
sbuf_printf(sb, "Plex %s:\tSize:\t%9jd bytes (%jd MB)\n",
p->name, (intmax_t)p->size, (intmax_t)p->size / MEGABYTE);
sbuf_printf(sb, "\t\tSubdisks: %8d\n", p->sdcount);
sbuf_printf(sb, "\t\tState: %s\n\t\tOrganization: %s",
gv_plexstate(p->state), gv_plexorg(p->org));
sbuf_printf(sb, "\t\tState: %s\n", gv_plexstate(p->state));
if ((p->flags & GV_PLEX_SYNCING) ||
(p->flags & GV_PLEX_GROWING) ||
(p->flags & GV_PLEX_REBUILDING)) {
sbuf_printf(sb, "\t\tSynced: ");
sbuf_printf(sb, "%16jd bytes (%d%%)\n",
(intmax_t)p->synced,
(p->size > 0) ? (int)((p->synced * 100) / p->size) :
0);
}
sbuf_printf(sb, "\t\tOrganization: %s", gv_plexorg(p->org));
if (gv_is_striped(p)) {
sbuf_printf(sb, "\tStripe size: %s\n",
gv_roughlength(p->stripesize, 1));
}
sbuf_printf(sb, "\t\tFlags: %d\n", p->flags);
if (p->vol_sc != NULL) {
sbuf_printf(sb, "\t\tPart of volume %s\n", p->volume);
}
} else {
sbuf_printf(sb, "P %-18s %2s State: %s\tSubdisks: %5d"
"\tSize: %s\n", p->name, gv_plexorg_short(p->org),
gv_plexstate(p->state), p->sdcount,
sbuf_printf(sb, "P %-18s %2s State: ", p->name,
gv_plexorg_short(p->org));
if ((p->flags & GV_PLEX_SYNCING) ||
(p->flags & GV_PLEX_GROWING) ||
(p->flags & GV_PLEX_REBUILDING)) {
sbuf_printf(sb, "S %d%%\t", (int)((p->synced * 100) /
p->size));
} else {
sbuf_printf(sb, "%s\t", gv_plexstate(p->state));
}
sbuf_printf(sb, "Subdisks: %5d\tSize: %s\n", p->sdcount,
gv_roughlength(p->size, 0));
}
@ -396,6 +413,7 @@ gv_lsi(struct gv_sd *s, struct sbuf *sb, int flags)
s->drive_sc == NULL ? "*missing*" : s->drive_sc->name,
(intmax_t)s->drive_offset,
gv_roughlength(s->drive_offset, 1));
sbuf_printf(sb, "\t\tFlags: %d\n", s->flags);
} else {
sbuf_printf(sb, "S %-21s State: ", s->name);
if (s->state == GV_SD_INITIALIZING ||
@ -455,6 +473,7 @@ gv_ldi(struct gv_drive *d, struct sbuf *sb, int flags)
sbuf_printf(sb, "\t\tAvailable: %11jd bytes (%jd MB)\n",
(intmax_t)d->avail, (intmax_t)d->avail / MEGABYTE);
sbuf_printf(sb, "\t\tState: %s\n", gv_drivestate(d->state));
sbuf_printf(sb, "\t\tFlags: %d\n", d->flags);
/* Be very verbose. */
if (flags & GV_FLAG_VV) {
@ -469,7 +488,7 @@ gv_ldi(struct gv_drive *d, struct sbuf *sb, int flags)
sbuf_printf(sb, "D %-21s State: %s\t/dev/%s\tA: %jd/%jd MB "
"(%d%%)\n", d->name, gv_drivestate(d->state), d->device,
(intmax_t)d->avail / MEGABYTE, (intmax_t)d->size / MEGABYTE,
(int)((d->avail * 100) / d->size));
d->size > 0 ? (int)((d->avail * 100) / d->size) : 0);
}
/* Recursive listing. */

106
sys/geom/vinum/geom_vinum_move.c
View File

@ -32,26 +32,21 @@
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/libkern.h>
#include <sys/kernel.h>
#include <sys/malloc.h>
#include <geom/geom.h>
#include <geom/vinum/geom_vinum_var.h>
#include <geom/vinum/geom_vinum.h>
#include <geom/vinum/geom_vinum_share.h>
static int gv_move_sd(struct gv_softc *, struct gctl_req *,
struct gv_sd *, char *, int);
void
gv_move(struct g_geom *gp, struct gctl_req *req)
{
struct gv_softc *sc;
struct gv_sd *s;
struct gv_drive *d;
char buf[20], *destination, *object;
int *argc, err, *flags, i, type;
int *argc, *flags, i, type;
sc = gp->softc;
@ -74,6 +69,7 @@ gv_move(struct g_geom *gp, struct gctl_req *req)
gctl_error(req, "destination '%s' is not a drive", destination);
return;
}
d = gv_find_drive(sc, destination);
/*
* We start with 1 here, because argv[0] on the command line is the
@ -97,67 +93,59 @@ gv_move(struct g_geom *gp, struct gctl_req *req)
gctl_error(req, "unknown subdisk '%s'", object);
return;
}
err = gv_move_sd(sc, req, s, destination, *flags);
if (err)
return;
gv_post_event(sc, GV_EVENT_MOVE_SD, s, d, *flags, 0);
}
gv_save_config_all(sc);
}
/* Move a subdisk. */
static int
gv_move_sd(struct gv_softc *sc, struct gctl_req *req, struct gv_sd *cursd, char *destination, int flags)
int
gv_move_sd(struct gv_softc *sc, struct gv_sd *cursd,
struct gv_drive *destination, int flags)
{
struct gv_drive *d;
struct gv_sd *newsd, *s, *s2;
struct gv_plex *p;
struct g_consumer *cp;
char errstr[ERRBUFSIZ];
int err;
g_topology_assert();
KASSERT(cursd != NULL, ("gv_move_sd: NULL cursd"));
KASSERT(destination != NULL, ("gv_move_sd: NULL destination"));
cp = cursd->consumer;
d = cursd->drive_sc;
if (cp != NULL && (cp->acr || cp->acw || cp->ace)) {
gctl_error(req, "subdisk '%s' is busy", cursd->name);
return (-1);
if ((gv_consumer_is_open(d->consumer) ||
gv_consumer_is_open(destination->consumer)) &&
!(flags && GV_FLAG_F)) {
G_VINUM_DEBUG(0, "consumers on current and destination drive "
" still open");
return (GV_ERR_ISBUSY);
}
if (!(flags && GV_FLAG_F)) {
gctl_error(req, "-f flag not passed; move would be "
G_VINUM_DEBUG(1, "-f flag not passed; move would be "
"destructive");
return (-1);
return (GV_ERR_INVFLAG);
}
d = gv_find_drive(sc, destination);
if (d == NULL) {
gctl_error(req, "destination drive '%s' not found",
destination);
return (-1);
}
if (d == cursd->drive_sc) {
gctl_error(req, "subdisk '%s' already on drive '%s'",
cursd->name, destination);
return (-1);
if (destination == cursd->drive_sc) {
G_VINUM_DEBUG(1, "subdisk '%s' already on drive '%s'",
cursd->name, destination->name);
return (GV_ERR_ISATTACHED);
}
/* XXX: Does it have to be part of a plex? */
p = gv_find_plex(sc, cursd->plex);
if (p == NULL) {
gctl_error(req, "subdisk '%s' is not part of a plex",
G_VINUM_DEBUG(0, "subdisk '%s' is not part of a plex",
cursd->name);
return (-1);
return (GV_ERR_NOTFOUND);
}
/* Stale the old subdisk. */
err = gv_set_sd_state(cursd, GV_SD_STALE,
GV_SETSTATE_FORCE | GV_SETSTATE_CONFIG);
if (err) {
gctl_error(req, "could not set the subdisk '%s' to state "
G_VINUM_DEBUG(0, "could not set the subdisk '%s' to state "
"'stale'", cursd->name);
return (err);
}
@ -171,54 +159,30 @@ gv_move_sd(struct gv_softc *sc, struct gctl_req *req, struct gv_sd *cursd, char
newsd->plex_offset = cursd->plex_offset;
newsd->size = cursd->size;
newsd->drive_offset = -1;
strncpy(newsd->name, cursd->name, GV_MAXSDNAME);
strncpy(newsd->drive, destination, GV_MAXDRIVENAME);
strncpy(newsd->plex, cursd->plex, GV_MAXPLEXNAME);
strlcpy(newsd->name, cursd->name, sizeof(newsd->name));
strlcpy(newsd->drive, destination->name, sizeof(newsd->drive));
strlcpy(newsd->plex, cursd->plex, sizeof(newsd->plex));
newsd->state = GV_SD_STALE;
newsd->vinumconf = cursd->vinumconf;
err = gv_sd_to_drive(sc, d, newsd, errstr, ERRBUFSIZ);
err = gv_sd_to_drive(newsd, destination);
if (err) {
/* XXX not enough free space? */
gctl_error(req, errstr);
g_free(newsd);
return (err);
}
/* Replace the old sd by the new one. */
if (cp != NULL)
g_detach(cp);
LIST_FOREACH_SAFE(s, &p->subdisks, in_plex, s2) {
if (s == cursd) {
p->sdcount--;
p->size -= s->size;
err = gv_rm_sd(sc, req, s, 0);
if (err)
return (err);
gv_rm_sd(sc, s);
}
}
gv_sd_to_plex(p, newsd, 1);
/* Creates the new providers.... */
gv_drive_modify(d);
/* And reconnect the consumer ... */
if (cp != NULL) {
newsd->consumer = cp;
err = g_attach(cp, newsd->provider);
if (err) {
g_destroy_consumer(cp);
gctl_error(req, "proposed move would create a loop "
"in GEOM config");
return (err);
}
}
gv_sd_to_plex(newsd, p);
LIST_INSERT_HEAD(&sc->subdisks, newsd, sd);
gv_save_config_all(sc);
/* Update volume size of plex. */
if (p->vol_sc != NULL)
gv_update_vol_size(p->vol_sc, gv_vol_size(p->vol_sc));
gv_save_config(p->vinumconf);
return (0);
}

1523
sys/geom/vinum/geom_vinum_plex.c
View File

File diff suppressed because it is too large Load Diff

380
sys/geom/vinum/geom_vinum_raid5.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -29,14 +29,8 @@ __FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/bio.h>
#include <sys/conf.h>
#include <sys/errno.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/libkern.h>
#include <sys/lock.h>
#include <sys/malloc.h>
#include <sys/mutex.h>
#include <sys/systm.h>
#include <geom/geom.h>
@ -44,8 +38,103 @@ __FBSDID("$FreeBSD$");
#include <geom/vinum/geom_vinum_raid5.h>
#include <geom/vinum/geom_vinum.h>
int gv_raid5_offset(struct gv_plex *, off_t, off_t, off_t *, off_t *,
int *, int *);
static int gv_raid5_offset(struct gv_plex *, off_t, off_t,
off_t *, off_t *, int *, int *, int);
static struct bio * gv_raid5_clone_bio(struct bio *, struct gv_sd *,
struct gv_raid5_packet *, caddr_t, int);
static int gv_raid5_request(struct gv_plex *, struct gv_raid5_packet *,
struct bio *, caddr_t, off_t, off_t, int *);
static int gv_raid5_check(struct gv_plex *, struct gv_raid5_packet *,
struct bio *, caddr_t, off_t, off_t);
static int gv_raid5_rebuild(struct gv_plex *, struct gv_raid5_packet *,
struct bio *, caddr_t, off_t, off_t);
struct gv_raid5_packet *
gv_raid5_start(struct gv_plex *p, struct bio *bp, caddr_t addr, off_t boff,
off_t bcount)
{
struct bio *cbp;
struct gv_raid5_packet *wp, *wp2;
struct gv_bioq *bq, *bq2;
int err, delay;
delay = 0;
wp = g_malloc(sizeof(*wp), M_WAITOK | M_ZERO);
wp->bio = bp;
wp->waiting = NULL;
wp->parity = NULL;
TAILQ_INIT(&wp->bits);
if (bp->bio_cflags & GV_BIO_REBUILD)
err = gv_raid5_rebuild(p, wp, bp, addr, boff, bcount);
else if (bp->bio_cflags & GV_BIO_CHECK)
err = gv_raid5_check(p, wp, bp, addr, boff, bcount);
else
err = gv_raid5_request(p, wp, bp, addr, boff, bcount, &delay);
/* Means we have a delayed request. */
if (delay) {
g_free(wp);
return (NULL);
}
/*
* Building the sub-request failed, we probably need to clean up a lot.
*/
if (err) {
G_VINUM_LOGREQ(0, bp, "raid5 plex request failed.");
TAILQ_FOREACH_SAFE(bq, &wp->bits, queue, bq2) {
TAILQ_REMOVE(&wp->bits, bq, queue);
g_free(bq);
}
if (wp->waiting != NULL) {
if (wp->waiting->bio_cflags & GV_BIO_MALLOC)
g_free(wp->waiting->bio_data);
g_destroy_bio(wp->waiting);
}
if (wp->parity != NULL) {
if (wp->parity->bio_cflags & GV_BIO_MALLOC)
g_free(wp->parity->bio_data);
g_destroy_bio(wp->parity);
}
g_free(wp);
TAILQ_FOREACH_SAFE(wp, &p->packets, list, wp2) {
if (wp->bio != bp)
continue;
TAILQ_REMOVE(&p->packets, wp, list);
TAILQ_FOREACH_SAFE(bq, &wp->bits, queue, bq2) {
TAILQ_REMOVE(&wp->bits, bq, queue);
g_free(bq);
}
g_free(wp);
}
cbp = bioq_takefirst(p->bqueue);
while (cbp != NULL) {
if (cbp->bio_cflags & GV_BIO_MALLOC)
g_free(cbp->bio_data);
g_destroy_bio(cbp);
cbp = bioq_takefirst(p->bqueue);
}
/* If internal, stop and reset state. */
if (bp->bio_cflags & GV_BIO_INTERNAL) {
if (bp->bio_cflags & GV_BIO_MALLOC)
g_free(cbp->bio_data);
g_destroy_bio(bp);
/* Reset flags. */
p->flags &= ~(GV_PLEX_SYNCING | GV_PLEX_REBUILDING |
GV_PLEX_GROWING);
return (NULL);
}
g_io_deliver(bp, err);
return (NULL);
}
return (wp);
}
/*
* Check if the stripe that the work packet wants is already being used by
@ -57,7 +146,7 @@ gv_stripe_active(struct gv_plex *p, struct bio *bp)
struct gv_raid5_packet *wp, *owp;
int overlap;
wp = bp->bio_driver1;
wp = bp->bio_caller2;
if (wp->lockbase == -1)
return (0);
@ -80,20 +169,20 @@ gv_stripe_active(struct gv_plex *p, struct bio *bp)
return (overlap);
}
int
gv_check_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
static int
gv_raid5_check(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
caddr_t addr, off_t boff, off_t bcount)
{
struct gv_sd *parity, *s;
struct gv_bioq *bq;
struct bio *cbp, *pbp;
struct bio *cbp;
int i, psdno;
off_t real_len, real_off;
if (p == NULL || LIST_EMPTY(&p->subdisks))
return (ENXIO);
gv_raid5_offset(p, boff, bcount, &real_off, &real_len, NULL, &psdno);
gv_raid5_offset(p, boff, bcount, &real_off, &real_len, NULL, &psdno, 1);
/* Find the right subdisk. */
parity = NULL;
@ -122,20 +211,16 @@ gv_check_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
/* Skip the parity subdisk. */
if (s == parity)
continue;
/* Skip growing subdisks. */
if (s->flags & GV_SD_GROW)
continue;
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, s, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_cmd = BIO_READ;
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = s->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
@ -143,51 +228,38 @@ gv_check_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
}
/* Read the parity data. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, parity, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_cmd = BIO_READ;
cbp->bio_data = g_malloc(real_len, M_WAITOK | M_ZERO);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = parity->consumer;
cbp->bio_driver1 = wp;
wp->waiting = cbp;
/*
* In case we want to rebuild the parity, create an extra BIO to write
* it out. It also acts as buffer for the XOR operations.
*/
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, parity, wp, addr, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_data = addr;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = parity->consumer;
cbp->bio_driver1 = wp;
wp->parity = cbp;
return (0);
}
/* Rebuild a degraded RAID5 plex. */
int
gv_rebuild_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
static int
gv_raid5_rebuild(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
caddr_t addr, off_t boff, off_t bcount)
{
struct gv_sd *broken, *s;
struct gv_bioq *bq;
struct bio *cbp, *pbp;
struct bio *cbp;
off_t real_len, real_off;
if (p == NULL || LIST_EMPTY(&p->subdisks))
return (ENXIO);
gv_raid5_offset(p, boff, bcount, &real_off, &real_len, NULL, NULL);
gv_raid5_offset(p, boff, bcount, &real_off, &real_len, NULL, NULL, 1);
/* Find the right subdisk. */
broken = NULL;
@ -210,6 +282,8 @@ gv_rebuild_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
G_VINUM_DEBUG(1, "sd %s is reviving", broken->name);
gv_set_sd_state(broken, GV_SD_REVIVING, GV_SETSTATE_FORCE);
/* Set this bit now, but should be set at end. */
broken->flags |= GV_SD_CANGOUP;
break;
case GV_SD_REVIVING:
@ -232,19 +306,16 @@ gv_rebuild_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
if (s == broken)
continue;
cbp = g_clone_bio(bp);
/* Skip growing subdisks. */
if (s->flags & GV_SD_GROW)
continue;
cbp = gv_raid5_clone_bio(bp, s, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_cmd = BIO_READ;
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = s->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
@ -252,34 +323,28 @@ gv_rebuild_raid5(struct gv_plex *p, struct gv_raid5_packet *wp, struct bio *bp,
}
/* Write the parity data. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, broken, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_data = g_malloc(real_len, M_WAITOK | M_ZERO);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = broken->consumer;
cbp->bio_driver1 = wp;
cbp->bio_cflags |= GV_BIO_REBUILD;
wp->parity = cbp;
p->synced = boff;
/* Post notification that we're finished. */
return (0);
}
/* Build a request group to perform (part of) a RAID5 request. */
int
gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
struct bio *bp, caddr_t addr, off_t boff, off_t bcount)
static int
gv_raid5_request(struct gv_plex *p, struct gv_raid5_packet *wp,
struct bio *bp, caddr_t addr, off_t boff, off_t bcount, int *delay)
{
struct g_geom *gp;
struct gv_sd *broken, *original, *parity, *s;
struct gv_bioq *bq;
struct bio *cbp, *pbp;
int i, psdno, sdno, type;
struct bio *cbp;
int i, psdno, sdno, type, grow;
off_t real_len, real_off;
gp = bp->bio_to->geom;
@ -295,7 +360,24 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
type = REQ_TYPE_NORMAL;
original = parity = broken = NULL;
gv_raid5_offset(p, boff, bcount, &real_off, &real_len, &sdno, &psdno);
/* XXX: The resize won't crash with rebuild or sync, but we should still
* be aware of it. Also this should perhaps be done on rebuild/check as
* well?
*/
/* If we're over, we must use the old. */
if (boff >= p->synced) {
grow = 1;
/* Or if over the resized offset, we use all drives. */
} else if (boff + bcount <= p->synced) {
grow = 0;
/* Else, we're in the middle, and must wait a bit. */
} else {
bioq_disksort(p->rqueue, bp);
*delay = 1;
return (0);
}
gv_raid5_offset(p, boff, bcount, &real_off, &real_len,
&sdno, &psdno, grow);
/* Find the right subdisks. */
i = 0;
@ -315,8 +397,13 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
/* Our data stripe is missing. */
if (original->state != GV_SD_UP)
type = REQ_TYPE_DEGRADED;
/* If synchronizing request, just write it if disks are stale. */
if (original->state == GV_SD_STALE && parity->state == GV_SD_STALE &&
bp->bio_cflags & GV_BIO_SYNCREQ && bp->bio_cmd == BIO_WRITE) {
type = REQ_TYPE_NORMAL;
/* Our parity stripe is missing. */
if (parity->state != GV_SD_UP) {
} else if (parity->state != GV_SD_UP) {
/* We cannot take another failure if we're already degraded. */
if (type != REQ_TYPE_NORMAL)
return (ENXIO);
@ -330,9 +417,15 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
KASSERT(wp->length >= 0, ("gv_build_raid5_request: wp->length < 0"));
if ((p->flags & GV_PLEX_SYNCING) && (boff + real_len < p->synced))
if ((p->flags & GV_PLEX_REBUILDING) && (boff + real_len < p->synced))
type = REQ_TYPE_NORMAL;
if ((p->flags & GV_PLEX_REBUILDING) && (boff + real_len >= p->synced)) {
bioq_disksort(p->rqueue, bp);
*delay = 1;
return (0);
}
switch (bp->bio_cmd) {
case BIO_READ:
/*
@ -346,18 +439,14 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
/* Skip the broken subdisk. */
if (s == broken)
continue;
cbp = g_clone_bio(bp);
/* Skip growing if within offset. */
if (grow && s->flags & GV_SD_GROW)
continue;
cbp = gv_raid5_clone_bio(bp, s, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = s->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
@ -366,16 +455,11 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
/* A normal read can be fulfilled with the original subdisk. */
} else {
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, original, wp, addr, 0);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_data = addr;
cbp->bio_done = g_std_done;
cbp->bio_caller2 = original->consumer;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
}
wp->lockbase = -1;
@ -394,20 +478,16 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
/* Skip the broken and the parity subdisk. */
if ((s == broken) || (s == parity))
continue;
/* Skip growing if within offset. */
if (grow && s->flags & GV_SD_GROW)
continue;
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, s, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_cmd = BIO_READ;
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = s->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
@ -415,34 +495,21 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
}
/* Write the parity data. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, parity, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
bcopy(addr, cbp->bio_data, real_len);
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = parity->consumer;
cbp->bio_driver1 = wp;
bcopy(addr, cbp->bio_data, wp->length);
wp->parity = cbp;
/*
* When the parity stripe is missing we just write out the data.
*/
} else if (type == REQ_TYPE_NOPARITY) {
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, original, wp, addr, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_data = addr;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = original->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
@ -455,54 +522,33 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
*/
} else {
/* Read old parity. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, parity, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_cmd = BIO_READ;
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = parity->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
TAILQ_INSERT_TAIL(&wp->bits, bq, queue);
/* Read old data. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, original, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_cmd = BIO_READ;
cbp->bio_data = g_malloc(real_len, M_WAITOK);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = original->consumer;
cbp->bio_driver1 = wp;
GV_ENQUEUE(bp, cbp, pbp);
bioq_insert_tail(p->bqueue, cbp);
bq = g_malloc(sizeof(*bq), M_WAITOK | M_ZERO);
bq->bp = cbp;
TAILQ_INSERT_TAIL(&wp->bits, bq, queue);
/* Write new data. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, original, wp, addr, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_data = addr;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = original->consumer;
cbp->bio_driver1 = wp;
/*
* We must not write the new data until the old data
@ -512,16 +558,9 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
wp->waiting = cbp;
/* The final bio for the parity. */
cbp = g_clone_bio(bp);
cbp = gv_raid5_clone_bio(bp, parity, wp, NULL, 1);
if (cbp == NULL)
return (ENOMEM);
cbp->bio_data = g_malloc(real_len, M_WAITOK | M_ZERO);
cbp->bio_cflags |= GV_BIO_MALLOC;
cbp->bio_offset = real_off;
cbp->bio_length = real_len;
cbp->bio_done = gv_plex_done;
cbp->bio_caller2 = parity->consumer;
cbp->bio_driver1 = wp;
/* Remember that this is the BIO for the parity data. */
wp->parity = cbp;
@ -535,21 +574,36 @@ gv_build_raid5_req(struct gv_plex *p, struct gv_raid5_packet *wp,
return (0);
}
/* Calculate the offsets in the various subdisks for a RAID5 request. */
int
/*
* Calculate the offsets in the various subdisks for a RAID5 request. Also take
* care of new subdisks in an expanded RAID5 array.
* XXX: This assumes that the new subdisks are inserted after the others (which
* is okay as long as plex_offset is larger). If subdisks are inserted into the
* plexlist before, we get problems.
*/
static int
gv_raid5_offset(struct gv_plex *p, off_t boff, off_t bcount, off_t *real_off,
off_t *real_len, int *sdno, int *psdno)
off_t *real_len, int *sdno, int *psdno, int growing)
{
int sd, psd;
struct gv_sd *s;
int sd, psd, sdcount;
off_t len_left, stripeend, stripeoff, stripestart;
sdcount = p->sdcount;
if (growing) {
LIST_FOREACH(s, &p->subdisks, in_plex) {
if (s->flags & GV_SD_GROW)
sdcount--;
}
}
/* The number of the subdisk containing the parity stripe. */
psd = p->sdcount - 1 - ( boff / (p->stripesize * (p->sdcount - 1))) %
p->sdcount;
psd = sdcount - 1 - ( boff / (p->stripesize * (sdcount - 1))) %
sdcount;
KASSERT(psdno >= 0, ("gv_raid5_offset: psdno < 0"));
/* Offset of the start address from the start of the stripe. */
stripeoff = boff % (p->stripesize * (p->sdcount - 1));
stripeoff = boff % (p->stripesize * (sdcount - 1));
KASSERT(stripeoff >= 0, ("gv_raid5_offset: stripeoff < 0"));
/* The number of the subdisk where the stripe resides. */
@ -561,7 +615,7 @@ gv_raid5_offset(struct gv_plex *p, off_t boff, off_t bcount, off_t *real_off,
sd++;
/* The offset of the stripe on this subdisk. */
stripestart = (boff - stripeoff) / (p->sdcount - 1);
stripestart = (boff - stripeoff) / (sdcount - 1);
KASSERT(stripestart >= 0, ("gv_raid5_offset: stripestart < 0"));
stripeoff %= p->stripesize;
@ -582,3 +636,27 @@ gv_raid5_offset(struct gv_plex *p, off_t boff, off_t bcount, off_t *real_off,
return (0);
}
static struct bio *
gv_raid5_clone_bio(struct bio *bp, struct gv_sd *s, struct gv_raid5_packet *wp,
caddr_t addr, int use_wp)
{
struct bio *cbp;
cbp = g_clone_bio(bp);
if (cbp == NULL)
return (NULL);
if (addr == NULL) {
cbp->bio_data = g_malloc(wp->length, M_WAITOK | M_ZERO);
cbp->bio_cflags |= GV_BIO_MALLOC;
} else
cbp->bio_data = addr;
cbp->bio_offset = wp->lockbase + s->drive_offset;
cbp->bio_length = wp->length;
cbp->bio_done = gv_done;
cbp->bio_caller1 = s;
if (use_wp)
cbp->bio_caller2 = wp;
return (cbp);
}

28
sys/geom/vinum/geom_vinum_raid5.h
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -35,26 +35,10 @@
* transaction (read or write).
*/
#define GV_ENQUEUE(bp, cbp, pbp) \
do { \
if (bp->bio_driver1 == NULL) { \
bp->bio_driver1 = cbp; \
} else { \
pbp = bp->bio_driver1; \
while (pbp->bio_caller1 != NULL) \
pbp = pbp->bio_caller1; \
pbp->bio_caller1 = cbp; \
} \
} while (0)
struct gv_raid5_packet {
caddr_t data; /* Data buffer of this sub-request- */
off_t length; /* Size of data buffer. */
off_t lockbase; /* Deny access to our plex offset. */
off_t offset; /* The drive offset of the subdisk. */
int bufmalloc; /* Flag if data buffer was malloced. */
int active; /* Count of active subrequests. */
int rqcount; /* Count of subrequests. */
struct bio *bio; /* Pointer to the original bio. */
struct bio *parity; /* The bio containing the parity data. */
@ -64,14 +48,8 @@ struct gv_raid5_packet {
TAILQ_ENTRY(gv_raid5_packet) list; /* Entry in plex's packet list. */
};
struct gv_raid5_packet * gv_raid5_start(struct gv_plex *, struct bio *,
caddr_t, off_t, off_t);
int gv_stripe_active(struct gv_plex *, struct bio *);
int gv_build_raid5_req(struct gv_plex *, struct gv_raid5_packet *,
struct bio *, caddr_t, off_t, off_t);
int gv_check_raid5(struct gv_plex *, struct gv_raid5_packet *,
struct bio *, caddr_t, off_t, off_t);
int gv_rebuild_raid5(struct gv_plex *, struct gv_raid5_packet *,
struct bio *, caddr_t, off_t, off_t);
void gv_raid5_worker(void *);
void gv_plex_done(struct bio *);
#endif /* !_GEOM_VINUM_RAID5_H_ */

245
sys/geom/vinum/geom_vinum_rename.c
View File

@ -34,22 +34,11 @@ __FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/libkern.h>
#include <sys/kernel.h>
#include <sys/malloc.h>
#include <geom/geom.h>
#include <geom/vinum/geom_vinum_var.h>
#include <geom/vinum/geom_vinum.h>
#include <geom/vinum/geom_vinum_share.h>
static int gv_rename_drive(struct gv_softc *, struct gctl_req *,
struct gv_drive *, char *, int);
static int gv_rename_plex(struct gv_softc *, struct gctl_req *,
struct gv_plex *, char *, int);
static int gv_rename_sd(struct gv_softc *, struct gctl_req *,
struct gv_sd *, char *, int);
static int gv_rename_vol(struct gv_softc *, struct gctl_req *,
struct gv_volume *, char *, int);
void
gv_rename(struct g_geom *gp, struct gctl_req *req)
@ -59,8 +48,8 @@ gv_rename(struct g_geom *gp, struct gctl_req *req)
struct gv_plex *p;
struct gv_sd *s;
struct gv_drive *d;
char *newname, *object;
int err, *flags, type;
char *newname, *object, *name;
int *flags, type;
sc = gp->softc;
@ -90,9 +79,9 @@ gv_rename(struct g_geom *gp, struct gctl_req *req)
gctl_error(req, "unknown volume '%s'", object);
return;
}
err = gv_rename_vol(sc, req, v, newname, *flags);
if (err)
return;
name = g_malloc(GV_MAXVOLNAME, M_WAITOK | M_ZERO);
strlcpy(name, newname, GV_MAXVOLNAME);
gv_post_event(sc, GV_EVENT_RENAME_VOL, v, name, *flags, 0);
break;
case GV_TYPE_PLEX:
p = gv_find_plex(sc, object);
@ -100,9 +89,9 @@ gv_rename(struct g_geom *gp, struct gctl_req *req)
gctl_error(req, "unknown plex '%s'", object);
return;
}
err = gv_rename_plex(sc, req, p, newname, *flags);
if (err)
return;
name = g_malloc(GV_MAXPLEXNAME, M_WAITOK | M_ZERO);
strlcpy(name, newname, GV_MAXPLEXNAME);
gv_post_event(sc, GV_EVENT_RENAME_PLEX, p, name, *flags, 0);
break;
case GV_TYPE_SD:
s = gv_find_sd(sc, object);
@ -110,9 +99,9 @@ gv_rename(struct g_geom *gp, struct gctl_req *req)
gctl_error(req, "unknown subdisk '%s'", object);
return;
}
err = gv_rename_sd(sc, req, s, newname, *flags);
if (err)
return;
name = g_malloc(GV_MAXSDNAME, M_WAITOK | M_ZERO);
strlcpy(name, newname, GV_MAXSDNAME);
gv_post_event(sc, GV_EVENT_RENAME_SD, s, name, *flags, 0);
break;
case GV_TYPE_DRIVE:
d = gv_find_drive(sc, object);
@ -120,122 +109,88 @@ gv_rename(struct g_geom *gp, struct gctl_req *req)
gctl_error(req, "unknown drive '%s'", object);
return;
}
err = gv_rename_drive(sc, req, d, newname, *flags);
if (err)
return;
name = g_malloc(GV_MAXDRIVENAME, M_WAITOK | M_ZERO);
strlcpy(name, newname, GV_MAXDRIVENAME);
gv_post_event(sc, GV_EVENT_RENAME_DRIVE, d, name, *flags, 0);
break;
default:
gctl_error(req, "unknown object '%s'", object);
return;
}
gv_save_config_all(sc);
}
static int
gv_rename_drive(struct gv_softc *sc, struct gctl_req *req, struct gv_drive *d, char *newname, int flags)
int
gv_rename_drive(struct gv_softc *sc, struct gv_drive *d, char *newname,
int flags)
{
struct gv_sd *s;
g_topology_assert();
KASSERT(d != NULL, ("gv_rename_drive: NULL d"));
if (gv_object_type(sc, newname) != -1) {
gctl_error(req, "drive name '%s' already in use", newname);
return (-1);
if (gv_object_type(sc, newname) != GV_ERR_NOTFOUND) {
G_VINUM_DEBUG(1, "drive name '%s' already in use", newname);
return (GV_ERR_NAMETAKEN);
}
strncpy(d->name, newname, GV_MAXDRIVENAME);
strncpy(d->hdr->label.name, newname, GV_MAXDRIVENAME);
/* XXX can we rename providers here? */
strlcpy(d->name, newname, sizeof(d->name));
if (d->hdr != NULL)
strlcpy(d->hdr->label.name, newname, sizeof(d->hdr->label.name));
LIST_FOREACH(s, &d->subdisks, from_drive)
strncpy(s->drive, d->name, GV_MAXDRIVENAME);
strlcpy(s->drive, d->name, sizeof(s->drive));
return (0);
}
static int
gv_rename_plex(struct gv_softc *sc, struct gctl_req *req, struct gv_plex *p, char *newname, int flags)
int
gv_rename_plex(struct gv_softc *sc, struct gv_plex *p, char *newname, int flags)
{
char newsd[GV_MAXSDNAME];
struct gv_sd *s;
char *plexnum, *plexnump, *oldplex, *oldplexp;
char *newsd, *oldsd, *oldsdp;
char *ptr;
int err;
g_topology_assert();
KASSERT(p != NULL, ("gv_rename_plex: NULL p"));
err = 0;
if (gv_object_type(sc, newname) != -1) {
gctl_error(req, "plex name '%s' already in use", newname);
return (-1);
if (gv_object_type(sc, newname) != GV_ERR_NOTFOUND) {
G_VINUM_DEBUG(1, "plex name '%s' already in use", newname);
return (GV_ERR_NAMETAKEN);
}
/* Needed for sanity checking. */
plexnum = g_malloc(GV_MAXPLEXNAME, M_WAITOK | M_ZERO);
strncpy(plexnum, newname, GV_MAXPLEXNAME);
plexnump = plexnum;
oldplex = g_malloc(GV_MAXPLEXNAME, M_WAITOK | M_ZERO);
strncpy(oldplex, p->name, GV_MAXPLEXNAME);
oldplexp = oldplex;
/*
* Locate the plex number part of the plex names.
*
* XXX: can we be sure that the current plex name has the format
* 'foo.pX'?
* XXX: might be a good idea to sanitize input a bit more
*/
strsep(&oldplexp, ".");
strsep(&plexnump, ".");
if (plexnump == NULL || *plexnump == '\0') {
gctl_error(req, "proposed plex name '%s' is not a valid plex "
ptr = strrchr(newname, '.');
if (ptr == NULL) {
G_VINUM_DEBUG(0, "proposed plex name '%s' is not a valid plex "
"name", newname);
err = -1;
goto failure;
}
if (strcmp(oldplexp, plexnump)) {
gctl_error(req, "current and proposed plex numbers (%s, %s) "
"do not match", plexnump, oldplexp);
err = -1;
goto failure;
return (GV_ERR_INVNAME);
}
strncpy(p->name, newname, GV_MAXPLEXNAME);
/* XXX can we rename providers here? */
strlcpy(p->name, newname, sizeof(p->name));
/* Fix up references and potentially rename subdisks. */
LIST_FOREACH(s, &p->subdisks, in_plex) {
strncpy(s->plex, p->name, GV_MAXPLEXNAME);
strlcpy(s->plex, p->name, sizeof(s->plex));
if (flags && GV_FLAG_R) {
newsd = g_malloc(GV_MAXSDNAME, M_WAITOK | M_ZERO);
oldsd = g_malloc(GV_MAXSDNAME, M_WAITOK | M_ZERO);
oldsdp = oldsd;
strncpy(oldsd, s->name, GV_MAXSDNAME);
/*
* XXX: can we be sure that the current sd name has the
* format 'foo.pX.sY'?
* Look for the two last dots in the string, and assume
* that the old value was ok.
*/
strsep(&oldsdp, ".");
strsep(&oldsdp, ".");
snprintf(newsd, GV_MAXSDNAME, "%s.%s", p->name, oldsdp);
err = gv_rename_sd(sc, req, s, newsd, flags);
g_free(newsd);
g_free(oldsd);
ptr = strrchr(s->name, '.');
if (ptr == NULL)
return (GV_ERR_INVNAME);
ptr++;
snprintf(newsd, sizeof(newsd), "%s.%s", p->name, ptr);
err = gv_rename_sd(sc, s, newsd, flags);
if (err)
goto failure;
return (err);
}
}
failure:
g_free(plexnum);
g_free(oldplex);
return (err);
return (0);
}
/*
@ -243,106 +198,64 @@ gv_rename_plex(struct gv_softc *sc, struct gctl_req *req, struct gv_plex *p, cha
* since there are no structures below a subdisk. Similarly, we don't have to
* clean up any references elsewhere to the subdisk's name.
*/
static int
gv_rename_sd(struct gv_softc *sc, struct gctl_req *req, struct gv_sd *s, char * newname, int flags)
int
gv_rename_sd(struct gv_softc *sc, struct gv_sd *s, char *newname, int flags)
{
char *new, *newp, *old, *oldp;
int err;
char *dot1, *dot2;
g_topology_assert();
KASSERT(s != NULL, ("gv_rename_sd: NULL s"));
err = 0;
if (gv_object_type(sc, newname) != -1) {
gctl_error(req, "subdisk name %s already in use", newname);
return (-1);
if (gv_object_type(sc, newname) != GV_ERR_NOTFOUND) {
G_VINUM_DEBUG(1, "subdisk name %s already in use", newname);
return (GV_ERR_NAMETAKEN);
}
/* Needed for sanity checking. */
new = g_malloc(GV_MAXSDNAME, M_WAITOK | M_ZERO);
strncpy(new, newname, GV_MAXSDNAME);
newp = new;
old = g_malloc(GV_MAXSDNAME, M_WAITOK | M_ZERO);
strncpy(old, s->name, GV_MAXSDNAME);
oldp = old;
/*
* Locate the sd number part of the sd names.
*
* XXX: can we be sure that the current sd name has the format
* 'foo.pX.sY'?
*/
strsep(&oldp, ".");
strsep(&oldp, ".");
strsep(&newp, ".");
if (newp == NULL || *newp == '\0') {
gctl_error(req, "proposed sd name '%s' is not a valid sd name",
/* Locate the sd number part of the sd names. */
dot1 = strchr(newname, '.');
if (dot1 == NULL || (dot2 = strchr(dot1 + 1, '.')) == NULL) {
G_VINUM_DEBUG(0, "proposed sd name '%s' is not a valid sd name",
newname);
err = -1;
goto fail;
return (GV_ERR_INVNAME);
}
strsep(&newp, ".");
if (newp == NULL || *newp == '\0') {
gctl_error(req, "proposed sd name '%s' is not a valid sd name",
newname);
err = -1;
goto fail;
}
if (strcmp(newp, oldp)) {
gctl_error(req, "current and proposed sd numbers (%s, %s) do "
"not match", oldp, newp);
err = -1;
goto fail;
}
strncpy(s->name, newname, GV_MAXSDNAME);
/* XXX: can we rename providers here? */
fail:
g_free(new);
g_free(old);
return (err);
strlcpy(s->name, newname, sizeof(s->name));
return (0);
}
static int
gv_rename_vol(struct gv_softc *sc, struct gctl_req *req, struct gv_volume *v, char *newname, int flags)
int
gv_rename_vol(struct gv_softc *sc, struct gv_volume *v, char *newname,
int flags)
{
struct g_provider *pp;
struct gv_plex *p;
char *new, *old, *oldp;
char newplex[GV_MAXPLEXNAME], *ptr;
int err;
g_topology_assert();
KASSERT(v != NULL, ("gv_rename_vol: NULL v"));
pp = v->provider;
KASSERT(pp != NULL, ("gv_rename_vol: NULL pp"));
if (gv_object_type(sc, newname) != -1) {
gctl_error(req, "volume name %s already in use", newname);
return (-1);
if (gv_object_type(sc, newname) != GV_ERR_NOTFOUND) {
G_VINUM_DEBUG(1, "volume name %s already in use", newname);
return (GV_ERR_NAMETAKEN);
}
/* Rename the volume. */
strncpy(v->name, newname, GV_MAXVOLNAME);
strlcpy(v->name, newname, sizeof(v->name));
/* Fix up references and potentially rename plexes. */
LIST_FOREACH(p, &v->plexes, in_volume) {
strncpy(p->volume, v->name, GV_MAXVOLNAME);
strlcpy(p->volume, v->name, sizeof(p->volume));
if (flags && GV_FLAG_R) {
new = g_malloc(GV_MAXPLEXNAME, M_WAITOK | M_ZERO);
old = g_malloc(GV_MAXPLEXNAME, M_WAITOK | M_ZERO);
oldp = old;
strncpy(old, p->name, GV_MAXPLEXNAME);
/*
* XXX: can we be sure that the current plex name has
* the format 'foo.pX'?
* Look for the last dot in the string, and assume that
* the old value was ok.
*/
strsep(&oldp, ".");
snprintf(new, GV_MAXPLEXNAME, "%s.%s", v->name, oldp);
err = gv_rename_plex(sc, req, p, new, flags);
g_free(new);
g_free(old);
ptr = strrchr(p->name, '.');
ptr++;
snprintf(newplex, sizeof(newplex), "%s.%s", v->name, ptr);
err = gv_rename_plex(sc, p, newplex, flags);
if (err)
return (err);
}

351
sys/geom/vinum/geom_vinum_rm.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -30,20 +30,11 @@ __FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/libkern.h>
#include <sys/kernel.h>
#include <sys/malloc.h>
#include <geom/geom.h>
#include <geom/vinum/geom_vinum_var.h>
#include <geom/vinum/geom_vinum.h>
#include <geom/vinum/geom_vinum_share.h>
static int gv_rm_drive(struct gv_softc *, struct gctl_req *,
struct gv_drive *, int);
static int gv_rm_plex(struct gv_softc *, struct gctl_req *,
struct gv_plex *, int);
static int gv_rm_vol(struct gv_softc *, struct gctl_req *,
struct gv_volume *, int);
/* General 'remove' routine. */
void
@ -56,7 +47,7 @@ gv_remove(struct g_geom *gp, struct gctl_req *req)
struct gv_drive *d;
int *argc, *flags;
char *argv, buf[20];
int i, type, err;
int i, type;
argc = gctl_get_paraml(req, "argc", sizeof(*argc));
@ -73,6 +64,8 @@ gv_remove(struct g_geom *gp, struct gctl_req *req)
sc = gp->softc;
/* XXX config locking */
for (i = 0; i < *argc; i++) {
snprintf(buf, sizeof(buf), "argv%d", i);
argv = gctl_get_param(req, buf, NULL);
@ -82,184 +75,173 @@ gv_remove(struct g_geom *gp, struct gctl_req *req)
switch (type) {
case GV_TYPE_VOL:
v = gv_find_vol(sc, argv);
if (v == NULL) {
gctl_error(req, "unknown volume '%s'", argv);
/*
* If this volume has plexes, we want a recursive
* removal.
*/
if (!LIST_EMPTY(&v->plexes) && !(*flags & GV_FLAG_R)) {
gctl_error(req, "volume '%s' has attached "
"plexes - need recursive removal", v->name);
return;
}
err = gv_rm_vol(sc, req, v, *flags);
if (err)
return;
gv_post_event(sc, GV_EVENT_RM_VOLUME, v, NULL, 0, 0);
break;
case GV_TYPE_PLEX:
p = gv_find_plex(sc, argv);
if (p == NULL) {
gctl_error(req, "unknown plex '%s'", argv);
/*
* If this plex has subdisks, we want a recursive
* removal.
*/
if (!LIST_EMPTY(&p->subdisks) &&
!(*flags & GV_FLAG_R)) {
gctl_error(req, "plex '%s' has attached "
"subdisks - need recursive removal",
p->name);
return;
}
err = gv_rm_plex(sc, req, p, *flags);
if (err)
/* Don't allow removal of the only plex of a volume. */
if (p->vol_sc != NULL && p->vol_sc->plexcount == 1) {
gctl_error(req, "plex '%s' is still attached "
"to volume '%s'", p->name, p->volume);
return;
}
gv_post_event(sc, GV_EVENT_RM_PLEX, p, NULL, 0, 0);
break;
case GV_TYPE_SD:
s = gv_find_sd(sc, argv);
if (s == NULL) {
gctl_error(req, "unknown subdisk '%s'", argv);
/* Don't allow removal if attached to a plex. */
if (s->plex_sc != NULL) {
gctl_error(req, "subdisk '%s' is still attached"
" to plex '%s'", s->name, s->plex_sc->name);
return;
}
err = gv_rm_sd(sc, req, s, *flags);
if (err)
return;
gv_post_event(sc, GV_EVENT_RM_SD, s, NULL, 0, 0);
break;
case GV_TYPE_DRIVE:
d = gv_find_drive(sc, argv);
if (d == NULL) {
gctl_error(req, "unknown drive '%s'", argv);
/* We don't allow to remove open drives. */
if (gv_consumer_is_open(d->consumer) &&
!(*flags & GV_FLAG_F)) {
gctl_error(req, "drive '%s' is open", d->name);
return;
}
err = gv_rm_drive(sc, req, d, *flags);
if (err)
/* A drive with subdisks needs a recursive removal. */
/* if (!LIST_EMPTY(&d->subdisks) &&
!(*flags & GV_FLAG_R)) {
gctl_error(req, "drive '%s' still has subdisks"
" - need recursive removal", d->name);
return;
}*/
gv_post_event(sc, GV_EVENT_RM_DRIVE, d, NULL, *flags,
0);
break;
default:
gctl_error(req, "unknown object '%s'", argv);
return;
}
}
gv_save_config_all(sc);
gv_post_event(sc, GV_EVENT_SAVE_CONFIG, sc, NULL, 0, 0);
}
/* Resets configuration */
int
gv_resetconfig(struct g_geom *gp, struct gctl_req *req)
gv_resetconfig(struct gv_softc *sc)
{
struct gv_softc *sc;
struct gv_drive *d, *d2;
struct gv_volume *v, *v2;
struct gv_plex *p, *p2;
struct gv_sd *s, *s2;
int flags;
d = NULL;
d2 = NULL;
p = NULL;
p2 = NULL;
s = NULL;
s2 = NULL;
flags = GV_FLAG_R;
sc = gp->softc;
/* First loop through to make sure no volumes are up */
LIST_FOREACH_SAFE(v, &sc->volumes, volume, v2) {
if (gv_is_open(v->geom)) {
gctl_error(req, "volume '%s' is busy", v->name);
return (-1);
/* First make sure nothing is open. */
LIST_FOREACH_SAFE(d, &sc->drives, drive, d2) {
if (gv_consumer_is_open(d->consumer)) {
return (GV_ERR_ISBUSY);
}
}
/* Then if not, we remove everything. */
LIST_FOREACH_SAFE(v, &sc->volumes, volume, v2)
gv_rm_vol(sc, req, v, flags);
LIST_FOREACH_SAFE(p, &sc->plexes, plex, p2)
gv_rm_plex(sc, req, p, flags);
LIST_FOREACH_SAFE(s, &sc->subdisks, sd, s2)
gv_rm_sd(sc, req, s, flags);
gv_rm_sd(sc, s);
LIST_FOREACH_SAFE(d, &sc->drives, drive, d2)
gv_rm_drive(sc, req, d, flags);
gv_save_config_all(sc);
gv_rm_drive(sc, d, 0);
LIST_FOREACH_SAFE(p, &sc->plexes, plex, p2)
gv_rm_plex(sc, p);
LIST_FOREACH_SAFE(v, &sc->volumes, volume, v2)
gv_rm_vol(sc, v);
gv_post_event(sc, GV_EVENT_SAVE_CONFIG, sc, NULL, 0, 0);
return (0);
}
/* Remove a volume. */
static int
gv_rm_vol(struct gv_softc *sc, struct gctl_req *req, struct gv_volume *v, int flags)
void
gv_rm_vol(struct gv_softc *sc, struct gv_volume *v)
{
struct g_geom *gp;
struct g_provider *pp;
struct gv_plex *p, *p2;
int err;
g_topology_assert();
KASSERT(v != NULL, ("gv_rm_vol: NULL v"));
/* If this volume has plexes, we want a recursive removal. */
if (!LIST_EMPTY(&v->plexes) && !(flags & GV_FLAG_R)) {
gctl_error(req, "volume '%s' has attached plexes", v->name);
return (-1);
}
gp = v->geom;
pp = v->provider;
KASSERT(pp != NULL, ("gv_rm_vol: NULL pp"));
/* Check if any of our consumers is open. */
if (gp != NULL && gv_is_open(gp)) {
gctl_error(req, "volume '%s' is busy", v->name);
return (-1);
if (gv_provider_is_open(pp)) {
G_VINUM_DEBUG(0, "Unable to remove %s: volume still in use",
v->name);
return;
}
/* Remove the plexes our volume has. */
LIST_FOREACH_SAFE(p, &v->plexes, in_volume, p2) {
v->plexcount--;
LIST_REMOVE(p, in_volume);
p->vol_sc = NULL;
LIST_FOREACH_SAFE(p, &v->plexes, in_volume, p2)
gv_rm_plex(sc, p);
err = gv_rm_plex(sc, req, p, flags);
if (err)
return (err);
}
/* Clean up and let our geom fade away. */
/* Clean up. */
LIST_REMOVE(v, volume);
gv_kill_vol_thread(v);
g_free(v);
if (gp != NULL) {
gp->softc = NULL;
g_wither_geom(gp, ENXIO);
}
return (0);
/* Get rid of the volume's provider. */
if (pp != NULL) {
g_topology_lock();
pp->flags |= G_PF_WITHER;
g_orphan_provider(pp, ENXIO);
g_topology_unlock();
}
}
/* Remove a plex. */
static int
gv_rm_plex(struct gv_softc *sc, struct gctl_req *req, struct gv_plex *p, int flags)
void
gv_rm_plex(struct gv_softc *sc, struct gv_plex *p)
{
struct g_geom *gp;
struct gv_volume *v;
struct gv_sd *s, *s2;
int err;
g_topology_assert();
KASSERT(p != NULL, ("gv_rm_plex: NULL p"));
/* If this plex has subdisks, we want a recursive removal. */
if (!LIST_EMPTY(&p->subdisks) && !(flags & GV_FLAG_R)) {
gctl_error(req, "plex '%s' has attached subdisks", p->name);
return (-1);
}
if (p->vol_sc != NULL && p->vol_sc->plexcount == 1) {
gctl_error(req, "plex '%s' is still attached to volume '%s'",
p->name, p->volume);
return (-1);
}
gp = p->geom;
v = p->vol_sc;
/* Check if any of our consumers is open. */
if (gp != NULL && gv_is_open(gp)) {
gctl_error(req, "plex '%s' is busy", p->name);
return (-1);
if (v != NULL && gv_provider_is_open(v->provider) && v->plexcount < 2) {
G_VINUM_DEBUG(0, "Unable to remove %s: volume still in use",
p->name);
return;
}
/* Remove the subdisks our plex has. */
LIST_FOREACH_SAFE(s, &p->subdisks, in_plex, s2) {
#if 0
LIST_REMOVE(s, in_plex);
s->plex_sc = NULL;
#endif
err = gv_rm_sd(sc, req, s, flags);
if (err)
return (err);
}
LIST_FOREACH_SAFE(s, &p->subdisks, in_plex, s2)
gv_rm_sd(sc, s);
v = p->vol_sc;
/* Clean up and let our geom fade away. */
@ -272,35 +254,25 @@ gv_rm_plex(struct gv_softc *sc, struct gctl_req *req, struct gv_plex *p, int fla
gv_update_vol_size(v, gv_vol_size(v));
}
gv_kill_plex_thread(p);
g_free(p);
if (gp != NULL) {
gp->softc = NULL;
g_wither_geom(gp, ENXIO);
}
return (0);
}
/* Remove a subdisk. */
int
gv_rm_sd(struct gv_softc *sc, struct gctl_req *req, struct gv_sd *s, int flags)
void
gv_rm_sd(struct gv_softc *sc, struct gv_sd *s)
{
struct g_provider *pp;
struct gv_plex *p;
struct gv_volume *v;
KASSERT(s != NULL, ("gv_rm_sd: NULL s"));
pp = s->provider;
p = s->plex_sc;
v = NULL;
/* Clean up. */
if (p != NULL) {
LIST_REMOVE(s, in_plex);
s->plex_sc = NULL;
p->sdcount--;
/* Update the plexsize. */
p->size = gv_plex_size(p);
@ -310,77 +282,64 @@ gv_rm_sd(struct gv_softc *sc, struct gctl_req *req, struct gv_sd *s, int flags)
gv_update_vol_size(v, gv_vol_size(v));
}
}
if (s->drive_sc)
if (s->drive_sc && !(s->drive_sc->flags & GV_DRIVE_REFERENCED))
LIST_REMOVE(s, from_drive);
LIST_REMOVE(s, sd);
gv_free_sd(s);
g_free(s);
/* If the subdisk has a provider we need to clean up this one too. */
if (pp != NULL) {
pp->flags |= G_PF_WITHER;
g_orphan_provider(pp, ENXIO);
}
return (0);
}
/* Remove a drive. */
static int
gv_rm_drive(struct gv_softc *sc, struct gctl_req *req, struct gv_drive *d, int flags)
void
gv_rm_drive(struct gv_softc *sc, struct gv_drive *d, int flags)
{
struct g_geom *gp;
struct g_consumer *cp;
struct gv_freelist *fl, *fl2;
struct gv_plex *p;
struct gv_sd *s, *s2;
struct gv_volume *v;
struct gv_drive *d2;
int err;
KASSERT(d != NULL, ("gv_rm_drive: NULL d"));
gp = d->geom;
KASSERT(gp != NULL, ("gv_rm_drive: NULL gp"));
/* We don't allow to remove open drives. */
if (gv_is_open(gp)) {
gctl_error(req, "drive '%s' is open", d->name);
return (-1);
}
cp = d->consumer;
/* A drive with subdisks needs a recursive removal. */
if (!LIST_EMPTY(&d->subdisks) && !(flags & GV_FLAG_R)) {
gctl_error(req, "drive '%s' still has subdisks", d->name);
return (-1);
}
if (cp != NULL) {
g_topology_lock();
err = g_access(cp, 0, 1, 0);
g_topology_unlock();
cp = LIST_FIRST(&gp->consumer);
err = g_access(cp, 0, 1, 0);
if (err) {
G_VINUM_DEBUG(0, "%s: unable to access '%s', errno: "
"%d", __func__, cp->provider->name, err);
return (err);
}
if (err) {
G_VINUM_DEBUG(0, "%s: couldn't access '%s', "
"errno: %d", __func__, cp->provider->name, err);
return;
}
/* Clear the Vinum Magic. */
d->hdr->magic = GV_NOMAGIC;
g_topology_unlock();
err = gv_write_header(cp, d->hdr);
if (err) {
G_VINUM_DEBUG(0, "%s: unable to write header to '%s'"
", errno: %d", __func__, cp->provider->name, err);
d->hdr->magic = GV_MAGIC;
/* Clear the Vinum Magic. */
d->hdr->magic = GV_NOMAGIC;
err = gv_write_header(cp, d->hdr);
if (err)
G_VINUM_DEBUG(0, "gv_rm_drive: couldn't write header to"
" '%s', errno: %d", cp->provider->name, err);
g_topology_lock();
g_access(cp, -cp->acr, -cp->acw, -cp->ace);
g_detach(cp);
g_destroy_consumer(cp);
g_topology_unlock();
}
g_topology_lock();
g_access(cp, 0, -1, 0);
/* Remove all associated subdisks, plexes, volumes. */
if (!LIST_EMPTY(&d->subdisks)) {
LIST_FOREACH_SAFE(s, &d->subdisks, from_drive, s2) {
p = s->plex_sc;
if (p != NULL) {
v = p->vol_sc;
if (v != NULL)
gv_rm_vol(sc, req, v, flags);
if (flags & GV_FLAG_R) {
if (!LIST_EMPTY(&d->subdisks)) {
LIST_FOREACH_SAFE(s, &d->subdisks, from_drive, s2) {
p = s->plex_sc;
if (p != NULL) {
v = p->vol_sc;
if (v != NULL)
gv_rm_vol(sc, v);
}
}
}
}
@ -390,15 +349,33 @@ gv_rm_drive(struct gv_softc *sc, struct gctl_req *req, struct gv_drive *d, int f
LIST_REMOVE(fl, freelist);
g_free(fl);
}
LIST_REMOVE(d, drive);
gv_kill_drive_thread(d);
gp = d->geom;
d->geom = NULL;
g_free(d->hdr);
g_free(d);
gv_save_config_all(sc);
g_wither_geom(gp, ENXIO);
return (err);
/* Put ourself into referenced state if we have subdisks. */
if (d->sdcount > 0) {
d->consumer = NULL;
d->hdr = NULL;
d->flags |= GV_DRIVE_REFERENCED;
snprintf(d->device, sizeof(d->device), "???");
d->size = 0;
d->avail = 0;
d->freelist_entries = 0;
LIST_FOREACH(s, &d->subdisks, from_drive) {
s->flags |= GV_SD_TASTED;
gv_set_sd_state(s, GV_SD_DOWN, GV_SETSTATE_FORCE);
}
/* Shuffle around so we keep gv_is_newer happy. */
LIST_REMOVE(d, drive);
d2 = LIST_FIRST(&sc->drives);
if (d2 == NULL)
LIST_INSERT_HEAD(&sc->drives, d, drive);
else
LIST_INSERT_AFTER(d2, d, drive);
return;
}
g_free(d);
gv_save_config(sc);
}

53
sys/geom/vinum/geom_vinum_share.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* Copyright (c) 1997, 1998, 1999
* Nan Yang Computer Services Limited. All rights reserved.
*
@ -45,10 +45,6 @@ __FBSDID("$FreeBSD$");
#include <sys/param.h>
#ifdef _KERNEL
#include <sys/bio.h>
#include <sys/conf.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/malloc.h>
#include <sys/systm.h>
@ -63,7 +59,6 @@ __FBSDID("$FreeBSD$");
#define g_free free
#endif /* _KERNEL */
#include <sys/lock.h>
#include <sys/mutex.h>
#include <sys/queue.h>
@ -237,6 +232,8 @@ gv_sdstatei(char *buf)
return (GV_SD_UP);
else if (!strcmp(buf, "reviving"))
return (GV_SD_REVIVING);
else if (!strcmp(buf, "initializing"))
return (GV_SD_INITIALIZING);
else if (!strcmp(buf, "stale"))
return (GV_SD_STALE);
else
@ -273,6 +270,8 @@ gv_plexstatei(char *buf)
return (GV_PLEX_INITIALIZING);
else if (!strcmp(buf, "degraded"))
return (GV_PLEX_DEGRADED);
else if (!strcmp(buf, "growable"))
return (GV_PLEX_GROWABLE);
else
return (GV_PLEX_DOWN);
}
@ -288,6 +287,8 @@ gv_plexstate(int state)
return "initializing";
case GV_PLEX_DEGRADED:
return "degraded";
case GV_PLEX_GROWABLE:
return "growable";
case GV_PLEX_UP:
return "up";
default:
@ -378,14 +379,13 @@ gv_new_drive(int max, char *token[])
return (NULL);
#ifdef _KERNEL
d = g_malloc(sizeof(struct gv_drive), M_WAITOK | M_ZERO);
d = g_malloc(sizeof(struct gv_drive), M_NOWAIT);
#else
d = malloc(sizeof(struct gv_drive));
#endif
if (d == NULL)
return (NULL);
bzero(d, sizeof(struct gv_drive));
#endif
errors = 0;
for (j = 1; j < max; j++) {
@ -406,10 +406,10 @@ gv_new_drive(int max, char *token[])
if (strncmp(ptr, "/dev/", 5) == 0)
ptr += 5;
strncpy(d->device, ptr, GV_MAXDRIVENAME);
strlcpy(d->device, ptr, sizeof(d->device));
} else {
/* We assume this is the drive name. */
strncpy(d->name, token[j], GV_MAXDRIVENAME);
strlcpy(d->name, token[j], sizeof(d->name));
}
}
@ -435,14 +435,13 @@ gv_new_volume(int max, char *token[])
return (NULL);
#ifdef _KERNEL
v = g_malloc(sizeof(struct gv_volume), M_WAITOK | M_ZERO);
v = g_malloc(sizeof(struct gv_volume), M_NOWAIT);
#else
v = malloc(sizeof(struct gv_volume));
#endif
if (v == NULL)
return (NULL);
bzero(v, sizeof(struct gv_volume));
#endif
errors = 0;
for (j = 1; j < max; j++) {
@ -455,7 +454,7 @@ gv_new_volume(int max, char *token[])
v->state = gv_volstatei(token[j]);
} else {
/* We assume this is the volume name. */
strncpy(v->name, token[j], GV_MAXVOLNAME);
strlcpy(v->name, token[j], sizeof(v->name));
}
}
@ -481,13 +480,13 @@ gv_new_plex(int max, char *token[])
return (NULL);
#ifdef _KERNEL
p = g_malloc(sizeof(struct gv_plex), M_WAITOK | M_ZERO);
p = g_malloc(sizeof(struct gv_plex), M_NOWAIT);
#else
p = malloc(sizeof(struct gv_plex));
#endif
if (p == NULL)
return (NULL);
bzero(p, sizeof(struct gv_plex));
#endif
errors = 0;
for (j = 1; j < max; j++) {
@ -497,7 +496,7 @@ gv_new_plex(int max, char *token[])
errors++;
break;
}
strncpy(p->name, token[j], GV_MAXPLEXNAME);
strlcpy(p->name, token[j], sizeof(p->name));
} else if (!strcmp(token[j], "org")) {
j++;
if (j >= max) {
@ -532,7 +531,7 @@ gv_new_plex(int max, char *token[])
errors++;
break;
}
strncpy(p->volume, token[j], GV_MAXVOLNAME);
strlcpy(p->volume, token[j], sizeof(p->volume));
} else {
errors++;
break;
@ -555,16 +554,16 @@ gv_new_sd(int max, char *token[])
int j, errors;
if (token[1] == NULL || *token[1] == '\0')
return NULL;
return (NULL);
#ifdef _KERNEL
s = g_malloc(sizeof(struct gv_sd), M_WAITOK | M_ZERO);
s = g_malloc(sizeof(struct gv_sd), M_NOWAIT);
#else
s = malloc(sizeof(struct gv_sd));
if (s == NULL)
return NULL;
bzero(s, sizeof(struct gv_sd));
#endif
if (s == NULL)
return (NULL);
bzero(s, sizeof(struct gv_sd));
s->plex_offset = -1;
s->size = -1;
@ -577,21 +576,21 @@ gv_new_sd(int max, char *token[])
errors++;
break;
}
strncpy(s->name, token[j], GV_MAXSDNAME);
strlcpy(s->name, token[j], sizeof(s->name));
} else if (!strcmp(token[j], "drive")) {
j++;
if (j >= max) {
errors++;
break;
}
strncpy(s->drive, token[j], GV_MAXDRIVENAME);
strlcpy(s->drive, token[j], sizeof(s->drive));
} else if (!strcmp(token[j], "plex")) {
j++;
if (j >= max) {
errors++;
break;
}
strncpy(s->plex, token[j], GV_MAXPLEXNAME);
strlcpy(s->plex, token[j], sizeof(s->plex));
} else if (!strcmp(token[j], "state")) {
j++;
if (j >= max) {

2
sys/geom/vinum/geom_vinum_share.h
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without

206
sys/geom/vinum/geom_vinum_state.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -27,8 +27,6 @@
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/kernel.h>
#include <sys/libkern.h>
#include <sys/malloc.h>
@ -43,8 +41,10 @@ gv_setstate(struct g_geom *gp, struct gctl_req *req)
struct gv_softc *sc;
struct gv_sd *s;
struct gv_drive *d;
struct gv_volume *v;
struct gv_plex *p;
char *obj, *state;
int err, f, *flags, newstate, type;
int f, *flags, type;
f = 0;
obj = gctl_get_param(req, "object", NULL);
@ -72,43 +72,52 @@ gv_setstate(struct g_geom *gp, struct gctl_req *req)
type = gv_object_type(sc, obj);
switch (type) {
case GV_TYPE_VOL:
if (gv_volstatei(state) < 0) {
gctl_error(req, "invalid volume state '%s'", state);
break;
}
v = gv_find_vol(sc, obj);
gv_post_event(sc, GV_EVENT_SET_VOL_STATE, v, NULL,
gv_volstatei(state), f);
break;
case GV_TYPE_PLEX:
gctl_error(req, "volume or plex state cannot be set currently");
if (gv_plexstatei(state) < 0) {
gctl_error(req, "invalid plex state '%s'", state);
break;
}
p = gv_find_plex(sc, obj);
gv_post_event(sc, GV_EVENT_SET_PLEX_STATE, p, NULL,
gv_plexstatei(state), f);
break;
case GV_TYPE_SD:
newstate = gv_sdstatei(state);
if (newstate < 0) {
if (gv_sdstatei(state) < 0) {
gctl_error(req, "invalid subdisk state '%s'", state);
break;
}
s = gv_find_sd(sc, obj);
err = gv_set_sd_state(s, newstate, f);
if (err)
gctl_error(req, "cannot set subdisk state");
gv_post_event(sc, GV_EVENT_SET_SD_STATE, s, NULL,
gv_sdstatei(state), f);
break;
case GV_TYPE_DRIVE:
newstate = gv_drivestatei(state);
if (newstate < 0) {
if (gv_drivestatei(state) < 0) {
gctl_error(req, "invalid drive state '%s'", state);
break;
}
d = gv_find_drive(sc, obj);
err = gv_set_drive_state(d, newstate, f);
if (err)
gctl_error(req, "cannot set drive state");
gv_post_event(sc, GV_EVENT_SET_DRIVE_STATE, d, NULL,
gv_drivestatei(state), f);
break;
default:
gctl_error(req, "unknown object '%s'", obj);
break;
}
return;
}
/* Update drive state; return 0 if the state changes, otherwise -1. */
/* Update drive state; return 0 if the state changes, otherwise error. */
int
gv_set_drive_state(struct gv_drive *d, int newstate, int flags)
{
@ -123,9 +132,9 @@ gv_set_drive_state(struct gv_drive *d, int newstate, int flags)
return (0);
/* We allow to take down an open drive only with force. */
if ((newstate == GV_DRIVE_DOWN) && gv_is_open(d->geom) &&
if ((newstate == GV_DRIVE_DOWN) && gv_consumer_is_open(d->consumer) &&
(!(flags & GV_SETSTATE_FORCE)))
return (-1);
return (GV_ERR_ISBUSY);
d->state = newstate;
@ -136,7 +145,7 @@ gv_set_drive_state(struct gv_drive *d, int newstate, int flags)
/* Save the config back to disk. */
if (flags & GV_SETSTATE_CONFIG)
gv_save_config_all(d->vinumconf);
gv_save_config(d->vinumconf);
return (0);
}
@ -165,14 +174,24 @@ gv_set_sd_state(struct gv_sd *s, int newstate, int flags)
* force.
*/
if ((s->plex_sc != NULL) && !(flags & GV_SETSTATE_FORCE))
return (-1);
return (GV_ERR_ISATTACHED);
break;
case GV_SD_REVIVING:
case GV_SD_INITIALIZING:
/*
* Only do this if we're forced, since it usually is done
* internally, and then we do use the force flag.
*/
if (!flags & GV_SETSTATE_FORCE)
return (GV_ERR_SETSTATE);
break;
case GV_SD_UP:
/* We can't bring the subdisk up if our drive is dead. */
d = s->drive_sc;
if ((d == NULL) || (d->state != GV_DRIVE_UP))
return (-1);
return (GV_ERR_SETSTATE);
/* Check from where we want to be brought up. */
switch (s->state) {
@ -201,12 +220,15 @@ gv_set_sd_state(struct gv_sd *s, int newstate, int flags)
if (p->org != GV_PLEX_RAID5)
break;
else if (flags & GV_SETSTATE_FORCE)
else if (s->flags & GV_SD_CANGOUP) {
s->flags &= ~GV_SD_CANGOUP;
break;
} else if (flags & GV_SETSTATE_FORCE)
break;
else
s->state = GV_SD_STALE;
status = -1;
status = GV_ERR_SETSTATE;
break;
case GV_SD_STALE:
@ -221,21 +243,24 @@ gv_set_sd_state(struct gv_sd *s, int newstate, int flags)
if (p == NULL || flags & GV_SETSTATE_FORCE)
break;
if ((p->org != GV_PLEX_RAID5) &&
(p->vol_sc->plexcount == 1))
if ((p->org != GV_PLEX_RAID5 &&
p->vol_sc->plexcount == 1) ||
(p->flags & GV_PLEX_SYNCING &&
p->synced > 0 &&
p->org == GV_PLEX_RAID5))
break;
else
return (-1);
return (GV_ERR_SETSTATE);
default:
return (-1);
return (GV_ERR_INVSTATE);
}
break;
/* Other state transitions are only possible with force. */
default:
if (!(flags & GV_SETSTATE_FORCE))
return (-1);
return (GV_ERR_SETSTATE);
}
/* We can change the state and do it. */
@ -248,11 +273,102 @@ gv_set_sd_state(struct gv_sd *s, int newstate, int flags)
/* Save the config back to disk. */
if (flags & GV_SETSTATE_CONFIG)
gv_save_config_all(s->vinumconf);
gv_save_config(s->vinumconf);
return (status);
}
int
gv_set_plex_state(struct gv_plex *p, int newstate, int flags)
{
struct gv_volume *v;
int oldstate, plexdown;
KASSERT(p != NULL, ("gv_set_plex_state: NULL p"));
oldstate = p->state;
v = p->vol_sc;
plexdown = 0;
if (newstate == oldstate)
return (0);
switch (newstate) {
case GV_PLEX_UP:
/* Let update_plex handle if the plex can come up */
gv_update_plex_state(p);
if (p->state != GV_PLEX_UP && !(flags & GV_SETSTATE_FORCE))
return (GV_ERR_SETSTATE);
p->state = newstate;
break;
case GV_PLEX_DOWN:
/*
* Set state to GV_PLEX_DOWN only if no-one is using the plex,
* or if the state is forced.
*/
if (v != NULL) {
/* If the only one up, force is needed. */
plexdown = gv_plexdown(v);
if ((v->plexcount == 1 ||
(v->plexcount - plexdown == 1)) &&
((flags & GV_SETSTATE_FORCE) == 0))
return (GV_ERR_SETSTATE);
}
p->state = newstate;
break;
case GV_PLEX_DEGRADED:
/* Only used internally, so we have to be forced. */
if (flags & GV_SETSTATE_FORCE)
p->state = newstate;
break;
}
/* Update our volume if we have one. */
if (v != NULL)
gv_update_vol_state(v);
/* Save config. */
if (flags & GV_SETSTATE_CONFIG)
gv_save_config(p->vinumconf);
return (0);
}
int
gv_set_vol_state(struct gv_volume *v, int newstate, int flags)
{
int oldstate;
KASSERT(v != NULL, ("gv_set_vol_state: NULL v"));
oldstate = v->state;
if (newstate == oldstate)
return (0);
switch (newstate) {
case GV_VOL_UP:
/* Let update handle if the volume can come up. */
gv_update_vol_state(v);
if (v->state != GV_VOL_UP && !(flags & GV_SETSTATE_FORCE))
return (GV_ERR_SETSTATE);
v->state = newstate;
break;
case GV_VOL_DOWN:
/*
* Set state to GV_VOL_DOWN only if no-one is using the volume,
* or if the state should be forced.
*/
if (!gv_provider_is_open(v->provider) &&
!(flags & GV_SETSTATE_FORCE))
return (GV_ERR_ISBUSY);
v->state = newstate;
break;
}
/* Save config */
if (flags & GV_SETSTATE_CONFIG)
gv_save_config(v->vinumconf);
return (0);
}
/* Update the state of a subdisk based on its environment. */
void
@ -268,15 +384,19 @@ gv_update_sd_state(struct gv_sd *s)
oldstate = s->state;
/* If our drive isn't up we cannot be up either. */
if (d->state != GV_DRIVE_UP)
if (d->state != GV_DRIVE_UP) {
s->state = GV_SD_DOWN;
/* If this subdisk was just created, we assume it is good.*/
else if (s->flags & GV_SD_NEWBORN) {
} else if (s->flags & GV_SD_NEWBORN) {
s->state = GV_SD_UP;
s->flags &= ~GV_SD_NEWBORN;
} else if (s->state != GV_SD_UP)
s->state = GV_SD_STALE;
else
} else if (s->state != GV_SD_UP) {
if (s->flags & GV_SD_CANGOUP) {
s->state = GV_SD_UP;
s->flags &= ~GV_SD_CANGOUP;
} else
s->state = GV_SD_STALE;
} else
s->state = GV_SD_UP;
if (s->state != oldstate)
@ -292,6 +412,7 @@ gv_update_sd_state(struct gv_sd *s)
void
gv_update_plex_state(struct gv_plex *p)
{
struct gv_sd *s;
int sdstates;
int oldstate;
@ -316,13 +437,24 @@ gv_update_plex_state(struct gv_plex *p)
/* Some of our subdisks are initializing. */
} else if (sdstates & GV_SD_INITSTATE) {
if (p->flags & GV_PLEX_SYNCING)
if (p->flags & GV_PLEX_SYNCING ||
p->flags & GV_PLEX_REBUILDING)
p->state = GV_PLEX_DEGRADED;
else
p->state = GV_PLEX_DOWN;
} else
p->state = GV_PLEX_DOWN;
if (p->state == GV_PLEX_UP) {
LIST_FOREACH(s, &p->subdisks, in_plex) {
if (s->flags & GV_SD_GROW) {
p->state = GV_PLEX_GROWABLE;
break;
}
}
}
if (p->state != oldstate)
G_VINUM_DEBUG(1, "plex %s state change: %s -> %s", p->name,
gv_plexstate(oldstate), gv_plexstate(p->state));

923
sys/geom/vinum/geom_vinum_subr.c
View File

File diff suppressed because it is too large Load Diff

143
sys/geom/vinum/geom_vinum_var.h
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2004, 2007 Lukas Ertl
* Copyright (c) 1997, 1998, 1999
* Nan Yang Computer Services Limited. All rights reserved.
*
@ -112,11 +112,29 @@
#define GV_BIO_MALLOC 0x02
#define GV_BIO_ONHOLD 0x04
#define GV_BIO_SYNCREQ 0x08
#define GV_BIO_SUCCEED 0x10
#define GV_BIO_INIT 0x10
#define GV_BIO_REBUILD 0x20
#define GV_BIO_CHECK 0x40
#define GV_BIO_PARITY 0x80
#define GV_BIO_RETRY 0x100
#define GV_BIO_INTERNAL \
(GV_BIO_SYNCREQ | GV_BIO_INIT | GV_BIO_REBUILD |GV_BIO_CHECK)
/* Error codes to be used within gvinum. */
#define GV_ERR_SETSTATE (-1) /* Error setting state. */
#define GV_ERR_BADSIZE (-2) /* Object has wrong size. */
#define GV_ERR_INVTYPE (-3) /* Invalid object type. */
#define GV_ERR_CREATE (-4) /* Error creating gvinum object. */
#define GV_ERR_ISBUSY (-5) /* Object is busy. */
#define GV_ERR_ISATTACHED (-6) /* Object is attached to another. */
#define GV_ERR_INVFLAG (-7) /* Invalid flag passed. */
#define GV_ERR_INVSTATE (-8) /* Invalid state. */
#define GV_ERR_NOTFOUND (-9) /* Object not found. */
#define GV_ERR_NAMETAKEN (-10) /* Object name is taken. */
#define GV_ERR_NOSPACE (-11) /* No space left on drive/subdisk. */
#define GV_ERR_BADOFFSET (-12) /* Invalid offset specified. */
#define GV_ERR_INVNAME (-13) /* Invalid object name. */
#define GV_ERR_PLEXORG (-14) /* Invalid plex organization. */
/*
* hostname is 256 bytes long, but we don't need to shlep multiple copies in
@ -162,16 +180,65 @@ struct gv_bioq {
TAILQ_ENTRY(gv_bioq) queue;
};
#define GV_EVENT_DRIVE_TASTED 1
#define GV_EVENT_DRIVE_LOST 2
#define GV_EVENT_THREAD_EXIT 3
#define GV_EVENT_CREATE_DRIVE 4
#define GV_EVENT_CREATE_VOLUME 5
#define GV_EVENT_CREATE_PLEX 6
#define GV_EVENT_CREATE_SD 7
#define GV_EVENT_SAVE_CONFIG 8
#define GV_EVENT_RM_VOLUME 9
#define GV_EVENT_RM_PLEX 10
#define GV_EVENT_RM_SD 11
#define GV_EVENT_RM_DRIVE 12
#define GV_EVENT_SET_SD_STATE 13
#define GV_EVENT_SET_DRIVE_STATE 14
#define GV_EVENT_SET_VOL_STATE 15
#define GV_EVENT_SET_PLEX_STATE 16
#define GV_EVENT_RESET_CONFIG 17
#define GV_EVENT_PARITY_REBUILD 18
#define GV_EVENT_PARITY_CHECK 19
#define GV_EVENT_START_PLEX 20
#define GV_EVENT_START_VOLUME 21
#define GV_EVENT_ATTACH_PLEX 22
#define GV_EVENT_ATTACH_SD 23
#define GV_EVENT_DETACH_PLEX 24
#define GV_EVENT_DETACH_SD 25
#define GV_EVENT_RENAME_VOL 26
#define GV_EVENT_RENAME_PLEX 27
#define GV_EVENT_RENAME_SD 28
#define GV_EVENT_RENAME_DRIVE 29
#define GV_EVENT_MOVE_SD 30
#define GV_EVENT_SETUP_OBJECTS 31
#ifdef _KERNEL
struct gv_event {
int type;
void *arg1;
void *arg2;
intmax_t arg3;
intmax_t arg4;
TAILQ_ENTRY(gv_event) events;
};
#endif
/* This struct contains the main vinum config. */
struct gv_softc {
/*struct mtx config_mtx; XXX not yet */
/* Linked lists of all objects in our setup. */
LIST_HEAD(,gv_drive) drives; /* All drives. */
LIST_HEAD(,gv_plex) plexes; /* All plexes. */
LIST_HEAD(,gv_sd) subdisks; /* All subdisks. */
LIST_HEAD(,gv_volume) volumes; /* All volumes. */
TAILQ_HEAD(,gv_event) equeue; /* Event queue. */
struct mtx queue_mtx; /* Queue lock. */
struct mtx config_mtx; /* Configuration lock. */
#ifdef _KERNEL
struct bio_queue_head *bqueue; /* BIO queue. */
#else
char *padding;
#endif
struct g_geom *geom; /* Pointer to our VINUM geom. */
};
@ -188,26 +255,19 @@ struct gv_drive {
int sdcount; /* Number of subdisks. */
int flags;
#define GV_DRIVE_THREAD_ACTIVE 0x01 /* Drive has an active worker thread. */
#define GV_DRIVE_THREAD_DIE 0x02 /* Signal the worker thread to die. */
#define GV_DRIVE_THREAD_DEAD 0x04 /* The worker thread has died. */
#define GV_DRIVE_NEWBORN 0x08 /* The drive was just created. */
#define GV_DRIVE_REFERENCED 0x01 /* The drive isn't really existing,
but was referenced by a subdisk
during taste. */
struct gv_hdr *hdr; /* The drive header. */
struct gv_hdr *hdr; /* The drive header. */
struct g_consumer *consumer; /* Consumer attached to this drive. */
int freelist_entries; /* Count of freelist entries. */
LIST_HEAD(,gv_freelist) freelist; /* List of freelist entries. */
LIST_HEAD(,gv_sd) subdisks; /* Subdisks on this drive. */
LIST_ENTRY(gv_drive) drive; /* Entry in the vinum config. */
#ifdef _KERNEL
struct bio_queue_head *bqueue; /* BIO queue of this drive. */
#else
char *padding;
#endif
struct mtx bqueue_mtx; /* Mtx. to protect the queue. */
struct g_geom *geom; /* The geom of this drive. */
struct gv_softc *vinumconf; /* Pointer to the vinum conf. */
};
@ -230,8 +290,10 @@ struct gv_sd {
int init_error; /* Flag error on initialization. */
int flags;
#define GV_SD_NEWBORN 0x01 /* Subdisk was just created. */
#define GV_SD_INITCANCEL 0x02 /* Cancel initialization process. */
#define GV_SD_NEWBORN 0x01 /* Subdisk is created by user. */
#define GV_SD_TASTED 0x02 /* Subdisk is created during taste. */
#define GV_SD_CANGOUP 0x04 /* Subdisk can go up immediately. */
#define GV_SD_GROW 0x08 /* Subdisk is added to striped plex. */
char drive[GV_MAXDRIVENAME]; /* Name of underlying drive. */
char plex[GV_MAXPLEXNAME]; /* Name of associated plex. */
@ -239,9 +301,6 @@ struct gv_sd {
struct gv_drive *drive_sc; /* Pointer to underlying drive. */
struct gv_plex *plex_sc; /* Pointer to associated plex. */
struct g_provider *provider; /* The provider this sd represents. */
struct g_consumer *consumer; /* Consumer attached to our provider. */
LIST_ENTRY(gv_sd) from_drive; /* Subdisk list of underlying drive. */
LIST_ENTRY(gv_sd) in_plex; /* Subdisk list of associated plex. */
LIST_ENTRY(gv_sd) sd; /* Entry in the vinum config. */
@ -257,7 +316,8 @@ struct gv_plex {
#define GV_PLEX_DOWN 0
#define GV_PLEX_INITIALIZING 1
#define GV_PLEX_DEGRADED 2
#define GV_PLEX_UP 3
#define GV_PLEX_GROWABLE 3
#define GV_PLEX_UP 4
int org; /* The plex organisation. */
#define GV_PLEX_DISORG 0
@ -270,6 +330,7 @@ struct gv_plex {
char volume[GV_MAXVOLNAME]; /* Name of associated volume. */
struct gv_volume *vol_sc; /* Pointer to associated volume. */
int sddetached; /* Number of detached subdisks. */
int sdcount; /* Number of subdisks in this plex. */
int sddown; /* Number of subdisks that are down. */
int flags;
@ -279,26 +340,25 @@ struct gv_plex {
#define GV_PLEX_THREAD_DIE 0x08 /* Signal the RAID5 thread to die. */
#define GV_PLEX_THREAD_DEAD 0x10 /* The RAID5 thread has died. */
#define GV_PLEX_NEWBORN 0x20 /* The plex was just created. */
#define GV_PLEX_REBUILDING 0x40 /* The plex is rebuilding. */
#define GV_PLEX_GROWING 0x80 /* The plex is growing. */
off_t synced; /* Count of synced bytes. */
struct mtx bqueue_mtx; /* Lock for the BIO queue. */
#ifdef _KERNEL
struct bio_queue_head *bqueue; /* BIO queue. */
struct bio_queue_head *wqueue; /* Waiting BIO queue. */
#else
char *bpad, *wpad;
#endif
TAILQ_HEAD(,gv_raid5_packet) packets; /* RAID5 sub-requests. */
LIST_HEAD(,gv_sd) subdisks; /* List of attached subdisks. */
LIST_ENTRY(gv_plex) in_volume; /* Plex list of associated volume. */
LIST_ENTRY(gv_plex) plex; /* Entry in the vinum config. */
struct g_provider *provider; /* The provider this plex represents. */
struct g_consumer *consumer; /* Consumer attached to our provider. */
#ifdef _KERNEL
struct bio_queue_head *bqueue; /* BIO queue. */
struct bio_queue_head *wqueue; /* Waiting BIO queue. */
struct bio_queue_head *rqueue; /* Rebuild waiting BIO queue. */
#else
char *bpad, *wpad, *rpad; /* Padding for userland. */
#endif
struct g_geom *geom; /* The geom of this plex. */
struct gv_softc *vinumconf; /* Pointer to the vinum config. */
};
@ -315,19 +375,20 @@ struct gv_volume {
#define GV_VOL_THREAD_ACTIVE 0x01 /* Volume has an active thread. */
#define GV_VOL_THREAD_DIE 0x02 /* Signal the thread to die. */
#define GV_VOL_THREAD_DEAD 0x04 /* The thread has died. */
#define GV_VOL_NEWBORN 0x08 /* The volume was just created. */
struct mtx bqueue_mtx; /* Lock for the BIO queue. */
#ifdef _KERNEL
struct bio_queue_head *bqueue; /* BIO queue. */
LIST_HEAD(,gv_plex) plexes; /* List of attached plexes. */
LIST_ENTRY(gv_volume) volume; /* Entry in vinum config. */
struct g_provider *provider; /* Provider of this volume. */
#ifdef _KERNEL
struct bio_queue_head *wqueue; /* BIO delayed request queue. */
#else
char *padding;
char *wpad; /* Padding for userland. */
#endif
LIST_HEAD(,gv_plex) plexes; /* List of attached plexes. */
LIST_ENTRY(gv_volume) volume; /* Entry in vinum config. */
struct gv_plex *last_read_plex;
struct g_geom *geom; /* The geom of this volume. */
struct gv_softc *vinumconf; /* Pointer to the vinum config. */
};

430
sys/geom/vinum/geom_vinum_volume.c
View File

@ -1,5 +1,5 @@
/*-
* Copyright (c) 2004 Lukas Ertl
* Copyright (c) 2007 Lukas Ertl
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -29,416 +29,136 @@ __FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/bio.h>
#include <sys/conf.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/libkern.h>
#include <sys/lock.h>
#include <sys/malloc.h>
#include <sys/module.h>
#include <sys/mutex.h>
#include <sys/systm.h>
#include <geom/geom.h>
#include <geom/vinum/geom_vinum_var.h>
#include <geom/vinum/geom_vinum.h>
static void gv_vol_completed_request(struct gv_volume *, struct bio *);
static void gv_vol_normal_request(struct gv_volume *, struct bio *);
void
gv_volume_flush(struct gv_volume *v)
{
struct gv_softc *sc;
struct bio *bp;
static void
gv_volume_orphan(struct g_consumer *cp)
KASSERT(v != NULL, ("NULL v"));
sc = v->vinumconf;
KASSERT(sc != NULL, ("NULL sc"));
bp = bioq_takefirst(v->wqueue);
while (bp != NULL) {
gv_volume_start(sc, bp);
bp = bioq_takefirst(v->wqueue);
}
}
void
gv_volume_start(struct gv_softc *sc, struct bio *bp)
{
struct g_geom *gp;
struct gv_volume *v;
int error;
struct gv_plex *p, *lp;
int numwrites;
g_topology_assert();
gp = cp->geom;
g_trace(G_T_TOPOLOGY, "gv_volume_orphan(%s)", gp->name);
if (cp->acr != 0 || cp->acw != 0 || cp->ace != 0)
g_access(cp, -cp->acr, -cp->acw, -cp->ace);
error = cp->provider->error;
if (error == 0)
error = ENXIO;
g_detach(cp);
g_destroy_consumer(cp);
if (!LIST_EMPTY(&gp->consumer))
return;
v = gp->softc;
if (v != NULL) {
gv_kill_vol_thread(v);
v->geom = NULL;
}
gp->softc = NULL;
g_wither_geom(gp, error);
}
/* We end up here after the requests to our plexes are done. */
static void
gv_volume_done(struct bio *bp)
{
struct gv_volume *v;
v = bp->bio_from->geom->softc;
bp->bio_cflags |= GV_BIO_DONE;
mtx_lock(&v->bqueue_mtx);
bioq_insert_tail(v->bqueue, bp);
wakeup(v);
mtx_unlock(&v->bqueue_mtx);
}
static void
gv_volume_start(struct bio *bp)
{
struct gv_volume *v;
switch(bp->bio_cmd) {
case BIO_READ:
case BIO_WRITE:
case BIO_DELETE:
break;
case BIO_GETATTR:
default:
g_io_deliver(bp, EOPNOTSUPP);
return;
}
v = bp->bio_to->geom->softc;
if (v->state != GV_VOL_UP) {
gp = sc->geom;
v = bp->bio_to->private;
if (v == NULL || v->state != GV_VOL_UP) {
g_io_deliver(bp, ENXIO);
return;
}
mtx_lock(&v->bqueue_mtx);
bioq_disksort(v->bqueue, bp);
wakeup(v);
mtx_unlock(&v->bqueue_mtx);
}
static void
gv_vol_worker(void *arg)
{
struct bio *bp;
struct gv_volume *v;
v = arg;
KASSERT(v != NULL, ("NULL v"));
mtx_lock(&v->bqueue_mtx);
for (;;) {
/* We were signaled to exit. */
if (v->flags & GV_VOL_THREAD_DIE)
break;
/* Take the first BIO from our queue. */
bp = bioq_takefirst(v->bqueue);
if (bp == NULL) {
msleep(v, &v->bqueue_mtx, PRIBIO, "-", hz/10);
continue;
}
mtx_unlock(&v->bqueue_mtx);
if (bp->bio_cflags & GV_BIO_DONE)
gv_vol_completed_request(v, bp);
else
gv_vol_normal_request(v, bp);
mtx_lock(&v->bqueue_mtx);
}
mtx_unlock(&v->bqueue_mtx);
v->flags |= GV_VOL_THREAD_DEAD;
wakeup(v);
kproc_exit(ENXIO);
}
static void
gv_vol_completed_request(struct gv_volume *v, struct bio *bp)
{
struct bio *pbp;
struct g_geom *gp;
struct g_consumer *cp, *cp2;
pbp = bp->bio_parent;
if (pbp->bio_error == 0)
pbp->bio_error = bp->bio_error;
switch (pbp->bio_cmd) {
case BIO_READ:
if (bp->bio_error == 0)
break;
if (pbp->bio_cflags & GV_BIO_RETRY)
break;
/* Check if we have another plex left. */
cp = bp->bio_from;
gp = cp->geom;
cp2 = LIST_NEXT(cp, consumer);
if (cp2 == NULL)
break;
if (LIST_NEXT(cp2, consumer) == NULL)
pbp->bio_cflags |= GV_BIO_RETRY;
g_destroy_bio(bp);
pbp->bio_children--;
mtx_lock(&v->bqueue_mtx);
bioq_disksort(v->bqueue, pbp);
mtx_unlock(&v->bqueue_mtx);
return;
case BIO_WRITE:
case BIO_DELETE:
/* Remember if this write request succeeded. */
if (bp->bio_error == 0)
pbp->bio_cflags |= GV_BIO_SUCCEED;
break;
}
/* When the original request is finished, we deliver it. */
pbp->bio_inbed++;
if (pbp->bio_inbed == pbp->bio_children) {
if (pbp->bio_cflags & GV_BIO_SUCCEED)
pbp->bio_error = 0;
pbp->bio_completed = bp->bio_length;
g_io_deliver(pbp, pbp->bio_error);
}
g_destroy_bio(bp);
}
static void
gv_vol_normal_request(struct gv_volume *v, struct bio *bp)
{
struct bio_queue_head queue;
struct g_geom *gp;
struct gv_plex *p, *lp;
struct bio *cbp;
gp = v->geom;
switch (bp->bio_cmd) {
case BIO_READ:
cbp = g_clone_bio(bp);
if (cbp == NULL) {
g_io_deliver(bp, ENOMEM);
return;
}
cbp->bio_done = gv_volume_done;
/*
* Try to find a good plex where we can send the request to.
* The plex either has to be up, or it's a degraded RAID5 plex.
* Try to find a good plex where we can send the request to,
* round-robin-style. The plex either has to be up, or it's a
* degraded RAID5 plex. Check if we have delayed requests. Put
* this request on the delayed queue if so. This makes sure that
* we don't read old values.
*/
if (bioq_first(v->wqueue) != NULL) {
bioq_insert_tail(v->wqueue, bp);
break;
}
lp = v->last_read_plex;
if (lp == NULL)
lp = LIST_FIRST(&v->plexes);
p = LIST_NEXT(lp, in_volume);
if (p == NULL)
p = LIST_FIRST(&v->plexes);
do {
if (p == NULL)
p = LIST_FIRST(&v->plexes);
if (p == NULL) {
p = lp;
break;
}
if ((p->state > GV_PLEX_DEGRADED) ||
(p->state >= GV_PLEX_DEGRADED &&
p->org == GV_PLEX_RAID5))
break;
p = LIST_NEXT(p, in_volume);
if (p == NULL)
p = LIST_FIRST(&v->plexes);
} while (p != lp);
if (p == NULL ||
if ((p == NULL) ||
(p->org == GV_PLEX_RAID5 && p->state < GV_PLEX_DEGRADED) ||
(p->org != GV_PLEX_RAID5 && p->state <= GV_PLEX_DEGRADED)) {
g_destroy_bio(cbp);
bp->bio_children--;
g_io_deliver(bp, ENXIO);
return;
}
g_io_request(cbp, p->consumer);
v->last_read_plex = p;
/* Hand it down to the plex logic. */
gv_plex_start(p, bp);
break;
case BIO_WRITE:
case BIO_DELETE:
bioq_init(&queue);
/* Delay write-requests if any plex is synchronizing. */
LIST_FOREACH(p, &v->plexes, in_volume) {
if (p->flags & GV_PLEX_SYNCING) {
bioq_insert_tail(v->wqueue, bp);
return;
}
}
numwrites = 0;
/* Give the BIO to each plex of this volume. */
LIST_FOREACH(p, &v->plexes, in_volume) {
if (p->state < GV_PLEX_DEGRADED)
continue;
cbp = g_clone_bio(bp);
if (cbp == NULL) {
for (cbp = bioq_first(&queue); cbp != NULL;
cbp = bioq_first(&queue)) {
bioq_remove(&queue, cbp);
g_destroy_bio(cbp);
}
if (bp->bio_error == 0)
bp->bio_error = ENOMEM;
g_io_deliver(bp, bp->bio_error);
return;
}
bioq_insert_tail(&queue, cbp);
cbp->bio_done = gv_volume_done;
cbp->bio_caller1 = p->consumer;
}
/* Fire off all sub-requests. */
for (cbp = bioq_first(&queue); cbp != NULL;
cbp = bioq_first(&queue)) {
bioq_remove(&queue, cbp);
g_io_request(cbp, cbp->bio_caller1);
gv_plex_start(p, bp);
numwrites++;
}
if (numwrites == 0)
g_io_deliver(bp, ENXIO);
break;
}
}
static int
gv_volume_access(struct g_provider *pp, int dr, int dw, int de)
void
gv_bio_done(struct gv_softc *sc, struct bio *bp)
{
struct g_geom *gp;
struct g_consumer *cp, *cp2;
int error;
gp = pp->geom;
error = ENXIO;
LIST_FOREACH(cp, &gp->consumer, consumer) {
error = g_access(cp, dr, dw, de);
if (error) {
LIST_FOREACH(cp2, &gp->consumer, consumer) {
if (cp == cp2)
break;
g_access(cp2, -dr, -dw, -de);
}
return (error);
}
}
return (error);
}
static struct g_geom *
gv_volume_taste(struct g_class *mp, struct g_provider *pp, int flags __unused)
{
struct g_geom *gp;
struct g_provider *pp2;
struct g_consumer *cp, *ocp;
struct gv_softc *sc;
struct gv_volume *v;
struct gv_plex *p;
int error, first;
struct gv_sd *s;
g_trace(G_T_TOPOLOGY, "gv_volume_taste(%s, %s)", mp->name, pp->name);
g_topology_assert();
s = bp->bio_caller1;
KASSERT(s != NULL, ("gv_bio_done: NULL s"));
p = s->plex_sc;
KASSERT(p != NULL, ("gv_bio_done: NULL p"));
v = p->vol_sc;
KASSERT(v != NULL, ("gv_bio_done: NULL v"));
/* First, find the VINUM class and its associated geom. */
gp = find_vinum_geom();
if (gp == NULL)
return (NULL);
sc = gp->softc;
KASSERT(sc != NULL, ("gv_volume_taste: NULL sc"));
gp = pp->geom;
/* We only want to attach to plexes. */
if (strcmp(gp->class->name, "VINUMPLEX"))
return (NULL);
first = 0;
p = gp->softc;
/* Let's see if the volume this plex wants is already configured. */
v = gv_find_vol(sc, p->volume);
if (v == NULL)
return (NULL);
if (v->geom == NULL) {
gp = g_new_geomf(mp, "%s", p->volume);
gp->start = gv_volume_start;
gp->orphan = gv_volume_orphan;
gp->access = gv_volume_access;
gp->softc = v;
first++;
} else
gp = v->geom;
/* Create bio queue, queue mutex, and worker thread, if necessary. */
if (v->bqueue == NULL) {
v->bqueue = g_malloc(sizeof(struct bio_queue_head),
M_WAITOK | M_ZERO);
bioq_init(v->bqueue);
switch (p->org) {
case GV_PLEX_CONCAT:
case GV_PLEX_STRIPED:
gv_plex_normal_done(p, bp);
break;
case GV_PLEX_RAID5:
gv_plex_raid5_done(p, bp);
break;
}
if (mtx_initialized(&v->bqueue_mtx) == 0)
mtx_init(&v->bqueue_mtx, "gv_plex", NULL, MTX_DEF);
if (!(v->flags & GV_VOL_THREAD_ACTIVE)) {
kproc_create(gv_vol_worker, v, NULL, 0, 0, "gv_v %s",
v->name);
v->flags |= GV_VOL_THREAD_ACTIVE;
}
/*
* Create a new consumer and attach it to the plex geom. Since this
* volume might already have a plex attached, we need to adjust the
* access counts of the new consumer.
*/
ocp = LIST_FIRST(&gp->consumer);
cp = g_new_consumer(gp);
g_attach(cp, pp);
if ((ocp != NULL) && (ocp->acr > 0 || ocp->acw > 0 || ocp->ace > 0)) {
error = g_access(cp, ocp->acr, ocp->acw, ocp->ace);
if (error) {
G_VINUM_DEBUG(0, "failed g_access %s -> %s; "
"errno %d", v->name, p->name, error);
g_detach(cp);
g_destroy_consumer(cp);
if (first)
g_destroy_geom(gp);
return (NULL);
}
}
p->consumer = cp;
if (p->vol_sc != v) {
p->vol_sc = v;
v->plexcount++;
LIST_INSERT_HEAD(&v->plexes, p, in_volume);
}
/* We need to setup a new VINUMVOLUME geom. */
if (first) {
pp2 = g_new_providerf(gp, "gvinum/%s", v->name);
pp2->mediasize = pp->mediasize;
pp2->sectorsize = pp->sectorsize;
g_error_provider(pp2, 0);
v->size = pp2->mediasize;
v->geom = gp;
return (gp);
}
return (NULL);
}
static int
gv_volume_destroy_geom(struct gctl_req *req, struct g_class *mp,
struct g_geom *gp)
{
struct gv_volume *v;
g_trace(G_T_TOPOLOGY, "gv_volume_destroy_geom: %s", gp->name);
g_topology_assert();
v = gp->softc;
gv_kill_vol_thread(v);
g_wither_geom(gp, ENXIO);
return (0);
}
#define VINUMVOLUME_CLASS_NAME "VINUMVOLUME"
static struct g_class g_vinum_volume_class = {
.name = VINUMVOLUME_CLASS_NAME,
.version = G_VERSION,
.taste = gv_volume_taste,
.destroy_geom = gv_volume_destroy_geom,
};
DECLARE_GEOM_CLASS(g_vinum_volume_class, g_vinum_volume);

4
sys/modules/geom/geom_vinum/Makefile
View File

@ -3,10 +3,10 @@
.PATH: ${.CURDIR}/../../../geom/vinum
KMOD= geom_vinum
SRCS= geom_vinum.c geom_vinum_drive.c geom_vinum_plex.c \
SRCS= geom_vinum.c geom_vinum_create.c geom_vinum_drive.c geom_vinum_plex.c \
geom_vinum_volume.c geom_vinum_subr.c geom_vinum_raid5.c \
geom_vinum_share.c geom_vinum_list.c geom_vinum_rm.c \
geom_vinum_init.c geom_vinum_state.c geom_vinum_rename.c \
geom_vinum_move.c
geom_vinum_move.c geom_vinum_events.c
.include <bsd.kmod.mk>