d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3406a2a083
freebsd-skq/sys/cam/scsi/scsi_enc.c

1034 lines
24 KiB
C
Raw Normal View History

MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
/*-
* Copyright (c) 2000 Matthew Jacob
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions, and the following disclaimer,
* without modification, immediately at the beginning of the file.
* 2. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/conf.h>
#include <sys/errno.h>
#include <sys/fcntl.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/lock.h>
#include <sys/malloc.h>
#include <sys/mutex.h>
#include <sys/queue.h>
#include <sys/sx.h>
#include <sys/systm.h>
#include <sys/sysctl.h>
#include <sys/types.h>
#include <machine/stdarg.h>
#include <cam/cam.h>
#include <cam/cam_ccb.h>
#include <cam/cam_debug.h>
#include <cam/cam_periph.h>
#include <cam/cam_xpt_periph.h>
#include <cam/scsi/scsi_all.h>
#include <cam/scsi/scsi_message.h>
#include <cam/scsi/scsi_enc.h>
#include <cam/scsi/scsi_enc_internal.h>
Fix SES_ENABLE_PASSTHROUGH kernel option, unexpectedly broken during driver overhaul. MFC after: 3 days
2013-09-01 12:18:44 +00:00
#include <opt_ses.h>
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
MALLOC_DEFINE(M_SCSIENC, "SCSI ENC", "SCSI ENC buffers");
/* Enclosure type independent driver */
static d_open_t enc_open;
static d_close_t enc_close;
static d_ioctl_t enc_ioctl;
static periph_init_t enc_init;
static periph_ctor_t enc_ctor;
static periph_oninv_t enc_oninvalidate;
static periph_dtor_t enc_dtor;
static void enc_async(void *, uint32_t, struct cam_path *, void *);
static enctyp enc_type(struct ccb_getdev *);
SYSCTL_NODE(_kern_cam, OID_AUTO, enc, CTLFLAG_RD, 0,
"CAM Enclosure Services driver");
static struct periph_driver encdriver = {
enc_init, "ses",
TAILQ_HEAD_INITIALIZER(encdriver.units), /* generation */ 0
};
PERIPHDRIVER_DECLARE(enc, encdriver);
static struct cdevsw enc_cdevsw = {
.d_version = D_VERSION,
.d_open = enc_open,
.d_close = enc_close,
.d_ioctl = enc_ioctl,
.d_name = "ses",
Work around a race condition in devfs by changing the way closes are handled in most CAM peripheral drivers that are not handled by GEOM's disk class. The usual character driver open and close semantics are that the driver gets N open calls, but only one close, when the last caller closes the device. CAM peripheral drivers expect that behavior to be honored to the letter, and the CAM peripheral driver code (specifically cam_periph_release_locked_busses()) panics if it is done incorrectly. Since devfs has to drop its locks while it calls a driver's close routine, and it does not have a way to delay or prevent open calls while it is calling the close routine, there is a race. The sequence of events, simplified a bit, is: - devfs acquires a lock - devfs checks the reference count, and if it is 1, continues to close. - devfs releases the lock - 2nd process open call on the device happens here - devfs calls the driver's close routine - devfs acquires a lock - devfs decrements the reference count - devfs releases the lock - 2nd process close call on the device happens here At the second close, we get a panic in cam_periph_release_locked_busses(), complaining that peripheral has been released when the reference count is already 0. This is because we have gotten two closes in a row, which should not happen. The fix is to add the D_TRACKCLOSE flag to the driver's cdevsw, so that we get a close() call for each open(). That does happen reliably, so we can make sure that our reference counts are correct. Note that the sa(4) and pt(4) drivers only allow one context through the open routine. So these drivers aren't exposed to the same race condition. scsi_ch.c, scsi_enc.c, scsi_enc_internal.h, scsi_pass.c, scsi_sg.c: For these drivers, change the open() routine to increment the reference count for every open, and just decrement the reference count in the close. Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. scsi_pt.c: Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. MFC after: 3 days
2012-05-27 06:11:09 +00:00
.d_flags = D_TRACKCLOSE,
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
};
static void
enc_init(void)
{
cam_status status;
/*
* Install a global async callback. This callback will
* receive async callbacks like "new device found".
*/
status = xpt_register_async(AC_FOUND_DEVICE, enc_async, NULL, NULL);
if (status != CAM_REQ_CMP) {
printf("enc: Failed to attach master async callback "
"due to status 0x%x!\n", status);
}
}
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
static void
enc_devgonecb(void *arg)
{
struct cam_periph *periph;
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
struct enc_softc *enc;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct mtx *mtx;
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
int i;
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
periph = (struct cam_periph *)arg;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx = cam_periph_mtx(periph);
mtx_lock(mtx);
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
enc = (struct enc_softc *)periph->softc;
/*
* When we get this callback, we will get no more close calls from
* devfs. So if we have any dangling opens, we need to release the
* reference held for that particular context.
*/
for (i = 0; i < enc->open_count; i++)
cam_periph_release_locked(periph);
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
enc->open_count = 0;
/*
* Release the reference held for the device node, it is gone now.
*/
cam_periph_release_locked(periph);
/*
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
* We reference the lock directly here, instead of using
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
* cam_periph_unlock(). The reason is that the final call to
* cam_periph_release_locked() above could result in the periph
* getting freed. If that is the case, dereferencing the periph
* with a cam_periph_unlock() call would cause a page fault.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(mtx);
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
}
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
static void
enc_oninvalidate(struct cam_periph *periph)
{
struct enc_softc *enc;
enc = periph->softc;
enc->enc_flags |= ENC_FLAG_INVALID;
/* If the sub-driver has an invalidate routine, call it */
if (enc->enc_vec.softc_invalidate != NULL)
enc->enc_vec.softc_invalidate(enc);
/*
* Unregister any async callbacks.
*/
xpt_register_async(0, enc_async, periph, periph->path);
/*
* Shutdown our daemon.
*/
enc->enc_flags |= ENC_FLAG_SHUTDOWN;
if (enc->enc_daemon != NULL) {
Remove sleep() from invalidate call in ses driver, waiting for daemon process exit. Instead use CAM's standard reference counting to prevent periph going away until process won't complete. I think that sleep in single CAM SWI thread is not a good idea and may lead to deadlocks if daemon process waits for some command completion. Combined with recent patch avoiding use of CAM SWI for ATA it just causes panics because of sleeps prohibited in interrupt thread context.
2012-05-25 07:57:17 +00:00
/* Signal the ses daemon to terminate. */
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
wakeup(enc->enc_daemon);
}
callout_drain(&enc->status_updater);
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
destroy_dev_sched_cb(enc->enc_dev, enc_devgonecb, periph);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
}
static void
enc_dtor(struct cam_periph *periph)
{
struct enc_softc *enc;
enc = periph->softc;
/* If the sub-driver has a cleanup routine, call it */
if (enc->enc_vec.softc_cleanup != NULL)
enc->enc_vec.softc_cleanup(enc);
if (enc->enc_boot_hold_ch.ich_func != NULL) {
config_intrhook_disestablish(&enc->enc_boot_hold_ch);
enc->enc_boot_hold_ch.ich_func = NULL;
}
ENC_FREE(enc);
}
static void
enc_async(void *callback_arg, uint32_t code, struct cam_path *path, void *arg)
{
struct cam_periph *periph;
periph = (struct cam_periph *)callback_arg;
switch(code) {
case AC_FOUND_DEVICE:
{
struct ccb_getdev *cgd;
cam_status status;
path_id_t path_id;
cgd = (struct ccb_getdev *)arg;
if (arg == NULL) {
break;
}
if (enc_type(cgd) == ENC_NONE) {
/*
* Schedule announcement of the ENC bindings for
* this device if it is managed by a SEP.
*/
path_id = xpt_path_path_id(path);
xpt_lock_buses();
TAILQ_FOREACH(periph, &encdriver.units, unit_links) {
struct enc_softc *softc;
softc = (struct enc_softc *)periph->softc;
if (xpt_path_path_id(periph->path) != path_id
|| softc == NULL
|| (softc->enc_flags & ENC_FLAG_INITIALIZED)
== 0
|| softc->enc_vec.device_found == NULL)
continue;
softc->enc_vec.device_found(softc);
}
xpt_unlock_buses();
return;
}
status = cam_periph_alloc(enc_ctor, enc_oninvalidate,
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
enc_dtor, NULL, "ses", CAM_PERIPH_BIO,
path, enc_async, AC_FOUND_DEVICE, cgd);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
if (status != CAM_REQ_CMP && status != CAM_REQ_INPROG) {
printf("enc_async: Unable to probe new device due to "
"status 0x%x\n", status);
}
break;
}
default:
cam_periph_async(periph, code, path, arg);
break;
}
}
static int
enc_open(struct cdev *dev, int flags, int fmt, struct thread *td)
{
struct cam_periph *periph;
struct enc_softc *softc;
int error = 0;
periph = (struct cam_periph *)dev->si_drv1;
if (periph == NULL) {
return (ENXIO);
}
if (cam_periph_acquire(periph) != CAM_REQ_CMP)
return (ENXIO);
cam_periph_lock(periph);
softc = (struct enc_softc *)periph->softc;
if ((softc->enc_flags & ENC_FLAG_INITIALIZED) == 0) {
error = ENXIO;
goto out;
}
if (softc->enc_flags & ENC_FLAG_INVALID) {
error = ENXIO;
goto out;
}
out:
Work around a race condition in devfs by changing the way closes are handled in most CAM peripheral drivers that are not handled by GEOM's disk class. The usual character driver open and close semantics are that the driver gets N open calls, but only one close, when the last caller closes the device. CAM peripheral drivers expect that behavior to be honored to the letter, and the CAM peripheral driver code (specifically cam_periph_release_locked_busses()) panics if it is done incorrectly. Since devfs has to drop its locks while it calls a driver's close routine, and it does not have a way to delay or prevent open calls while it is calling the close routine, there is a race. The sequence of events, simplified a bit, is: - devfs acquires a lock - devfs checks the reference count, and if it is 1, continues to close. - devfs releases the lock - 2nd process open call on the device happens here - devfs calls the driver's close routine - devfs acquires a lock - devfs decrements the reference count - devfs releases the lock - 2nd process close call on the device happens here At the second close, we get a panic in cam_periph_release_locked_busses(), complaining that peripheral has been released when the reference count is already 0. This is because we have gotten two closes in a row, which should not happen. The fix is to add the D_TRACKCLOSE flag to the driver's cdevsw, so that we get a close() call for each open(). That does happen reliably, so we can make sure that our reference counts are correct. Note that the sa(4) and pt(4) drivers only allow one context through the open routine. So these drivers aren't exposed to the same race condition. scsi_ch.c, scsi_enc.c, scsi_enc_internal.h, scsi_pass.c, scsi_sg.c: For these drivers, change the open() routine to increment the reference count for every open, and just decrement the reference count in the close. Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. scsi_pt.c: Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. MFC after: 3 days
2012-05-27 06:11:09 +00:00
if (error != 0)
cam_periph_release_locked(periph);
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
else
softc->open_count++;
Work around a race condition in devfs by changing the way closes are handled in most CAM peripheral drivers that are not handled by GEOM's disk class. The usual character driver open and close semantics are that the driver gets N open calls, but only one close, when the last caller closes the device. CAM peripheral drivers expect that behavior to be honored to the letter, and the CAM peripheral driver code (specifically cam_periph_release_locked_busses()) panics if it is done incorrectly. Since devfs has to drop its locks while it calls a driver's close routine, and it does not have a way to delay or prevent open calls while it is calling the close routine, there is a race. The sequence of events, simplified a bit, is: - devfs acquires a lock - devfs checks the reference count, and if it is 1, continues to close. - devfs releases the lock - 2nd process open call on the device happens here - devfs calls the driver's close routine - devfs acquires a lock - devfs decrements the reference count - devfs releases the lock - 2nd process close call on the device happens here At the second close, we get a panic in cam_periph_release_locked_busses(), complaining that peripheral has been released when the reference count is already 0. This is because we have gotten two closes in a row, which should not happen. The fix is to add the D_TRACKCLOSE flag to the driver's cdevsw, so that we get a close() call for each open(). That does happen reliably, so we can make sure that our reference counts are correct. Note that the sa(4) and pt(4) drivers only allow one context through the open routine. So these drivers aren't exposed to the same race condition. scsi_ch.c, scsi_enc.c, scsi_enc_internal.h, scsi_pass.c, scsi_sg.c: For these drivers, change the open() routine to increment the reference count for every open, and just decrement the reference count in the close. Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. scsi_pt.c: Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. MFC after: 3 days
2012-05-27 06:11:09 +00:00
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
cam_periph_unlock(periph);
Work around a race condition in devfs by changing the way closes are handled in most CAM peripheral drivers that are not handled by GEOM's disk class. The usual character driver open and close semantics are that the driver gets N open calls, but only one close, when the last caller closes the device. CAM peripheral drivers expect that behavior to be honored to the letter, and the CAM peripheral driver code (specifically cam_periph_release_locked_busses()) panics if it is done incorrectly. Since devfs has to drop its locks while it calls a driver's close routine, and it does not have a way to delay or prevent open calls while it is calling the close routine, there is a race. The sequence of events, simplified a bit, is: - devfs acquires a lock - devfs checks the reference count, and if it is 1, continues to close. - devfs releases the lock - 2nd process open call on the device happens here - devfs calls the driver's close routine - devfs acquires a lock - devfs decrements the reference count - devfs releases the lock - 2nd process close call on the device happens here At the second close, we get a panic in cam_periph_release_locked_busses(), complaining that peripheral has been released when the reference count is already 0. This is because we have gotten two closes in a row, which should not happen. The fix is to add the D_TRACKCLOSE flag to the driver's cdevsw, so that we get a close() call for each open(). That does happen reliably, so we can make sure that our reference counts are correct. Note that the sa(4) and pt(4) drivers only allow one context through the open routine. So these drivers aren't exposed to the same race condition. scsi_ch.c, scsi_enc.c, scsi_enc_internal.h, scsi_pass.c, scsi_sg.c: For these drivers, change the open() routine to increment the reference count for every open, and just decrement the reference count in the close. Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. scsi_pt.c: Call cam_periph_release_locked() in some scenarios to avoid additional lock and unlock calls. MFC after: 3 days
2012-05-27 06:11:09 +00:00
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
return (error);
}
static int
enc_close(struct cdev *dev, int flag, int fmt, struct thread *td)
{
struct cam_periph *periph;
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
struct enc_softc *enc;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct mtx *mtx;
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
periph = (struct cam_periph *)dev->si_drv1;
if (periph == NULL)
return (ENXIO);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx = cam_periph_mtx(periph);
mtx_lock(mtx);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
enc = periph->softc;
enc->open_count--;
cam_periph_release_locked(periph);
/*
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
* We reference the lock directly here, instead of using
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
* cam_periph_unlock(). The reason is that the call to
* cam_periph_release_locked() above could result in the periph
* getting freed. If that is the case, dereferencing the periph
* with a cam_periph_unlock() call would cause a page fault.
*
* cam_periph_release() avoids this problem using the same method,
* but we're manually acquiring and dropping the lock here to
* protect the open count and avoid another lock acquisition and
* release.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(mtx);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
return (0);
}
int
enc_error(union ccb *ccb, uint32_t cflags, uint32_t sflags)
{
struct enc_softc *softc;
struct cam_periph *periph;
periph = xpt_path_periph(ccb->ccb_h.path);
softc = (struct enc_softc *)periph->softc;
return (cam_periph_error(ccb, cflags, sflags, &softc->saved_ccb));
}
static int
enc_ioctl(struct cdev *dev, u_long cmd, caddr_t arg_addr, int flag,
struct thread *td)
{
struct cam_periph *periph;
encioc_enc_status_t tmp;
encioc_string_t sstr;
encioc_elm_status_t elms;
encioc_elm_desc_t elmd;
encioc_elm_devnames_t elmdn;
encioc_element_t *uelm;
enc_softc_t *enc;
enc_cache_t *cache;
void *addr;
int error, i;
if (arg_addr)
addr = *((caddr_t *) arg_addr);
else
addr = NULL;
periph = (struct cam_periph *)dev->si_drv1;
if (periph == NULL)
return (ENXIO);
CAM_DEBUG(periph->path, CAM_DEBUG_TRACE, ("entering encioctl\n"));
cam_periph_lock(periph);
enc = (struct enc_softc *)periph->softc;
cache = &enc->enc_cache;
/*
* Now check to see whether we're initialized or not.
* This actually should never fail as we're not supposed
* to get past enc_open w/o successfully initializing
* things.
*/
if ((enc->enc_flags & ENC_FLAG_INITIALIZED) == 0) {
cam_periph_unlock(periph);
return (ENXIO);
}
cam_periph_unlock(periph);
error = 0;
CAM_DEBUG(periph->path, CAM_DEBUG_TRACE,
("trying to do ioctl %#lx\n", cmd));
/*
* If this command can change the device's state,
* we must have the device open for writing.
*
* For commands that get information about the
* device- we don't need to lock the peripheral
* if we aren't running a command. The periph
* also can't go away while a user process has
* it open.
*/
switch (cmd) {
case ENCIOC_GETNELM:
case ENCIOC_GETELMMAP:
case ENCIOC_GETENCSTAT:
case ENCIOC_GETELMSTAT:
case ENCIOC_GETELMDESC:
case ENCIOC_GETELMDEVNAMES:
break;
default:
if ((flag & FWRITE) == 0) {
return (EBADF);
}
}
/*
* XXX The values read here are only valid for the current
* configuration generation. We need these ioctls
* to also pass in/out a generation number.
*/
sx_slock(&enc->enc_cache_lock);
switch (cmd) {
case ENCIOC_GETNELM:
error = copyout(&cache->nelms, addr, sizeof (cache->nelms));
break;
case ENCIOC_GETELMMAP:
for (uelm = addr, i = 0; i != cache->nelms; i++) {
encioc_element_t kelm;
kelm.elm_idx = i;
kelm.elm_subenc_id = cache->elm_map[i].subenclosure;
kelm.elm_type = cache->elm_map[i].enctype;
error = copyout(&kelm, &uelm[i], sizeof(kelm));
if (error)
break;
}
break;
case ENCIOC_GETENCSTAT:
cam_periph_lock(periph);
error = enc->enc_vec.get_enc_status(enc, 1);
if (error) {
cam_periph_unlock(periph);
break;
}
tmp = cache->enc_status;
cam_periph_unlock(periph);
error = copyout(&tmp, addr, sizeof(tmp));
cache->enc_status = tmp;
break;
case ENCIOC_SETENCSTAT:
error = copyin(addr, &tmp, sizeof(tmp));
if (error)
break;
cam_periph_lock(periph);
error = enc->enc_vec.set_enc_status(enc, tmp, 1);
cam_periph_unlock(periph);
break;
case ENCIOC_GETSTRING:
case ENCIOC_SETSTRING:
if (enc->enc_vec.handle_string == NULL) {
error = EINVAL;
break;
}
error = copyin(addr, &sstr, sizeof(sstr));
if (error)
break;
cam_periph_lock(periph);
error = enc->enc_vec.handle_string(enc, &sstr, cmd);
cam_periph_unlock(periph);
break;
case ENCIOC_GETELMSTAT:
error = copyin(addr, &elms, sizeof(elms));
if (error)
break;
if (elms.elm_idx >= cache->nelms) {
error = EINVAL;
break;
}
cam_periph_lock(periph);
error = enc->enc_vec.get_elm_status(enc, &elms, 1);
cam_periph_unlock(periph);
if (error)
break;
error = copyout(&elms, addr, sizeof(elms));
break;
case ENCIOC_GETELMDESC:
error = copyin(addr, &elmd, sizeof(elmd));
if (error)
break;
if (elmd.elm_idx >= cache->nelms) {
error = EINVAL;
break;
}
if (enc->enc_vec.get_elm_desc != NULL) {
error = enc->enc_vec.get_elm_desc(enc, &elmd);
if (error)
break;
} else
elmd.elm_desc_len = 0;
error = copyout(&elmd, addr, sizeof(elmd));
break;
case ENCIOC_GETELMDEVNAMES:
if (enc->enc_vec.get_elm_devnames == NULL) {
error = EINVAL;
break;
}
error = copyin(addr, &elmdn, sizeof(elmdn));
if (error)
break;
if (elmdn.elm_idx >= cache->nelms) {
error = EINVAL;
break;
}
cam_periph_lock(periph);
error = (*enc->enc_vec.get_elm_devnames)(enc, &elmdn);
cam_periph_unlock(periph);
if (error)
break;
error = copyout(&elmdn, addr, sizeof(elmdn));
break;
case ENCIOC_SETELMSTAT:
error = copyin(addr, &elms, sizeof(elms));
if (error)
break;
if (elms.elm_idx >= cache->nelms) {
error = EINVAL;
break;
}
cam_periph_lock(periph);
error = enc->enc_vec.set_elm_status(enc, &elms, 1);
cam_periph_unlock(periph);
break;
case ENCIOC_INIT:
cam_periph_lock(periph);
error = enc->enc_vec.init_enc(enc);
cam_periph_unlock(periph);
break;
default:
cam_periph_lock(periph);
error = cam_periph_ioctl(periph, cmd, arg_addr, enc_error);
cam_periph_unlock(periph);
break;
}
sx_sunlock(&enc->enc_cache_lock);
return (error);
}
int
enc_runcmd(struct enc_softc *enc, char *cdb, int cdbl, char *dptr, int *dlenp)
{
int error, dlen, tdlen;
ccb_flags ddf;
union ccb *ccb;
CAM_DEBUG(enc->periph->path, CAM_DEBUG_TRACE,
("entering enc_runcmd\n"));
if (dptr) {
if ((dlen = *dlenp) < 0) {
dlen = -dlen;
ddf = CAM_DIR_OUT;
} else {
ddf = CAM_DIR_IN;
}
} else {
dlen = 0;
ddf = CAM_DIR_NONE;
}
if (cdbl > IOCDBLEN) {
cdbl = IOCDBLEN;
}
Remove one more numeric priority constant.
2012-10-27 08:52:33 +00:00
ccb = cam_periph_getccb(enc->periph, CAM_PRIORITY_NORMAL);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
if (enc->enc_type == ENC_SEMB_SES || enc->enc_type == ENC_SEMB_SAFT) {
tdlen = min(dlen, 1020);
tdlen = (tdlen + 3) & ~3;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_fill_ataio(&ccb->ataio, 0, NULL, ddf, 0, dptr, tdlen,
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
30 * 1000);
if (cdb[0] == RECEIVE_DIAGNOSTIC)
ata_28bit_cmd(&ccb->ataio,
ATA_SEP_ATTN, cdb[2], 0x02, tdlen / 4);
else if (cdb[0] == SEND_DIAGNOSTIC)
ata_28bit_cmd(&ccb->ataio,
ATA_SEP_ATTN, dlen > 0 ? dptr[0] : 0,
0x82, tdlen / 4);
else if (cdb[0] == READ_BUFFER)
ata_28bit_cmd(&ccb->ataio,
ATA_SEP_ATTN, cdb[2], 0x00, tdlen / 4);
else
ata_28bit_cmd(&ccb->ataio,
ATA_SEP_ATTN, dlen > 0 ? dptr[0] : 0,
0x80, tdlen / 4);
} else {
tdlen = dlen;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_fill_csio(&ccb->csio, 0, NULL, ddf, MSG_SIMPLE_Q_TAG,
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
dptr, dlen, sizeof (struct scsi_sense_data), cdbl,
60 * 1000);
bcopy(cdb, ccb->csio.cdb_io.cdb_bytes, cdbl);
}
error = cam_periph_runccb(ccb, enc_error, ENC_CFLAGS, ENC_FLAGS, NULL);
if (error) {
if (dptr) {
*dlenp = dlen;
}
} else {
if (dptr) {
if (ccb->ccb_h.func_code == XPT_ATA_IO)
*dlenp = ccb->ataio.resid;
else
*dlenp = ccb->csio.resid;
*dlenp += tdlen - dlen;
}
}
xpt_release_ccb(ccb);
CAM_DEBUG(enc->periph->path, CAM_DEBUG_SUBTRACE,
("exiting enc_runcmd: *dlenp = %d\n", *dlenp));
return (error);
}
void
enc_log(struct enc_softc *enc, const char *fmt, ...)
{
va_list ap;
printf("%s%d: ", enc->periph->periph_name, enc->periph->unit_number);
va_start(ap, fmt);
vprintf(fmt, ap);
va_end(ap);
}
/*
* The code after this point runs on many platforms,
* so forgive the slightly awkward and nonconforming
* appearance.
*/
/*
* Is this a device that supports enclosure services?
*
s/ is is / is /g s/ a a / a /g Approved by: cperciva MFC after: 3 days
2012-09-14 22:00:03 +00:00
* It's a pretty simple ruleset- if it is device type
1. Remove SEN support. I doubt there are any working examples of this hardware still running (close to twenty years now). 2. Quiesece and use ENC_VLOG instead of ENC_LOG for most complaints. That is, they're visible with bootverbose, but otherwise quiesced and not repeatedly spamming messages with constant reminders that hardware in this space is rarely fully compliant. MFC after: 1 month
2012-08-12 17:01:07 +00:00
* 0x0D (13), it's an ENCLOSURE device.
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
*/
#define SAFTE_START 44
#define SAFTE_END 50
#define SAFTE_LEN SAFTE_END-SAFTE_START
static enctyp
enc_type(struct ccb_getdev *cgd)
{
int buflen;
unsigned char *iqd;
if (cgd->protocol == PROTO_SEMB) {
iqd = (unsigned char *)&cgd->ident_data;
if (STRNCMP(iqd + 43, "S-E-S", 5) == 0)
return (ENC_SEMB_SES);
else if (STRNCMP(iqd + 43, "SAF-TE", 6) == 0)
return (ENC_SEMB_SAFT);
return (ENC_NONE);
} else if (cgd->protocol != PROTO_SCSI)
return (ENC_NONE);
iqd = (unsigned char *)&cgd->inq_data;
buflen = min(sizeof(cgd->inq_data),
SID_ADDITIONAL_LENGTH(&cgd->inq_data));
if ((iqd[0] & 0x1f) == T_ENCLOSURE) {
1. Remove SEN support. I doubt there are any working examples of this hardware still running (close to twenty years now). 2. Quiesece and use ENC_VLOG instead of ENC_LOG for most complaints. That is, they're visible with bootverbose, but otherwise quiesced and not repeatedly spamming messages with constant reminders that hardware in this space is rarely fully compliant. MFC after: 1 month
2012-08-12 17:01:07 +00:00
if ((iqd[2] & 0x7) > 2) {
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
return (ENC_SES);
} else {
return (ENC_SES_SCSI2);
}
return (ENC_NONE);
}
Fix SES_ENABLE_PASSTHROUGH kernel option, unexpectedly broken during driver overhaul. MFC after: 3 days
2013-09-01 12:18:44 +00:00
#ifdef SES_ENABLE_PASSTHROUGH
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
if ((iqd[6] & 0x40) && (iqd[2] & 0x7) >= 2) {
/*
* PassThrough Device.
*/
Fix SES_ENABLE_PASSTHROUGH kernel option, unexpectedly broken during driver overhaul. MFC after: 3 days
2013-09-01 12:18:44 +00:00
return (ENC_SES_PASSTHROUGH);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
}
#endif
/*
* The comparison is short for a reason-
* some vendors were chopping it short.
*/
if (buflen < SAFTE_END - 2) {
return (ENC_NONE);
}
if (STRNCMP((char *)&iqd[SAFTE_START], "SAF-TE", SAFTE_LEN - 2) == 0) {
return (ENC_SAFT);
}
return (ENC_NONE);
}
/*================== Enclosure Monitoring/Processing Daemon ==================*/
/**
* \brief Queue an update request for a given action, if needed.
*
* \param enc SES softc to queue the request for.
* \param action Action requested.
*/
void
enc_update_request(enc_softc_t *enc, uint32_t action)
{
if ((enc->pending_actions & (0x1 << action)) == 0) {
enc->pending_actions |= (0x1 << action);
ENC_DLOG(enc, "%s: queing requested action %d\n",
__func__, action);
if (enc->current_action == ENC_UPDATE_NONE)
wakeup(enc->enc_daemon);
} else {
ENC_DLOG(enc, "%s: ignoring requested action %d - "
"Already queued\n", __func__, action);
}
}
/**
* \brief Invoke the handler of the highest priority pending
* state in the SES state machine.
*
* \param enc The SES instance invoking the state machine.
*/
static void
enc_fsm_step(enc_softc_t *enc)
{
union ccb *ccb;
uint8_t *buf;
struct enc_fsm_state *cur_state;
int error;
uint32_t xfer_len;
ENC_DLOG(enc, "%s enter %p\n", __func__, enc);
enc->current_action = ffs(enc->pending_actions) - 1;
enc->pending_actions &= ~(0x1 << enc->current_action);
cur_state = &enc->enc_fsm_states[enc->current_action];
buf = NULL;
if (cur_state->buf_size != 0) {
cam_periph_unlock(enc->periph);
buf = malloc(cur_state->buf_size, M_SCSIENC, M_WAITOK|M_ZERO);
cam_periph_lock(enc->periph);
}
error = 0;
ccb = NULL;
if (cur_state->fill != NULL) {
ccb = cam_periph_getccb(enc->periph, CAM_PRIORITY_NORMAL);
error = cur_state->fill(enc, cur_state, ccb, buf);
if (error != 0)
goto done;
error = cam_periph_runccb(ccb, cur_state->error,
ENC_CFLAGS,
ENC_FLAGS|SF_QUIET_IR, NULL);
}
if (ccb != NULL) {
if (ccb->ccb_h.func_code == XPT_ATA_IO)
xfer_len = ccb->ataio.dxfer_len - ccb->ataio.resid;
else
xfer_len = ccb->csio.dxfer_len - ccb->csio.resid;
} else
xfer_len = 0;
cam_periph_unlock(enc->periph);
cur_state->done(enc, cur_state, ccb, &buf, error, xfer_len);
cam_periph_lock(enc->periph);
done:
ENC_DLOG(enc, "%s exit - result %d\n", __func__, error);
ENC_FREE_AND_NULL(buf);
if (ccb != NULL)
xpt_release_ccb(ccb);
}
/**
* \invariant Called with cam_periph mutex held.
*/
static void
enc_status_updater(void *arg)
{
enc_softc_t *enc;
enc = arg;
if (enc->enc_vec.poll_status != NULL)
enc->enc_vec.poll_status(enc);
}
static void
enc_daemon(void *arg)
{
enc_softc_t *enc;
enc = arg;
cam_periph_lock(enc->periph);
while ((enc->enc_flags & ENC_FLAG_SHUTDOWN) == 0) {
if (enc->pending_actions == 0) {
struct intr_config_hook *hook;
/*
* Reset callout and msleep, or
* issue timed task completion
* status command.
*/
enc->current_action = ENC_UPDATE_NONE;
/*
* We've been through our state machine at least
* once. Allow the transition to userland.
*/
hook = &enc->enc_boot_hold_ch;
if (hook->ich_func != NULL) {
config_intrhook_disestablish(hook);
hook->ich_func = NULL;
}
callout_reset(&enc->status_updater, 60*hz,
enc_status_updater, enc);
cam_periph_sleep(enc->periph, enc->enc_daemon,
PUSER, "idle", 0);
} else {
enc_fsm_step(enc);
}
}
enc->enc_daemon = NULL;
cam_periph_unlock(enc->periph);
Remove sleep() from invalidate call in ses driver, waiting for daemon process exit. Instead use CAM's standard reference counting to prevent periph going away until process won't complete. I think that sleep in single CAM SWI thread is not a good idea and may lead to deadlocks if daemon process waits for some command completion. Combined with recent patch avoiding use of CAM SWI for ATA it just causes panics because of sleeps prohibited in interrupt thread context.
2012-05-25 07:57:17 +00:00
cam_periph_release(enc->periph);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
kproc_exit(0);
}
static int
enc_kproc_init(enc_softc_t *enc)
{
int result;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
callout_init_mtx(&enc->status_updater, cam_periph_mtx(enc->periph), 0);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
Remove sleep() from invalidate call in ses driver, waiting for daemon process exit. Instead use CAM's standard reference counting to prevent periph going away until process won't complete. I think that sleep in single CAM SWI thread is not a good idea and may lead to deadlocks if daemon process waits for some command completion. Combined with recent patch avoiding use of CAM SWI for ATA it just causes panics because of sleeps prohibited in interrupt thread context.
2012-05-25 07:57:17 +00:00
if (cam_periph_acquire(enc->periph) != CAM_REQ_CMP)
return (ENXIO);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
result = kproc_create(enc_daemon, enc, &enc->enc_daemon, /*flags*/0,
/*stackpgs*/0, "enc_daemon%d",
enc->periph->unit_number);
if (result == 0) {
/* Do an initial load of all page data. */
cam_periph_lock(enc->periph);
enc->enc_vec.poll_status(enc);
cam_periph_unlock(enc->periph);
Remove sleep() from invalidate call in ses driver, waiting for daemon process exit. Instead use CAM's standard reference counting to prevent periph going away until process won't complete. I think that sleep in single CAM SWI thread is not a good idea and may lead to deadlocks if daemon process waits for some command completion. Combined with recent patch avoiding use of CAM SWI for ATA it just causes panics because of sleeps prohibited in interrupt thread context.
2012-05-25 07:57:17 +00:00
} else
cam_periph_release(enc->periph);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
return (result);
}
/**
* \brief Interrupt configuration hook callback associated with
* enc_boot_hold_ch.
*
* Since interrupts are always functional at the time of enclosure
* configuration, there is nothing to be done when the callback occurs.
* This hook is only registered to hold up boot processing while initial
* eclosure processing occurs.
*
* \param arg The enclosure softc, but currently unused in this callback.
*/
static void
enc_nop_confighook_cb(void *arg __unused)
{
}
static cam_status
enc_ctor(struct cam_periph *periph, void *arg)
{
cam_status status = CAM_REQ_CMP_ERR;
int err;
enc_softc_t *enc;
struct ccb_getdev *cgd;
char *tname;
cgd = (struct ccb_getdev *)arg;
if (cgd == NULL) {
printf("enc_ctor: no getdev CCB, can't register device\n");
goto out;
}
enc = ENC_MALLOCZ(sizeof(*enc));
if (enc == NULL) {
printf("enc_ctor: Unable to probe new device. "
"Unable to allocate enc\n");
goto out;
}
enc->periph = periph;
enc->current_action = ENC_UPDATE_INVALID;
enc->enc_type = enc_type(cgd);
sx_init(&enc->enc_cache_lock, "enccache");
switch (enc->enc_type) {
case ENC_SES:
case ENC_SES_SCSI2:
case ENC_SES_PASSTHROUGH:
case ENC_SEMB_SES:
err = ses_softc_init(enc);
break;
case ENC_SAFT:
case ENC_SEMB_SAFT:
err = safte_softc_init(enc);
break;
case ENC_NONE:
default:
ENC_FREE(enc);
return (CAM_REQ_CMP_ERR);
}
if (err) {
xpt_print(periph->path, "error %d initializing\n", err);
goto out;
}
/*
* Hold off userland until we have made at least one pass
* through our state machine so that physical path data is
* present.
*/
if (enc->enc_vec.poll_status != NULL) {
enc->enc_boot_hold_ch.ich_func = enc_nop_confighook_cb;
enc->enc_boot_hold_ch.ich_arg = enc;
config_intrhook_establish(&enc->enc_boot_hold_ch);
}
/*
* The softc field is set only once the enc is fully initialized
* so that we can rely on this field to detect partially
* initialized periph objects in the AC_FOUND_DEVICE handler.
*/
periph->softc = enc;
cam_periph_unlock(periph);
if (enc->enc_vec.poll_status != NULL) {
err = enc_kproc_init(enc);
if (err) {
xpt_print(periph->path,
Remove sleep() from invalidate call in ses driver, waiting for daemon process exit. Instead use CAM's standard reference counting to prevent periph going away until process won't complete. I think that sleep in single CAM SWI thread is not a good idea and may lead to deadlocks if daemon process waits for some command completion. Combined with recent patch avoiding use of CAM SWI for ATA it just causes panics because of sleeps prohibited in interrupt thread context.
2012-05-25 07:57:17 +00:00
"error %d starting enc_daemon\n", err);
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
goto out;
}
}
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4) drivers. The bug occurrs when a userland process has the driver instance open and the underlying device goes away. We get the devfs callback that the device node has been destroyed, but not all of the closes necessary to fully decrement the reference count on the CAM peripheral. The reason is that once devfs calls back and says the device has been destroyed, it is moved off to deadfs, and devfs guarantees that there will be no more open or close calls. So the solution is to keep track of how many outstanding open calls there are on the device, and just release that many references when we get the callback from devfs. scsi_pass.c, scsi_enc.c, scsi_enc_internal.h: Add an open count to the softc in these drivers. Increment it on open and decrement it on close. When we get a devfs callback to say that the device node has gone away, decrement the peripheral reference count by the number of still outstanding opens. Make sure we don't access the peripheral with cam_periph_unlock() after what might be the final call to cam_periph_release_locked(). The peripheral might have been freed, and we will be dereferencing freed memory. scsi_ch.c, scsi_sg.c: For the ch(4) and sg(4) drivers, add the same changes described above, and in addition, fix another bug that was previously fixed in the pass(4) and enc(4) drivers. These drivers were calling destroy_dev() from their cleanup routine, but that could cause a deadlock because the cleanup routine could be indirectly called from the driver's close routine. This would cause a deadlock, because the device node is being held open by the active close call, and can't be destroyed. Sponsored by: Spectra Logic Corporation MFC after: 1 week
2012-12-08 04:03:04 +00:00
/*
* Acquire a reference to the periph before we create the devfs
* instance for it. We'll release this reference once the devfs
* instance has been freed.
*/
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
if (cam_periph_acquire(periph) != CAM_REQ_CMP) {
xpt_print(periph->path, "%s: lost periph during "
"registration!\n", __func__);
cam_periph_lock(periph);
return (CAM_REQ_CMP_ERR);
}
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
enc->enc_dev = make_dev(&enc_cdevsw, periph->unit_number,
UID_ROOT, GID_OPERATOR, 0600, "%s%d",
periph->periph_name, periph->unit_number);
Fix several reference counting and object lifetime issues between the pass(4) and enc(4) drivers and devfs. The pass(4) driver uses the destroy_dev_sched() routine to schedule its device node for destruction in a separate thread context. It does this because the passcleanup() routine can get called indirectly from the passclose() routine, and that would cause a deadlock if the close routine tried to destroy its own device node. In any case, once a particular passthrough driver number, e.g. pass3, is destroyed, CAM considers that unit number (3 in this case) available for reuse. The problem is that devfs may not be done cleaning up the previous instance of pass3, and will panic if isn't done cleaning up the previous instance. The solution is to get a callback from devfs when the device node is removed, and make sure we hold a reference to the peripheral until that happens. Testing exposed some other cases where we have reference counting issues, and those were also fixed in the pass(4) driver. cam_periph.c: In camperiphfree(), reorder some of the operations. The peripheral destructor needs to be called before the peripheral is removed from the peripheral is removed from the list. This is because once we remove the peripheral from the list, and drop the topology lock, the peripheral number may be reused. But if the destructor hasn't been called yet, there may still be resources hanging around (like devfs nodes) that haven't been fully cleaned up. cam_xpt.c: Add an argument to xpt_remove_periph() to indicate whether the topology lock is already held. scsi_enc.c: Acquire an extra reference to the peripheral during registration, and release it once we get a callback from devfs indicating that the device node is gone. Call destroy_dev_sched_cb() in enc_oninvalidate() instead of calling destroy_dev() in the cleanup routine. scsi_pass.c: Add reference counting to handle peripheral and devfs object lifetime issues. Add a reference to the peripheral and the devfs node in the peripheral registration. Don't attempt to add a physical path alias if the peripheral has been marked invalid. Release the devfs reference once the initial physical path alias taskqueue run has completed. Schedule devfs node destruction in the passoninvalidate(), and release our peripheral reference in a new routine, passdevgonecb() once the devfs node is gone. This allows the peripheral to fully go away, and the peripheral destructor, passcleanup(), will get called. MFC after: 3 days Sponsored by: Spectra Logic
2012-06-20 17:08:00 +00:00
MFprojects/zfsd: Revamp the CAM enclosure services driver. This updated driver uses an in-kernel daemon to track state changes and publishes physical path location information\for disk elements into the CAM device database. Sponsored by: Spectra Logic Corporation Sponsored by: iXsystems, Inc. Submitted by: gibbs, will, mav
2012-05-24 14:07:44 +00:00
cam_periph_lock(periph);
enc->enc_dev->si_drv1 = periph;
enc->enc_flags |= ENC_FLAG_INITIALIZED;
/*
* Add an async callback so that we get notified if this
* device goes away.
*/
xpt_register_async(AC_LOST_DEVICE, enc_async, periph, periph->path);
switch (enc->enc_type) {
default:
case ENC_NONE:
tname = "No ENC device";
break;
case ENC_SES_SCSI2:
tname = "SCSI-2 ENC Device";
break;
case ENC_SES:
tname = "SCSI-3 ENC Device";
break;
case ENC_SES_PASSTHROUGH:
tname = "ENC Passthrough Device";
break;
case ENC_SAFT:
tname = "SAF-TE Compliant Device";
break;
case ENC_SEMB_SES:
tname = "SEMB SES Device";
break;
case ENC_SEMB_SAFT:
tname = "SEMB SAF-TE Device";
break;
}
xpt_announce_periph(periph, tname);
status = CAM_REQ_CMP;
out:
if (status != CAM_REQ_CMP)
enc_dtor(periph);
return (status);
}
Reference in New Issue Copy Permalink