d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ea0ccffc39
freebsd-skq/sys/cam/cam_xpt.c

5299 lines
136 KiB
C
Raw Normal View History

Start each of the license/copyright comments with /*-
2005-01-05 22:34:37 +00:00
/*-
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* Implementation of the Common Access Method Transport (XPT) layer.
*
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
* Copyright (c) 1997, 1998, 1999 Justin T. Gibbs.
Generalize the quirk entry for the Conner CFP* drives. It did just cover the CFP2107, but it appears (not surprisingly) that the 1 gig and 4 gig versions of that drive have the same problem with tagged queueing. Also, fix the problem reported in PR kern/9482. The XPT_DEV_MATCH case in xptioctl() wasn't putting a proper path in the CCB before it called xpt_action(). When CAMDEBUG is defined, and CAM_DEBUG_TRACE debugging is turned on, the CAM_DEBUG statement at the beginning of xpt_action would end up deferencing a NULL path pointer. That of course caused a panic. My solution is to just stick the xpt peripheral's path in the CCB. PR: kern/9482 Reviewed by: gibbs
1999-01-19 01:02:47 +00:00
* Copyright (c) 1997, 1998, 1999 Kenneth D. Merry.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions, and the following disclaimer,
* without modification, immediately at the beginning of the file.
* 2. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
Use __FBSDID().
2003-06-10 17:50:20 +00:00
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <sys/param.h>
- Remove the inthand2_t type and use the equivalent driver_intr_t type from newbus for referencing device interrupt handlers. - Move the 'struct intrec' type which describes interrupt sources into sys/interrupt.h instead of making it just be a x86 structure. - Don't create 'ithd' and 'intrec' typedefs, instead, just use 'struct ithd' and 'struct intrec' - Move the code to translate new-bus interrupt flags into an interrupt thread priority out of the x86 nexus code and into a MI ithread_priority() function in sys/kern/kern_intr.c. - Remove now-uneeded x86-specific headers from sys/dev/ata/ata-all.c and sys/pci/pci_compat.c.
2000-09-13 18:33:25 +00:00
#include <sys/bus.h>
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <sys/systm.h>
#include <sys/types.h>
#include <sys/malloc.h>
#include <sys/kernel.h>
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
#include <sys/time.h>
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <sys/conf.h>
#include <sys/fcntl.h>
a more correct change that should meet the sniff test
1998-12-24 06:01:15 +00:00
#include <sys/interrupt.h>
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
#include <sys/proc.h>
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
#include <sys/sbuf.h>
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
#include <sys/smp.h>
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
#include <sys/taskqueue.h>
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Provide locking for the ccb_bioq. This allows xpt_done() to be called without Giant held. In camisr(), move the ccb_bioq elements to a temporary local list and then process the elements off of that list. This enables the list to be processed by only taking the ccb_bioq_lock once and only for a very short time. ccb_bioq_lock is a leaf mutex, so it's fine to call xpt_done() with other locks held. This is just a very minor step in the work to lock CAM, but it allows us to avoid some messy locking/unlock dances in certain drivers.
2005-02-09 11:44:15 +00:00
#include <sys/lock.h>
#include <sys/mutex.h>
Make the exploring of all luns supported by an HBA more of a tunable (until we get REPORT LUNS in place). If we're probing luns, and each probe succeeds, we keep going past lun 7 if we're a SCSI3 or better device (until we fail to probe). If we're probing luns, and a probe fails, we only keep going if we're quirked *for* it (CAM_QUIRK_HILUNS), and if we're not quirked *against* it (CAM_QUIRK_NOHILUNS), or we're a SCSI3 or better device and the tunable (kern.cam.cam_srch_hi) is set non-zero. Reviewed by: nate@rootlabs.org, gibbs@scsiguy.com, ken@kdm.com, scottl@samsco.org MFC after: 1 week
2005-09-16 01:26:17 +00:00
#include <sys/sysctl.h>
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
#include <sys/kthread.h>
Provide locking for the ccb_bioq. This allows xpt_done() to be called without Giant held. In camisr(), move the ccb_bioq elements to a temporary local list and then process the elements off of that list. This enables the list to be processed by only taking the ccb_bioq_lock once and only for a very short time. ccb_bioq_lock is a leaf mutex, so it's fine to call xpt_done() with other locks held. This is just a very minor step in the work to lock CAM, but it allows us to avoid some messy locking/unlock dances in certain drivers.
2005-02-09 11:44:15 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <cam/cam.h>
#include <cam/cam_ccb.h>
#include <cam/cam_periph.h>
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
#include <cam/cam_queue.h>
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <cam/cam_sim.h>
#include <cam/cam_xpt.h>
#include <cam/cam_xpt_sim.h>
#include <cam/cam_xpt_periph.h>
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
#include <cam/cam_xpt_internal.h>
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <cam/cam_debug.h>
Add infrastructure for doing compatibility shims, as has been sorely needed for the last 10 years. Far too much of the internal API is exposed, and every small adjustment causes applications to stop working. To kick this off, bump the API version to 0x17 as should have been done with r246713, but add shims to compensate. Thanks to the shims, there should be no visible change in application behavior. I have plans to do a significant overhaul of the API to harnen it for the future, but until then, I welcome others to add shims for older versions of the API. Obtained from: Netflix
2013-06-17 08:57:09 +00:00
#include <cam/cam_compat.h>
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include <cam/scsi/scsi_all.h>
#include <cam/scsi/scsi_message.h>
#include <cam/scsi/scsi_pass.h>
For sparc64 also adjust the geometry of da(4) driven disks to not overflow the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The geometry chosen for disks above that limit allows to use disks up to 2TB, which is the limit of the extended VTOC8 format. The geometry used for disks smaller than the 16-bit cylinders limit stays the same as used by cam_calc_geometry(9) for extended translation. Thanks to Hans-Joerg Sirtl for providing hardware for testing this change. MFC after: 3 days
2011-11-27 15:43:40 +00:00
#include <machine/md_var.h> /* geometry translation */
Add an xpt_print function to reduce most of the xpt_print_path/printf pairs. Convert the core code to use it. Reviewed by: scsi (various folks weighing in with good advice) MFC after: 2 weeks
2006-12-05 07:45:28 +00:00
#include <machine/stdarg.h> /* for xpt_print below */
For sparc64 also adjust the geometry of da(4) driven disks to not overflow the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The geometry chosen for disks above that limit allows to use disks up to 2TB, which is the limit of the extended VTOC8 format. The geometry used for disks smaller than the 16-bit cylinders limit stays the same as used by cam_calc_geometry(9) for extended translation. Thanks to Hans-Joerg Sirtl for providing hardware for testing this change. MFC after: 3 days
2011-11-27 15:43:40 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
#include "opt_cam.h"
/*
* This is the maximum number of high powered commands (e.g. start unit)
* that can be outstanding at a particular time.
*/
#ifndef CAM_MAX_HIGHPOWER
#define CAM_MAX_HIGHPOWER 4
#endif
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
/* Datastructures internal to the xpt layer */
MALLOC_DEFINE(M_CAMXPT, "CAM XPT", "CAM XPT buffers");
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
MALLOC_DEFINE(M_CAMDEV, "CAM DEV", "CAM devices");
MALLOC_DEFINE(M_CAMCCB, "CAM CCB", "CAM CCBs");
MALLOC_DEFINE(M_CAMPATH, "CAM path", "CAM paths");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
/* Object for defering XPT actions to a taskqueue */
struct xpt_task {
struct task task;
void *data1;
uintptr_t data2;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
};
Make the exploring of all luns supported by an HBA more of a tunable (until we get REPORT LUNS in place). If we're probing luns, and each probe succeeds, we keep going past lun 7 if we're a SCSI3 or better device (until we fail to probe). If we're probing luns, and a probe fails, we only keep going if we're quirked *for* it (CAM_QUIRK_HILUNS), and if we're not quirked *against* it (CAM_QUIRK_NOHILUNS), or we're a SCSI3 or better device and the tunable (kern.cam.cam_srch_hi) is set non-zero. Reviewed by: nate@rootlabs.org, gibbs@scsiguy.com, ken@kdm.com, scottl@samsco.org MFC after: 1 week
2005-09-16 01:26:17 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct xpt_softc {
Restore the CAM XPT peripheral generation counter, and export it via sysctl. Define it as an atomic uint32_t. These increments happen infrequently enough for the atomic overhead to be a problem, and since they're now independent atomics, they won't contend with xpt_lock_buses(). This counter is useful as a means of cheaply identifying whether any changes have been made to the CAM peripheral list. Userland programs have no guarantee that the counter won't change on them while being returned or while processing the information, so they must be written accordingly. Discussed with: ken, mav (in general) MFC after: 1 week Sponsored by: Spectra Logic
2015-01-20 21:15:33 +00:00
uint32_t xpt_generation;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
/* number of high powered commands that can go through right now */
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
struct mtx xpt_highpower_lock;
MFprojects/camlock r249505: Change CCB queue resize logic to be able safely handle overallocations: - (re)allocate queue space in power of 2 chunks with 64 elements minimum and never shrink it; with only 4/8 bytes per element size is insignificant. - automatically reallocate the queue to double size if it is overflowed. - if queue reallocation failed, store extra CCBs in unsorted TAILQ, fetching them back as soon as some queue element is freed. To free space in CCB for TAILQ linking, change highpowerq from keeping high-power CCBs to keeping devices frozen due to high-power CCBs. This encloses all pieces of queue resize logic inside of cam_queue.[ch], removing some not obvious duties from xpt_release_ccb().
2013-08-05 11:48:40 +00:00
STAILQ_HEAD(highpowerlist, cam_ed) highpowerq;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
int num_highpower;
/* queue for handling async rescan requests. */
TAILQ_HEAD(, ccb_hdr) ccb_scanq;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
int buses_to_config;
int buses_config_done;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
/* Registered busses */
TAILQ_HEAD(,cam_eb) xpt_busses;
u_int bus_generation;
struct intr_config_hook *xpt_config_hook;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
int boot_delay;
struct callout boot_callout;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
struct mtx xpt_topo_lock;
struct mtx xpt_lock;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct taskqueue *xpt_taskq;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
};
typedef enum {
DM_RET_COPY = 0x01,
DM_RET_FLAG_MASK = 0x0f,
DM_RET_NONE = 0x00,
DM_RET_STOP = 0x10,
DM_RET_DESCEND = 0x20,
DM_RET_ERROR = 0x30,
DM_RET_ACTION_MASK = 0xf0
} dev_match_ret;
typedef enum {
XPT_DEPTH_BUS,
XPT_DEPTH_TARGET,
XPT_DEPTH_DEVICE,
XPT_DEPTH_PERIPH
} xpt_traverse_depth;
struct xpt_traverse_config {
xpt_traverse_depth depth;
void *tr_func;
void *tr_arg;
};
typedef int xpt_busfunc_t (struct cam_eb *bus, void *arg);
typedef int xpt_targetfunc_t (struct cam_et *target, void *arg);
typedef int xpt_devicefunc_t (struct cam_ed *device, void *arg);
typedef int xpt_periphfunc_t (struct cam_periph *periph, void *arg);
typedef int xpt_pdrvfunc_t (struct periph_driver **pdrv, void *arg);
/* Transport layer configuration information */
static struct xpt_softc xsoftc;
Make periphdriver_register() take XPT lock when modifying the periph_drivers array. This fixes a panic that sometimes occured when kldloading ctl.ko. Reviewed by: mav@ MFC after: 1 month Sponsored by: The FreeBSD Foundation
2015-03-02 22:48:11 +00:00
MTX_SYSINIT(xpt_topo_init, &xsoftc.xpt_topo_lock, "XPT topology lock", MTX_DEF);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
SYSCTL_INT(_kern_cam, OID_AUTO, boot_delay, CTLFLAG_RDTUN,
&xsoftc.boot_delay, 0, "Bus registration wait time");
Restore the CAM XPT peripheral generation counter, and export it via sysctl. Define it as an atomic uint32_t. These increments happen infrequently enough for the atomic overhead to be a problem, and since they're now independent atomics, they won't contend with xpt_lock_buses(). This counter is useful as a means of cheaply identifying whether any changes have been made to the CAM peripheral list. Userland programs have no guarantee that the counter won't change on them while being returned or while processing the information, so they must be written accordingly. Discussed with: ken, mav (in general) MFC after: 1 week Sponsored by: Spectra Logic
2015-01-20 21:15:33 +00:00
SYSCTL_UINT(_kern_cam, OID_AUTO, xpt_generation, CTLFLAG_RD,
&xsoftc.xpt_generation, 0, "CAM peripheral generation count");
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_doneq {
struct mtx_padalign cam_doneq_mtx;
STAILQ_HEAD(, ccb_hdr) cam_doneq;
int cam_doneq_sleep;
};
static struct cam_doneq cam_doneqs[MAXCPU];
static int cam_num_doneqs;
static struct proc *cam_proc;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
SYSCTL_INT(_kern_cam, OID_AUTO, num_doneqs, CTLFLAG_RDTUN,
&cam_num_doneqs, 0, "Number of completion queues/threads");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*sigh* I must have been on something that night. Make xpt_periph an extern with the original in cam_xpt.c instead of replicating xpt_periph in all the sources using it (and hence not initialising it)
2000-05-07 18:04:50 +00:00
struct cam_periph *xpt_periph;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static periph_init_t xpt_periph_init;
static struct periph_driver xpt_driver =
{
xpt_periph_init, "xpt",
MFp4: - Reduce code duplication in ATA XPT and PMP driver. - Move PIO size setting from ada driver to ATA XPT. It is XPT business to negotiate transfer details. ada driver is now stateless. - Report PIO size to SIM. It is required for correct PATA SIM operation. - Tune PMP scan timings. It workarounds some problems with SiI. - If reset hapens during PMP initialization - restart it. - Introduce early-initialized periph drivers, which are used during initial scan process. Use it for xpt, probe, aprobe and pmp. It gives pmp chance to finish scan before mountroot and numerate devices in right order.
2009-10-31 10:43:38 +00:00
TAILQ_HEAD_INITIALIZER(xpt_driver.units), /* generation */ 0,
CAM_PERIPH_DRV_EARLY
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
};
Change the peripheral driver list from a linker set to module driven driver registration. This should allow things like da, sa, cd etc to be in seperate KLD's to the cam core and make them preloadable.
2001-02-07 07:05:59 +00:00
PERIPHDRIVER_DECLARE(xpt, xpt_driver);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static d_open_t xptopen;
static d_close_t xptclose;
static d_ioctl_t xptioctl;
Add infrastructure for doing compatibility shims, as has been sorely needed for the last 10 years. Far too much of the internal API is exposed, and every small adjustment causes applications to stop working. To kick this off, bump the API version to 0x17 as should have been done with r246713, but add shims to compensate. Thanks to the shims, there should be no visible change in application behavior. I have plans to do a significant overhaul of the API to harnen it for the future, but until then, I welcome others to add shims for older versions of the API. Obtained from: Netflix
2013-06-17 08:57:09 +00:00
static d_ioctl_t xptdoioctl;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
This commit should be a extensive NO-OP: Reformat and initialize correctly all "struct cdevsw". Initialize the d_maj and d_bmaj fields. The d_reset field was not removed, although it is never used. I used a program to do most of this, so all the files now use the same consistent format. Please keep it that way. Vinum and i4b not modified, patches emailed to respective authors.
1999-05-30 16:53:49 +00:00
static struct cdevsw xpt_cdevsw = {
Device megapatch 4/6: Introduce d_version field in struct cdevsw, this must always be initialized to D_VERSION. Flip sense of D_NOGIANT flag to D_NEEDGIANT, this involves removing four D_NOGIANT flags and adding 145 D_NEEDGIANT flags.
2004-02-21 21:10:55 +00:00
.d_version = D_VERSION,
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
.d_flags = 0,
Gigacommit to improve device-driver source compatibility between branches: Initialize struct cdevsw using C99 sparse initializtion and remove all initializations to default values. This patch is automatically generated and has been tested by compiling LINT with all the fields in struct cdevsw in reverse order on alpha, sparc64 and i386. Approved by: re(scottl)
2003-03-03 12:15:54 +00:00
.d_open = xptopen,
.d_close = xptclose,
.d_ioctl = xptioctl,
.d_name = "xpt",
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
};
/* Storage for debugging datastructures */
struct cam_path *cam_dpath;
Implement the usage of Report Luns as part of SCSI probing for SCP3 or better devices. This can be disabled on a per-device basis using quirks as well. This also handles the case where there is actually no connected LUN 0 (which can definitely be the case for storage arrays). Reviewed by: scsi@ MFC after: 1 month
2010-06-08 16:17:25 +00:00
u_int32_t cam_dflags = CAM_DEBUG_FLAGS;
Pull in r267961 and r267973 again. Fix for issues reported will follow.
2014-06-28 03:56:17 +00:00
SYSCTL_UINT(_kern_cam, OID_AUTO, dflags, CTLFLAG_RWTUN,
To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO, CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default. List of these flags can be modified with CAM_DEBUG_COMPILE kernel option. CAMDEBUG kernel option still enables all possible debug, if not overriden. Additional 50KB of kernel size is a good price for the ability to debug problems without rebuilding the kernel. In case where size is important, debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.
2012-06-07 10:05:51 +00:00
&cam_dflags, 0, "Enabled debug flags");
u_int32_t cam_debug_delay = CAM_DEBUG_DELAY;
Pull in r267961 and r267973 again. Fix for issues reported will follow.
2014-06-28 03:56:17 +00:00
SYSCTL_UINT(_kern_cam, OID_AUTO, debug_delay, CTLFLAG_RWTUN,
To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO, CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default. List of these flags can be modified with CAM_DEBUG_COMPILE kernel option. CAMDEBUG kernel option still enables all possible debug, if not overriden. Additional 50KB of kernel size is a good price for the ability to debug problems without rebuilding the kernel. In case where size is important, debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.
2012-06-07 10:05:51 +00:00
&cam_debug_delay, 0, "Delay in us after each debug message");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Call the xpt_init() hook during at the start of the configure() process via SYSINIT(). This gets a little closer to making cam useable as a module.
1999-01-19 00:13:05 +00:00
/* Our boot-time initialization hook */
CAM, the module: scbus, da, cd, and st wrapped in one module. Make the umass driver depend on this module. Makes it possible to compile the kernel without SCSI support and load it when for example a USB floppy is conencted.
2000-08-13 18:49:40 +00:00
static int cam_module_event_handler(module_t, int /*modeventtype_t*/, void *);
static moduledata_t cam_moduledata = {
"cam",
cam_module_event_handler,
NULL
};
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
static int xpt_init(void *);
CAM, the module: scbus, da, cd, and st wrapped in one module. Make the umass driver depend on this module. Makes it possible to compile the kernel without SCSI support and load it when for example a USB floppy is conencted.
2000-08-13 18:49:40 +00:00
DECLARE_MODULE(cam, cam_moduledata, SI_SUB_CONFIGURE, SI_ORDER_SECOND);
MODULE_VERSION(cam, 1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static void xpt_async_bcast(struct async_list *async_head,
u_int32_t async_code,
struct cam_path *path,
void *async_arg);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
static path_id_t xptnextfreepathid(void);
static path_id_t xptpathid(const char *sim_name, int sim_unit, int sim_bus);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static union ccb *xpt_get_ccb(struct cam_periph *periph);
static union ccb *xpt_get_ccb_nowait(struct cam_periph *periph);
static void xpt_run_allocq(struct cam_periph *periph, int sleep);
static void xpt_run_allocq_task(void *context, int pending);
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
static void xpt_run_devq(struct cam_devq *devq);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static timeout_t xpt_release_devq_timeout;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
static void xpt_release_simq_timeout(void *arg) __unused;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static void xpt_acquire_bus(struct cam_eb *bus);
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
static void xpt_release_bus(struct cam_eb *bus);
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
static uint32_t xpt_freeze_devq_device(struct cam_ed *dev, u_int count);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static int xpt_release_devq_device(struct cam_ed *dev, u_int count,
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
int run_queue);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static struct cam_et*
xpt_alloc_target(struct cam_eb *bus, target_id_t target_id);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static void xpt_acquire_target(struct cam_et *target);
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
static void xpt_release_target(struct cam_et *target);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static struct cam_eb*
xpt_find_bus(path_id_t path_id);
static struct cam_et*
xpt_find_target(struct cam_eb *bus, target_id_t target_id);
static struct cam_ed*
xpt_find_device(struct cam_et *target, lun_id_t lun_id);
static void xpt_config(void *arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static int xpt_schedule_dev(struct camq *queue, cam_pinfo *dev_pinfo,
u_int32_t new_priority);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static xpt_devicefunc_t xptpassannouncefunc;
static void xptaction(struct cam_sim *sim, union ccb *work_ccb);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
static void xptpoll(struct cam_sim *sim);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static void camisr_runqueue(void);
static void xpt_done_process(struct ccb_hdr *ccb_h);
static void xpt_done_td(void *);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static dev_match_ret xptbusmatch(struct dev_match_pattern *patterns,
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
u_int num_patterns, struct cam_eb *bus);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static dev_match_ret xptdevicematch(struct dev_match_pattern *patterns,
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
u_int num_patterns,
struct cam_ed *device);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static dev_match_ret xptperiphmatch(struct dev_match_pattern *patterns,
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
u_int num_patterns,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_periph *periph);
static xpt_busfunc_t xptedtbusfunc;
static xpt_targetfunc_t xptedttargetfunc;
static xpt_devicefunc_t xptedtdevicefunc;
static xpt_periphfunc_t xptedtperiphfunc;
static xpt_pdrvfunc_t xptplistpdrvfunc;
static xpt_periphfunc_t xptplistperiphfunc;
static int xptedtmatch(struct ccb_dev_match *cdm);
static int xptperiphlistmatch(struct ccb_dev_match *cdm);
static int xptbustraverse(struct cam_eb *start_bus,
xpt_busfunc_t *tr_func, void *arg);
static int xpttargettraverse(struct cam_eb *bus,
struct cam_et *start_target,
xpt_targetfunc_t *tr_func, void *arg);
static int xptdevicetraverse(struct cam_et *target,
struct cam_ed *start_device,
xpt_devicefunc_t *tr_func, void *arg);
static int xptperiphtraverse(struct cam_ed *device,
struct cam_periph *start_periph,
xpt_periphfunc_t *tr_func, void *arg);
static int xptpdrvtraverse(struct periph_driver **start_pdrv,
xpt_pdrvfunc_t *tr_func, void *arg);
static int xptpdperiphtraverse(struct periph_driver **pdrv,
struct cam_periph *start_periph,
xpt_periphfunc_t *tr_func,
void *arg);
static xpt_busfunc_t xptdefbusfunc;
static xpt_targetfunc_t xptdeftargetfunc;
static xpt_devicefunc_t xptdefdevicefunc;
static xpt_periphfunc_t xptdefperiphfunc;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
static void xpt_finishconfig_task(void *context, int pending);
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
static void xpt_dev_async_default(u_int32_t async_code,
struct cam_eb *bus,
struct cam_et *target,
struct cam_ed *device,
void *async_arg);
static struct cam_ed * xpt_alloc_device_default(struct cam_eb *bus,
struct cam_et *target,
lun_id_t lun_id);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static xpt_devicefunc_t xptsetasyncfunc;
static xpt_busfunc_t xptsetasyncbusfunc;
static cam_status xptregister(struct cam_periph *periph,
void *arg);
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
static __inline int device_is_queued(struct cam_ed *device);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static __inline int
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
xpt_schedule_devq(struct cam_devq *devq, struct cam_ed *dev)
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
{
int retval;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_assert(&devq->send_mtx, MA_OWNED);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
if ((dev->ccbq.queue.entries > 0) &&
(dev->ccbq.dev_openings > 0) &&
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
(dev->ccbq.queue.qfrozen_cnt == 0)) {
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
/*
* The priority of a device waiting for controller
Fix typos - remove duplicate "the". PR: bin/154928 Submitted by: Eitan Adler <lists at eitanadler.com> MFC after: 3 days
2011-02-21 09:01:34 +00:00
* resources is that of the highest priority CCB
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
* enqueued.
*/
retval =
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
xpt_schedule_dev(&devq->send_queue,
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&dev->devq_entry,
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
CAMQ_GET_PRIO(&dev->ccbq.queue));
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
} else {
retval = 0;
}
return (retval);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static __inline int
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
device_is_queued(struct cam_ed *device)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
return (device->devq_entry.index != CAM_UNQUEUED_INDEX);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static void
xpt_periph_init()
{
Convert the xpt, pass, pt and target drivers to use the new make_dev()/destroy_dev() interface.
1999-11-17 04:59:09 +00:00
make_dev(&xpt_cdevsw, 0, UID_ROOT, GID_OPERATOR, 0600, "xpt0");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static int
Do the dreaded s/dev_t/struct cdev */ Bump __FreeBSD_version accordingly.
2004-06-16 09:47:26 +00:00
xptopen(struct cdev *dev, int flags, int fmt, struct thread *td)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Check to make sure that this device is opened read-write, not just read only. Previously, if the device was chmoded 644, someone could open it with the O_RDONLY flag and issue any ioctl to the device. Reviewed by: imp, gibbs
1998-09-16 00:11:53 +00:00
/*
* Only allow read-write access.
*/
if (((flags & FWRITE) == 0) || ((flags & FREAD) == 0))
return(EPERM);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* We don't allow nonblocking access.
*/
if ((flags & O_NONBLOCK) != 0) {
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
printf("%s: can't do nonblocking access\n", devtoname(dev));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return(ENODEV);
}
return(0);
}
static int
Do the dreaded s/dev_t/struct cdev */ Bump __FreeBSD_version accordingly.
2004-06-16 09:47:26 +00:00
xptclose(struct cdev *dev, int flag, int fmt, struct thread *td)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
return(0);
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
/*
* Don't automatically grab the xpt softc lock here even though this is going
* through the xpt device. The xpt device is really just a back door for
* accessing other devices and SIMs, so the right thing to do is to grab
* the appropriate SIM lock once the bus/SIM is located.
*/
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static int
Do the dreaded s/dev_t/struct cdev */ Bump __FreeBSD_version accordingly.
2004-06-16 09:47:26 +00:00
xptioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flag, struct thread *td)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
int error;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Add infrastructure for doing compatibility shims, as has been sorely needed for the last 10 years. Far too much of the internal API is exposed, and every small adjustment causes applications to stop working. To kick this off, bump the API version to 0x17 as should have been done with r246713, but add shims to compensate. Thanks to the shims, there should be no visible change in application behavior. I have plans to do a significant overhaul of the API to harnen it for the future, but until then, I welcome others to add shims for older versions of the API. Obtained from: Netflix
2013-06-17 08:57:09 +00:00
if ((error = xptdoioctl(dev, cmd, addr, flag, td)) == ENOTTY) {
Re-do r255853. Along with adding back the API/ABI changes from the original, this hides the contents of cam_compat.h from ktrace/kdump/truss, avoiding problems there. There are no user-servicable parts in there, so no need for those tools to be groping around in there. Approved by: re
2013-09-25 15:55:56 +00:00
error = cam_compat_ioctl(dev, cmd, addr, flag, td, xptdoioctl);
Add infrastructure for doing compatibility shims, as has been sorely needed for the last 10 years. Far too much of the internal API is exposed, and every small adjustment causes applications to stop working. To kick this off, bump the API version to 0x17 as should have been done with r246713, but add shims to compensate. Thanks to the shims, there should be no visible change in application behavior. I have plans to do a significant overhaul of the API to harnen it for the future, but until then, I welcome others to add shims for older versions of the API. Obtained from: Netflix
2013-06-17 08:57:09 +00:00
}
return (error);
}
static int
xptdoioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flag, struct thread *td)
{
int error;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
error = 0;
switch(cmd) {
/*
* For the transport layer CAMIOCOMMAND ioctl, we really only want
* to accept CCB types that don't quite make sense to send through a
Allow XPT_PATH_INQ to be performed on the XPT device, as per the CAM spec. Return ENOTSUP for XPT_ENG_INQ and XPT_ENG_EXEC when performed on the XPT device. Reviewed by: gibbs
2000-10-30 23:30:28 +00:00
* passthrough driver. XPT_PATH_INQ is an exception to this, as stated
* in the CAM spec.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
case CAMIOCOMMAND: {
union ccb *ccb;
union ccb *inccb;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
struct cam_eb *bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
inccb = (union ccb *)addr;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
bus = xpt_find_bus(inccb->ccb_h.path_id);
Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN). It's less resource comsumptive than scanning a whole bus when the caller knows only one target has changes. Reviewed by: scsi@ Sponsored by: Panasas MFC after: 1 month
2010-05-26 22:49:42 +00:00
if (bus == NULL)
return (EINVAL);
switch (inccb->ccb_h.func_code) {
case XPT_SCAN_BUS:
case XPT_RESET_BUS:
if (inccb->ccb_h.target_id != CAM_TARGET_WILDCARD ||
inccb->ccb_h.target_lun != CAM_LUN_WILDCARD) {
xpt_release_bus(bus);
return (EINVAL);
}
break;
case XPT_SCAN_TGT:
if (inccb->ccb_h.target_id == CAM_TARGET_WILDCARD ||
inccb->ccb_h.target_lun != CAM_LUN_WILDCARD) {
xpt_release_bus(bus);
return (EINVAL);
}
break;
default:
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
break;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
switch(inccb->ccb_h.func_code) {
case XPT_SCAN_BUS:
case XPT_RESET_BUS:
Allow XPT_PATH_INQ to be performed on the XPT device, as per the CAM spec. Return ENOTSUP for XPT_ENG_INQ and XPT_ENG_EXEC when performed on the XPT device. Reviewed by: gibbs
2000-10-30 23:30:28 +00:00
case XPT_PATH_INQ:
Treat engine inquiry the same as path inquiry in xptioctl(). Return ENOTSUP for any opcode that is not supported by the XPT device. Add back a missing local declaration that seems to have been deleted by my last commit.
2000-10-31 22:07:02 +00:00
case XPT_ENG_INQ:
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_SCAN_LUN:
Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN). It's less resource comsumptive than scanning a whole bus when the caller knows only one target has changes. Reviewed by: scsi@ Sponsored by: Panasas MFC after: 1 month
2010-05-26 22:49:42 +00:00
case XPT_SCAN_TGT:
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Revert a driver API change to xpt_alloc_ccb that isn't necessary. Fix a couple of associated error checks.
2007-04-18 04:58:53 +00:00
ccb = xpt_alloc_ccb();
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Create a path using the bus, target, and lun the
* user passed in.
*/
MFprojects/camlock r248982: Stop abusing xpt_periph in random plases that really have no periph related to CCB, for example, bus scanning. NULL value is fine in such cases and it is correctly logged in debug messages as "noperiph". If at some point we need some real XPT periphs (alike to pmpX now), quite likely they will be per-bus, and not a single global instance as xpt_periph now.
2013-04-14 09:55:48 +00:00
if (xpt_create_path(&ccb->ccb_h.path, NULL,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
inccb->ccb_h.path_id,
inccb->ccb_h.target_id,
inccb->ccb_h.target_lun) !=
CAM_REQ_CMP){
error = EINVAL;
xpt_free_ccb(ccb);
break;
}
/* Ensure all of our fields are correct */
xpt_setup_ccb(&ccb->ccb_h, ccb->ccb_h.path,
inccb->ccb_h.pinfo.priority);
xpt_merge_ccb(ccb, inccb);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_lock(ccb->ccb_h.path);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cam_periph_runccb(ccb, NULL, 0, 0, NULL);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_unlock(ccb->ccb_h.path);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
bcopy(ccb, inccb, sizeof(union ccb));
xpt_free_path(ccb->ccb_h.path);
xpt_free_ccb(ccb);
break;
case XPT_DEBUG: {
union ccb ccb;
/*
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
* This is an immediate CCB, so it's okay to
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* allocate it on the stack.
*/
/*
* Create a path using the bus, target, and lun the
* user passed in.
*/
MFprojects/camlock r248982: Stop abusing xpt_periph in random plases that really have no periph related to CCB, for example, bus scanning. NULL value is fine in such cases and it is correctly logged in debug messages as "noperiph". If at some point we need some real XPT periphs (alike to pmpX now), quite likely they will be per-bus, and not a single global instance as xpt_periph now.
2013-04-14 09:55:48 +00:00
if (xpt_create_path(&ccb.ccb_h.path, NULL,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
inccb->ccb_h.path_id,
inccb->ccb_h.target_id,
inccb->ccb_h.target_lun) !=
CAM_REQ_CMP){
error = EINVAL;
break;
}
/* Ensure all of our fields are correct */
xpt_setup_ccb(&ccb.ccb_h, ccb.ccb_h.path,
inccb->ccb_h.pinfo.priority);
xpt_merge_ccb(&ccb, inccb);
xpt_action(&ccb);
bcopy(&ccb, inccb, sizeof(union ccb));
xpt_free_path(ccb.ccb_h.path);
break;
}
case XPT_DEV_MATCH: {
struct cam_periph_map_info mapinfo;
Generalize the quirk entry for the Conner CFP* drives. It did just cover the CFP2107, but it appears (not surprisingly) that the 1 gig and 4 gig versions of that drive have the same problem with tagged queueing. Also, fix the problem reported in PR kern/9482. The XPT_DEV_MATCH case in xptioctl() wasn't putting a proper path in the CCB before it called xpt_action(). When CAMDEBUG is defined, and CAM_DEBUG_TRACE debugging is turned on, the CAM_DEBUG statement at the beginning of xpt_action would end up deferencing a NULL path pointer. That of course caused a panic. My solution is to just stick the xpt peripheral's path in the CCB. PR: kern/9482 Reviewed by: gibbs
1999-01-19 01:02:47 +00:00
struct cam_path *old_path;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* We can't deal with physical addresses for this
* type of transaction.
*/
Reform the busdma API so that new types may be added without modifying every architecture's busdma_machdep.c. It is done by unifying the bus_dmamap_load_buffer() routines so that they may be called from MI code. The MD busdma is then given a chance to do any final processing in the complete() callback. The cam changes unify the bus_dmamap_load* handling in cam drivers. The arm and mips implementations are updated to track virtual addresses for sync(). Previously this was done in a type specific way. Now it is done in a generic way by recording the list of virtuals in the map. Submitted by: jeff (sponsored by EMC/Isilon) Reviewed by: kan (previous version), scottl, mjacob (isp(4), no objections for target mode changes) Discussed with: ian (arm changes) Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris), amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)
2013-02-12 16:57:20 +00:00
if ((inccb->ccb_h.flags & CAM_DATA_MASK) !=
CAM_DATA_VADDR) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
error = EINVAL;
break;
}
Generalize the quirk entry for the Conner CFP* drives. It did just cover the CFP2107, but it appears (not surprisingly) that the 1 gig and 4 gig versions of that drive have the same problem with tagged queueing. Also, fix the problem reported in PR kern/9482. The XPT_DEV_MATCH case in xptioctl() wasn't putting a proper path in the CCB before it called xpt_action(). When CAMDEBUG is defined, and CAM_DEBUG_TRACE debugging is turned on, the CAM_DEBUG statement at the beginning of xpt_action would end up deferencing a NULL path pointer. That of course caused a panic. My solution is to just stick the xpt peripheral's path in the CCB. PR: kern/9482 Reviewed by: gibbs
1999-01-19 01:02:47 +00:00
/*
* Save this in case the caller had it set to
* something in particular.
*/
old_path = inccb->ccb_h.path;
/*
* We really don't need a path for the matching
* code. The path is needed because of the
* debugging statements in xpt_action(). They
* assume that the CCB has a valid path.
*/
inccb->ccb_h.path = xpt_periph->path;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
bzero(&mapinfo, sizeof(mapinfo));
/*
* Map the pattern and match buffers into kernel
* virtual address space.
*/
Make pass, sg and targ drivers respect HBA's maxio. Previous limitation of 64K (DFLTPHYS) is quite annoying.
2015-09-30 13:31:37 +00:00
error = cam_periph_mapmem(inccb, &mapinfo, MAXPHYS);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Generalize the quirk entry for the Conner CFP* drives. It did just cover the CFP2107, but it appears (not surprisingly) that the 1 gig and 4 gig versions of that drive have the same problem with tagged queueing. Also, fix the problem reported in PR kern/9482. The XPT_DEV_MATCH case in xptioctl() wasn't putting a proper path in the CCB before it called xpt_action(). When CAMDEBUG is defined, and CAM_DEBUG_TRACE debugging is turned on, the CAM_DEBUG statement at the beginning of xpt_action would end up deferencing a NULL path pointer. That of course caused a panic. My solution is to just stick the xpt peripheral's path in the CCB. PR: kern/9482 Reviewed by: gibbs
1999-01-19 01:02:47 +00:00
if (error) {
inccb->ccb_h.path = old_path;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
Generalize the quirk entry for the Conner CFP* drives. It did just cover the CFP2107, but it appears (not surprisingly) that the 1 gig and 4 gig versions of that drive have the same problem with tagged queueing. Also, fix the problem reported in PR kern/9482. The XPT_DEV_MATCH case in xptioctl() wasn't putting a proper path in the CCB before it called xpt_action(). When CAMDEBUG is defined, and CAM_DEBUG_TRACE debugging is turned on, the CAM_DEBUG statement at the beginning of xpt_action would end up deferencing a NULL path pointer. That of course caused a panic. My solution is to just stick the xpt peripheral's path in the CCB. PR: kern/9482 Reviewed by: gibbs
1999-01-19 01:02:47 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* This is an immediate CCB, we can send it on directly.
*/
xpt_action(inccb);
/*
* Map the buffers back into user space.
*/
cam_periph_unmapmem(inccb, &mapinfo);
Generalize the quirk entry for the Conner CFP* drives. It did just cover the CFP2107, but it appears (not surprisingly) that the 1 gig and 4 gig versions of that drive have the same problem with tagged queueing. Also, fix the problem reported in PR kern/9482. The XPT_DEV_MATCH case in xptioctl() wasn't putting a proper path in the CCB before it called xpt_action(). When CAMDEBUG is defined, and CAM_DEBUG_TRACE debugging is turned on, the CAM_DEBUG statement at the beginning of xpt_action would end up deferencing a NULL path pointer. That of course caused a panic. My solution is to just stick the xpt peripheral's path in the CCB. PR: kern/9482 Reviewed by: gibbs
1999-01-19 01:02:47 +00:00
inccb->ccb_h.path = old_path;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
error = 0;
break;
}
default:
Treat engine inquiry the same as path inquiry in xptioctl(). Return ENOTSUP for any opcode that is not supported by the XPT device. Add back a missing local declaration that seems to have been deleted by my last commit.
2000-10-31 22:07:02 +00:00
error = ENOTSUP;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
}
Release the bus reference that is acquired when doing a CAMIOCOMMAND ioctl. This should make SIM removal a bit more robust.
2007-05-16 16:57:21 +00:00
xpt_release_bus(bus);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
}
/*
* This is the getpassthru ioctl. It takes a XPT_GDEVLIST ccb as input,
* with the periphal driver name and unit name filled in. The other
* fields don't really matter as input. The passthrough driver name
* ("pass"), and unit number are passed back in the ccb. The current
* device generation number, and the index into the device peripheral
* driver list, and the status are also passed back. Note that
* since we do everything in one pass, unlike the XPT_GDEVLIST ccb,
* we never return a status of CAM_GDEVLIST_LIST_CHANGED. It is
* (or rather should be) impossible for the device peripheral driver
* list to change since we look at the whole thing in one pass, and
Retire the spl() markers. Add in some minor missed locking as a result.
2007-04-19 23:34:51 +00:00
* we do it with lock protection.
style(9)
2008-01-02 01:45:31 +00:00
*
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
case CAMGETPASSTHRU: {
union ccb *ccb;
struct cam_periph *periph;
struct periph_driver **p_drv;
char *name;
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
u_int unit;
Add a "dummy light" (actually two dummy lights) to catch people who don't have the passthrough device configured in their kernel. This will hopefully reduce the number of people complaining that they can't get {camcontrol, xmcd, tosha, cdrecord, etc.} to work. Reviewed by: gibbs
1998-10-12 21:54:13 +00:00
int base_periph_found;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
ccb = (union ccb *)addr;
unit = ccb->cgdl.unit_number;
name = ccb->cgdl.periph_name;
Add a "dummy light" (actually two dummy lights) to catch people who don't have the passthrough device configured in their kernel. This will hopefully reduce the number of people complaining that they can't get {camcontrol, xmcd, tosha, cdrecord, etc.} to work. Reviewed by: gibbs
1998-10-12 21:54:13 +00:00
base_periph_found = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Sanity check -- make sure we don't get a null peripheral
* driver name.
*/
if (*ccb->cgdl.periph_name == '\0') {
error = EINVAL;
break;
}
/* Keep the list from changing while we traverse it */
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_lock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/* first find our driver in the list of drivers */
Change the peripheral driver list from a linker set to module driven driver registration. This should allow things like da, sa, cd etc to be in seperate KLD's to the cam core and make them preloadable.
2001-02-07 07:05:59 +00:00
for (p_drv = periph_drivers; *p_drv != NULL; p_drv++)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (strcmp((*p_drv)->driver_name, name) == 0)
break;
if (*p_drv == NULL) {
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
ccb->ccb_h.status = CAM_REQ_CMP_ERR;
ccb->cgdl.status = CAM_GDEVLIST_ERROR;
*ccb->cgdl.periph_name = '\0';
ccb->cgdl.unit_number = 0;
error = ENOENT;
break;
style(9)
2008-01-02 01:45:31 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Run through every peripheral instance of this driver
* and check to see whether it matches the unit passed
* in by the user. If it does, get out of the loops and
* find the passthrough driver associated with that
* peripheral driver.
*/
for (periph = TAILQ_FIRST(&(*p_drv)->units); periph != NULL;
periph = TAILQ_NEXT(periph, unit_links)) {
Remove droping topology mutex after iterating 100 periphs in CAMGETPASSTHRU. That is not so slow and so often operation to handle unneeded otherwise xsoftc.xpt_generation and respective locking complications.
2013-08-07 11:34:20 +00:00
if (periph->unit_number == unit)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
}
/*
* If we found the peripheral driver that the user passed
* in, go through all of the peripheral drivers for that
* particular device and look for a passthrough driver.
*/
if (periph != NULL) {
struct cam_ed *device;
int i;
Add a "dummy light" (actually two dummy lights) to catch people who don't have the passthrough device configured in their kernel. This will hopefully reduce the number of people complaining that they can't get {camcontrol, xmcd, tosha, cdrecord, etc.} to work. Reviewed by: gibbs
1998-10-12 21:54:13 +00:00
base_periph_found = 1;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
device = periph->path->device;
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
for (i = 0, periph = SLIST_FIRST(&device->periphs);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
periph != NULL;
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
periph = SLIST_NEXT(periph, periph_links), i++) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Check to see whether we have a
style(9)
2008-01-02 01:45:31 +00:00
* passthrough device or not.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
if (strcmp(periph->periph_name, "pass") == 0) {
/*
* Fill in the getdevlist fields.
*/
strcpy(ccb->cgdl.periph_name,
periph->periph_name);
ccb->cgdl.unit_number =
periph->unit_number;
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
if (SLIST_NEXT(periph, periph_links))
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
ccb->cgdl.status =
CAM_GDEVLIST_MORE_DEVS;
else
ccb->cgdl.status =
CAM_GDEVLIST_LAST_DEVICE;
ccb->cgdl.generation =
device->generation;
ccb->cgdl.index = i;
/*
* Fill in some CCB header fields
* that the user may want.
*/
ccb->ccb_h.path_id =
periph->path->bus->path_id;
ccb->ccb_h.target_id =
periph->path->target->target_id;
ccb->ccb_h.target_lun =
periph->path->device->lun_id;
ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
}
}
/*
* If the periph is null here, one of two things has
* happened. The first possibility is that we couldn't
* find the unit number of the particular peripheral driver
* that the user is asking about. e.g. the user asks for
* the passthrough driver for "da11". We find the list of
* "da" peripherals all right, but there is no unit 11.
* The other possibility is that we went through the list
* of peripheral drivers attached to the device structure,
* but didn't find one with the name "pass". Either way,
* we return ENOENT, since we couldn't find something.
*/
if (periph == NULL) {
ccb->ccb_h.status = CAM_REQ_CMP_ERR;
ccb->cgdl.status = CAM_GDEVLIST_ERROR;
*ccb->cgdl.periph_name = '\0';
ccb->cgdl.unit_number = 0;
error = ENOENT;
Add a "dummy light" (actually two dummy lights) to catch people who don't have the passthrough device configured in their kernel. This will hopefully reduce the number of people complaining that they can't get {camcontrol, xmcd, tosha, cdrecord, etc.} to work. Reviewed by: gibbs
1998-10-12 21:54:13 +00:00
/*
* It is unfortunate that this is even necessary,
* but there are many, many clueless users out there.
* If this is true, the user is looking for the
* passthrough driver, but doesn't have one in his
* kernel.
*/
if (base_periph_found == 1) {
printf("xptioctl: pass driver is not in the "
"kernel\n");
Correct the kernel config hint in a printf statement. Submitted by: R.Mahmatkhanov <cvs-src@yandex.ru> Reviewed by: scottl Approved by: rwatson (mentor) MFC after: 3 days
2008-01-28 18:13:03 +00:00
printf("xptioctl: put \"device pass\" in "
Add a "dummy light" (actually two dummy lights) to catch people who don't have the passthrough device configured in their kernel. This will hopefully reduce the number of people complaining that they can't get {camcontrol, xmcd, tosha, cdrecord, etc.} to work. Reviewed by: gibbs
1998-10-12 21:54:13 +00:00
"your kernel config file\n");
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
}
default:
error = ENOTTY;
break;
}
return(error);
}
CAM, the module: scbus, da, cd, and st wrapped in one module. Make the umass driver depend on this module. Makes it possible to compile the kernel without SCSI support and load it when for example a USB floppy is conencted.
2000-08-13 18:49:40 +00:00
static int
cam_module_event_handler(module_t mod, int what, void *arg)
{
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
int error;
switch (what) {
case MOD_LOAD:
if ((error = xpt_init(NULL)) != 0)
return (error);
break;
case MOD_UNLOAD:
CAM, the module: scbus, da, cd, and st wrapped in one module. Make the umass driver depend on this module. Makes it possible to compile the kernel without SCSI support and load it when for example a USB floppy is conencted.
2000-08-13 18:49:40 +00:00
return EBUSY;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
default:
Do a pass over all modules in the kernel and make them return EOPNOTSUPP for unknown events. A number of modules return EINVAL in this instance, and I have left those alone for now and instead taught MOD_QUIESCE to accept this as "didn't do anything".
2004-07-15 08:26:07 +00:00
return EOPNOTSUPP;
CAM, the module: scbus, da, cd, and st wrapped in one module. Make the umass driver depend on this module. Makes it possible to compile the kernel without SCSI support and load it when for example a USB floppy is conencted.
2000-08-13 18:49:40 +00:00
}
return 0;
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
static void
xpt_rescan_done(struct cam_periph *periph, union ccb *done_ccb)
{
if (done_ccb->ccb_h.ppriv_ptr1 == NULL) {
xpt_free_path(done_ccb->ccb_h.path);
xpt_free_ccb(done_ccb);
} else {
done_ccb->ccb_h.cbfcnp = done_ccb->ccb_h.ppriv_ptr1;
(*done_ccb->ccb_h.cbfcnp)(periph, done_ccb);
}
xpt_release_boot();
}
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
/* thread to handle bus rescans */
static void
xpt_scanner_thread(void *dummy)
{
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
union ccb *ccb;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_path path;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_lock_buses();
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
for (;;) {
Fix some instances where CAM rescans get hung up or take a long time to complete. Also, allow xpt_rescan() to rescan a LUN instead of a full bus. MFC after: 3 days Sponsored by: Copan Systems, Inc.
2009-09-15 00:15:24 +00:00
if (TAILQ_EMPTY(&xsoftc.ccb_scanq))
msleep(&xsoftc.ccb_scanq, &xsoftc.xpt_topo_lock, PRIBIO,
Report more readable state "-" for idle CAM scan thread.
2014-04-13 11:08:57 +00:00
"-", 0);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
if ((ccb = (union ccb *)TAILQ_FIRST(&xsoftc.ccb_scanq)) != NULL) {
TAILQ_REMOVE(&xsoftc.ccb_scanq, &ccb->ccb_h, sim_links.tqe);
xpt_unlock_buses();
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/*
* Since lock can be dropped inside and path freed
* by completion callback even before return here,
* take our own path copy for reference.
*/
xpt_copy_path(&path, ccb->ccb_h.path);
xpt_path_lock(&path);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_action(ccb);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_unlock(&path);
xpt_release_path(&path);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_lock_buses();
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
}
}
}
void
xpt_rescan(union ccb *ccb)
{
struct ccb_hdr *hdr;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/* Prepare request */
Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN). It's less resource comsumptive than scanning a whole bus when the caller knows only one target has changes. Reviewed by: scsi@ Sponsored by: Panasas MFC after: 1 month
2010-05-26 22:49:42 +00:00
if (ccb->ccb_h.path->target->target_id == CAM_TARGET_WILDCARD &&
Make xpt_rescan() more correct, as it was before r197208: do not use XPT_SCAN_LUN for wildcard LUN, fall back to XPT_SCAN_BUS.
2010-02-23 18:42:07 +00:00
ccb->ccb_h.path->device->lun_id == CAM_LUN_WILDCARD)
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
ccb->ccb_h.func_code = XPT_SCAN_BUS;
Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN). It's less resource comsumptive than scanning a whole bus when the caller knows only one target has changes. Reviewed by: scsi@ Sponsored by: Panasas MFC after: 1 month
2010-05-26 22:49:42 +00:00
else if (ccb->ccb_h.path->target->target_id != CAM_TARGET_WILDCARD &&
ccb->ccb_h.path->device->lun_id == CAM_LUN_WILDCARD)
ccb->ccb_h.func_code = XPT_SCAN_TGT;
else if (ccb->ccb_h.path->target->target_id != CAM_TARGET_WILDCARD &&
ccb->ccb_h.path->device->lun_id != CAM_LUN_WILDCARD)
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
ccb->ccb_h.func_code = XPT_SCAN_LUN;
Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN). It's less resource comsumptive than scanning a whole bus when the caller knows only one target has changes. Reviewed by: scsi@ Sponsored by: Panasas MFC after: 1 month
2010-05-26 22:49:42 +00:00
else {
xpt_print(ccb->ccb_h.path, "illegal scan path\n");
xpt_free_path(ccb->ccb_h.path);
xpt_free_ccb(ccb);
return;
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
ccb->ccb_h.ppriv_ptr1 = ccb->ccb_h.cbfcnp;
ccb->ccb_h.cbfcnp = xpt_rescan_done;
xpt_setup_ccb(&ccb->ccb_h, ccb->ccb_h.path, CAM_PRIORITY_XPT);
/* Don't make duplicate entries for the same paths. */
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xpt_lock_buses();
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
if (ccb->ccb_h.ppriv_ptr1 == NULL) {
TAILQ_FOREACH(hdr, &xsoftc.ccb_scanq, sim_links.tqe) {
if (xpt_path_comp(hdr->path, ccb->ccb_h.path) == 0) {
wakeup(&xsoftc.ccb_scanq);
xpt_unlock_buses();
xpt_print(ccb->ccb_h.path, "rescan already queued\n");
xpt_free_path(ccb->ccb_h.path);
xpt_free_ccb(ccb);
return;
}
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
}
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
TAILQ_INSERT_TAIL(&xsoftc.ccb_scanq, &ccb->ccb_h, sim_links.tqe);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xsoftc.buses_to_config++;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
wakeup(&xsoftc.ccb_scanq);
xpt_unlock_buses();
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/* Functions accessed by the peripheral drivers */
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
static int
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
xpt_init(void *dummy)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
struct cam_sim *xpt_sim;
struct cam_path *path;
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
struct cam_devq *devq;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cam_status status;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
int error, i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
TAILQ_INIT(&xsoftc.xpt_busses);
TAILQ_INIT(&xsoftc.ccb_scanq);
STAILQ_INIT(&xsoftc.highpowerq);
xsoftc.num_highpower = CAM_MAX_HIGHPOWER;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
mtx_init(&xsoftc.xpt_lock, "XPT lock", NULL, MTX_DEF);
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_init(&xsoftc.xpt_highpower_lock, "XPT highpower lock", NULL, MTX_DEF);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xsoftc.xpt_taskq = taskqueue_create("CAM XPT task", M_WAITOK,
taskqueue_thread_enqueue, /*context*/&xsoftc.xpt_taskq);
Provide locking for the ccb_bioq. This allows xpt_done() to be called without Giant held. In camisr(), move the ccb_bioq elements to a temporary local list and then process the elements off of that list. This enables the list to be processed by only taking the ccb_bioq_lock once and only for a very short time. ccb_bioq_lock is a leaf mutex, so it's fine to call xpt_done() with other locks held. This is just a very minor step in the work to lock CAM, but it allows us to avoid some messy locking/unlock dances in certain drivers.
2005-02-09 11:44:15 +00:00
Expose CAM_BOOT_DELAY as a kernel conf item now. This allows users who boot without loader to adjust their environments around slightly buggy or slow hardware. PR: kern/161809 Submitted by: rozhuk.im@gmail.com MFC after: 2 weeks
2013-04-20 00:33:37 +00:00
#ifdef CAM_BOOT_DELAY
/*
* Override this value at compile time to assist our users
* who don't use loader to boot a kernel.
*/
xsoftc.boot_delay = CAM_BOOT_DELAY;
#endif
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
sys/cam: spelling fixes in comments. No functional change.
2016-04-29 21:05:48 +00:00
* The xpt layer is, itself, the equivalent of a SIM.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* Allow 16 ccbs in the ccb pool for it. This should
* give decent parallelism when we probe busses and
* perform other XPT functions.
*/
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
devq = cam_simq_alloc(16);
xpt_sim = cam_sim_alloc(xptaction,
xptpoll,
"xpt",
/*softc*/NULL,
/*unit*/0,
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
/*mtx*/&xsoftc.xpt_lock,
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
/*max_dev_transactions*/0,
/*max_tagged_dev_transactions*/0,
devq);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
if (xpt_sim == NULL)
return (ENOMEM);
mtx_lock(&xsoftc.xpt_lock);
Prepare for future integration between CAM and newbus. xpt_bus_register now takes a device_t to be the parent of the bus that is being created. Most SIMs have been updated with a reasonable argument, but a few exceptions just pass NULL for now. This argument isn't used yet and the newbus integration likely won't be ready until after 7.0-RELEASE.
2007-06-17 05:55:54 +00:00
if ((status = xpt_bus_register(xpt_sim, NULL, 0)) != CAM_SUCCESS) {
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
mtx_unlock(&xsoftc.xpt_lock);
fix botch
2006-12-21 23:20:51 +00:00
printf("xpt_init: xpt_bus_register failed with status %#x,"
Pay attention to return value from xpt_bus_register in xpt_init. Obtained from: Xin Li (Coverity) MFC after: 3 days
2006-12-21 20:06:30 +00:00
" failing attach\n", status);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
return (EINVAL);
Pay attention to return value from xpt_bus_register in xpt_init. Obtained from: Xin Li (Coverity) MFC after: 3 days
2006-12-21 20:06:30 +00:00
}
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_unlock(&xsoftc.xpt_lock);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Looking at the XPT from the SIM layer, the XPT is
sys/cam: spelling fixes in comments. No functional change.
2016-04-29 21:05:48 +00:00
* the equivalent of a peripheral driver. Allocate
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* a peripheral driver entry for us.
*/
if ((status = xpt_create_path(&path, NULL, CAM_XPT_PATH_ID,
CAM_TARGET_WILDCARD,
CAM_LUN_WILDCARD)) != CAM_REQ_CMP) {
printf("xpt_init: xpt_create_path failed with status %#x,"
" failing attach\n", status);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
return (EINVAL);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
xpt_path_lock(path);
Fix a problem with the way we handled device invalidation when attaching to a device failed. In theory, the same steps that happen when we get an AC_LOST_DEVICE async notification should have been taken when a driver fails to attach. In practice, that wasn't the case. This only affected the da, cd and ch drivers, but the fix affects all peripheral drivers. There were several possible problems: - In the da driver, we didn't remove the peripheral's softc from the da driver's linked list of softcs. Once the peripheral and softc got removed, we'd get a kernel panic the next time the timeout routine called dasendorderedtag(). - In the da, cd and possibly ch drivers, we didn't remove the peripheral's devstat structure from the devstat queue. Once the peripheral and softc were removed, this could cause a panic if anyone tried to access device statistics. (one component of the linked list wouldn't exist anymore) - In the cd driver, we didn't take the peripheral off the changer run queue if it was scheduled to run. In practice, it's highly unlikely, and maybe impossible that the peripheral would have been on the changer run queue at that stage of the probe process. The fix is: - Add a new peripheral callback function (the "oninvalidate" function) that is called the first time cam_periph_invalidate() is called for a peripheral. - Create new foooninvalidate() routines for each peripheral driver. This routine is always called at splsoftcam(), and contains all the stuff that used to be in the AC_LOST_DEVICE case of the async callback handler. - Move the devstat cleanup call to the destructor/cleanup routines, since some of the drivers do I/O in their close routines. - Make sure that when we're flushing the buffer queue, we traverse it at splbio(). - Add a check for the invalid flag in the pt driver's open routine. Reviewed by: gibbs
1998-10-22 22:16:56 +00:00
cam_periph_alloc(xptregister, NULL, NULL, NULL, "xpt", CAM_PERIPH_BIO,
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
path, NULL, 0, xpt_sim);
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
xpt_path_unlock(path);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_free_path(path);
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (cam_num_doneqs < 1)
cam_num_doneqs = 1 + mp_ncpus / 6;
else if (cam_num_doneqs > MAXCPU)
cam_num_doneqs = MAXCPU;
for (i = 0; i < cam_num_doneqs; i++) {
mtx_init(&cam_doneqs[i].cam_doneq_mtx, "CAM doneq", NULL,
MTX_DEF);
STAILQ_INIT(&cam_doneqs[i].cam_doneq);
error = kproc_kthread_add(xpt_done_td, &cam_doneqs[i],
&cam_proc, NULL, 0, 0, "cam", "doneq%d", i);
if (error != 0) {
cam_num_doneqs = i;
break;
}
}
if (cam_num_doneqs < 1) {
printf("xpt_init: Cannot init completion queues "
"- failing attach\n");
return (ENOMEM);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Register a callback for when interrupts are enabled.
*/
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xsoftc.xpt_config_hook =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
(struct intr_config_hook *)malloc(sizeof(struct intr_config_hook),
Eliminate the use of M_TEMP.
2007-05-14 21:48:53 +00:00
M_CAMXPT, M_NOWAIT | M_ZERO);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
if (xsoftc.xpt_config_hook == NULL) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
printf("xpt_init: Cannot malloc config hook "
"- failing attach\n");
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
return (ENOMEM);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xsoftc.xpt_config_hook->ich_func = xpt_config;
if (config_intrhook_establish(xsoftc.xpt_config_hook) != 0) {
Eliminate the use of M_TEMP.
2007-05-14 21:48:53 +00:00
free (xsoftc.xpt_config_hook, M_CAMXPT);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
printf("xpt_init: config_intrhook_establish failed "
"- failing attach\n");
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
return (0);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static cam_status
xptregister(struct cam_periph *periph, void *arg)
{
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
struct cam_sim *xpt_sim;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (periph == NULL) {
printf("xptregister: periph was NULL!!\n");
return(CAM_REQ_CMP_ERR);
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xpt_sim = (struct cam_sim *)arg;
xpt_sim->softc = periph;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_periph = periph;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
periph->softc = NULL;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return(CAM_REQ_CMP);
}
int32_t
xpt_add_periph(struct cam_periph *periph)
{
struct cam_ed *device;
int32_t status;
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
TASK_INIT(&periph->periph_run_task, 0, xpt_run_allocq_task, periph);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
device = periph->path->device;
status = CAM_REQ_CMP;
if (device != NULL) {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&device->target->bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
device->generation++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
SLIST_INSERT_HEAD(&device->periphs, periph, periph_links);
mtx_unlock(&device->target->bus->eb_mtx);
Restore the CAM XPT peripheral generation counter, and export it via sysctl. Define it as an atomic uint32_t. These increments happen infrequently enough for the atomic overhead to be a problem, and since they're now independent atomics, they won't contend with xpt_lock_buses(). This counter is useful as a means of cheaply identifying whether any changes have been made to the CAM peripheral list. Userland programs have no guarantee that the counter won't change on them while being returned or while processing the information, so they must be written accordingly. Discussed with: ken, mav (in general) MFC after: 1 week Sponsored by: Spectra Logic
2015-01-20 21:15:33 +00:00
atomic_add_32(&xsoftc.xpt_generation, 1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return (status);
}
void
Remove droping topology mutex after iterating 100 periphs in CAMGETPASSTHRU. That is not so slow and so often operation to handle unneeded otherwise xsoftc.xpt_generation and respective locking complications.
2013-08-07 11:34:20 +00:00
xpt_remove_periph(struct cam_periph *periph)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
struct cam_ed *device;
device = periph->path->device;
if (device != NULL) {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&device->target->bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
device->generation++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
SLIST_REMOVE(&device->periphs, periph, cam_periph, periph_links);
mtx_unlock(&device->target->bus->eb_mtx);
Restore the CAM XPT peripheral generation counter, and export it via sysctl. Define it as an atomic uint32_t. These increments happen infrequently enough for the atomic overhead to be a problem, and since they're now independent atomics, they won't contend with xpt_lock_buses(). This counter is useful as a means of cheaply identifying whether any changes have been made to the CAM peripheral list. Userland programs have no guarantee that the counter won't change on them while being returned or while processing the information, so they must be written accordingly. Discussed with: ken, mav (in general) MFC after: 1 week Sponsored by: Spectra Logic
2015-01-20 21:15:33 +00:00
atomic_add_32(&xsoftc.xpt_generation, 1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
void
xpt_announce_periph(struct cam_periph *periph, char *announce_string)
{
Virtualize transport part of periph announcement.
2010-02-22 19:17:17 +00:00
struct cam_path *path = periph->path;
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_periph_assert(periph, MA_OWNED);
Unify periph invalidation and destruction reporting. Print message containing device model and serial number on invalidation. Requested by: glebius MFC after: 1 week
2013-10-15 17:59:41 +00:00
periph->flags |= CAM_PERIPH_ANNOUNCED;
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
printf("%s%d at %s%d bus %d scbus%d target %d lun %jx\n",
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
periph->periph_name, periph->unit_number,
path->bus->sim->sim_name,
path->bus->sim->unit_number,
path->bus->sim->bus_id,
MFp4: Report scbusX in xpt_announce_periph() to less confuse users by two different bus addressing schemes.
2009-09-06 18:48:18 +00:00
path->bus->path_id,
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
path->target->target_id,
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
(uintmax_t)path->device->lun_id);
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
printf("%s%d: ", periph->periph_name, periph->unit_number);
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
if (path->device->protocol == PROTO_SCSI)
Virtualize transport part of periph announcement.
2010-02-22 19:17:17 +00:00
scsi_print_inquiry(&path->device->inq_data);
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
else if (path->device->protocol == PROTO_ATA ||
path->device->protocol == PROTO_SATAPM)
ata_print_ident(&path->device->ident_data);
MFprojects/zfsd: - Add low-level support for SATA Enclosure Management Bridge (SEMB) devices -- SATA equivalents of the SCSI SES/SAF-TE devices. - Add some utility functions for SCSI SAF-TE devices access. Sponsored by: iXsystems, Inc.
2012-05-24 11:07:39 +00:00
else if (path->device->protocol == PROTO_SEMB)
semb_print_ident(
(struct sep_identify_data *)&path->device->ident_data);
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
else
Virtualize transport part of periph announcement.
2010-02-22 19:17:17 +00:00
printf("Unknown protocol device\n");
Unhide "Serial Number" lines from bootverbose. That information may be useful for system administration to have in hard copy (in logs) if one of several devices suddenly dies. Requested by: glebius MFC after: 1 week
2013-10-15 12:59:40 +00:00
if (path->device->serial_num_len > 0) {
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
/* Don't wrap the screen - print only the first 60 chars */
printf("%s%d: Serial Number %.60s\n", periph->periph_name,
periph->unit_number, path->device->serial_num);
}
Virtualize transport part of periph announcement.
2010-02-22 19:17:17 +00:00
/* Announce transport details. */
(*(path->bus->xport->announce))(periph);
/* Announce command queueing. */
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
if (path->device->inq_flags & SID_CmdQue
|| path->device->flags & CAM_DEV_TAG_AFTER_COUNT) {
Fix newlines broken at r204220.
2010-02-26 10:33:48 +00:00
printf("%s%d: Command Queueing enabled\n",
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
periph->periph_name, periph->unit_number);
}
Virtualize transport part of periph announcement.
2010-02-22 19:17:17 +00:00
/* Announce caller's details if they've passed in. */
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
if (announce_string != NULL)
printf("%s%d: %s\n", periph->periph_name,
periph->unit_number, announce_string);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Added output of device QUIRKS for CAM and AHCI devices during boot. Reviewed by: mav Approved by: pjd (mentor) MFC after: 2 weeks
2013-05-18 23:36:21 +00:00
void
xpt_announce_quirks(struct cam_periph *periph, int quirks, char *bit_string)
{
if (quirks != 0) {
printf("%s%d: quirks=0x%b\n", periph->periph_name,
periph->unit_number, quirks, bit_string);
}
}
Unify periph invalidation and destruction reporting. Print message containing device model and serial number on invalidation. Requested by: glebius MFC after: 1 week
2013-10-15 17:59:41 +00:00
void
xpt_denounce_periph(struct cam_periph *periph)
{
struct cam_path *path = periph->path;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_periph_assert(periph, MA_OWNED);
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
printf("%s%d at %s%d bus %d scbus%d target %d lun %jx\n",
Unify periph invalidation and destruction reporting. Print message containing device model and serial number on invalidation. Requested by: glebius MFC after: 1 week
2013-10-15 17:59:41 +00:00
periph->periph_name, periph->unit_number,
path->bus->sim->sim_name,
path->bus->sim->unit_number,
path->bus->sim->bus_id,
path->bus->path_id,
path->target->target_id,
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
(uintmax_t)path->device->lun_id);
Unify periph invalidation and destruction reporting. Print message containing device model and serial number on invalidation. Requested by: glebius MFC after: 1 week
2013-10-15 17:59:41 +00:00
printf("%s%d: ", periph->periph_name, periph->unit_number);
if (path->device->protocol == PROTO_SCSI)
scsi_print_inquiry_short(&path->device->inq_data);
else if (path->device->protocol == PROTO_ATA ||
path->device->protocol == PROTO_SATAPM)
ata_print_ident_short(&path->device->ident_data);
else if (path->device->protocol == PROTO_SEMB)
semb_print_ident_short(
(struct sep_identify_data *)&path->device->ident_data);
else
printf("Unknown protocol device");
if (path->device->serial_num_len > 0)
printf(" s/n %.60s", path->device->serial_num);
printf(" detached\n");
}
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
int
xpt_getattr(char *buf, size_t len, const char *attr, struct cam_path *path)
{
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
int ret = -1, l;
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
struct ccb_dev_advinfo cdai;
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
struct scsi_vpd_id_descriptor *idd;
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_assert(path, MA_OWNED);
Protect xpt_getattr() calls with the SIM lock and assert that. Submitted by: ken@ (earlier version)
2012-10-12 17:18:24 +00:00
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
memset(&cdai, 0, sizeof(cdai));
xpt_setup_ccb(&cdai.ccb_h, path, CAM_PRIORITY_NORMAL);
cdai.ccb_h.func_code = XPT_DEV_ADVINFO;
cdai.bufsiz = len;
if (!strcmp(attr, "GEOM::ident"))
cdai.buftype = CDAI_TYPE_SERIAL_NUM;
else if (!strcmp(attr, "GEOM::physpath"))
cdai.buftype = CDAI_TYPE_PHYS_PATH;
Add new attribute lunname to report only textual LUN-specific device IDs. While lunid attribute prefers to report numeric ones, having both may be useful in some situations.
2013-08-24 09:42:14 +00:00
else if (strcmp(attr, "GEOM::lunid") == 0 ||
strcmp(attr, "GEOM::lunname") == 0) {
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
cdai.buftype = CDAI_TYPE_SCSI_DEVID;
cdai.bufsiz = CAM_SCSI_DEVID_MAXLEN;
} else
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
goto out;
cdai.buf = malloc(cdai.bufsiz, M_CAMXPT, M_NOWAIT|M_ZERO);
if (cdai.buf == NULL) {
ret = ENOMEM;
goto out;
}
xpt_action((union ccb *)&cdai); /* can only be synchronous */
if ((cdai.ccb_h.status & CAM_DEV_QFRZN) != 0)
cam_release_devq(cdai.ccb_h.path, 0, 0, 0, FALSE);
if (cdai.provsiz == 0)
goto out;
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
if (cdai.buftype == CDAI_TYPE_SCSI_DEVID) {
Add new attribute lunname to report only textual LUN-specific device IDs. While lunid attribute prefers to report numeric ones, having both may be useful in some situations.
2013-08-24 09:42:14 +00:00
if (strcmp(attr, "GEOM::lunid") == 0) {
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
idd = scsi_get_devid((struct scsi_vpd_device_id *)cdai.buf,
Add new attribute lunname to report only textual LUN-specific device IDs. While lunid attribute prefers to report numeric ones, having both may be useful in some situations.
2013-08-24 09:42:14 +00:00
cdai.provsiz, scsi_devid_is_lun_naa);
if (idd == NULL)
idd = scsi_get_devid((struct scsi_vpd_device_id *)cdai.buf,
cdai.provsiz, scsi_devid_is_lun_eui64);
} else
idd = NULL;
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
if (idd == NULL)
idd = scsi_get_devid((struct scsi_vpd_device_id *)cdai.buf,
cdai.provsiz, scsi_devid_is_lun_t10);
if (idd == NULL)
idd = scsi_get_devid((struct scsi_vpd_device_id *)cdai.buf,
cdai.provsiz, scsi_devid_is_lun_name);
if (idd == NULL)
goto out;
ret = 0;
When passing LUN IDs through treat ASCII values as fixed-length, not interpreating NULLs as EOLs, but converting them to spaces. SPC-4 does not tell that T10-based IDs should be NULL-terminated/padded. And while it tells that it should include only ASCII chars (0x20-0x7F), there are some USB sticks (SanDisk Ultra Fit), that have NULLs inside the value. Treating NULLs as EOLs there made those LUN IDs non-unique. MFC after: 1 week
2014-12-01 15:21:54 +00:00
if ((idd->proto_codeset & SVPD_ID_CODESET_MASK) == SVPD_ID_CODESET_ASCII) {
if (idd->length < len) {
for (l = 0; l < idd->length; l++)
buf[l] = idd->identifier[l] ?
idd->identifier[l] : ' ';
buf[l] = 0;
} else
ret = EFAULT;
} else if ((idd->proto_codeset & SVPD_ID_CODESET_MASK) == SVPD_ID_CODESET_UTF8) {
Make CAM return and GEOM DISK pass through new GEOM::lunid attribute. SPC-4 specification states that serial number may be property of device, but not a specific logical unit. People reported about FC storages using serial number in that way, making it unusable for purposes of LUN multipath detection. SPC-4 states that designators associated with logical unit from the VPD page 83h "Device Identification" should be used for that purpose. Report first of them in the new attribute in such preference order: NAA, EUI-64, T10 and SCSI name string. While there, make GEOM DISK properly report GEOM::ident in XML output also using d_getattr() method, if available. This fixes serial numbers reporting for SCSI disks in `geom disk list` output and confxml. Discussed with: gibbs, ken Sponsored by: iXsystems, Inc. MFC after: 2 weeks
2013-06-12 13:36:20 +00:00
l = strnlen(idd->identifier, idd->length);
if (l < len) {
bcopy(idd->identifier, buf, l);
buf[l] = 0;
} else
ret = EFAULT;
} else {
if (idd->length * 2 < len) {
for (l = 0; l < idd->length; l++)
sprintf(buf + l * 2, "%02x",
idd->identifier[l]);
} else
ret = EFAULT;
}
} else {
ret = 0;
if (strlcpy(buf, cdai.buf, len) >= len)
ret = EFAULT;
}
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
out:
if (cdai.buf != NULL)
free(cdai.buf, M_CAMXPT);
return ret;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static dev_match_ret
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
xptbusmatch(struct dev_match_pattern *patterns, u_int num_patterns,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_eb *bus)
{
dev_match_ret retval;
cam: unsign some types to match their definitions and avoid overflows. numpatterns is u_int. ctl: CTL_NUM_MODE_PAGES comes from sizeof(). In struct:ctl_scsiio, kern_sg_entries is uint32_t. MFC after: 2 weeks
2016-04-27 15:35:05 +00:00
u_int i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = DM_RET_NONE;
/*
* If we aren't given something to match against, that's an error.
*/
if (bus == NULL)
return(DM_RET_ERROR);
/*
* If there are no match entries, then this bus matches no
* matter what.
*/
if ((patterns == NULL) || (num_patterns == 0))
return(DM_RET_DESCEND | DM_RET_COPY);
for (i = 0; i < num_patterns; i++) {
struct bus_match_pattern *cur_pattern;
/*
* If the pattern in question isn't for a bus node, we
* aren't interested. However, we do indicate to the
* calling routine that we should continue descending the
* tree, since the user wants to match against lower-level
* EDT elements.
*/
if (patterns[i].type != DEV_MATCH_BUS) {
if ((retval & DM_RET_ACTION_MASK) == DM_RET_NONE)
retval |= DM_RET_DESCEND;
continue;
}
cur_pattern = &patterns[i].pattern.bus_pattern;
/*
* If they want to match any bus node, we give them any
* device node.
*/
if (cur_pattern->flags == BUS_MATCH_ANY) {
/* set the copy flag */
retval |= DM_RET_COPY;
/*
* If we've already decided on an action, go ahead
* and return.
*/
if ((retval & DM_RET_ACTION_MASK) != DM_RET_NONE)
return(retval);
}
/*
* Not sure why someone would do this...
*/
if (cur_pattern->flags == BUS_MATCH_NONE)
continue;
if (((cur_pattern->flags & BUS_MATCH_PATH) != 0)
&& (cur_pattern->path_id != bus->path_id))
continue;
if (((cur_pattern->flags & BUS_MATCH_BUS_ID) != 0)
&& (cur_pattern->bus_id != bus->sim->bus_id))
continue;
if (((cur_pattern->flags & BUS_MATCH_UNIT) != 0)
&& (cur_pattern->unit_number != bus->sim->unit_number))
continue;
if (((cur_pattern->flags & BUS_MATCH_NAME) != 0)
&& (strncmp(cur_pattern->dev_name, bus->sim->sim_name,
DEV_IDLEN) != 0))
continue;
/*
style(9)
2008-01-02 01:45:31 +00:00
* If we get to this point, the user definitely wants
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* information on this bus. So tell the caller to copy the
* data out.
*/
retval |= DM_RET_COPY;
/*
* If the return action has been set to descend, then we
* know that we've already seen a non-bus matching
* expression, therefore we need to further descend the tree.
* This won't change by continuing around the loop, so we
* go ahead and return. If we haven't seen a non-bus
* matching expression, we keep going around the loop until
* we exhaust the matching expressions. We'll set the stop
* flag once we fall out of the loop.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_DESCEND)
return(retval);
}
/*
* If the return action hasn't been set to descend yet, that means
* we haven't seen anything other than bus matching patterns. So
* tell the caller to stop descending the tree -- the user doesn't
* want to match against lower level tree elements.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_NONE)
retval |= DM_RET_STOP;
return(retval);
}
static dev_match_ret
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
xptdevicematch(struct dev_match_pattern *patterns, u_int num_patterns,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_ed *device)
{
dev_match_ret retval;
cam: unsign some types to match their definitions and avoid overflows. numpatterns is u_int. ctl: CTL_NUM_MODE_PAGES comes from sizeof(). In struct:ctl_scsiio, kern_sg_entries is uint32_t. MFC after: 2 weeks
2016-04-27 15:35:05 +00:00
u_int i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = DM_RET_NONE;
/*
* If we aren't given something to match against, that's an error.
*/
if (device == NULL)
return(DM_RET_ERROR);
/*
* If there are no match entries, then this device matches no
* matter what.
*/
Compare the *number* of patterns to zero, not the *pointer* to the patterns. (These lines are correct the other two times they appear.) Reported by: "Ted Unangst" <tedu@coverity.com> Approved by: rwatson (mentor), ken (scsi)
2004-02-28 12:59:56 +00:00
if ((patterns == NULL) || (num_patterns == 0))
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return(DM_RET_DESCEND | DM_RET_COPY);
for (i = 0; i < num_patterns; i++) {
struct device_match_pattern *cur_pattern;
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
struct scsi_vpd_device_id *device_id_page;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* If the pattern in question isn't for a device node, we
* aren't interested.
*/
if (patterns[i].type != DEV_MATCH_DEVICE) {
if ((patterns[i].type == DEV_MATCH_PERIPH)
&& ((retval & DM_RET_ACTION_MASK) == DM_RET_NONE))
retval |= DM_RET_DESCEND;
continue;
}
cur_pattern = &patterns[i].pattern.device_pattern;
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
/* Error out if mutually exclusive options are specified. */
if ((cur_pattern->flags & (DEV_MATCH_INQUIRY|DEV_MATCH_DEVID))
== (DEV_MATCH_INQUIRY|DEV_MATCH_DEVID))
return(DM_RET_ERROR);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* If they want to match any device node, we give them any
* device node.
*/
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
if (cur_pattern->flags == DEV_MATCH_ANY)
goto copy_dev_node;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Not sure why someone would do this...
*/
if (cur_pattern->flags == DEV_MATCH_NONE)
continue;
if (((cur_pattern->flags & DEV_MATCH_PATH) != 0)
&& (cur_pattern->path_id != device->target->bus->path_id))
continue;
if (((cur_pattern->flags & DEV_MATCH_TARGET) != 0)
&& (cur_pattern->target_id != device->target->target_id))
continue;
if (((cur_pattern->flags & DEV_MATCH_LUN) != 0)
&& (cur_pattern->target_lun != device->lun_id))
continue;
if (((cur_pattern->flags & DEV_MATCH_INQUIRY) != 0)
&& (cam_quirkmatch((caddr_t)&device->inq_data,
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
(caddr_t)&cur_pattern->data.inq_pat,
1, sizeof(cur_pattern->data.inq_pat),
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
scsi_static_inquiry_match) == NULL))
continue;
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
device_id_page = (struct scsi_vpd_device_id *)device->device_id;
if (((cur_pattern->flags & DEV_MATCH_DEVID) != 0)
&& (device->device_id_len < SVPD_DEVICE_ID_HDR_LEN
|| scsi_devid_match((uint8_t *)device_id_page->desc_list,
device->device_id_len
- SVPD_DEVICE_ID_HDR_LEN,
cur_pattern->data.devid_pat.id,
cur_pattern->data.devid_pat.id_len) != 0))
continue;
copy_dev_node:
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
style(9)
2008-01-02 01:45:31 +00:00
* If we get to this point, the user definitely wants
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* information on this device. So tell the caller to copy
* the data out.
*/
retval |= DM_RET_COPY;
/*
* If the return action has been set to descend, then we
* know that we've already seen a peripheral matching
* expression, therefore we need to further descend the tree.
* This won't change by continuing around the loop, so we
* go ahead and return. If we haven't seen a peripheral
* matching expression, we keep going around the loop until
* we exhaust the matching expressions. We'll set the stop
* flag once we fall out of the loop.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_DESCEND)
return(retval);
}
/*
* If the return action hasn't been set to descend yet, that means
* we haven't seen any peripheral matching patterns. So tell the
* caller to stop descending the tree -- the user doesn't want to
* match against lower level tree elements.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_NONE)
retval |= DM_RET_STOP;
return(retval);
}
/*
* Match a single peripheral against any number of match patterns.
*/
static dev_match_ret
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
xptperiphmatch(struct dev_match_pattern *patterns, u_int num_patterns,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_periph *periph)
{
dev_match_ret retval;
cam: unsign some types to match their definitions and avoid overflows. numpatterns is u_int. ctl: CTL_NUM_MODE_PAGES comes from sizeof(). In struct:ctl_scsiio, kern_sg_entries is uint32_t. MFC after: 2 weeks
2016-04-27 15:35:05 +00:00
u_int i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* If we aren't given something to match against, that's an error.
*/
if (periph == NULL)
return(DM_RET_ERROR);
/*
* If there are no match entries, then this peripheral matches no
* matter what.
*/
if ((patterns == NULL) || (num_patterns == 0))
return(DM_RET_STOP | DM_RET_COPY);
/*
* There aren't any nodes below a peripheral node, so there's no
* reason to descend the tree any further.
*/
retval = DM_RET_STOP;
for (i = 0; i < num_patterns; i++) {
struct periph_match_pattern *cur_pattern;
/*
* If the pattern in question isn't for a peripheral, we
* aren't interested.
*/
if (patterns[i].type != DEV_MATCH_PERIPH)
continue;
cur_pattern = &patterns[i].pattern.periph_pattern;
/*
* If they want to match on anything, then we will do so.
*/
if (cur_pattern->flags == PERIPH_MATCH_ANY) {
/* set the copy flag */
retval |= DM_RET_COPY;
/*
* We've already set the return action to stop,
* since there are no nodes below peripherals in
* the tree.
*/
return(retval);
}
/*
* Not sure why someone would do this...
*/
if (cur_pattern->flags == PERIPH_MATCH_NONE)
continue;
if (((cur_pattern->flags & PERIPH_MATCH_PATH) != 0)
&& (cur_pattern->path_id != periph->path->bus->path_id))
continue;
/*
* For the target and lun id's, we have to make sure the
* target and lun pointers aren't NULL. The xpt peripheral
* has a wildcard target and device.
*/
if (((cur_pattern->flags & PERIPH_MATCH_TARGET) != 0)
&& ((periph->path->target == NULL)
||(cur_pattern->target_id != periph->path->target->target_id)))
continue;
if (((cur_pattern->flags & PERIPH_MATCH_LUN) != 0)
&& ((periph->path->device == NULL)
|| (cur_pattern->target_lun != periph->path->device->lun_id)))
continue;
if (((cur_pattern->flags & PERIPH_MATCH_UNIT) != 0)
&& (cur_pattern->unit_number != periph->unit_number))
continue;
if (((cur_pattern->flags & PERIPH_MATCH_NAME) != 0)
&& (strncmp(cur_pattern->periph_name, periph->periph_name,
DEV_IDLEN) != 0))
continue;
/*
style(9)
2008-01-02 01:45:31 +00:00
* If we get to this point, the user definitely wants
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* information on this peripheral. So tell the caller to
* copy the data out.
*/
retval |= DM_RET_COPY;
/*
* The return action has already been set to stop, since
* peripherals don't have any nodes below them in the EDT.
*/
return(retval);
}
/*
* If we get to this point, the peripheral that was passed in
* doesn't match any of the patterns.
*/
return(retval);
}
static int
xptedtbusfunc(struct cam_eb *bus, void *arg)
{
struct ccb_dev_match *cdm;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_et *target;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
dev_match_ret retval;
cdm = (struct ccb_dev_match *)arg;
/*
* If our position is for something deeper in the tree, that means
* that we've already seen this node. So, we keep going down.
*/
if ((cdm->pos.position_type & CAM_DEV_POS_BUS)
&& (cdm->pos.cookie.bus == bus)
&& (cdm->pos.position_type & CAM_DEV_POS_TARGET)
&& (cdm->pos.cookie.target != NULL))
retval = DM_RET_DESCEND;
else
retval = xptbusmatch(cdm->patterns, cdm->num_patterns, bus);
/*
* If we got an error, bail out of the search.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_ERROR) {
cdm->status = CAM_DEV_MATCH_ERROR;
return(0);
}
/*
* If the copy flag is set, copy this bus out.
*/
if (retval & DM_RET_COPY) {
int spaceleft, j;
spaceleft = cdm->match_buf_len - (cdm->num_matches *
sizeof(struct dev_match_result));
/*
* If we don't have enough space to put in another
* match result, save our position and tell the
* user there are more devices to check.
*/
if (spaceleft < sizeof(struct dev_match_result)) {
bzero(&cdm->pos, sizeof(cdm->pos));
style(9)
2008-01-02 01:45:31 +00:00
cdm->pos.position_type =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEV_POS_EDT | CAM_DEV_POS_BUS;
cdm->pos.cookie.bus = bus;
cdm->pos.generations[CAM_BUS_GENERATION]=
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xsoftc.bus_generation;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cdm->status = CAM_DEV_MATCH_MORE;
return(0);
}
j = cdm->num_matches;
cdm->num_matches++;
cdm->matches[j].type = DEV_MATCH_BUS;
cdm->matches[j].result.bus_result.path_id = bus->path_id;
cdm->matches[j].result.bus_result.bus_id = bus->sim->bus_id;
cdm->matches[j].result.bus_result.unit_number =
bus->sim->unit_number;
strncpy(cdm->matches[j].result.bus_result.dev_name,
bus->sim->sim_name, DEV_IDLEN);
}
/*
* If the user is only interested in busses, there's no
* reason to descend to the next level in the tree.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_STOP)
return(1);
/*
* If there is a target generation recorded, check it to
* make sure the target list hasn't changed.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((cdm->pos.position_type & CAM_DEV_POS_BUS)
&& (cdm->pos.cookie.bus == bus)
&& (cdm->pos.position_type & CAM_DEV_POS_TARGET)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.target != NULL)) {
if ((cdm->pos.generations[CAM_TARGET_GENERATION] !=
bus->generation)) {
mtx_unlock(&bus->eb_mtx);
cdm->status = CAM_DEV_MATCH_LIST_CHANGED;
return (0);
}
target = (struct cam_et *)cdm->pos.cookie.target;
target->refcount++;
} else
target = NULL;
mtx_unlock(&bus->eb_mtx);
return (xpttargettraverse(bus, target, xptedttargetfunc, arg));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static int
xptedttargetfunc(struct cam_et *target, void *arg)
{
struct ccb_dev_match *cdm;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus;
struct cam_ed *device;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cdm = (struct ccb_dev_match *)arg;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
bus = target->bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* If there is a device list generation recorded, check it to
* make sure the device list hasn't changed.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((cdm->pos.position_type & CAM_DEV_POS_BUS)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.bus == bus)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
&& (cdm->pos.position_type & CAM_DEV_POS_TARGET)
&& (cdm->pos.cookie.target == target)
&& (cdm->pos.position_type & CAM_DEV_POS_DEVICE)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.device != NULL)) {
if (cdm->pos.generations[CAM_DEV_GENERATION] !=
target->generation) {
mtx_unlock(&bus->eb_mtx);
cdm->status = CAM_DEV_MATCH_LIST_CHANGED;
return(0);
}
device = (struct cam_ed *)cdm->pos.cookie.device;
device->refcount++;
} else
device = NULL;
mtx_unlock(&bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
return (xptdevicetraverse(target, device, xptedtdevicefunc, arg));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static int
xptedtdevicefunc(struct cam_ed *device, void *arg)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus;
struct cam_periph *periph;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct ccb_dev_match *cdm;
dev_match_ret retval;
cdm = (struct ccb_dev_match *)arg;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
bus = device->target->bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* If our position is for something deeper in the tree, that means
* that we've already seen this node. So, we keep going down.
*/
if ((cdm->pos.position_type & CAM_DEV_POS_DEVICE)
&& (cdm->pos.cookie.device == device)
&& (cdm->pos.position_type & CAM_DEV_POS_PERIPH)
&& (cdm->pos.cookie.periph != NULL))
retval = DM_RET_DESCEND;
else
retval = xptdevicematch(cdm->patterns, cdm->num_patterns,
device);
if ((retval & DM_RET_ACTION_MASK) == DM_RET_ERROR) {
cdm->status = CAM_DEV_MATCH_ERROR;
return(0);
}
/*
* If the copy flag is set, copy this device out.
*/
if (retval & DM_RET_COPY) {
int spaceleft, j;
spaceleft = cdm->match_buf_len - (cdm->num_matches *
sizeof(struct dev_match_result));
/*
* If we don't have enough space to put in another
* match result, save our position and tell the
* user there are more devices to check.
*/
if (spaceleft < sizeof(struct dev_match_result)) {
bzero(&cdm->pos, sizeof(cdm->pos));
style(9)
2008-01-02 01:45:31 +00:00
cdm->pos.position_type =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEV_POS_EDT | CAM_DEV_POS_BUS |
CAM_DEV_POS_TARGET | CAM_DEV_POS_DEVICE;
cdm->pos.cookie.bus = device->target->bus;
cdm->pos.generations[CAM_BUS_GENERATION]=
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xsoftc.bus_generation;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cdm->pos.cookie.target = device->target;
cdm->pos.generations[CAM_TARGET_GENERATION] =
device->target->bus->generation;
cdm->pos.cookie.device = device;
style(9)
2008-01-02 01:45:31 +00:00
cdm->pos.generations[CAM_DEV_GENERATION] =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
device->target->generation;
cdm->status = CAM_DEV_MATCH_MORE;
return(0);
}
j = cdm->num_matches;
cdm->num_matches++;
cdm->matches[j].type = DEV_MATCH_DEVICE;
cdm->matches[j].result.device_result.path_id =
device->target->bus->path_id;
cdm->matches[j].result.device_result.target_id =
device->target->target_id;
cdm->matches[j].result.device_result.target_lun =
device->lun_id;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cdm->matches[j].result.device_result.protocol =
device->protocol;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
bcopy(&device->inq_data,
&cdm->matches[j].result.device_result.inq_data,
sizeof(struct scsi_inquiry_data));
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
bcopy(&device->ident_data,
&cdm->matches[j].result.device_result.ident_data,
sizeof(struct ata_params));
Add a number of interrelated CAM feature enhancements and bug fixes. NOTE: These changes will require recompilation of any userland applications, like cdrecord, xmcd, etc., that use the CAM passthrough interface. A make world is recommended. camcontrol.[c8]: - We now support two new commands, "tags" and "negotiate". - The tags commands allows users to view the number of tagged openings for a device as well as a number of other related parameters, and it allows users to set tagged openings for a device. - The negotiate command allows users to enable and disable disconnection and tagged queueing, set sync rates, offsets and bus width. Note that not all of those features are available for all controllers. Only the adv, ahc, and ncr drivers fully support all of the features at this point. Some cards do not allow the setting of sync rates, offsets and the like, and some of the drivers don't have any facilities to do so. Some drivers, like the adw driver, only support enabling or disabling sync negotiation, but do not support setting sync rates. - new description in the camcontrol man page of how to format a disk - cleanup of the camcontrol inquiry command - add support in the 'devlist' command for skipping unconfigured devices if -v was not specified on the command line. - make use of the new base_transfer_speed in the path inquiry CCB. - fix CCB bzero cases cam_xpt.c, cam_sim.[ch], cam_ccb.h: - new flags on many CCB function codes to designate whether they're non-immediate, use a user-supplied CCB, and can only be passed from userland programs via the xpt device. Use these flags in the transport layer and pass driver to categorize CCBs. - new flag in the transport layer device matching code for device nodes that indicates whether a device is unconfigured - bump the CAM version from 0x10 to 0x11 - Change the CAM ioctls to use the version as their group code, so we can force users to recompile code even when the CCB size doesn't change. - add + fill in a new value in the path inquiry CCB, base_transfer_speed. Remove a corresponding field from the cam_sim structure, and add code to every SIM to set this field to the proper value. - Fix the set transfer settings code in the transport layer. scsi_cd.c: - make some variables volatile instead of just casting them in various places - fix a race condition in the changer code - attach unless we get a "logical unit not supported" error. This should fix all of the cases where people have devices that return weird errors when they don't have media in the drive. scsi_da.c: - attach unless we get a "logical unit not supported" error scsi_pass.c: - for immediate CCBs, just malloc a CCB to send the user request in. This gets rid of the 'held' count problem in camcontrol tags. scsi_pass.h: - change the CAM ioctls to use the CAM version as their group code. adv driver: - Allow changing the sync rate and offset separately. adw driver - Allow changing the sync rate and offset separately. aha driver: - Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs. ahc driver: - Allow setting offset and sync rate separately bt driver: - Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs. NCR driver: - Fix the ultra/ultra 2 negotiation bug - allow setting both the sync rate and offset separately Other HBA drivers: - Put code in to set the base_transfer_speed field for XPT_GET_TRAN_SETTINGS CCBs. Reviewed by: gibbs, mjacob (isp), imp (aha)
1999-05-06 20:16:39 +00:00
/* Let the user know whether this device is unconfigured */
if (device->flags & CAM_DEV_UNCONFIGURED)
cdm->matches[j].result.device_result.flags =
DEV_RESULT_UNCONFIGURED;
else
cdm->matches[j].result.device_result.flags =
DEV_RESULT_NOFLAG;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
/*
* If the user isn't interested in peripherals, don't descend
* the tree any further.
*/
if ((retval & DM_RET_ACTION_MASK) == DM_RET_STOP)
return(1);
/*
* If there is a peripheral list generation recorded, make sure
* it hasn't changed.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_lock_buses();
mtx_lock(&bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((cdm->pos.position_type & CAM_DEV_POS_BUS)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.bus == bus)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
&& (cdm->pos.position_type & CAM_DEV_POS_TARGET)
&& (cdm->pos.cookie.target == device->target)
&& (cdm->pos.position_type & CAM_DEV_POS_DEVICE)
&& (cdm->pos.cookie.device == device)
&& (cdm->pos.position_type & CAM_DEV_POS_PERIPH)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.periph != NULL)) {
if (cdm->pos.generations[CAM_PERIPH_GENERATION] !=
device->generation) {
mtx_unlock(&bus->eb_mtx);
xpt_unlock_buses();
cdm->status = CAM_DEV_MATCH_LIST_CHANGED;
return(0);
}
periph = (struct cam_periph *)cdm->pos.cookie.periph;
periph->refcount++;
} else
periph = NULL;
mtx_unlock(&bus->eb_mtx);
xpt_unlock_buses();
return (xptperiphtraverse(device, periph, xptedtperiphfunc, arg));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static int
xptedtperiphfunc(struct cam_periph *periph, void *arg)
{
struct ccb_dev_match *cdm;
dev_match_ret retval;
cdm = (struct ccb_dev_match *)arg;
retval = xptperiphmatch(cdm->patterns, cdm->num_patterns, periph);
if ((retval & DM_RET_ACTION_MASK) == DM_RET_ERROR) {
cdm->status = CAM_DEV_MATCH_ERROR;
return(0);
}
/*
* If the copy flag is set, copy this peripheral out.
*/
if (retval & DM_RET_COPY) {
int spaceleft, j;
spaceleft = cdm->match_buf_len - (cdm->num_matches *
sizeof(struct dev_match_result));
/*
* If we don't have enough space to put in another
* match result, save our position and tell the
* user there are more devices to check.
*/
if (spaceleft < sizeof(struct dev_match_result)) {
bzero(&cdm->pos, sizeof(cdm->pos));
style(9)
2008-01-02 01:45:31 +00:00
cdm->pos.position_type =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEV_POS_EDT | CAM_DEV_POS_BUS |
CAM_DEV_POS_TARGET | CAM_DEV_POS_DEVICE |
CAM_DEV_POS_PERIPH;
cdm->pos.cookie.bus = periph->path->bus;
cdm->pos.generations[CAM_BUS_GENERATION]=
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xsoftc.bus_generation;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cdm->pos.cookie.target = periph->path->target;
cdm->pos.generations[CAM_TARGET_GENERATION] =
periph->path->bus->generation;
cdm->pos.cookie.device = periph->path->device;
style(9)
2008-01-02 01:45:31 +00:00
cdm->pos.generations[CAM_DEV_GENERATION] =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
periph->path->target->generation;
cdm->pos.cookie.periph = periph;
cdm->pos.generations[CAM_PERIPH_GENERATION] =
periph->path->device->generation;
cdm->status = CAM_DEV_MATCH_MORE;
return(0);
}
j = cdm->num_matches;
cdm->num_matches++;
cdm->matches[j].type = DEV_MATCH_PERIPH;
cdm->matches[j].result.periph_result.path_id =
periph->path->bus->path_id;
cdm->matches[j].result.periph_result.target_id =
periph->path->target->target_id;
cdm->matches[j].result.periph_result.target_lun =
periph->path->device->lun_id;
cdm->matches[j].result.periph_result.unit_number =
periph->unit_number;
strncpy(cdm->matches[j].result.periph_result.periph_name,
periph->periph_name, DEV_IDLEN);
}
return(1);
}
static int
xptedtmatch(struct ccb_dev_match *cdm)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
int ret;
cdm->num_matches = 0;
/*
* Check the bus list generation. If it has changed, the user
* needs to reset everything and start over.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_lock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((cdm->pos.position_type & CAM_DEV_POS_BUS)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.bus != NULL)) {
if (cdm->pos.generations[CAM_BUS_GENERATION] !=
xsoftc.bus_generation) {
xpt_unlock_buses();
cdm->status = CAM_DEV_MATCH_LIST_CHANGED;
return(0);
}
bus = (struct cam_eb *)cdm->pos.cookie.bus;
bus->refcount++;
} else
bus = NULL;
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
ret = xptbustraverse(bus, xptedtbusfunc, cdm);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* If we get back 0, that means that we had to stop before fully
* traversing the EDT. It also means that one of the subroutines
* has set the status field to the proper value. If we get back 1,
* we've fully traversed the EDT and copied out any matching entries.
*/
if (ret == 1)
cdm->status = CAM_DEV_MATCH_LAST;
return(ret);
}
static int
xptplistpdrvfunc(struct periph_driver **pdrv, void *arg)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_periph *periph;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct ccb_dev_match *cdm;
cdm = (struct ccb_dev_match *)arg;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_lock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((cdm->pos.position_type & CAM_DEV_POS_PDPTR)
&& (cdm->pos.cookie.pdrv == pdrv)
&& (cdm->pos.position_type & CAM_DEV_POS_PERIPH)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
&& (cdm->pos.cookie.periph != NULL)) {
if (cdm->pos.generations[CAM_PERIPH_GENERATION] !=
(*pdrv)->generation) {
xpt_unlock_buses();
cdm->status = CAM_DEV_MATCH_LIST_CHANGED;
return(0);
}
periph = (struct cam_periph *)cdm->pos.cookie.periph;
periph->refcount++;
} else
periph = NULL;
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
return (xptpdperiphtraverse(pdrv, periph, xptplistperiphfunc, arg));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static int
xptplistperiphfunc(struct cam_periph *periph, void *arg)
{
struct ccb_dev_match *cdm;
dev_match_ret retval;
cdm = (struct ccb_dev_match *)arg;
retval = xptperiphmatch(cdm->patterns, cdm->num_patterns, periph);
if ((retval & DM_RET_ACTION_MASK) == DM_RET_ERROR) {
cdm->status = CAM_DEV_MATCH_ERROR;
return(0);
}
/*
* If the copy flag is set, copy this peripheral out.
*/
if (retval & DM_RET_COPY) {
int spaceleft, j;
spaceleft = cdm->match_buf_len - (cdm->num_matches *
sizeof(struct dev_match_result));
/*
* If we don't have enough space to put in another
* match result, save our position and tell the
* user there are more devices to check.
*/
if (spaceleft < sizeof(struct dev_match_result)) {
struct periph_driver **pdrv;
pdrv = NULL;
bzero(&cdm->pos, sizeof(cdm->pos));
style(9)
2008-01-02 01:45:31 +00:00
cdm->pos.position_type =
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEV_POS_PDRV | CAM_DEV_POS_PDPTR |
CAM_DEV_POS_PERIPH;
/*
* This may look a bit non-sensical, but it is
* actually quite logical. There are very few
* peripheral drivers, and bloating every peripheral
* structure with a pointer back to its parent
* peripheral driver linker set entry would cost
* more in the long run than doing this quick lookup.
*/
Change the peripheral driver list from a linker set to module driven driver registration. This should allow things like da, sa, cd etc to be in seperate KLD's to the cam core and make them preloadable.
2001-02-07 07:05:59 +00:00
for (pdrv = periph_drivers; *pdrv != NULL; pdrv++) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (strcmp((*pdrv)->driver_name,
periph->periph_name) == 0)
break;
}
Check the correct periph pointer after search for it. Submitted by: Coverity Prevent analysis tool
2005-02-20 23:51:20 +00:00
if (*pdrv == NULL) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cdm->status = CAM_DEV_MATCH_ERROR;
return(0);
}
cdm->pos.cookie.pdrv = pdrv;
/*
* The periph generation slot does double duty, as
* does the periph pointer slot. They are used for
* both edt and pdrv lookups and positioning.
*/
cdm->pos.cookie.periph = periph;
cdm->pos.generations[CAM_PERIPH_GENERATION] =
(*pdrv)->generation;
cdm->status = CAM_DEV_MATCH_MORE;
return(0);
}
j = cdm->num_matches;
cdm->num_matches++;
cdm->matches[j].type = DEV_MATCH_PERIPH;
cdm->matches[j].result.periph_result.path_id =
periph->path->bus->path_id;
/*
* The transport layer peripheral doesn't have a target or
* lun.
*/
if (periph->path->target)
cdm->matches[j].result.periph_result.target_id =
periph->path->target->target_id;
else
Replace several instances of -1 with appropriate CAM_*_WILDCARD and types. It was equal before r259397, but for good or bad, not any more for LUNs. This change fixes at least CAM debugging.
2014-01-10 12:18:05 +00:00
cdm->matches[j].result.periph_result.target_id =
CAM_TARGET_WILDCARD;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (periph->path->device)
cdm->matches[j].result.periph_result.target_lun =
periph->path->device->lun_id;
else
Replace several instances of -1 with appropriate CAM_*_WILDCARD and types. It was equal before r259397, but for good or bad, not any more for LUNs. This change fixes at least CAM debugging.
2014-01-10 12:18:05 +00:00
cdm->matches[j].result.periph_result.target_lun =
CAM_LUN_WILDCARD;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cdm->matches[j].result.periph_result.unit_number =
periph->unit_number;
strncpy(cdm->matches[j].result.periph_result.periph_name,
periph->periph_name, DEV_IDLEN);
}
return(1);
}
static int
xptperiphlistmatch(struct ccb_dev_match *cdm)
{
int ret;
cdm->num_matches = 0;
/*
* At this point in the edt traversal function, we check the bus
* list generation to make sure that no busses have been added or
* removed since the user last sent a XPT_DEV_MATCH ccb through.
* For the peripheral driver list traversal function, however, we
* don't have to worry about new peripheral driver types coming or
* going; they're in a linker set, and therefore can't change
* without a recompile.
*/
if ((cdm->pos.position_type & CAM_DEV_POS_PDPTR)
&& (cdm->pos.cookie.pdrv != NULL))
ret = xptpdrvtraverse(
cam_periph.c: Move handling of CAM_AUTOSENSE_FAIL into block dealing with all other scsi status errors. cam_queue.c: cam_queue.h: Fix 'off by one' heap bug in a more efficient manner. Since heap algorithms like to deal with indexes started from 1, offset our heap array pointer at allocation time to make this so for a C environment. This makes the implementation of the algorithm a bit more efficient. cam_xpt.c: Use macros for accessing the head of the heap so that code is isolated from implementation details of the heap.
1999-04-19 21:26:08 +00:00
(struct periph_driver **)cdm->pos.cookie.pdrv,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xptplistpdrvfunc, cdm);
else
ret = xptpdrvtraverse(NULL, xptplistpdrvfunc, cdm);
/*
* If we get back 0, that means that we had to stop before fully
* traversing the peripheral driver tree. It also means that one of
* the subroutines has set the status field to the proper value. If
* we get back 1, we've fully traversed the EDT and copied out any
* matching entries.
*/
if (ret == 1)
cdm->status = CAM_DEV_MATCH_LAST;
return(ret);
}
static int
xptbustraverse(struct cam_eb *start_bus, xpt_busfunc_t *tr_func, void *arg)
{
struct cam_eb *bus, *next_bus;
int retval;
retval = 1;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (start_bus)
bus = start_bus;
else {
xpt_lock_buses();
bus = TAILQ_FIRST(&xsoftc.xpt_busses);
if (bus == NULL) {
xpt_unlock_buses();
return (retval);
}
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
bus->refcount++;
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
for (; bus != NULL; bus = next_bus) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = tr_func(bus, arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (retval == 0) {
xpt_release_bus(bus);
break;
}
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_lock_buses();
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
next_bus = TAILQ_NEXT(bus, links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (next_bus)
next_bus->refcount++;
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
xpt_release_bus(bus);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return(retval);
}
static int
xpttargettraverse(struct cam_eb *bus, struct cam_et *start_target,
xpt_targetfunc_t *tr_func, void *arg)
{
struct cam_et *target, *next_target;
int retval;
retval = 1;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (start_target)
target = start_target;
else {
mtx_lock(&bus->eb_mtx);
target = TAILQ_FIRST(&bus->et_entries);
if (target == NULL) {
mtx_unlock(&bus->eb_mtx);
return (retval);
}
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
target->refcount++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&bus->eb_mtx);
}
for (; target != NULL; target = next_target) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = tr_func(target, arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (retval == 0) {
xpt_release_target(target);
break;
}
mtx_lock(&bus->eb_mtx);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
next_target = TAILQ_NEXT(target, links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (next_target)
next_target->refcount++;
mtx_unlock(&bus->eb_mtx);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
xpt_release_target(target);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return(retval);
}
static int
xptdevicetraverse(struct cam_et *target, struct cam_ed *start_device,
xpt_devicefunc_t *tr_func, void *arg)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_ed *device, *next_device;
int retval;
retval = 1;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
bus = target->bus;
if (start_device)
device = start_device;
else {
mtx_lock(&bus->eb_mtx);
device = TAILQ_FIRST(&target->ed_entries);
if (device == NULL) {
mtx_unlock(&bus->eb_mtx);
return (retval);
}
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
device->refcount++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&bus->eb_mtx);
}
for (; device != NULL; device = next_device) {
mtx_lock(&device->device_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = tr_func(device, arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&device->device_mtx);
if (retval == 0) {
xpt_release_device(device);
break;
}
mtx_lock(&bus->eb_mtx);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
next_device = TAILQ_NEXT(device, links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (next_device)
next_device->refcount++;
mtx_unlock(&bus->eb_mtx);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
xpt_release_device(device);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return(retval);
}
static int
xptperiphtraverse(struct cam_ed *device, struct cam_periph *start_periph,
xpt_periphfunc_t *tr_func, void *arg)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_periph *periph, *next_periph;
int retval;
retval = 1;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
bus = device->target->bus;
if (start_periph)
periph = start_periph;
else {
xpt_lock_buses();
mtx_lock(&bus->eb_mtx);
periph = SLIST_FIRST(&device->periphs);
while (periph != NULL && (periph->flags & CAM_PERIPH_FREE) != 0)
periph = SLIST_NEXT(periph, periph_links);
if (periph == NULL) {
mtx_unlock(&bus->eb_mtx);
xpt_unlock_buses();
return (retval);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
}
periph->refcount++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&bus->eb_mtx);
xpt_unlock_buses();
}
for (; periph != NULL; periph = next_periph) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = tr_func(periph, arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (retval == 0) {
Fix lock recursion, triggered by `smartctl -a /dev/adaX`.
2013-11-01 00:14:15 +00:00
cam_periph_release_locked(periph);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
break;
}
xpt_lock_buses();
mtx_lock(&bus->eb_mtx);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
next_periph = SLIST_NEXT(periph, periph_links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
while (next_periph != NULL &&
(next_periph->flags & CAM_PERIPH_FREE) != 0)
Use proper variable when looping through periphs with CAM_PERIPH_FREE. PR: 194256 Submitted by: Scott M. Ferris <smferris@gmail.com> MFC after: 3 days Sponsored by: EMC/Isilon Storage Division
2014-10-09 05:53:58 +00:00
next_periph = SLIST_NEXT(next_periph, periph_links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (next_periph)
next_periph->refcount++;
mtx_unlock(&bus->eb_mtx);
xpt_unlock_buses();
cam_periph_release_locked(periph);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return(retval);
}
static int
xptpdrvtraverse(struct periph_driver **start_pdrv,
xpt_pdrvfunc_t *tr_func, void *arg)
{
struct periph_driver **pdrv;
int retval;
retval = 1;
/*
* We don't traverse the peripheral driver list like we do the
* other lists, because it is a linker set, and therefore cannot be
* changed during runtime. If the peripheral driver list is ever
* re-done to be something other than a linker set (i.e. it can
* change while the system is running), the list traversal should
* be modified to work like the other traversal functions.
*/
Change the peripheral driver list from a linker set to module driven driver registration. This should allow things like da, sa, cd etc to be in seperate KLD's to the cam core and make them preloadable.
2001-02-07 07:05:59 +00:00
for (pdrv = (start_pdrv ? start_pdrv : periph_drivers);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*pdrv != NULL; pdrv++) {
retval = tr_func(pdrv, arg);
if (retval == 0)
return(retval);
}
return(retval);
}
static int
xptpdperiphtraverse(struct periph_driver **pdrv,
struct cam_periph *start_periph,
xpt_periphfunc_t *tr_func, void *arg)
{
struct cam_periph *periph, *next_periph;
int retval;
retval = 1;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (start_periph)
periph = start_periph;
else {
xpt_lock_buses();
periph = TAILQ_FIRST(&(*pdrv)->units);
while (periph != NULL && (periph->flags & CAM_PERIPH_FREE) != 0)
periph = TAILQ_NEXT(periph, unit_links);
if (periph == NULL) {
xpt_unlock_buses();
return (retval);
Protect periph drivers list and rearrange things to minimize the chance of stepping oneself during probing. Don't blindly decrement a periph probe count. Reviewed by: scsi@ Obtained from: Alexander Motin, Atillio Rao, Others MFC after: 1 month
2010-06-02 18:06:32 +00:00
}
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
periph->refcount++;
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
xpt_unlock_buses();
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
for (; periph != NULL; periph = next_periph) {
cam_periph_lock(periph);
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
retval = tr_func(periph, arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_periph_unlock(periph);
if (retval == 0) {
cam_periph_release(periph);
break;
}
xpt_lock_buses();
Fix a race condition in CAM peripheral free handling, locking in the CAM XPT bus traversal code, and a number of other periph level issues. cam_periph.h, cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID flag prior to allowing a reference count to be gained on a peripheral. Callers of this function will receive CAM_REQ_CMP_ERR status in the situation of attempting to reference an invalidated periph. This guarantees that a peripheral scheduled for a deferred free will not be accessed during its wait for destruction. Panic during attempts to drop a reference count on a peripheral that already has a zero reference count. In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN set so that mallocs do not occur while the xpt topology lock is held, regardless of the allocation policy of the passed in sbuf. Add a new routine, cam_periph_release_locked_buses(), that can be called when the caller already holds the CAM topology lock. Add some extra debugging for duplicate peripheral allocations in cam_periph_alloc(). Treat CAM_DEV_NOT_THERE much the same as a selection timeout (AC_LOST_DEVICE is emitted), but forgo retries. cam_xpt.c: Revamp the way the EDT traversal code does locking and reference counting. This was broken, since it assumed that the EDT would not change during traversal, but that assumption is no longer valid. So, to prevent devices from going away while we traverse the EDT, make sure we properly lock everything and hold references on devices that we are using. The two peripheral driver traversal routines should be examined. xptpdperiphtraverse() holds the topology lock for the entire time it runs. xptperiphtraverse() is now locked properly, but only holds the topology lock while it is traversing the list, and not while the traversal function is running. The bus locking code in xptbustraverse() should also be revisited at a later time, since it is complex and should probably be simplified. scsi_da.c: Pay attention to the return value from cam_periph_acquire(). Return 0 always from daclose() even if the disk is now gone. Add some rudimentary error injection support. scsi_sg.c: Fix reference counting in the sg(4) driver. The sg driver was calling cam_periph_release() on close, but never called cam_periph_acquire() (which increments the reference count) on open. The periph code correctly complained that the sg(4) driver was trying to decrement the refcount when it was already 0. Sponsored by: Spectra Logic MFC after: 2 weeks
2012-01-12 00:41:48 +00:00
next_periph = TAILQ_NEXT(periph, unit_links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
while (next_periph != NULL &&
(next_periph->flags & CAM_PERIPH_FREE) != 0)
Use proper variable when looping through periphs with CAM_PERIPH_FREE. PR: 194256 Submitted by: Scott M. Ferris <smferris@gmail.com> MFC after: 3 days Sponsored by: EMC/Isilon Storage Division
2014-10-09 05:53:58 +00:00
next_periph = TAILQ_NEXT(next_periph, unit_links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (next_periph)
next_periph->refcount++;
xpt_unlock_buses();
cam_periph_release(periph);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return(retval);
}
static int
xptdefbusfunc(struct cam_eb *bus, void *arg)
{
struct xpt_traverse_config *tr_config;
tr_config = (struct xpt_traverse_config *)arg;
if (tr_config->depth == XPT_DEPTH_BUS) {
xpt_busfunc_t *tr_func;
tr_func = (xpt_busfunc_t *)tr_config->tr_func;
return(tr_func(bus, tr_config->tr_arg));
} else
return(xpttargettraverse(bus, NULL, xptdeftargetfunc, arg));
}
static int
xptdeftargetfunc(struct cam_et *target, void *arg)
{
struct xpt_traverse_config *tr_config;
tr_config = (struct xpt_traverse_config *)arg;
if (tr_config->depth == XPT_DEPTH_TARGET) {
xpt_targetfunc_t *tr_func;
tr_func = (xpt_targetfunc_t *)tr_config->tr_func;
return(tr_func(target, tr_config->tr_arg));
} else
return(xptdevicetraverse(target, NULL, xptdefdevicefunc, arg));
}
static int
xptdefdevicefunc(struct cam_ed *device, void *arg)
{
struct xpt_traverse_config *tr_config;
tr_config = (struct xpt_traverse_config *)arg;
if (tr_config->depth == XPT_DEPTH_DEVICE) {
xpt_devicefunc_t *tr_func;
tr_func = (xpt_devicefunc_t *)tr_config->tr_func;
return(tr_func(device, tr_config->tr_arg));
} else
return(xptperiphtraverse(device, NULL, xptdefperiphfunc, arg));
}
static int
xptdefperiphfunc(struct cam_periph *periph, void *arg)
{
struct xpt_traverse_config *tr_config;
xpt_periphfunc_t *tr_func;
tr_config = (struct xpt_traverse_config *)arg;
tr_func = (xpt_periphfunc_t *)tr_config->tr_func;
/*
* Unlike the other default functions, we don't check for depth
* here. The peripheral driver level is the last level in the EDT,
* so if we're here, we should execute the function in question.
*/
return(tr_func(periph, tr_config->tr_arg));
}
/*
* Execute the given function for every bus in the EDT.
*/
static int
xpt_for_all_busses(xpt_busfunc_t *tr_func, void *arg)
{
struct xpt_traverse_config tr_config;
tr_config.depth = XPT_DEPTH_BUS;
tr_config.tr_func = tr_func;
tr_config.tr_arg = arg;
return(xptbustraverse(NULL, xptdefbusfunc, &tr_config));
}
/*
* Execute the given function for every device in the EDT.
*/
static int
xpt_for_all_devices(xpt_devicefunc_t *tr_func, void *arg)
{
struct xpt_traverse_config tr_config;
tr_config.depth = XPT_DEPTH_DEVICE;
tr_config.tr_func = tr_func;
tr_config.tr_arg = arg;
return(xptbustraverse(NULL, xptdefbusfunc, &tr_config));
}
static int
xptsetasyncfunc(struct cam_ed *device, void *arg)
{
struct cam_path path;
struct ccb_getdev cgd;
Change the way in which fake async events generated. Do not use taskqueue for lock decoupling, as it causes unwanted races.
2010-02-02 18:07:16 +00:00
struct ccb_setasync *csa = (struct ccb_setasync *)arg;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
/*
* Don't report unconfigured devices (Wildcard devs,
* devices only for target mode, device instances
* that have been invalidated but are waiting for
* their last reference count to be released).
*/
if ((device->flags & CAM_DEV_UNCONFIGURED) != 0)
return (1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_compile_path(&path,
NULL,
device->target->bus->path_id,
device->target->target_id,
device->lun_id);
Replace most of priority numbers with defines. No logical changes.
2009-10-23 08:27:55 +00:00
xpt_setup_ccb(&cgd.ccb_h, &path, CAM_PRIORITY_NORMAL);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cgd.ccb_h.func_code = XPT_GDEV_TYPE;
xpt_action((union ccb *)&cgd);
Change the way in which fake async events generated. Do not use taskqueue for lock decoupling, as it causes unwanted races.
2010-02-02 18:07:16 +00:00
csa->callback(csa->callback_arg,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
AC_FOUND_DEVICE,
&path, &cgd);
xpt_release_path(&path);
return(1);
}
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static int
xptsetasyncbusfunc(struct cam_eb *bus, void *arg)
{
struct cam_path path;
struct ccb_pathinq cpi;
Change the way in which fake async events generated. Do not use taskqueue for lock decoupling, as it causes unwanted races.
2010-02-02 18:07:16 +00:00
struct ccb_setasync *csa = (struct ccb_setasync *)arg;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_compile_path(&path, /*periph*/NULL,
Close the race on path ID allocation in xpt_bus_register() if two buses are registered simultaneously. Due to topology unlock between the ID allocation and the bus registration there is a chance that two buses may get the same IDs. That is supposed reason of lock assertion panic in CAM during initial bus scanning after new iscsid initiates two sessions same time. Reported by: trasz Approved by: re (glebus, marius) MFC after: 2 weeks
2013-10-09 12:09:01 +00:00
bus->path_id,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_TARGET_WILDCARD,
CAM_LUN_WILDCARD);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_lock(&path);
Replace most of priority numbers with defines. No logical changes.
2009-10-23 08:27:55 +00:00
xpt_setup_ccb(&cpi.ccb_h, &path, CAM_PRIORITY_NORMAL);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cpi.ccb_h.func_code = XPT_PATH_INQ;
xpt_action((union ccb *)&cpi);
Change the way in which fake async events generated. Do not use taskqueue for lock decoupling, as it causes unwanted races.
2010-02-02 18:07:16 +00:00
csa->callback(csa->callback_arg,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
AC_PATH_REGISTERED,
&path, &cpi);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_unlock(&path);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_release_path(&path);
return(1);
}
void
xpt_action(union ccb *start_ccb)
{
Run xpt_action at splsoftcam() so that it cannot be re-entered by our SWI. The XPT doesn't have a problem with this itself, but some controllers drivers may have been caught off guard by the old behavior. XPT_CONT_TARGET_IO is also a valid ccb type for cam_periph_unmapmem.
1999-05-25 20:17:29 +00:00
Improve cam tracing a little by including the function code in the traces for xpt_action. Note up-calls (down-calls?) to the SIM as well. Differential Review: https://reviews.freebsd.org/D4382
2015-12-07 04:14:29 +00:00
CAM_DEBUG(start_ccb->ccb_h.path, CAM_DEBUG_TRACE,
("xpt_action: func=%#x\n", start_ccb->ccb_h.func_code));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_REQ_INPROG;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
(*(start_ccb->ccb_h.path->bus->xport->action))(start_ccb);
}
void
xpt_action_default(union ccb *start_ccb)
{
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
struct cam_path *path;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_sim *sim;
int lock;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
path = start_ccb->ccb_h.path;
Improve cam tracing a little by including the function code in the traces for xpt_action. Note up-calls (down-calls?) to the SIM as well. Differential Review: https://reviews.freebsd.org/D4382
2015-12-07 04:14:29 +00:00
CAM_DEBUG(path, CAM_DEBUG_TRACE,
("xpt_action_default: func=%#x\n", start_ccb->ccb_h.func_code));
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
switch (start_ccb->ccb_h.func_code) {
case XPT_SCSI_IO:
Add a new CAM debugging mode, CAM_DEBUG_CDB. This causes the kernel to print out a one line description/dump of every SCSI CDB sent to a particular debugging target or targets. This is a good bit more useful than the other debugging modes, I think. Change some things in LINT to note the availability of this new option. Fix an erroneous argument to scsi_cdb_string() in scsi_all.c Reviewed by: gibbs
1998-10-02 21:00:58 +00:00
{
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
struct cam_ed *device;
Add a new CAM debugging mode, CAM_DEBUG_CDB. This causes the kernel to print out a one line description/dump of every SCSI CDB sent to a particular debugging target or targets. This is a good bit more useful than the other debugging modes, I think. Change some things in LINT to note the availability of this new option. Fix an erroneous argument to scsi_cdb_string() in scsi_all.c Reviewed by: gibbs
1998-10-02 21:00:58 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* For the sake of compatibility with SCSI-1
* devices that may not understand the identify
* message, we include lun information in the
* second byte of all commands. SCSI-1 specifies
* that luns are a 3 bit value and reserves only 3
* bits for lun information in the CDB. Later
* revisions of the SCSI spec allow for more than 8
* luns, but have deprecated lun information in the
* CDB. So, if the lun won't fit, we must omit.
*
* Also be aware that during initial probing for devices,
* the inquiry information is unknown but initialized to 0.
* This means that this code will be exercised while probing
* devices with an ANSI revision greater than 2.
*/
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
device = path->device;
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
if (device->protocol_version <= SCSI_REV_2
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
&& start_ccb->ccb_h.target_lun < 8
&& (start_ccb->ccb_h.flags & CAM_CDB_POINTER) == 0) {
start_ccb->csio.cdb_io.cdb_bytes[1] |=
start_ccb->ccb_h.target_lun << 5;
}
start_ccb->csio.scsi_status = SCSI_STATUS_OK;
Add a new CAM debugging mode, CAM_DEBUG_CDB. This causes the kernel to print out a one line description/dump of every SCSI CDB sent to a particular debugging target or targets. This is a good bit more useful than the other debugging modes, I think. Change some things in LINT to note the availability of this new option. Fix an erroneous argument to scsi_cdb_string() in scsi_all.c Reviewed by: gibbs
1998-10-02 21:00:58 +00:00
}
Add /* FALLTHROUGH */ Move /* FALLTHROUGH */ to correct location. Remove unused variable(s). Found by: FlexeLint
2003-05-31 20:46:21 +00:00
/* FALLTHROUGH */
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_TARGET_IO:
case XPT_CONT_TARGET_IO:
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
start_ccb->csio.sense_resid = 0;
start_ccb->csio.resid = 0;
/* FALLTHROUGH */
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
case XPT_ATA_IO:
MFprojects/camlock r248928: Move CAM_DEBUG_CDB messages from the point of queuing to the point of sending to SIM. That allows to inspect real requests execution order, respecting priorities, freezing, etc. MFC after: 2 weeks
2013-04-04 18:59:29 +00:00
if (start_ccb->ccb_h.func_code == XPT_ATA_IO)
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
start_ccb->ataio.resid = 0;
For unhandled actions in xpt_action_default, remember to call xpt_done for queued requests. This solves the problem of indefinite hangs for unspecified transports when XPT_SCAN_BUS is called. A few minor cosmetics elsewhere. MFC after: 1 week
2010-03-31 17:47:57 +00:00
/* FALLTHROUGH */
Fix Bus Device Resets from userland, and specifically from camcontrol. camcontrol was setup to use the old scheme of going through the xpt(4) device, which never worked properly (and has been disabled for a while). camcontrol now sends BDRs through the pass(4) device, and XPT_RESET_DEV CCBs are put on the device queue in the transport layer, as they should be. Submitted by: luoqi Reviewed by: ken
1999-10-31 02:39:44 +00:00
case XPT_RESET_DEV:
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_ENG_EXEC:
Add Serial Management Protocol (SMP) passthrough support to CAM. This includes support in the kernel, camcontrol(8), libcam and the mps(4) driver for SMP passthrough. The CAM SCSI probe code has been modified to fetch Inquiry VPD page 0x00 to determine supported pages, and will now fetch page 0x83 in addition to page 0x80 if supported. Add two new CAM CCBs, XPT_SMP_IO, and XPT_GDEV_ADVINFO. The SMP CCB is intended for SMP requests and responses. The ADVINFO is currently used to fetch cached VPD page 0x83 data from the transport layer, but is intended to be extensible to fetch other types of device-specific data. SMP-only devices are not currently represented in the CAM topology, and so the current semantics are that the SIM will route SMP CCBs to either the addressed device, if it contains an SMP target, or its parent, if it contains an SMP target. (This is noted in cam_ccb.h, since it will change later once we have the ability to have SMP-only devices in CAM's topology.) smp_all.c, smp_all.h: New helper routines for SMP. This includes SMP request building routines, response parsing routines, error decoding routines, and structure definitions for a number of SMP commands. libcam/Makefile: Add smp_all.c to libcam, so that SMP functionality is available to userland applications. camcontrol.8, camcontrol.c: Add smp passthrough support to camcontrol. Several new subcommands are now available: 'smpcmd' functions much like 'cmd', except that it allows the user to send generic SMP commands. 'smprg' sends the SMP report general command, and displays the decoded output. It will automatically fetch extended output if it is available. 'smppc' sends the SMP phy control command, with any number of potential options. Among other things, this allows the user to reset a phy on a SAS expander, or disable a phy on an expander. 'smpmaninfo' sends the SMP report manufacturer information and displays the decoded output. 'smpphylist' displays a list of phys on an expander, and the CAM devices attached to those phys, if any. cam.h, cam.c: Add a status value for SMP errors (CAM_SMP_STATUS_ERROR). Add a missing description for CAM_SCSI_IT_NEXUS_LOST. Add support for SMP commands to cam_error_string(). cam_ccb.h: Rename the CAM_DIR_RESV flag to CAM_DIR_BOTH. SMP commands are by nature bi-directional, and we may need to support bi-directional SCSI commands later. Add the XPT_SMP_IO CCB. Since SMP commands are bi-directional, there are pointers for both the request and response. Add a fill routine for SMP CCBs. Add the XPT_GDEV_ADVINFO CCB. This is currently used to fetch cached page 0x83 data from the transport later, but is extensible to fetch many other types of data. cam_periph.c: Add support in cam_periph_mapmem() for XPT_SMP_IO and XPT_GDEV_ADVINFO CCBs. cam_xpt.c: Add support for executing XPT_SMP_IO CCBs. cam_xpt_internal.h: Add fields for VPD pages 0x00 and 0x83 in struct cam_ed. scsi_all.c: Add scsi_get_sas_addr(), a function that parses VPD page 0x83 data and pulls out a SAS address. scsi_all.h: Add VPD page 0x00 and 0x83 structures, and a prototype for scsi_get_sas_addr(). scsi_pass.c: Add support for mapping buffers in XPT_SMP_IO and XPT_GDEV_ADVINFO CCBs. scsi_xpt.c: In the SCSI probe code, first ask the device for VPD page 0x00. If any VPD pages are supported, that page is required to be implemented. Based on the response, we may probe for the serial number (page 0x80) or device id (page 0x83). Add support for the XPT_GDEV_ADVINFO CCB. sys/conf/files: Add smp_all.c. mps.c: Add support for passing in a uio in mps_map_command(), so we can map a S/G list at once. Add support for SMP passthrough commands in mps_data_cb(). SMP is a special case, because the first buffer in the S/G list is outbound and the second buffer is inbound. Add support for warning the user if the busdma code comes back with more buffers than will work for the command. This will, for example, help the user determine why an SMP command failed if busdma comes back with three buffers. mps_pci.c: Add sys/uio.h. mps_sas.c: Add the SAS address and the parent handle to the list of fields we pull from device page 0 and cache in struct mpssas_target. These are needed for SMP passthrough. Add support for the XPT_SMP_IO CCB. For now, this CCB is routed to the addressed device if it supports SMP, or to its parent if it does not and the parent does. This is necessary because CAM does not currently support SMP-only nodes in the topology. Make SMP passthrough support conditional on __FreeBSD_version >= 900026. This will make it easier to MFC this change to the driver without MFCing the CAM changes as well. mps_user.c: Un-staticize mpi_init_sge() so we can use it for the SMP passthrough code. mpsvar.h: Add a uio and iovecs into struct mps_command for SMP passthrough commands. Add a cm_max_segs field to struct mps_command so that we can warn the user if busdma comes back with too many segments. Clear the cm_reply when a command gets freed. If it is not cleared, reply frames will eventually get freed into the pool multiple times and corrupt the pool. (This fix is from scottl.) Add a prototype for mpi_init_sge(). sys/param.h: Bump __FreeBSD_version to 900026 for the for the inclusion of the XPT_GDEV_ADVINFO and XPT_SMP_IO CAM CCBs.
2010-11-30 22:39:46 +00:00
case XPT_SMP_IO:
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
{
struct cam_devq *devq;
devq = path->bus->sim->devq;
mtx_lock(&devq->send_mtx);
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
cam_ccbq_insert_ccb(&path->device->ccbq, start_ccb);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (xpt_schedule_devq(devq, path->device) != 0)
xpt_run_devq(devq);
mtx_unlock(&devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_CALC_GEOMETRY:
/* Filter out garbage */
if (start_ccb->ccg.block_size == 0
|| start_ccb->ccg.volume_size == 0) {
start_ccb->ccg.cylinders = 0;
start_ccb->ccg.heads = 0;
start_ccb->ccg.secs_per_track = 0;
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
For sparc64 also adjust the geometry of da(4) driven disks to not overflow the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The geometry chosen for disks above that limit allows to use disks up to 2TB, which is the limit of the extended VTOC8 format. The geometry used for disks smaller than the 16-bit cylinders limit stays the same as used by cam_calc_geometry(9) for extended translation. Thanks to Hans-Joerg Sirtl for providing hardware for testing this change. MFC after: 3 days
2011-11-27 15:43:40 +00:00
#if defined(PC98) || defined(__sparc64__)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* In a PC-98 system, geometry translation depens on
* the "real" device geometry obtained from mode page 4.
* SCSI geometry translation is performed in the
* initialization routine of the SCSI BIOS and the result
* stored in host memory. If the translation is available
* in host memory, use it. If not, rely on the default
* translation the device driver performs.
For sparc64 also adjust the geometry of da(4) driven disks to not overflow the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The geometry chosen for disks above that limit allows to use disks up to 2TB, which is the limit of the extended VTOC8 format. The geometry used for disks smaller than the 16-bit cylinders limit stays the same as used by cam_calc_geometry(9) for extended translation. Thanks to Hans-Joerg Sirtl for providing hardware for testing this change. MFC after: 3 days
2011-11-27 15:43:40 +00:00
* For sparc64, we may need adjust the geometry of large
* disks in order to fit the limitations of the 16-bit
* fields of the VTOC8 disk label.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
if (scsi_da_bios_params(&start_ccb->ccg) != 0) {
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
#endif
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
goto call_sim;
Wire up the XPT_ABORT and XPT_RESET_DEV ccb function codes so they can be delivered to controller drivers. Adjust for changes to the ccb_hdr list types in cam_queue.h
1998-12-15 08:13:10 +00:00
case XPT_ABORT:
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
{
union ccb* abort_ccb;
abort_ccb = start_ccb->cab.abort_ccb;
if (XPT_FC_IS_DEV_QUEUED(abort_ccb)) {
if (abort_ccb->ccb_h.pinfo.index >= 0) {
struct cam_ccbq *ccbq;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
struct cam_ed *device;
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
device = abort_ccb->ccb_h.path->device;
ccbq = &device->ccbq;
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
cam_ccbq_remove_ccb(ccbq, abort_ccb);
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
abort_ccb->ccb_h.status =
CAM_REQ_ABORTED|CAM_DEV_QFRZN;
xpt_freeze_devq(abort_ccb->ccb_h.path, 1);
xpt_done(abort_ccb);
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
if (abort_ccb->ccb_h.pinfo.index == CAM_UNQUEUED_INDEX
&& (abort_ccb->ccb_h.status & CAM_SIM_QUEUED) == 0) {
/*
* We've caught this ccb en route to
* the SIM. Flag it for abort and the
* SIM will do so just before starting
* real work on the CCB.
*/
abort_ccb->ccb_h.status =
CAM_REQ_ABORTED|CAM_DEV_QFRZN;
xpt_freeze_devq(abort_ccb->ccb_h.path, 1);
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
style(9)
2008-01-02 01:45:31 +00:00
}
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
if (XPT_FC_IS_QUEUED(abort_ccb)
&& (abort_ccb->ccb_h.pinfo.index == CAM_DONEQ_INDEX)) {
/*
* It's already completed but waiting
* for our SWI to get to it.
*/
start_ccb->ccb_h.status = CAM_UA_ABORT;
break;
}
/*
* If we weren't able to take care of the abort request
* in the XPT, pass the request down to the SIM for processing.
*/
}
Add /* FALLTHROUGH */ Move /* FALLTHROUGH */ to correct location. Remove unused variable(s). Found by: FlexeLint
2003-05-31 20:46:21 +00:00
/* FALLTHROUGH */
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_ACCEPT_TARGET_IO:
case XPT_EN_LUN:
case XPT_IMMED_NOTIFY:
case XPT_NOTIFY_ACK:
case XPT_RESET_BUS:
Add 8Gb support (isp_2500). Fix a fair number of configuration and firmware loading bugs. Target mode support has received some serious attention to make it more usable and stable. Some backward compatible additions to CAM have been made that make target mode async events easier to deal with have also been put into place. Further refinement and better support for NP-IV (N-port Virtualization) is now in place. Code for release prior to RELENG_7 has been stripped away for code clarity. Sponsored by: Copan Systems Reviewed by: scottl, ken, jung-uk kim Approved by: re
2009-08-01 01:04:26 +00:00
case XPT_IMMEDIATE_NOTIFY:
case XPT_NOTIFY_ACKNOWLEDGE:
Move to new value for XPT_GET_SIM_KNOB to avoid clash with XPT_ATA_IO.
2016-03-10 06:25:05 +00:00
case XPT_GET_SIM_KNOB_OLD:
Add 8Gb support (isp_2500). Fix a fair number of configuration and firmware loading bugs. Target mode support has received some serious attention to make it more usable and stable. Some backward compatible additions to CAM have been made that make target mode async events easier to deal with have also been put into place. Further refinement and better support for NP-IV (N-port Virtualization) is now in place. Code for release prior to RELENG_7 has been stripped away for code clarity. Sponsored by: Copan Systems Reviewed by: scottl, ken, jung-uk kim Approved by: re
2009-08-01 01:04:26 +00:00
case XPT_GET_SIM_KNOB:
case XPT_SET_SIM_KNOB:
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
case XPT_GET_TRAN_SETTINGS:
case XPT_SET_TRAN_SETTINGS:
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
case XPT_PATH_INQ:
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
call_sim:
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
sim = path->bus->sim;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
lock = (mtx_owned(sim->mtx) == 0);
if (lock)
CAM_SIM_LOCK(sim);
Improve cam tracing a little by including the function code in the traces for xpt_action. Note up-calls (down-calls?) to the SIM as well. Differential Review: https://reviews.freebsd.org/D4382
2015-12-07 04:14:29 +00:00
CAM_DEBUG(path, CAM_DEBUG_TRACE,
("sim->sim_action: func=%#x\n", start_ccb->ccb_h.func_code));
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
(*(sim->sim_action))(sim, start_ccb);
Improve cam tracing a little by including the function code in the traces for xpt_action. Note up-calls (down-calls?) to the SIM as well. Differential Review: https://reviews.freebsd.org/D4382
2015-12-07 04:14:29 +00:00
CAM_DEBUG(path, CAM_DEBUG_TRACE,
("sim->sim_action: status=%#x\n", start_ccb->ccb_h.status));
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (lock)
CAM_SIM_UNLOCK(sim);
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
break;
case XPT_PATH_STATS:
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
start_ccb->cpis.last_reset = path->bus->last_reset;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_GDEV_TYPE:
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
{
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
struct cam_ed *dev;
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
dev = path->device;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
if ((dev->flags & CAM_DEV_UNCONFIGURED) != 0) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_DEV_NOT_THERE;
} else {
struct ccb_getdev *cgd;
cgd = &start_ccb->cgd;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cgd->protocol = dev->protocol;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cgd->inq_data = dev->inq_data;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cgd->ident_data = dev->ident_data;
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
cgd->inq_flags = dev->inq_flags;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cgd->ccb_h.status = CAM_REQ_CMP;
cgd->serial_num_len = dev->serial_num_len;
if ((dev->serial_num_len > 0)
&& (dev->serial_num != NULL))
bcopy(dev->serial_num, cgd->serial_num,
dev->serial_num_len);
}
style(9)
2008-01-02 01:45:31 +00:00
break;
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
}
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
case XPT_GDEV_STATS:
{
struct cam_ed *dev;
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
dev = path->device;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
if ((dev->flags & CAM_DEV_UNCONFIGURED) != 0) {
start_ccb->ccb_h.status = CAM_DEV_NOT_THERE;
} else {
struct ccb_getdevstats *cgds;
struct cam_eb *bus;
struct cam_et *tar;
Update CAM CCB accounting for the new status quo. devq_openings counter lost its meaning after allocation queues has gone. held counter is still meaningful, but problematic to update due to separate locking of CCB allocation and queuing. To fix that replace devq_openings counter with allocated counter. held is now calculated on request as difference between number of allocated, queued and active CCBs. MFC after: 1 month
2014-09-14 11:59:49 +00:00
struct cam_devq *devq;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
cgds = &start_ccb->cgds;
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
bus = path->bus;
tar = path->target;
Update CAM CCB accounting for the new status quo. devq_openings counter lost its meaning after allocation queues has gone. held counter is still meaningful, but problematic to update due to separate locking of CCB allocation and queuing. To fix that replace devq_openings counter with allocated counter. held is now calculated on request as difference between number of allocated, queued and active CCBs. MFC after: 1 month
2014-09-14 11:59:49 +00:00
devq = bus->sim->devq;
mtx_lock(&devq->send_mtx);
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
cgds->dev_openings = dev->ccbq.dev_openings;
cgds->dev_active = dev->ccbq.dev_active;
Update CAM CCB accounting for the new status quo. devq_openings counter lost its meaning after allocation queues has gone. held counter is still meaningful, but problematic to update due to separate locking of CCB allocation and queuing. To fix that replace devq_openings counter with allocated counter. held is now calculated on request as difference between number of allocated, queued and active CCBs. MFC after: 1 month
2014-09-14 11:59:49 +00:00
cgds->allocated = dev->ccbq.allocated;
cgds->queued = cam_ccbq_pending_ccb_count(&dev->ccbq);
cgds->held = cgds->allocated - cgds->dev_active -
cgds->queued;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
cgds->last_reset = tar->last_reset;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cgds->maxtags = dev->maxtags;
cgds->mintags = dev->mintags;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
if (timevalcmp(&tar->last_reset, &bus->last_reset, <))
cgds->last_reset = bus->last_reset;
Update CAM CCB accounting for the new status quo. devq_openings counter lost its meaning after allocation queues has gone. held counter is still meaningful, but problematic to update due to separate locking of CCB allocation and queuing. To fix that replace devq_openings counter with allocated counter. held is now calculated on request as difference between number of allocated, queued and active CCBs. MFC after: 1 month
2014-09-14 11:59:49 +00:00
mtx_unlock(&devq->send_mtx);
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
cgds->ccb_h.status = CAM_REQ_CMP;
}
break;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
case XPT_GDEVLIST:
{
struct cam_periph *nperiph;
struct periph_list *periph_head;
struct ccb_getdevlist *cgdl;
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
u_int i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_ed *device;
int found;
found = 0;
/*
* Don't want anyone mucking with our data.
*/
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
device = path->device;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
periph_head = &device->periphs;
cgdl = &start_ccb->cgdl;
/*
* Check and see if the list has changed since the user
* last requested a list member. If so, tell them that the
style(9)
2008-01-02 01:45:31 +00:00
* list has changed, and therefore they need to start over
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* from the beginning.
*/
style(9)
2008-01-02 01:45:31 +00:00
if ((cgdl->index != 0) &&
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
(cgdl->generation != device->generation)) {
cgdl->status = CAM_GDEVLIST_LIST_CHANGED;
break;
}
/*
style(9)
2008-01-02 01:45:31 +00:00
* Traverse the list of peripherals and attempt to find
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* the requested peripheral.
*/
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
for (nperiph = SLIST_FIRST(periph_head), i = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
(nperiph != NULL) && (i <= cgdl->index);
Mechanical change to use <sys/queue.h> macro API instead of fondling implementation details. Created with: sed(1) Reviewed by: md5(1)
2001-02-04 13:13:25 +00:00
nperiph = SLIST_NEXT(nperiph, periph_links), i++) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (i == cgdl->index) {
strncpy(cgdl->periph_name,
nperiph->periph_name,
DEV_IDLEN);
cgdl->unit_number = nperiph->unit_number;
found = 1;
}
}
if (found == 0) {
cgdl->status = CAM_GDEVLIST_ERROR;
break;
}
if (nperiph == NULL)
cgdl->status = CAM_GDEVLIST_LAST_DEVICE;
else
cgdl->status = CAM_GDEVLIST_MORE_DEVS;
cgdl->index++;
cgdl->generation = device->generation;
cgdl->ccb_h.status = CAM_REQ_CMP;
break;
}
case XPT_DEV_MATCH:
{
dev_pos_type position_type;
struct ccb_dev_match *cdm;
cdm = &start_ccb->cdm;
/*
* There are two ways of getting at information in the EDT.
* The first way is via the primary EDT tree. It starts
* with a list of busses, then a list of targets on a bus,
* then devices/luns on a target, and then peripherals on a
* device/lun. The "other" way is by the peripheral driver
* lists. The peripheral driver lists are organized by
* peripheral driver. (obviously) So it makes sense to
* use the peripheral driver list if the user is looking
* for something like "da1", or all "da" devices. If the
* user is looking for something on a particular bus/target
* or lun, it's generally better to go through the EDT tree.
*/
if (cdm->pos.position_type != CAM_DEV_POS_NONE)
position_type = cdm->pos.position_type;
else {
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
u_int i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
position_type = CAM_DEV_POS_NONE;
for (i = 0; i < cdm->num_patterns; i++) {
if ((cdm->patterns[i].type == DEV_MATCH_BUS)
||(cdm->patterns[i].type == DEV_MATCH_DEVICE)){
position_type = CAM_DEV_POS_EDT;
break;
}
}
if (cdm->num_patterns == 0)
position_type = CAM_DEV_POS_EDT;
else if (position_type == CAM_DEV_POS_NONE)
position_type = CAM_DEV_POS_PDRV;
}
switch(position_type & CAM_DEV_POS_TYPEMASK) {
case CAM_DEV_POS_EDT:
Add /* FALLTHROUGH */ Move /* FALLTHROUGH */ to correct location. Remove unused variable(s). Found by: FlexeLint
2003-05-31 20:46:21 +00:00
xptedtmatch(cdm);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
case CAM_DEV_POS_PDRV:
Add /* FALLTHROUGH */ Move /* FALLTHROUGH */ to correct location. Remove unused variable(s). Found by: FlexeLint
2003-05-31 20:46:21 +00:00
xptperiphlistmatch(cdm);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
default:
cdm->status = CAM_DEV_MATCH_ERROR;
break;
}
if (cdm->status == CAM_DEV_MATCH_ERROR)
start_ccb->ccb_h.status = CAM_REQ_CMP_ERR;
else
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
case XPT_SASYNC_CB:
{
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
struct ccb_setasync *csa;
struct async_node *cur_entry;
struct async_list *async_head;
u_int32_t added;
csa = &start_ccb->csa;
added = csa->event_enable;
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
async_head = &path->device->asyncs;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
* If there is already an entry for us, simply
* update it.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
cur_entry = SLIST_FIRST(async_head);
while (cur_entry != NULL) {
if ((cur_entry->callback_arg == csa->callback_arg)
&& (cur_entry->callback == csa->callback))
break;
cur_entry = SLIST_NEXT(cur_entry, links);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
if (cur_entry != NULL) {
/*
* If the request has no flags set,
* remove the entry.
*/
added &= ~cur_entry->event_enable;
if (csa->event_enable == 0) {
SLIST_REMOVE(async_head, cur_entry,
async_node, links);
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
xpt_release_device(path->device);
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
free(cur_entry, M_CAMXPT);
} else {
cur_entry->event_enable = csa->event_enable;
}
Change the way in which fake async events generated. Do not use taskqueue for lock decoupling, as it causes unwanted races.
2010-02-02 18:07:16 +00:00
csa->event_enable = added;
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
} else {
cur_entry = malloc(sizeof(*cur_entry), M_CAMXPT,
M_NOWAIT);
if (cur_entry == NULL) {
csa->ccb_h.status = CAM_RESRC_UNAVAIL;
break;
}
cur_entry->event_enable = csa->event_enable;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cur_entry->event_lock =
mtx_owned(path->bus->sim->mtx) ? 1 : 0;
For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus and device lists instead of decoupling the whole operation. This avoids problems with SIMs going away.
2007-04-16 19:55:36 +00:00
cur_entry->callback_arg = csa->callback_arg;
cur_entry->callback = csa->callback;
SLIST_INSERT_HEAD(async_head, cur_entry, links);
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
xpt_acquire_device(path->device);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
case XPT_REL_SIMQ:
{
struct ccb_relsim *crs;
struct cam_ed *dev;
crs = &start_ccb->crs;
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
dev = path->device;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (dev == NULL) {
crs->ccb_h.status = CAM_DEV_NOT_THERE;
break;
}
if ((crs->release_flags & RELSIM_ADJUST_OPENINGS) != 0) {
Allow to change number of openings (used tags) for ATA/SATA devices via `camcontrol tags ... -N ...`. There is no need to tune it in usual cases, but some users want to have it for debugging purposes. MFC after: 2 weeks
2012-05-29 14:28:46 +00:00
/* Don't ever go below one opening */
if (crs->openings > 0) {
xpt_dev_ccbq_resize(path, crs->openings);
if (bootverbose) {
xpt_print(path,
"number of openings is now %d\n",
crs->openings);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&dev->sim->devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((crs->release_flags & RELSIM_RELEASE_AFTER_TIMEOUT) != 0) {
if ((dev->flags & CAM_DEV_REL_TIMEOUT_PENDING) != 0) {
/*
* Just extend the old timeout and decrement
* the freeze count so that a single timeout
* is sufficient for releasing the queue.
*/
start_ccb->ccb_h.flags &= ~CAM_DEV_QFREEZE;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
callout_stop(&dev->callout);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
} else {
start_ccb->ccb_h.flags |= CAM_DEV_QFREEZE;
}
Prevent overflow issues in timeout processing Previously, any timeout value for which (timeout * hz) will overflow the signed integer, will give weird results, since callout(9) routines will convert negative values of ticks to '1'. For unsigned integer overflow we will get sufficiently smaller timeout values than expected. Switch from callout_reset, which requires conversion to int based ticks to callout_reset_sbt to avoid this. Also correct isci to correctly resolve ccb timeout. This was based on the original work done by Eygene Ryabinkin <rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid the overlow. Differential Revision: https://reviews.freebsd.org/D1157 Reviewed by: mav, davide MFC after: 1 month Sponsored by: Multiplay
2014-11-21 21:01:24 +00:00
callout_reset_sbt(&dev->callout,
SBT_1MS * crs->release_timeout, 0,
xpt_release_devq_timeout, dev, 0);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
dev->flags |= CAM_DEV_REL_TIMEOUT_PENDING;
}
if ((crs->release_flags & RELSIM_RELEASE_AFTER_CMDCMPLT) != 0) {
if ((dev->flags & CAM_DEV_REL_ON_COMPLETE) != 0) {
/*
* Decrement the freeze count so that a single
* completion is still sufficient to unfreeze
* the queue.
*/
start_ccb->ccb_h.flags &= ~CAM_DEV_QFREEZE;
} else {
style(9)
2008-01-02 01:45:31 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
dev->flags |= CAM_DEV_REL_ON_COMPLETE;
start_ccb->ccb_h.flags |= CAM_DEV_QFREEZE;
}
}
if ((crs->release_flags & RELSIM_RELEASE_AFTER_QEMPTY) != 0) {
if ((dev->flags & CAM_DEV_REL_ON_QUEUE_EMPTY) != 0
|| (dev->ccbq.dev_active == 0)) {
start_ccb->ccb_h.flags &= ~CAM_DEV_QFREEZE;
} else {
style(9)
2008-01-02 01:45:31 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
dev->flags |= CAM_DEV_REL_ON_QUEUE_EMPTY;
start_ccb->ccb_h.flags |= CAM_DEV_QFREEZE;
}
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&dev->sim->devq->send_mtx);
Retire the spl() markers. Add in some minor missed locking as a result.
2007-04-19 23:34:51 +00:00
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
if ((start_ccb->ccb_h.flags & CAM_DEV_QFREEZE) == 0)
xpt_release_devq(path, /*count*/1, /*run_queue*/TRUE);
start_ccb->crs.qfrozen_cnt = dev->ccbq.queue.qfrozen_cnt;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
case XPT_DEBUG: {
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
struct cam_path *oldpath;
To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO, CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default. List of these flags can be modified with CAM_DEBUG_COMPILE kernel option. CAMDEBUG kernel option still enables all possible debug, if not overriden. Additional 50KB of kernel size is a good price for the ability to debug problems without rebuilding the kernel. In case where size is important, debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.
2012-06-07 10:05:51 +00:00
/* Check that all request bits are supported. */
Add CAM_DEBUG_INFO debug messages for periph created/invalidated/destroyed and for asyncs sent.
2012-06-07 10:53:42 +00:00
if (start_ccb->cdbg.flags & ~(CAM_DEBUG_COMPILE)) {
To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO, CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default. List of these flags can be modified with CAM_DEBUG_COMPILE kernel option. CAMDEBUG kernel option still enables all possible debug, if not overriden. Additional 50KB of kernel size is a good price for the ability to debug problems without rebuilding the kernel. In case where size is important, debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.
2012-06-07 10:05:51 +00:00
start_ccb->ccb_h.status = CAM_FUNC_NOTAVAIL;
break;
}
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
cam_dflags = CAM_DEBUG_NONE;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (cam_dpath != NULL) {
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
oldpath = cam_dpath;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cam_dpath = NULL;
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
xpt_free_path(oldpath);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
if (start_ccb->cdbg.flags != CAM_DEBUG_NONE) {
MFprojects/camlock r248982: Stop abusing xpt_periph in random plases that really have no periph related to CCB, for example, bus scanning. NULL value is fine in such cases and it is correctly logged in debug messages as "noperiph". If at some point we need some real XPT periphs (alike to pmpX now), quite likely they will be per-bus, and not a single global instance as xpt_periph now.
2013-04-14 09:55:48 +00:00
if (xpt_create_path(&cam_dpath, NULL,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.path_id,
start_ccb->ccb_h.target_id,
start_ccb->ccb_h.target_lun) !=
CAM_REQ_CMP) {
start_ccb->ccb_h.status = CAM_RESRC_UNAVAIL;
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
} else {
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
cam_dflags = start_ccb->cdbg.flags;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_REQ_CMP;
Add an xpt_print function to reduce most of the xpt_print_path/printf pairs. Convert the core code to use it. Reviewed by: scsi (various folks weighing in with good advice) MFC after: 2 weeks
2006-12-05 07:45:28 +00:00
xpt_print(cam_dpath, "debugging flags now %x\n",
cam_dflags);
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
}
Fix XPT_DEBUG paths operations locking: - Extend the lock to cover xpt_path_release() for the new path. - While xpt_action() is called while holding right SIM lock for the new bus, the old path release may require different SIM lock. So we have to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
} else
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
}
case XPT_NOOP:
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
if ((start_ccb->ccb_h.flags & CAM_DEV_QFREEZE) != 0)
Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.
2011-04-29 07:14:37 +00:00
xpt_freeze_devq(path, 1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_REQ_CMP;
break;
default:
case XPT_SDEV_TYPE:
case XPT_TERM_IO:
case XPT_ENG_INQ:
/* XXX Implement */
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
printf("%s: CCB type %#x not supported\n", __func__,
start_ccb->ccb_h.func_code);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
start_ccb->ccb_h.status = CAM_PROVIDE_FAIL;
For unhandled actions in xpt_action_default, remember to call xpt_done for queued requests. This solves the problem of indefinite hangs for unspecified transports when XPT_SCAN_BUS is called. A few minor cosmetics elsewhere. MFC after: 1 week
2010-03-31 17:47:57 +00:00
if (start_ccb->ccb_h.func_code & XPT_FC_DEV_QUEUED) {
xpt_done(start_ccb);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
}
}
void
xpt_polled_action(union ccb *start_ccb)
{
u_int32_t timeout;
style(9)
2008-01-02 01:45:31 +00:00
struct cam_sim *sim;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_devq *devq;
struct cam_ed *dev;
Decrease poll interval from 1000 to 100us. This significantly reduces kernel dump time, especially with minidump.
2010-09-22 05:17:18 +00:00
timeout = start_ccb->ccb_h.timeout * 10;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
sim = start_ccb->ccb_h.path->bus->sim;
devq = sim->devq;
dev = start_ccb->ccb_h.path->device;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&dev->device_mtx);
Introduce CAM_SIM_POLLED SIM flag, indicating that it works in polling mode. It blocks CAM SWI usage on requests completion, unneeded because of polling and denied during kernel dumping because of blocked scheduler. Before r198899 there was periph flag CAM_PERIPH_POLLED, but that was wrong, because there is whole SIM is polled or handled by SWI, not a single periph. Tested by: kib MFC after: 1 month
2011-11-17 21:07:56 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Steal an opening so that no other queued requests
* can get it before us while we simulate interrupts.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&devq->send_mtx);
style(9)
2008-01-02 01:45:31 +00:00
dev->ccbq.dev_openings--;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
while((devq->send_openings <= 0 || dev->ccbq.dev_openings < 0) &&
(--timeout > 0)) {
mtx_unlock(&devq->send_mtx);
Decrease poll interval from 1000 to 100us. This significantly reduces kernel dump time, especially with minidump.
2010-09-22 05:17:18 +00:00
DELAY(100);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
CAM_SIM_LOCK(sim);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
(*(sim->sim_poll))(sim);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
CAM_SIM_UNLOCK(sim);
camisr_runqueue();
mtx_lock(&devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
dev->ccbq.dev_openings++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&devq->send_mtx);
style(9)
2008-01-02 01:45:31 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (timeout != 0) {
xpt_action(start_ccb);
while(--timeout > 0) {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
CAM_SIM_LOCK(sim);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
(*(sim->sim_poll))(sim);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
CAM_SIM_UNLOCK(sim);
camisr_runqueue();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if ((start_ccb->ccb_h.status & CAM_STATUS_MASK)
!= CAM_REQ_INPROG)
break;
Decrease poll interval from 1000 to 100us. This significantly reduces kernel dump time, especially with minidump.
2010-09-22 05:17:18 +00:00
DELAY(100);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
if (timeout == 0) {
/*
* XXX Is it worth adding a sim_timeout entry
* point so we can attempt recovery? If
* this is only used for dumps, I don't think
* it is.
*/
start_ccb->ccb_h.status = CAM_CMD_TIMEOUT;
}
} else {
start_ccb->ccb_h.status = CAM_RESRC_UNAVAIL;
}
Introduce CAM_SIM_POLLED SIM flag, indicating that it works in polling mode. It blocks CAM SWI usage on requests completion, unneeded because of polling and denied during kernel dumping because of blocked scheduler. Before r198899 there was periph flag CAM_PERIPH_POLLED, but that was wrong, because there is whole SIM is polled or handled by SWI, not a single periph. Tested by: kib MFC after: 1 month
2011-11-17 21:07:56 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&dev->device_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
style(9)
2008-01-02 01:45:31 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
Correct a comment
2014-09-17 18:59:25 +00:00
* Schedule a peripheral driver to receive a ccb when its
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
* target device has space for more transactions.
*/
void
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_schedule(struct cam_periph *periph, u_int32_t new_priority)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
CAM_DEBUG(periph->path, CAM_DEBUG_TRACE, ("xpt_schedule\n"));
cam_periph_assert(periph, MA_OWNED);
if (new_priority < periph->scheduled_priority) {
periph->scheduled_priority = new_priority;
xpt_run_allocq(periph, 0);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
/*
* Schedule a device to run on a given queue.
* If the device was inserted as a new entry on the queue,
* return 1 meaning the device queue should be run. If we
* were already queued, implying someone else has already
* started the queue, return 0 so the caller doesn't attempt
Retire the spl() markers. Add in some minor missed locking as a result.
2007-04-19 23:34:51 +00:00
* to run the queue.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static int
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_schedule_dev(struct camq *queue, cam_pinfo *pinfo,
u_int32_t new_priority)
{
int retval;
u_int32_t old_priority;
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT, ("xpt_schedule_dev\n"));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
old_priority = pinfo->priority;
/*
* Are we already queued?
*/
if (pinfo->index != CAM_UNQUEUED_INDEX) {
/* Simply reorder based on new priority */
if (new_priority < old_priority) {
camq_change_priority(queue, pinfo->index,
new_priority);
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
("changed priority to %d\n",
new_priority));
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
retval = 1;
} else
retval = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
} else {
/* New entry on the queue */
if (new_priority < old_priority)
pinfo->priority = new_priority;
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
("Inserting onto queue\n"));
Remove camq_regen(). We already perform modular comparisons for generation counts, so no further steps to deal with generation count wrap are required. Fix an off by one problem in the camq heap code.
1999-04-07 22:57:48 +00:00
pinfo->generation = ++queue->generation;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
camq_insert(queue, pinfo);
retval = 1;
}
return (retval);
}
static void
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_run_allocq_task(void *context, int pending)
{
struct cam_periph *periph = context;
cam_periph_lock(periph);
periph->flags &= ~CAM_PERIPH_RUN_TASK;
xpt_run_allocq(periph, 1);
cam_periph_unlock(periph);
cam_periph_release(periph);
}
static void
xpt_run_allocq(struct cam_periph *periph, int sleep)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_ed *device;
union ccb *ccb;
uint32_t prio;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_periph_assert(periph, MA_OWNED);
if (periph->periph_allocating)
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
return;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
periph->periph_allocating = 1;
CAM_DEBUG_PRINT(CAM_DEBUG_XPT, ("xpt_run_allocq(%p)\n", periph));
device = periph->path->device;
ccb = NULL;
restart:
while ((prio = min(periph->scheduled_priority,
periph->immediate_priority)) != CAM_PRIORITY_NONE &&
(periph->periph_allocated - (ccb != NULL ? 1 : 0) <
device->ccbq.total_openings || prio <= CAM_PRIORITY_OOB)) {
if (ccb == NULL &&
(ccb = xpt_get_ccb_nowait(periph)) == NULL) {
if (sleep) {
ccb = xpt_get_ccb(periph);
goto restart;
}
Unconditionally acquire periph reference on CCB allocation failure. cam_periph_acquire() can return error if periph already invalidated, but that may be unacceptable and cause deadlock if the invalidated periph can't be destroyed without "executing" the scheduled request. Coverity CID: 1109822 MFC after: 2 months
2013-10-22 12:58:22 +00:00
if (periph->flags & CAM_PERIPH_RUN_TASK)
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
break;
Take additional reference on SCSI probe periph to cover its freeze count. Otherwise periph may be invalidated and freed before single-stepping freeze is dropped, causing use after free panic.
2014-01-11 13:35:36 +00:00
cam_periph_doacquire(periph);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
periph->flags |= CAM_PERIPH_RUN_TASK;
taskqueue_enqueue(xsoftc.xpt_taskq,
&periph->periph_run_task);
break;
}
xpt_setup_ccb(&ccb->ccb_h, periph->path, prio);
if (prio == periph->immediate_priority) {
periph->immediate_priority = CAM_PRIORITY_NONE;
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT,
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
("waking cam_periph_getccb()\n"));
SLIST_INSERT_HEAD(&periph->ccb_list, &ccb->ccb_h,
periph_links.sle);
wakeup(&periph->ccb_list);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
} else {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
periph->scheduled_priority = CAM_PRIORITY_NONE;
CAM_DEBUG_PRINT(CAM_DEBUG_XPT,
("calling periph_start()\n"));
periph->periph_start(periph, ccb);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
ccb = NULL;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (ccb != NULL)
xpt_release_ccb(ccb);
periph->periph_allocating = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
static void
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
xpt_run_devq(struct cam_devq *devq)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
MFprojects/camlock r248928: Move CAM_DEBUG_CDB messages from the point of queuing to the point of sending to SIM. That allows to inspect real requests execution order, respecting priorities, freezing, etc. MFC after: 2 weeks
2013-04-04 18:59:29 +00:00
char cdb_str[(SCSI_MAX_CDBLEN * 3) + 1];
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
int lock;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT, ("xpt_run_devq\n"));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
devq->send_queue.qfrozen_cnt++;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
while ((devq->send_queue.entries > 0)
MFp4: Remove code that years ago was closing race between request submission to SIM and device/SIM freeze. That race become impossible after moving from spl to mutex locking, while this workaround causes some unexpected effects.
2009-11-14 20:23:20 +00:00
&& (devq->send_openings > 0)
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
&& (devq->send_queue.qfrozen_cnt <= 1)) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct cam_ed *device;
union ccb *work_ccb;
struct cam_sim *sim;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
device = (struct cam_ed *)camq_remove(&devq->send_queue,
cam_periph.c: Move handling of CAM_AUTOSENSE_FAIL into block dealing with all other scsi status errors. cam_queue.c: cam_queue.h: Fix 'off by one' heap bug in a more efficient manner. Since heap algorithms like to deal with indexes started from 1, offset our heap array pointer at allocation time to make this so for a C environment. This makes the implementation of the algorithm a bit more efficient. cam_xpt.c: Use macros for accessing the head of the heap so that code is isolated from implementation details of the heap.
1999-04-19 21:26:08 +00:00
CAMQ_HEAD);
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT,
Fix several potential buffer overrun conditions. These changes have been tested both in the kernel and in userland. Also, fix a couple of printf warnings that show up when CAMDEBUG is defined. Reviewed by: imp Partially submitted by: imp
1998-10-15 19:08:58 +00:00
("running device %p\n", device));
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cam_periph.c: Move handling of CAM_AUTOSENSE_FAIL into block dealing with all other scsi status errors. cam_queue.c: cam_queue.h: Fix 'off by one' heap bug in a more efficient manner. Since heap algorithms like to deal with indexes started from 1, offset our heap array pointer at allocation time to make this so for a C environment. This makes the implementation of the algorithm a bit more efficient. cam_xpt.c: Use macros for accessing the head of the heap so that code is isolated from implementation details of the heap.
1999-04-19 21:26:08 +00:00
work_ccb = cam_ccbq_peek_ccb(&device->ccbq, CAMQ_HEAD);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (work_ccb == NULL) {
Add \n to various printfs. grep shows these should be the last ones.
2002-08-27 18:43:14 +00:00
printf("device on run queue with no ccbs???\n");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
continue;
}
if ((work_ccb->ccb_h.flags & CAM_HIGH_POWER) != 0) {
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_lock(&xsoftc.xpt_highpower_lock);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
if (xsoftc.num_highpower <= 0) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* We got a high power command, but we
* don't have any available slots. Freeze
* the device queue until we have a slot
* available.
*/
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
xpt_freeze_devq_device(device, 1);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
STAILQ_INSERT_TAIL(&xsoftc.highpowerq, device,
MFprojects/camlock r249505: Change CCB queue resize logic to be able safely handle overallocations: - (re)allocate queue space in power of 2 chunks with 64 elements minimum and never shrink it; with only 4/8 bytes per element size is insignificant. - automatically reallocate the queue to double size if it is overflowed. - if queue reallocation failed, store extra CCBs in unsorted TAILQ, fetching them back as soon as some queue element is freed. To free space in CCB for TAILQ linking, change highpowerq from keeping high-power CCBs to keeping devices frozen due to high-power CCBs. This encloses all pieces of queue resize logic inside of cam_queue.[ch], removing some not obvious duties from xpt_release_ccb().
2013-08-05 11:48:40 +00:00
highpowerq_entry);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_unlock(&xsoftc.xpt_highpower_lock);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
continue;
} else {
/*
* Consume a high power slot while
* this ccb runs.
*/
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xsoftc.num_highpower--;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_unlock(&xsoftc.xpt_highpower_lock);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
cam_ccbq_remove_ccb(&device->ccbq, work_ccb);
cam_ccbq_send_ccb(&device->ccbq, work_ccb);
devq->send_openings--;
style(9)
2008-01-02 01:45:31 +00:00
devq->send_active++;
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
xpt_schedule_devq(devq, device);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
if ((work_ccb->ccb_h.flags & CAM_DEV_QFREEZE) != 0) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* The client wants to freeze the queue
* after this CCB is sent.
*/
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_freeze_devq(work_ccb->ccb_h.path, 1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Whoops- forgot to commit this other pearl from Justin- only set or clear CAM_TAG_ACTION_VALID if this is an XPT_SCSI_IO CCB (otherwise, the peripheral driver knows best...).. Obtained from:gibbs@freebsd.org
2000-07-17 00:43:47 +00:00
/* In Target mode, the peripheral driver knows best... */
if (work_ccb->ccb_h.func_code == XPT_SCSI_IO) {
if ((device->inq_flags & SID_CmdQue) != 0
&& work_ccb->csio.tag_action != CAM_TAG_ACTION_NONE)
work_ccb->ccb_h.flags |= CAM_TAG_ACTION_VALID;
else
/*
* Clear this in case of a retried CCB that
* failed due to a rejected tag.
*/
work_ccb->ccb_h.flags &= ~CAM_TAG_ACTION_VALID;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock r248928: Move CAM_DEBUG_CDB messages from the point of queuing to the point of sending to SIM. That allows to inspect real requests execution order, respecting priorities, freezing, etc. MFC after: 2 weeks
2013-04-04 18:59:29 +00:00
switch (work_ccb->ccb_h.func_code) {
case XPT_SCSI_IO:
CAM_DEBUG(work_ccb->ccb_h.path,
CAM_DEBUG_CDB,("%s. CDB: %s\n",
scsi_op_desc(work_ccb->csio.cdb_io.cdb_bytes[0],
&device->inq_data),
scsi_cdb_string(work_ccb->csio.cdb_io.cdb_bytes,
cdb_str, sizeof(cdb_str))));
break;
case XPT_ATA_IO:
CAM_DEBUG(work_ccb->ccb_h.path,
CAM_DEBUG_CDB,("%s. ACB: %s\n",
ata_op_string(&work_ccb->ataio.cmd),
ata_cmd_string(&work_ccb->ataio.cmd,
cdb_str, sizeof(cdb_str))));
break;
default:
break;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
* Device queues can be shared among multiple SIM instances
* that reside on different busses. Use the SIM from the
* queued device, rather than the one from the calling bus.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
sim = device->sim;
lock = (mtx_owned(sim->mtx) == 0);
if (lock)
CAM_SIM_LOCK(sim);
New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the same as before. The common scheduling bits have moved from inline code in each of the CAM periph drivers into a library that implements the default scheduling. In addition, a number of rate-limiting and I/O preference options can be enabled by adding CAM_IOSCHED_NETFLIX to your config file. A number of extra stats are also maintained. CAM_IOSCHED_NETFLIX isn't on by default because it uses a separate BIO_READ and BIO_WRITE queue, so doesn't honor BIO_ORDERED between these two types of operations. We already didn't honor it for BIO_DELETE, and we don't depend on BIO_ORDERED between reads and writes anywhere in the system (it is currently used with BIO_FLUSH in ZFS to make sure some writes are complete before others start and as a poor-man's soft dependency in one place in UFS where we won't be issuing READs until after the operation completes). However, out of an abundance of caution, it isn't enabled by default. Plus, this also brings in NCQ TRIM support for those SSDs that support it. A black list is also provided for known rogues that use NCQ trim as an excuse to corrupt the drive. It was difficult to separate out into a separate commit. This code has run in production at Netflix for over a year now. Sponsored by: Netflix, Inc Differential Revision: https://reviews.freebsd.org/D4609
2016-04-14 21:47:58 +00:00
work_ccb->ccb_h.qos.sim_data = sbinuptime(); // xxx uintprt_t too small 32bit platforms
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
(*(sim->sim_action))(sim, work_ccb);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (lock)
CAM_SIM_UNLOCK(sim);
mtx_lock(&devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
devq->send_queue.qfrozen_cnt--;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
/*
* This function merges stuff from the slave ccb into the master ccb, while
* keeping important fields in the master ccb constant.
*/
void
xpt_merge_ccb(union ccb *master_ccb, union ccb *slave_ccb)
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* Pull fields that are valid for peripheral drivers to set
* into the master CCB along with the CCB "payload".
*/
master_ccb->ccb_h.retry_count = slave_ccb->ccb_h.retry_count;
master_ccb->ccb_h.func_code = slave_ccb->ccb_h.func_code;
master_ccb->ccb_h.timeout = slave_ccb->ccb_h.timeout;
master_ccb->ccb_h.flags = slave_ccb->ccb_h.flags;
bcopy(&(&slave_ccb->ccb_h)[1], &(&master_ccb->ccb_h)[1],
sizeof(union ccb) - sizeof(struct ccb_hdr));
}
void
Add asynchronous command support to the pass(4) driver, and the new camdd(8) utility. CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and completed CCBs may be retrieved via the CAMIOGET ioctl. User processes can use poll(2) or kevent(2) to get notification when I/O has completed. While the existing CAMIOCOMMAND blocking ioctl interface only supports user virtual data pointers in a CCB (generally only one per CCB), the new CAMIOQUEUE ioctl supports user virtual and physical address pointers, as well as user virtual and physical scatter/gather lists. This allows user applications to have more flexibility in their data handling operations. Kernel memory for data transferred via the queued interface is allocated from the zone allocator in MAXPHYS sized chunks, and user data is copied in and out. This is likely faster than the vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in configurations with many processors (there are more TLB shootdowns caused by the mapping/unmapping operation) but may not be as fast as running with unmapped I/O. The new memory handling model for user requests also allows applications to send CCBs with request sizes that are larger than MAXPHYS. The pass(4) driver now limits queued requests to the I/O size listed by the SIM driver in the maxio field in the Path Inquiry (XPT_PATH_INQ) CCB. There are some things things would be good to add: 1. Come up with a way to do unmapped I/O on multiple buffers. Currently the unmapped I/O interface operates on a struct bio, which includes only one address and length. It would be nice to be able to send an unmapped scatter/gather list down to busdma. This would allow eliminating the copy we currently do for data. 2. Add an ioctl to list currently outstanding CCBs in the various queues. 3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do that. 4. Test physical address support. Virtual pointers and scatter gather lists have been tested, but I have not yet tested physical addresses or scatter/gather lists. 5. Investigate multiple queue support. At the moment there is one queue of commands per pass(4) device. If multiple processes open the device, they will submit I/O into the same queue and get events for the same completions. This is probably the right model for most applications, but it is something that could be changed later on. Also, add a new utility, camdd(8) that uses the asynchronous pass(4) driver interface. This utility is intended to be a basic data transfer/copy utility, a simple benchmark utility, and an example of how to use the asynchronous pass(4) interface. It can copy data to and from pass(4) devices using any target queue depth, starting offset and blocksize for the input and ouptut devices. It currently only supports SCSI devices, but could be easily extended to support ATA devices. It can also copy data to and from regular files, block devices, tape devices, pipes, stdin, and stdout. It does not support queueing multiple commands to any of those targets, since it uses the standard read(2)/write(2)/writev(2)/readv(2) system calls. The I/O is done by two threads, one for the reader and one for the writer. The reader thread sends completed read requests to the writer thread in strictly sequential order, even if they complete out of order. That could be modified later on for random I/O patterns or slightly out of order I/O. camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from the pass(4) driver and also to send request notifications internally. For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR) per CAM CCB on the reading side, and a scatter/gather list (CAM_DATA_SG) on the writing side. In addition to testing both interfaces, this makes any potential reblocking of I/O easier. No data is copied between the reader and the writer, but rather the reader's buffers are split into multiple I/O requests or combined into a single I/O request depending on the input and output blocksize. For the file I/O path, camdd(8) also uses a single buffer (read(2), write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list (readv(2), writev(2), preadv(2), pwritev(2)) on writes. Things that would be nice to do for camdd(8) eventually: 1. Add support for I/O pattern generation. Patterns like all zeros, all ones, LBA-based patterns, random patterns, etc. Right Now you can always use /dev/zero, /dev/random, etc. 2. Add support for a "sink" mode, so we do only reads with no writes. Right now, you can use /dev/null. 3. Add support for automatic queue depth probing, so that we can figure out the right queue depth on the input and output side for maximum throughput. At the moment it defaults to 6. 4. Add support for SATA device passthrough I/O. 5. Add support for random LBAs and/or lengths on the input and output sides. 6. Track average per-I/O latency and busy time. The busy time and latency could also feed in to the automatic queue depth determination. sys/cam/scsi/scsi_pass.h: Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue and fetch asynchronous CAM CCBs respectively. Although these ioctls do not have a declared argument, they both take a union ccb pointer. If we declare a size here, the ioctl code in sys/kern/sys_generic.c will malloc and free a buffer for either the CCB or the CCB pointer (depending on how it is declared). Since we have to keep a copy of the CCB (which is fairly large) anyway, having the ioctl malloc and free a CCB for each call is wasteful. sys/cam/scsi/scsi_pass.c: Add asynchronous CCB support. Add two new ioctls, CAMIOQUEUE and CAMIOGET. CAMIOQUEUE adds a CCB to the incoming queue. The CCB is executed immediately (and moved to the active queue) if it is an immediate CCB, but otherwise it will be executed in passstart() when a CCB is available from the transport layer. When CCBs are completed (because they are immediate or passdone() if they are queued), they are put on the done queue. If we get the final close on the device before all pending I/O is complete, all active I/O is moved to the abandoned queue and we increment the peripheral reference count so that the peripheral driver instance doesn't go away before all pending I/O is done. The new passcreatezone() function is called on the first call to the CAMIOQUEUE ioctl on a given device to allocate the UMA zones for I/O requests and S/G list buffers. This may be good to move off to a taskqueue at some point. The new passmemsetup() function allocates memory and scatter/gather lists to hold the user's data, and copies in any data that needs to be written. For virtual pointers (CAM_DATA_VADDR), the kernel buffer is malloced from the new pass(4) driver malloc bucket. For virtual scatter/gather lists (CAM_DATA_SG), buffers are allocated from a new per-pass(9) UMA zone in MAXPHYS-sized chunks. Physical pointers are passed in unchanged. We have support for up to 16 scatter/gather segments (for the user and kernel S/G lists) in the default struct pass_io_req, so requests with longer S/G lists require an extra kernel malloc. The new passcopysglist() function copies a user scatter/gather list to a kernel scatter/gather list. The number of elements in each list may be different, but (obviously) the amount of data stored has to be identical. The new passmemdone() function copies data out for the CAM_DATA_VADDR and CAM_DATA_SG cases. The new passiocleanup() function restores data pointers in user CCBs and frees memory. Add new functions to support kqueue(2)/kevent(2): passreadfilt() tells kevent whether or not the done queue is empty. passkqfilter() adds a knote to our list. passreadfiltdetach() removes a knote from our list. Add a new function, passpoll(), for poll(2)/select(2) to use. Add devstat(9) support for the queued CCB path. sys/cam/ata/ata_da.c: Add support for the BIO_VLIST bio type. sys/cam/cam_ccb.h: Add a new enumeration for the xflags field in the CCB header. (This doesn't change the CCB header, just adds an enumeration to use.) sys/cam/cam_xpt.c: Add a new function, xpt_setup_ccb_flags(), that allows specifying CCB flags. sys/cam/cam_xpt.h: Add a prototype for xpt_setup_ccb_flags(). sys/cam/scsi/scsi_da.c: Add support for BIO_VLIST. sys/dev/md/md.c: Add BIO_VLIST support to md(4). sys/geom/geom_disk.c: Add BIO_VLIST support to the GEOM disk class. Re-factor the I/O size limiting code in g_disk_start() a bit. sys/kern/subr_bus_dma.c: Change _bus_dmamap_load_vlist() to take a starting offset and length. Add a new function, _bus_dmamap_load_pages(), that will load a list of physical pages starting at an offset. Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios. Allow unmapped I/O to start at an offset. sys/kern/subr_uio.c: Add two new functions, physcopyin_vlist() and physcopyout_vlist(). sys/pc98/include/bus.h: Guard kernel-only parts of the pc98 machine/bus.h header with #ifdef _KERNEL. This allows userland programs to include <machine/bus.h> to get the definition of bus_addr_t and bus_size_t. sys/sys/bio.h: Add a new bio flag, BIO_VLIST. sys/sys/uio.h: Add prototypes for physcopyin_vlist() and physcopyout_vlist(). share/man/man4/pass.4: Document the CAMIOQUEUE and CAMIOGET ioctls. usr.sbin/Makefile: Add camdd. usr.sbin/camdd/Makefile: Add a makefile for camdd(8). usr.sbin/camdd/camdd.8: Man page for camdd(8). usr.sbin/camdd/camdd.c: The new camdd(8) utility. Sponsored by: Spectra Logic MFC after: 1 week
2015-12-03 20:54:55 +00:00
xpt_setup_ccb_flags(struct ccb_hdr *ccb_h, struct cam_path *path,
u_int32_t priority, u_int32_t flags)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEBUG(path, CAM_DEBUG_TRACE, ("xpt_setup_ccb\n"));
ccb_h->pinfo.priority = priority;
ccb_h->path = path;
ccb_h->path_id = path->bus->path_id;
if (path->target)
ccb_h->target_id = path->target->target_id;
else
ccb_h->target_id = CAM_TARGET_WILDCARD;
if (path->device) {
ccb_h->target_lun = path->device->lun_id;
Remove camq_regen(). We already perform modular comparisons for generation counts, so no further steps to deal with generation count wrap are required. Fix an off by one problem in the camq heap code.
1999-04-07 22:57:48 +00:00
ccb_h->pinfo.generation = ++path->device->ccbq.queue.generation;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
} else {
ccb_h->target_lun = CAM_TARGET_WILDCARD;
}
ccb_h->pinfo.index = CAM_UNQUEUED_INDEX;
Add asynchronous command support to the pass(4) driver, and the new camdd(8) utility. CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and completed CCBs may be retrieved via the CAMIOGET ioctl. User processes can use poll(2) or kevent(2) to get notification when I/O has completed. While the existing CAMIOCOMMAND blocking ioctl interface only supports user virtual data pointers in a CCB (generally only one per CCB), the new CAMIOQUEUE ioctl supports user virtual and physical address pointers, as well as user virtual and physical scatter/gather lists. This allows user applications to have more flexibility in their data handling operations. Kernel memory for data transferred via the queued interface is allocated from the zone allocator in MAXPHYS sized chunks, and user data is copied in and out. This is likely faster than the vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in configurations with many processors (there are more TLB shootdowns caused by the mapping/unmapping operation) but may not be as fast as running with unmapped I/O. The new memory handling model for user requests also allows applications to send CCBs with request sizes that are larger than MAXPHYS. The pass(4) driver now limits queued requests to the I/O size listed by the SIM driver in the maxio field in the Path Inquiry (XPT_PATH_INQ) CCB. There are some things things would be good to add: 1. Come up with a way to do unmapped I/O on multiple buffers. Currently the unmapped I/O interface operates on a struct bio, which includes only one address and length. It would be nice to be able to send an unmapped scatter/gather list down to busdma. This would allow eliminating the copy we currently do for data. 2. Add an ioctl to list currently outstanding CCBs in the various queues. 3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do that. 4. Test physical address support. Virtual pointers and scatter gather lists have been tested, but I have not yet tested physical addresses or scatter/gather lists. 5. Investigate multiple queue support. At the moment there is one queue of commands per pass(4) device. If multiple processes open the device, they will submit I/O into the same queue and get events for the same completions. This is probably the right model for most applications, but it is something that could be changed later on. Also, add a new utility, camdd(8) that uses the asynchronous pass(4) driver interface. This utility is intended to be a basic data transfer/copy utility, a simple benchmark utility, and an example of how to use the asynchronous pass(4) interface. It can copy data to and from pass(4) devices using any target queue depth, starting offset and blocksize for the input and ouptut devices. It currently only supports SCSI devices, but could be easily extended to support ATA devices. It can also copy data to and from regular files, block devices, tape devices, pipes, stdin, and stdout. It does not support queueing multiple commands to any of those targets, since it uses the standard read(2)/write(2)/writev(2)/readv(2) system calls. The I/O is done by two threads, one for the reader and one for the writer. The reader thread sends completed read requests to the writer thread in strictly sequential order, even if they complete out of order. That could be modified later on for random I/O patterns or slightly out of order I/O. camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from the pass(4) driver and also to send request notifications internally. For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR) per CAM CCB on the reading side, and a scatter/gather list (CAM_DATA_SG) on the writing side. In addition to testing both interfaces, this makes any potential reblocking of I/O easier. No data is copied between the reader and the writer, but rather the reader's buffers are split into multiple I/O requests or combined into a single I/O request depending on the input and output blocksize. For the file I/O path, camdd(8) also uses a single buffer (read(2), write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list (readv(2), writev(2), preadv(2), pwritev(2)) on writes. Things that would be nice to do for camdd(8) eventually: 1. Add support for I/O pattern generation. Patterns like all zeros, all ones, LBA-based patterns, random patterns, etc. Right Now you can always use /dev/zero, /dev/random, etc. 2. Add support for a "sink" mode, so we do only reads with no writes. Right now, you can use /dev/null. 3. Add support for automatic queue depth probing, so that we can figure out the right queue depth on the input and output side for maximum throughput. At the moment it defaults to 6. 4. Add support for SATA device passthrough I/O. 5. Add support for random LBAs and/or lengths on the input and output sides. 6. Track average per-I/O latency and busy time. The busy time and latency could also feed in to the automatic queue depth determination. sys/cam/scsi/scsi_pass.h: Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue and fetch asynchronous CAM CCBs respectively. Although these ioctls do not have a declared argument, they both take a union ccb pointer. If we declare a size here, the ioctl code in sys/kern/sys_generic.c will malloc and free a buffer for either the CCB or the CCB pointer (depending on how it is declared). Since we have to keep a copy of the CCB (which is fairly large) anyway, having the ioctl malloc and free a CCB for each call is wasteful. sys/cam/scsi/scsi_pass.c: Add asynchronous CCB support. Add two new ioctls, CAMIOQUEUE and CAMIOGET. CAMIOQUEUE adds a CCB to the incoming queue. The CCB is executed immediately (and moved to the active queue) if it is an immediate CCB, but otherwise it will be executed in passstart() when a CCB is available from the transport layer. When CCBs are completed (because they are immediate or passdone() if they are queued), they are put on the done queue. If we get the final close on the device before all pending I/O is complete, all active I/O is moved to the abandoned queue and we increment the peripheral reference count so that the peripheral driver instance doesn't go away before all pending I/O is done. The new passcreatezone() function is called on the first call to the CAMIOQUEUE ioctl on a given device to allocate the UMA zones for I/O requests and S/G list buffers. This may be good to move off to a taskqueue at some point. The new passmemsetup() function allocates memory and scatter/gather lists to hold the user's data, and copies in any data that needs to be written. For virtual pointers (CAM_DATA_VADDR), the kernel buffer is malloced from the new pass(4) driver malloc bucket. For virtual scatter/gather lists (CAM_DATA_SG), buffers are allocated from a new per-pass(9) UMA zone in MAXPHYS-sized chunks. Physical pointers are passed in unchanged. We have support for up to 16 scatter/gather segments (for the user and kernel S/G lists) in the default struct pass_io_req, so requests with longer S/G lists require an extra kernel malloc. The new passcopysglist() function copies a user scatter/gather list to a kernel scatter/gather list. The number of elements in each list may be different, but (obviously) the amount of data stored has to be identical. The new passmemdone() function copies data out for the CAM_DATA_VADDR and CAM_DATA_SG cases. The new passiocleanup() function restores data pointers in user CCBs and frees memory. Add new functions to support kqueue(2)/kevent(2): passreadfilt() tells kevent whether or not the done queue is empty. passkqfilter() adds a knote to our list. passreadfiltdetach() removes a knote from our list. Add a new function, passpoll(), for poll(2)/select(2) to use. Add devstat(9) support for the queued CCB path. sys/cam/ata/ata_da.c: Add support for the BIO_VLIST bio type. sys/cam/cam_ccb.h: Add a new enumeration for the xflags field in the CCB header. (This doesn't change the CCB header, just adds an enumeration to use.) sys/cam/cam_xpt.c: Add a new function, xpt_setup_ccb_flags(), that allows specifying CCB flags. sys/cam/cam_xpt.h: Add a prototype for xpt_setup_ccb_flags(). sys/cam/scsi/scsi_da.c: Add support for BIO_VLIST. sys/dev/md/md.c: Add BIO_VLIST support to md(4). sys/geom/geom_disk.c: Add BIO_VLIST support to the GEOM disk class. Re-factor the I/O size limiting code in g_disk_start() a bit. sys/kern/subr_bus_dma.c: Change _bus_dmamap_load_vlist() to take a starting offset and length. Add a new function, _bus_dmamap_load_pages(), that will load a list of physical pages starting at an offset. Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios. Allow unmapped I/O to start at an offset. sys/kern/subr_uio.c: Add two new functions, physcopyin_vlist() and physcopyout_vlist(). sys/pc98/include/bus.h: Guard kernel-only parts of the pc98 machine/bus.h header with #ifdef _KERNEL. This allows userland programs to include <machine/bus.h> to get the definition of bus_addr_t and bus_size_t. sys/sys/bio.h: Add a new bio flag, BIO_VLIST. sys/sys/uio.h: Add prototypes for physcopyin_vlist() and physcopyout_vlist(). share/man/man4/pass.4: Document the CAMIOQUEUE and CAMIOGET ioctls. usr.sbin/Makefile: Add camdd. usr.sbin/camdd/Makefile: Add a makefile for camdd(8). usr.sbin/camdd/camdd.8: Man page for camdd(8). usr.sbin/camdd/camdd.c: The new camdd(8) utility. Sponsored by: Spectra Logic MFC after: 1 week
2015-12-03 20:54:55 +00:00
ccb_h->flags = flags;
Make sure the CCB xflags field is initialized to zero so that CAM_EXTLUN_VALID is not erroneously set. Also add an XPORT_SRP identifier to the known SCSI transports for the SCSI RDMA protocol, as used, for example with Infiniband storage. Reviewed by: scottl Approved by: re (marius)
2013-09-27 16:02:40 +00:00
ccb_h->xflags = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Add asynchronous command support to the pass(4) driver, and the new camdd(8) utility. CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and completed CCBs may be retrieved via the CAMIOGET ioctl. User processes can use poll(2) or kevent(2) to get notification when I/O has completed. While the existing CAMIOCOMMAND blocking ioctl interface only supports user virtual data pointers in a CCB (generally only one per CCB), the new CAMIOQUEUE ioctl supports user virtual and physical address pointers, as well as user virtual and physical scatter/gather lists. This allows user applications to have more flexibility in their data handling operations. Kernel memory for data transferred via the queued interface is allocated from the zone allocator in MAXPHYS sized chunks, and user data is copied in and out. This is likely faster than the vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in configurations with many processors (there are more TLB shootdowns caused by the mapping/unmapping operation) but may not be as fast as running with unmapped I/O. The new memory handling model for user requests also allows applications to send CCBs with request sizes that are larger than MAXPHYS. The pass(4) driver now limits queued requests to the I/O size listed by the SIM driver in the maxio field in the Path Inquiry (XPT_PATH_INQ) CCB. There are some things things would be good to add: 1. Come up with a way to do unmapped I/O on multiple buffers. Currently the unmapped I/O interface operates on a struct bio, which includes only one address and length. It would be nice to be able to send an unmapped scatter/gather list down to busdma. This would allow eliminating the copy we currently do for data. 2. Add an ioctl to list currently outstanding CCBs in the various queues. 3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do that. 4. Test physical address support. Virtual pointers and scatter gather lists have been tested, but I have not yet tested physical addresses or scatter/gather lists. 5. Investigate multiple queue support. At the moment there is one queue of commands per pass(4) device. If multiple processes open the device, they will submit I/O into the same queue and get events for the same completions. This is probably the right model for most applications, but it is something that could be changed later on. Also, add a new utility, camdd(8) that uses the asynchronous pass(4) driver interface. This utility is intended to be a basic data transfer/copy utility, a simple benchmark utility, and an example of how to use the asynchronous pass(4) interface. It can copy data to and from pass(4) devices using any target queue depth, starting offset and blocksize for the input and ouptut devices. It currently only supports SCSI devices, but could be easily extended to support ATA devices. It can also copy data to and from regular files, block devices, tape devices, pipes, stdin, and stdout. It does not support queueing multiple commands to any of those targets, since it uses the standard read(2)/write(2)/writev(2)/readv(2) system calls. The I/O is done by two threads, one for the reader and one for the writer. The reader thread sends completed read requests to the writer thread in strictly sequential order, even if they complete out of order. That could be modified later on for random I/O patterns or slightly out of order I/O. camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from the pass(4) driver and also to send request notifications internally. For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR) per CAM CCB on the reading side, and a scatter/gather list (CAM_DATA_SG) on the writing side. In addition to testing both interfaces, this makes any potential reblocking of I/O easier. No data is copied between the reader and the writer, but rather the reader's buffers are split into multiple I/O requests or combined into a single I/O request depending on the input and output blocksize. For the file I/O path, camdd(8) also uses a single buffer (read(2), write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list (readv(2), writev(2), preadv(2), pwritev(2)) on writes. Things that would be nice to do for camdd(8) eventually: 1. Add support for I/O pattern generation. Patterns like all zeros, all ones, LBA-based patterns, random patterns, etc. Right Now you can always use /dev/zero, /dev/random, etc. 2. Add support for a "sink" mode, so we do only reads with no writes. Right now, you can use /dev/null. 3. Add support for automatic queue depth probing, so that we can figure out the right queue depth on the input and output side for maximum throughput. At the moment it defaults to 6. 4. Add support for SATA device passthrough I/O. 5. Add support for random LBAs and/or lengths on the input and output sides. 6. Track average per-I/O latency and busy time. The busy time and latency could also feed in to the automatic queue depth determination. sys/cam/scsi/scsi_pass.h: Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue and fetch asynchronous CAM CCBs respectively. Although these ioctls do not have a declared argument, they both take a union ccb pointer. If we declare a size here, the ioctl code in sys/kern/sys_generic.c will malloc and free a buffer for either the CCB or the CCB pointer (depending on how it is declared). Since we have to keep a copy of the CCB (which is fairly large) anyway, having the ioctl malloc and free a CCB for each call is wasteful. sys/cam/scsi/scsi_pass.c: Add asynchronous CCB support. Add two new ioctls, CAMIOQUEUE and CAMIOGET. CAMIOQUEUE adds a CCB to the incoming queue. The CCB is executed immediately (and moved to the active queue) if it is an immediate CCB, but otherwise it will be executed in passstart() when a CCB is available from the transport layer. When CCBs are completed (because they are immediate or passdone() if they are queued), they are put on the done queue. If we get the final close on the device before all pending I/O is complete, all active I/O is moved to the abandoned queue and we increment the peripheral reference count so that the peripheral driver instance doesn't go away before all pending I/O is done. The new passcreatezone() function is called on the first call to the CAMIOQUEUE ioctl on a given device to allocate the UMA zones for I/O requests and S/G list buffers. This may be good to move off to a taskqueue at some point. The new passmemsetup() function allocates memory and scatter/gather lists to hold the user's data, and copies in any data that needs to be written. For virtual pointers (CAM_DATA_VADDR), the kernel buffer is malloced from the new pass(4) driver malloc bucket. For virtual scatter/gather lists (CAM_DATA_SG), buffers are allocated from a new per-pass(9) UMA zone in MAXPHYS-sized chunks. Physical pointers are passed in unchanged. We have support for up to 16 scatter/gather segments (for the user and kernel S/G lists) in the default struct pass_io_req, so requests with longer S/G lists require an extra kernel malloc. The new passcopysglist() function copies a user scatter/gather list to a kernel scatter/gather list. The number of elements in each list may be different, but (obviously) the amount of data stored has to be identical. The new passmemdone() function copies data out for the CAM_DATA_VADDR and CAM_DATA_SG cases. The new passiocleanup() function restores data pointers in user CCBs and frees memory. Add new functions to support kqueue(2)/kevent(2): passreadfilt() tells kevent whether or not the done queue is empty. passkqfilter() adds a knote to our list. passreadfiltdetach() removes a knote from our list. Add a new function, passpoll(), for poll(2)/select(2) to use. Add devstat(9) support for the queued CCB path. sys/cam/ata/ata_da.c: Add support for the BIO_VLIST bio type. sys/cam/cam_ccb.h: Add a new enumeration for the xflags field in the CCB header. (This doesn't change the CCB header, just adds an enumeration to use.) sys/cam/cam_xpt.c: Add a new function, xpt_setup_ccb_flags(), that allows specifying CCB flags. sys/cam/cam_xpt.h: Add a prototype for xpt_setup_ccb_flags(). sys/cam/scsi/scsi_da.c: Add support for BIO_VLIST. sys/dev/md/md.c: Add BIO_VLIST support to md(4). sys/geom/geom_disk.c: Add BIO_VLIST support to the GEOM disk class. Re-factor the I/O size limiting code in g_disk_start() a bit. sys/kern/subr_bus_dma.c: Change _bus_dmamap_load_vlist() to take a starting offset and length. Add a new function, _bus_dmamap_load_pages(), that will load a list of physical pages starting at an offset. Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios. Allow unmapped I/O to start at an offset. sys/kern/subr_uio.c: Add two new functions, physcopyin_vlist() and physcopyout_vlist(). sys/pc98/include/bus.h: Guard kernel-only parts of the pc98 machine/bus.h header with #ifdef _KERNEL. This allows userland programs to include <machine/bus.h> to get the definition of bus_addr_t and bus_size_t. sys/sys/bio.h: Add a new bio flag, BIO_VLIST. sys/sys/uio.h: Add prototypes for physcopyin_vlist() and physcopyout_vlist(). share/man/man4/pass.4: Document the CAMIOQUEUE and CAMIOGET ioctls. usr.sbin/Makefile: Add camdd. usr.sbin/camdd/Makefile: Add a makefile for camdd(8). usr.sbin/camdd/camdd.8: Man page for camdd(8). usr.sbin/camdd/camdd.c: The new camdd(8) utility. Sponsored by: Spectra Logic MFC after: 1 week
2015-12-03 20:54:55 +00:00
void
xpt_setup_ccb(struct ccb_hdr *ccb_h, struct cam_path *path, u_int32_t priority)
{
xpt_setup_ccb_flags(ccb_h, path, priority, /*flags*/ 0);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/* Path manipulation functions */
cam_status
xpt_create_path(struct cam_path **new_path_ptr, struct cam_periph *perph,
path_id_t path_id, target_id_t target_id, lun_id_t lun_id)
{
struct cam_path *path;
cam_status status;
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
path = (struct cam_path *)malloc(sizeof(*path), M_CAMPATH, M_NOWAIT);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (path == NULL) {
status = CAM_RESRC_UNAVAIL;
return(status);
}
status = xpt_compile_path(path, perph, path_id, target_id, lun_id);
if (status != CAM_REQ_CMP) {
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
free(path, M_CAMPATH);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
path = NULL;
}
*new_path_ptr = path;
return (status);
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
cam_status
xpt_create_path_unlocked(struct cam_path **new_path_ptr,
struct cam_periph *periph, path_id_t path_id,
target_id_t target_id, lun_id_t lun_id)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
return (xpt_create_path(new_path_ptr, periph, path_id, target_id,
lun_id));
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
}
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cam_status
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_compile_path(struct cam_path *new_path, struct cam_periph *perph,
path_id_t path_id, target_id_t target_id, lun_id_t lun_id)
{
struct cam_eb *bus;
struct cam_et *target;
struct cam_ed *device;
cam_status status;
status = CAM_REQ_CMP; /* Completed without error */
target = NULL; /* Wildcarded */
device = NULL; /* Wildcarded */
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
/*
* We will potentially modify the EDT, so block interrupts
* that may attempt to create cam paths.
*/
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
bus = xpt_find_bus(path_id);
if (bus == NULL) {
status = CAM_PATH_INVALID;
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
} else {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_lock_buses();
mtx_lock(&bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
target = xpt_find_target(bus, target_id);
if (target == NULL) {
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
/* Create one */
struct cam_et *new_target;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
new_target = xpt_alloc_target(bus, target_id);
if (new_target == NULL) {
status = CAM_RESRC_UNAVAIL;
} else {
target = new_target;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_unlock_buses();
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
if (target != NULL) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
device = xpt_find_device(target, lun_id);
if (device == NULL) {
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
/* Create one */
struct cam_ed *new_device;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
new_device =
(*(bus->xport->alloc_device))(bus,
target,
lun_id);
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
if (new_device == NULL) {
status = CAM_RESRC_UNAVAIL;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
} else {
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
device = new_device;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&bus->eb_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
/*
* Only touch the user's data if we are successful.
*/
if (status == CAM_REQ_CMP) {
new_path->periph = perph;
new_path->bus = bus;
new_path->target = target;
new_path->device = device;
CAM_DEBUG(new_path, CAM_DEBUG_TRACE, ("xpt_compile_path\n"));
} else {
if (device != NULL)
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
xpt_release_device(device);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (target != NULL)
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
xpt_release_target(target);
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
if (bus != NULL)
xpt_release_bus(bus);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
return (status);
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_status
xpt_clone_path(struct cam_path **new_path_ptr, struct cam_path *path)
{
struct cam_path *new_path;
new_path = (struct cam_path *)malloc(sizeof(*path), M_CAMPATH, M_NOWAIT);
if (new_path == NULL)
return(CAM_RESRC_UNAVAIL);
xpt_copy_path(new_path, path);
*new_path_ptr = new_path;
return (CAM_REQ_CMP);
}
void
xpt_copy_path(struct cam_path *new_path, struct cam_path *path)
{
*new_path = *path;
if (path->bus != NULL)
xpt_acquire_bus(path->bus);
if (path->target != NULL)
xpt_acquire_target(path->target);
if (path->device != NULL)
xpt_acquire_device(path->device);
}
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
void
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_release_path(struct cam_path *path)
{
CAM_DEBUG(path, CAM_DEBUG_TRACE, ("xpt_release_path\n"));
Release bus entries in the EDT when a path is released. Null out path entries when the path is released to catch bugs. Free our configuration hook when we're done with it.
1999-04-23 23:25:48 +00:00
if (path->device != NULL) {
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
xpt_release_device(path->device);
Release bus entries in the EDT when a path is released. Null out path entries when the path is released to catch bugs. Free our configuration hook when we're done with it.
1999-04-23 23:25:48 +00:00
path->device = NULL;
}
if (path->target != NULL) {
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
xpt_release_target(path->target);
Release bus entries in the EDT when a path is released. Null out path entries when the path is released to catch bugs. Free our configuration hook when we're done with it.
1999-04-23 23:25:48 +00:00
path->target = NULL;
}
if (path->bus != NULL) {
xpt_release_bus(path->bus);
path->bus = NULL;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
void
xpt_free_path(struct cam_path *path)
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEBUG(path, CAM_DEBUG_TRACE, ("xpt_free_path\n"));
xpt_release_path(path);
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
free(path, M_CAMPATH);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Fixes for sure bus reference miscounting and potential device and target reference miscounts. It also adds a helper function to get the current reference counts for components of cam_path for debug aid. One minor style(9) change. Partially Obtained from: Chuck Tuffli (Emulex) Reviewed by: scsi@ (ken) Approved by: re (kib) MFC after: 1 month
2011-08-12 20:09:38 +00:00
void
xpt_path_counts(struct cam_path *path, uint32_t *bus_ref,
uint32_t *periph_ref, uint32_t *target_ref, uint32_t *device_ref)
{
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_lock_buses();
Fixes for sure bus reference miscounting and potential device and target reference miscounts. It also adds a helper function to get the current reference counts for components of cam_path for debug aid. One minor style(9) change. Partially Obtained from: Chuck Tuffli (Emulex) Reviewed by: scsi@ (ken) Approved by: re (kib) MFC after: 1 month
2011-08-12 20:09:38 +00:00
if (bus_ref) {
if (path->bus)
*bus_ref = path->bus->refcount;
else
*bus_ref = 0;
}
if (periph_ref) {
if (path->periph)
*periph_ref = path->periph->refcount;
else
*periph_ref = 0;
}
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
xpt_unlock_buses();
Fixes for sure bus reference miscounting and potential device and target reference miscounts. It also adds a helper function to get the current reference counts for components of cam_path for debug aid. One minor style(9) change. Partially Obtained from: Chuck Tuffli (Emulex) Reviewed by: scsi@ (ken) Approved by: re (kib) MFC after: 1 month
2011-08-12 20:09:38 +00:00
if (target_ref) {
if (path->target)
*target_ref = path->target->refcount;
else
*target_ref = 0;
}
if (device_ref) {
if (path->device)
*device_ref = path->device->refcount;
else
*device_ref = 0;
}
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
* Return -1 for failure, 0 for exact match, 1 for match with wildcards
* in path1, 2 for match with wildcards in path2.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
int
xpt_path_comp(struct cam_path *path1, struct cam_path *path2)
{
int retval = 0;
if (path1->bus != path2->bus) {
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
if (path1->bus->path_id == CAM_BUS_WILDCARD)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
retval = 1;
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
else if (path2->bus->path_id == CAM_BUS_WILDCARD)
retval = 2;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
else
return (-1);
}
if (path1->target != path2->target) {
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
if (path1->target->target_id == CAM_TARGET_WILDCARD) {
if (retval == 0)
retval = 1;
} else if (path2->target->target_id == CAM_TARGET_WILDCARD)
retval = 2;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
else
return (-1);
}
if (path1->device != path2->device) {
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
if (path1->device->lun_id == CAM_LUN_WILDCARD) {
if (retval == 0)
retval = 1;
} else if (path2->device->lun_id == CAM_LUN_WILDCARD)
retval = 2;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
else
return (-1);
}
return (retval);
}
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
int
xpt_path_comp_dev(struct cam_path *path, struct cam_ed *dev)
{
int retval = 0;
if (path->bus != dev->target->bus) {
if (path->bus->path_id == CAM_BUS_WILDCARD)
retval = 1;
else if (dev->target->bus->path_id == CAM_BUS_WILDCARD)
retval = 2;
else
return (-1);
}
if (path->target != dev->target) {
if (path->target->target_id == CAM_TARGET_WILDCARD) {
if (retval == 0)
retval = 1;
} else if (dev->target->target_id == CAM_TARGET_WILDCARD)
retval = 2;
else
return (-1);
}
if (path->device != dev) {
if (path->device->lun_id == CAM_LUN_WILDCARD) {
if (retval == 0)
retval = 1;
} else if (dev->lun_id == CAM_LUN_WILDCARD)
retval = 2;
else
return (-1);
}
return (retval);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
void
xpt_print_path(struct cam_path *path)
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (path == NULL)
printf("(nopath): ");
else {
if (path->periph != NULL)
printf("(%s%d:", path->periph->periph_name,
path->periph->unit_number);
else
printf("(noperiph:");
if (path->bus != NULL)
printf("%s%d:%d:", path->bus->sim->sim_name,
path->bus->sim->unit_number,
path->bus->sim->bus_id);
else
printf("nobus:");
if (path->target != NULL)
printf("%d:", path->target->target_id);
else
printf("X:");
if (path->device != NULL)
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
printf("%jx): ", (uintmax_t)path->device->lun_id);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
else
printf("X): ");
}
}
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
void
xpt_print_device(struct cam_ed *device)
{
if (device == NULL)
printf("(nopath): ");
else {
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
printf("(noperiph:%s%d:%d:%d:%jx): ", device->sim->sim_name,
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
device->sim->unit_number,
device->sim->bus_id,
device->target->target_id,
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
(uintmax_t)device->lun_id);
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
}
}
Add an xpt_print function to reduce most of the xpt_print_path/printf pairs. Convert the core code to use it. Reviewed by: scsi (various folks weighing in with good advice) MFC after: 2 weeks
2006-12-05 07:45:28 +00:00
void
xpt_print(struct cam_path *path, const char *fmt, ...)
{
va_list ap;
xpt_print_path(path);
va_start(ap, fmt);
vprintf(fmt, ap);
va_end(ap);
}
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
int
xpt_path_string(struct cam_path *path, char *str, size_t str_len)
{
struct sbuf sb;
sbuf_new(&sb, str, str_len, 0);
if (path == NULL)
sbuf_printf(&sb, "(nopath): ");
else {
if (path->periph != NULL)
sbuf_printf(&sb, "(%s%d:", path->periph->periph_name,
path->periph->unit_number);
else
sbuf_printf(&sb, "(noperiph:");
if (path->bus != NULL)
sbuf_printf(&sb, "%s%d:%d:", path->bus->sim->sim_name,
path->bus->sim->unit_number,
path->bus->sim->bus_id);
else
sbuf_printf(&sb, "nobus:");
if (path->target != NULL)
sbuf_printf(&sb, "%d:", path->target->target_id);
else
sbuf_printf(&sb, "X:");
if (path->device != NULL)
printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t. MFC after: 2 weeks
2013-10-30 14:13:15 +00:00
sbuf_printf(&sb, "%jx): ",
(uintmax_t)path->device->lun_id);
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
else
sbuf_printf(&sb, "X): ");
}
sbuf_finish(&sb);
return(sbuf_len(&sb));
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
path_id_t
xpt_path_path_id(struct cam_path *path)
{
return(path->bus->path_id);
}
target_id_t
xpt_path_target_id(struct cam_path *path)
{
if (path->target != NULL)
return (path->target->target_id);
else
return (CAM_TARGET_WILDCARD);
}
lun_id_t
xpt_path_lun_id(struct cam_path *path)
{
if (path->device != NULL)
return (path->device->lun_id);
else
return (CAM_LUN_WILDCARD);
}
struct cam_sim *
xpt_path_sim(struct cam_path *path)
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (path->bus->sim);
}
struct cam_periph*
xpt_path_periph(struct cam_path *path)
{
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (path->periph);
}
/*
* Release a CAM control block for the caller. Remit the cost of the structure
* to the device referenced by the path. If the this device had no 'credits'
* and peripheral drivers have registered async callbacks for this notification
* call them now.
*/
void
xpt_release_ccb(union ccb *free_ccb)
{
struct cam_ed *device;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_periph *periph;
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
use CAM_DEBUG_XPT to track XPT; correct a misspelling
1998-12-06 00:06:48 +00:00
CAM_DEBUG_PRINT(CAM_DEBUG_XPT, ("xpt_release_ccb\n"));
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_assert(free_ccb->ccb_h.path, MA_OWNED);
device = free_ccb->ccb_h.path->device;
periph = free_ccb->ccb_h.path->periph;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_free_ccb(free_ccb);
periph->periph_allocated--;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cam_ccbq_release_opening(&device->ccbq);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_run_allocq(periph, 0);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
/* Functions accessed by SIM drivers */
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
static struct xpt_xport xport_default = {
.alloc_device = xpt_alloc_device_default,
.action = xpt_action_default,
.async = xpt_dev_async_default,
};
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
* A sim structure, listing the SIM entry points and instance
* identification info is passed to xpt_bus_register to hook the SIM
* into the CAM framework. xpt_bus_register creates a cam_eb entry
* for this new bus and places it in the array of busses and assigns
* it a path_id. The path_id may be influenced by "hard wiring"
* information specified by the user. Once interrupt services are
Make "kldunload atapicam" return EBUSY instead of deadlocking when a device created by atapicam is being kept opened or mounted. This is probably just a temporary solution until we invent something better. Reviewed by: scottl Approved by: rwatson (mentor) Sponsored by: FreeBSD Foundation Reported by: Jaakko Heinonen
2009-01-08 17:26:51 +00:00
* available, the bus will be probed.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
int32_t
Prepare for future integration between CAM and newbus. xpt_bus_register now takes a device_t to be the parent of the bus that is being created. Most SIMs have been updated with a reasonable argument, but a few exceptions just pass NULL for now. This argument isn't used yet and the newbus integration likely won't be ready until after 7.0-RELEASE.
2007-06-17 05:55:54 +00:00
xpt_bus_register(struct cam_sim *sim, device_t parent, u_int32_t bus)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
struct cam_eb *new_bus;
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
struct cam_eb *old_bus;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
struct ccb_pathinq cpi;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
struct cam_path *path;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cam_status status;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
mtx_assert(sim->mtx, MA_OWNED);
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
sim->bus_id = bus;
new_bus = (struct cam_eb *)malloc(sizeof(*new_bus),
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
M_CAMXPT, M_NOWAIT|M_ZERO);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (new_bus == NULL) {
/* Couldn't satisfy request */
return (CAM_RESRC_UNAVAIL);
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_init(&new_bus->eb_mtx, "CAM bus lock", NULL, MTX_DEF);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
TAILQ_INIT(&new_bus->et_entries);
Add SIM refcounting. This is slightly different from what DragonFly does - in DragonFly, it's cam_sim_release() what actually frees the SIM; cam_sim_free does nothing more than calling cam_sim_release(). Here, we drain in cam_sim_free, waiting for refcount to drop to zero. We cannot do the same think DragonFly does, because after cam_sim_free returns, client would destroy the sim->mtx, and CAM would trip over an initialized mutex. Reviewed by: scottl Approved by: rwatson (mentor) Sponsored by: FreeBSD Foundation
2008-12-16 16:57:33 +00:00
cam_sim_hold(sim);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
new_bus->sim = sim;
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
timevalclear(&new_bus->last_reset);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
new_bus->flags = 0;
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
new_bus->refcount = 1; /* Held until a bus_deregister event */
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
new_bus->generation = 0;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_lock_buses();
Close the race on path ID allocation in xpt_bus_register() if two buses are registered simultaneously. Due to topology unlock between the ID allocation and the bus registration there is a chance that two buses may get the same IDs. That is supposed reason of lock assertion panic in CAM during initial bus scanning after new iscsid initiates two sessions same time. Reported by: trasz Approved by: re (glebus, marius) MFC after: 2 weeks
2013-10-09 12:09:01 +00:00
sim->path_id = new_bus->path_id =
xptpathid(sim->sim_name, sim->unit_number, sim->bus_id);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
old_bus = TAILQ_FIRST(&xsoftc.xpt_busses);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
while (old_bus != NULL
&& old_bus->path_id < new_bus->path_id)
old_bus = TAILQ_NEXT(old_bus, links);
if (old_bus != NULL)
TAILQ_INSERT_BEFORE(old_bus, new_bus, links);
else
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
TAILQ_INSERT_TAIL(&xsoftc.xpt_busses, new_bus, links);
xsoftc.bus_generation++;
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
/*
* Set a default transport so that a PATH_INQ can be issued to
* the SIM. This will then allow for probing and attaching of
* a more appropriate transport.
*/
new_bus->xport = &xport_default;
Don't duplicate path/ccb allocation code, use existing functions.
2012-10-11 19:57:11 +00:00
status = xpt_create_path(&path, /*periph*/NULL, sim->path_id,
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD);
Really handle xpt_compile_path() error in xpt_bus_register() instead of print error message and probably crash just after it on NULL dereference. Found by: Clang Static Analyzer
2012-10-10 18:34:15 +00:00
if (status != CAM_REQ_CMP) {
xpt_release_bus(new_bus);
free(path, M_CAMXPT);
return (CAM_RESRC_UNAVAIL);
}
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_setup_ccb(&cpi.ccb_h, path, CAM_PRIORITY_NORMAL);
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
cpi.ccb_h.func_code = XPT_PATH_INQ;
xpt_action((union ccb *)&cpi);
if (cpi.ccb_h.status == CAM_REQ_CMP) {
switch (cpi.transport) {
case XPORT_SPI:
case XPORT_SAS:
case XPORT_FC:
case XPORT_USB:
Fix iSCSI initiator and vpo driver operation, broken by CAM changes. Reviewed by: scottl, Danny Braniss Approved by: re (rwatson)
2009-08-18 08:46:54 +00:00
case XPORT_ISCSI:
Make sure the CCB xflags field is initialized to zero so that CAM_EXTLUN_VALID is not erroneously set. Also add an XPORT_SRP identifier to the known SCSI transports for the SCSI RDMA protocol, as used, for example with Infiniband storage. Reviewed by: scottl Approved by: re (marius)
2013-09-27 16:02:40 +00:00
case XPORT_SRP:
Fix iSCSI initiator and vpo driver operation, broken by CAM changes. Reviewed by: scottl, Danny Braniss Approved by: re (rwatson)
2009-08-18 08:46:54 +00:00
case XPORT_PPB:
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
new_bus->xport = scsi_get_xport();
break;
case XPORT_ATA:
case XPORT_SATA:
new_bus->xport = ata_get_xport();
break;
default:
new_bus->xport = &xport_default;
break;
}
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/* Notify interested parties */
if (sim->path_id != CAM_XPT_PATH_ID) {
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_async(AC_PATH_REGISTERED, path, &cpi);
CAM and mps(4) driver scanning changes. Add a PIM_NOSCAN flag to the CAM path inquiry CCB. This tells CAM not to perform a rescan on a bus when it is registered. We now use this flag in the mps(4) driver. Since it knows what devices it has attached, it is more efficient for it to just issue a target rescan on the targets that are attached. Also, remove the private rescan thread from the mps(4) driver in favor of the rescan thread already built into CAM. Without this change, but with the change above, the MPS scanner could run before or during CAM's initial setup, which would cause duplicate device reprobes and announcements. sys/param.h: Bump __FreeBSD_version to 1000039 for the inclusion of the PIM_RESCAN CAM path inquiry flag. sys/cam/cam_ccb.h: sys/cam/cam_xpt.c: Added a PIM_NOSCAN flag. If a SIM sets this in the path inquiry ccb, then CAM won't rescan the bus in xpt_bus_regsister. sys/dev/mps/mps_sas.c For versions of FreeBSD that have the PIM_NOSCAN path inquiry flag, don't freeze the sim queue during scanning, because CAM won't be scanning this bus. Instead, hold up the boot. Don't call mpssas_rescan_target in mpssas_startup_decrement; it's redundant and I don't know why it was in there. Set PIM_NOSCAN in path inquiry CCBs. Remove methods related to the internal rescan daemon. Always use async events to trigger a probe for EEDP support. In older versions of FreeBSD where AC_ADVINFO_CHANGED is not available, use AC_FOUND_DEVICE and issue the necessary READ CAPACITY manually. Provide a path to xpt_register_async() so that we only receive events for our own SCSI domain. Improve error reporting in cases where setup for EEDP detection fails. sys/dev/mps/mps_sas.h: Remove softc flags and data related to the scanner thread. sys/dev/mps/mps_sas_lsi.c: Unconditionally rescan the target whenever a device is added. Sponsored by: Spectra Logic MFC after: 1 week
2013-07-22 18:37:07 +00:00
if ((cpi.hba_misc & PIM_NOSCAN) == 0) {
union ccb *scan_ccb;
/* Initiate bus rescan. */
scan_ccb = xpt_alloc_ccb_nowait();
if (scan_ccb != NULL) {
scan_ccb->ccb_h.path = path;
scan_ccb->ccb_h.func_code = XPT_SCAN_BUS;
scan_ccb->crcn.flags = 0;
xpt_rescan(scan_ccb);
Fix memory and references leak due to unfreed path in case we can't allocate bus scan CCB. MFC after: 2 weeks
2014-01-21 23:15:23 +00:00
} else {
CAM and mps(4) driver scanning changes. Add a PIM_NOSCAN flag to the CAM path inquiry CCB. This tells CAM not to perform a rescan on a bus when it is registered. We now use this flag in the mps(4) driver. Since it knows what devices it has attached, it is more efficient for it to just issue a target rescan on the targets that are attached. Also, remove the private rescan thread from the mps(4) driver in favor of the rescan thread already built into CAM. Without this change, but with the change above, the MPS scanner could run before or during CAM's initial setup, which would cause duplicate device reprobes and announcements. sys/param.h: Bump __FreeBSD_version to 1000039 for the inclusion of the PIM_RESCAN CAM path inquiry flag. sys/cam/cam_ccb.h: sys/cam/cam_xpt.c: Added a PIM_NOSCAN flag. If a SIM sets this in the path inquiry ccb, then CAM won't rescan the bus in xpt_bus_regsister. sys/dev/mps/mps_sas.c For versions of FreeBSD that have the PIM_NOSCAN path inquiry flag, don't freeze the sim queue during scanning, because CAM won't be scanning this bus. Instead, hold up the boot. Don't call mpssas_rescan_target in mpssas_startup_decrement; it's redundant and I don't know why it was in there. Set PIM_NOSCAN in path inquiry CCBs. Remove methods related to the internal rescan daemon. Always use async events to trigger a probe for EEDP support. In older versions of FreeBSD where AC_ADVINFO_CHANGED is not available, use AC_FOUND_DEVICE and issue the necessary READ CAPACITY manually. Provide a path to xpt_register_async() so that we only receive events for our own SCSI domain. Improve error reporting in cases where setup for EEDP detection fails. sys/dev/mps/mps_sas.h: Remove softc flags and data related to the scanner thread. sys/dev/mps/mps_sas_lsi.c: Unconditionally rescan the target whenever a device is added. Sponsored by: Spectra Logic MFC after: 1 week
2013-07-22 18:37:07 +00:00
xpt_print(path,
"Can't allocate CCB to scan bus\n");
Fix memory and references leak due to unfreed path in case we can't allocate bus scan CCB. MFC after: 2 weeks
2014-01-21 23:15:23 +00:00
xpt_free_path(path);
}
Make some improvements to r253322 to really rescan target, not a bus. Add there and in two more places checks for NULL on xpt_alloc_ccb_nowait().
2013-07-15 18:17:31 +00:00
} else
CAM and mps(4) driver scanning changes. Add a PIM_NOSCAN flag to the CAM path inquiry CCB. This tells CAM not to perform a rescan on a bus when it is registered. We now use this flag in the mps(4) driver. Since it knows what devices it has attached, it is more efficient for it to just issue a target rescan on the targets that are attached. Also, remove the private rescan thread from the mps(4) driver in favor of the rescan thread already built into CAM. Without this change, but with the change above, the MPS scanner could run before or during CAM's initial setup, which would cause duplicate device reprobes and announcements. sys/param.h: Bump __FreeBSD_version to 1000039 for the inclusion of the PIM_RESCAN CAM path inquiry flag. sys/cam/cam_ccb.h: sys/cam/cam_xpt.c: Added a PIM_NOSCAN flag. If a SIM sets this in the path inquiry ccb, then CAM won't rescan the bus in xpt_bus_regsister. sys/dev/mps/mps_sas.c For versions of FreeBSD that have the PIM_NOSCAN path inquiry flag, don't freeze the sim queue during scanning, because CAM won't be scanning this bus. Instead, hold up the boot. Don't call mpssas_rescan_target in mpssas_startup_decrement; it's redundant and I don't know why it was in there. Set PIM_NOSCAN in path inquiry CCBs. Remove methods related to the internal rescan daemon. Always use async events to trigger a probe for EEDP support. In older versions of FreeBSD where AC_ADVINFO_CHANGED is not available, use AC_FOUND_DEVICE and issue the necessary READ CAPACITY manually. Provide a path to xpt_register_async() so that we only receive events for our own SCSI domain. Improve error reporting in cases where setup for EEDP detection fails. sys/dev/mps/mps_sas.h: Remove softc flags and data related to the scanner thread. sys/dev/mps/mps_sas_lsi.c: Unconditionally rescan the target whenever a device is added. Sponsored by: Spectra Logic MFC after: 1 week
2013-07-22 18:37:07 +00:00
xpt_free_path(path);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
} else
xpt_free_path(path);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (CAM_SUCCESS);
}
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
int32_t
xpt_bus_deregister(path_id_t pathid)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
struct cam_path bus_path;
cam_status status;
status = xpt_compile_path(&bus_path, NULL, pathid,
CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD);
if (status != CAM_REQ_CMP)
return (status);
xpt_async(AC_LOST_DEVICE, &bus_path, NULL);
xpt_async(AC_PATH_DEREGISTERED, &bus_path, NULL);
When deregistering a bus, attempt to flush out all outstanding operations before returning. Point the bus at a dummy cam_sim structure so that any CCBs will complete immediately with a CAM_DEV_NOT_THERE status, and ensure that any xpt_schedule() calls on the bus's devices will immediately call the peripheral's periph_start() routine. Also repeat the async messages because devices that were part of the way through being probed may appear after the original AC_LOST_DEVICE was sent, and would otherwise never go away. These changes make it possible to deregister a bus and free the SIM at most stages during bus probing without the usual crashes in camisr(). In particular, plugging in a umass device and then unplugging it as soon as the first probe messages appeared would almost always result in a crash. Now the device just goes away with a few CAM errors and all references to the CAM bus, target and device are dropped correctly.
2006-01-11 02:06:08 +00:00
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
/* Release the reference count held while registered. */
xpt_release_bus(bus_path.bus);
xpt_release_path(&bus_path);
return (CAM_REQ_CMP);
}
static path_id_t
xptnextfreepathid(void)
{
struct cam_eb *bus;
path_id_t pathid;
Hints overhaul: - Replace some very poorly thought out API hacks that should have been fixed a long while ago. - Provide some much more flexible search functions (resource_find_*()) - Use strings for storage instead of an outgrowth of the rather inconvenient temporary ioconf table from config(). We already had a fallback to using strings before malloc/vm was running anyway.
2001-06-12 09:40:04 +00:00
const char *strval;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Close the race on path ID allocation in xpt_bus_register() if two buses are registered simultaneously. Due to topology unlock between the ID allocation and the bus registration there is a chance that two buses may get the same IDs. That is supposed reason of lock assertion panic in CAM during initial bus scanning after new iscsid initiates two sessions same time. Reported by: trasz Approved by: re (glebus, marius) MFC after: 2 weeks
2013-10-09 12:09:01 +00:00
mtx_assert(&xsoftc.xpt_topo_lock, MA_OWNED);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
pathid = 0;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
bus = TAILQ_FIRST(&xsoftc.xpt_busses);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
retry:
/* Find an unoccupied pathid */
Add an xpt_rescan function and a thread that will field rescan requests. The purpose of this is to allow a SIM (or other entities) to request a bus rescan and have it then fielded in a different (process) context from the caller. There are probably better ways to accomplish this, but it's a very small change that helps solve a number of problems. Reviewed by: Justin, Ken and Scott. MFC after: 2 weeks
2007-02-23 05:47:36 +00:00
while (bus != NULL && bus->path_id <= pathid) {
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
if (bus->path_id == pathid)
pathid++;
bus = TAILQ_NEXT(bus, links);
}
/*
* Ensure that this pathid is not reserved for
* a bus that may be registered in the future.
*/
A minimal update to the device wiring code so that it looks at the dynamic resource tables rather than relying on a duplicated cam-specific table generated by config(8) in ioconf.c. This was a major holdup to getting loader / userconfig driven configuration of scsi wiring.
2000-01-23 18:03:38 +00:00
if (resource_string_value("scbus", pathid, "at", &strval) == 0) {
++pathid;
/* Start the search over */
goto retry;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
return (pathid);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
static path_id_t
xptpathid(const char *sim_name, int sim_unit, int sim_bus)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
path_id_t pathid;
A minimal update to the device wiring code so that it looks at the dynamic resource tables rather than relying on a duplicated cam-specific table generated by config(8) in ioconf.c. This was a major holdup to getting loader / userconfig driven configuration of scsi wiring.
2000-01-23 18:03:38 +00:00
int i, dunit, val;
Slight cleanup - this is logically equivalent code but means one less use of the evil resource_locate() function.
2000-10-15 10:17:55 +00:00
char buf[32];
Hints overhaul: - Replace some very poorly thought out API hacks that should have been fixed a long while ago. - Provide some much more flexible search functions (resource_find_*()) - Use strings for storage instead of an outgrowth of the rather inconvenient temporary ioconf table from config(). We already had a fallback to using strings before malloc/vm was running anyway.
2001-06-12 09:40:04 +00:00
const char *dname;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
pathid = CAM_XPT_PATH_ID;
A minimal update to the device wiring code so that it looks at the dynamic resource tables rather than relying on a duplicated cam-specific table generated by config(8) in ioconf.c. This was a major holdup to getting loader / userconfig driven configuration of scsi wiring.
2000-01-23 18:03:38 +00:00
snprintf(buf, sizeof(buf), "%s%d", sim_name, sim_unit);
Close the race on path ID allocation in xpt_bus_register() if two buses are registered simultaneously. Due to topology unlock between the ID allocation and the bus registration there is a chance that two buses may get the same IDs. That is supposed reason of lock assertion panic in CAM during initial bus scanning after new iscsid initiates two sessions same time. Reported by: trasz Approved by: re (glebus, marius) MFC after: 2 weeks
2013-10-09 12:09:01 +00:00
if (strcmp(buf, "xpt0") == 0 && sim_bus == 0)
return (pathid);
Hints overhaul: - Replace some very poorly thought out API hacks that should have been fixed a long while ago. - Provide some much more flexible search functions (resource_find_*()) - Use strings for storage instead of an outgrowth of the rather inconvenient temporary ioconf table from config(). We already had a fallback to using strings before malloc/vm was running anyway.
2001-06-12 09:40:04 +00:00
i = 0;
while ((resource_find_match(&i, &dname, &dunit, "at", buf)) == 0) {
if (strcmp(dname, "scbus")) {
Slight cleanup - this is logically equivalent code but means one less use of the evil resource_locate() function.
2000-10-15 10:17:55 +00:00
/* Avoid a bit of foot shooting. */
continue;
}
A minimal update to the device wiring code so that it looks at the dynamic resource tables rather than relying on a duplicated cam-specific table generated by config(8) in ioconf.c. This was a major holdup to getting loader / userconfig driven configuration of scsi wiring.
2000-01-23 18:03:38 +00:00
if (dunit < 0) /* unwired?! */
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
continue;
A minimal update to the device wiring code so that it looks at the dynamic resource tables rather than relying on a duplicated cam-specific table generated by config(8) in ioconf.c. This was a major holdup to getting loader / userconfig driven configuration of scsi wiring.
2000-01-23 18:03:38 +00:00
if (resource_int_value("scbus", dunit, "bus", &val) == 0) {
if (sim_bus == val) {
pathid = dunit;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
}
A minimal update to the device wiring code so that it looks at the dynamic resource tables rather than relying on a duplicated cam-specific table generated by config(8) in ioconf.c. This was a major holdup to getting loader / userconfig driven configuration of scsi wiring.
2000-01-23 18:03:38 +00:00
} else if (sim_bus == 0) {
/* Unspecified matches bus 0 */
pathid = dunit;
break;
} else {
printf("Ambiguous scbus configuration for %s%d "
"bus %d, cannot wire down. The kernel "
"config entry for scbus%d should "
"specify a controller bus.\n"
"Scbus will be assigned dynamically.\n",
sim_name, sim_unit, sim_bus, dunit);
break;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
if (pathid == CAM_XPT_PATH_ID)
pathid = xptnextfreepathid();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (pathid);
}
Add CAM_DEBUG_INFO debug messages for periph created/invalidated/destroyed and for asyncs sent.
2012-06-07 10:53:42 +00:00
static const char *
xpt_async_string(u_int32_t async_code)
{
switch (async_code) {
case AC_BUS_RESET: return ("AC_BUS_RESET");
case AC_UNSOL_RESEL: return ("AC_UNSOL_RESEL");
case AC_SCSI_AEN: return ("AC_SCSI_AEN");
case AC_SENT_BDR: return ("AC_SENT_BDR");
case AC_PATH_REGISTERED: return ("AC_PATH_REGISTERED");
case AC_PATH_DEREGISTERED: return ("AC_PATH_DEREGISTERED");
case AC_FOUND_DEVICE: return ("AC_FOUND_DEVICE");
case AC_LOST_DEVICE: return ("AC_LOST_DEVICE");
case AC_TRANSFER_NEG: return ("AC_TRANSFER_NEG");
case AC_INQ_CHANGED: return ("AC_INQ_CHANGED");
case AC_GETDEV_CHANGED: return ("AC_GETDEV_CHANGED");
case AC_CONTRACT: return ("AC_CONTRACT");
case AC_ADVINFO_CHANGED: return ("AC_ADVINFO_CHANGED");
Implement media change notification for DA and CD removable media devices. It includes three parts: 1) Modifications to CAM to detect media media changes and report them to disk(9) layer. For modern SATA (and potentially UAS) devices it utilizes Asynchronous Notification mechanism to receive events from hardware. Active polling with TEST UNIT READY commands with 3 seconds period is used for incapable hardware. After that both CD and DA drivers work the same way, detecting two conditions: "NOT READY: Medium not present" after medium was detected previously, and "UNIT ATTENTION: Not ready to ready change, medium may have changed". First one reported to disk(9) as media removal, second as media insert/change. To reliably receive second event new AC_UNIT_ATTENTION async added to make UAs broadcasted to all periphs by generic error handling code in cam_periph_error(). 2) Modifications to GEOM core to handle media remove and change events. Media removal handled by spoiling all consumers attached to the provider. Media change event also schedules provider retaste after spoiling to probe new media. New flag G_CF_ORPHAN was added to consumers to reflect that consumer is in process of destruction. It allows retaste to create new geom instance of the same class, while previous one is still dying. 3) Modifications to some GEOM classes: DEV -- to report media change events to devd; VFS -- to handle spoiling same as orphan to prevent accessing replaced media. PART class already handles spoiling alike to orphan. Reviewed by: silence on geom@ and scsi@ Tested by: avg Sponsored by: iXsystems, Inc. / PC-BSD MFC after: 2 months
2012-07-29 11:51:48 +00:00
case AC_UNIT_ATTENTION: return ("AC_UNIT_ATTENTION");
Add CAM_DEBUG_INFO debug messages for periph created/invalidated/destroyed and for asyncs sent.
2012-06-07 10:53:42 +00:00
}
return ("AC_UNKNOWN");
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static int
xpt_async_size(u_int32_t async_code)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
switch (async_code) {
case AC_BUS_RESET: return (0);
case AC_UNSOL_RESEL: return (0);
case AC_SCSI_AEN: return (0);
case AC_SENT_BDR: return (0);
case AC_PATH_REGISTERED: return (sizeof(struct ccb_pathinq));
case AC_PATH_DEREGISTERED: return (0);
case AC_FOUND_DEVICE: return (sizeof(struct ccb_getdev));
case AC_LOST_DEVICE: return (0);
case AC_TRANSFER_NEG: return (sizeof(struct ccb_trans_settings));
case AC_INQ_CHANGED: return (0);
case AC_GETDEV_CHANGED: return (0);
case AC_CONTRACT: return (sizeof(struct ac_contract));
case AC_ADVINFO_CHANGED: return (-1);
case AC_UNIT_ATTENTION: return (sizeof(struct ccb_scsiio));
}
return (0);
}
static int
xpt_async_process_dev(struct cam_ed *device, void *arg)
{
union ccb *ccb = arg;
struct cam_path *path = ccb->ccb_h.path;
void *async_arg = ccb->casync.async_arg_ptr;
u_int32_t async_code = ccb->casync.async_code;
int relock;
if (path->device != device
&& path->device->lun_id != CAM_LUN_WILDCARD
&& device->lun_id != CAM_LUN_WILDCARD)
return (1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
/*
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
* The async callback could free the device.
* If it is a broadcast async, it doesn't hold
* device reference, so take our own reference.
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_acquire_device(device);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/*
* If async for specific device is to be delivered to
* the wildcard client, take the specific device lock.
* XXX: We may need a way for client to specify it.
*/
if ((device->lun_id == CAM_LUN_WILDCARD &&
path->device->lun_id != CAM_LUN_WILDCARD) ||
(device->target->target_id == CAM_TARGET_WILDCARD &&
path->target->target_id != CAM_TARGET_WILDCARD) ||
(device->target->bus->path_id == CAM_BUS_WILDCARD &&
path->target->bus->path_id != CAM_BUS_WILDCARD)) {
mtx_unlock(&device->device_mtx);
xpt_path_lock(path);
relock = 1;
} else
relock = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
(*(device->target->bus->xport->async))(async_code,
device->target->bus, device->target, device, async_arg);
xpt_async_bcast(&device->asyncs, async_code, path, async_arg);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (relock) {
xpt_path_unlock(path);
mtx_lock(&device->device_mtx);
}
xpt_release_device(device);
return (1);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static int
xpt_async_process_tgt(struct cam_et *target, void *arg)
{
union ccb *ccb = arg;
struct cam_path *path = ccb->ccb_h.path;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (path->target != target
&& path->target->target_id != CAM_TARGET_WILDCARD
&& target->target_id != CAM_TARGET_WILDCARD)
return (1);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (ccb->casync.async_code == AC_SENT_BDR) {
/* Update our notion of when the last reset occurred */
microtime(&target->last_reset);
}
Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb types allow the reporting of error counts and other statistics. Currently we provide information on the last BDR or bus reset as well as active transaction inforamtion, but this will be expanded as more information is added to aid in error recovery. Use the 'last reset' information to better handle bus settle delays. Peripheral drivers now control whether a bus settle delay occurs and for how long. This allows target mode peripheral drivers to avoid having their device queue frozen by the XPT for what shoudl only be initiator type behavior. Don't perform a bus reset if the target device is incapable of performing transfer negotiation (e.g. Fiber Channel). If we don't perform a bus reset but the controller is capable of transfer negotiations, force negotiations on the first transaction to go to the device. This ensures that we aren't tripped up by a left over negotiation from the prom, BIOS, loader, etc. Add a default async handler funstion to cam_periph.c to remove duplicated code in all initiator type peripheral drivers. Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.
1999-05-22 21:58:47 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
return (xptdevicetraverse(target, NULL, xpt_async_process_dev, ccb));
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static void
xpt_async_process(struct cam_periph *periph, union ccb *ccb)
{
struct cam_eb *bus;
struct cam_path *path;
void *async_arg;
u_int32_t async_code;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
path = ccb->ccb_h.path;
async_code = ccb->casync.async_code;
async_arg = ccb->casync.async_arg_ptr;
CAM_DEBUG(path, CAM_DEBUG_TRACE | CAM_DEBUG_INFO,
("xpt_async(%s)\n", xpt_async_string(async_code)));
bus = path->bus;
Allow 5 untagged commands to go to a device before enabling tags after enabling transfer negotiations, a BDR, or a bus reset to allow the controller driver to negotiate without tagged messages getting in the way. Some devices are confused by attempts to negotiate and tag at the same time. Some controllers (e.g. BT MultiMaster with certain firmware revs) will never negotiate if you don't give them an untagged "window" to perform negotiation in. Bump the maximum tag count to 255. The system reclaims unused tag space as the tag count is dropped anyway, so we might as well try the max. We should probably use a larger type than u_int8_t to hold our tag value as SCSI over certain mediums allows for higher values. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1998-09-23 03:03:19 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (async_code == AC_BUS_RESET) {
/* Update our notion of when the last reset occurred */
microtime(&bus->last_reset);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
style(9)
2008-01-02 01:45:31 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpttargettraverse(bus, NULL, xpt_async_process_tgt, ccb);
Add support for wildcard device entries in the EDT. The target mode 'Black Hole' device uses this feature to schedule itself against any target or lun attached to a controller that receives an unwanted request from an initiator instead of having an instance per potential target/lun request. Use the wildcard entries to simplify wildcard async callback storage. Don't announce devices twice to peripheral drivers. The devices will be announced as soon as the AC_PATH_REGISTERED event is registered by the peripheral driver, so no manaul push of this event is required. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1999-01-14 06:03:59 +00:00
/*
* If this wasn't a fully wildcarded async, tell all
* clients that want all async events.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (bus != xpt_periph->path->bus) {
xpt_path_lock(xpt_periph->path);
xpt_async_process_dev(xpt_periph->path->device, ccb);
xpt_path_unlock(xpt_periph->path);
}
if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD)
xpt_release_devq(path, 1, TRUE);
else
xpt_release_simq(path->bus->sim, TRUE);
if (ccb->casync.async_arg_size > 0)
free(async_arg, M_CAMXPT);
xpt_free_path(path);
xpt_free_ccb(ccb);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static void
xpt_async_bcast(struct async_list *async_head,
u_int32_t async_code,
struct cam_path *path, void *async_arg)
{
struct async_node *cur_entry;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
int lock;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cur_entry = SLIST_FIRST(async_head);
while (cur_entry != NULL) {
struct async_node *next_entry;
/*
* Grab the next list entry before we call the current
* entry's callback. This is because the callback function
* can delete its async callback entry.
*/
next_entry = SLIST_NEXT(cur_entry, links);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if ((cur_entry->event_enable & async_code) != 0) {
lock = cur_entry->event_lock;
if (lock)
CAM_SIM_LOCK(path->device->sim);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cur_entry->callback(cur_entry->callback_arg,
async_code, path,
async_arg);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (lock)
CAM_SIM_UNLOCK(path->device->sim);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cur_entry = next_entry;
}
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
void
xpt_async(u_int32_t async_code, struct cam_path *path, void *async_arg)
{
union ccb *ccb;
int size;
ccb = xpt_alloc_ccb_nowait();
if (ccb == NULL) {
xpt_print(path, "Can't allocate CCB to send %s\n",
xpt_async_string(async_code));
return;
}
if (xpt_clone_path(&ccb->ccb_h.path, path) != CAM_REQ_CMP) {
xpt_print(path, "Can't allocate path to send %s\n",
xpt_async_string(async_code));
xpt_free_ccb(ccb);
return;
}
ccb->ccb_h.path->periph = NULL;
ccb->ccb_h.func_code = XPT_ASYNC;
ccb->ccb_h.cbfcnp = xpt_async_process;
ccb->ccb_h.flags |= CAM_UNLOCKED;
ccb->casync.async_code = async_code;
ccb->casync.async_arg_size = 0;
size = xpt_async_size(async_code);
if (size > 0 && async_arg != NULL) {
ccb->casync.async_arg_ptr = malloc(size, M_CAMXPT, M_NOWAIT);
if (ccb->casync.async_arg_ptr == NULL) {
xpt_print(path, "Can't allocate argument to send %s\n",
xpt_async_string(async_code));
xpt_free_path(ccb->ccb_h.path);
xpt_free_ccb(ccb);
return;
}
memcpy(ccb->casync.async_arg_ptr, async_arg, size);
ccb->casync.async_arg_size = size;
Initialize async_arg_ptr in xpt_async when called with async_code AC_ADVINFO_CHANGED. Without this change, newly inserted hard disks won't always have their physical path device nodes created. The problem reproduces most readily when attaching a large number of disks at once. Differential Revision: https://reviews.freebsd.org/D2290 Reviewed by: mav, imp MFC after: 2 weeks Sponsored by: Spectra Logic
2015-04-14 16:33:33 +00:00
} else if (size < 0) {
ccb->casync.async_arg_ptr = async_arg;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
ccb->casync.async_arg_size = size;
Initialize async_arg_ptr in xpt_async when called with async_code AC_ADVINFO_CHANGED. Without this change, newly inserted hard disks won't always have their physical path device nodes created. The problem reproduces most readily when attaching a large number of disks at once. Differential Revision: https://reviews.freebsd.org/D2290 Reviewed by: mav, imp MFC after: 2 weeks Sponsored by: Spectra Logic
2015-04-14 16:33:33 +00:00
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD)
xpt_freeze_devq(path, 1);
else
xpt_freeze_simq(path->bus->sim, 1);
xpt_done(ccb);
}
Fix async notifications for listners registered to wildcard nodes. For example, a client registered to receive specific events for bus 0, target *, lun *, was not receiving notifications. Reviewed by: ken@FreeBSD.org
2000-11-06 20:12:07 +00:00
static void
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
xpt_dev_async_default(u_int32_t async_code, struct cam_eb *bus,
struct cam_et *target, struct cam_ed *device,
void *async_arg)
Fix async notifications for listners registered to wildcard nodes. For example, a client registered to receive specific events for bus 0, target *, lun *, was not receiving notifications. Reviewed by: ken@FreeBSD.org
2000-11-06 20:12:07 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/*
* We only need to handle events for real devices.
*/
if (target->target_id == CAM_TARGET_WILDCARD
|| device->lun_id == CAM_LUN_WILDCARD)
return;
For unhandled actions in xpt_action_default, remember to call xpt_done for queued requests. This solves the problem of indefinite hangs for unspecified transports when XPT_SCAN_BUS is called. A few minor cosmetics elsewhere. MFC after: 1 week
2010-03-31 17:47:57 +00:00
printf("%s called\n", __func__);
Fix async notifications for listners registered to wildcard nodes. For example, a client registered to receive specific events for bus 0, target *, lun *, was not receiving notifications. Reviewed by: ken@FreeBSD.org
2000-11-06 20:12:07 +00:00
}
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
static uint32_t
xpt_freeze_devq_device(struct cam_ed *dev, u_int count)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_devq *devq;
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
uint32_t freeze;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
devq = dev->sim->devq;
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_assert(&devq->send_mtx, MA_OWNED);
CAM_DEBUG_DEV(dev, CAM_DEBUG_TRACE,
("xpt_freeze_devq_device(%d) %u->%u\n", count,
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
dev->ccbq.queue.qfrozen_cnt, dev->ccbq.queue.qfrozen_cnt + count));
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
freeze = (dev->ccbq.queue.qfrozen_cnt += count);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/* Remove frozen device from sendq. */
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (device_is_queued(dev))
camq_remove(&devq->send_queue, dev->devq_entry.index);
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
return (freeze);
}
u_int32_t
xpt_freeze_devq(struct cam_path *path, u_int count)
{
struct cam_ed *dev = path->device;
struct cam_devq *devq;
uint32_t freeze;
devq = dev->sim->devq;
mtx_lock(&devq->send_mtx);
CAM_DEBUG(path, CAM_DEBUG_TRACE, ("xpt_freeze_devq(%d)\n", count));
freeze = xpt_freeze_devq_device(dev, count);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&devq->send_mtx);
return (freeze);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
u_int32_t
xpt_freeze_simq(struct cam_sim *sim, u_int count)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_devq *devq;
uint32_t freeze;
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
devq = sim->devq;
mtx_lock(&devq->send_mtx);
freeze = (devq->send_queue.qfrozen_cnt += count);
mtx_unlock(&devq->send_mtx);
return (freeze);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static void
xpt_release_devq_timeout(void *arg)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_ed *dev;
struct cam_devq *devq;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
dev = (struct cam_ed *)arg;
CAM_DEBUG_DEV(dev, CAM_DEBUG_TRACE, ("xpt_release_devq_timeout\n"));
devq = dev->sim->devq;
mtx_assert(&devq->send_mtx, MA_OWNED);
if (xpt_release_devq_device(dev, /*count*/1, /*run_queue*/TRUE))
xpt_run_devq(devq);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
void
Implement the guts of the XPT_ABORT function code. Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just as we do for XPT_SCSI_IO calls. Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by allowing the caller to decrement the count more than once per call. Change the semantics of xpt_path_cmp to return 1 for a match with a wildcard from path1 and 2 for a match with wildcards from path2. This change doesn't affect most callers as they only check for >=0 (exact match or one containing a wildcard). When cleaning up a device node, ensure that there are no timeouts outstanding that reference it.
1999-08-16 22:22:41 +00:00
xpt_release_devq(struct cam_path *path, u_int count, int run_queue)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_ed *dev;
struct cam_devq *devq;
Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the SCSI drivers.
2003-11-09 02:22:33 +00:00
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
CAM_DEBUG(path, CAM_DEBUG_TRACE, ("xpt_release_devq(%d, %d)\n",
count, run_queue));
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
dev = path->device;
devq = dev->sim->devq;
mtx_lock(&devq->send_mtx);
if (xpt_release_devq_device(dev, count, run_queue))
xpt_run_devq(dev->sim->devq);
mtx_unlock(&devq->send_mtx);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static int
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
xpt_release_devq_device(struct cam_ed *dev, u_int count, int run_queue)
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
{
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_assert(&dev->sim->devq->send_mtx, MA_OWNED);
Add debug trace points for freeze/release device queue.
2013-09-01 17:37:19 +00:00
CAM_DEBUG_DEV(dev, CAM_DEBUG_TRACE,
("xpt_release_devq_device(%d, %d) %u->%u\n", count, run_queue,
dev->ccbq.queue.qfrozen_cnt, dev->ccbq.queue.qfrozen_cnt - count));
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
if (count > dev->ccbq.queue.qfrozen_cnt) {
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
#ifdef INVARIANTS
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
printf("xpt_release_devq(): requested %u > present %u\n",
count, dev->ccbq.queue.qfrozen_cnt);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
#endif
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
count = dev->ccbq.queue.qfrozen_cnt;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
}
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
dev->ccbq.queue.qfrozen_cnt -= count;
if (dev->ccbq.queue.qfrozen_cnt == 0) {
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/*
* No longer need to wait for a successful
* command completion.
*/
dev->flags &= ~CAM_DEV_REL_ON_COMPLETE;
/*
* Remove any timeouts that might be scheduled
* to release this queue.
*/
if ((dev->flags & CAM_DEV_REL_TIMEOUT_PENDING) != 0) {
callout_stop(&dev->callout);
dev->flags &= ~CAM_DEV_REL_TIMEOUT_PENDING;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/*
* Now that we are unfrozen schedule the
* device so any pending transactions are
* run.
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_schedule_devq(dev->sim->devq, dev);
} else
run_queue = 0;
return (run_queue);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
void
xpt_release_simq(struct cam_sim *sim, int run_queue)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_devq *devq;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
devq = sim->devq;
mtx_lock(&devq->send_mtx);
if (devq->send_queue.qfrozen_cnt <= 0) {
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
#ifdef INVARIANTS
printf("xpt_release_simq: requested 1 > present %u\n",
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
devq->send_queue.qfrozen_cnt);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
#endif
} else
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
devq->send_queue.qfrozen_cnt--;
if (devq->send_queue.qfrozen_cnt == 0) {
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/*
* If there is a timeout scheduled to release this
* sim queue, remove it. The queue frozen count is
* already at 0.
*/
if ((sim->flags & CAM_SIM_REL_TIMEOUT_PENDING) != 0){
callout_stop(&sim->callout);
sim->flags &= ~CAM_SIM_REL_TIMEOUT_PENDING;
}
if (run_queue) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/*
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
* Now that we are unfrozen run the send queue.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
xpt_run_devq(sim->devq);
Retire the spl() markers. Add in some minor missed locking as a result.
2007-04-19 23:34:51 +00:00
}
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
/*
* XXX Appears to be unused.
*/
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static void
xpt_release_simq_timeout(void *arg)
{
struct cam_sim *sim;
sim = (struct cam_sim *)arg;
xpt_release_simq(sim, /* run_queue */ TRUE);
}
void
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
xpt_done(union ccb *done_ccb)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_doneq *queue;
int run, hash;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEBUG(done_ccb->ccb_h.path, CAM_DEBUG_TRACE, ("xpt_done\n"));
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if ((done_ccb->ccb_h.func_code & XPT_FC_QUEUED) == 0)
return;
Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI to allow drivers to handle request completion directly without passing them to the CAM SWI thread removing extra context switch. Modify all ATA/SATA drivers to use them. Reviewed by: gibbs, ken MFC after: 2 weeks
2012-05-12 13:55:36 +00:00
New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the same as before. The common scheduling bits have moved from inline code in each of the CAM periph drivers into a library that implements the default scheduling. In addition, a number of rate-limiting and I/O preference options can be enabled by adding CAM_IOSCHED_NETFLIX to your config file. A number of extra stats are also maintained. CAM_IOSCHED_NETFLIX isn't on by default because it uses a separate BIO_READ and BIO_WRITE queue, so doesn't honor BIO_ORDERED between these two types of operations. We already didn't honor it for BIO_DELETE, and we don't depend on BIO_ORDERED between reads and writes anywhere in the system (it is currently used with BIO_FLUSH in ZFS to make sure some writes are complete before others start and as a poor-man's soft dependency in one place in UFS where we won't be issuing READs until after the operation completes). However, out of an abundance of caution, it isn't enabled by default. Plus, this also brings in NCQ TRIM support for those SSDs that support it. A black list is also provided for known rogues that use NCQ trim as an excuse to corrupt the drive. It was difficult to separate out into a separate commit. This code has run in production at Netflix for over a year now. Sponsored by: Netflix, Inc Differential Revision: https://reviews.freebsd.org/D4609
2016-04-14 21:47:58 +00:00
/* Store the time the ccb was in the sim */
done_ccb->ccb_h.qos.sim_data = sbinuptime() - done_ccb->ccb_h.qos.sim_data;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
hash = (done_ccb->ccb_h.path_id + done_ccb->ccb_h.target_id +
done_ccb->ccb_h.target_lun) % cam_num_doneqs;
queue = &cam_doneqs[hash];
mtx_lock(&queue->cam_doneq_mtx);
run = (queue->cam_doneq_sleep && STAILQ_EMPTY(&queue->cam_doneq));
STAILQ_INSERT_TAIL(&queue->cam_doneq, &done_ccb->ccb_h, sim_links.stqe);
done_ccb->ccb_h.pinfo.index = CAM_DONEQ_INDEX;
mtx_unlock(&queue->cam_doneq_mtx);
if (run)
wakeup(&queue->cam_doneq);
Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI to allow drivers to handle request completion directly without passing them to the CAM SWI thread removing extra context switch. Modify all ATA/SATA drivers to use them. Reviewed by: gibbs, ken MFC after: 2 weeks
2012-05-12 13:55:36 +00:00
}
void
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_done_direct(union ccb *done_ccb)
Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI to allow drivers to handle request completion directly without passing them to the CAM SWI thread removing extra context switch. Modify all ATA/SATA drivers to use them. Reviewed by: gibbs, ken MFC after: 2 weeks
2012-05-12 13:55:36 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
CAM_DEBUG(done_ccb->ccb_h.path, CAM_DEBUG_TRACE, ("xpt_done_direct\n"));
if ((done_ccb->ccb_h.func_code & XPT_FC_QUEUED) == 0)
return;
New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the same as before. The common scheduling bits have moved from inline code in each of the CAM periph drivers into a library that implements the default scheduling. In addition, a number of rate-limiting and I/O preference options can be enabled by adding CAM_IOSCHED_NETFLIX to your config file. A number of extra stats are also maintained. CAM_IOSCHED_NETFLIX isn't on by default because it uses a separate BIO_READ and BIO_WRITE queue, so doesn't honor BIO_ORDERED between these two types of operations. We already didn't honor it for BIO_DELETE, and we don't depend on BIO_ORDERED between reads and writes anywhere in the system (it is currently used with BIO_FLUSH in ZFS to make sure some writes are complete before others start and as a poor-man's soft dependency in one place in UFS where we won't be issuing READs until after the operation completes). However, out of an abundance of caution, it isn't enabled by default. Plus, this also brings in NCQ TRIM support for those SSDs that support it. A black list is also provided for known rogues that use NCQ trim as an excuse to corrupt the drive. It was difficult to separate out into a separate commit. This code has run in production at Netflix for over a year now. Sponsored by: Netflix, Inc Differential Revision: https://reviews.freebsd.org/D4609
2016-04-14 21:47:58 +00:00
/* Store the time the ccb was in the sim */
done_ccb->ccb_h.qos.sim_data = sbinuptime() - done_ccb->ccb_h.qos.sim_data;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_done_process(&done_ccb->ccb_h);
Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI to allow drivers to handle request completion directly without passing them to the CAM SWI thread removing extra context switch. Modify all ATA/SATA drivers to use them. Reviewed by: gibbs, ken MFC after: 2 weeks
2012-05-12 13:55:36 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
union ccb *
Revert a driver API change to xpt_alloc_ccb that isn't necessary. Fix a couple of associated error checks.
2007-04-18 04:58:53 +00:00
xpt_alloc_ccb()
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
union ccb *new_ccb;
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
new_ccb = malloc(sizeof(*new_ccb), M_CAMCCB, M_ZERO|M_WAITOK);
- Providing fine-grained malloc statistic by replacing M_DEVBUF with module-specific malloc types. These should help us to pinpoint the possible memory leakage in the future. - Implementing xpt_alloc_ccb_nowait() and replacing all malloc/free based CCB memory management with xpt_alloc_ccb[_nowait]/xpt_free_ccb. Hopefully this would be helpful if someday we move the CCB allocator to use UMA instead of malloc(). Encouraged by: jeffr, rwatson Reviewed by: gibbs, scottl Approved by: re (scottl)
2005-07-01 15:21:30 +00:00
return (new_ccb);
}
union ccb *
Revert a driver API change to xpt_alloc_ccb that isn't necessary. Fix a couple of associated error checks.
2007-04-18 04:58:53 +00:00
xpt_alloc_ccb_nowait()
- Providing fine-grained malloc statistic by replacing M_DEVBUF with module-specific malloc types. These should help us to pinpoint the possible memory leakage in the future. - Implementing xpt_alloc_ccb_nowait() and replacing all malloc/free based CCB memory management with xpt_alloc_ccb[_nowait]/xpt_free_ccb. Hopefully this would be helpful if someday we move the CCB allocator to use UMA instead of malloc(). Encouraged by: jeffr, rwatson Reviewed by: gibbs, scottl Approved by: re (scottl)
2005-07-01 15:21:30 +00:00
{
union ccb *new_ccb;
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
new_ccb = malloc(sizeof(*new_ccb), M_CAMCCB, M_ZERO|M_NOWAIT);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (new_ccb);
}
void
xpt_free_ccb(union ccb *free_ccb)
{
Use separate malloc buckets for CAM devices, CCBs and paths. This will make it easier to track down the source of any use after free problems. Submitted by: ken@
2012-10-11 20:14:11 +00:00
free(free_ccb, M_CAMCCB);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
/* Private XPT functions */
/*
* Get a CAM control block for the caller. Charge the structure to the device
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
* referenced by the path. If we don't have sufficient resources to allocate
* more ccbs, we return NULL.
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
*/
static union ccb *
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_get_ccb_nowait(struct cam_periph *periph)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
union ccb *new_ccb;
Be consistent with M_ZERO when allocating ccbs. There are four places, all in cam_xpt.c, where ccbs are malloc'ed. Two of these use M_ZERO, two don't. The two that don't meant that allocated ccbs had trash in them making it hard to debug errors where they showed up. Due to this, use M_ZERO all the time when allocating ccbs. Submitted by: Scott Ferris <scott.ferris@isilon.com> Sponsored by: EMC/Isilon Storage Division Reviewed by: scottl, imp Differential: https://reviews.freebsd.org/D2120
2015-03-23 18:45:37 +00:00
new_ccb = malloc(sizeof(*new_ccb), M_CAMCCB, M_ZERO|M_NOWAIT);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (new_ccb == NULL)
return (NULL);
periph->periph_allocated++;
cam_ccbq_take_opening(&periph->path->device->ccbq);
return (new_ccb);
}
static union ccb *
xpt_get_ccb(struct cam_periph *periph)
{
union ccb *new_ccb;
cam_periph_unlock(periph);
Be consistent with M_ZERO when allocating ccbs. There are four places, all in cam_xpt.c, where ccbs are malloc'ed. Two of these use M_ZERO, two don't. The two that don't meant that allocated ccbs had trash in them making it hard to debug errors where they showed up. Due to this, use M_ZERO all the time when allocating ccbs. Submitted by: Scott Ferris <scott.ferris@isilon.com> Sponsored by: EMC/Isilon Storage Division Reviewed by: scottl, imp Differential: https://reviews.freebsd.org/D2120
2015-03-23 18:45:37 +00:00
new_ccb = malloc(sizeof(*new_ccb), M_CAMCCB, M_ZERO|M_WAITOK);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_periph_lock(periph);
periph->periph_allocated++;
cam_ccbq_take_opening(&periph->path->device->ccbq);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (new_ccb);
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
union ccb *
cam_periph_getccb(struct cam_periph *periph, u_int32_t priority)
{
struct ccb_hdr *ccb_h;
CAM_DEBUG(periph->path, CAM_DEBUG_TRACE, ("cam_periph_getccb\n"));
cam_periph_assert(periph, MA_OWNED);
while ((ccb_h = SLIST_FIRST(&periph->ccb_list)) == NULL ||
ccb_h->pinfo.priority != priority) {
if (priority < periph->immediate_priority) {
periph->immediate_priority = priority;
xpt_run_allocq(periph, 0);
} else
cam_periph_sleep(periph, &periph->ccb_list, PRIBIO,
"cgticb", 0);
}
SLIST_REMOVE_HEAD(&periph->ccb_list, periph_links.sle);
return ((union ccb *)ccb_h);
}
static void
xpt_acquire_bus(struct cam_eb *bus)
{
xpt_lock_buses();
bus->refcount++;
xpt_unlock_buses();
}
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
static void
xpt_release_bus(struct cam_eb *bus)
{
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_lock_buses();
Fixes for sure bus reference miscounting and potential device and target reference miscounts. It also adds a helper function to get the current reference counts for components of cam_path for debug aid. One minor style(9) change. Partially Obtained from: Chuck Tuffli (Emulex) Reviewed by: scsi@ (ken) Approved by: re (kib) MFC after: 1 month
2011-08-12 20:09:38 +00:00
KASSERT(bus->refcount >= 1, ("bus->refcount >= 1"));
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
if (--bus->refcount > 0) {
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
return;
}
TAILQ_REMOVE(&xsoftc.xpt_busses, bus, links);
xsoftc.bus_generation++;
xpt_unlock_buses();
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
KASSERT(TAILQ_EMPTY(&bus->et_entries),
("destroying bus, but target list is not empty"));
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
cam_sim_release(bus->sim);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_destroy(&bus->eb_mtx);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
free(bus, M_CAMXPT);
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static struct cam_et *
xpt_alloc_target(struct cam_eb *bus, target_id_t target_id)
{
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
struct cam_et *cur_target, *target;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_assert(&xsoftc.xpt_topo_lock, MA_OWNED);
mtx_assert(&bus->eb_mtx, MA_OWNED);
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
target = (struct cam_et *)malloc(sizeof(*target), M_CAMXPT,
M_NOWAIT|M_ZERO);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
if (target == NULL)
return (NULL);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
TAILQ_INIT(&target->ed_entries);
target->bus = bus;
target->target_id = target_id;
target->refcount = 1;
target->generation = 0;
target->luns = NULL;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_init(&target->luns_mtx, "CAM LUNs lock", NULL, MTX_DEF);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
timevalclear(&target->last_reset);
/*
* Hold a reference to our parent bus so it
* will not go away before we do.
*/
bus->refcount++;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
/* Insertion sort into our bus's target list */
cur_target = TAILQ_FIRST(&bus->et_entries);
while (cur_target != NULL && cur_target->target_id < target_id)
cur_target = TAILQ_NEXT(cur_target, links);
if (cur_target != NULL) {
TAILQ_INSERT_BEFORE(cur_target, target, links);
} else {
TAILQ_INSERT_TAIL(&bus->et_entries, target, links);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
bus->generation++;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (target);
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static void
xpt_acquire_target(struct cam_et *target)
{
struct cam_eb *bus = target->bus;
mtx_lock(&bus->eb_mtx);
target->refcount++;
mtx_unlock(&bus->eb_mtx);
}
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
static void
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
xpt_release_target(struct cam_et *target)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus = target->bus;
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&bus->eb_mtx);
if (--target->refcount > 0) {
mtx_unlock(&bus->eb_mtx);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
return;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
TAILQ_REMOVE(&bus->et_entries, target, links);
bus->generation++;
mtx_unlock(&bus->eb_mtx);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
KASSERT(TAILQ_EMPTY(&target->ed_entries),
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
("destroying target, but device list is not empty"));
xpt_release_bus(bus);
mtx_destroy(&target->luns_mtx);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
if (target->luns)
free(target->luns, M_CAMXPT);
free(target, M_CAMXPT);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static struct cam_ed *
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
xpt_alloc_device_default(struct cam_eb *bus, struct cam_et *target,
lun_id_t lun_id)
{
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
struct cam_ed *device;
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
device = xpt_alloc_device(bus, target, lun_id);
if (device == NULL)
return (NULL);
device->mintags = 1;
device->maxtags = 1;
return (device);
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
static void
xpt_destroy_device(void *context, int pending)
{
struct cam_ed *device = context;
mtx_lock(&device->device_mtx);
mtx_destroy(&device->device_mtx);
free(device, M_CAMDEV);
}
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
struct cam_ed *
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_alloc_device(struct cam_eb *bus, struct cam_et *target, lun_id_t lun_id)
{
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
struct cam_ed *cur_device, *device;
struct cam_devq *devq;
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
cam_status status;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_assert(&bus->eb_mtx, MA_OWNED);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
/* Make space for us in the device queue on our bus */
devq = bus->sim->devq;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&devq->send_mtx);
MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905, r248917, r248918, r248978, r249001, r249014, r249030: Remove multilevel freezing mechanism, implemented to handle specifics of the ATA/SATA error recovery, when post-reset recovery commands should be allocated when queues are already full of payload requests. Instead of removing frozen CCBs with specified range of priorities from the queue to provide free openings, use simple hack, allowing explicit CCBs over- allocation for requests with priority higher (numerically lower) then CAM_PRIORITY_OOB threshold. Simplify CCB allocation logic by removing SIM-level allocation queue. After that SIM-level queue manages only CCBs execution, while allocation logic is localized within each single device. Suggested by: gibbs
2013-04-14 09:28:14 +00:00
status = cam_devq_resize(devq, devq->send_queue.array_size + 1);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&devq->send_mtx);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
if (status != CAM_REQ_CMP)
return (NULL);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
device = (struct cam_ed *)malloc(sizeof(*device),
M_CAMDEV, M_NOWAIT|M_ZERO);
if (device == NULL)
return (NULL);
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
cam_init_pinfo(&device->devq_entry);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
device->target = target;
device->lun_id = lun_id;
device->sim = bus->sim;
if (cam_ccbq_init(&device->ccbq,
bus->sim->max_dev_openings) != 0) {
free(device, M_CAMDEV);
return (NULL);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
SLIST_INIT(&device->asyncs);
SLIST_INIT(&device->periphs);
device->generation = 0;
device->flags = CAM_DEV_UNCONFIGURED;
device->tag_delay_count = 0;
device->tag_saved_openings = 0;
device->refcount = 1;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_init(&device->device_mtx, "CAM device lock", NULL, MTX_DEF);
callout_init_mtx(&device->callout, &devq->send_mtx, 0);
TASK_INIT(&device->device_destroy_task, 0, xpt_destroy_device, device);
/*
* Hold a reference to our parent bus so it
* will not go away before we do.
*/
target->refcount++;
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
cur_device = TAILQ_FIRST(&target->ed_entries);
while (cur_device != NULL && cur_device->lun_id < lun_id)
cur_device = TAILQ_NEXT(cur_device, links);
if (cur_device != NULL)
TAILQ_INSERT_BEFORE(cur_device, device, links);
else
TAILQ_INSERT_TAIL(&target->ed_entries, device, links);
target->generation++;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (device);
}
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
void
xpt_acquire_device(struct cam_ed *device)
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus = device->target->bus;
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&bus->eb_mtx);
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
device->refcount++;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&bus->eb_mtx);
MFp4: Fix reference counting bug, when device unreferenced before then invalidated. To do it, do not handle validity flag as another reference, but explicitly modify reference count each time flag is modified. Discovered by: thompsa
2009-11-01 11:31:06 +00:00
}
void
xpt_release_device(struct cam_ed *device)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_eb *bus = device->target->bus;
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
struct cam_devq *devq;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&bus->eb_mtx);
if (--device->refcount > 0) {
mtx_unlock(&bus->eb_mtx);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
return;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
TAILQ_REMOVE(&device->target->ed_entries, device,links);
device->target->generation++;
mtx_unlock(&bus->eb_mtx);
/* Release our slot in the devq */
devq = bus->sim->devq;
mtx_lock(&devq->send_mtx);
cam_devq_resize(devq, devq->send_queue.array_size - 1);
mtx_unlock(&devq->send_mtx);
Add CAM infrastructure to allow reporting when a drive's long read capacity data changes. cam_ccb.h: Add a new advanced information type, CDAI_TYPE_RCAPLONG, for long read capacity data. cam_xpt_internal.h: Add a read capacity data pointer and length to struct cam_ed. cam_xpt.c: Free the read capacity buffer when a device goes away. While we're here, make sure we don't leak memory for other malloced fields in struct cam_ed. scsi_all.c: Update the scsi_read_capacity_16() to take a uint8_t * and a length instead of just a pointer to the parameter data structure. This will hopefully make this function somewhat immune to future changes in the parameter data. scsi_all.h: Add some extra bit definitions to struct scsi_read_capacity_data_long, and bump up the structure size to the full size specified by SBC-3. Change the prototype for scsi_read_capacity_16(). scsi_da.c: Register changes in read capacity data with the transport layer. This allows the transport layer to send out an async notification to interested parties. Update the dasetgeom() API. Use scsi_extract_sense_len() instead of scsi_extract_sense(). scsi_xpt.c: Add support for the new CDAI_TYPE_RCAPLONG advanced information type. Make sure we set the physpath pointer to NULL after freeing it. This allows blindly freeing it in the struct cam_ed destructor. sys/param.h: Bump __FreeBSD_version from 1000005 to 1000006 to make it easier for third party drivers to determine that the read capacity data async notification is available. camcontrol.c, mptutil/mpt_cam.c: Update these for the new scsi_read_capacity_16() argument structure. Sponsored by: Spectra Logic
2012-01-26 18:09:28 +00:00
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
KASSERT(SLIST_EMPTY(&device->periphs),
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
("destroying device, but periphs list is not empty"));
KASSERT(device->devq_entry.index == CAM_UNQUEUED_INDEX,
("destroying device while still queued for ccbs"));
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
if ((device->flags & CAM_DEV_REL_TIMEOUT_PENDING) != 0)
callout_stop(&device->callout);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_release_target(device->target);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
cam_ccbq_fini(&device->ccbq);
/*
* Free allocated memory. free(9) does nothing if the
* supplied pointer is NULL, so it is safe to call without
* checking.
*/
free(device->supported_vpds, M_CAMXPT);
free(device->device_id, M_CAMXPT);
Add support for probing the SCSI VPD Extended Inquiry page (0x86). This VPD page is effectively an extension of the standard Inquiry data page, and includes lots of additional bits. This commit includes support for probing the page in the SCSI probe code, and an additional request type for the XPT_DEV_ADVINFO CCB. CTL already supports the Extended Inquiry page. Support for querying this page in the sa(4) driver will come later. sys/cam/scsi/scsi_xpt.c: Probe the Extended Inquiry page, if the device supports it, and return it in response to a XPT_DEV_ADVINFO CCB if it is requested. sys/cam/scsi/cam_ccb.h: Define a new advanced information CCB data type, CDAI_TYPE_EXT_INQ. sys/cam/cam_xpt.c: Free the extended inquiry data in a device when the device goes away. sys/cam/cam_xpt_internal.h: Add an extended inquiry data pointer and length to struct cam_ed. sys/sys/param.h Bump __FreeBSD_version for the addition of the new CDAI_TYPE_EXT_INQ advanced information type. Sponsored by: Spectra Logic MFC after: 1 week
2015-02-05 00:12:21 +00:00
free(device->ext_inq, M_CAMXPT);
MFprojects/camlock: r249017: Some cosmetic things: - Unify device to target insertion inside xpt_alloc_device() instead of duplicating it three times. - Remove extra checks for empty lists of devices and targets on release since zero refcount check also implies it. - Reformat code to reduce indentation. r249103: - Add lock assertions to every point where reference counters are modified. - When reference counters are reaching zero, add assertions that there are no children items left. - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
free(device->physpath, M_CAMXPT);
free(device->rcap_buf, M_CAMXPT);
free(device->serial_num, M_CAMXPT);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
taskqueue_enqueue(xsoftc.xpt_taskq, &device->device_destroy_task);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
u_int32_t
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
xpt_dev_ccbq_resize(struct cam_path *path, int newopenings)
{
int result;
struct cam_ed *dev;
dev = path->device;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&dev->sim->devq->send_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
result = cam_ccbq_resize(&dev->ccbq, newopenings);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&dev->sim->devq->send_mtx);
Restore queue depth settings across tag disable events. The system often disables tag queuing temporarily in order to allow controllers a window to safely perform transfer negotiation with non-compliant devices. Before this change, CAM would restore the queue depth to the controller specified maximum or device quirk level rather than any depth determined by reactions to QUEUE FULL/BUSY events or an explicit user setting. During device probe, initialize the flags field for XPT_SCAN_BUS. The uninitialized value often confused CAM into not bothering to issue an AC_FOUND_DEVICE async event for new devices. The reason this bug wasn't reported earlier is that CAM manually announces devices after the initial system bus scans. MFC: 3 days
2005-03-23 16:43:29 +00:00
if ((dev->flags & CAM_DEV_TAG_AFTER_COUNT) != 0
|| (dev->inq_flags & SID_CmdQue) != 0)
dev->tag_saved_openings = newopenings;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (result);
}
static struct cam_eb *
xpt_find_bus(path_id_t path_id)
{
struct cam_eb *bus;
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_lock_buses();
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
for (bus = TAILQ_FIRST(&xsoftc.xpt_busses);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
bus != NULL;
bus = TAILQ_NEXT(bus, links)) {
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
if (bus->path_id == path_id) {
bus->refcount++;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
break;
Protect access to the EDT with splcam() to protect against corruption caused by temporary EDT allocations performed by controller drivers in their interrupt routiens. Reference count bus entries in the EDT in preparation for support for dynamic controller arrival and departure. Have children of the EDT hold references to their parents. Correct routing of the XPT_IMMED_NOTIFY ccb type for use in target mode applications. Fix a few cases where the generation count for EDT data members was not being updated when a modification occurred. splcam() problem Noticed by: Tor Egge <tegge@FreeBSD.org>
1999-03-05 23:18:16 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock) to unify the code.
2013-04-04 10:19:05 +00:00
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
return (bus);
}
static struct cam_et *
xpt_find_target(struct cam_eb *bus, target_id_t target_id)
{
struct cam_et *target;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_assert(&bus->eb_mtx, MA_OWNED);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
for (target = TAILQ_FIRST(&bus->et_entries);
target != NULL;
target = TAILQ_NEXT(target, links)) {
if (target->target_id == target_id) {
target->refcount++;
break;
}
}
return (target);
}
static struct cam_ed *
xpt_find_device(struct cam_et *target, lun_id_t lun_id)
{
struct cam_ed *device;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_assert(&target->bus->eb_mtx, MA_OWNED);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
for (device = TAILQ_FIRST(&target->ed_entries);
device != NULL;
device = TAILQ_NEXT(device, links)) {
if (device->lun_id == lun_id) {
device->refcount++;
break;
}
}
return (device);
}
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
void
Fix a few problems with the tag delay code: - Tagged devices were limited to one transaction (oops) - We revert to untagged with a tag delay if the user changed the transfer negotiation values (via perhaps camcontrol some day). - xpt_async did not use the expanded path in some cases which could cause a panic.
1998-09-24 22:43:54 +00:00
xpt_start_tags(struct cam_path *path)
{
Allow 5 untagged commands to go to a device before enabling tags after enabling transfer negotiations, a BDR, or a bus reset to allow the controller driver to negotiate without tagged messages getting in the way. Some devices are confused by attempts to negotiate and tag at the same time. Some controllers (e.g. BT MultiMaster with certain firmware revs) will never negotiate if you don't give them an untagged "window" to perform negotiation in. Bump the maximum tag count to 255. The system reclaims unused tag space as the tag count is dropped anyway, so we might as well try the max. We should probably use a larger type than u_int8_t to hold our tag value as SCSI over certain mediums allows for higher values. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1998-09-23 03:03:19 +00:00
struct ccb_relsim crs;
struct cam_ed *device;
struct cam_sim *sim;
int newopenings;
device = path->device;
sim = path->bus->sim;
device->flags &= ~CAM_DEV_TAG_AFTER_COUNT;
xpt_freeze_devq(path, /*count*/1);
device->inq_flags |= SID_CmdQue;
Restore queue depth settings across tag disable events. The system often disables tag queuing temporarily in order to allow controllers a window to safely perform transfer negotiation with non-compliant devices. Before this change, CAM would restore the queue depth to the controller specified maximum or device quirk level rather than any depth determined by reactions to QUEUE FULL/BUSY events or an explicit user setting. During device probe, initialize the flags field for XPT_SCAN_BUS. The uninitialized value often confused CAM into not bothering to issue an AC_FOUND_DEVICE async event for new devices. The reason this bug wasn't reported earlier is that CAM manually announces devices after the initial system bus scans. MFC: 3 days
2005-03-23 16:43:29 +00:00
if (device->tag_saved_openings != 0)
newopenings = device->tag_saved_openings;
else
Separate the parallel scsi knowledge out of the core of the XPT, and modularize it so that new transports can be created. Add a transport for SATA Add a periph+protocol layer for ATA Add a driver for AHCI-compliant hardware. Add a maxio field to CAM so that drivers can advertise their max I/O capability. Modify various drivers so that they are insulated from the value of MAXPHYS. The new ATA/SATA code supports AHCI-compliant hardware, and will override the classic ATA driver if it is loaded as a module at boot time or compiled into the kernel. The stack now support NCQ (tagged queueing) for increased performance on modern SATA drives. It also supports port multipliers. ATA drives are accessed via 'ada' device nodes. ATAPI drives are accessed via 'cd' device nodes. They can all be enumerated and manipulated via camcontrol, just like SCSI drives. SCSI commands are not translated to their ATA equivalents; ATA native commands are used throughout the entire stack, including camcontrol. See the camcontrol manpage for further details. Testing this code may require that you update your fstab, and possibly modify your BIOS to enable AHCI functionality, if available. This code is very experimental at the moment. The userland ABI/API has changed, so applications will need to be recompiled. It may change further in the near future. The 'ada' device name may also change as more infrastructure is completed in this project. The goal is to eventually put all CAM busses and devices until newbus, allowing for interesting topology and management options. Few functional changes will be seen with existing SCSI/SAS/FC drivers, though the userland ABI has still changed. In the future, transports specific modules for SAS and FC may appear in order to better support the topologies and capabilities of these technologies. The modularization of CAM and the addition of the ATA/SATA modules is meant to break CAM out of the mold of being specific to SCSI, letting it grow to be a framework for arbitrary transports and protocols. It also allows drivers to be written to support discrete hardware without jeopardizing the stability of non-related hardware. While only an AHCI driver is provided now, a Silicon Image driver is also in the works. Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware is possible and encouraged. Help with new transports is also encouraged. Submitted by: scottl, mav Approved by: re
2009-07-10 08:18:08 +00:00
newopenings = min(device->maxtags,
Restore queue depth settings across tag disable events. The system often disables tag queuing temporarily in order to allow controllers a window to safely perform transfer negotiation with non-compliant devices. Before this change, CAM would restore the queue depth to the controller specified maximum or device quirk level rather than any depth determined by reactions to QUEUE FULL/BUSY events or an explicit user setting. During device probe, initialize the flags field for XPT_SCAN_BUS. The uninitialized value often confused CAM into not bothering to issue an AC_FOUND_DEVICE async event for new devices. The reason this bug wasn't reported earlier is that CAM manually announces devices after the initial system bus scans. MFC: 3 days
2005-03-23 16:43:29 +00:00
sim->max_tagged_dev_openings);
Fix a few problems with the tag delay code: - Tagged devices were limited to one transaction (oops) - We revert to untagged with a tag delay if the user changed the transfer negotiation values (via perhaps camcontrol some day). - xpt_async did not use the expanded path in some cases which could cause a panic.
1998-09-24 22:43:54 +00:00
xpt_dev_ccbq_resize(path, newopenings);
Use AC_GETDEV_CHANGED async to notify ada driver about DMA and NCQ status change. Now that allows switching between PIO and DMA modes on the fly.
2012-06-01 09:32:37 +00:00
xpt_async(AC_GETDEV_CHANGED, path, NULL);
Replace most of priority numbers with defines. No logical changes.
2009-10-23 08:27:55 +00:00
xpt_setup_ccb(&crs.ccb_h, path, CAM_PRIORITY_NORMAL);
Allow 5 untagged commands to go to a device before enabling tags after enabling transfer negotiations, a BDR, or a bus reset to allow the controller driver to negotiate without tagged messages getting in the way. Some devices are confused by attempts to negotiate and tag at the same time. Some controllers (e.g. BT MultiMaster with certain firmware revs) will never negotiate if you don't give them an untagged "window" to perform negotiation in. Bump the maximum tag count to 255. The system reclaims unused tag space as the tag count is dropped anyway, so we might as well try the max. We should probably use a larger type than u_int8_t to hold our tag value as SCSI over certain mediums allows for higher values. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1998-09-23 03:03:19 +00:00
crs.ccb_h.func_code = XPT_REL_SIMQ;
crs.release_flags = RELSIM_RELEASE_AFTER_QEMPTY;
crs.openings
style(9)
2008-01-02 01:45:31 +00:00
= crs.release_timeout
Allow 5 untagged commands to go to a device before enabling tags after enabling transfer negotiations, a BDR, or a bus reset to allow the controller driver to negotiate without tagged messages getting in the way. Some devices are confused by attempts to negotiate and tag at the same time. Some controllers (e.g. BT MultiMaster with certain firmware revs) will never negotiate if you don't give them an untagged "window" to perform negotiation in. Bump the maximum tag count to 255. The system reclaims unused tag space as the tag count is dropped anyway, so we might as well try the max. We should probably use a larger type than u_int8_t to hold our tag value as SCSI over certain mediums allows for higher values. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1998-09-23 03:03:19 +00:00
= crs.qfrozen_cnt
= 0;
xpt_action((union ccb *)&crs);
}
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
void
xpt_stop_tags(struct cam_path *path)
{
struct ccb_relsim crs;
struct cam_ed *device;
struct cam_sim *sim;
device = path->device;
sim = path->bus->sim;
device->flags &= ~CAM_DEV_TAG_AFTER_COUNT;
device->tag_delay_count = 0;
xpt_freeze_devq(path, /*count*/1);
device->inq_flags &= ~SID_CmdQue;
xpt_dev_ccbq_resize(path, sim->max_dev_openings);
Use AC_GETDEV_CHANGED async to notify ada driver about DMA and NCQ status change. Now that allows switching between PIO and DMA modes on the fly.
2012-06-01 09:32:37 +00:00
xpt_async(AC_GETDEV_CHANGED, path, NULL);
MFp4: - Move tagged queueing control from ADA to ATA XPT. It allows to control device command queue length correctly. First step to support < 32 tags. - Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4). - Implement quirk matching for ATA devices. - Move xpt_schedule_dev_sendq() from header to source file. - Move delayed queue shrinking to the more expected place - element freeing. - Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
xpt_setup_ccb(&crs.ccb_h, path, CAM_PRIORITY_NORMAL);
crs.ccb_h.func_code = XPT_REL_SIMQ;
crs.release_flags = RELSIM_RELEASE_AFTER_QEMPTY;
crs.openings
= crs.release_timeout
= crs.qfrozen_cnt
= 0;
xpt_action((union ccb *)&crs);
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
static void
xpt_boot_delay(void *arg)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_release_boot();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
static void
xpt_config(void *arg)
{
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
/*
* Now that interrupts are enabled, go find our devices
*/
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (taskqueue_start_threads(&xsoftc.xpt_taskq, 1, PRIBIO, "CAM taskq"))
printf("xpt_config: failed to create taskqueue thread.\n");
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO, CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default. List of these flags can be modified with CAM_DEBUG_COMPILE kernel option. CAMDEBUG kernel option still enables all possible debug, if not overriden. Additional 50KB of kernel size is a good price for the ability to debug problems without rebuilding the kernel. In case where size is important, debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.
2012-06-07 10:05:51 +00:00
/* Setup debugging path */
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
if (cam_dflags != CAM_DEBUG_NONE) {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (xpt_create_path(&cam_dpath, NULL,
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
CAM_DEBUG_BUS, CAM_DEBUG_TARGET,
CAM_DEBUG_LUN) != CAM_REQ_CMP) {
printf("xpt_config: xpt_create_path() failed for debug"
" target %d:%d:%d, debugging disabled\n",
CAM_DEBUG_BUS, CAM_DEBUG_TARGET, CAM_DEBUG_LUN);
cam_dflags = CAM_DEBUG_NONE;
}
} else
cam_dpath = NULL;
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
periphdriver_init(1);
xpt_hold_boot();
callout_init(&xsoftc.boot_callout, 1);
Prevent overflow issues in timeout processing Previously, any timeout value for which (timeout * hz) will overflow the signed integer, will give weird results, since callout(9) routines will convert negative values of ticks to '1'. For unsigned integer overflow we will get sufficiently smaller timeout values than expected. Switch from callout_reset, which requires conversion to int based ticks to callout_reset_sbt to avoid this. Also correct isci to correctly resolve ccb timeout. This was based on the original work done by Eygene Ryabinkin <rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid the overlow. Differential Revision: https://reviews.freebsd.org/D1157 Reviewed by: mav, davide MFC after: 1 month Sponsored by: Multiplay
2014-11-21 21:01:24 +00:00
callout_reset_sbt(&xsoftc.boot_callout, SBT_1MS * xsoftc.boot_delay, 0,
xpt_boot_delay, NULL, 0);
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/* Fire up rescan thread. */
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (kproc_kthread_add(xpt_scanner_thread, NULL, &cam_proc, NULL, 0, 0,
"cam", "scanner")) {
MFp4: Make CAM to stop all attached devices on system shutdown. It allows devices to park heads, reducing stress on power loss. Add `kern.cam.power_down` tunable and sysctl to controll it.
2010-02-03 08:42:08 +00:00
printf("xpt_config: failed to create rescan thread.\n");
MFp4: - Reduce code duplication in ATA XPT and PMP driver. - Move PIO size setting from ada driver to ATA XPT. It is XPT business to negotiate transfer details. ada driver is now stateless. - Report PIO size to SIM. It is required for correct PATA SIM operation. - Tune PMP scan timings. It workarounds some problems with SiI. - If reset hapens during PMP initialization - restart it. - Introduce early-initialized periph drivers, which are used during initial scan process. Use it for xpt, probe, aprobe and pmp. It gives pmp chance to finish scan before mountroot and numerate devices in right order.
2009-10-31 10:43:38 +00:00
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
}
void
xpt_hold_boot(void)
{
xpt_lock_buses();
xsoftc.buses_to_config++;
xpt_unlock_buses();
}
void
xpt_release_boot(void)
{
xpt_lock_buses();
xsoftc.buses_to_config--;
if (xsoftc.buses_to_config == 0 && xsoftc.buses_config_done == 0) {
struct xpt_task *task;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xsoftc.buses_config_done = 1;
xpt_unlock_buses();
Only deregister out configuration hook manually if there are no SCSI busses to configure.
1998-09-20 05:03:34 +00:00
/* Call manually because we don't have any busses */
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
task = malloc(sizeof(struct xpt_task), M_CAMXPT, M_NOWAIT);
if (task != NULL) {
TASK_INIT(&task->task, 0, xpt_finishconfig_task, task);
taskqueue_enqueue(taskqueue_thread, &task->task);
If the bus delay is >= 2 seconds, notify the user that we are waiting for devices to settle. This will hopefully allay any 'first installation' fears that the machine has hung.
1998-10-10 21:10:36 +00:00
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
} else
xpt_unlock_buses();
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
/*
* If the given device only has one peripheral attached to it, and if that
* peripheral is the passthrough driver, announce it. This insures that the
* user sees some sort of announcement for every peripheral in their system.
*/
static int
xptpassannouncefunc(struct cam_ed *device, void *arg)
{
struct cam_periph *periph;
int i;
for (periph = SLIST_FIRST(&device->periphs), i = 0; periph != NULL;
periph = SLIST_NEXT(periph, periph_links), i++);
periph = SLIST_FIRST(&device->periphs);
if ((i == 1)
&& (strncmp(periph->periph_name, "pass", 4) == 0))
xpt_announce_periph(periph, NULL);
return(1);
}
static void
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
xpt_finishconfig_task(void *context, int pending)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
periphdriver_init(2);
/*
* Check for devices with no "standard" peripheral driver
* attached. For any devices like that, announce the
* passthrough driver so the user will see something.
*/
MFprojects/zfsd: - Add low-level support for SATA Enclosure Management Bridge (SEMB) devices -- SATA equivalents of the SCSI SES/SAF-TE devices. - Add some utility functions for SCSI SAF-TE devices access. Sponsored by: iXsystems, Inc.
2012-05-24 11:07:39 +00:00
if (!bootverbose)
xpt_for_all_devices(xptpassannouncefunc, NULL);
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
/* Release our hook so that the boot can continue. */
config_intrhook_disestablish(xsoftc.xpt_config_hook);
free(xsoftc.xpt_config_hook, M_CAMXPT);
xsoftc.xpt_config_hook = NULL;
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
free(context, M_CAMXPT);
}
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
cam_status
xpt_register_async(int event, ac_callback_t *cbfunc, void *cbarg,
struct cam_path *path)
{
struct ccb_setasync csa;
cam_status status;
int xptpath = 0;
if (path == NULL) {
status = xpt_create_path(&path, /*periph*/NULL, CAM_XPT_PATH_ID,
CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (status != CAM_REQ_CMP)
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
return (status);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_lock(path);
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
xptpath = 1;
}
MFp4: Large set of CAM inprovements. - Unify bus reset/probe sequence. Whenever bus attached at boot or later, CAM will automatically reset and scan it. It allows to remove duplicate code from many drivers. - Any bus, attached before CAM completed it's boot-time initialization, will equally join to the process, delaying boot if needed. - New kern.cam.boot_delay loader tunable should help controllers that are still unable to register their buses in time (such as slow USB/ PCCard/ CardBus devices), by adding one more event to wait on boot. - To allow synchronization between different CAM levels, concept of requests priorities was extended. Priorities now split between several "run levels". Device can be freezed at specified level, allowing higher priority requests to pass. For example, no payload requests allowed, until PMP driver enable port. ATA XPT negotiate transfer parameters, periph driver configure caching and so on. - Frozen requests are no more counted by request allocation scheduler. It fixes deadlocks, when frozen low priority payload requests occupying slots, required by higher levels to manage theit execution. - Two last changes were holding proper ATA reinitialization and error recovery implementation. Now it is done: SATA controllers and Port Multipliers now implement automatic hot-plug and should correctly recover from timeouts and bus resets. - Improve SCSI error recovery for devices on buses without automatic sense reporting, such as ATAPI or USB. For example, it allows CAM to wait, while CD drive loads disk, instead of immediately return error status. - Decapitalize diagnostic messages and make them more readable and sensible. - Teach PMP driver to limit maximum speed on fan-out ports. - Make boot wait for PMP scan completes, and make rescan more reliable. - Fix pass driver, to return CCB to user level in case of error. - Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
xpt_setup_ccb(&csa.ccb_h, path, CAM_PRIORITY_NORMAL);
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
csa.ccb_h.func_code = XPT_SASYNC_CB;
csa.event_enable = event;
csa.callback = cbfunc;
csa.callback_arg = cbarg;
xpt_action((union ccb *)&csa);
status = csa.ccb_h.status;
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
if (xptpath) {
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_path_unlock(path);
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
xpt_free_path(path);
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
}
Change the way in which fake async events generated. Do not use taskqueue for lock decoupling, as it causes unwanted races.
2010-02-02 18:07:16 +00:00
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
if ((status == CAM_REQ_CMP) &&
(csa.event_enable & AC_FOUND_DEVICE)) {
/*
* Get this peripheral up to date with all
* the currently existing devices.
*/
xpt_for_all_devices(xptsetasyncfunc, &csa);
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
}
Lay groundwork in CAM for recording and reporting physical path and other device attributes stored in the CAM Existing Device Table (EDT). This includes some infrastructure requried by the enclosure services driver to export physical path information. Make the CAM device advanced info interface accept store requests. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: - Replace scsi_get_sas_addr() with a scsi_get_devid() which takes a callback that decides whether to accept a particular descriptor. Provide callbacks for NAA IEEE Registered addresses and for SAS addresses, replacing the old function. This is needed because the old function doesn't work for an enclosure address for a SAS device, which is not flagged as a SAS address, but is NAA IEEE Registered. It may be worthwhile merging this interface with the devid match interface. - Add a few more defines for some device ID fields. sbin/camcontrol/camcontrol.c: - Update for the CCB_DEV_ADVINFO interface change. cam/cam_xpt_internal.h: - Add the new fields for the physical path string to the CAM EDT. cam/cam_ccb.h: - Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb structure to ccb_dev_advinfo. - Add a flag that changes this CCB's action to store, rather than the default, retrieve. - Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT physpath field. - Remove the never-implemented transport & proto flags. cam/cam_xpt.c: cam/cam_xpt.h: - Add xpt_getattr(), which provides a wrapper for fetching a device's attribute using the GEOM strings as key. This method currently supports "GEOM::ident" and "GEOM::physpath". Submitted by: will Reviewed by : gibbs Extend the XPT_DEV_MATCH api to allow a device search by device ID. As far as the API is concerned, device ID is a binary blob to be interpreted by the transport layer. The SCSI implementation assumes it is an array of VPD device ID descriptors. sys/cam/cam_ccb.h: Create a new structure, device_id_match_pattern, and update the XPT_DEV_MATCH datastructures and flags so that this pattern type can be used. sys/cam/cam_xpt.c: - A single pattern matching on both inquiry data and device ID is invalid. Report any violators. - Pass device ID match requests through to the new routine scsi_devid_match(). The direct call of a SCSI routine is a layering violation, but no worse than the one a few lines up that checks inquiry data. Defer cleaning this up until our future, larger, rototilling of CAM. - Zero out cam_ed and cam_et nodes on allocation. Prior to this change, device_id_len and device_id were not inialized, preventing proper detection of the presence of this information. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add the scsi_match_devid() routine. Add a helper function for extracting peripherial driver names sys/cam/cam_periph.c: sys/cam/cam_periph.h: Add the cam_periph_list() method which fills an sbuf with a comma delimited list of the peripheral instances associated with a given CAM path. Add a helper functions for SCSI commands used by the SES driver. sys/cam/scsi/scsi_all.c: sys/cam/scsi/scsi_all.h: Add structure definitions and csio filling functions for the receive diagnostic results and send diagnostic commands. Misc CAM XPT cleanups. sys/cam/cam_xpt.c: Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED events at the time async event handlers are attached even when registering just for events on a partitular SIM. Previously, you had to register for these events on all SIMs in the system in order to get the initial broadcast even though subsequent device and path arrivals would be delivered. sys/cam/cam_xpt.c: Remove SIM mutex held asserts from path accessors. CAM paths are reference counted and it is this reference count, not the sim mutex, that garantees they are stable. Sponsored by: Spectra Logic Corporation
2011-06-14 14:53:17 +00:00
if ((status == CAM_REQ_CMP) &&
(csa.event_enable & AC_PATH_REGISTERED)) {
/*
* Get this peripheral up to date with all
* the currently existing busses.
*/
xpt_for_all_busses(xptsetasyncbusfunc, &csa);
}
Add a helper function for registering async callbacks. Besides eliminating a lot of duplicated code, this also fixes a locking edge case.
2007-05-16 16:54:23 +00:00
return (status);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
static void
xptaction(struct cam_sim *sim, union ccb *work_ccb)
{
CAM_DEBUG(work_ccb->ccb_h.path, CAM_DEBUG_TRACE, ("xptaction\n"));
switch (work_ccb->ccb_h.func_code) {
/* Common cases first */
case XPT_PATH_INQ: /* Path routing inquiry */
{
struct ccb_pathinq *cpi;
cpi = &work_ccb->cpi;
cpi->version_num = 1; /* XXX??? */
cpi->hba_inquiry = 0;
cpi->target_sprt = 0;
cpi->hba_misc = 0;
cpi->hba_eng_cnt = 0;
cpi->max_target = 0;
cpi->max_lun = 0;
cpi->initiator_id = 0;
strncpy(cpi->sim_vid, "FreeBSD", SIM_IDLEN);
strncpy(cpi->hba_vid, "", HBA_IDLEN);
strncpy(cpi->dev_name, sim->sim_name, DEV_IDLEN);
cpi->unit_number = sim->unit_number;
cpi->bus_id = sim->bus_id;
Add a number of interrelated CAM feature enhancements and bug fixes. NOTE: These changes will require recompilation of any userland applications, like cdrecord, xmcd, etc., that use the CAM passthrough interface. A make world is recommended. camcontrol.[c8]: - We now support two new commands, "tags" and "negotiate". - The tags commands allows users to view the number of tagged openings for a device as well as a number of other related parameters, and it allows users to set tagged openings for a device. - The negotiate command allows users to enable and disable disconnection and tagged queueing, set sync rates, offsets and bus width. Note that not all of those features are available for all controllers. Only the adv, ahc, and ncr drivers fully support all of the features at this point. Some cards do not allow the setting of sync rates, offsets and the like, and some of the drivers don't have any facilities to do so. Some drivers, like the adw driver, only support enabling or disabling sync negotiation, but do not support setting sync rates. - new description in the camcontrol man page of how to format a disk - cleanup of the camcontrol inquiry command - add support in the 'devlist' command for skipping unconfigured devices if -v was not specified on the command line. - make use of the new base_transfer_speed in the path inquiry CCB. - fix CCB bzero cases cam_xpt.c, cam_sim.[ch], cam_ccb.h: - new flags on many CCB function codes to designate whether they're non-immediate, use a user-supplied CCB, and can only be passed from userland programs via the xpt device. Use these flags in the transport layer and pass driver to categorize CCBs. - new flag in the transport layer device matching code for device nodes that indicates whether a device is unconfigured - bump the CAM version from 0x10 to 0x11 - Change the CAM ioctls to use the version as their group code, so we can force users to recompile code even when the CCB size doesn't change. - add + fill in a new value in the path inquiry CCB, base_transfer_speed. Remove a corresponding field from the cam_sim structure, and add code to every SIM to set this field to the proper value. - Fix the set transfer settings code in the transport layer. scsi_cd.c: - make some variables volatile instead of just casting them in various places - fix a race condition in the changer code - attach unless we get a "logical unit not supported" error. This should fix all of the cases where people have devices that return weird errors when they don't have media in the drive. scsi_da.c: - attach unless we get a "logical unit not supported" error scsi_pass.c: - for immediate CCBs, just malloc a CCB to send the user request in. This gets rid of the 'held' count problem in camcontrol tags. scsi_pass.h: - change the CAM ioctls to use the CAM version as their group code. adv driver: - Allow changing the sync rate and offset separately. adw driver - Allow changing the sync rate and offset separately. aha driver: - Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs. ahc driver: - Allow setting offset and sync rate separately bt driver: - Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs. NCR driver: - Fix the ultra/ultra 2 negotiation bug - allow setting both the sync rate and offset separately Other HBA drivers: - Put code in to set the base_transfer_speed field for XPT_GET_TRAN_SETTINGS CCBs. Reviewed by: gibbs, mjacob (isp), imp (aha)
1999-05-06 20:16:39 +00:00
cpi->base_transfer_speed = 0;
Rewrite of the CAM error recovery code. Some of the major changes include: - The SCSI error handling portion of cam_periph_error() has been broken out into a number of subfunctions to better modularize the code that handles the hierarchy of SCSI errors. As a result, the code is now much easier to read. - String handling and error printing has been significantly revamped. We now use sbufs to do string formatting instead of using printfs (for the kernel) and snprintf/strncat (for userland) as before. There is a new catchall error printing routine, cam_error_print() and its string-based counterpart, cam_error_string() that allow the kernel and userland applications to pass in a CCB and have errors printed out properly, whether or not they're SCSI errors. Among other things, this helped eliminate a fair amount of duplicate code in camcontrol. We now print out more information than before, including the CAM status and SCSI status and the error recovery action taken to remedy the problem. - sbufs are now available in userland, via libsbuf. This change was necessary since most of the error printing code is shared between libcam and the kernel. - A new transfer settings interface is included in this checkin. This code is #ifdef'ed out, and is primarily intended to aid discussion with HBA driver authors on the final form the interface should take. There is example code in the ahc(4) driver that implements the HBA driver side of the new interface. The new transfer settings code won't be enabled until we're ready to switch all HBA drivers over to the new interface. src/Makefile.inc1, lib/Makefile: Add libsbuf. It must be built before libcam, since libcam uses sbuf routines. libcam/Makefile: libcam now depends on libsbuf. libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the sbuf sources from sys/kern. bsd.libnames.mk: Add LIBSBUF. camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically linked, we can't depend on the dynamic linker to pull in libsbuf. camcontrol.c: Use cam_error_print() instead of checking for CAM_SCSI_STATUS_ERROR on every failed CCB. sbuf.9: Change the prototypes for sbuf_cat() and sbuf_cpy() so that the source string is now a const char *. This is more in line wth the standard system string functions, and helps eliminate warnings when dealing with a const source buffer. Fix a typo. cam.c: Add description strings for the various CAM error status values, as well as routines to look up those strings. Add new cam_error_string() and cam_error_print() routines for userland and the kernel. cam.h: Add a new CAM flag, CAM_RETRY_SELTO. Add enumerated types for the various options available with cam_error_print() and cam_error_string(). cam_ccb.h: Add new transfer negotiation structures/types. Change inq_len in the ccb_getdev structure to be "reserved". This field has never been filled in, and will be removed when we next bump the CAM version. cam_debug.h: Fix typo. cam_periph.c: Modularize cam_periph_error(). The SCSI error handling part of cam_periph_error() is now in camperiphscsistatuserror() and camperiphscsisenseerror(). In cam_periph_lock(), increase the reference count on the periph while we wait for our lock attempt to succeed so that the periph won't go away while we're sleeping. cam_xpt.c: Add new transfer negotiation code. (ifdefed out) Add a new function, xpt_path_string(). This is a string/sbuf analog to xpt_print_path(). scsi_all.c: Revamp string handing and error printing code. We now use sbufs for much of the string formatting code. More of that code is shared between userland the kernel. scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly useful in the first place. Add a new error action, SS_REQSENSE. (Send a request sense and then retry the command.) This is useful when the controller hasn't performed autosense for some reason. Change the default actions around a bit. scsi_cd.c, scsi_da.c, scsi_pt.c, scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection timeouts shouldn't be covered by a sense flag. scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Get rid of the last vestiges of a read/write interface. libkern/bsearch.c, sys/libkern.h, conf/files: Add bsearch.c, which is needed for some of the new table lookup routines. aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if CAM_NEW_TRAN_CODE is defined. sbuf.h, subr_sbuf.c: Add the appropriate #ifdefs so sbufs can compile and run in userland. Change sbuf_printf() to use vsnprintf() instead of kvprintf(), which is only available in the kernel. Change the source string for sbuf_cpy() and sbuf_cat() to be a const char *. Add __BEGIN_DECLS and __END_DECLS around function prototypes since they're now exported to userland. kdump/mkioctls: Include stdio.h before cam.h since cam.h now includes a function with a FILE * argument. Submitted by: gibbs (mostly) Reviewed by: jdp, marcel (libsbuf makefile changes) Reviewed by: des (sbuf changes) Reviewed by: ken
2001-03-27 05:45:52 +00:00
cpi->protocol = PROTO_UNSPECIFIED;
cpi->protocol_version = PROTO_VERSION_UNSPECIFIED;
cpi->transport = XPORT_UNSPECIFIED;
cpi->transport_version = XPORT_VERSION_UNSPECIFIED;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
cpi->ccb_h.status = CAM_REQ_CMP;
xpt_done(work_ccb);
break;
}
default:
work_ccb->ccb_h.status = CAM_REQ_INVALID;
xpt_done(work_ccb);
break;
}
}
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
/*
* The xpt as a "controller" has no interrupt sources, so polling
* is a no-op.
*/
static void
xptpoll(struct cam_sim *sim)
{
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
void
xpt_lock_buses(void)
{
mtx_lock(&xsoftc.xpt_topo_lock);
}
void
xpt_unlock_buses(void)
{
mtx_unlock(&xsoftc.xpt_topo_lock);
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct mtx *
xpt_path_mtx(struct cam_path *path)
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
return (&path->device->device_mtx);
Split the camisr into per-SIM done queues. This optimizes the locking a little bit and allows for direct dispatch of the doneq from certain contexts that would otherwise face recursive locking problems.
2007-04-19 14:28:43 +00:00
}
Provide locking for the ccb_bioq. This allows xpt_done() to be called without Giant held. In camisr(), move the ccb_bioq elements to a temporary local list and then process the elements off of that list. This enables the list to be processed by only taking the ccb_bioq_lock once and only for a very short time. ccb_bioq_lock is a leaf mutex, so it's fine to call xpt_done() with other locks held. This is just a very minor step in the work to lock CAM, but it allows us to avoid some messy locking/unlock dances in certain drivers.
2005-02-09 11:44:15 +00:00
Split the camisr into per-SIM done queues. This optimizes the locking a little bit and allows for direct dispatch of the doneq from certain contexts that would otherwise face recursive locking problems.
2007-04-19 14:28:43 +00:00
static void
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
xpt_done_process(struct ccb_hdr *ccb_h)
Split the camisr into per-SIM done queues. This optimizes the locking a little bit and allows for direct dispatch of the doneq from certain contexts that would otherwise face recursive locking problems.
2007-04-19 14:28:43 +00:00
{
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
struct cam_sim *sim;
struct cam_devq *devq;
struct mtx *mtx = NULL;
Split the camisr into per-SIM done queues. This optimizes the locking a little bit and allows for direct dispatch of the doneq from certain contexts that would otherwise face recursive locking problems.
2007-04-19 14:28:43 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (ccb_h->flags & CAM_HIGH_POWER) {
struct highpowerlist *hphead;
struct cam_ed *device;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_lock(&xsoftc.xpt_highpower_lock);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
hphead = &xsoftc.highpowerq;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
device = STAILQ_FIRST(hphead);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/*
* Increment the count since this command is done.
*/
xsoftc.num_highpower++;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/*
* Any high powered commands queued up?
*/
if (device != NULL) {
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
STAILQ_REMOVE_HEAD(hphead, highpowerq_entry);
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_unlock(&xsoftc.xpt_highpower_lock);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_lock(&device->sim->devq->send_mtx);
xpt_release_devq_device(device,
/*count*/1, /*runqueue*/TRUE);
mtx_unlock(&device->sim->devq->send_mtx);
} else
Some CAM locks polishing: - Fix LOR and possible lock recursion when handling high-power commands. Introduce new lock to protect left power quota and list of frozen devices. - Correct locking around xpt periph creation. - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
mtx_unlock(&xsoftc.xpt_highpower_lock);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
sim = ccb_h->path->bus->sim;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (ccb_h->status & CAM_RELEASE_SIMQ) {
xpt_release_simq(sim, /*run_queue*/FALSE);
ccb_h->status &= ~CAM_RELEASE_SIMQ;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if ((ccb_h->flags & CAM_DEV_QFRZDIS)
&& (ccb_h->status & CAM_DEV_QFRZN)) {
Move xpt_run_devq() call before request completion callback where it was originally. I am not sure why exactly have I moved it during one of many refactorings during camlock project, but obviously it opens race window that may cause use after free panics during SIM (in reported cases umass(4)) detach. MFC after: 2 weeks
2014-01-11 16:52:09 +00:00
xpt_release_devq(ccb_h->path, /*count*/1, /*run_queue*/TRUE);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
ccb_h->status &= ~CAM_DEV_QFRZN;
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
devq = sim->devq;
if ((ccb_h->func_code & XPT_FC_USER_CCB) == 0) {
struct cam_ed *dev = ccb_h->path->device;
mtx_lock(&devq->send_mtx);
devq->send_active--;
devq->send_openings++;
cam_ccbq_ccb_done(&dev->ccbq, (union ccb *)ccb_h);
if (((dev->flags & CAM_DEV_REL_ON_QUEUE_EMPTY) != 0
&& (dev->ccbq.dev_active == 0))) {
dev->flags &= ~CAM_DEV_REL_ON_QUEUE_EMPTY;
xpt_release_devq_device(dev, /*count*/1,
/*run_queue*/FALSE);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will use to synchornize and protect all data objects that are used for that SIM. Drivers that are not yet MPSAFE register Giant and operate as usual. RIght now, no drivers are MPSAFE, though a few will be changed in the coming week as this work settles down. The driver API has changed, so all CAM drivers will need to be recompiled. The userland API has not changed, so tools like camcontrol do not need to be recompiled.
2007-04-15 08:49:19 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (((dev->flags & CAM_DEV_REL_ON_COMPLETE) != 0
&& (ccb_h->status&CAM_STATUS_MASK) != CAM_REQUEUE_REQ)) {
dev->flags &= ~CAM_DEV_REL_ON_COMPLETE;
xpt_release_devq_device(dev, /*count*/1,
/*run_queue*/FALSE);
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if (!device_is_queued(dev))
(void)xpt_schedule_devq(devq, dev);
Move xpt_run_devq() call before request completion callback where it was originally. I am not sure why exactly have I moved it during one of many refactorings during camlock project, but obviously it opens race window that may cause use after free panics during SIM (in reported cases umass(4)) detach. MFC after: 2 weeks
2014-01-11 16:52:09 +00:00
xpt_run_devq(devq);
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
mtx_unlock(&devq->send_mtx);
style(9)
2008-01-02 01:45:31 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if ((dev->flags & CAM_DEV_TAG_AFTER_COUNT) != 0) {
mtx = xpt_path_mtx(ccb_h->path);
mtx_lock(mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Allow 5 untagged commands to go to a device before enabling tags after enabling transfer negotiations, a BDR, or a bus reset to allow the controller driver to negotiate without tagged messages getting in the way. Some devices are confused by attempts to negotiate and tag at the same time. Some controllers (e.g. BT MultiMaster with certain firmware revs) will never negotiate if you don't give them an untagged "window" to perform negotiation in. Bump the maximum tag count to 255. The system reclaims unused tag space as the tag count is dropped anyway, so we might as well try the max. We should probably use a larger type than u_int8_t to hold our tag value as SCSI over certain mediums allows for higher values. Reviewed by: Kenneth Merry <ken@FreeBSD.org>
1998-09-23 03:03:19 +00:00
if ((dev->flags & CAM_DEV_TAG_AFTER_COUNT) != 0
&& (--dev->tag_delay_count == 0))
xpt_start_tags(ccb_h->path);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
if ((ccb_h->flags & CAM_UNLOCKED) == 0) {
if (mtx == NULL) {
mtx = xpt_path_mtx(ccb_h->path);
mtx_lock(mtx);
}
} else {
if (mtx != NULL) {
mtx_unlock(mtx);
mtx = NULL;
style(9)
2008-01-02 01:45:31 +00:00
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
}
Clean up the xpt_sim creation to mirror how HBA drivers perform this task. Clean up node creation in the EDT so that initialization follows member declaration. Sort registered paths by pathid so that we probe busses in order of ascending pathid. This makes hardwiring of busses without wiring individual targets do what the user expects. (submitted by tegge@FreeBSD.org) Fix an EDT node leak. Target nodes would never go away. Implement xpt_bus_deregister(). (prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/* Call the peripheral driver's callback */
Move CAM_UNQUEUED_INDEX setting to the last moment and under the periph lock. This fixes race condition with cam_periph_ccbwait(), causing use-after-free.
2013-10-23 12:53:05 +00:00
ccb_h->pinfo.index = CAM_UNQUEUED_INDEX;
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
(*ccb_h->cbfcnp)(ccb_h->path->periph, (union ccb *)ccb_h);
if (mtx != NULL)
mtx_unlock(mtx);
}
void
xpt_done_td(void *arg)
{
struct cam_doneq *queue = arg;
struct ccb_hdr *ccb_h;
STAILQ_HEAD(, ccb_hdr) doneq;
STAILQ_INIT(&doneq);
mtx_lock(&queue->cam_doneq_mtx);
while (1) {
while (STAILQ_EMPTY(&queue->cam_doneq)) {
queue->cam_doneq_sleep = 1;
msleep(&queue->cam_doneq, &queue->cam_doneq_mtx,
PRIBIO, "-", 0);
queue->cam_doneq_sleep = 0;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
STAILQ_CONCAT(&doneq, &queue->cam_doneq);
mtx_unlock(&queue->cam_doneq_mtx);
THREAD_NO_SLEEPING();
while ((ccb_h = STAILQ_FIRST(&doneq)) != NULL) {
STAILQ_REMOVE_HEAD(&doneq, sim_links.stqe);
xpt_done_process(ccb_h);
}
THREAD_SLEEPING_OK();
mtx_lock(&queue->cam_doneq_mtx);
}
}
static void
camisr_runqueue(void)
{
struct ccb_hdr *ccb_h;
struct cam_doneq *queue;
int i;
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
Merge CAM locking changes from the projects/camlock branch to radically reduce lock congestion and improve SMP scalability of the SCSI/ATA stack, preparing the ground for the coming next GEOM direct dispatch support. Replace big per-SIM locks with bunch of smaller ones: - per-LUN locks to protect device and peripheral drivers state; - per-target locks to protect list of LUNs on target; - per-bus locks to protect reference counting; - per-send queue locks to protect queue of CCBs to be sent; - per-done queue locks to protect queue of completed CCBs; - remaining per-SIM locks now protect only HBA driver internals. While holding LUN lock it is allowed (while not recommended for performance reasons) to take SIM lock. The opposite acquisition order is forbidden. All the other locks are leaf locks, that can be taken anywhere, but should not be cascaded. Many functions, such as: xpt_action(), xpt_done(), xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM lock to be held. To keep compatibility and solve cases where SIM lock can't be dropped, all xpt_async() calls in addition to xpt_done() calls are queued to completion threads for async processing in clean environment without SIM lock held. Instead of single CAM SWI thread, used for commands completion processing before, use multiple (depending on number of CPUs) threads. Load balanced between them using "hash" of the device B:T:L address. HBA drivers that can drop SIM lock during completion processing and have sufficient number of completion threads to efficiently scale to multiple CPUs can use new function xpt_done_direct() to avoid extra context switch. Make ahci(4) driver to use this mechanism depending on hardware setup. Sponsored by: iXsystems, Inc. MFC after: 2 months
2013-10-21 12:00:26 +00:00
/* Process global queues. */
for (i = 0; i < cam_num_doneqs; i++) {
queue = &cam_doneqs[i];
mtx_lock(&queue->cam_doneq_mtx);
while ((ccb_h = STAILQ_FIRST(&queue->cam_doneq)) != NULL) {
STAILQ_REMOVE_HEAD(&queue->cam_doneq, sim_links.stqe);
mtx_unlock(&queue->cam_doneq_mtx);
xpt_done_process(ccb_h);
mtx_lock(&queue->cam_doneq_mtx);
}
mtx_unlock(&queue->cam_doneq_mtx);
CAM Transport Layer (XPT). Submitted by: The CAM Team
1998-09-15 06:33:23 +00:00
}
}
Reference in New Issue Copy Permalink