802d215dc1
17 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Scott Long
|
fe8391035a |
Rate limit the 'out of chain frame' messages to once per 60 seconds.
Obtained from: Netflix MFC after: 3 days |
||
|
Scott Long
|
d9802deb4e |
Sometimes a device misbehaves so badly that it disrupts the entire system.
Add a tunable that allows such a device to be excluded from the driver. The id parameter is the target id that the driver assigns to a given device. dev.mps.X.exclude_ids=<id>,<id> Obtained from: Netflix MFC after: 3 days |
||
Kenneth D. Merry
|
9b91b192fb |
Merge in phase 14+ -> 16 mps driver fixes from LSI:
--------------------------------------------------------------- System panics during a Port reset with ouststanding I/O --------------------------------------------------------------- It is possible to call mps_mapping_free_memory after this memory is already freed, causing a panic. Removed this extra call to mps_mappiing_free_memory and call mps_mapping_exit in place of the mps_mapping_free_memory call so that any outstanding mapping items can be flushed before memory is freed. --------------------------------------------------------------- Correct memory leak during a Port reset with ouststanding I/O --------------------------------------------------------------- In mps_reinit function, the mapping memory was not being freed before being re-allocated. Added line to call the memory free function for mapping memory. --------------------------------------------------------------- Use CAM_SIM_QUEUED flag in Driver IO path. --------------------------------------------------------------- This flag informs the XPT that successful abort of a CCB requires an abort ccb to be issued to the SIM. While processing SCSI IO's, set the CAM_SIM_QUEUED flag in the status for the IO. When the command completes, clear this flag. --------------------------------------------------------------- Check for CAM_REQ_INPROG in I/O path. --------------------------------------------------------------- Added a check in mpssas_action_scsiio for the In Progress status for the IO. If this flag is set, the IO has already been aborted by the upper layer (before CAM_SIM_QUEUED was set) and there is no need to send the IO. The request will be completed without error. --------------------------------------------------------------- Improve "doorbell handshake method" for mps_get_iocfacts --------------------------------------------------------------- Removed call to get Port Facts since this information is not used currently. Added mps_iocfacts_allocate function to allocate memory that is based on IOC Facts data. Added mps_iocfacts_free function to free memory that is based on IOC Facts data. Both of the functions are used when a Diag Reset is performed or when the driver is attached/detached. This is needed in case IOC Facts changes after a Diag Reset, which could happen if FW is upgraded. Moved call of mps_bases_static_config_pages from the attach routine to after the IOC is ready to process accesses based on the new memory allocations (instead of polling through the Doorbell). --------------------------------------------------------------- Set TimeStamp in INIT message in millisecond format Set the IOC --------------------------------------------------------------- --------------------------------------------------------------- Prefer mps_wait_command to mps_request_polled --------------------------------------------------------------- Instead of using mps_request_polled, call mps_wait_command whenever possible. Change the mps_wait_command function to check the current context and either use interrupt context or poll if required by using the pause or DELAY function. Added a check after waiting 50mSecs to see if the command has timed out. This is only done if polliing, the msleep command will automatically timeout if the command has taken too long to complete. --------------------------------------------------------------- Integrated RAID: Volume Activation Failed error message is displayed though the volume has been activated. --------------------------------------------------------------- Instead of failing an IOCTL request that does not have a large enough buffer to hold the complete reply, copy as much data from the reply as possible into the user's buffer and log a message saying that the user's buffer was smaller than the returned data. --------------------------------------------------------------- mapping_add_new_device failure due to persistent table FULL --------------------------------------------------------------- When a new device is added, if it is determined that the device persistent table is being used and is full, instead of displaying a message for this condition every time, only log a message if the MPS_INFO bit is set in the debug_flags. Submitted by: LSI MFC after: 1 week |
||
|
Scott Long
|
1610f95c56 |
Overhaul error, information, and debug logging.
Obtained from: Netflix MFC after: 3 days |
||
Konstantin Belousov
|
dd0b4fb6d5 |
Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the bus_dmamap_load_buffer() routines so that they may be called from MI code. The MD busdma is then given a chance to do any final processing in the complete() callback. The cam changes unify the bus_dmamap_load* handling in cam drivers. The arm and mips implementations are updated to track virtual addresses for sync(). Previously this was done in a type specific way. Now it is done in a generic way by recording the list of virtuals in the map. Submitted by: jeff (sponsored by EMC/Isilon) Reviewed by: kan (previous version), scottl, mjacob (isp(4), no objections for target mode changes) Discussed with: ian (arm changes) Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris), amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>) |
||
Alexander Motin
|
61c49b4dd1 |
Several fixes to allow firmware/BIOS flash access from user-level:
- remove special handling of zero length transfers in mpi_pre_fw_upload(); - add missing MPS_CM_FLAGS_DATAIN flag in mpi_pre_fw_upload(); - move mps_user_setup_request() call into proper place; - increase user command timeout from 30 to 60 seconds; - avoid NULL dereference panic in case of firmware crash. Set max DMA segment size to 24bit, as MPI SGE supports it. Use mps_add_dmaseg() to add empty SGE instead of custom code. Tune endianness safety. Reviewed by: Desai, Kashyap <Kashyap.Desai@lsi.com> Sponsored by: iXsystems, Inc. |
||
|
Kenneth D. Merry
|
be4aa869c1 |
Bring in LSI's latest mps(4) 6Gb SAS and WarpDrive driver, version
14.00.00.01-fbsd. Their description of the changes is as follows: 1. Copyright contents has been changed in all respective .c and .h files 2. Support for WRITE12 and READ12 for direct-io (warpdrive only) has been added. 3. Driver has added checks to see if Drive has READ_CAP_16 support before sending it down to the device. If SPC3_SID_PROTECT flag is set in the inquiry data, the device supports protection information, and must support the 16 byte read capacity command, otherwise continue without sending read cap 16. This will optimize driver performance, since it will not send READ_CAP_16 to the drive which does not have support of READ_CAP_16. 4. With new approach, "MPTIOCTL_RESET_ADAPTER" IOCTL will not use DELAY() which is busy loop implementation. It will use <msleep> (Better way to sleep without busy loop). Also from the HBA reset code path and some other places, DELAY() is replaced with msleep() or "pause()", which is based on sleep/wakeup style calls. Driver use msleep()/pause() instead of DELAY based on CAN_SLEEP/NO_SLEEP flags to avoid busy loop which is not required all the time.e.a a. While driver is getting loaded, driver calls most of the commands with NO_SLEEP. b. When Driver is functional and it needs Reinit of HBA, CAN_SLEEP flag is used. 5. <mpslsi> driver is not Endian safe. It will not work on Big Endian machines like Sparc and PowerPC platforms because it assumes it is running on a Little Endian machine. Driver code is modified such way that it does not assume CPU arch is Little Endian. a. All places where Driver interacts from HBA to Host, it converts Little Endian format to CPU format. b. All places where Driver interacts from Host to HBA, it converts CPU format to Little Endian. 6. Findout memory leaks in FreeBSD Driver and resolve those, such as memory leak in targ's luns creation/deletion. Also added additional checks to see memory allocation success/fail. 7. Add loginfo prints as debug message, i.e. When FW sends any loginfo, Driver should print those as debug message. This will help for debugging purpose. 8. There is possibility to get config request timeout. Current driver is able to detect config request timetout, but it does not do anything on config_request timeout. Driver should call mps_reinit() if any request_poll (which is called as part of config_request) is time out. 9. cdb length check is required for 32 byte CDB. Add correct mpi control value for 32 bit CDB as below while submitting SCSI IO Request to controller. mpi_control |= 4 << MPI2_SCSIIO_CONTROL_ADDCDBLEN_SHIFT; 10. Check the actual status of Message unit reset (mps_message_unit_reset).Previously FreeBSD Driver just writes MPI2_FUNCTION_IOC_MESSAGE_UNIT_RESET and never check the ack (it just wait for 50 millisecond). So, Driver now check the status of "MPI2_FUNCTION_IOC_MESSAGE_UNIT_RESET" after writing it to the FW. Now it also checking for whether doorbell ack uses msleep with proper sleep flags, instead of <DELAY>. 11. Previously CAM does not detect Multi-Lun Devices. In order to detect Multi-Lun Devices by CAM the driver needs following change set: a. There is "max_lun" field which Driver need to set based on hw/fw support. Currently LSI released driver does not set this field. b. Default of "max_lun" should not be 0 in OS, but it is currently set to 0 in CAM layer. c. Export max_lun capacity to 255 12. Driver will not reset target info after port enable complete and also do Device removal when Device remove from FW. The detail description is as follows a. When Driver receive WD PD add events, it will add all information in driver local data structure. b. Only for WD, we have below checks after port enable completes, where driver clear off all information retrieved at #1. if ((sc->WD_available && (sc->WD_hide_expose == MPS_WD_HIDE_ALWAYS)) || (sc->WD_valid_config && (sc->WD_hide_expose == MPS_WD_HIDE_IF_VOLUME)) { // clear off target data structure. } It is mainly not to attach PDs to OS. FreeBSD does bus rescan as older Parallel scsi style. So Driver needs to handle which Drive is visible to OS. That is a reason we have to clear off targ information for PDs. Again, above logic was implemented long time ago. Similar concept we have for non-wd also. For that, LSI have introduced different logic to hide PDs. Eventually, because of above gap, when Phy goes offline, we observe below failure. That is what Driver is not doing complete removal of device with FW. (which was pointed by Scott) Apr 5 02:39:24 Freebsd7 kernel: mpslsi0: mpssas_prepare_remove Apr 5 02:39:24 Freebsd7 kernel: mpssas_prepare_remove 497 : invalid handle 0xe Now Driver will not reset target info after port enable complete and also will do Device removal when Device remove from FW. 13. Returning "CAM_SEL_TIMEOUT" instead of "CAM_TID_INVALID" error code on request to the Target IDs that have no devices conected at that moment. As if "CAM_TID_INVALID" error code is returned to the CAM Layaer then it results in a huge chain of errors in verbose kernel messages on boot and every hot-plug event. Submitted by: Sreekanth Reddy <Sreekanth.Reddy@lsi.com> MFC after: 3 days |
||
|
Kenneth D. Merry
|
653c521f8d |
Bring in a number of mps(4) driver fixes from LSI:
1. Fixed timeout specification for the msleep in mps_wait_command().
Added 30 second timeout for mps_wait_command() calls in mps_user.c.
2. Make sure we call mps_detach_user() from the kldunload path.
3. Raid Hotplug behavior change.
The driver now removes a volume when it goes to a failed state,
so we also need to add volume back to the OS when it goes to
opitimal/degraded/online from failed/missing.
Handle raid volume add and remove from the IR_Volume event.
4. Added some more debugging information.
5. Replace xpt_async(AC_LOST_DEVICE, path, NULL) with
mpssas_rescan_target().
This is to work around a panic in CAM that shows up when adding a
drive with a rescan and removing another device from the driver thread
with an AC_LOST_DEVICE async notification.
This problem was encountered in testing with the LSI sas2ircu utility,
which was used to create a RAID volume from physical disks. The driver
has to create the RAID volume target and remove the physical disk
targets, and triggered a panic in the process.
The CAM issue needs to be fully diagnosed and fixed, but this works
around the issue for now.
6. Fix some memory initialization issues in mps_free_command().
7. Resolve the "devq freeze forever" issue. This was caused by the
internal read capacity command issued in the non-head version of the
driver. When the command completed with an error, the driver wasn't
unfreezing thd device queue.
The version in head uses the CAM infrastructure for getting the read
capacity information, and therefore doesn't have the same issue.
8. Bump the version to 13.00.00.00-fbsd. (this is very close to LSI's
internal stable driver 13.00.00.00)
Submitted by: Kashyap Desai <Kashyap.Desai@lsi.com>
MFC after: 3 days
|
||
|
Kenneth D. Merry
|
d043c56453 |
Bring in the LSI-supported version of the mps(4) driver.
This involves significant changes to the mps(4) driver, but is not a complete rewrite. Some of the changes in this version of the driver: - Integrated RAID (IR) support. - Support for WarpDrive controllers. - Support for SCSI protection information (EEDP). - Support for TLR (Transport Level Retries), needed for tape drives. - Improved error recovery code. - ioctl interface compatible with LSI utilities. mps.4: Update the mps(4) driver man page somewhat for the driver changes. The list of supported hardware still needs to be updated to reflect the full list of supported cards. conf/files: Add the new driver files. mps/mpi/*: Updated version of the MPI header files, with a BSD style copyright. mps/*: See above for a description of the new driver features. modules/mps/Makefile: Add the new mps(4) driver files. Submitted by: Kashyap Desai <Kashyap.Desai@lsi.com> Reviewed by: ken MFC after: 1 week |
||
|
Kenneth D. Merry
|
750ffe843d |
Silence 'out of chain frames' warnings and bump the number of frames.
mps.c: Hide the 'out of chain frames' warning behind MPS_INFO. mps_sas.c: Hide the SIM queue freeze/unfreeze messages behind MPS_INFO. mpsvar.h: Bump the number of chain frames from 1024 to 2048. From testing, it looks like this makes it less likely that we'll run out of chain frames, and it doesn't cost much memory (32K). MFC after: 3 days |
||
|
Kenneth D. Merry
|
550e2acdfc |
Fix several issues with the mps(4) driver.
When the driver ran out of DMA chaining buffers, it kept the timeout for the I/O, and I/O would stall. The driver was not freezing the device queue on errors. mps.c: Pull command completion logic into a separate function, and call the callback/wakeup for commands that are never sent due to lack of chain buffers. Add a number of extra diagnostic sysctl variables. Handle pre-hardware errors for configuration I/O. This doesn't panic the system, but it will fail the configuration I/O and there is no retry mechanism. So the device probe will not succeed. This should be a very uncommon situation, however. mps_sas.c: Freeze the SIM queue when we run out of chain buffers, and unfreeze it when more commands complete. Freeze the device queue when errors occur, so that CAM can insure proper command ordering. Report pre-hardware errors for task management commands. In general, that shouldn't be possible because task management commands don't have S/G lists, and that is currently the only error path before we get to the hardware. Handle pre-hardware errors (like out of chain elements) for SMP requests. That shouldn't happen either, since we should have enough space for two S/G elements in the standard request. For commands that end with MPI2_IOCSTATUS_SCSI_IOC_TERMINATED and MPI2_IOCSTATUS_SCSI_EXT_TERMINATED, return them with CAM_REQUEUE_REQ to retry them unconditionally. These seem to be related to back end, transport related problems that are hopefully transient. We don't want to go through the retry count for something that is not a permanent error. Keep track of the number of outstanding I/Os. mpsvar.h: Track the number of free chain elements. Add variables for the number of outstanding I/Os, and I/O high water mark. Add variables to track the number of free chain buffers and the chain low water mark, as well as the number of chain allocation failures. Add I/O state flags and an attach done flag. MFC after: 3 days |
||
|
Kenneth D. Merry
|
9866848a62 |
In the MPS driver, during device removal processing, don't assume that
the controller firmware will return all of our commands. Instead, keep track of outstanding I/Os and return them to CAM once device removal processing completes. mpsvar.h: Declare the new "io_list" in the mps_softc. mps.c: Initialize the new "io_list" in the mps softc. mps_sas.c: o Track SCSI I/O requests on the io_list from the time of mpssas_action() through mpssas_scsiio_complete(). o Zero out the request structures used for device removal commands prior to filling them out. o Once the target reset task management function completes during device removal processing, assume any SCSI I/O commands that are still oustanding will never return from the controller, and process them manually. Submitted by: gibbs MFC after: 3 days |
||
|
Kenneth D. Merry
|
49dfe4a2d4 |
Fix an event handling bug with the mps(4) driver.
This bug manifested itself after repeated device arrivals and departures. The root of the problem was that the last entry in the reply array wasn't initialized/allocated. So every time we got around to that event, we had a bogus address. There were a couple more problems with the code that are also fixed: - The reply mechanism was being treated as sequential (indexed by sc->replycurindex) even though the spec says that the driver should use the ReplyFrameAddress field of the post queue descriptor to figure out where the reply is. There is no guarantee that the reply descriptors will be used in sequential order. - The second word of the reply post queue descriptor wasn't being checked in mps_intr_locked() to make sure that it wasn't 0xffffffff. So the driver could potentially come across a partially DMAed descriptor. - The number of replies allocated was one less than the actual size of the queue. Instead, it was the size of the number of replies that can be used at one time. (Which is one less than the size of the queue.) mps.c: When initializing the entries in the reply free queue, make sure we initialize the full number that we tell the chip we have (sc->fqdepth), not the number that can be used at any one time (sc->num_replies). When allocating replies, make sure we allocate the number of replies that we've told the chip exist, not just the number that can be used simultaneously. Use the ReplyFrameAddress field of the post queue descriptor to figure out which reply is being referenced. This is what the spec says to do, and the spec doesn't guarantee that the replies will be used in order. Put a check in to verify that the reply address passed back from the card is valid. (Panic if it isn't, we'll panic when we try to deference the reply pointer in any case.) In mps_intr_locked(), verify that the second word of the post queue descriptor is not 0xffffffff in addition to verifying that the unused flag is not set, so we can make sure we didn't get a partially DMAed descriptor. Remove references to sc->replycurindex, it isn't needed now. mpsvar.h: Remove replycurindex from the softc, it isn't needed now. Reviewed by: scottl |
||
|
Kenneth D. Merry
|
06e794928b |
Add Serial Management Protocol (SMP) passthrough support to CAM.
This includes support in the kernel, camcontrol(8), libcam and the mps(4) driver for SMP passthrough. The CAM SCSI probe code has been modified to fetch Inquiry VPD page 0x00 to determine supported pages, and will now fetch page 0x83 in addition to page 0x80 if supported. Add two new CAM CCBs, XPT_SMP_IO, and XPT_GDEV_ADVINFO. The SMP CCB is intended for SMP requests and responses. The ADVINFO is currently used to fetch cached VPD page 0x83 data from the transport layer, but is intended to be extensible to fetch other types of device-specific data. SMP-only devices are not currently represented in the CAM topology, and so the current semantics are that the SIM will route SMP CCBs to either the addressed device, if it contains an SMP target, or its parent, if it contains an SMP target. (This is noted in cam_ccb.h, since it will change later once we have the ability to have SMP-only devices in CAM's topology.) smp_all.c, smp_all.h: New helper routines for SMP. This includes SMP request building routines, response parsing routines, error decoding routines, and structure definitions for a number of SMP commands. libcam/Makefile: Add smp_all.c to libcam, so that SMP functionality is available to userland applications. camcontrol.8, camcontrol.c: Add smp passthrough support to camcontrol. Several new subcommands are now available: 'smpcmd' functions much like 'cmd', except that it allows the user to send generic SMP commands. 'smprg' sends the SMP report general command, and displays the decoded output. It will automatically fetch extended output if it is available. 'smppc' sends the SMP phy control command, with any number of potential options. Among other things, this allows the user to reset a phy on a SAS expander, or disable a phy on an expander. 'smpmaninfo' sends the SMP report manufacturer information and displays the decoded output. 'smpphylist' displays a list of phys on an expander, and the CAM devices attached to those phys, if any. cam.h, cam.c: Add a status value for SMP errors (CAM_SMP_STATUS_ERROR). Add a missing description for CAM_SCSI_IT_NEXUS_LOST. Add support for SMP commands to cam_error_string(). cam_ccb.h: Rename the CAM_DIR_RESV flag to CAM_DIR_BOTH. SMP commands are by nature bi-directional, and we may need to support bi-directional SCSI commands later. Add the XPT_SMP_IO CCB. Since SMP commands are bi-directional, there are pointers for both the request and response. Add a fill routine for SMP CCBs. Add the XPT_GDEV_ADVINFO CCB. This is currently used to fetch cached page 0x83 data from the transport later, but is extensible to fetch many other types of data. cam_periph.c: Add support in cam_periph_mapmem() for XPT_SMP_IO and XPT_GDEV_ADVINFO CCBs. cam_xpt.c: Add support for executing XPT_SMP_IO CCBs. cam_xpt_internal.h: Add fields for VPD pages 0x00 and 0x83 in struct cam_ed. scsi_all.c: Add scsi_get_sas_addr(), a function that parses VPD page 0x83 data and pulls out a SAS address. scsi_all.h: Add VPD page 0x00 and 0x83 structures, and a prototype for scsi_get_sas_addr(). scsi_pass.c: Add support for mapping buffers in XPT_SMP_IO and XPT_GDEV_ADVINFO CCBs. scsi_xpt.c: In the SCSI probe code, first ask the device for VPD page 0x00. If any VPD pages are supported, that page is required to be implemented. Based on the response, we may probe for the serial number (page 0x80) or device id (page 0x83). Add support for the XPT_GDEV_ADVINFO CCB. sys/conf/files: Add smp_all.c. mps.c: Add support for passing in a uio in mps_map_command(), so we can map a S/G list at once. Add support for SMP passthrough commands in mps_data_cb(). SMP is a special case, because the first buffer in the S/G list is outbound and the second buffer is inbound. Add support for warning the user if the busdma code comes back with more buffers than will work for the command. This will, for example, help the user determine why an SMP command failed if busdma comes back with three buffers. mps_pci.c: Add sys/uio.h. mps_sas.c: Add the SAS address and the parent handle to the list of fields we pull from device page 0 and cache in struct mpssas_target. These are needed for SMP passthrough. Add support for the XPT_SMP_IO CCB. For now, this CCB is routed to the addressed device if it supports SMP, or to its parent if it does not and the parent does. This is necessary because CAM does not currently support SMP-only nodes in the topology. Make SMP passthrough support conditional on __FreeBSD_version >= 900026. This will make it easier to MFC this change to the driver without MFCing the CAM changes as well. mps_user.c: Un-staticize mpi_init_sge() so we can use it for the SMP passthrough code. mpsvar.h: Add a uio and iovecs into struct mps_command for SMP passthrough commands. Add a cm_max_segs field to struct mps_command so that we can warn the user if busdma comes back with too many segments. Clear the cm_reply when a command gets freed. If it is not cleared, reply frames will eventually get freed into the pool multiple times and corrupt the pool. (This fix is from scottl.) Add a prototype for mpi_init_sge(). sys/param.h: Bump __FreeBSD_version to 900026 for the for the inclusion of the XPT_GDEV_ADVINFO and XPT_SMP_IO CAM CCBs. |
||
Matthew D Fleming
|
13ac67a784 |
Re-work the internals of adding items to the driver's scatter-gather
list. Use the new internals to simplify adding transaction context elements, and in future diffs, more complicated SGLs. |
||
|
Kenneth D. Merry
|
1476ba4087 |
Turn on serialization of task management commands going down to the
controller, but make it optional. After a problem report from Andrew Boyer, it looks like the LSI chip may have issues (the watchdog timer fired) if too many aborts are sent down to the chip at the same time. We know that task management commands are serialized, and although the manual doesn't say it, it may be a good idea to just send one at a time. But, since I'm not certain that this is necessary, add a tunable and sysctl variable (hw.mps.%d.allow_multiple_tm_cmds) to control the driver's behavior. mps.c: Add support for the sysctl and tunable, and add a comment about the possible return values to mps_map_command(). mps_sas.c: Run all task management commands through two new routines, mpssas_issue_tm_request() and mpssas_complete_tm_request(). This allows us to optionally serialize task management commands. Also, change things so that the response to a task management command always comes back through the callback. (Before it could come via the callback or the return value.) mpsvar.h: Add softc variables for the list of active task management commands, the number of active commands, and whether we should allow multiple active task management commands. Add an active command flag. mps.4: Describe the new sysctl/loader tunable variable. Sponsored by: Spectra Logic Corporation |
||
|
Kenneth D. Merry
|
d3c7b9a08a |
MFp4 (//depot/projects/mps/...)
Bring in a driver for the LSI Logic MPT2 6Gb SAS controllers. This driver supports basic I/O, and works with SAS and SATA drives and expanders. Basic error recovery works (i.e. timeouts and aborts) as well. Integrated RAID isn't supported yet, and there are some known bugs. So this isn't ready for production use, but is certainly ready for testing and additional development. For the moment, new commits to this driver should go into the FreeBSD Perforce repository first (//depot/projects/mps/...) and then get merged into -current once they've been vetted. This has only been added to the amd64 GENERIC, since that is the only architecture I have tested this driver with. Submitted by: scottl Discussed with: imp, gibbs, will Sponsored by: Yahoo, Spectra Logic Corporation |