AC_ADVINFO_CHANGED.
Without this change, newly inserted hard disks won't always have their
physical path device nodes created. The problem reproduces most readily
when attaching a large number of disks at once.
Differential Revision: https://reviews.freebsd.org/D2290
Reviewed by: mav, imp
MFC after: 2 weeks
Sponsored by: Spectra Logic
There are four places, all in cam_xpt.c, where ccbs are malloc'ed. Two of
these use M_ZERO, two don't. The two that don't meant that allocated ccbs
had trash in them making it hard to debug errors where they showed up. Due
to this, use M_ZERO all the time when allocating ccbs.
Submitted by: Scott Ferris <scott.ferris@isilon.com>
Sponsored by: EMC/Isilon Storage Division
Reviewed by: scottl, imp
Differential: https://reviews.freebsd.org/D2120
The only drives I have discovered so far that support medium type
reports are newer HP LTO (LTO-5 and LTO-6) drives. IBM drives
only support the density reports.
sys/cam/scsi/scsi_sa.h:
The number of possible density codes in the medium type
report is 9, not 8. This caused problems parsing all of
the medium type report after this point in the structure.
usr.bin/mt/mt.c:
Run the density codes returned in the medium type report
through denstostring(), just like the primary and secondary
density codes in the density report. This will print the
density code in hex, and give a text description if it
is available.
Thanks to Rudolf Cejka for doing extensive testing with HP LTO drives
and Bacula and discovering these problems.
Tested by: Rudolf Cejka <cejkar at fit.vutbr.cz>
Sponsored by: Spectra Logic
MFC after: 4 days
SCSI-2 devices.
Some older tape devices claim to be SCSI-2, but actually do support
long position information. (Long position information includes
the current file mark.) For example, the COMPAQ SuperDLT1.
So we now only disable the check on SCSI-1 and older devices.
sys/cam/scsi/scsi_sa.c:
In saregister(), only disable fetching long position
information on SCSI-1 and older drives. Update the
comment to explain why.
Confirmed by: dvl
Sponsored by: Spectra Logic
MFC after: 3 weeks
tracking.
It is important to subtract the residual from the requested
transfer size to see how much data was actually transferred. With
tape drives in particular, it is common to request more data than is
returned.
Also, add I/O latency tracking for CAM requests issued by
cam_periph_runccb().
If the caller supplies a struct devstat, and the I/O is a SCSI or
ATA I/O, we will track the elapsed time to provide I/O latency
statistics for the request.
sys/cam/scsi/cam_periph.c:
In cam_periph_runccb(), subtract the residual when reporting I/O
totals to devstat(9) for SCSI and ATA passthrough requests.
In cam_periph_runccb(), grab the I/O start time and supply
the start time to devstat_end_transaction() so that it can
calculate the elapsed I/O time.
Sponsored by: Spectra Logic
MFC after: 1 week
The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.
Significant changes and new features include:
o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.
o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.
o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.
The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.
This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.
For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.
o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.
o New, extensible tape driver parameter get/set interface.
o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.
o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.
o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.
o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.
o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.
o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2
contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.
lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.
This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.
rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.
src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.
src/share/man/man4/sa.4
Update BUGS and maintainer section.
sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.
sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.
Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.
Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.
Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.
Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.
Add a new MTIOCRBLIM ioctl to get block limits information.
Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().
scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.
Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.
For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.
Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.
Add a partition number to the sa(4) softc.
Improve device departure handling. The previous implementation
led to hangs when the device was open.
If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.
So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.
Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.
Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().
Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.
Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.
This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.
However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.
The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.
sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.
This includes definitions for the following new ioctls:
MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */
usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.
usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.
usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.
Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.
The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.
The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.
Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.
Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.
Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.
Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.
Sponsored by: Spectra Logic
MFC after: 1 month
properly.
If there is garbage in the flags field, it can sometimes include a
set CDAI_FLAG_STORE flag, which may cause either an error or
perhaps result in overwriting the field that was intended to be
read.
sys/cam/cam_ccb.h:
Add a new flag to the XPT_DEV_ADVINFO CCB, CDAI_FLAG_NONE,
that callers can use to set the flags field when no store
is desired.
sys/cam/scsi/scsi_enc_ses.c:
In ses_setphyspath_callback(), explicitly set the
XPT_DEV_ADVINFO flags to CDAI_FLAG_NONE when fetching the
physical path information. Instead of ORing in the
CDAI_FLAG_STORE flag when storing the physical path, set
the flags field to CDAI_FLAG_STORE.
sys/cam/scsi/scsi_sa.c:
Set the XPT_DEV_ADVINFO flags field to CDAI_FLAG_NONE when
fetching extended inquiry information.
sys/cam/scsi/scsi_da.c:
When storing extended READ CAPACITY information, set the
XPT_DEV_ADVINFO flags field to CDAI_FLAG_STORE instead of
ORing it into a field that isn't initialized.
sys/dev/mpr/mpr_sas.c,
sys/dev/mps/mps_sas.c:
When fetching extended READ CAPACITY information, set the
XPT_DEV_ADVINFO flags field to CDAI_FLAG_NONE instead of
setting it to 0.
sbin/camcontrol/camcontrol.c:
When fetching a device ID, set the XPT_DEV_ADVINFO flags
field to CDAI_FLAG_NONE instead of 0.
sys/sys/param.h:
Bump __FreeBSD_version to 1100061 for the new XPT_DEV_ADVINFO
CCB flag, CDAI_FLAG_NONE.
Sponsored by: Spectra Logic
MFC after: 1 week
This change by 2-3 times improves performance of misaligned XCOPY and WUT
commands by avoiding unneeded read-modify-write cycles inside ZFS.
MFC after: 1 week
This change by 2-3 times improves performance of misaligned WRITE SAME
commands by avoiding unneeded read-modify-write cycles inside ZFS.
MFC after: 1 week
target iSCSI offload. Add mechanism to query maximum receive data segment
size supported by chosen hardware offload module, and use it in ctld(8)
to determine the value to advertise to the other side.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
This VPD page is effectively an extension of the standard Inquiry
data page, and includes lots of additional bits.
This commit includes support for probing the page in the SCSI probe code,
and an additional request type for the XPT_DEV_ADVINFO CCB. CTL already
supports the Extended Inquiry page.
Support for querying this page in the sa(4) driver will come later.
sys/cam/scsi/scsi_xpt.c:
Probe the Extended Inquiry page, if the device supports it, and
return it in response to a XPT_DEV_ADVINFO CCB if it is requested.
sys/cam/scsi/cam_ccb.h:
Define a new advanced information CCB data type, CDAI_TYPE_EXT_INQ.
sys/cam/cam_xpt.c:
Free the extended inquiry data in a device when the device goes
away.
sys/cam/cam_xpt_internal.h:
Add an extended inquiry data pointer and length to struct cam_ed.
sys/sys/param.h
Bump __FreeBSD_version for the addition of the new
CDAI_TYPE_EXT_INQ advanced information type.
Sponsored by: Spectra Logic
MFC after: 1 week
While ctld(8) still does not allow multiple portal groups per target
to be configured, kernel should now be able to handle it.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
VMware returns BUSY status when storage has transient connectivity issues.
It is often better to wait and let VM admin fix the problem then crash.
Discussed with: ken
MFC after: 1 week
Replace iSCSI-specific LUN mapping mechanism with new one, working for any
ports. By default all ports are created without LUN mapping, exposing all
CTL LUNs as before. But, if needed, LUN mapping can be manually set on
per-port basis via ctladm. For its iSCSI ports ctld does it via ioctl(2).
The next step will be to teach ctld to work with FibreChannel ports also.
Respecting additional flexibility of the new mechanism, ctl.conf now allows
alternative syntax for LUN definition. LUNs can now be defined in global
context, and then referenced from targets by unique name, as needed. It
allows same LUN to be exposed several times via multiple targets.
While there, increase limit for LUNs per target in ctld from 256 to 1024.
Some initiators do not support LUNs above 255, but that is not our problem.
Discussed with: trasz
MFC after: 2 weeks
Relnotes: yes
Sponsored by: iXsystems, Inc.
sys/cam/scsi/scsi_all.h:
In struct scsi_extended_inquiry_data:
- Increase the length field to 2 bytes, as it is 2 bytes in SPC-4.
- Add bit definitions for the various Activiate Microcode actions.
- Add the Sequential Access Logical Block Protection support bit,
since we need that in the sa(4) driver. (For modifications
that will come later.)
- Add definitions for the various Multi I_T Nexus Microcode
Download modes.
sys/cam/ctl/ctl.c:
As of SPC-4, a single report of "REPORTED LUNS DATA HAS CHANGED"
is to be given per I_T nexus. Once it is reported, the unit
attention condition should be cleared for all LUNS attached to
an I_T nexus.
Previously that only happened when a REPORT LUNS command was
processed.
This behavior may be different (according to SAM-5) when the
UA_INTLCK_CTRL bits are non-zero in the control mode page but
CTL does not currently support that.
So, in view of the spec, whenever we report a LUN inventory
change unit attention, clear it on all LUNs for that
particular I_T nexus.
Add a new function, ctl_clear_ua() that will clear a unit
attention on all LUNs for the given I_T nexus.
One field in the extended inquiry data that we could potentially
report at some point is the maximum supported sense data length.
To do that, we would the SIM to report (via path inquiry
perhaps) how much sense data it is able to send.
Add comments to explain some of the bits that are set in the
Extended Inquiry VPD page.
Add a few comments to make it more clear which functions handle
various VPD pages.
Sponsored by: Spectra Logic
MFC after: 1 week
This could cause data corruption due to accessing wrong LUN in case of
retries on write errors. Failed writes were retried to read LUN.
MFC after: 3 days
Define it as an atomic uint32_t. These increments happen infrequently
enough for the atomic overhead to be a problem, and since they're now
independent atomics, they won't contend with xpt_lock_buses().
This counter is useful as a means of cheaply identifying whether any changes
have been made to the CAM peripheral list. Userland programs have no guarantee
that the counter won't change on them while being returned or while processing
the information, so they must be written accordingly.
Discussed with: ken, mav (in general)
MFC after: 1 week
Sponsored by: Spectra Logic
If we aggregated status sending with data move and got error, allow status
to be updated and resent again separately. Without this command may stuck
without status sent at all.
MFC after: 2 weeks
This includes a new summary mode (-s) for camcontrol defects that
quickly tells the user the most important thing: how many defects
are in the requested list. The actual location of the defects is
less important.
Modern drives frequently have more than the 8191 defects that can
be reported by the READ DEFECT DATA (10) command. If they don't
have that many grown defects, they certainly have more than 8191
defects in the primary (i.e. factory) defect list.
The READ DEFECT DATA (12) command allows for longer parameter
lists, as well as indexing into the list of defects, and so allows
reporting many more defects.
This has been tested with HGST drives and Seagate drives, but
does not fully work with Seagate drives. Once I have a Seagate
spec I may be able to determine whether it is possible to make it
work with Seagate drives.
scsi_da.h: Add a definition for the new long block defect
format.
Add bit and mask definitions for the new extended
physical sector and bytes from index defect
formats.
Add a prototype for the new scsi_read_defects() CDB
building function.
scsi_da.c: Add a new scsi_read_defects() CDB building function.
camcontrol(8) was previously composing CDBs manually.
This is long overdue.
camcontrol.c: Revamp the camcontrol defects subcommand. We now
go through multiple stages in trying to get defect
data off the drive while avoiding various drive
firmware quirks.
We start off by requesting the defect header with
the 10 byte command. If we're in summary mode (-s)
and the drive reports fewer defects than can be
represented in the 10 byte header, we're done.
Otherwise, we know that we need to issue the
12 byte command if the drive reports the maximum
number of defects.
If we're in summary mode, we're done if we get a
good response back when asking for the 12 byte header.
If the user has asked for the full list, then we
use the address descriptor index field in the 12
byte CDB to step through the list in 64K chunks.
64K is small enough to work with most any ancient
or modern SCSI controller.
Add support for printing the new long block defect
format, as well as the extended physical sector and
bytes from index formats. I don't have any drives
that support the new formats.
Add a hexadecimal output format that can be turned
on with -X.
Add a quiet mode (-q) that can be turned on with
the summary mode (-s) to just print out a number.
Revamp the error detection and recovery code for
the defects command to work with HGST drives.
Call the new scsi_read_defects() CDB building
function instead of rolling the CDB ourselves.
Pay attention to the residual from the defect list
request when printing it out, so we don't run off
the end of the list.
Use the new scsi_nv library routines to convert
from strings to numbers and back.
camcontrol.8: Document the new defect formats (longblock, extbfi,
extphys) and command line options (-q, -s, -S and
-X) for the defects subcommand.
Explain a little more about what drives generally
do and don't support.
Sponsored by: Spectra Logic
MFC after: 1 week
data to go undetected.
The probe code does an MD5 checksum of the inquiry data (and page
0x80 serial number if available) before doing a reprobe of an
existing device, and then compares a checksum after the probe to
see whether the device has changed.
This check was broken in January, 2000 by change 56146 when the extended
inquiry probe code was added.
In the extended inquiry probe case, it was calculating the checksum
a second time. The second time it included the updated inquiry
data from the short inquiry probe (first 36 bytes). So it wouldn't
catch cases where the vendor, product, revision, etc. changed.
This change will have the effect that when a device's inquiry data is
updated and a rescan is issued, it will disappear and then reappear.
This is the appropriate action, because if the inquiry data or serial
number changes, it is either a different device or the device
configuration may have changed significantly. (e.g. with updated
firmware.)
scsi_xpt.c: Don't calculate the initial MD5 checksum on
standard inquiry data and the page 0x80 serial
number if we have already calculated it.
MFC after: 1 week
Sponsored by: Spectra Logic
While in most cases CTL should correctly fetch those values from backing
storages, there are some initiators (like MS SQL), that may not like large
physical block sizes, even if they are true. For such cases allow override
fetched values with supported ones (like 4K).
MFC after: 1 week
While we don't support MCS, hole in received sequence numbers may mean
only PDU loss. While we don't support lost PDU recovery, terminate the
connection to avoid stuck commands.
While there, improve handling of sequence numbers wrap after 2^32 PDUs.
MFC after: 2 weeks
Technically read requests can be executed in any order or simultaneously
since they are not changing any data. But ZFS prefetcher goes crasy when
it receives consecutive requests from different threads. Since prefetcher
works on level of separate blocks, instead of two consecutive 128K requests
it may receive 32 8K requests in mixed order.
This patch is more workaround then a real fix, and it does not fix all of
prefetcher problems, but it improves sequential read speed by 3-4x times
in some configurations. On the other side it may hurt performance if
some backing store has no prefetch, that is why it is disabled by default
for raw devices.
MFC after: 2 weeks
While without UNMAP support there is not much initiator can do about it,
the administrator still better be notified about the storage overflow.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Previously it was supported only for ZVOL-backed LUNs, but now should work
for file-backed LUNs too. Used value in this case is a space occupied by
the backing file, while available value is an available space on file
system. Pool thresholds are still not implemented in this case.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
It is implemented for LUNs backed by ZVOLs in "dev" mode and files.
GEOM has no such API, so for LUNs backed by raw devices all LBAs will
be reported as mapped/unknown.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
After recent optimizations this change is no longer blocked by CTL memory
consumption. Those limits are still not free, but much cheaper now.
MFC after: 1 week
Relnotes: yes
Sponsored by: iXsystems, Inc.
Abusing ability of major UAs cover minor ones we may not account UAs for
inactive ports. Allocate UAs storage for port and start accounting only
after some initiator from that port fetched its first POWER ON OCCURRED.
This reduces per-LUN CTL memory usage from >1MB to less then 100K.
MFC after: 1 month
In configurations with many ports, like iSCSI, each LUN is typically
accessed only by limited subset of ports. Allocating that memory on
demand allows to reduce CTL memory usage from 5.3MB/LUN to 1.3MB/LUN.
MFC after: 1 month
interpreating NULLs as EOLs, but converting them to spaces.
SPC-4 does not tell that T10-based IDs should be NULL-terminated/padded.
And while it tells that it should include only ASCII chars (0x20-0x7F),
there are some USB sticks (SanDisk Ultra Fit), that have NULLs inside
the value. Treating NULLs as EOLs there made those LUN IDs non-unique.
MFC after: 1 week
Make CTL core and block backend set success status before initiating last
data move for read commands. Make CAM target and iSCSI frontends detect
such condition and send command status together with data. New I/O flag
allows to skip duplicate status sending on later fe_done() call.
For Fibre Channel this change saves one of three interrupts per read command,
increasing performance from 126K to 160K IOPS. For iSCSI this change saves
one of three PDUs per read command, increasing performance from 1M to 1.2M
IOPS.
MFC after: 1 month
Sponsored by: iXsystems, Inc.
Old allocator created significant lock congestion protecting its lists
of preallocated I/Os, while UMA provides much better SMP scalability.
The downside of UMA is lack of reliable preallocation, that could guarantee
successful allocation in non-sleepable environments. But careful code
review shown, that only CAM target frontend really has that requirement.
Fix that making that frontend preallocate and statically bind CTL I/O for
every ATIO/INOT it preallocates any way. That allows to avoid allocations
in hot I/O path. Other frontends either may sleep in allocation context
or can properly handle allocation errors.
On 40-core server with 6 ZVOL-backed LUNs and 7 iSCSI client connections
this change increases peak performance from ~700K to >1M IOPS! Yay! :)
MFC after: 1 month
Sponsored by: iXsystems, Inc.
Previously, any timeout value for which (timeout * hz) will overflow the
signed integer, will give weird results, since callout(9) routines will
convert negative values of ticks to '1'. For unsigned integer overflow we
will get sufficiently smaller timeout values than expected.
Switch from callout_reset, which requires conversion to int based ticks
to callout_reset_sbt to avoid this.
Also correct isci to correctly resolve ccb timeout.
This was based on the original work done by Eygene Ryabinkin
<rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid
the overlow.
Differential Revision: https://reviews.freebsd.org/D1157
Reviewed by: mav, davide
MFC after: 1 month
Sponsored by: Multiplay
In this mode one head is in Active state, supporting all commands, while
another is in Standby state, supporting only minimal LUN discovery subset.
It is still incomplete since Standby state requires reservation support,
which is impossible to do right without having interlink between heads.
But it allows to run some basic experiments.
related cleanups:
- Require each driver to initalize a mutex in the scsi_low_softc that
is shared with the scsi_low code. This mutex is used for CAM SIMs,
timers, and interrupt handlers.
- Replace the osdep function switch with direct calls to the relevant
CAM functions and direct manipulation of timers via callout(9).
- Collapse the CAM-specific scsi_low_osdep_interface substructure
directly into scsi_low_softc.
- Use bus_*() instead of bus_space_*().
- Return BUS_PROBE_DEFAULT from probe routines instead of 0.
- No need to zero softcs.
- Pass 0ul and ~0ul instead of 0 and ~0 to bus_alloc_resource().
- Spell "dettach" as "detach".
- Remove unused 'dvname' variables.
- De-spl().
Tested by: no one
With command serialization used in CTL, there are no other commands to abort
when PREEMPT AND ABORT gets to run, so it is practically equal to PREEMPT.
MFC after: 1 week
For ZVOL-backed LUNs this allows to inform initiators if storage's used or
available spaces get above/below the configured thresholds.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
This makes VMWare VAAI Thin Provisioning Stun primitive activate, pausing
the virtual machine, when backing storage (ZFS pool) is getting overflowed.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
This prevents BIO_DELETE requests getting stuck in the TRIM queue which
results in a panic on shutdown due to outstanding requests.
PR: 194606
Reported by: Guido Falsi
Reviewed by: mav
MFC after: 3 days
Sponsored by: Multiplay
- Wrong integer type was specified.
- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.
- Logical OR where binary OR was expected.
- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.
- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.
- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.
- Updated "EXAMPLES" section in SYSCTL manual page.
MFC after: 3 days
Sponsored by: Mellanox Technologies
in userland rename in-kernel getenv()/setenv() to kern_setenv()/kern_getenv().
This fixes a namespace collision with libc symbols.
Submitted by: kmacy
Tested by: make universe
This includes support for:
- Read-Write Error Recovery mode page;
- Informational Exceptions Control mode page;
- Logical Block Provisioning mode page;
- LOG SENSE command.
No real Informational Exceptions features yet. This is only a placeholder.
Sponsored by: iXsystems, Inc.
SPC-4 r2 allows to return empty defect list if the list is not supported.
We don't reallu support defect data lists, but this suppresses some errors.
MFC after: 1 week
Make this subcommand less FC-specific, reporting target and port addresses
in more generic way. Also make it report list of connected initiators in
unified way, working for both FC and iSCSI, and potentially others.
MFC after: 1 week
Queued async events handling in CAM opened race, that may lead to duplicate
AC_PATH_REGISTERED events delivery during boot. That was not happening
before r272935 because the driver was initialized later. After that change
it started create duplicate ports in CTL.
Target mode operation does not depend on the initiator mode scan process.
This change allows the target driver to attach earlier and receive some
async events (like AC_CONTRACT) that could be lost otherwise.
MFC after: 1 week
Such LUNs will be visible to initiators, but return "not ready" status
on media access commands. If backing storage become available later,
`ctladm modify ...` or `service ctld reload` can trigger its reopen.
It allows to push out some final data from the send queue to the socket
before its close. In particular, it increases chances for logout response
to be delivered to the initiator.
Before this change target could send R2T request for write transfer of any
size, that could violate iSCSI RFC, which allows initiator to limit maximum
R2T size by negotiating MaxBurstLength connection parameter.
Also report an error in case of write underflow, when initiator provides
less data than initiator expects. Previously in such case our target
sent R2T request for non-existing data, violating the RFC, and confusing
some initiators. SCSI specs don't explicitly define how write underflows
should be handled and there are different oppinions, but reporting error
is hopefully better then violating iSCSI RFC with unpredictable results.
MFC after: 2 weeks
SPC-2 tells REPORT LUNS shall be supported by devices supporting LUNs other
then LUN 0. If we see LUN 0 disconnected, guess there may be others, and
so REPORT LUNS shall be supported.
MFC after: 1 month
Previous logic was not differentiating disconnected LUNs and absent targets.
That made it to stop scan if LUN 0 was not found for any reason. That made
problematic, for example, using iSCSI targets declaring SPC-2 compliance and
having no LUN 0 configured.
The new logic continues sequential LUN scan if:
-- we have more configured LUNs that need recheck;
-- this LUN is connected and its SCSI version allows more LUNs;
-- this LUN is disconnected, its SCSI version allows more LUNs and we
guess they may be connected (we haven't scanned first 8 LUNs yet or
kern.cam.cam_srch_hi sysctl is set to scan more).
Reported by: trasz
MFC after: 1 month
or I_T NEXUS LOSS, clear all minor UAs for the LUN, redundant in this case.
All SAM specifications tell that target MAY do it, but libiscsi initiator
seems require it to be done, terminating connection with error if some more
UAs happen to be reported during iSCSI connection.
MFC after: 3 days
Using pointer from the cdev directly is dangerous since we have no reference
on it, and it may change any time. That caused panic if device has gone.
While there, report capacity change only if it really changed.
MFC after: 3 days
Without clustering support we any way have only one group of permanently
active ports, but that gives us one more supported VMWare feature. ;)
Solaris' Comstar also reports it even when only one port is present.
devq_openings counter lost its meaning after allocation queues has gone.
held counter is still meaningful, but problematic to update due to separate
locking of CCB allocation and queuing.
To fix that replace devq_openings counter with allocated counter. held is
now calculated on request as difference between number of allocated, queued
and active CCBs.
MFC after: 1 month
It allows to bypass range checks between UNMAP and READ/WRITE commands,
which may introduce additional delays while waiting for UNMAP parameters.
READ and WRITE commands are always processed in safe order since their
range checks are almost free.
Before this change UNMAP completely blocked other I/Os while running.
Now it blocks only colliding ones, slowing down others only due to ZFS
locks collisions.
Sponsored by: iXsystems, Inc.
None of existing STEC devices need UNMAP or even support it well, having
many limitations and even hanging sometimes executing those commands.
New devices that may use UNMAP going to be released under HGST name.
MFC after: 3 days
kern.cam.ctl.iscsi.ping_timeout to 0. This fixes interoperability with
some initiators that don't properly support NOP-Ins, namely iPXE/gPXE.
Submitted by: Chen Wen <pokkys@gmail.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
At this moment it works only for files and ZVOLs in device mode since BIOs
have no respective respective cache control flags (DPO/FUA).
MFC after: 1 month
Sponsored by: iXsystems, Inc.
that's ATAPI specific. Instead, skip to PROBE_SET_MULTI instead for
non ATAPI protocols. The prior code incorrectly terminated the probe
with a break, rather than arranging for probedone to get called. This
caused panics or worse on some systems.
that's only mostly similar. Specifically word 78 bits are defined for
IDENTIFY DEVICE as
5 Supports Hardware Feature Control
while a IDENTIFY PACKET DEVICE defines them as
5 Asynchronous notification supported
Therefore, only pay attention to bit 5 when we're talking to ATAPI
devices (we don't use the hardware feature control at this time).
Ignore it for ATA devices. Remove kludge that papered over this issue
for Samsung SATA SSDs, since Micron drives also have the bit set and
the error was caused by this bad interpretation of the spec (which is
quite easy to do, since bits aren't normally overlapping like this).
sizeof(struct scsi_inquiry_data) of 256 bytes combined with off-by-one
error in the changed code gave total INQUIRY data length above 255 bytes,
that was maximal INQUIRY length in SPC-2. While SPC-3 increased the
maximal length to 64K, at least sg3_utils are still confused by that.
MFC after: 1 week
This allows to avoid extra network traffic when copying files on NTFS iSCSI
disks within one storage host by drag'n'dropping them in Windows Explorer
of Windows 8/2012. It should also accelerate Hyper-V VM operations, etc.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
left some of the decisions based on its counterpart, SA_CCB_BUFFER_IO
being random. As a result, propagation of the residual information
for the SPACE command was broken, so the number of filemarks
encountered during a SPACE operation was miscalculated. Consequently,
systems relying on properly tracked filemark counters (like Bacula)
fell apart.
The change also removes a switch/case in sadone() which r256843
degraded to a single remaining case label.
PR: 192285
Approved by: ken
MFC after: 2 weeks
Unlike disk devices ZVOLs process all requests synchronously. That makes
impossible sending multiple requests to them from single thread. From the
other side ZVOLs have real d_read/d_write methods, which unlike d_strategy
can handle uio scatter/gather and have no strict I/O size limitations.
So, if ZVOL in "dev" mode is detected, use of d_read/d_write methods instead
of d_strategy allows to avoid pointless splitting of large requests into
MAXPHYS (128K) sized chunks.
MFC after: 1 week
link_elf_obj: symbol icl_pdu_new_bhs undefined
PR: 192031
Submitted by: Nils Beyer (earlier version)
MFC after: 3 days
Sponsored by: FreeBSD Foundation
