Commit Graph

98 Commits

Author SHA1 Message Date
Pedro F. Giffuni
1ffe58516c sys/cam: spelling fixes in comments.
No functional change.
2016-04-29 21:05:48 +00:00
Alan Somers
f94033f407 Add the ability to read a SAS device's Target Port NAA designator
sys/cam/scsi/scsi_all.h
sys/cam/scsi/scsi_all.c
	Add the scsi_devid_is_port_naa helper function

Reviewed by:	ken
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D5975
2016-04-18 15:01:59 +00:00
Scott Long
4b35e39c6a Add scsi_cdb_sbuf() for handling CDB strings. Reimplement scsi_cdb_string()
in terms of it.

Reviewed by:	imp, mav, ken
MFC after:	3 days
Sponsored by:	Netflix
Differential Revision:	D5934
2016-04-13 15:43:11 +00:00
Alexander Motin
4cace9ebf6 Add couple new constants from SPC5r08. 2016-04-09 18:39:57 +00:00
Alexander Motin
4ef0129a46 Add new report types to REPORT LUNS command.
This is only for completeness, since we have nothing new to report there.
2015-09-24 12:22:47 +00:00
Alexander Motin
a6daea64fd Update WRITE ATOMIC(16) support to sbc4r8 draft.
This is only a cosmetic change.  We still don't support atomic boundary
field in the CDB, but at least now we do it formally.
2015-09-24 08:04:47 +00:00
Alexander Motin
de988746be Add support for READ BUFFER(16) command. 2015-09-24 07:16:34 +00:00
Alexander Motin
c53993057b Add support for Control extension mode page. 2015-09-22 14:55:46 +00:00
Alexander Motin
c39d464164 Make CAM log errors that make it wait.
Waiting can take minutes, and it would be good for user to know what is
going on.

MFC after:	2 weeks
2015-09-15 10:57:16 +00:00
Alexander Motin
84e2fad15a Check for obsolete NUL bin in CSCD descriptor. 2015-09-12 20:45:09 +00:00
Kenneth D. Merry
0e358df062 Revamp camcontrol(8) fwdownload support and add the opcodes subcommand.
The significant changes and bugs fixed here are:

1. Fixed a bug in the progress display code:

   When the user's filename is too big, or his terminal width is too
   small, the progress code could wind up using a negative number for
   the length of the "stars" that it uses to indicate progress.

   This negative value was assigned to an unsigned variable, resulting
   in a very large positive value.

   The result is that we wound up writing garbage from memory to the
   user's terminal.

   With an 80 column terminal, a file name length of more than 35
   characters would generate this problem.

   To address this, we now set a minimum progress bar length, and
   truncate the user's file name as needed.

   This has been tested with large filenames and small terminals, and
   at least produces reasonable results.  If the terminal is too
   narrow, the progress display takes up an additional line with each
   update, but this is more user friendly than writing garbage to the
   tty.

2. SATA drives connected via a SATA controller didn't have SCSI Inquiry
   data populated in struct cam_device.  This meant that the code in
   fw_get_vendor() in fwdownload.c would try to match a zero-length
   vendor ID, and so return the first entry in the vendor table.  (Which
   used to be HITACHI.)  Fixed by grabbing identify data, passing the
   identify buffer into fw_get_vendor(), and matching against the model
   name.

3. SATA drives connected via a SAS controller do have Inquiry data
   populated.  The table included a couple of entries -- "ATA ST" and
   "ATA HDS", intended to handle Seagate and Hitachi SATA drives attached
   via a SAS controller.  SCSI to ATA translation layers use a vendor
   ID of "ATA" (which is standard), and then the model name from the ATA
   identify data as the SCSI product name when they are returning data on
   SATA disks.  The cam_strmatch code will match the first part of the
   string (because the length it is given is the length of the vendor,
   "ATA"), and return 0 (i.e. a match).  So all SATA drives attached to
   a SAS controller would be programmed using the Seagate method
   (WRITE BUFFER mode 7) of SCSI firmware downloading.

4. Issue #2 above covered up a bug in fw_download_img() -- if the
   maximum packet size in the vendor table was 0, it tried to default
   to a packet size of 32K.  But then it didn't actually succeed in
   doing that, because it set the packet size to the value that was
   in the vendor table (0).  Now that we actually have ATA attached
   drives fall use the VENDOR_ATA case, we need a reasonable default
   packet size.  So this is fixed to properly set the default packet size.

5. Add support for downloading firmware to IBM LTO drives, and add a
   firmware file validation method to make sure that the firmware
   file matches the drive type.  IBM tape drives include a Load ID and
   RU name in their vendor-specific VPD page 0x3.  Those should match
   the IDs in the header of the firmware file to insure that the
   proper firmware file is loaded.

6. This also adds a new -q option to the camcontrol fwdownload
   subcommand to suppress informational output.  When -q is used in
   combination with -y, the firmware upgrade will happen without
   prompting and without output except if an error condition occurs.

7. Re-add support for printing out SCSI inquiry information when
   asking the user to confirm that they want to download firmware, and
   add printing of ATA Identify data if it is a SATA disk.  This was
   removed in r237281 when support for flashing ATA disks was added.

8. Add a new camcontrol(8) "opcodes" subcommand, and use the
   underlying code to get recommended timeout values for drive
   firmware downloads.

   Many SCSI devices support the REPORT SUPPORTED OPERATION CODES
   command, and some support the optional timeout descriptor that
   specifies nominal and recommended timeouts for the commands
   supported by the device.

   The new camcontrol opcodes subcommand allows displaying all
   opcodes supported by a drive, information about which fields
   in a SCSI CDB are actually used by a given SCSI device, and the
   nominal and recommended timeout values for each command.

   Since firmware downloads can take a long time in some devices, and
   the time varies greatly between different types of devices, take
   advantage of the infrastructure used by the camcontrol opcodes
   subcommand to determine the best timeout to use for the WRITE
   BUFFER command in SCSI device firmware downloads.

   If the device recommends a timeout, it is likely to be more
   accurate than the default 50 second timeout used by the firmware
   download code.  If the user specifies a timeout, it will override
   the default or device recommended timeout.  If the device doesn't
   support timeout descriptors, we fall back to the default.

9. Instead of downloading firmware to SATA drives behind a SAS controller
   using WRITE BUFFER, use the SCSI ATA PASS-THROUGH command to compose
   an ATA DOWNLOAD MICROCODE command and it to the drive.  The previous
   version of this code attempted to send a SCSI WRITE BUFFER command to
   SATA drives behind a SAS controller.  Although that is part of the
   SAT-3 spec, it doesn't work with the parameters used with LSI
   controllers at least.

10.Add a new mechanism for making common ATA passthrough and
   ATA-behind-SCSI passthrough commands.

   The existing camcontrol(8) ATA command mechanism checks the device
   type on every command executed.  That works fine for individual
   commands, but is cumbersome for things like a firmware download
   that send a number of commands.

   The fwdownload code detects the device type up front, and then
   sends the appropriate commands.

11.In simulation mode (-s), if the user specifies the -v flag, print out
   the SCSI CDB or ATA registers that would be sent to the drive.  This will
   aid in debugging any firmware download issues.

sbin/camcontrol/fwdownload.c:
	Add a device type to the fw_vendor structure, so that we can
	specify different download methods for different devices from the
	same vendor.  In this case, IBM hard drives (from when they
	still made hard drives) and tape drives.

	Add a tur_status field to the fw_vendor structure so that we can
	specify whether the drive to be upgraded should be ready, not
	ready, or whether it doesn't matter.  Add the corresponding
	capability in fw_download_img().

	Add comments describing each of the vendor table fields.

	Add HGST and SmrtStor to the supported SCSI vendors list.

	In fw_get_vendor(), look at ATA identify data if we have a SATA
	device to try to identify what the drive vendor is.

	Add IBM firmware file validation.  This gets VPD page 0x3, and
	compares the Load ID and RU name in the page to the values
	included in the header.  The validation code will refuse to load
	a firmware file if the values don't match.  This does allow the
	user to attempt a downgrade; whether or not it succeeds will
	likely depend on the drive settings.

	Add a -q option, and disable all informative output
	(progress bars, etc.) when this is enabled.

	Re-add the inquiry in the confirmation dialog so the user has
	a better idea of which device he is talking to.  Add support for
	displaying ATA identify data.

	Don't automatically disable confirmation in simulation (-s) mode.
	This allows the user to see the inquiry or identify data in the
	dialog, and see exactly what they would see when the command
	actually runs.  Also, in simulation mode, if the user specifies
	the -v flag, print out the SCSI CDB or ATA registers that would
	be sent to the drive.  This will aid in debugging any firmware
	download issues.

	Add a timeout field and timeout type to the firmware download
	vendor table.  This allows specifying a default timeout and allows
	specifying whether we should attempt to probe for a recommended
	timeout from the drive.

	Add a new fuction, fw_get_timeout(), that will determine
	which timeout to use for the WRITE BUFFER command.  If the
	user specifies a timeout, we always use that.  Otherwise,
	we will use the drive recommended timeout, if available,
	and fall back to the default when a drive recommended
	timeout isn't available.

	When we prompt the user, tell him what timeout we're going
	to use, and the source of the timeout.

	Revamp the way SATA devices are handled.

	In fwdownload(), use the new get_device_type() function to
	determine what kind of device we're talking to.

	Allow firmware downloads to any SATA device, but restrict
	SCSI downloads to known devices.  (The latter is not a
	change in behavior.)

	Break out the "ready" check from fw_download_img() into a
	new subfunction, fw_check_device_ready().  This sends the
	appropriate command to the device in question -- a TEST
	UNIT READY or an IDENTIFY.  The IDENTIFY for SATA devices
 	a SAT layer is done using the SCSI ATA PASS-THROUGH
	command.

	Use the new build_ata_cmd() function to build either a SCSI or
	ATA I/O CCB to issue the DOWNLOAD MICROCODE command to SATA
	devices.  build_ata_cmd() figures looks at the devtype argument
	and fills in the correct CCB type and CDB or ATA registers.

	Revamp the vendor table to remove the previous
	vendor-specific ATA entries and use a generic ATA vendor
	placeholder.  We currently use the same method for all ATA
	drives, although we may have to add vendor-specific
	behavior once we test this with more drives.

sbin/camcontrol/progress.c:
	In progress_draw(), make barlength a signed value so that
	we can easily detect a negative value.

	If barlength (the length of the progress bar) would wind up
	negative due to a small TTY width or a large filename,
	set the bar length to the new minimum (10 stars) and
	truncate the user's filename.  We will truncate it down to
	0 characters if necessary.

	Calculate a new prefix_len variable (user's filename length)
	and use it as the precision when printing the filename.

sbin/camcontrol/camcontrol.c:
	Implement a new camcontrol(8) subcommand, "opcodes".  The
	opcodes subcommand allows displaying the entire list of
	SCSI commands supported by a device, or details on an
	individual command.  In either case, it can display
	nominal and recommended timeout values.

	Add the scsiopcodes() function, which calls the new
	scsigetopcodes() function to fetch opcode data from a
	drive.

	Add two new functions, scsiprintoneopcode() and
	scsiprintopcodes(), which print information about one
	opcode or all opcodes, respectively.

	Remove the get_disk_type() function.  It is no longer used.

	Add a new function, dev_has_vpd_page(), that fetches the
	supported INQUIRY VPD list from a device and tells the
	caller whether the requested VPD page is available.

	Add a new function, get_device_type(), that returns a more
	precise device type than the old get_disk_type() function.
	The get_disk_type() function only distinguished between
	SCSI and ATA devices, and SATA devices behind a SCSI to ATA
	translation layer were considered to be "SCSI".

	get_device_type() offers a third type, CC_DT_ATA_BEHIND_SCSI.
	We need to know this to know whether to attempt to send ATA
	passthrough commands.  If the device has the ATA
	Information VPD page (0x89), then it is an ATA device
	behind a SCSI to ATA translation layer.

	Remove the type argument from the fwdownload() subcommand.

	Add a new function, build_ata_cmd(), that will take one set
	of common arguments and build either a SCSI or ATA I/O CCB,
	depending on the device type passed in.

sbin/camcontrol/camcontrol.h:
	Add a prototype for scsigetopcodes().

	Add a new enumeration, camcontrol_devtype.

	Add prototypes for dev_has_vpd_page(), get_device_type()
	and build_ata_cmd().

	Remove the type argument from the fwdownload() subcommand.

sbin/camcontrol/camcontrol.8
	Explain that the fwdownload subcommand will use the drive
	recommended timeout if available, and that the user can
	override the timeout.

	Document the new opcodes subcommand.

	Explain that we will attempt to download firmware to any
	SATA device.

	Document supported SCSI vendors, and models tested if known.

	Explain the commands used to download firmware for the
	three different drive and controller combinations.

	Document that the -v flag in simulation mode for the fwdownload
	subcommand will print out the SCSI CDBs or ATA registers that would
	be used.

sys/cam/scsi/scsi_all.h:
	Add new bit definitions for the one opcode descriptor for
	the REPORT SUPPORTED OPCODES command.

	Add a function prototype for scsi_report_supported_opcodes().

sys/cam/scsi/scsi_all.c:
	Add a new CDB building function, scsi_report_supported_opcodes().

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-08-20 16:07:51 +00:00
Kenneth D. Merry
5672fac935 Add support for reading MAM attributes to camcontrol(8) and libcam(3).
MAM is Medium Auxiliary Memory and is most commonly found as flash
chips on tapes.

This includes support for reading attributes and decoding most
known attributes, but does not yet include support for writing
attributes or reporting attributes in XML format.

libsbuf/Makefile:
	Add subr_prf.c for the new sbuf_hexdump() function.  This
	function is essentially the same function.

libsbuf/Symbol.map:
	Add a new shared library minor version, and include the
	sbuf_hexdump() function.

libsbuf/Version.def:
	Add version 1.4 of the libsbuf library.

libutil/hexdump.3:
	Document sbuf_hexdump() alongside hexdump(3), since it is
	essentially the same function.

camcontrol/Makefile:
	Add attrib.c.

camcontrol/attrib.c:
	Implementation of READ ATTRIBUTE support for camcontrol(8).

camcontrol/camcontrol.8:
	Document the new 'camcontrol attrib' subcommand.

camcontrol/camcontrol.c:
	Add the new 'camcontrol attrib' subcommand.

camcontrol/camcontrol.h:
	Add a function prototype for scsiattrib().

share/man/man9/sbuf.9:
	Document the existence of sbuf_hexdump() and point users to
	the hexdump(3) man page for more details.

sys/cam/scsi/scsi_all.c:
	Add a table of known attributes, text descriptions and
	handler functions.

	Add a new scsi_attrib_sbuf() function along with a number
	of other related functions that help decode attributes.

	scsi_attrib_ascii_sbuf() decodes ASCII format attributes.

	scsi_attrib_int_sbuf() decodes binary format attributes, and
	will pass them off to scsi_attrib_hexdump_sbuf() if they're
	bigger than 8 bytes.

	scsi_attrib_vendser_sbuf() decodes the vendor and drive
	serial number attribute.

	scsi_attrib_volcoh_sbuf() decodes the Volume Coherency
	Information attribute that LTFS writes out.

sys/cam/scsi/scsi_all.h:
	Add a number of attribute-related structure definitions and
	other defines.

	Add function prototypes for all of the functions added in
	scsi_all.c.

sys/kern/subr_prf.c:
	Add a new function, sbuf_hexdump().  This is the same as
	the existing hexdump(9) function, except that it puts the
	result in an sbuf.

	This also changes subr_prf.c so that it can be compiled in
	userland for includsion in libsbuf.

	We should work to change this so that the kernel hexdump
	implementation is a wrapper around sbuf_hexdump() with a
	statically allocated sbuf with a drain.  That will require
	a drain function that goes to the kernel printf() buffer
	that can take a non-NUL terminated string as input.
	That is because an sbuf isn't NUL-terminated until it is
	finished, and we don't want to finish it while we're still
	using it.

	We should also work to consolidate the userland hexdump and
	kernel hexdump implemenatations, which are currently
	separate.  This would also mean making applications that
	currently link in libutil link in libsbuf.

sys/sys/sbuf.h:
	Add the prototype for sbuf_hexdump(), and add another copy
	of the hexdump flag values if they aren't already defined.

	Ideally the flags should be defined in one place but the
	implemenation makes it difficult to do properly.  (See
	above.)

Sponsored by:	Spectra Logic Corporation
MFC after:	1 week
2015-06-09 21:39:38 +00:00
Kenneth D. Merry
43518607b2 Significant upgrades to sa(4) and mt(1).
The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

 o sa(4) driver status and parameter information is now exported via an
   XML structure.  This will allow for changes and improvements later
   on that will not break userland applications.  The old MTIOCGET
   status ioctl remains, so applications using the existing interface
   will not break.

 o 'mt status' now reports drive-reported tape position information
   as well as the previously available calculated tape position
   information.  These numbers will be different at times, because
   the drive-reported block numbers are relative to BOP (Beginning
   of Partition), but the block numbers calculated previously via
   sa(4) (and still provided) are relative to the last filemark.
   Both numbers are now provided.  'mt status' now also shows the
   drive INQUIRY information, serial number and any position flags
   (BOP, EOT, etc.) provided with the tape position information.
   'mt status -v' adds information on the maximum possible I/O size,
   and the underlying values used to calculate it.

 o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

   The extra devices were originally added as place holders for
   density-specific device nodes.  Some OSes (NetBSD, NetApp's OnTap
   and Solaris) have had device nodes that, when you write to them,
   will automatically select a given density for particular tape drives.

   This is a convenient way of switching densities, but it was never
   implemented in FreeBSD.  Only the device nodes were there, and that
   sometimes confused users.

   For modern tape devices, the density is generally not selectable
   (e.g. with LTO) or defaults to the highest availble density when
   the tape is rewritten from BOT (e.g. TS11X0).  So, for most users,
   density selection won't be necessary.  If they do need to select
   the density, it is easy enough to use 'mt density' to change it.

 o Protection information is now supported.  This is either a
   Reed-Solomon CRC or CRC32 that is included at the end of each block
   read and written.  On write, the tape drive verifies the CRC, and
   on read, the tape drive provides a CRC for the userland application
   to verify.

 o New, extensible tape driver parameter get/set interface.

 o Density reporting information.  For drives that support it,
   'mt getdensity' will show detailed information on what formats the
   tape drive supports, and what formats the tape drive supports.

 o Some mt(1) functionality moved into a new mt(3) library so that
   external applications can reuse the code.

 o The new mt(3) library includes helper routines to aid in parsing
   the XML output of the sa(4) driver, and build a tree of driver
   metadata.

 o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
   (write filemark immediate) ioctls needed by IBM's LTFS
   implementation.

 o Improve device departure behavior for the sa(4) driver.  The previous
   implementation led to hangs when the device was open.

 o This has been tested on the following types of drives:
	IBM TS1150
	IBM TS1140
	IBM LTO-6
	IBM LTO-5
	HP LTO-2
	Seagate DDS-4
	Quantum DLT-4000
	Exabyte 8505
	Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
	Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
	New mt(3) library that contains functions moved from mt(1) and
	new functions needed to interact with the updated sa(4) driver.

	This includes XML parser helper functions that application writers
	can use when writing code to query tape parameters.

rescue/rescue/Makefile:
	Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
	Clarify this man page a bit, and since it contains what is
	essentially the mtio.h header file, add new ioctls and structure
	definitions from mtio.h.

src/share/man/man4/sa.4
	Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
	Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
	functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
	Many tape driver changes, largely outlined above.

	Increase the sa(4) driver read/write timeout from 4 to 32
	minutes.  This is based on the recommended values for IBM LTO
	5/6 drives.  This may also avoid timeouts for other tape
	hardware that can take a long time to do retries and error
	recovery.  Longer term, a better way to handle this is to ask
	the drive for recommended timeout values using the REPORT
	SUPPORTED OPCODES command.  Modern IBM and Oracle tape drives
	at least support that command, and it would allow for more
	accurate timeout values.

	Add XML status generation.  This is done with a series of
	macros to eliminate as much duplicate code as possible.  The
	new XML-based status values are reported through the new
	MTIOCEXTGET ioctl.

	Add XML driver parameter reporting, using the new MTIOCPARAMGET
	ioctl.

	Add a new driver parameter setting interface, using the new
	MTIOCPARAMSET and MTIOCSETLIST ioctls.

	Add a new MTIOCRBLIM ioctl to get block limits information.

	Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
	and scsi_read_position_10().

	scsi_locate_10 implements the LOCATE command, as does the
	existing scsi_set_position() command.  It just supports
	additional arguments and features.  If/when we figure out a
	good way to provide backward compatibility for older
	applications using the old function API, we can just revamp
	scsi_set_position().  The same goes for
	scsi_read_position_10() and the existing scsi_read_position()
	function.

	Revamp sasetpos() to take the new mtlocate structure as an
	argument.  It now will use either scsi_locate_10() or
	scsi_locate_16(), depending upon the arguments the user
	supplies.  As before, once we change position we don't have a
	clear idea of what the current logical position of the tape
	drive is.

	For tape drives that support long form position data, we
	read the current position and store that for later reporting
	after changing the position.  This should help applications
	like Bacula speed tape access under FreeBSD once they are
	modified to support the new ioctls.

	Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
	drives that report SCSI-2 or older, as well as drives that
	report an Illegal Request type error for READ POSITION with
	the long format.  So we should automatically detect drives
	that don't support the long form and stop asking for it after
	an initial try.

	Add a partition number to the sa(4) softc.

	Improve device departure handling. The previous implementation
	led to hangs when the device was open.

	If an application had the sa(4) driver open, and attempted to
	close it after it went away, the cam_periph_release() call in
	saclose() would cause the periph to get destroyed because that
	was the last reference to it.  Because destroy_dev() was
	called from the sa(4) driver's cleanup routine (sacleanup()),
	and would block waiting for the close to happen, a deadlock
	would result.

	So instead of calling destroy_dev() from the cleanup routine,
	call destroy_dev_sched_cb() from saoninvalidate() and wait for
	the callback.

	Acquire a reference for devfs in saregister(), and release it
	in the new sadevgonecb() routine when all devfs devices for
	the particular sa(4) driver instance are gone.

	Add a new function, sasetupdev(), to centralize setting
	per-instance devfs device parameters instead of repeating the
	code in saregister().

	Add an open count to the softc, so we know how many
	peripheral driver references are a result of open
       	sessions.

	Add the D_TRACKCLOSE flag to the cdevsw flags so
	that we get a 1:1 mapping of open to close calls
	instead of a N:1 mapping.

	This should be a no-op for everything except the
	control device, since we don't allow more than one
	open on non-control devices.

	However, since we do allow multiple opens on the
	control device, the combination of the open count
	and the D_TRACKCLOSE flag should result in an
	accurate peripheral driver reference count, and an
	accurate open count.

	The accurate open count allows us to release all
	peripheral driver references that are the result
	of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
	Add a number of new mt(4) ioctls and the requisite data
	structures.  None of the existing interfaces been removed
	or changed.

	This includes definitions for the following new ioctls:

	MTIOCRBLIM      /* get block limits */
	MTIOCEXTLOCATE	/* seek to position */
	MTIOCEXTGET     /* get tape status */
	MTIOCPARAMGET	/* get tape params */
	MTIOCPARAMSET	/* set tape params */
	MTIOCSETLIST	/* set N params */

usr.bin/mt/Makefile:
	mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
	Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
	Implement support for mt(1) subcommands that need to
	use getopt(3) for their arguments.

	Implement a new 'mt status' command to replace the old
	'mt status' command.  The old status command has been
	renamed 'ostatus'.

	The new status function uses the MTIOCEXTGET ioctl, and
	therefore parses the XML data to determine drive status.
	The -x argument to 'mt status' allows the user to dump out
	the raw XML reported by the kernel.

	The new status display is mostly the same as the old status
	display, except that it doesn't print the redundant density
	mode information, and it does print the current partition
	number and position flags.

	Add a new command, 'mt locate', that will supersede the
	old 'mt setspos' and 'mt sethpos' commands.  'mt locate'
	implements all of the functionality of the MTIOCEXTLOCATE
	ioctl, and allows the user to change the logical position
	of the tape drive in a number of ways.  (Partition,
	block number, file number, set mark number, end of data.)
	The immediate bit and the explicit address bits are
	implemented, but not documented in the man page.

	Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
	This allows the user to ask the drive to write a filemark
	without waiting around for the operation to complete.

	Add a new 'mt getdensity' command that gets the XML-based
	tape drive density report from the sa(4) driver and displays
	it.  This uses the SCSI REPORT DENSITY SUPPORT command
	to get comprehensive information from the tape drive about
	what formats it is able to read and write.

	Add a new 'mt protect' command that allows getting and setting
	tape drive protection information.  The protection information
	is a CRC tacked on to the end of every read/write from and to
	the tape drive.

Sponsored by:	Spectra Logic
MFC after:	1 month
2015-02-23 21:59:30 +00:00
Alexander Motin
2c8cab2a4e Add support for General Statistics and Performance log page.
CTL already collects most of statistics reported there, so why not.

MFC after:	2 weeks
2015-02-11 16:10:31 +00:00
Kenneth D. Merry
e761f855a0 Improve SCSI Extended Inquiry VPD page (0x86) support.
sys/cam/scsi/scsi_all.h:
	In struct scsi_extended_inquiry_data:
	- Increase the length field to 2 bytes, as it is 2 bytes in SPC-4.
	- Add bit definitions for the various Activiate Microcode actions.
	- Add the Sequential Access Logical Block Protection support bit,
	  since we need that in the sa(4) driver.  (For modifications
	  that will come later.)
	- Add definitions for the various Multi I_T Nexus Microcode
	  Download modes.

sys/cam/ctl/ctl.c:
	As of SPC-4, a single report of "REPORTED LUNS DATA HAS CHANGED"
	is to be given per I_T nexus.  Once it is reported, the unit
	attention condition should be cleared for all LUNS attached to
	an I_T nexus.

	Previously that only happened when a REPORT LUNS command was
	processed.

	This behavior may be different (according to SAM-5) when the
	UA_INTLCK_CTRL bits are non-zero in the control mode page but
	CTL does not currently support that.

	So, in view of the spec, whenever we report a LUN inventory
	change unit attention, clear it on all LUNs for that
	particular I_T nexus.

	Add a new function, ctl_clear_ua() that will clear a unit
	attention on all LUNs for the given I_T nexus.

	One field in the extended inquiry data that we could potentially
	report at some point is the maximum supported sense data length.
	To do that, we would the SIM to report (via path inquiry
	perhaps) how much sense data it is able to send.

	Add comments to explain some of the bits that are set in the
	Extended Inquiry VPD page.

	Add a few comments to make it more clear which functions handle
	various VPD pages.

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-01-30 05:23:39 +00:00
Alexander Motin
ef8daf3fed Add GET LBA STATUS command support to CTL.
It is implemented for LUNs backed by ZVOLs in "dev" mode and files.
GEOM has no such API, so for LUNs backed by raw devices all LBAs will
be reported as mapped/unknown.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-12-04 11:34:19 +00:00
Alexander Motin
50d75c5b57 Fix check for vendor-specific peripheral qualifier.
Submitted by:	anton.rang@isilon.com
MFC after:	1 week
2014-11-13 18:15:05 +00:00
Alexander Motin
c3e7ba3e6d Add to CTL support for logical block provisioning threshold notifications.
For ZVOL-backed LUNs this allows to inform initiators if storage's used or
available spaces get above/below the configured thresholds.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-11-06 00:48:36 +00:00
Alexander Motin
9a0190c9a1 Remove couple Copan's vendor-specific mode pages.
Those pages are highly system-/hardware-specific, the code is incomplete,
and so they hardly can be useful for anybody else.
2014-10-14 11:28:25 +00:00
Alexander Motin
523f047ea2 Some groundwork for later Informational Exceptions support.
This includes support for:
 - Read-Write Error Recovery mode page;
 - Informational Exceptions Control mode page;
 - Logical Block Provisioning mode page;
 - LOG SENSE command.

No real Informational Exceptions features yet. This is only a placeholder.

Sponsored by:	iXsystems, Inc.
2014-10-14 10:14:14 +00:00
Alexander Motin
8a41675372 Add support for WRITE ATOMIC (16) command and report SBC-4 compliance.
Atomic writes are only supported for ZVOLs in "dev" mode.  In other cases
atomicity can not be guarantied and so the command is blocked.
2014-10-08 07:48:36 +00:00
Alexander Motin
fb767c2ba2 Allow more commands to pass persistent reservation according to SPC-4 r37. 2014-09-18 22:22:14 +00:00
Alexander Motin
64c5167c91 Add support for "no Data-Out Buffer" (NDOB) flag of WRITE SAME (16) command. 2014-09-18 21:39:00 +00:00
Alexander Motin
13378399d6 Fix typo in defined ROD types in r269497.
MFC after:	3 days
2014-09-17 06:46:37 +00:00
Alexander Motin
5e5ac52b42 Add support for Extended INQUIRY Data (0x86) VPD page. 2014-09-11 22:40:11 +00:00
Alexander Motin
a3c5994cdf Oops, missed piece of r271311. 2014-09-09 14:20:55 +00:00
Alexander Motin
3406a2a083 Fix several issues and inconsistencies in UNMAP capabilities reporting.
This makes Windows 2012 to start using UNMAP on our disks.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-08-06 08:54:31 +00:00
Alexander Motin
25eee848cd Add support for Windows dialect of EXTENDED COPY command, aka Microsoft ODX.
This allows to avoid extra network traffic when copying files on NTFS iSCSI
disks within one storage host by drag'n'dropping them in Windows Explorer
of Windows 8/2012.  It should also accelerate Hyper-V VM operations, etc.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-08-04 01:16:20 +00:00
Alexander Motin
984a2ea91f Add support for VMWare dialect of EXTENDED COPY command, aka VAAI Clone.
This allows to clone VMs and move them between LUNs inside one storage
host without generating extra network traffic to the initiator and back,
and without being limited by network bandwidth.

LUNs participating in copy operation should have UNIQUE NAA or EUI IDs set.
For LUNs without these IDs VMWare will use traditional copy operations.

Beware: the above LUN IDs explicitly set to values non-unique from the VM
cluster point of view may cause data corruption if wrong LUN is addressed!

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-07-16 15:57:17 +00:00
Alexander Motin
b33b96e352 Enable TAS feature: notify initiator if its command was aborted by other.
That should make operation more kind to multi-initiator environment.
Without this, other initiators may find out that something bad happened
to their commands only via command timeout.
2014-07-08 16:38:05 +00:00
Alexander Motin
99ae56ac82 Add support for SCSI Ports (88h) VPD page. 2014-07-06 07:34:18 +00:00
Kenneth D. Merry
08df2e3eaf Add persistent reservation support to camcontrol(8).
camcontrol(8) now supports a new 'persist' subcommand that allows users to
issue SCSI PERSISTENT RESERVE IN / OUT commands.

sbin/camcontrol/Makefile:
	Add persist.c.

sbin/camcontrol/persist.c:
	New persistent reservation support for camcontrol(8).

	We have support for all known operation modes for PERSISTENT RESERVE
	IN and PERSISTENT RESERVE OUT.
	exceptions noted above.

sbin/camcontrol/camcontrol.8:
	Document the new 'persist' subcommand.

	In the section on the Transport ID (-I) option, explain what
	Transport IDs for each protocol should look like.  At some point
	some of this information could probably get moved off in a
	separate man page, either on Transport IDs alone or a man page
	documenting the Transport ID parsing code.

	Add a number of examples of persistent reservation commands.
	Persistent Reservations are complex enough that the average user
	probably won't be able to get the commands exactly right by just
	reading the man page.  These examples show a few basic and
	advanced examples of how to use persistent reservations.

sbin/camcontrol/camcontrol.h:
	Move the definition for camcontrol_optret here, so we can use it
	for the persistent reservation code.

	Add a definition for the new scsipersist() function.

sbin/camcontrol/camcontrol.c:
	Add 'persist' to the list of subcommands.

	Document 'persist' in the help text.

sys/cam/scsi/scsi_all.c:
	Add the scsi_persistent_reserve_in() and
	scsi_persistent_reserve_out() CCB building functions.

	Add a new function, scsi_transportid_sbuf().  This takes a
	SCSI Transport ID (documented in SPC-4), and prints it to
	an sbuf(9).  There are some transports (like ATA, USB, and
	SSA) for which there is no transport defined.  We need to
	come up with a reasonable thing to do if we're presented
	with a Transport ID that claims to be for one of those
	protocols.

	Add new routines scsi_get_nv() and scsi_nv_to_str().

	These functions do a table lookup to go between a string and an
	integer.  There are lots of table lookups needed in the
	persistent reservation code in camcontrol(8).

	Add a new function, scsi_parse_transportid(), along with leaf node
	functions to parse:
	FC, 1394 and SAS (scsi_parse_transportid_64bit())
	iSCSI (scsi_parse_transportid_iscsi())
	SPI (scsi_parse_transportid_spi())
	RDMA (scsi_parse_transportid_rdma())
	PCIe (scsi_parse_transportid_sop())

	Transport IDs.  Given a string with the general form proto,id these
	functions create a SCSI Transport ID structure.

sys/cam/scsi/scsi_all.h:
	Update the various persistent reservation data structures to
	SPC4r36l, but also rename some fields that were previously
	obsolete with the proper names from older SCSI specs.  This
	allows using older, obsolete persistent reservation types when
	desired.

	Add function prototypes for the new persistent reservation CCB
	building functions.

	Add a data strucure for the READ FULL STATUS service action
	of the PERSISTENT RESERVE IN command.

	Add Transport ID structures for all protocols described in SPC-4.

	Add a new series of SCSI_PROTO_XXX definitions, and
	redefine other defines in terms of these new definitions.

	Add a prototype for scsi_transportid_sbuf().

	Change a couple of "obsolete" persistent reservation data
	structure fields into something more meaningful, based on
	what the field was called when it was defined in the spec.
	(e.g. SPC, SPC-2, etc.)

	Create a new define, SPRI_MAX_LEN, for the maximum allocation
	length allowed for the PERSISTENT RESERVE IN command.

	Add data structures and enumerations for the new name/value
	translation functions.

	Add data structures for SCSI over PCIe Routing IDs.

	Bring the PERSISTENT RESERVE OUT Register and Move parameter list
	structure (struct scsi_per_res_out_parms) up to date with SPC-4.

	Add a data structure for the transport IDs that can optionally be
	appended to the basic PERSISTENT RESERVE OUT parameter list.

	Move SCSI protocol macro definitions out of the VPD page 0x83
	definition and combine them with the more up to date protocol
	definitions higher in the file.

	Add function prototypes for scsi_nv_to_str(), scsi_get_nv(),
	scsi_parse_transportid_64bit(), scsi_parse_transportid_spi(),
	scsi_parse_transportid_rdma(), scsi_parse_transportid_iscsi(),
	scsi_parse_transportid_sop(), and scsi_parse_transportid().

Sponsored by:	Spectra Logic Corporation
MFC after:	1 week
2014-07-03 23:09:44 +00:00
Alexander Motin
25c9d5e593 Add support for REPORT TIMESTAMP command.
MFC after:	2 weeks
2014-07-01 16:52:41 +00:00
Alexander Motin
1b08cb4ee7 Add more formal and strict command parsing and validation.
For every supported command define CDB length and mask of bits that are
allowed to be set.  This allows to remove bunch of checks through the code
and still make the validation more strict.  To properly do it for commands
supporting multiple service actions, formalize their parsing by adding
subtables for each of such commands.

As visible effect, this change allows to add support for REPORT SUPPORTED
OPERATION CODES command, reporting to client all the data about supported
SCSI commands, except timeouts.

MFC after:	2 weeks
2014-07-01 15:05:23 +00:00
Alexander Motin
f82388fd84 Allow MODE SENSE commands through Write Exclusive persistent reservation,
as required by SPC-4.

Report that fact in persistent reservation capabilities.

MFC after:	2 weeks
2014-06-26 09:42:00 +00:00
Alexander Motin
11b569f7cb Add support for VERIFY(10/12/16) and COMPARE AND WRITE SCSI commands.
Make data_submit backends method support not only read and write requests,
but also two new ones: verify and compare.  Verify just checks readability
of the data in specified location without transferring them outside.
Compare reads the specified data and compares them to received data,
returning error if they are different.

VERIFY(10/12/16) commands request either verify or compare from backend,
depending on BYTCHK CDB field.  COMPARE AND WRITE command executed in two
stages: first it requests compare, and then, if succeesed, requests write.
Atomicity of operation is guarantied by CTL request ordering code.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-06-16 11:00:14 +00:00
Alexander Motin
f7ad1c4625 Oops! Few quick fixes for r264274. 2014-04-08 21:30:10 +00:00
Alexander Motin
ee7f31c068 Add support for SCSI UNMAP commands to CTL.
This patch adds support for three new SCSI commands: UNMAP, WRITE SAME(10)
and WRITE SAME(16).  WRITE SAME commands support both normal write mode
and UNMAP flag.  To properly report UNMAP capabilities this patch also adds
support for reporting two new VPD pages: Block limits and Logical Block
Provisioning.

UNMAP support can be enabled per-LUN by adding "-o unmap=on" to `ctladm
create` command line or "option unmap on" to lun sections of /etc/ctl.conf.

At this moment UNMAP supported for ramdisks and device-backed block LUNs.
It was tested to work great with ZFS ZVOLs.  For file-backed LUNs UNMAP
support is unfortunately missing due to absence of respective VFS KPI.

Reviewed by:	ken
MFC after:	1 month
Sponsored by:	iXsystems, Inc
2014-04-08 20:50:48 +00:00
Steven Hartland
c28078e903 Improve ZFS N-way mirror read performance by using load and locality
information.

The existing algorithm selects a preferred leaf vdev based on offset of the zio
request modulo the number of members in the mirror. It assumes the devices are
of equal performance and that spreading the requests randomly over both drives
will be sufficient to saturate them. In practice this results in the leaf vdevs
being under utilized.

The new algorithm takes into the following additional factors:
* Load of the vdevs (number outstanding I/O requests)
* The locality of last queued I/O vs the new I/O request.

Within the locality calculation additional knowledge about the underlying vdev
is considered such as; is the device backing the vdev a rotating media device.

This results in performance increases across the board as well as significant
increases for predominantly streaming loads and for configurations which don't
have evenly performing devices.

The following are results from a setup with 3 Way Mirror with 2 x HD's and
1 x SSD from a basic test running multiple parrallel dd's.

With pre-fetch disabled (vfs.zfs.prefetch_disable=1):

== Stripe Balanced (default) ==
Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s
== Load Balanced (zfslinux) ==
Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s
== Load Balanced (locality freebsd) ==
Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s

With pre-fetch enabled (vfs.zfs.prefetch_disable=0):

== Stripe Balanced (default) ==
Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s
== Load Balanced (zfslinux) ==
Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s
== Load Balanced (locality freebsd) ==
Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s

In addition to the performance changes the code was also restructured, with
the help of Justin Gibbs, to provide a more logical flow which also ensures
vdevs loads are only calculated from the set of valid candidates.

The following additional sysctls where added to allow the administrator
to tune the behaviour of the load algorithm:
* vfs.zfs.vdev.mirror.rotating_inc
* vfs.zfs.vdev.mirror.rotating_seek_inc
* vfs.zfs.vdev.mirror.rotating_seek_offset
* vfs.zfs.vdev.mirror.non_rotating_inc
* vfs.zfs.vdev.mirror.non_rotating_seek_inc

These changes where based on work started by the zfsonlinux developers:
https://github.com/zfsonlinux/zfs/pull/1487

Reviewed by:	gibbs, mav, will
MFC after:	2 weeks
Sponsored by:	Multiplay
2013-10-23 09:54:58 +00:00
Alexander Motin
8d36a71b76 Unify periph invalidation and destruction reporting.
Print message containing device model and serial number on invalidation.

Requested by:   glebius
MFC after:	1 week
2013-10-15 17:59:41 +00:00
Edward Tomasz Napierala
83fd94a416 Fix the (unused for now) SCSI_PROTO_iSCSI define to match style(9). 2013-08-21 07:45:47 +00:00
Alexander Motin
0181d54b6b Improve handling of 0x3F/0x0E "Reported LUNs data has changed" and 0x25/0x00
"Logical unit not supported" errors.  First initiates specific target rescan,
second -- destroys specific LUN.  That allows to automatically detect changes
in list of device LUNs.  This mechanism doesn't work when target is completely
idle, but probably that is all what can be done without active polling.

Reviewed by:	ken
Sponsored by:	iXsystems, Inc.
2013-07-13 13:35:09 +00:00
Steven Hartland
ef27aa40a5 Corrected ATA Passthrough defines from decimal to hex
Reviewed by:	scottl
MFC after:	1 week
2013-06-20 21:38:08 +00:00
Alexander Motin
ccba710262 Make CAM return and GEOM DISK pass through new GEOM::lunid attribute.
SPC-4 specification states that serial number may be property of device,
but not a specific logical unit.  People reported about FC storages using
serial number in that way, making it unusable for purposes of LUN multipath
detection.  SPC-4 states that designators associated with logical unit from
the VPD page 83h "Device Identification" should be used for that purpose.
Report first of them in the new attribute in such preference order: NAA,
EUI-64, T10 and SCSI name string.

While there, make GEOM DISK properly report GEOM::ident in XML output also
using d_getattr() method, if available.  This fixes serial numbers reporting
for SCSI disks in `geom disk list` output and confxml.

Discussed with:	gibbs, ken
Sponsored by:	iXsystems, Inc.
MFC after:	2 weeks
2013-06-12 13:36:20 +00:00
Steven Hartland
a428909068 Use the existence of ATA Information VPD to determine if we should attempt
to query ATA functionality via ATA Pass-Through (16) as this page is defined
as "must" for SATL devices, hence indicating that the device is at least
likely to support Pass-Through (16).

This eliminates errors produced by CTL when ATA Pass-Through (16) fails.

Switch ATA probe daerror call to SF_NO_PRINT to avoid errors printing out
for devices which return invalid errors.

Output details about supported and choosen delete method when verbose booted.

Reviewed by:	mav
Approved by:	pjd (mentor)
MFC after:	 1 week
2013-05-02 14:37:23 +00:00
Steven Hartland
90edda31ba Added automatic detection of non-rotating media which disables the
use of BIO queue sorting, hence optimising performance for devices
such as SSD's

Reviewed by:	scottl
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-04-26 16:31:03 +00:00
Steven Hartland
e88aa3fd2c Refactored scsi_xpt use of device_has_vpd to generic scsi_vpd_supported_page
so its available for use in generic scsi code.

This is a pre-requirement for using VPD queries to determine available SCSI
delete methods within scsi_da.

Reviewed by:	mav
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-04-26 16:11:03 +00:00
Steven Hartland
b1da0a9868 Added the ability to send ATA identify and Data Set Management (DSM) TRIM
commands to an ATA device attached via a SCSI control.

sys/cam/scsi/scsi_all.c:
        - Added scsi_ata_identify, scsi_ata_trim
          Which use ATA Pass-Through to send commands to the attached disk.

sys/cam/scsi/scsi_all.h:
        - Added defines for all missing ATA Pass-Through commands values.

        - Added scsi_ata_identify, scsi_ata_trim methods used in ATA TRIM
          support.

        - Added scsi_vpd_logical_block_prov structure used when querying for
          the supported sizes UNMAP commands.

        - Added scsi_vpd_block_limits structure used when querying for the
          supported sizes of the UNMAP command.

Reviewed by:	mav
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-04-26 15:53:22 +00:00
Steven Hartland
b3cc74dc3b Added ATA Pass-Through support to CAM
sys/cam/scsi/scsi_all.c:
        - Added scsi_ata_pass_16 method
          Which use ATA Pass-Through to send commands to the attached disk.

sys/cam/scsi/scsi_all.h:
        - Added defines for all missing ATA Pass-Through commands values.

        - Added scsi_ata_pass_16 method.

        - Fixed a comment typo while I'm here

Reviewed by:	mav
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-04-02 00:11:35 +00:00
Konstantin Belousov
abc1e60e0e Support unmapped i/o for the md(4).
The vnode-backed md(4) has to map the unmapped bio because VOP_READ()
and VOP_WRITE() interfaces do not allow to pass unmapped requests to
the filesystem. Vnode-backed md(4) uses pbufs instead of relying on
the bio_transient_map, to avoid usual md deadlock.

Sponsored by:	The FreeBSD Foundation
Tested by:	pho, scottl
2013-03-19 15:01:50 +00:00