freebsd-dev/sys/cam
Kenneth D. Merry 9a6844d55f Add support for managing Shingled Magnetic Recording (SMR) drives.
This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to
the Zoned ATA Command Set or ZAC spec) behind SAS expanders.

This includes full management support through the GEOM BIO interface, and
through a new userland utility, zonectl(8), and through camcontrol(8).

This is now ready for filesystems to use to detect and manage zoned drives.
(There is no work in progress that I know of to use this for ZFS or UFS, if
anyone is interested, let me know and I may have some suggestions.)

Also, improve ATA command passthrough and dispatch support, both via ATA
and ATA passthrough over SCSI.

Also, add support to camcontrol(8) for the ATA Extended Power Conditions
feature set.  You can now manage ATA device power states, and set various
idle time thresholds for a drive to enter lower power states.

Note that this change cannot be MFCed in full, because it depends on
changes to the struct bio API that break compatilibity.  In order to
avoid breaking the stable API, only changes that don't touch or depend on
the struct bio changes can be merged.  For example, the camcontrol(8)
changes don't depend on the new bio API, but zonectl(8) and the probe
changes to the da(4) and ada(4) drivers do depend on it.

Also note that the SMR changes have not yet been tested with an actual
SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports
ZBC to ZAC translation.  I have not yet gotten a suitable drive or SAT
layer, so any testing help would be appreciated.  These changes have been
tested with Seagate Host Aware SATA drives attached to both SAS and SATA
controllers.  Also, I do not have any SATA Host Managed devices, and I
suspect that it may take additional (hopefully minor) changes to support
them.

Thanks to Seagate for supplying the test hardware and answering questions.

sbin/camcontrol/Makefile:
	Add epc.c and zone.c.

sbin/camcontrol/camcontrol.8:
	Document the zone and epc subcommands.

sbin/camcontrol/camcontrol.c:
	Add the zone and epc subcommands.

	Add auxiliary register support to build_ata_cmd().  Make sure to
	set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA
	flags as appropriate for ATA commands.

	Add a new get_ata_status() function to parse ATA result from SCSI
	sense descriptors (for ATA passthrough over SCSI) and ATA I/O
	requests.

sbin/camcontrol/camcontrol.h:
	Update the build_ata_cmd() prototype

	Add get_ata_status(), zone(), and epc().

sbin/camcontrol/epc.c:
	Support for ATA Extended Power Conditions features.  This includes
	support for all features documented in the ACS-4 Revision 12
	specification from t13.org (dated February 18, 2016).

	The EPC feature set allows putting a drive into a power power mode
	immediately, or setting timeouts so that the drive will
	automatically enter progressively lower power states after various
	idle times.

sbin/camcontrol/fwdownload.c:
	Update the firmware download code for the new build_ata_cmd()
	arguments.

sbin/camcontrol/zone.c:
	Implement support for Shingled Magnetic Recording (SMR) drives
	via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA
	Command Set (ZAC).

	These specs were developed in concert, and are functionally
	identical.  The primary differences are due to SCSI and ATA
	differences.  (SCSI is big endian, ATA is little endian, for
	example.)

	This includes support for all commands defined in the ZBC and
	ZAC specs.

sys/cam/ata/ata_all.c:
	Decode a number of additional ATA command names in ata_op_string().

	Add a new CCB building function, ata_read_log().

	Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building
	functions.  These support both DMA and NCQ encapsulation.

sys/cam/ata/ata_all.h:
	Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and
	ata_zac_mgmt_in().

sys/cam/ata/ata_da.c:
	Revamp the ada(4) driver to support zoned devices.

	Add four new probe states to gather information needed for zone
	support.

	Add a new adasetflags() function to avoid duplication of large
	blocks of flag setting between the async handler and register
	functions.

	Add new sysctl variables that describe zone support and paramters.

	Add support for the new BIO_ZONE bio, and all of its subcommands:
	DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
	DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

sys/cam/scsi/scsi_all.c:
	Add command descriptions for the ZBC IN/OUT commands.

	Add descriptions for ZBC Host Managed devices.

	Add a new function, scsi_ata_pass() to do ATA passthrough over
	SCSI.  This will eventually replace scsi_ata_pass_16() -- it
	can create the 12, 16, and 32-byte variants of the ATA
	PASS-THROUGH command, and supports setting all of the
	registers defined as of SAT-4, Revision 5 (March 11, 2016).

	Change scsi_ata_identify() to use scsi_ata_pass() instead of
	scsi_ata_pass_16().

	Add a new scsi_ata_read_log() function to facilitate reading
	ATA logs via SCSI.

sys/cam/scsi/scsi_all.h:
	Add the new ATA PASS-THROUGH(32) command CDB.  Add extended and
	variable CDB opcodes.

	Add Zoned Block Device Characteristics VPD page.

	Add ATA Return SCSI sense descriptor.

	Add prototypes for scsi_ata_read_log() and scsi_ata_pass().

sys/cam/scsi/scsi_da.c:
	Revamp the da(4) driver to support zoned devices.

	Add five new probe states, four of which are needed for ATA
	devices.

	Add five new sysctl variables that describe zone support and
	parameters.

	The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC
	devices when they are attached via a SCSI to ATA Translation (SAT)
	layer.  Since ZBC -> ZAC translation is a new feature in the T10
	SAT-4 spec, most SATA drives will be supported via ATA commands
	sent via the SCSI ATA PASS-THROUGH command.  The da(4) driver will
	prefer the ZBC interface, if it is available, for performance
	reasons, but will use the ATA PASS-THROUGH interface to the ZAC
	command set if the SAT layer doesn't support translation yet.
	As I mentioned above, ZBC command support is untested.

	Add support for the new BIO_ZONE bio, and all of its subcommands:
	DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
	DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

	Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.

	Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB
	building functions.  Note that these have return values, unlike
	almost all other CCB building functions in CAM.  The reason is
	that they can fail, depending upon the particular combination
	of input parameters.  The primary failure case is if the user
	wants NCQ, but fails to specify additional CDB storage.  NCQ
	requires using the 32-byte version of the SCSI ATA PASS-THROUGH
	command, and the current CAM CDB size is 16 bytes.

sys/cam/scsi/scsi_da.h:
	Add ZBC IN and ZBC OUT CDBs and opcodes.

	Add SCSI Report Zones data structures.

	Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and
	scsi_ata_zac_mgmt_in() prototypes.

sys/dev/ahci/ahci.c:
	Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.

	ahci_setup_fis() previously set the top bits of the sector count
	register in the FIS to 0 for FPDMA commands.  This is okay for
	read and write, because the PRIO field is in the only thing in
	those bits, and we don't implement that further up the stack.

	But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that
	byte, so it needs to be transmitted to the drive.

	In ahci_setup_fis(), always set the the top 8 bits of the
	sector count register.  We need it in both the standard
	and NCQ / FPDMA cases.

sys/geom/eli/g_eli.c:
	Pass BIO_ZONE commands through the GELI class.

sys/geom/geom.h:
	Add g_io_zonecmd() prototype.

sys/geom/geom_dev.c:
	Add new DIOCZONECMD ioctl, which allows sending zone commands to
	disks.

sys/geom/geom_disk.c:
	Add support for BIO_ZONE commands.

sys/geom/geom_disk.h:
	Add a new flag, DISKFLAG_CANZONE, that indicates that a given
	GEOM disk client can handle BIO_ZONE commands.

sys/geom/geom_io.c:
	Add a new function, g_io_zonecmd(), that handles execution of
	BIO_ZONE commands.

	Add permissions check for BIO_ZONE commands.

	Add command decoding for BIO_ZONE commands.

sys/geom/geom_subr.c:
	Add DDB command decoding for BIO_ZONE commands.

sys/kern/subr_devstat.c:
	Record statistics for REPORT ZONES commands.  Note that the
	number of bytes transferred for REPORT ZONES won't quite match
	what is received from the harware.  This is because we're
	necessarily counting bytes coming from the da(4) / ada(4) drivers,
	which are using the disk_zone.h interface to communicate up
	the stack.  The structure sizes it uses are slightly different
	than the SCSI and ATA structure sizes.

sys/sys/ata.h:
	Add many bit and structure definitions for ZAC, NCQ, and EPC
	command support.

sys/sys/bio.h:
	Convert the bio_cmd field to a straight enumeration.  This will
	yield more space for additional commands in the future.  After
	change r297955 and other related changes, this is now possible.
	Converting to an enumeration will also prevent use as a bitmask
	in the future.

sys/sys/disk.h:
	Define the DIOCZONECMD ioctl.

sys/sys/disk_zone.h:
	Add a new API for managing zoned disks.  This is very close to
	the SCSI ZBC and ATA ZAC standards, but uses integers in native
	byte order instead of big endian (SCSI) or little endian (ATA)
	byte arrays.

	This is intended to offer to the complete feature set of the ZBC
	and ZAC disk management without requiring the application developer
	to include SCSI or ATA headers.  We also use one set of headers
	for ioctl consumers and kernel bio-level consumers.

sys/sys/param.h:
	Bump __FreeBSD_version for sys/bio.h command changes, and inclusion
	of SMR support.

usr.sbin/Makefile:
	Add the zonectl utility.

usr.sbin/diskinfo/diskinfo.c
	Add disk zoning capability to the 'diskinfo -v' output.

usr.sbin/zonectl/Makefile:
	Add zonectl makefile.

usr.sbin/zonectl/zonectl.8
	zonectl(8) man page.

usr.sbin/zonectl/zonectl.c
	The zonectl(8) utility.  This allows managing SCSI or ATA zoned
	disks via the disk_zone.h API.  You can report zones, reset write
	pointers, get parameters, etc.

Sponsored by:	Spectra Logic
Differential Revision:	https://reviews.freebsd.org/D6147
Reviewed by:	wblock (documentation)
2016-05-19 14:08:36 +00:00
..
ata Add support for managing Shingled Magnetic Recording (SMR) drives. 2016-05-19 14:08:36 +00:00
ctl Add icl_conn_connect() ICL method, required for iSER. 2016-05-17 11:10:44 +00:00
scsi Add support for managing Shingled Magnetic Recording (SMR) drives. 2016-05-19 14:08:36 +00:00
cam_ccb.h Add "camcontrol reprobe" subcommand, and implement it for da(4). 2016-05-10 15:46:33 +00:00
cam_compat.c Make pass, sg and targ drivers respect HBA's maxio. 2015-09-30 13:31:37 +00:00
cam_compat.h Widen lun_id_t to 64 bits. This is a follow-on to r257345 to let the kernel 2013-12-14 22:07:40 +00:00
cam_debug.h Replace several instances of -1 with appropriate CAM_*_WILDCARD and types. 2014-01-10 12:18:05 +00:00
cam_iosched.c Fix tinderbox LINT build. 2016-04-18 08:24:13 +00:00
cam_iosched.h Add in missing files from r298002. 2016-04-14 22:13:44 +00:00
cam_periph.c Don't repeat the the word 'the' 2016-05-17 12:52:31 +00:00
cam_periph.h Make pass, sg and targ drivers respect HBA's maxio. 2015-09-30 13:31:37 +00:00
cam_queue.c sys/cam: spelling fixes. 2016-04-29 21:04:06 +00:00
cam_queue.h Update CAM CCB accounting for the new status quo. 2014-09-14 11:59:49 +00:00
cam_sim.c Merge CAM locking changes from the projects/camlock branch to radically 2013-10-21 12:00:26 +00:00
cam_sim.h Merge CAM locking changes from the projects/camlock branch to radically 2013-10-21 12:00:26 +00:00
cam_xpt_internal.h Add support for probing the SCSI VPD Extended Inquiry page (0x86). 2015-02-05 00:12:21 +00:00
cam_xpt_periph.h Unify periph invalidation and destruction reporting. 2013-10-15 17:59:41 +00:00
cam_xpt_sim.h Merge CAM locking changes from the projects/camlock branch to radically 2013-10-21 12:00:26 +00:00
cam_xpt.c Add "camcontrol reprobe" subcommand, and implement it for da(4). 2016-05-10 15:46:33 +00:00
cam_xpt.h Add asynchronous command support to the pass(4) driver, and the new 2015-12-03 20:54:55 +00:00
cam.c Remove slightly used const values that can be replaced with nitems(). 2016-04-21 15:38:28 +00:00
cam.h sys/cam: spelling fixes in comments. 2016-04-29 21:05:48 +00:00
README.quirks

/* $FreeBSD$ */

                     FreeBSD Quirk Guidelines

                  Nate Lawson - njl at freebsd org

0. Introduction

FreeBSD drivers make every attempt possible to support the standards
behind hardware. Where possible and not in conflict with the standard,
they also attempt to work around hardware which doesn't strictly
conform. However, some devices have flaws which can't be worked
around while keeping the driver compatible with the standard. For
these devices, we have created a quirks mechanism to indicate to
the driver that it must avoid certain commands or use them differently
with a specific model and/or version of hardware. This document
focuses on identifying and committing quirks for storage hardware
involving CAM and UMASS but is applicable to other areas.

CAM provides a generic transport for SCSI-like devices. Many different
transports use SCSI command sets including parallel SCSI, firewire
(1394), USB UMASS, fibre channel, and ATAPI. For block devices (i.e.
hard drives, flash adapters, cameras) there are two standards, SBC
and RBC. SCSI hard drives are usually SBC-compliant and smaller
devices like flash drives are usually RBC-compliant. Multimedia
devices including CDROMs and DVD-RW are usually MMC-compliant.

Please follow these guidelines to get your device working as soon
as possible. If you are a committer, please do NOT commit quirks
directly but follow this process also.

1. Determing the problem

The first step is to determine what's wrong. If the device should
be supported but hangs while attaching, it's possible a quirk can
help. The types of things a quirk can fix are:
`
 * cam/cam_xpt.c quirks 

  o CAM_QUIRK_NOLUNS - do not probe luns other than 0 since device
  responds to all inquiries with "lun present".

  o CAM_QUIRK_NOSERIAL - do not send an inquiry for serial number. 

  o CAM_QUIRK_HILUNS - probe all luns even if some respond "not present"
  since device has a sparse lun space. 

 * cam/scsi/scsi_da.c quirks 

  o DA_Q_NO_SYNC_CACHE - The sync cache command is used to force a
  drive to write out all changes to disk before shutting down. Some
  drives hang when receiving this command even though it is required
  by all SBC and RBC standards. Note that a warning message on
  console is NOT sufficient to add this quirk. The warning messages
  are harmless and only a device or system hang is cause for adding
  this quirk.

  o DA_Q_NO_6_BYTE - The RBC spec (see Links below) does not allow
  for 6-byte READ/WRITE commands. Some manufacturers took that too
  literally and crash when receiving 6-byte commands. This quirk
  causes FreeBSD to only send 10-byte commands. Since the CAM subsystem
  has been modified to not send 6-byte commands to USB, 1394, and
  other transports that don't support SBC, this quirk should be very
  rare.

  o DA_Q_NO_PREVENT - Don't use the prevent/allow commands to keep a
  removable medium from being ejected. Some systems can't handle these
  commands (rare).

 * cam/scsi/scsi_cd.c quirks 

  o CD_Q_NO_TOUCH - not implemented 

  o CD_Q_BCD_TRACKS - convert start/end track to BCD 

  o CD_Q_NO_CHANGER - never treat as a changer 

  o CD_Q_CHANGER - always treat as a changer 

 * cam/scsi/scsi_ch.c quirks 
  o CH_Q_NO_DBD - disable block descriptors in mode sense 

 * cam/scsi/scsi_sa.c quirks 

  o SA_QUIRK_NOCOMP - Can't deal with compression at all 

  o SA_QUIRK_FIXED - Force fixed mode 

  o SA_QUIRK_VARIABLE - Force variable mode 

  o SA_QUIRK_2FM - Needs Two File Marks at EOD 

  o SA_QUIRK_1FM - No more than 1 File Mark at EOD 

  o SA_QUIRK_NODREAD - Don't try and dummy read density 

  o SA_QUIRK_NO_MODESEL - Don't do mode select at all 

  o SA_QUIRK_NO_CPAGE - Don't use DEVICE COMPRESSION page 

 * dev/usb/umass.c quirks 

  o NO_TEST_UNIT_READY - The drive does not support Test Unit Ready.
  Convert to Start Unit. This command is a simple no-op for most
  firmware but some of them hang when this command is sent.

  o RS_NO_CLEAR_UA - The drive does not reset the Unit Attention state
  after REQUEST SENSE has been sent. The INQUIRY command does not
  reset the UA either, and so CAM runs in circles trying to retrieve
  the initial INQUIRY data. This quirk signifies that after a unit
  attention condition, don't try to clear the condition with a request
  sense command.

  o NO_START_STOP - Like test unit ready, don't send this command if it hangs the device. 

  o FORCE_SHORT_INQUIRY - Don't ask for full inquiry data (256
  bytes). Some drives can only handle the shorter inquiry length
  (36 bytes).

  o SHUTTLE_INIT - Needs to be initialised the Shuttle way. Haven't
  looked into what this does but apparently it's mostly Shuttle
  devices.

  o ALT_IFACE_1 - Drive needs to be switched to alternate interface 1. Rare.

  o FLOPPY_SPEED - Drive does not do 1Mb/s, but just floppy speeds (20kb/s). 

  o IGNORE_RESIDUE - The device can't count and gets the residue
  of transfers wrong. This is sometimes needed for devices where
  large transfers cause stalls.

  o NO_GETMAXLUN - Get maximum LUN is a command to identify multiple
  devices sharing the same ID. For instance, a multislot compact
  flash reader might be on two LUNS. Some non-standard devices hang
  when receiving this command so this quirk disables it.

  o WRONG_CSWSIG - The device uses a weird CSWSIGNATURE. Rare. 

  o NO_INQUIRY - Device cannot handle INQUIRY so fake a generic
  response. INQUIRY is one of the most basic commands but some
  drives can't even handle it. (No idea how such devices even work
  at all on other OS's.) This quirk fakes up a valid but generic
  response for devices that can't handle INQUIRY.

  o NO_INQUIRY_EVPD - Device cannot handle an extended INQUIRY
  asking for vital product data (EVPD) so just return a "no data"
  response (check condition) without sending the command to the
  device.

2. Testing a Quirk

After you have an idea what you want to try, edit the proper file
above, using wildcarding to be sure your device is matched. Here
is a list of the common things to try. Note that some devices require
multiple quirks or quirks in different drivers. For example, some
USB pen drives or flash readers require quirks in both da(4) and
umass(4).

* umass(4) device (sys/dev/usb/umass.c) -- this quirk matches an Asahi Optical device with any product ID or revision ID. 
* 
*         { USB_VENDOR_ASAHIOPTICAL, PID_WILDCARD, RID_WILDCARD,
*           UMASS_PROTO_ATAPI | UMASS_PROTO_CBI_I,
*           RS_NO_CLEAR_UA
*         },
* da(4) device (sys/cam/scsi/scsi_da.c) -- this quirk matches a Creative device with a name of "NOMAD_MUVO" and any revision. 
* 
*         {
*                 /*
*                  * Creative Nomad MUVO mp3 player (USB)
*                  * PR: kern/53094
*                  */
*                 {T_DIRECT, SIP_MEDIA_REMOVABLE, "CREATIVE", "NOMAD_MUVO", "*"},
*                 /*quirks*/ DA_Q_NO_SYNC_CACHE|DA_Q_NO_PREVENT
*         },

3. Filing a PR

All quirk submissions MUST go through GNATS. For information on how
to submit a PR, see this page.

Please include the following in your PR:

 * Subject: QUIRK: FooCo USB DVD-RAM drive 
 * Output of "camcontrol inquiry yourdevice" 
 * Manufacturer name, model number, etc. 
 * Transport type (FC, SCSI, USB, Firewire) 
 * Output from dmesg for failed attach attempts 
 * Output from dmesg for successful attach attempts (after quirk added) 
 * Output of "usbdevs -v" with device attached 
 * Valid email address 

Here are some examples of well-formed PRs: 

 * kern/43580 
 * kern/49054 

4. What happens next

I will review your submission, respond with comments, and once the
quirk is deemed necessary and ready for committing, I'll commit it,
referencing the PR. (Again, all quirks must be submitted as PRs).
Questions? Email njl AT freebsd.org.

5. Note to Committers

Please insert quirks in the right section in scsi_da.c, sorted by
PR number. Always include the name and PR number for scsi_da.c (see
above for an example.) Please sort quirks alphabetically in umass.c.
Follow the surrounding style in all drivers. Be sure to correspond
with the submitter to be sure the quirk you are adding is the minimum
necessary, not quirking other useful features and not overly broad
(i.e., too many wildcards).