This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to
the Zoned ATA Command Set or ZAC spec) behind SAS expanders.
This includes full management support through the GEOM BIO interface, and
through a new userland utility, zonectl(8), and through camcontrol(8).
This is now ready for filesystems to use to detect and manage zoned drives.
(There is no work in progress that I know of to use this for ZFS or UFS, if
anyone is interested, let me know and I may have some suggestions.)
Also, improve ATA command passthrough and dispatch support, both via ATA
and ATA passthrough over SCSI.
Also, add support to camcontrol(8) for the ATA Extended Power Conditions
feature set. You can now manage ATA device power states, and set various
idle time thresholds for a drive to enter lower power states.
Note that this change cannot be MFCed in full, because it depends on
changes to the struct bio API that break compatilibity. In order to
avoid breaking the stable API, only changes that don't touch or depend on
the struct bio changes can be merged. For example, the camcontrol(8)
changes don't depend on the new bio API, but zonectl(8) and the probe
changes to the da(4) and ada(4) drivers do depend on it.
Also note that the SMR changes have not yet been tested with an actual
SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports
ZBC to ZAC translation. I have not yet gotten a suitable drive or SAT
layer, so any testing help would be appreciated. These changes have been
tested with Seagate Host Aware SATA drives attached to both SAS and SATA
controllers. Also, I do not have any SATA Host Managed devices, and I
suspect that it may take additional (hopefully minor) changes to support
them.
Thanks to Seagate for supplying the test hardware and answering questions.
sbin/camcontrol/Makefile:
Add epc.c and zone.c.
sbin/camcontrol/camcontrol.8:
Document the zone and epc subcommands.
sbin/camcontrol/camcontrol.c:
Add the zone and epc subcommands.
Add auxiliary register support to build_ata_cmd(). Make sure to
set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA
flags as appropriate for ATA commands.
Add a new get_ata_status() function to parse ATA result from SCSI
sense descriptors (for ATA passthrough over SCSI) and ATA I/O
requests.
sbin/camcontrol/camcontrol.h:
Update the build_ata_cmd() prototype
Add get_ata_status(), zone(), and epc().
sbin/camcontrol/epc.c:
Support for ATA Extended Power Conditions features. This includes
support for all features documented in the ACS-4 Revision 12
specification from t13.org (dated February 18, 2016).
The EPC feature set allows putting a drive into a power power mode
immediately, or setting timeouts so that the drive will
automatically enter progressively lower power states after various
idle times.
sbin/camcontrol/fwdownload.c:
Update the firmware download code for the new build_ata_cmd()
arguments.
sbin/camcontrol/zone.c:
Implement support for Shingled Magnetic Recording (SMR) drives
via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA
Command Set (ZAC).
These specs were developed in concert, and are functionally
identical. The primary differences are due to SCSI and ATA
differences. (SCSI is big endian, ATA is little endian, for
example.)
This includes support for all commands defined in the ZBC and
ZAC specs.
sys/cam/ata/ata_all.c:
Decode a number of additional ATA command names in ata_op_string().
Add a new CCB building function, ata_read_log().
Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building
functions. These support both DMA and NCQ encapsulation.
sys/cam/ata/ata_all.h:
Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and
ata_zac_mgmt_in().
sys/cam/ata/ata_da.c:
Revamp the ada(4) driver to support zoned devices.
Add four new probe states to gather information needed for zone
support.
Add a new adasetflags() function to avoid duplication of large
blocks of flag setting between the async handler and register
functions.
Add new sysctl variables that describe zone support and paramters.
Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.
sys/cam/scsi/scsi_all.c:
Add command descriptions for the ZBC IN/OUT commands.
Add descriptions for ZBC Host Managed devices.
Add a new function, scsi_ata_pass() to do ATA passthrough over
SCSI. This will eventually replace scsi_ata_pass_16() -- it
can create the 12, 16, and 32-byte variants of the ATA
PASS-THROUGH command, and supports setting all of the
registers defined as of SAT-4, Revision 5 (March 11, 2016).
Change scsi_ata_identify() to use scsi_ata_pass() instead of
scsi_ata_pass_16().
Add a new scsi_ata_read_log() function to facilitate reading
ATA logs via SCSI.
sys/cam/scsi/scsi_all.h:
Add the new ATA PASS-THROUGH(32) command CDB. Add extended and
variable CDB opcodes.
Add Zoned Block Device Characteristics VPD page.
Add ATA Return SCSI sense descriptor.
Add prototypes for scsi_ata_read_log() and scsi_ata_pass().
sys/cam/scsi/scsi_da.c:
Revamp the da(4) driver to support zoned devices.
Add five new probe states, four of which are needed for ATA
devices.
Add five new sysctl variables that describe zone support and
parameters.
The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC
devices when they are attached via a SCSI to ATA Translation (SAT)
layer. Since ZBC -> ZAC translation is a new feature in the T10
SAT-4 spec, most SATA drives will be supported via ATA commands
sent via the SCSI ATA PASS-THROUGH command. The da(4) driver will
prefer the ZBC interface, if it is available, for performance
reasons, but will use the ATA PASS-THROUGH interface to the ZAC
command set if the SAT layer doesn't support translation yet.
As I mentioned above, ZBC command support is untested.
Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.
Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.
Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB
building functions. Note that these have return values, unlike
almost all other CCB building functions in CAM. The reason is
that they can fail, depending upon the particular combination
of input parameters. The primary failure case is if the user
wants NCQ, but fails to specify additional CDB storage. NCQ
requires using the 32-byte version of the SCSI ATA PASS-THROUGH
command, and the current CAM CDB size is 16 bytes.
sys/cam/scsi/scsi_da.h:
Add ZBC IN and ZBC OUT CDBs and opcodes.
Add SCSI Report Zones data structures.
Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and
scsi_ata_zac_mgmt_in() prototypes.
sys/dev/ahci/ahci.c:
Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.
ahci_setup_fis() previously set the top bits of the sector count
register in the FIS to 0 for FPDMA commands. This is okay for
read and write, because the PRIO field is in the only thing in
those bits, and we don't implement that further up the stack.
But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that
byte, so it needs to be transmitted to the drive.
In ahci_setup_fis(), always set the the top 8 bits of the
sector count register. We need it in both the standard
and NCQ / FPDMA cases.
sys/geom/eli/g_eli.c:
Pass BIO_ZONE commands through the GELI class.
sys/geom/geom.h:
Add g_io_zonecmd() prototype.
sys/geom/geom_dev.c:
Add new DIOCZONECMD ioctl, which allows sending zone commands to
disks.
sys/geom/geom_disk.c:
Add support for BIO_ZONE commands.
sys/geom/geom_disk.h:
Add a new flag, DISKFLAG_CANZONE, that indicates that a given
GEOM disk client can handle BIO_ZONE commands.
sys/geom/geom_io.c:
Add a new function, g_io_zonecmd(), that handles execution of
BIO_ZONE commands.
Add permissions check for BIO_ZONE commands.
Add command decoding for BIO_ZONE commands.
sys/geom/geom_subr.c:
Add DDB command decoding for BIO_ZONE commands.
sys/kern/subr_devstat.c:
Record statistics for REPORT ZONES commands. Note that the
number of bytes transferred for REPORT ZONES won't quite match
what is received from the harware. This is because we're
necessarily counting bytes coming from the da(4) / ada(4) drivers,
which are using the disk_zone.h interface to communicate up
the stack. The structure sizes it uses are slightly different
than the SCSI and ATA structure sizes.
sys/sys/ata.h:
Add many bit and structure definitions for ZAC, NCQ, and EPC
command support.
sys/sys/bio.h:
Convert the bio_cmd field to a straight enumeration. This will
yield more space for additional commands in the future. After
change r297955 and other related changes, this is now possible.
Converting to an enumeration will also prevent use as a bitmask
in the future.
sys/sys/disk.h:
Define the DIOCZONECMD ioctl.
sys/sys/disk_zone.h:
Add a new API for managing zoned disks. This is very close to
the SCSI ZBC and ATA ZAC standards, but uses integers in native
byte order instead of big endian (SCSI) or little endian (ATA)
byte arrays.
This is intended to offer to the complete feature set of the ZBC
and ZAC disk management without requiring the application developer
to include SCSI or ATA headers. We also use one set of headers
for ioctl consumers and kernel bio-level consumers.
sys/sys/param.h:
Bump __FreeBSD_version for sys/bio.h command changes, and inclusion
of SMR support.
usr.sbin/Makefile:
Add the zonectl utility.
usr.sbin/diskinfo/diskinfo.c
Add disk zoning capability to the 'diskinfo -v' output.
usr.sbin/zonectl/Makefile:
Add zonectl makefile.
usr.sbin/zonectl/zonectl.8
zonectl(8) man page.
usr.sbin/zonectl/zonectl.c
The zonectl(8) utility. This allows managing SCSI or ATA zoned
disks via the disk_zone.h API. You can report zones, reset write
pointers, get parameters, etc.
Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D6147
Reviewed by: wblock (documentation)
This makes it possible to manually force updating capacity data
after the disk got resized. Without it it might be neccessary to
reboot before FreeBSD notices updated disk size under eg VMWare.
Discussed with: imp@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6108
The significant changes and bugs fixed here are:
1. Fixed a bug in the progress display code:
When the user's filename is too big, or his terminal width is too
small, the progress code could wind up using a negative number for
the length of the "stars" that it uses to indicate progress.
This negative value was assigned to an unsigned variable, resulting
in a very large positive value.
The result is that we wound up writing garbage from memory to the
user's terminal.
With an 80 column terminal, a file name length of more than 35
characters would generate this problem.
To address this, we now set a minimum progress bar length, and
truncate the user's file name as needed.
This has been tested with large filenames and small terminals, and
at least produces reasonable results. If the terminal is too
narrow, the progress display takes up an additional line with each
update, but this is more user friendly than writing garbage to the
tty.
2. SATA drives connected via a SATA controller didn't have SCSI Inquiry
data populated in struct cam_device. This meant that the code in
fw_get_vendor() in fwdownload.c would try to match a zero-length
vendor ID, and so return the first entry in the vendor table. (Which
used to be HITACHI.) Fixed by grabbing identify data, passing the
identify buffer into fw_get_vendor(), and matching against the model
name.
3. SATA drives connected via a SAS controller do have Inquiry data
populated. The table included a couple of entries -- "ATA ST" and
"ATA HDS", intended to handle Seagate and Hitachi SATA drives attached
via a SAS controller. SCSI to ATA translation layers use a vendor
ID of "ATA" (which is standard), and then the model name from the ATA
identify data as the SCSI product name when they are returning data on
SATA disks. The cam_strmatch code will match the first part of the
string (because the length it is given is the length of the vendor,
"ATA"), and return 0 (i.e. a match). So all SATA drives attached to
a SAS controller would be programmed using the Seagate method
(WRITE BUFFER mode 7) of SCSI firmware downloading.
4. Issue #2 above covered up a bug in fw_download_img() -- if the
maximum packet size in the vendor table was 0, it tried to default
to a packet size of 32K. But then it didn't actually succeed in
doing that, because it set the packet size to the value that was
in the vendor table (0). Now that we actually have ATA attached
drives fall use the VENDOR_ATA case, we need a reasonable default
packet size. So this is fixed to properly set the default packet size.
5. Add support for downloading firmware to IBM LTO drives, and add a
firmware file validation method to make sure that the firmware
file matches the drive type. IBM tape drives include a Load ID and
RU name in their vendor-specific VPD page 0x3. Those should match
the IDs in the header of the firmware file to insure that the
proper firmware file is loaded.
6. This also adds a new -q option to the camcontrol fwdownload
subcommand to suppress informational output. When -q is used in
combination with -y, the firmware upgrade will happen without
prompting and without output except if an error condition occurs.
7. Re-add support for printing out SCSI inquiry information when
asking the user to confirm that they want to download firmware, and
add printing of ATA Identify data if it is a SATA disk. This was
removed in r237281 when support for flashing ATA disks was added.
8. Add a new camcontrol(8) "opcodes" subcommand, and use the
underlying code to get recommended timeout values for drive
firmware downloads.
Many SCSI devices support the REPORT SUPPORTED OPERATION CODES
command, and some support the optional timeout descriptor that
specifies nominal and recommended timeouts for the commands
supported by the device.
The new camcontrol opcodes subcommand allows displaying all
opcodes supported by a drive, information about which fields
in a SCSI CDB are actually used by a given SCSI device, and the
nominal and recommended timeout values for each command.
Since firmware downloads can take a long time in some devices, and
the time varies greatly between different types of devices, take
advantage of the infrastructure used by the camcontrol opcodes
subcommand to determine the best timeout to use for the WRITE
BUFFER command in SCSI device firmware downloads.
If the device recommends a timeout, it is likely to be more
accurate than the default 50 second timeout used by the firmware
download code. If the user specifies a timeout, it will override
the default or device recommended timeout. If the device doesn't
support timeout descriptors, we fall back to the default.
9. Instead of downloading firmware to SATA drives behind a SAS controller
using WRITE BUFFER, use the SCSI ATA PASS-THROUGH command to compose
an ATA DOWNLOAD MICROCODE command and it to the drive. The previous
version of this code attempted to send a SCSI WRITE BUFFER command to
SATA drives behind a SAS controller. Although that is part of the
SAT-3 spec, it doesn't work with the parameters used with LSI
controllers at least.
10.Add a new mechanism for making common ATA passthrough and
ATA-behind-SCSI passthrough commands.
The existing camcontrol(8) ATA command mechanism checks the device
type on every command executed. That works fine for individual
commands, but is cumbersome for things like a firmware download
that send a number of commands.
The fwdownload code detects the device type up front, and then
sends the appropriate commands.
11.In simulation mode (-s), if the user specifies the -v flag, print out
the SCSI CDB or ATA registers that would be sent to the drive. This will
aid in debugging any firmware download issues.
sbin/camcontrol/fwdownload.c:
Add a device type to the fw_vendor structure, so that we can
specify different download methods for different devices from the
same vendor. In this case, IBM hard drives (from when they
still made hard drives) and tape drives.
Add a tur_status field to the fw_vendor structure so that we can
specify whether the drive to be upgraded should be ready, not
ready, or whether it doesn't matter. Add the corresponding
capability in fw_download_img().
Add comments describing each of the vendor table fields.
Add HGST and SmrtStor to the supported SCSI vendors list.
In fw_get_vendor(), look at ATA identify data if we have a SATA
device to try to identify what the drive vendor is.
Add IBM firmware file validation. This gets VPD page 0x3, and
compares the Load ID and RU name in the page to the values
included in the header. The validation code will refuse to load
a firmware file if the values don't match. This does allow the
user to attempt a downgrade; whether or not it succeeds will
likely depend on the drive settings.
Add a -q option, and disable all informative output
(progress bars, etc.) when this is enabled.
Re-add the inquiry in the confirmation dialog so the user has
a better idea of which device he is talking to. Add support for
displaying ATA identify data.
Don't automatically disable confirmation in simulation (-s) mode.
This allows the user to see the inquiry or identify data in the
dialog, and see exactly what they would see when the command
actually runs. Also, in simulation mode, if the user specifies
the -v flag, print out the SCSI CDB or ATA registers that would
be sent to the drive. This will aid in debugging any firmware
download issues.
Add a timeout field and timeout type to the firmware download
vendor table. This allows specifying a default timeout and allows
specifying whether we should attempt to probe for a recommended
timeout from the drive.
Add a new fuction, fw_get_timeout(), that will determine
which timeout to use for the WRITE BUFFER command. If the
user specifies a timeout, we always use that. Otherwise,
we will use the drive recommended timeout, if available,
and fall back to the default when a drive recommended
timeout isn't available.
When we prompt the user, tell him what timeout we're going
to use, and the source of the timeout.
Revamp the way SATA devices are handled.
In fwdownload(), use the new get_device_type() function to
determine what kind of device we're talking to.
Allow firmware downloads to any SATA device, but restrict
SCSI downloads to known devices. (The latter is not a
change in behavior.)
Break out the "ready" check from fw_download_img() into a
new subfunction, fw_check_device_ready(). This sends the
appropriate command to the device in question -- a TEST
UNIT READY or an IDENTIFY. The IDENTIFY for SATA devices
a SAT layer is done using the SCSI ATA PASS-THROUGH
command.
Use the new build_ata_cmd() function to build either a SCSI or
ATA I/O CCB to issue the DOWNLOAD MICROCODE command to SATA
devices. build_ata_cmd() figures looks at the devtype argument
and fills in the correct CCB type and CDB or ATA registers.
Revamp the vendor table to remove the previous
vendor-specific ATA entries and use a generic ATA vendor
placeholder. We currently use the same method for all ATA
drives, although we may have to add vendor-specific
behavior once we test this with more drives.
sbin/camcontrol/progress.c:
In progress_draw(), make barlength a signed value so that
we can easily detect a negative value.
If barlength (the length of the progress bar) would wind up
negative due to a small TTY width or a large filename,
set the bar length to the new minimum (10 stars) and
truncate the user's filename. We will truncate it down to
0 characters if necessary.
Calculate a new prefix_len variable (user's filename length)
and use it as the precision when printing the filename.
sbin/camcontrol/camcontrol.c:
Implement a new camcontrol(8) subcommand, "opcodes". The
opcodes subcommand allows displaying the entire list of
SCSI commands supported by a device, or details on an
individual command. In either case, it can display
nominal and recommended timeout values.
Add the scsiopcodes() function, which calls the new
scsigetopcodes() function to fetch opcode data from a
drive.
Add two new functions, scsiprintoneopcode() and
scsiprintopcodes(), which print information about one
opcode or all opcodes, respectively.
Remove the get_disk_type() function. It is no longer used.
Add a new function, dev_has_vpd_page(), that fetches the
supported INQUIRY VPD list from a device and tells the
caller whether the requested VPD page is available.
Add a new function, get_device_type(), that returns a more
precise device type than the old get_disk_type() function.
The get_disk_type() function only distinguished between
SCSI and ATA devices, and SATA devices behind a SCSI to ATA
translation layer were considered to be "SCSI".
get_device_type() offers a third type, CC_DT_ATA_BEHIND_SCSI.
We need to know this to know whether to attempt to send ATA
passthrough commands. If the device has the ATA
Information VPD page (0x89), then it is an ATA device
behind a SCSI to ATA translation layer.
Remove the type argument from the fwdownload() subcommand.
Add a new function, build_ata_cmd(), that will take one set
of common arguments and build either a SCSI or ATA I/O CCB,
depending on the device type passed in.
sbin/camcontrol/camcontrol.h:
Add a prototype for scsigetopcodes().
Add a new enumeration, camcontrol_devtype.
Add prototypes for dev_has_vpd_page(), get_device_type()
and build_ata_cmd().
Remove the type argument from the fwdownload() subcommand.
sbin/camcontrol/camcontrol.8
Explain that the fwdownload subcommand will use the drive
recommended timeout if available, and that the user can
override the timeout.
Document the new opcodes subcommand.
Explain that we will attempt to download firmware to any
SATA device.
Document supported SCSI vendors, and models tested if known.
Explain the commands used to download firmware for the
three different drive and controller combinations.
Document that the -v flag in simulation mode for the fwdownload
subcommand will print out the SCSI CDBs or ATA registers that would
be used.
sys/cam/scsi/scsi_all.h:
Add new bit definitions for the one opcode descriptor for
the REPORT SUPPORTED OPCODES command.
Add a function prototype for scsi_report_supported_opcodes().
sys/cam/scsi/scsi_all.c:
Add a new CDB building function, scsi_report_supported_opcodes().
Sponsored by: Spectra Logic
MFC after: 1 week
MAM is Medium Auxiliary Memory and is most commonly found as flash
chips on tapes.
This includes support for reading attributes and decoding most
known attributes, but does not yet include support for writing
attributes or reporting attributes in XML format.
libsbuf/Makefile:
Add subr_prf.c for the new sbuf_hexdump() function. This
function is essentially the same function.
libsbuf/Symbol.map:
Add a new shared library minor version, and include the
sbuf_hexdump() function.
libsbuf/Version.def:
Add version 1.4 of the libsbuf library.
libutil/hexdump.3:
Document sbuf_hexdump() alongside hexdump(3), since it is
essentially the same function.
camcontrol/Makefile:
Add attrib.c.
camcontrol/attrib.c:
Implementation of READ ATTRIBUTE support for camcontrol(8).
camcontrol/camcontrol.8:
Document the new 'camcontrol attrib' subcommand.
camcontrol/camcontrol.c:
Add the new 'camcontrol attrib' subcommand.
camcontrol/camcontrol.h:
Add a function prototype for scsiattrib().
share/man/man9/sbuf.9:
Document the existence of sbuf_hexdump() and point users to
the hexdump(3) man page for more details.
sys/cam/scsi/scsi_all.c:
Add a table of known attributes, text descriptions and
handler functions.
Add a new scsi_attrib_sbuf() function along with a number
of other related functions that help decode attributes.
scsi_attrib_ascii_sbuf() decodes ASCII format attributes.
scsi_attrib_int_sbuf() decodes binary format attributes, and
will pass them off to scsi_attrib_hexdump_sbuf() if they're
bigger than 8 bytes.
scsi_attrib_vendser_sbuf() decodes the vendor and drive
serial number attribute.
scsi_attrib_volcoh_sbuf() decodes the Volume Coherency
Information attribute that LTFS writes out.
sys/cam/scsi/scsi_all.h:
Add a number of attribute-related structure definitions and
other defines.
Add function prototypes for all of the functions added in
scsi_all.c.
sys/kern/subr_prf.c:
Add a new function, sbuf_hexdump(). This is the same as
the existing hexdump(9) function, except that it puts the
result in an sbuf.
This also changes subr_prf.c so that it can be compiled in
userland for includsion in libsbuf.
We should work to change this so that the kernel hexdump
implementation is a wrapper around sbuf_hexdump() with a
statically allocated sbuf with a drain. That will require
a drain function that goes to the kernel printf() buffer
that can take a non-NUL terminated string as input.
That is because an sbuf isn't NUL-terminated until it is
finished, and we don't want to finish it while we're still
using it.
We should also work to consolidate the userland hexdump and
kernel hexdump implemenatations, which are currently
separate. This would also mean making applications that
currently link in libutil link in libsbuf.
sys/sys/sbuf.h:
Add the prototype for sbuf_hexdump(), and add another copy
of the hexdump flag values if they aren't already defined.
Ideally the flags should be defined in one place but the
implemenation makes it difficult to do properly. (See
above.)
Sponsored by: Spectra Logic Corporation
MFC after: 1 week
This includes a new summary mode (-s) for camcontrol defects that
quickly tells the user the most important thing: how many defects
are in the requested list. The actual location of the defects is
less important.
Modern drives frequently have more than the 8191 defects that can
be reported by the READ DEFECT DATA (10) command. If they don't
have that many grown defects, they certainly have more than 8191
defects in the primary (i.e. factory) defect list.
The READ DEFECT DATA (12) command allows for longer parameter
lists, as well as indexing into the list of defects, and so allows
reporting many more defects.
This has been tested with HGST drives and Seagate drives, but
does not fully work with Seagate drives. Once I have a Seagate
spec I may be able to determine whether it is possible to make it
work with Seagate drives.
scsi_da.h: Add a definition for the new long block defect
format.
Add bit and mask definitions for the new extended
physical sector and bytes from index defect
formats.
Add a prototype for the new scsi_read_defects() CDB
building function.
scsi_da.c: Add a new scsi_read_defects() CDB building function.
camcontrol(8) was previously composing CDBs manually.
This is long overdue.
camcontrol.c: Revamp the camcontrol defects subcommand. We now
go through multiple stages in trying to get defect
data off the drive while avoiding various drive
firmware quirks.
We start off by requesting the defect header with
the 10 byte command. If we're in summary mode (-s)
and the drive reports fewer defects than can be
represented in the 10 byte header, we're done.
Otherwise, we know that we need to issue the
12 byte command if the drive reports the maximum
number of defects.
If we're in summary mode, we're done if we get a
good response back when asking for the 12 byte header.
If the user has asked for the full list, then we
use the address descriptor index field in the 12
byte CDB to step through the list in 64K chunks.
64K is small enough to work with most any ancient
or modern SCSI controller.
Add support for printing the new long block defect
format, as well as the extended physical sector and
bytes from index formats. I don't have any drives
that support the new formats.
Add a hexadecimal output format that can be turned
on with -X.
Add a quiet mode (-q) that can be turned on with
the summary mode (-s) to just print out a number.
Revamp the error detection and recovery code for
the defects command to work with HGST drives.
Call the new scsi_read_defects() CDB building
function instead of rolling the CDB ourselves.
Pay attention to the residual from the defect list
request when printing it out, so we don't run off
the end of the list.
Use the new scsi_nv library routines to convert
from strings to numbers and back.
camcontrol.8: Document the new defect formats (longblock, extbfi,
extphys) and command line options (-q, -s, -S and
-X) for the defects subcommand.
Explain a little more about what drives generally
do and don't support.
Sponsored by: Spectra Logic
MFC after: 1 week
camcontrol(8) now supports a new 'persist' subcommand that allows users to
issue SCSI PERSISTENT RESERVE IN / OUT commands.
sbin/camcontrol/Makefile:
Add persist.c.
sbin/camcontrol/persist.c:
New persistent reservation support for camcontrol(8).
We have support for all known operation modes for PERSISTENT RESERVE
IN and PERSISTENT RESERVE OUT.
exceptions noted above.
sbin/camcontrol/camcontrol.8:
Document the new 'persist' subcommand.
In the section on the Transport ID (-I) option, explain what
Transport IDs for each protocol should look like. At some point
some of this information could probably get moved off in a
separate man page, either on Transport IDs alone or a man page
documenting the Transport ID parsing code.
Add a number of examples of persistent reservation commands.
Persistent Reservations are complex enough that the average user
probably won't be able to get the commands exactly right by just
reading the man page. These examples show a few basic and
advanced examples of how to use persistent reservations.
sbin/camcontrol/camcontrol.h:
Move the definition for camcontrol_optret here, so we can use it
for the persistent reservation code.
Add a definition for the new scsipersist() function.
sbin/camcontrol/camcontrol.c:
Add 'persist' to the list of subcommands.
Document 'persist' in the help text.
sys/cam/scsi/scsi_all.c:
Add the scsi_persistent_reserve_in() and
scsi_persistent_reserve_out() CCB building functions.
Add a new function, scsi_transportid_sbuf(). This takes a
SCSI Transport ID (documented in SPC-4), and prints it to
an sbuf(9). There are some transports (like ATA, USB, and
SSA) for which there is no transport defined. We need to
come up with a reasonable thing to do if we're presented
with a Transport ID that claims to be for one of those
protocols.
Add new routines scsi_get_nv() and scsi_nv_to_str().
These functions do a table lookup to go between a string and an
integer. There are lots of table lookups needed in the
persistent reservation code in camcontrol(8).
Add a new function, scsi_parse_transportid(), along with leaf node
functions to parse:
FC, 1394 and SAS (scsi_parse_transportid_64bit())
iSCSI (scsi_parse_transportid_iscsi())
SPI (scsi_parse_transportid_spi())
RDMA (scsi_parse_transportid_rdma())
PCIe (scsi_parse_transportid_sop())
Transport IDs. Given a string with the general form proto,id these
functions create a SCSI Transport ID structure.
sys/cam/scsi/scsi_all.h:
Update the various persistent reservation data structures to
SPC4r36l, but also rename some fields that were previously
obsolete with the proper names from older SCSI specs. This
allows using older, obsolete persistent reservation types when
desired.
Add function prototypes for the new persistent reservation CCB
building functions.
Add a data strucure for the READ FULL STATUS service action
of the PERSISTENT RESERVE IN command.
Add Transport ID structures for all protocols described in SPC-4.
Add a new series of SCSI_PROTO_XXX definitions, and
redefine other defines in terms of these new definitions.
Add a prototype for scsi_transportid_sbuf().
Change a couple of "obsolete" persistent reservation data
structure fields into something more meaningful, based on
what the field was called when it was defined in the spec.
(e.g. SPC, SPC-2, etc.)
Create a new define, SPRI_MAX_LEN, for the maximum allocation
length allowed for the PERSISTENT RESERVE IN command.
Add data structures and enumerations for the new name/value
translation functions.
Add data structures for SCSI over PCIe Routing IDs.
Bring the PERSISTENT RESERVE OUT Register and Move parameter list
structure (struct scsi_per_res_out_parms) up to date with SPC-4.
Add a data structure for the transport IDs that can optionally be
appended to the basic PERSISTENT RESERVE OUT parameter list.
Move SCSI protocol macro definitions out of the VPD page 0x83
definition and combine them with the more up to date protocol
definitions higher in the file.
Add function prototypes for scsi_nv_to_str(), scsi_get_nv(),
scsi_parse_transportid_64bit(), scsi_parse_transportid_spi(),
scsi_parse_transportid_rdma(), scsi_parse_transportid_iscsi(),
scsi_parse_transportid_sop(), and scsi_parse_transportid().
Sponsored by: Spectra Logic Corporation
MFC after: 1 week
disks such as SSD's
Adds the ability to run ATA commands via the SCSI ATA Pass-Through(16) comand
Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks
defect information it has before grabbing the full defect list.
This works around a bug with some Hitachi drives that generate data overrun
errors when they are asked for more defect data than they have.
The change is done in a spec-compliant way, so it should have no negative
impact on drives that don't have this issue.
This is based on work originally done at Sandvine.
scsi_da.h: Add a define for the maximum amount of data that can be
contained in a defect list.
camcontrol.c: Update the readdefects() function to issue an initial
command to determine the length of the defect list, and
then use that length in the request for the full defect
list.
camcontrol.8: Add a note that some drives will report 0 defects available
if you don't request either the PLIST or GLIST.
Submitted by: Mark Johnston <markjdb@gmail.com> (original version)
MFC after: 3 days
Firmware can be reprogrammed on devices from Hitachi, HP, IBM, Plextor,
Quantum, and Seagate. At least one device from each manufacturer has
been tested with some version of this code, and it has been used to
update thousands of drives so far.
The man page suggests having a backup of the drive's data, and the
operation must be confirmed, either interactively or on the command
line. (This is the same as the confirmation on the format command.)
This work is largely derived from fwprog.c by Andre Albsmeier.
Submitted by: Nima Misaghian
Sponsored by: Sandvine Incorporated
MFC after: 3 months
This includes support in the kernel, camcontrol(8), libcam and the mps(4)
driver for SMP passthrough.
The CAM SCSI probe code has been modified to fetch Inquiry VPD page 0x00
to determine supported pages, and will now fetch page 0x83 in addition to
page 0x80 if supported.
Add two new CAM CCBs, XPT_SMP_IO, and XPT_GDEV_ADVINFO. The SMP CCB is
intended for SMP requests and responses. The ADVINFO is currently used to
fetch cached VPD page 0x83 data from the transport layer, but is intended
to be extensible to fetch other types of device-specific data.
SMP-only devices are not currently represented in the CAM topology, and so
the current semantics are that the SIM will route SMP CCBs to either the
addressed device, if it contains an SMP target, or its parent, if it
contains an SMP target. (This is noted in cam_ccb.h, since it will change
later once we have the ability to have SMP-only devices in CAM's topology.)
smp_all.c,
smp_all.h: New helper routines for SMP. This includes
SMP request building routines, response parsing
routines, error decoding routines, and structure
definitions for a number of SMP commands.
libcam/Makefile: Add smp_all.c to libcam, so that SMP functionality
is available to userland applications.
camcontrol.8,
camcontrol.c: Add smp passthrough support to camcontrol. Several
new subcommands are now available:
'smpcmd' functions much like 'cmd', except that it
allows the user to send generic SMP commands.
'smprg' sends the SMP report general command, and
displays the decoded output. It will automatically
fetch extended output if it is available.
'smppc' sends the SMP phy control command, with any
number of potential options. Among other things,
this allows the user to reset a phy on a SAS
expander, or disable a phy on an expander.
'smpmaninfo' sends the SMP report manufacturer
information and displays the decoded output.
'smpphylist' displays a list of phys on an
expander, and the CAM devices attached to those
phys, if any.
cam.h,
cam.c: Add a status value for SMP errors
(CAM_SMP_STATUS_ERROR).
Add a missing description for CAM_SCSI_IT_NEXUS_LOST.
Add support for SMP commands to cam_error_string().
cam_ccb.h: Rename the CAM_DIR_RESV flag to CAM_DIR_BOTH. SMP
commands are by nature bi-directional, and we may
need to support bi-directional SCSI commands later.
Add the XPT_SMP_IO CCB. Since SMP commands are
bi-directional, there are pointers for both the
request and response.
Add a fill routine for SMP CCBs.
Add the XPT_GDEV_ADVINFO CCB. This is currently
used to fetch cached page 0x83 data from the
transport later, but is extensible to fetch many
other types of data.
cam_periph.c: Add support in cam_periph_mapmem() for XPT_SMP_IO
and XPT_GDEV_ADVINFO CCBs.
cam_xpt.c: Add support for executing XPT_SMP_IO CCBs.
cam_xpt_internal.h: Add fields for VPD pages 0x00 and 0x83 in struct
cam_ed.
scsi_all.c: Add scsi_get_sas_addr(), a function that parses
VPD page 0x83 data and pulls out a SAS address.
scsi_all.h: Add VPD page 0x00 and 0x83 structures, and a
prototype for scsi_get_sas_addr().
scsi_pass.c: Add support for mapping buffers in XPT_SMP_IO and
XPT_GDEV_ADVINFO CCBs.
scsi_xpt.c: In the SCSI probe code, first ask the device for
VPD page 0x00. If any VPD pages are supported,
that page is required to be implemented. Based on
the response, we may probe for the serial number
(page 0x80) or device id (page 0x83).
Add support for the XPT_GDEV_ADVINFO CCB.
sys/conf/files: Add smp_all.c.
mps.c: Add support for passing in a uio in mps_map_command(),
so we can map a S/G list at once.
Add support for SMP passthrough commands in
mps_data_cb(). SMP is a special case, because the
first buffer in the S/G list is outbound and the
second buffer is inbound.
Add support for warning the user if the busdma code
comes back with more buffers than will work for the
command. This will, for example, help the user
determine why an SMP command failed if busdma comes
back with three buffers.
mps_pci.c: Add sys/uio.h.
mps_sas.c: Add the SAS address and the parent handle to the
list of fields we pull from device page 0 and cache
in struct mpssas_target. These are needed for SMP
passthrough.
Add support for the XPT_SMP_IO CCB. For now, this
CCB is routed to the addressed device if it supports
SMP, or to its parent if it does not and the parent
does. This is necessary because CAM does not
currently support SMP-only nodes in the topology.
Make SMP passthrough support conditional on
__FreeBSD_version >= 900026. This will make it
easier to MFC this change to the driver without
MFCing the CAM changes as well.
mps_user.c: Un-staticize mpi_init_sge() so we can use it for
the SMP passthrough code.
mpsvar.h: Add a uio and iovecs into struct mps_command for
SMP passthrough commands.
Add a cm_max_segs field to struct mps_command so
that we can warn the user if busdma comes back with
too many segments.
Clear the cm_reply when a command gets freed. If
it is not cleared, reply frames will eventually get
freed into the pool multiple times and corrupt the
pool. (This fix is from scottl.)
Add a prototype for mpi_init_sge().
sys/param.h: Bump __FreeBSD_version to 900026 for the for the
inclusion of the XPT_GDEV_ADVINFO and XPT_SMP_IO
CAM CCBs.
It could be used for broad range of tasks, such as configuring drive
power management modes, caching, security and any other features and tasks,
not supported by existing drivers.
modularize it so that new transports can be created.
Add a transport for SATA
Add a periph+protocol layer for ATA
Add a driver for AHCI-compliant hardware.
Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.
The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.
ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.
This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.
Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.
The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.
Submitted by: scottl, mav
Approved by: re
will automatically issue the 16 byte verison of read capacity if the device
in question is larger than 2TB.
There are also a number of output options here (last block, number of
blocks, human readable) that should meet most needs, and also aid in
scripting.
Approved by: re (bmah)
MFC after: 1 week
REPORT LUNS command to a device.
camcontrol.[c8]: Implement reportluns. This tries to print the LUNs
out in a reasonable format. Only the periph
addressing method has been tested, since very little
hardware that I know of supports the other methods.
scsi_all.[ch]: Revamp the report luns CDB structure and helper
functions. This constitutes a little bit of an API
change, but since the old CDB length was 10 bytes,
and the REPORT LUNS CDB length is actually 12 bytes,
it's clear that no one was using this API in the
first place.
MFC After: 1 week
report on the status of a format already running on a drive.
Fix status reporting for 'camcontrol format'. This was broken in rev 1.34
of camcontrol.c, almost 4 years ago!
Submitted by: joerg (most of the reportonly changes)
MFC after: 3 days
