Commit Graph

1854 Commits

Author SHA1 Message Date
Alexander Motin
c71150ce5d Extend mask of VMware virtual disks. 2015-11-05 09:07:53 +00:00
Alexander Motin
10643dd70e Don't try to replicate mode pages not present on this device.
MFC after:	3 days
2015-10-26 14:14:56 +00:00
Alexander Motin
59f063d549 Give CTL support for PIM_EXTLUNS when talking to CAM.
CTL itself still lives in flat LUN space, but it can generate extended
numbers if CAM SIM reports such capability.
2015-10-24 17:24:19 +00:00
Alexander Motin
c98d2b1f1e Add partial support for QUERY TMF to CAM and isp(4).
This change allows to decode respective functions in isp(4) in target mode
and pass them through CAM to CTL.  Unfortunately neither CAM nor isp(4)
support returning response info for those task management functions now.

On the other side I just have no initiator to test this functionality.
2015-10-23 18:34:18 +00:00
Alexander Motin
aeb1faa040 Fix LUN disable in CAM broken at r285155.
MFC after:	1 week
2015-10-23 10:39:43 +00:00
Alexander Motin
5124012aae Make some panic strings mode informative. 2015-10-21 15:31:26 +00:00
Alexander Motin
0ac03010f6 Make delete method set via kern.cam.da.X.delete_method persistent.
This allows to set delete method via tunable, before device capabilities
are known.  Also allow ZERO method for devices not reporting LBP, if user
explicitly requests it -- it may be useful if storage supports compression
and WRITE SAME, but does not support UNMAP.

MFC after:	2 weeks
2015-10-11 18:26:06 +00:00
Alexander Motin
6854699543 Remove legacy CHS geometry from dmesg and unify capacity outputs. 2015-10-11 13:48:20 +00:00
Alexander Motin
4a3760bae6 Remove compatibility shims for legacy ATA device names.
We got new ATA stack in FreeBSD 8.x, switched to it at 9.x, completely
removed old stack at 10.x, so at 11.x it is time to remove compat shims.
2015-10-11 13:01:51 +00:00
Alexander Motin
361e885315 Remove lock upgrade attempt from ctl_be_block_open_file().
I am not sure what for it was done.  Now open routine should automatically
fall back to read-only if open for writing is impossible.  In such case
attempt to upgrade to write sounds strange.

MFC after:	1 week
2015-10-11 08:28:49 +00:00
Alexander Motin
3d5cb709bd Add missing vnode lock in case of file modify request.
Submitted by:	Richard Kojedzinszky
MFC after:	1 week
2015-10-08 07:34:30 +00:00
Alexander Motin
0952a19f7d More aggressively fill WUT read pipeline.
On some tests I've measured 5% copy speedup from this.
2015-10-01 19:07:15 +00:00
Alexander Motin
6ac1446d0e Make zero WUT use WRITE SAME with recently allowed NDOB flag. 2015-10-01 16:30:20 +00:00
Alexander Motin
f24b1dc90c Implement SPC-3 exceptions to SPC-2 RESERVE and RELEASE behavior. 2015-10-01 12:57:37 +00:00
Alexander Motin
f53270c858 Unify PR variable names to reduce confusion. 2015-10-01 12:15:36 +00:00
Alexander Motin
f5a2bbe6d6 Use proper STAILQ_* macros where possible. 2015-09-30 20:38:35 +00:00
Alexander Motin
de2393124c Make pass, sg and targ drivers respect HBA's maxio.
Previous limitation of 64K (DFLTPHYS) is quite annoying.
2015-09-30 13:31:37 +00:00
Alexander Motin
66b6967686 Really implement PREVENT ALLOW MEDIUM REMOVAL command. 2015-09-29 15:12:40 +00:00
Alexander Motin
7f3d6f1fe3 Don't report SYNC_NV bit set in SYNCHRONIZE CACHE as error.
While this bit is obsolete in SBC-3, behavior controlled by it is allowed
on device discretion.
2015-09-29 13:58:27 +00:00
Alexander Motin
862aedb0d6 Fix arguments order. 2015-09-29 12:53:41 +00:00
Alexander Motin
6908da41fa Report that we can read all flavours of DVD. Why not? 2015-09-29 10:44:37 +00:00
Alexander Motin
d6e7f6e741 Add CD/DVD Capabilities and Mechanical Status Page.
This page is obsolete since MMC-4, but still used by some software.
2015-09-29 09:09:37 +00:00
Alexander Motin
648dfc1a29 Umplement media load/eject support for removable devices.
In case of block backend eject really closes the backing store, while
load tries to open it back.  Failed store open is reported as no media.
2015-09-28 20:54:18 +00:00
Alexander Motin
91be33dc78 Add to CTL initial support for CDROMs and removable devices.
Relnotes:	yes
2015-09-27 13:47:28 +00:00
Alexander Motin
2e33ae99cf Allow LOG SENSE command on non-disk devices. 2015-09-26 13:51:29 +00:00
Alexander Motin
0b33261bbe Remove concept of control device. 2015-09-26 12:53:55 +00:00
Alexander Motin
6bff2b5bff Move ioctl frontend defines where they belong. 2015-09-26 11:56:28 +00:00
Alexander Motin
d3ab449cfa Remove few more unused variables. 2015-09-26 11:39:54 +00:00
Alexander Motin
9c887a4f86 Remove some duplicate, legacy, dead and questionable code. 2015-09-26 11:28:45 +00:00
Alexander Motin
0bb9989c43 Pass to sobind() copy of the address, not the original. 2015-09-26 01:23:29 +00:00
Alexander Motin
86b7954b92 Properly lock LUN in ctl_failover_lun(). 2015-09-25 22:45:23 +00:00
Alexander Motin
f44570f617 Fix bit masks changed for the wrong commands in r288221. 2015-09-25 19:46:34 +00:00
Alexander Motin
5d4dfee032 Constify ctl_serialize_table. 2015-09-25 18:49:25 +00:00
Alexander Motin
c30a4c1871 Remove some dead code found by Clang analyzer. 2015-09-25 18:15:34 +00:00
Alexander Motin
67cc546dfc Remove stale comments and some excessive empty lines. 2015-09-25 16:34:59 +00:00
Alexander Motin
e675024a02 Switch I/O time accounting from system time to uptime.
While there, make num_dmas accounted independently of CTL_TIME_IO.
2015-09-25 10:14:39 +00:00
Alexander Motin
6c068d4bf3 Collect DMA statistics on secondary HA node. 2015-09-25 09:55:51 +00:00
Alexander Motin
116c5818ec Make HA handle datamove and done in a universal way, using port methods.
Now from primary node point of view requests transferred from secondary
node should look almost normal and always have valid port.
2015-09-25 09:14:29 +00:00
Alexander Motin
21d963e528 Remove some control_softc references. 2015-09-25 07:27:23 +00:00
Alexander Motin
6c2acea564 Allow WRITE SAME with NDOB bit set but without UNMAP.
This combination was originally forbidden, but allowed at spc4r3.
2015-09-24 15:59:08 +00:00
Alexander Motin
4ef0129a46 Add new report types to REPORT LUNS command.
This is only for completeness, since we have nothing new to report there.
2015-09-24 12:22:47 +00:00
Alexander Motin
a6daea64fd Update WRITE ATOMIC(16) support to sbc4r8 draft.
This is only a cosmetic change.  We still don't support atomic boundary
field in the CDB, but at least now we do it formally.
2015-09-24 08:04:47 +00:00
Alexander Motin
de988746be Add support for READ BUFFER(16) command. 2015-09-24 07:16:34 +00:00
Alexander Motin
ca85b7c4e9 Synchronize mode pages between HA peers.
We allow to modify only few fields in mode pages now, but still it is
not good if they unexpectedly change during failover.  Also this fixes
reporting of "Mode parameters changed" UAs on secondary node.
2015-09-23 18:33:00 +00:00
Alexander Motin
a85700a912 Make HA peers announce their parameters on connect.
HA protocol requires strict version, parameters and configuration match.
Differences there may cause full set of problems up to kernel panic.
To avoid that, validate peer parameters on connect, and abort connection
immediately if some mismatch detected.
2015-09-23 15:49:27 +00:00
Alexander Motin
c53993057b Add support for Control extension mode page. 2015-09-22 14:55:46 +00:00
Alexander Motin
4ce7a0868c Remove duplicate and incomplete code handling LUN modify.
Instead reuse code from LUN creation.  This allows most of LUN media
options to be changed live with modify request without full restart.
2015-09-22 10:45:50 +00:00
Alexander Motin
8153ec9ad5 Log iSCSI session reinstatements.
False session reinstatements can be result of misconfiguration, when
several initiators use the same initiator name and ISID.
2015-09-21 13:25:29 +00:00
Alexander Motin
218d25d00b Mark with DMA flag I/Os waiting for iSCSI write data after R2T.
Reads and immediate writes are not blocking, so don't bother.
2015-09-20 11:39:19 +00:00
Alexander Motin
b22213694e Remove couple excess SGLIST I/O flags.
Those flags duplicated respective (sg_entries > 0) values.
2015-09-20 10:40:30 +00:00
Alexander Motin
921af50782 Always execute REPORT LUNS and REQUEST SENSE commands locally.
REPORT LUNS command is more related to target rather then specific LUN.
This node may be primary for LUNs for some reason unknown to another,
and command forwarded to another node won't be able to report them.

REQUEST SENSE is related to LUN, but in our implementation it reports
only UAs and CAs, that are stored locally rather then on primary node.
2015-09-19 19:30:55 +00:00
Alexander Motin
efbf6139a4 Split two command flags with different meaning.
This is only a cosmetical change.
2015-09-19 19:11:59 +00:00
Alexander Motin
027dd0cfd0 Pack struct ctl_ha_msg_hdr by 8 bytes. 2015-09-19 18:23:06 +00:00
Alexander Motin
6861045bda Fix memory corruption when >128K transferred through HA link.
While there, do some cleaning.
2015-09-19 13:12:09 +00:00
Alexander Motin
bc8749e2ff Mark I/Os with DMA flag while moving data through the HA link. 2015-09-18 20:11:10 +00:00
Alexander Motin
75a3108e13 Relax serseq option operation for reads.
Previously, with serseq enabled, next command was unblocked only after
previous completed.  With this change, for read operations, next command
is unblocked as soon as last media read completed.  This is important
for frontends that actually wait for data move completion (like camtgt),
or when data are moved through the HA link, or especially when both.
2015-09-18 19:43:14 +00:00
Alexander Motin
59bb97a934 Kill HA link and shutdown the threads on shutdown. 2015-09-18 12:08:00 +00:00
Alexander Motin
1c69dbd098 Update list of opcodes to 5/26/15. 2015-09-18 10:44:25 +00:00
Alexander Motin
f90e68de18 Update list of ASC/ASCQ codes from 5/20/12 to 8/12/15. 2015-09-18 10:23:17 +00:00
Alexander Motin
0c05f0dc16 Replicate initiators WWPNs and names between HA peers. 2015-09-17 21:51:11 +00:00
Alexander Motin
54713bce25 Replicate port->init_devid to HA peer. 2015-09-17 20:36:34 +00:00
Alexander Motin
6213882769 When reporting TPT UA, report which of thresholds was reached. 2015-09-17 17:00:36 +00:00
Alexander Motin
042e9bdc41 Report number of failed XCOPY segment. 2015-09-17 14:22:52 +00:00
Alexander Motin
7f7bb97a0f Report proper medium error code for VERIFY commands. 2015-09-17 12:52:18 +00:00
Alexander Motin
83981e319d Fix reading after end of file for file-backed LUNs.
If backing file is smaller then the LUN size, we have to explicitly clear
the rest of the buffer to not leak some random data from previous I/Os.
2015-09-16 21:43:51 +00:00
Alexander Motin
d6043e4643 Make COMPARE AND WRITE report offset of difference. 2015-09-16 18:33:04 +00:00
Alexander Motin
723c363f7f Fix fixed sense writing when passed more data then it can fit.
MFC after:	1 week
2015-09-16 17:56:24 +00:00
Alexander Motin
5c5fb901eb Frontends don't need to set errors themselves. 2015-09-16 13:25:35 +00:00
Alexander Motin
e2c3044b13 Don't flap the HA link if sysctl is reset to the same value. 2015-09-16 09:59:05 +00:00
Alexander Motin
c39d464164 Make CAM log errors that make it wait.
Waiting can take minutes, and it would be good for user to know what is
going on.

MFC after:	2 weeks
2015-09-15 10:57:16 +00:00
Alexander Motin
18e87f95f7 Fix completion/error status reporting. 2015-09-15 10:42:53 +00:00
Alexander Motin
923863546a Close potential race between datamove and HA failover. 2015-09-15 09:36:46 +00:00
Mark Johnston
87dd1668e0 Preserve the device queue status before retrying a sense request in
chdone(). Previously, the retry could clear the CAM_DEV_QFRZN bit in the
CCB status, leaving the queue frozen.

Submitted by:	Jeff Miller <Jeff.Miller@isilon.com>
Reviewed by:	ken
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-09-15 05:09:17 +00:00
Alexander Motin
d0f25804a7 Implement data/status aggregation for secondary HA node.
For short read requests this reduces latency by 30%, reporting command
completion after two interlink roundtrips instead of full three.
2015-09-14 15:21:23 +00:00
Alexander Motin
3ed019a919 Implement data/status aggregation for camsim backend.
This is almost pointless for primary HA node, but useful for secondary,
where between fe_datamove and fe_done calls goes another link roundtrip.
2015-09-14 15:15:32 +00:00
Alexander Motin
7b6660c9aa Remove CTL_PRIV_LBA_LEN from HA messages.
Previously it was used for statistics, but now just a 16 extra bytes.
2015-09-14 09:56:01 +00:00
Alexander Motin
ceff31dc0c Implement QUERY TASK, QUERY TASK SET and QUERY ASYNC EVENT.
Now we support most of SAM-5 task management.
2015-09-14 08:01:05 +00:00
Alexander Motin
bb978da493 Map CLEAR TASK SET and I_T NEXUS RESET for iSCSI.
The last should not be called without iSCSIProtocolLevel negotiation.
2015-09-13 20:40:00 +00:00
Alexander Motin
4cbc0bc5ae Implement iSCSI TARGET COLD RESET task management function.
Implement it as CTL_TASK_TARGET_RESET plus termination of all sessions.
2015-09-13 20:22:18 +00:00
Alexander Motin
6187d4722a Improve read-only support. 2015-09-13 16:49:41 +00:00
Alexander Motin
6b56dd0cfa Reannounce port to HA peer if LUN map changed after online. 2015-09-13 15:31:55 +00:00
Alexander Motin
cfa0987386 Report INQUIRY DATA HAS CHANGED for related LUNs on port on-/offline. 2015-09-13 14:52:31 +00:00
Alexander Motin
eebde34629 Report CTL_UA_LUN_CHANGE on LUN map change. 2015-09-13 14:04:42 +00:00
Alexander Motin
844c2a3a94 Make TARGET RESET respect LUN mapping. 2015-09-13 13:00:20 +00:00
Alexander Motin
84e2fad15a Check for obsolete NUL bin in CSCD descriptor. 2015-09-12 20:45:09 +00:00
Alexander Motin
051481a24e Add HA support for CTL_TASK_I_T_NEXUS_RESET. 2015-09-12 20:06:22 +00:00
Alexander Motin
0a4c638188 Fix false CTL_UA_RES_RELEASE on secondary HA node. 2015-09-12 18:29:05 +00:00
Alexander Motin
119c9aca64 Decode WRITE ATOMIC(16) command. 2015-09-12 17:53:49 +00:00
Alexander Motin
a65a997fd9 Improve XCOPY error reporting. 2015-09-12 16:30:01 +00:00
Alexander Motin
238b6b7c75 Report that we have no limit on POPULATE TOKEN segment size. 2015-09-12 14:20:11 +00:00
Alexander Motin
eed0d596cf Correct RTPG bitmask. 2015-09-12 13:53:41 +00:00
Alexander Motin
e184d2576a Some HA polishing. 2015-09-12 12:46:04 +00:00
Alexander Motin
d36c617616 CTL documentation update, mostly for HA. 2015-09-12 10:23:23 +00:00
Alexander Motin
572846e089 Make ctld restart on primary HA node less invasive for secondary. 2015-09-11 14:47:35 +00:00
Alexander Motin
ee4ad294d2 Close races between device close and request processing.
All requests arriving for processing after OFFLINE flag set are rejected
with BUSY status.  Races around OFFLINE flag setting are closed by calling
taskqueue_drain_all().
2015-09-11 14:33:05 +00:00
Alexander Motin
3236151ea8 Reference/release devices on every I/O, rather on open/close.
While this may be slower, it allows device destruction to complete,
rather then block waiting for indefinitely long time.
2015-09-11 12:50:52 +00:00
Alexander Motin
7ac58230ea Reimplement CTL High Availability.
CTL HA functionality was originally implemented by Copan many years ago,
but large part of the sources was never published.  This change includes
clean room implementation of the missing code and fixes for many bugs.

This code supports dual-node HA with ALUA in four modes:
 - Active/Unavailable without interlink between nodes;
 - Active/Standby with second node handling only basic LUN discovery and
reservation, synchronizing with the first node through the interlink;
 - Active/Active with both nodes processing commands and accessing the
backing storage, synchronizing with the first node through the interlink;
 - Active/Active with second node working as proxy, transfering all
commands to the first node for execution through the interlink.

Unlike original Copan's implementation, depending on specific hardware,
this code uses simple custom TCP-based protocol for interlink.  It has
no authentication, so it should never be enabled on public interfaces.

The code may still need some polishing, but generally it is functional.

Relnotes:	yes
Sponsored by:	iXsystems, Inc.
2015-09-10 12:40:31 +00:00
Alexander Motin
fb606ebabc Remove unused target and initiator IDs. 2015-09-10 10:46:21 +00:00
Alexander Motin
3f326305fc Disable CTL_IO_DELAY feature.
It is too developer-oriented to be enabled by default.
2015-09-10 09:27:22 +00:00
Alexander Motin
a3977bea20 Allow LUN options modification via CTL_LUNREQ_MODIFY.
Not all changes take effect, but that is a different question.
2015-09-06 11:23:01 +00:00
Alexander Motin
0bcd4ab6ba Move setting of media parameters inside open routines.
This is preparation for possibility to open/close media several times
per LUN life cycle.  While there, rename variables to reduce confusion.
As additional bonus this allows to open read-only media, such as ZFS
snapshots.
2015-09-06 09:54:56 +00:00
Alexander Motin
bd236ba5c0 Remove some dead code. 2015-09-04 09:19:01 +00:00
Alexander Motin
420abfcecd Small UA cleanup. 2015-09-03 12:56:57 +00:00
Alexander Motin
8be6a2df51 Fix copy-paste bug introduced in r275458.
MFC after:	3 days
2015-09-03 12:15:14 +00:00
Alexander Motin
1b7849d0f2 Make most of port methods optional and remove bunch of dummies. 2015-09-01 16:28:06 +00:00
Alexander Motin
7278725be5 Remove 600 bytes of port_priv from struct ctl_io_hdr.
This field used only for camtgt frontend, and once it any way preallocates
all requests, let it preallocate this memory too, not bothering core code.
2015-08-29 15:33:31 +00:00
Alexander Motin
9202485814 Attach pass driver to LUNs is OFFLINE state.
Previously such LUNs were silently ignored.  But while they indeed unable
to process most of SCSI commands, some, like RTPG, they still can.

MFC after:	1 month
2015-08-29 11:21:20 +00:00
Alexander Motin
f6295033c1 Fix type bug introduced at r286811. 2015-08-27 21:16:24 +00:00
Alexander Motin
4beec13537 Remove some code duplication by using biofinish().
Submitted by:	imp
MFC after:	1 week
2015-08-22 15:58:35 +00:00
Kenneth D. Merry
0e358df062 Revamp camcontrol(8) fwdownload support and add the opcodes subcommand.
The significant changes and bugs fixed here are:

1. Fixed a bug in the progress display code:

   When the user's filename is too big, or his terminal width is too
   small, the progress code could wind up using a negative number for
   the length of the "stars" that it uses to indicate progress.

   This negative value was assigned to an unsigned variable, resulting
   in a very large positive value.

   The result is that we wound up writing garbage from memory to the
   user's terminal.

   With an 80 column terminal, a file name length of more than 35
   characters would generate this problem.

   To address this, we now set a minimum progress bar length, and
   truncate the user's file name as needed.

   This has been tested with large filenames and small terminals, and
   at least produces reasonable results.  If the terminal is too
   narrow, the progress display takes up an additional line with each
   update, but this is more user friendly than writing garbage to the
   tty.

2. SATA drives connected via a SATA controller didn't have SCSI Inquiry
   data populated in struct cam_device.  This meant that the code in
   fw_get_vendor() in fwdownload.c would try to match a zero-length
   vendor ID, and so return the first entry in the vendor table.  (Which
   used to be HITACHI.)  Fixed by grabbing identify data, passing the
   identify buffer into fw_get_vendor(), and matching against the model
   name.

3. SATA drives connected via a SAS controller do have Inquiry data
   populated.  The table included a couple of entries -- "ATA ST" and
   "ATA HDS", intended to handle Seagate and Hitachi SATA drives attached
   via a SAS controller.  SCSI to ATA translation layers use a vendor
   ID of "ATA" (which is standard), and then the model name from the ATA
   identify data as the SCSI product name when they are returning data on
   SATA disks.  The cam_strmatch code will match the first part of the
   string (because the length it is given is the length of the vendor,
   "ATA"), and return 0 (i.e. a match).  So all SATA drives attached to
   a SAS controller would be programmed using the Seagate method
   (WRITE BUFFER mode 7) of SCSI firmware downloading.

4. Issue #2 above covered up a bug in fw_download_img() -- if the
   maximum packet size in the vendor table was 0, it tried to default
   to a packet size of 32K.  But then it didn't actually succeed in
   doing that, because it set the packet size to the value that was
   in the vendor table (0).  Now that we actually have ATA attached
   drives fall use the VENDOR_ATA case, we need a reasonable default
   packet size.  So this is fixed to properly set the default packet size.

5. Add support for downloading firmware to IBM LTO drives, and add a
   firmware file validation method to make sure that the firmware
   file matches the drive type.  IBM tape drives include a Load ID and
   RU name in their vendor-specific VPD page 0x3.  Those should match
   the IDs in the header of the firmware file to insure that the
   proper firmware file is loaded.

6. This also adds a new -q option to the camcontrol fwdownload
   subcommand to suppress informational output.  When -q is used in
   combination with -y, the firmware upgrade will happen without
   prompting and without output except if an error condition occurs.

7. Re-add support for printing out SCSI inquiry information when
   asking the user to confirm that they want to download firmware, and
   add printing of ATA Identify data if it is a SATA disk.  This was
   removed in r237281 when support for flashing ATA disks was added.

8. Add a new camcontrol(8) "opcodes" subcommand, and use the
   underlying code to get recommended timeout values for drive
   firmware downloads.

   Many SCSI devices support the REPORT SUPPORTED OPERATION CODES
   command, and some support the optional timeout descriptor that
   specifies nominal and recommended timeouts for the commands
   supported by the device.

   The new camcontrol opcodes subcommand allows displaying all
   opcodes supported by a drive, information about which fields
   in a SCSI CDB are actually used by a given SCSI device, and the
   nominal and recommended timeout values for each command.

   Since firmware downloads can take a long time in some devices, and
   the time varies greatly between different types of devices, take
   advantage of the infrastructure used by the camcontrol opcodes
   subcommand to determine the best timeout to use for the WRITE
   BUFFER command in SCSI device firmware downloads.

   If the device recommends a timeout, it is likely to be more
   accurate than the default 50 second timeout used by the firmware
   download code.  If the user specifies a timeout, it will override
   the default or device recommended timeout.  If the device doesn't
   support timeout descriptors, we fall back to the default.

9. Instead of downloading firmware to SATA drives behind a SAS controller
   using WRITE BUFFER, use the SCSI ATA PASS-THROUGH command to compose
   an ATA DOWNLOAD MICROCODE command and it to the drive.  The previous
   version of this code attempted to send a SCSI WRITE BUFFER command to
   SATA drives behind a SAS controller.  Although that is part of the
   SAT-3 spec, it doesn't work with the parameters used with LSI
   controllers at least.

10.Add a new mechanism for making common ATA passthrough and
   ATA-behind-SCSI passthrough commands.

   The existing camcontrol(8) ATA command mechanism checks the device
   type on every command executed.  That works fine for individual
   commands, but is cumbersome for things like a firmware download
   that send a number of commands.

   The fwdownload code detects the device type up front, and then
   sends the appropriate commands.

11.In simulation mode (-s), if the user specifies the -v flag, print out
   the SCSI CDB or ATA registers that would be sent to the drive.  This will
   aid in debugging any firmware download issues.

sbin/camcontrol/fwdownload.c:
	Add a device type to the fw_vendor structure, so that we can
	specify different download methods for different devices from the
	same vendor.  In this case, IBM hard drives (from when they
	still made hard drives) and tape drives.

	Add a tur_status field to the fw_vendor structure so that we can
	specify whether the drive to be upgraded should be ready, not
	ready, or whether it doesn't matter.  Add the corresponding
	capability in fw_download_img().

	Add comments describing each of the vendor table fields.

	Add HGST and SmrtStor to the supported SCSI vendors list.

	In fw_get_vendor(), look at ATA identify data if we have a SATA
	device to try to identify what the drive vendor is.

	Add IBM firmware file validation.  This gets VPD page 0x3, and
	compares the Load ID and RU name in the page to the values
	included in the header.  The validation code will refuse to load
	a firmware file if the values don't match.  This does allow the
	user to attempt a downgrade; whether or not it succeeds will
	likely depend on the drive settings.

	Add a -q option, and disable all informative output
	(progress bars, etc.) when this is enabled.

	Re-add the inquiry in the confirmation dialog so the user has
	a better idea of which device he is talking to.  Add support for
	displaying ATA identify data.

	Don't automatically disable confirmation in simulation (-s) mode.
	This allows the user to see the inquiry or identify data in the
	dialog, and see exactly what they would see when the command
	actually runs.  Also, in simulation mode, if the user specifies
	the -v flag, print out the SCSI CDB or ATA registers that would
	be sent to the drive.  This will aid in debugging any firmware
	download issues.

	Add a timeout field and timeout type to the firmware download
	vendor table.  This allows specifying a default timeout and allows
	specifying whether we should attempt to probe for a recommended
	timeout from the drive.

	Add a new fuction, fw_get_timeout(), that will determine
	which timeout to use for the WRITE BUFFER command.  If the
	user specifies a timeout, we always use that.  Otherwise,
	we will use the drive recommended timeout, if available,
	and fall back to the default when a drive recommended
	timeout isn't available.

	When we prompt the user, tell him what timeout we're going
	to use, and the source of the timeout.

	Revamp the way SATA devices are handled.

	In fwdownload(), use the new get_device_type() function to
	determine what kind of device we're talking to.

	Allow firmware downloads to any SATA device, but restrict
	SCSI downloads to known devices.  (The latter is not a
	change in behavior.)

	Break out the "ready" check from fw_download_img() into a
	new subfunction, fw_check_device_ready().  This sends the
	appropriate command to the device in question -- a TEST
	UNIT READY or an IDENTIFY.  The IDENTIFY for SATA devices
 	a SAT layer is done using the SCSI ATA PASS-THROUGH
	command.

	Use the new build_ata_cmd() function to build either a SCSI or
	ATA I/O CCB to issue the DOWNLOAD MICROCODE command to SATA
	devices.  build_ata_cmd() figures looks at the devtype argument
	and fills in the correct CCB type and CDB or ATA registers.

	Revamp the vendor table to remove the previous
	vendor-specific ATA entries and use a generic ATA vendor
	placeholder.  We currently use the same method for all ATA
	drives, although we may have to add vendor-specific
	behavior once we test this with more drives.

sbin/camcontrol/progress.c:
	In progress_draw(), make barlength a signed value so that
	we can easily detect a negative value.

	If barlength (the length of the progress bar) would wind up
	negative due to a small TTY width or a large filename,
	set the bar length to the new minimum (10 stars) and
	truncate the user's filename.  We will truncate it down to
	0 characters if necessary.

	Calculate a new prefix_len variable (user's filename length)
	and use it as the precision when printing the filename.

sbin/camcontrol/camcontrol.c:
	Implement a new camcontrol(8) subcommand, "opcodes".  The
	opcodes subcommand allows displaying the entire list of
	SCSI commands supported by a device, or details on an
	individual command.  In either case, it can display
	nominal and recommended timeout values.

	Add the scsiopcodes() function, which calls the new
	scsigetopcodes() function to fetch opcode data from a
	drive.

	Add two new functions, scsiprintoneopcode() and
	scsiprintopcodes(), which print information about one
	opcode or all opcodes, respectively.

	Remove the get_disk_type() function.  It is no longer used.

	Add a new function, dev_has_vpd_page(), that fetches the
	supported INQUIRY VPD list from a device and tells the
	caller whether the requested VPD page is available.

	Add a new function, get_device_type(), that returns a more
	precise device type than the old get_disk_type() function.
	The get_disk_type() function only distinguished between
	SCSI and ATA devices, and SATA devices behind a SCSI to ATA
	translation layer were considered to be "SCSI".

	get_device_type() offers a third type, CC_DT_ATA_BEHIND_SCSI.
	We need to know this to know whether to attempt to send ATA
	passthrough commands.  If the device has the ATA
	Information VPD page (0x89), then it is an ATA device
	behind a SCSI to ATA translation layer.

	Remove the type argument from the fwdownload() subcommand.

	Add a new function, build_ata_cmd(), that will take one set
	of common arguments and build either a SCSI or ATA I/O CCB,
	depending on the device type passed in.

sbin/camcontrol/camcontrol.h:
	Add a prototype for scsigetopcodes().

	Add a new enumeration, camcontrol_devtype.

	Add prototypes for dev_has_vpd_page(), get_device_type()
	and build_ata_cmd().

	Remove the type argument from the fwdownload() subcommand.

sbin/camcontrol/camcontrol.8
	Explain that the fwdownload subcommand will use the drive
	recommended timeout if available, and that the user can
	override the timeout.

	Document the new opcodes subcommand.

	Explain that we will attempt to download firmware to any
	SATA device.

	Document supported SCSI vendors, and models tested if known.

	Explain the commands used to download firmware for the
	three different drive and controller combinations.

	Document that the -v flag in simulation mode for the fwdownload
	subcommand will print out the SCSI CDBs or ATA registers that would
	be used.

sys/cam/scsi/scsi_all.h:
	Add new bit definitions for the one opcode descriptor for
	the REPORT SUPPORTED OPCODES command.

	Add a function prototype for scsi_report_supported_opcodes().

sys/cam/scsi/scsi_all.c:
	Add a new CDB building function, scsi_report_supported_opcodes().

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-08-20 16:07:51 +00:00
Alexander Motin
a15bbf1508 Polish sizes processing. 2015-08-15 18:22:16 +00:00
Alexander Motin
67ceb24bca Move "ioctl" CAM frontend into separate file.
It has nothing to share with too huge ctl.c other then device descriptor,
but even that may be counted as design error that may be fixed later.
At some point we may even want to have several ioctl ports.
2015-08-15 15:42:21 +00:00
Alexander Motin
2f444d157b Drop "internal" CTL frontend.
Its idea was to be a simple initiator and execute several commands from
kernel level, but FreeBSD never had consumer for that functionality,
while its implementation polluted many unrelated places..
2015-08-15 13:34:38 +00:00
Alexander Motin
4ddb055594 Remove verbose CTL messages.
Reporting SCSI errors to console is often useless, pollutes logs and may
affect performance.  For debugging there is kern.cam.ctl.debug sysctl

MFC after:	1 week
2015-08-09 09:54:29 +00:00
Alexander Motin
bac1eac93c Don't panic if disk lost TRIM support due to switching to PIO mode.
MFC after:	1 week
2015-08-08 11:22:45 +00:00
Alexander Motin
3301406331 Add more ifdefs to fix build with GCC after r286406. 2015-08-07 14:12:51 +00:00
Marcelo Araujo
8b3ae99560 Wrap some unused functions with notyet, it is necessary to be able to
build the modules/ctl directly.
Remove a dead MALLOC_DEFINE.

Differential Revision:	D3329
Reviewed by:		mav
Sponsored by:		gandi.net
2015-08-07 08:30:43 +00:00
Alexander Motin
7d0d4342e3 Pass SYNCHRONIZE CACHE command parameters to backends.
At this point IMMED flag is translated to MNT_NOWAIT flag of VOP_FSYNC(),
hoping that file system implements that (ZFS seems doesn't).

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2015-08-05 22:24:49 +00:00
Alexander Motin
f2a20b166a Relax serialization of SYNCHRONIZE CACHE commands.
Before this change SYNCHRONIZE CACHE commands were executed exclusively,
as if they had ORDERED tag.  But looking through SCSI specs I've found
no any reason to be so strict.  For reads this ordering seems pointless.
For writes it looks less obvious, so I left ordering against preceeding
write commands, while following ones are no longer required to wait.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2015-08-05 21:58:32 +00:00
Alexander Motin
73942c5ce0 Issue all reads of single XCOPY segment simultaneously.
During vMotion and Clone VMware by default runs multiple sequential 4MB
XCOPY requests same time.  If CTL issues reads sequentially in 1MB chunks
for each XCOPY command, reads from different commands are not detected
as sequential by serseq option code and allowed to execute simultaneously.
Such read pattern confused ZFS prefetcher, causing suboptimal disk access.
Issuing all reads same time make serseq code work properly, serializing
reads both within each XCOPY command and between them.

My tests with ZFS pool of 14 disks in RAID10 shows prefetcher efficiency
improved from 37% to 99.7%, copying speed improved by 10-60%, average
read latency reduced twice on HDD layer and by five times on zvol layer.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2015-08-05 13:46:15 +00:00
Jeff Roberson
fade8dd714 Refactor unmapped buffer address handling.
- Use pointer assignment rather than a combination of pointers and
   flags to switch buffers between unmapped and mapped.  This eliminates
   multiple flags and generally simplifies the logic.
 - Eliminate b_saveaddr since it is only used with pager bufs which have
   their b_data re-initialized on each allocation.
 - Gather up some convenience routines in the buffer cache for
   manipulating buf space and buf malloc space.
 - Add an inline, buf_mapped(), to standardize checks around unmapped
   buffers.

In collaboration with: mlaier
Reviewed by:	kib
Tested by:	pho (many small revisions ago)
Sponsored by:	EMC / Isilon Storage Division
2015-07-23 19:13:41 +00:00
Mateusz Guzik
8a08cec166 Create a dedicated function for ensuring that cdir and rdir are populated.
Previously several places were doing it on its own, partially
incorrectly (e.g. without the filedesc locked) or even actively harmful
by populating jdir or assigning rootvnode without vrefing it.

Reviewed by:	kib
2015-07-11 16:22:48 +00:00
Alexander Motin
d1f4058735 Make first step toward supporting target and initiator roles same time.
To avoid conflicts between target and initiator devices in CAM, make
CTL use target ID reported by HBA as its initiator_id in XPT_PATH_INQ.
That target ID is known to never be used for initiator role, so it won't
conflict.  For Fibre Channel and FireWire HBAs this specific ID choice
is irrelevant since all target IDs there are virtual. Same time for SPI
HBAs it seems could be even requirement to use same target ID for both
initiator and target roles.

While there are some more things to polish in isp(4) driver, first tests
of using both roles same time on the same port appeared successfull:

# camcontrol devlist -v
scbus0 on isp0 bus 0:
<FREEBSD CTLDISK 0001>             at scbus0 target 1 lun 0 (da20,pass21)
<>                                 at scbus0 target 256 lun 0 (ctl0)
<>                                 at scbus0 target -1 lun ffffffff (ctl1)
2015-07-05 03:38:58 +00:00
Alexander Motin
b9b4269c1d Fix couple panics on forced unmount of backing file.
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2015-07-02 12:53:22 +00:00
Alexander Motin
1ec1012dd2 Fix REPORT LUNS command output for the case when same LUN mapped to same
port several times.  While it is unusual configuration, it is not illegal.

MFC after:	1 week
2015-06-20 13:02:57 +00:00
Alexander Motin
7834ea8891 Bring per-port LUN enable/disable code up to date:
- remove last remnants of never implemented multiple targets support;
 - implement missing support for LUN mapping in this area.

Due to existing locking constraints LUN mapping code is practically
unlocked at this point.  Hopefully it is not racy enough to live until
somebody get idea how to call sleeping fronend methods under lock also
taken by the same frontend in non-sleepable context. :(
2015-06-20 12:43:54 +00:00
Alexander Motin
2d8b28765c Introduce separate lock for tokens to reduce ctl_lock scope. 2015-06-20 11:20:25 +00:00
Alexander Motin
754e493530 Remove some dead and duplicate LUN enabling code. 2015-06-20 07:33:06 +00:00
Alexander Motin
b9807a43b0 Remove device queue freeze handling and replace it with dummy.
At this point CTL has no known use case for device queue freezes.
Same time existing (considered to be broken) code was found to cause
modify-after-free issues.

Discussed with:	ken
MFC after:	1 week
2015-06-20 03:40:19 +00:00
Alexander Motin
521db0ac8d Change ATIO/INOT counting to prevent periph destruction while requests
are still running inside CTL.

MFC after:	1 week
2015-06-18 09:39:51 +00:00
Kenneth D. Merry
5672fac935 Add support for reading MAM attributes to camcontrol(8) and libcam(3).
MAM is Medium Auxiliary Memory and is most commonly found as flash
chips on tapes.

This includes support for reading attributes and decoding most
known attributes, but does not yet include support for writing
attributes or reporting attributes in XML format.

libsbuf/Makefile:
	Add subr_prf.c for the new sbuf_hexdump() function.  This
	function is essentially the same function.

libsbuf/Symbol.map:
	Add a new shared library minor version, and include the
	sbuf_hexdump() function.

libsbuf/Version.def:
	Add version 1.4 of the libsbuf library.

libutil/hexdump.3:
	Document sbuf_hexdump() alongside hexdump(3), since it is
	essentially the same function.

camcontrol/Makefile:
	Add attrib.c.

camcontrol/attrib.c:
	Implementation of READ ATTRIBUTE support for camcontrol(8).

camcontrol/camcontrol.8:
	Document the new 'camcontrol attrib' subcommand.

camcontrol/camcontrol.c:
	Add the new 'camcontrol attrib' subcommand.

camcontrol/camcontrol.h:
	Add a function prototype for scsiattrib().

share/man/man9/sbuf.9:
	Document the existence of sbuf_hexdump() and point users to
	the hexdump(3) man page for more details.

sys/cam/scsi/scsi_all.c:
	Add a table of known attributes, text descriptions and
	handler functions.

	Add a new scsi_attrib_sbuf() function along with a number
	of other related functions that help decode attributes.

	scsi_attrib_ascii_sbuf() decodes ASCII format attributes.

	scsi_attrib_int_sbuf() decodes binary format attributes, and
	will pass them off to scsi_attrib_hexdump_sbuf() if they're
	bigger than 8 bytes.

	scsi_attrib_vendser_sbuf() decodes the vendor and drive
	serial number attribute.

	scsi_attrib_volcoh_sbuf() decodes the Volume Coherency
	Information attribute that LTFS writes out.

sys/cam/scsi/scsi_all.h:
	Add a number of attribute-related structure definitions and
	other defines.

	Add function prototypes for all of the functions added in
	scsi_all.c.

sys/kern/subr_prf.c:
	Add a new function, sbuf_hexdump().  This is the same as
	the existing hexdump(9) function, except that it puts the
	result in an sbuf.

	This also changes subr_prf.c so that it can be compiled in
	userland for includsion in libsbuf.

	We should work to change this so that the kernel hexdump
	implementation is a wrapper around sbuf_hexdump() with a
	statically allocated sbuf with a drain.  That will require
	a drain function that goes to the kernel printf() buffer
	that can take a non-NUL terminated string as input.
	That is because an sbuf isn't NUL-terminated until it is
	finished, and we don't want to finish it while we're still
	using it.

	We should also work to consolidate the userland hexdump and
	kernel hexdump implemenatations, which are currently
	separate.  This would also mean making applications that
	currently link in libutil link in libsbuf.

sys/sys/sbuf.h:
	Add the prototype for sbuf_hexdump(), and add another copy
	of the hexdump flag values if they aren't already defined.

	Ideally the flags should be defined in one place but the
	implemenation makes it difficult to do properly.  (See
	above.)

Sponsored by:	Spectra Logic Corporation
MFC after:	1 week
2015-06-09 21:39:38 +00:00
Alexander Motin
6fbc43fdfb Fix wrong function name in error message.
MFC after:	3 days
2015-06-05 20:30:36 +00:00
Alexander Motin
7607afb12f Allow setting only WWNN or only WWPN.
MFC after:	2 weeks
2015-06-05 06:46:11 +00:00
Alexander Motin
0631de4a79 Handle EDQUOT backend storage errors same as ENOSPC.
MFC after:	1 week
2015-05-06 19:47:31 +00:00
Scott Long
16be867406 Revert r282227. It is clearly incorrect as it frees an object that is still
referenced.  I think that there does exist an unlikely edge case for a
memory leak, but only if a driver is incorrectly written and specifies no
valid range of targets to scan.  That can be fixed in a follow-up commit.

Obtained from:	Netflix, Inc.
2015-04-29 17:18:41 +00:00
Pedro F. Giffuni
bfce3bb269 Fix memory leak in scsi_scan_bus()
CID:	1007770
PR:	199671
2015-04-29 15:46:57 +00:00
Xin LI
90f851d22f Extend DA_Q_NO_RC16 to MXUB3* devices.
PR:		kern/198647
MFC after:	2 weeks
2015-04-21 22:55:52 +00:00
Pedro F. Giffuni
8188e2e04e scsi_parse_transportid_rdma(): fix mismatch in memoty access size.
Independently found by Coverity and gcc49.

CID:		1230006
Reviewed by:	ken
MFC after:	5 days
2015-04-20 21:44:55 +00:00
Alan Somers
738fd16625 Initialize async_arg_ptr in xpt_async when called with async_code
AC_ADVINFO_CHANGED.

Without this change, newly inserted hard disks won't always have their
physical path device nodes created.  The problem reproduces most readily
when attaching a large number of disks at once.

Differential Revision:	https://reviews.freebsd.org/D2290
Reviewed by:	mav, imp
MFC after:	2 weeks
Sponsored by:	Spectra Logic
2015-04-14 16:33:33 +00:00
Eitan Adler
9073a96a85 Add some additional quirks for various Western Digital Caviar MHDDs
Submitted by:	Jeremy Chadwick
PR:		188685
MFC After:	1 month
2015-03-30 09:05:20 +00:00
Hans Petter Selasky
a19f8579a6 Add DA_Q_NO_RC16 quirk for USB mass storage device.
PR:		198647
MFC after:	1 week
2015-03-25 13:28:13 +00:00
Kenneth D. Merry
873da79689 Remove some #if 0'ed code that apparently confuses cscope.
Requested by:	Peter Xu <xzpeter@gmail.com>
MFC after:	3 days
2015-03-24 21:42:28 +00:00
Benno Rice
d3995fdd6a Be consistent with M_ZERO when allocating ccbs.
There are four places, all in cam_xpt.c, where ccbs are malloc'ed. Two of
these use M_ZERO, two don't. The two that don't meant that allocated ccbs
had trash in them making it hard to debug errors where they showed up. Due
to this, use M_ZERO all the time when allocating ccbs.

Submitted by:	Scott Ferris <scott.ferris@isilon.com>
Sponsored by:	EMC/Isilon Storage Division
Reviewed by:	scottl, imp
Differential:	https://reviews.freebsd.org/D2120
2015-03-23 18:45:37 +00:00
Kenneth D. Merry
74a177ac50 Fix a couple of problems in the sa(4) media type reports.
The only drives I have discovered so far that support medium type
reports are newer HP LTO (LTO-5 and LTO-6) drives.  IBM drives
only support the density reports.

sys/cam/scsi/scsi_sa.h:
	The number of possible density codes in the medium type
	report is 9, not 8.  This caused problems parsing all of
	the medium type report after this point in the structure.

usr.bin/mt/mt.c:
	Run the density codes returned in the medium type report
	through denstostring(), just like the primary and secondary
	density codes in the density report.  This will print the
	density code in hex, and give a text description if it
	is available.

Thanks to Rudolf Cejka for doing extensive testing with HP LTO drives
and Bacula and discovering these problems.

Tested by:	Rudolf Cejka <cejkar at fit.vutbr.cz>
Sponsored by:	Spectra Logic
MFC after:	4 days
2015-03-18 20:52:34 +00:00
Alexander Motin
4f42bb1021 Improve ATA and SCSI versions printing.
There is no "SCSI-6" and "ATA-9", but there is "SPC-4" and "ACS-2".

MFC after:	2 weeks
2015-03-17 13:21:49 +00:00
Hans Petter Selasky
d7ea38a553 Add DA_Q_NO_RC16 quirk for USB mass storage device.
PR:		194062
MFC after:	1 week
2015-03-07 17:18:06 +00:00
Edward Tomasz Napierala
5719711f8b Make periphdriver_register() take XPT lock when modifying the periph_drivers
array.

This fixes a panic that sometimes occured when kldloading ctl.ko.

Reviewed by:	mav@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2015-03-02 22:48:11 +00:00
Kenneth D. Merry
54eb0be231 Change the sa(4) driver to check for long position support on
SCSI-2 devices.

Some older tape devices claim to be SCSI-2, but actually do support
long position information.  (Long position information includes
the current file mark.)  For example, the COMPAQ SuperDLT1.

So we now only disable the check on SCSI-1 and older devices.

sys/cam/scsi/scsi_sa.c:
	In saregister(), only disable fetching long position
	information on SCSI-1 and older drives.  Update the
	comment to explain why.

Confirmed by:	dvl
Sponsored by:	Spectra Logic
MFC after:	3 weeks
2015-03-02 18:09:49 +00:00
Kenneth D. Merry
fd8be18f3f Fix I/O size calculation for pass(4) driver requests and add latency
tracking.

It is important to subtract the residual from the requested
transfer size to see how much data was actually transferred.  With
tape drives in particular, it is common to request more data than is
returned.

Also, add I/O latency tracking for CAM requests issued by
cam_periph_runccb().

If the caller supplies a struct devstat, and the I/O is a SCSI or
ATA I/O, we will track the elapsed time to provide I/O latency
statistics for the request.

sys/cam/scsi/cam_periph.c:
	In cam_periph_runccb(), subtract the residual when reporting I/O
	totals to devstat(9) for SCSI and ATA passthrough requests.

	In cam_periph_runccb(), grab the I/O start time and supply
	the start time to devstat_end_transaction() so that it can
	calculate the elapsed I/O time.

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-02-27 21:35:36 +00:00
Kenneth D. Merry
62d67aa923 Fix printf format warnings on sparc64 and mips.
Sponsored by:	Spectra Logic
MFC after:	1 month
2015-02-24 05:43:16 +00:00