Commit Graph

2190 Commits

Author SHA1 Message Date
Edward Tomasz Napierala
c6646787ab Add SCSI descriptors for USB Mass Storage.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2017-01-22 15:32:17 +00:00
Alexander Motin
0c629e2884 Add initial support for CTL module unloading.
It is only a first step and not perfect, but better then nothing.
The main blocker is CAM target frontend, that can not be unloaded,
since CAM does not have mechanism to unregister periph driver now.

MFC after:	2 weeks
2017-01-21 19:38:26 +00:00
Alan Somers
c9bd03d87f Fix "camcontrol timestamp -s" with LTO-7 drives
The length of the scsi_set_timestamp_parameters struct was incorrect.  LTO-5
drives don't care, but LTO-7 drives do.

Reviewed by:	Sam Klopsch
MFC after:	2 weeks
Sponsored by:	Spectra Logic Corp
2017-01-20 17:54:24 +00:00
Alexander Motin
640603fbf6 Remove writing 'residual' field of struct ctl_scsiio.
This field has no practical use and never readed.  Initiators already
receive respective residual size from frontends.  Removed field had
different semantics, which looks useless, and was never passed through
by any frontend.

While there, fix kern_data_resid field support in case of HA, missed in
r312291.

MFC after:	13 days
2017-01-17 18:32:47 +00:00
Sean Bruno
23030355c6 Add 4k quirk for Micron 5100 and Intel S3610 SSDs
Submitted by:	Jason Wolfe <j@nitrology.com>
MFH:		1 week
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D9209
2017-01-17 14:52:48 +00:00
Alexander Motin
a8d32bff70 Improve error message on duplicate iSCSI port.
MFC after:	2 weeks
2017-01-17 14:13:14 +00:00
Alexander Motin
eb6ac6f9db Make CTL frontends report kern_data_resid for under-/overruns.
It seems like kern_data_resid was never really implemented.  This change
finally does it.  Now frontends update this field while transferring data,
while CTL/backends getting it can more flexibly handle the result.
At this point behavior should not change significantly, still reporting
errors on write overrun, but that may be changed later, if we decide so.

CAM target frontend still does not properly handle overruns due to CAM API
limitations.  We may need to add some fields to struct ccb_accept_tio to
pass information about initiator requested transfer size(s).

MFC after:	2 weeks
2017-01-16 16:19:55 +00:00
Conrad Meyer
1d64db52f3 Fix a variety of cosmetic typos and misspellings
No functional change.

PR:		216096, 216097, 216098, 216101, 216102, 216106, 216109, 216110
Reported by:	Bulat <bltsrc at mail.ru>
Sponsored by:	Dell EMC Isilon
2017-01-15 18:00:45 +00:00
Conrad Meyer
db4fcadf52 "Buses" is the preferred plural of "bus"
Replace archaic "busses" with modern form "buses."

Intentionally excluded:
* Old/random drivers I didn't recognize
  * Old hardware in general
* Use of "busses" in code as identifiers

No functional change.

http://grammarist.com/spelling/buses-busses/

PR:		216099
Reported by:	bltsrc at mail.ru
Sponsored by:	Dell EMC Isilon
2017-01-15 17:54:01 +00:00
Alexander Motin
953f538e0e Add under-/overrun support to IOCTL and CAM SIM frontends.
MFC after:	1 week
2017-01-15 13:57:42 +00:00
Alexander Motin
f319059631 When in kernel, map ctl_scsi_zero_io() to ctl_zero_io().
MFC after:	1 week
2017-01-15 13:51:44 +00:00
Alexander Motin
4902e14dc8 Improve CAM_CDB_POINTER support.
MFC after:	2 weeks
2017-01-13 08:31:55 +00:00
Alexander Motin
17160457b4 Report random flash storage as non-rotating to GEOM_DISK.
While doing it, introduce respective constants in geom_disk.h.

MFC after:	1 week
2017-01-12 08:53:10 +00:00
Alexander Motin
4982dca18d Do not wait for HA thread shutdown if scheduler is stopped.
This wait loop made system hang on panic instead of reboot.

MFC after:	1 week
2017-01-10 20:16:16 +00:00
Alexander Motin
7c07ea9aa2 Fix malloc(M_WAITOK) under mutex, introduced at r311787.
MFC after:	13 days
2017-01-10 10:33:36 +00:00
Alexander Motin
bb8f9017b3 Rewrite CTL statistics in more simple and scalable way.
Instead of collecting statistics for each combination of ports and logical
units, that consumed ~45KB per LU with present number of ports, collect
separate statistics for every port and every logical unit separately, that
consume only 176 bytes per each single LU/port.  This reduces struct
ctl_lun size down to just 6KB.

Also new IOCTL API/ABI does not hardcode number of LUs/ports, and should
allow handling of very large quantities.

MFC after:	2 weeks (probably keeping old API enabled for some time)
2017-01-09 18:18:15 +00:00
Alexander Motin
0e2d6474ca Allocate memory for prevent flags only for removable LUs.
This array takes 64KB of RAM now, that was more then half of struct ctl_lun
size.  If at some point we support more ports, this may need another tune.

MFC after:	2 weeks
2017-01-09 16:21:06 +00:00
Alexander Motin
81241f2828 Make CTL_GETSTATS ioctl return partial data if buffer is small.
MFC after:	2 weeks
2017-01-08 13:26:34 +00:00
Alexander Motin
54644e21e8 Make 'camcontrol modepage' support subpages.
MFC after:	2 weeks
2017-01-07 09:56:12 +00:00
Alexander Motin
f9b66e4c7d Fix bootverbose affecting code logic in r294558.
Reported by:	Jilles Tjoelker <jilles@stack.nl>
2017-01-05 16:30:13 +00:00
Alan Somers
4195c7de24 Always null-terminate ccb_pathinq.(sim_vid|hba_vid|dev_name)
The sim_vid, hba_vid, and dev_name fields of struct ccb_pathinq are
fixed-length strings. AFAICT the only place they're read is in
sbin/camcontrol/camcontrol.c, which assumes they'll be null-terminated.
However, the kernel doesn't null-terminate them. A bunch of copy-pasted code
uses strncpy to write them, and doesn't guarantee null-termination. For at
least 4 drivers (mpr, mps, ciss, and hyperv), the hba_vid field actually
overflows. You can see the result by doing "camcontrol negotiate da0 -v".

This change null-terminates those fields everywhere they're set in the
kernel. It also shortens a few strings to ensure they'll fit within the
16-character field.

PR:		215474
Reported by:	Coverity
CID:		1009997 1010000 1010001 1010002 1010003 1010004 1010005
CID:		1331519 1010006 1215097 1010007 1288967 1010008 1306000
CID:		1211924 1010009 1010010 1010011 1010012 1010013 1010014
CID:		1147190 1010017 1010016 1010018 1216435 1010020 1010021
CID:		1010022 1009666 1018185 1010023 1010025 1010026 1010027
CID:		1010028 1010029 1010030 1010031 1010033 1018186 1018187
CID:		1010035 1010036 1010042 1010041 1010040 1010039
Reviewed by:	imp, sephe, slm
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D9037
Differential Revision:	https://reviews.freebsd.org/D9038
2017-01-04 20:26:42 +00:00
Edward Tomasz Napierala
3a408ce90d Don't release the cfiscsi session refcount too early. It wasn't
observed to fix any actual error, but it's the right thing to do
from the correctness point of view.

Tested by:	Eugene M. Zheganin <emz at norma.perm.ru>
MFC after:	1 month
2017-01-04 12:50:44 +00:00
Alexander Motin
49fe3b378b Fix build with enabled debug after r310778.
MFC after:	2 weeks
2016-12-29 18:08:04 +00:00
Alexander Motin
9cbbfd2f5c Improve use of I/O's private area.
- Since I/Os are allocates from per-port pools, make allocations store
pointer to CTL softc there, and use it where needed instead of global.
 - Created bunch of helper macros to access LUN, port and CTL softc.

MFC after:	 2 weeks
2016-12-29 15:09:34 +00:00
Alexander Motin
e67ac203a6 Allow more efficient use of private area.
There are 16 bytes of space, so we may store two pointers in one.

MFC after:	2 weeks
2016-12-27 23:56:46 +00:00
Alexander Motin
ca734d8a13 Do not update "saved" mode page on every MODE SELECT.
We do not have non-volatile memory to really save those values, so we
neither report nor support this capability.  Also saved mode pages are
not replicated between HA peers now.

MFC after:	2 weeks
2016-12-27 22:38:44 +00:00
Alexander Motin
2e0a1f828e Fix/synchronize field types in struct ctl_modepage_header.
MFC after:	2 weeks
2016-12-27 22:31:06 +00:00
Alexander Motin
2c966e8500 Missed piece of r310640.
MFC after:	2 weeks
2016-12-27 22:28:14 +00:00
Alexander Motin
3722787c8a Add support for revert to defaults (RTD) bit in MODE SELECT.
MFC after:	2 weeks
2016-12-27 21:11:54 +00:00
Alexander Motin
ee5e44e076 Decouple limits on number of LUNs per port and LUs per CTL.
Those two values are not directly related, so make them independent.
This does not change any limits immediately, but makes number of LUNs
per port controllable via tunable/sysctl kern.cam.ctl.lun_map_size.
After this change increasing CTL_MAX_LUNS should be pretty cheap,
and even making it tunable should be easy.

MFC after:	2 weeks
2016-12-27 18:23:16 +00:00
Alexander Motin
92efdd4305 Fix improperly used nexus.targ_lun.
MFC after:	2 weeks
2016-12-26 08:47:35 +00:00
Alexander Motin
a16253002c Some random code cleaning.
- Reduce indentation.
 - Remove extra braces.
 - Add few missing savety checks.

MFC after:	2 weeks
2016-12-25 20:17:15 +00:00
Alexander Motin
4124315924 Remove CTL_MAX_LUNS from places where it is not required.
MFC after:	2 weeks
2016-12-25 13:34:02 +00:00
Alexander Motin
a3dd837892 Improve third-party copy error reporting.
For EXTENDED COPY:
 - improve parameters checking to report some errors before copy start;
 - forward sense data from copy target as descriptor in case of error;
 - report which CSCD reported error in sense key specific information.
For WRITE USING TOKEN:
 - pass through real sense data from copy target instead of reporting
our copy error, since for initiator its a "simple" write, not a copy.

MFC after:	2 weeks
2016-12-25 09:40:44 +00:00
Alexander Motin
4fc0d1d757 Improve length handling when writing sense data.
- Allow maximal sense size limitation via Control Extension mode page.
 - When sense size limited, include descriptors atomically: whole or none.
 - Set new SDAT_OVFL bit if some descriptors don't fit the limit.
 - Report real written sense length instead of static maximal 252 bytes.

MFC after:	2 weeks
2016-12-24 17:42:34 +00:00
Alexander Motin
5bb5027e27 Implement printing forwarded sense data.
MFC after:	2 weeks
2016-12-23 21:56:08 +00:00
Alexander Motin
368cbc2c67 Add place-holders for TAPE STREAM MIRRORING subcommands of XCOPY.
We do not support tapes, so this is only a comments.

MFC after:	2 weeks
2016-12-23 14:54:40 +00:00
Alexander Motin
2be7cd9bdb Add support for REPD bit in RSTMF command.
We have no real timeout values to report there, but its better then error.

MFC after:	2 weeks
2016-12-22 12:18:40 +00:00
Alexander Motin
8202971892 Fix REPORT SUPPORTED OPERATION CODES for READ/WRITE BUFFER commands.
Specifications require MODE parameter of those commands to be treated
the same as SERVICE ACTION parameter of other commands.

MFC after:	2 weeks
2016-12-22 11:10:11 +00:00
Alexander Motin
20549bf66f Add support for REPORTING OPTIONS == 3 in REPORT SUPPORTED OPERATION CODES.
MFC after:	2 weeks
2016-12-21 20:19:12 +00:00
Alexander Motin
33e7ba9cf9 Add support for SITUA bit in Logical Block Provisioning mode page.
VMware tries to enable this bit to avoid multiple threshold notifications
in case of multiple initiators connected to the same LUN.  Unfortunately
their code sends MODE SELECT(6) request with parameter length hardcoded
for the page without any thresholds.  Since we have four threshold and our
page is bigger, this attempt fails, that is correct in my understanding.
So all we can do about this now is to report proper error code and hope
VMware fix their code one day.

MFC after:	2 weeks
2016-12-21 15:17:47 +00:00
Alexander Motin
3157c36886 Fix UUID formatting added in r310360. Oops!
MFC after:	2 weeks
2016-12-21 09:49:16 +00:00
Alexander Motin
00d72d5714 Report UUID and MD5 LUN IDs.
MFC after:	2 weeks
2016-12-21 09:36:23 +00:00
Alexander Motin
f383d6efd5 Add support for locally assigned RFC 4122 UUID LUN identifiers.
MFC after:	2 weeks
2016-12-21 09:05:30 +00:00
Alexander Motin
171e4b68b3 Bump specifications support to SAM-6/SPC-5. 2016-12-20 21:58:43 +00:00
Alexander Motin
1f0694a680 Improve error handling when I/O split between several BIOs.
If we get several error codes, handle one with lowest offset.

MFC after:	2 weeks
2016-12-19 21:27:18 +00:00
Alexander Motin
32920cbfa3 When reporting "Logical block address out of range" error, report the LBA
in sense data INFORMATION field.

MFC after:	2 weeks
2016-12-19 19:00:03 +00:00
Alexander Motin
b9985489dd When writing fixed format sense data, set VALID bit only if provided value
for INFORMATION field fit into available 4 bytes (has no non-zero bytes
except last 4), as explicitly required by SPC-5 specification.

MFC after:	2 weeks
2016-12-19 18:55:10 +00:00
Alexander Motin
c329b295dc Fix typo in function name. 2016-12-19 15:49:59 +00:00
Alexander Motin
f60639b79e Add new bits into Extended Inquiry VPD page.
MFC after:	2 weeks
2016-12-19 15:18:31 +00:00
Alexander Motin
7f900be181 Add support for NUAR bit in Control mode page.
MFC after:	2 weeks
2016-12-19 14:19:52 +00:00
Alexander Motin
1188787581 Add set of macros to simplify code access to mode pages fields.
MFC after:	2 weeks
2016-12-19 13:25:53 +00:00
Alexander Motin
f3ee47571e Following SPC-5, make REQUEST SENSE report "Logical unit not supported"
in returned parameter data for not accessible LUNs.

MFC after:	2 weeks
2016-12-19 11:44:41 +00:00
Alexander Motin
d9ba4eefdc Improve support for informational exceptions.
While CTL still has no real events to report in this way (like SMART),
it is possible to trigger false event by manually setting TEST bit in
Informational Exceptions Control mode page, that can be useful for
initiator testing.  This code supports all flavours of IE reporting:
UNIT ATTENTION, RECOVERED ERROR and NO SENSE sense keys, REQUEST SENSE
command and Informational Exceptions log page.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2016-12-19 10:25:47 +00:00
Alexander Motin
b7599b7205 Don't treat informational exceptions (warnings and impending failures)
a.k.a. SCSI SMART events as errors.  Log them to console and continue.

MFC after:	2 weeks
2016-12-18 15:02:50 +00:00
Ed Schouten
4c484fd216 Add label annotations to CAM sysctls.
Under kern.cam we have certain sysctls that are per-device, such as the
ones under kern.cam.ada.[0-9]+.*. Add a "device_index" label annotation
to such sysctls, so that the Prometheus metrics exporter will give all
of those metrics the same name. The device number will be added to the
metric name as the "device_index" label.

Reviewed by:	cem
Differential Revision:	https://reviews.freebsd.org/D8775
2016-12-14 12:53:33 +00:00
Kenneth D. Merry
28db0a5e74 Add SCSI REPORT TIMESTAMP and SET TIMESTAMP support.
This adds support to camcontrol(8) and libcam(3) for getting and setting
the time on SCSI protocol drives.  This is more commonly found on tape
drives, but is a SPC (SCSI Primary Commands) command, and may be found
on any device that speaks SCSI.

The new camcontrol timestamp subcommand allows getting the current device
time or setting the time to the current system time or any arbitrary time.

sbin/camcontrol/Makefile:
	Add timestamp.c.

sbin/camcontrol/camcontrol.8:
	Document the new timestamp subcommand.

sbin/camcontrol/camcontrol.c:
	Add the timestamp subcommand to camcontrol.

sbin/camcontrol/camcontrol.h:
	Add the timestamp() function prototype.

sbin/camcontrol/timestamp.c:
	Timestamp setting and reporting functionality.

sys/cam/scsi/scsi_all.c:
	Add two new CCB building functions, scsi_set_timestamp() and
	scsi_report_timestamp().  Also, add a new helper function,
	scsi_create_timestamp().

sys/cam/scsi/scsi_all.h:
	Add CDB and parameter data for the the set and report timestamp
	commands.

	Add function declarations for the new CCB building and helper
	functions.

Submitted by:	Sam Klopsch
Sponsored by:	Spectra Logic
MFC After:	2 weeks
2016-12-01 22:20:27 +00:00
Alexander Motin
7027c01c56 Make SES status updates more aggressive.
- On control request update all status pages, since they may also be
affected if user enables/disables enclosure slots.
 - Periodically update element descriptors too, since there is some
hardware where they are changed dynamically.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2016-11-29 13:48:36 +00:00
Alexander Motin
ab4327bb3f Explicitly initialize cdai.flags.
In SES driver uninitialized value caused unreliable physpath reporting.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2016-11-29 11:13:43 +00:00
Alan Somers
2edc027cf0 Fix "camcontrol rescan" with SATA drives behind a SAS controller
A bug in CAM's serial number hash logic resulted in SATA drives behind a SAS
controller getting removed and readded anytime the drive was rescanned for
any reason.

PR:		212914
Submitted by:	kadesai
Reported by:	kadesai
Reviewed by:	asomers, ken
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
2016-11-17 20:42:56 +00:00
Bryan Drewery
28323add09 Fix improper use of "its".
Sponsored by:	Dell EMC Isilon
2016-11-08 23:59:41 +00:00
Conrad Meyer
8fc77fff81 cam: Zero bio pointer in user-supplied SCSI CCBs
The BUF_TRACKING bio pointer only makes sense for kernel consumers of
CCBs.

PR:		214250
Reported by:	trasz@
Reviewed by:	imp@, markj@
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D8477
2016-11-08 21:17:24 +00:00
Alexander Motin
bceff6e30a Add support for EIIOE flag in Additional Element Status.
It was added in SES-3 spec, and its support required to properly link
the Additional Element Status page data to the original elements.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2016-11-07 18:21:53 +00:00
Scott Long
34cc1bf804 If the da periph probe state machine gets an asc=0x44 error, the periph
is not going to recover until the system is reset.  Treat it as a special
case and don't allow it to fall through to quasi-success.

Reviewed by:	ken, imp
Obtained from:	Netflix
MFC after:	3 days
2016-11-04 17:08:52 +00:00
Scott Long
b575f33f67 asc/ascq 44/0 is typically a non-transient, permanent error (at least until
the components are reset).  Therefore retries are pointless.  This is very
visible in SATL systems, for example an LSI SAS controller and a SATA HDD/SSD.

Reviewed by:	ken
Obtained from:	Netflix
MFC after:	3 days
2016-11-04 16:56:36 +00:00
Edward Tomasz Napierala
a851d1fbfd Check for lengths being <= 0. Note that this interface can only
be accessed by root.  It uses unsigned ints instead of size_t
to preserve the ABI.

PR:		207627
Submitted by:	ryan@ryanday.net (with slight tweaks)
MFC after:	1 month
2016-11-03 10:11:59 +00:00
Conrad Meyer
8532d381a9 Add BUF_TRACKING and FULL_BUF_TRACKING buffer debugging
Upstream the BUF_TRACKING and FULL_BUF_TRACKING buffer debugging code.
This can be handy in tracking down what code touched hung bios and bufs
last. The full history is especially useful, but adds enough bloat that
it shouldn't be enabled in release builds.

Function names (or arbitrary string constants) are tracked in a
fixed-size ring in bufs. Bios gain a pointer to the upper buf for
tracking. SCSI CCBs gain a pointer to the upper bio for tracking.

Reviewed by:	markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D8366
2016-10-31 23:09:52 +00:00
Alexander Motin
a1a604ca90 Make pass driver better support CAM_CDB_POINTER flag.
Previously pass driver just ignored the flag, making random kernel code
access user-space pointer, sometime causing crashes even for correctly
written applications if user-level context was switched or swapped out.
This patch tries to copyin the CDB into kernel space to avoid it.

MFC after:	2 weeks
2016-10-17 15:02:17 +00:00
Alexander Motin
bc9d8bd59e Revert timeout part of r307507.
I misread 5 minutes as 5 seconds.  Timeout of 5 minutes may have sense.

MFC after:	2 weeks
2016-10-17 09:16:44 +00:00
Alexander Motin
55a1720717 Replicate r307507 for ATA disks.
MFC after:	2 weeks
2016-10-17 08:38:24 +00:00
Alexander Motin
3d946e4f5e Consider device as clean even if SYNCHRONIZE CACHE failed.
If device reservation was preempted by other initiator, our sync request
will always fail.  Without this change CAM tried to sync cache on every
following device close, including numerous GEOM tasting opens/closes,
causing lots of useless noise in logs.

While there, increase SYNCHRONIZE CACHE timeout to default value.

MFC after:	2 weeks
2016-10-17 08:35:56 +00:00
Alexander Motin
4928a0c224 Add LU option to control reported provisioning type.
MFC after:	2 weeks
2016-10-15 14:40:34 +00:00
Alexander Motin
15fd2e2412 Add LUN options to limit UNMAP and WRITE SAME sizes.
CTL itself has no limits on on UNMAP and WRITE SAME sizes.  But depending
on backends large requests may take too much time.  To avoid that new
configuration options allow to hint initiator maximal sizes it should not
exceed.

MFC after:	2 weeks
2016-10-15 10:29:33 +00:00
Alexander Motin
25ee4c0429 Use copyout() instead of pointing sbuf to user-space buffer.
MFC after:	2 weeks
2016-10-12 08:25:13 +00:00
Alexander Motin
7061250472 Fix copy/paste bug in r306747.
Submitted by:	emaste, pfg
MFC after:	3 days
2016-10-11 15:27:39 +00:00
Alexander Motin
d3c5b1fea9 Fix ABI compat shims, broken by adding NVMe support.
MFC after:	1 week
2016-10-06 01:14:10 +00:00
Mark Johnston
f30cad3355 CAM ccbq sanity: checks on insert and remove
KASSERT in cam_ccbq_insert_ccb that only XPT_FC_QUEUED ops are queued,
and XPT_FC_USER_CCB ops are not. Otherwise cam_ccbq_ccb_done may be
skipped.

Bounds check the index used for camq_remove in order to panic instead
of scribble on removal of an out-of-bounds index (e.g. consider the
effect of camq_remove of CAM_UNQUEUED_INDEX).

KASSERT in cam_ccbq_remove_ccb that the ccb removed by index was the
one sought.

Submitted by:	Ryan Libby <rlibby@gmail.com>
Reviewed by:	imp, mav
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D8151
2016-10-05 17:18:24 +00:00
Alexander Motin
769cbdb7a3 Do not retry on some security sense codes.
MFC after:	1 week
2016-10-05 00:45:58 +00:00
Mark Johnston
991b5d2630 cam_periph_ccbwait could return while ccb in progress
In cam_periph_runccb, cam_periph_ccbwait was using the value of the ccb
pinfo.index and status fields to determine whether the ccb was done,
but these fields are updated without a contending lock and could glitch
into states that would be erroneously interpreted as done.  Instead,
have cam_periph_ccbwait look for the explicit result of the function
cam_periph_done.

Submitted by:	Ryan Libby <rlibby@gmail.com>
Reviewed by:	mav
MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D8020
2016-09-30 21:00:09 +00:00
Warner Losh
cf3ec15167 Compute two new metrics. Disk load, the average number of transactions
we have queued up normaliazed to the queue size. Also compute buckets
of latency to help compute, in userland, estimates of Median, P90, P95
and P99 values.

Sponsored by: Netflix, Inc
2016-09-30 17:49:04 +00:00
Sepherosa Ziehau
a11463fd84 cam/ata: Allow drivers to veto ATA disk attachment.
This eventhandler is mainly used by VMs, e.g. Hyper-V, whose disk
controllers share the disks with the simulated ATA controllers.

Submitted by:	Hongjiang Zhang <honzhan microsoft com>
Discussed with:	mav
MFC after:	1 week
Sponsored by:	Microsoft
Differential Revision:	https://reviews.freebsd.org/D7693
2016-09-28 08:35:05 +00:00
Mark Johnston
dda945f50a Protect ccbq access with devq->send_mtx in the XPT_ABORT handler.
Submitted by:	Ryan Libby <rlibby@gmail.com>
Reviewed by:	mav
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D7985
2016-09-21 00:08:42 +00:00
Alexander Motin
cd3752643c Don't report to devd statuses that CAM doesn't consider errors.
Some statuses, such as "ATA pass through information available", are part
part of absolutely normal operation and do not worth reporting.

MFC after:	2 weeks
2016-09-08 13:33:33 +00:00
Alexander Motin
5d18110a7f "Extended copy information available" is not an error either.
MFC after:	2 weeks
2016-09-08 13:03:49 +00:00
Alexander Motin
6867747328 "ATA pass through information available" is not an error.
MFC after:	2 weeks
2016-09-08 12:58:33 +00:00
Alexander Motin
d4a08767ba Decode ATA Status Return descriptor.
MFC after:	2 weeks
2016-09-08 12:00:02 +00:00
Alexander Motin
1e3d53e2c4 Decode some new ATA commands found in ACS-3.
MFC after:	1 week
2016-08-27 19:51:37 +00:00
Navdeep Parhar
97b84d344d Make the iSCSI parameter negotiation more flexible.
Decouple the send and receive limits on the amount of data in a single
iSCSI PDU.  MaxRecvDataSegmentLength is declarative, not negotiated, and
is direction-specific so there is no reason for both ends to limit
themselves to the same min(initiator, target) value in both directions.

Allow iSCSI drivers to report their send, receive, first burst, and max
burst limits explicitly instead of using hardcoded values or trying to
derive all of them from the receive limit (which was the only limit
reported by the drivers prior to this change).

Display the send and receive limits separately in the userspace iSCSI
utilities.

Reviewed by:	jpaetzel@ (earlier version), trasz@
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7279
2016-08-25 05:22:53 +00:00
Warner Losh
3f04cc100f Improve the pattern matching so that internal *'s work, as well as
[set] notation. This fixes pattern matching for recently added drives
that would set the NCQ Trim being broken incorrectly.

PR: 210686
Tested-by: Tomoaki AOKI
MFC After: 3 days
2016-08-19 04:30:29 +00:00
Sepherosa Ziehau
0ea702cd46 cam/da: Add quirk for I-O Data USB Flash Disk
PR:		211716
Submitted by:	Jun Su <junsu microsoft com>
Reported by:	Jun Su <junsu microsoft com>
MFC after:	1 week
Sponsored by:	Microsoft
2016-08-11 03:12:56 +00:00
Pedro F. Giffuni
a061aa46fe sys: replace comma with semicolon when pertinent.
Uses of commas instead of a semicolons can easily go undetected. The comma
can serve as a statement separator but this shouldn't be abused when
statements are meant to be standalone.

Detected with devel/coccinelle following a hint from DragonFlyBSD.

MFC after:	1 month
2016-08-09 19:42:20 +00:00
Warner Losh
08f1387933 Move protocol specific stuff into a linker set object that's
per-protocol. This reduces the number scsi symbols references by
cam_xpt significantly, and eliminates all ata / nvme symbols. There's
still some NVME / ATA specific code for dealing with XPT_NVME_IO and
XPT_ATA_IO respectively, and a bunch of scsi-specific code, but this
is progress.

Differential Revision: https://reviews.freebsd.org/D7289
2016-07-28 22:55:21 +00:00
Warner Losh
ded2b70617 Switch to linker sets to find the xport callback object. This
eliminates the need to special case everything in cam_xpt for new
transports. It is now a failure to not have a transport object when
registering the bus as well. You can still, however, create a
transport that's unspecified (XPT_)

Differential Revision: https://reviews.freebsd.org/D7289
2016-07-28 22:55:14 +00:00
Warner Losh
964fee41a3 Remove some extraneous printfs. 2016-07-25 19:16:30 +00:00
Warner Losh
e0ce51cb80 Remove some bogus comments and printfs. Also, we can't
cam_periph_releaes_locked() at the end of nvme_probe_start because we
hit an assertion which may be bogus. Instead, leak a periph until we
sort it out. Since these devices don't arrive and depart often, so
this is the lessor of two evils.

MFC after: 1 week
2016-07-21 03:11:37 +00:00
Warner Losh
336b7b6665 Fix mismerge and add NVME xport support to devstat (or rather, exclude
it from SCSI).

MFC after: 1 week
2016-07-21 03:11:36 +00:00
Warner Losh
b24ced67d0 Fix mismerge and include the nvme support.
Also, print out the name of any CCB's functions that's not supported.

MFC after: 1 week
2016-07-21 03:11:35 +00:00
Alexander Motin
db8e94bb3b Restore PIM_ATA_EXT flag handling, lost at r300207.
This re-enables NCQ TRIM usage on capable hardware (bhyve).
2016-07-17 14:17:58 +00:00
Warner Losh
035ec48e55 Tidy up loose ends from Netflix I/O sched rename to dynamic I/O sched.
Rename kern.cam.do_netflix_iosched sysctl to
kern.cam.do_dynamic_iosched.

Approved by: re (kib@)
2016-07-07 20:31:35 +00:00
Sean Bruno
3f7606d5a3 Correct PERSISTENT RESERVE OUT command and populate scsi_cmd->length.
PR:	202625
Submitted by:	niakrisn@gmail.com
Reviewed by:	scottl kenm
Approved by:	re (gjb)
MFC after:	2 weeks
2016-06-29 16:41:37 +00:00
Sean Bruno
795b21ec94 Revert svn r302253 at the request/review of Ken M. This commit is
incorrect.

PR:		202625
Approved by:	re (implicit)
2016-06-28 18:32:15 +00:00
Sean Bruno
9df32773f3 Correct PERSISTENT RESERVE OUT command and populate scsi_cmd->length.
PR:		202625
Submitted by:	niakrisn@gmail.com
Reviewed by:	scottl
Approved by:	re (hrs)
MFC after:	2 weeks
2016-06-28 18:08:47 +00:00
Warner Losh
df2362478e Rename CAM_NETFLIX_IOSCHED to CAM_IOSCHED_DYNAMIC to better reflect
its nature.

Approved by: re
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D6811
2016-06-23 23:20:58 +00:00
Warner Losh
f24c011beb Commit the bits of nda that were missed. This should fix the build.
Approved by: re@
2016-06-10 06:04:53 +00:00
Warner Losh
baabaca31b New NVMe front end (nda). 2016-06-09 22:39:02 +00:00
Warner Losh
bbb19fc720 Add place holder for SDIO CAM stuff for CCB XPT type.
Sponsored by: Netflix
2016-06-09 16:05:56 +00:00
Warner Losh
7b05c3e3c1 Add NVME IO type. 2016-06-09 16:05:49 +00:00
Warner Losh
69be012ff2 Improve debugging of xpt.
Sponsored by: Netflix
2016-06-09 16:05:40 +00:00
Conrad Meyer
75548271a9 Fix memory leaks in (a|)daregister introduced in r298002
In the case where cam_iosched_init() fails, the ada and da softcs were leaked.
Instead, free them.

Reported by:	Coverity
CID:		1356039
Sponsored by:	EMC / Isilon Storage Division
2016-06-07 20:33:55 +00:00
Edward Tomasz Napierala
4e5408f10c Report negotiated MaxBurstLength and FirstBurstLength in "iscsictl -v"
and "ctladm islist -v" outputs.

MFC after:	1 month
2016-06-05 08:48:37 +00:00
Alan Somers
583febf416 Strip leading spaces off of a SCSI disk's serial number
sys/cam/scsi/scsi_xpt.c
	Strip leading spaces off of a SCSI disk's reported serial number
	when populating the CAM serial number. This affects the output of
	"diskinfo -v" and the names of /dev/diskid/DISK-* device nodes,
	among other things.

	SPC5r05 says that the Product Serial Number field from the Unit
	Serial Number VPD page is right-aligned. So any leading spaces are
	not part of the actual serial number. Most devices don't left-pad
	their serial numbers, but some do. In particular, the SN VPD page
	that an LSI HBA emulates for a SATA drive contains enough
	left-padding to fill a 20-byte field.

UPDATING
	Add a note to UPDATING, because some users may have to update
	/etc/fstab or geom labels.

Reviewed by:	ken, mav
MFC after:	Never
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D6516
2016-05-27 22:26:43 +00:00
Kenneth D. Merry
600fd98ff3 Fix a few ada(4) driver issues:
o Some Samsung drives do not support the ATA READ LOG EXT or READ
   LOG DMA EXT commands, despite indicating that they do in their
   IDENTIFY data.  So, fix this in two ways:
	1. Only start the log directory probe (ADA_STATE_LOGDIR) if
	   the drive claims to be an SMR drive in the first place.
	   We don't need to do the extra probing for other devices.
	   This will also serve to prevent problems with other
	   drives that have the same issue.
	2. Add quirks for the two Samsung drives that have been
	   reported so far (thanks to Oleg Nauman and Alex Petrov).
	   If there is a reason to do a Read Log later on, we will
	   know that it doesn't work on these drives.

 o Add a quirk entry to mark Seagate Lamarr Drive Managed drives as
   drive managed.  They don't report this in their Identify data.

sys/cam/ata/ata_da.c:
	Add two new quirks:
	1. ADA_Q_LOG_BROKEN, for drives that claim to support Read
	   Log but don't really.
	2. ADA_Q_SMR_DM, for drives that are Drive Managed SMR, but
	   don't report it.  This can matter for software that
	   wants to know when it should make an extra effort to
	   write sequentially.

	Record two Samsung drives that don't support Read Log, and
	one Seagate drive that doesn't report that it is a SMR drive.
	The Seagate drive is already recorded in the da(4) driver.

	We may have to come up with a similar solution in the da(4)
	driver for SATA drives that don't properly support Read Log.

	In adasetflags(), Dont' set the ADA_FLAG_CAN_LOG bit if the
	device has the LOG_BROKEN quirk set.  Also, look at the
	SMR_DM quirk and set the device type accordingly if it is
	actually a drive managed drive.

	When deciding whether to go into the LOGDIR probe state,
	look to see whether the device claims to be an SMR device.
	If not, don't bother with the LOGDIR probe state.

Sponsored by:	Spectra Logic
2016-05-25 01:37:39 +00:00
Edward Tomasz Napierala
b891159418 Add mechanism for choosing iSER-capable ICL modules.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-05-24 08:44:45 +00:00
Don Lewis
95320acebc Fix multiple Coverity Out-of-bounds access false postive issues in CAM
The currently used idiom for clearing the part of a ccb after its
header generates one or two Coverity errors for each time it is
used.  All instances generate an Out-of-bounds access (ARRAY_VS_SINGLETON)
error because of the treatment of the header as a two element array,
with a pointer to the non-existent second element being passed as
the starting address to bzero().  Some instances also alsp generate
Out-of-bounds access (OVERRUN) errors, probably because the space
being cleared is larger than the sizeofstruct ccb_hdr).

In addition, this idiom is difficult for humans to understand and
it is error prone.  The user has to chose the proper struct ccb_*
type (which does not appear in the surrounding code) for the sizeof()
in the length calculation.  I found several instances where the
length was incorrect, which could cause either an actual out of
bounds write, or incompletely clear the ccb.

A better way is to write the code to clear the ccb itself starting
at sizeof(ccb_hdr) bytes from the start of the ccb, and calculate
the length based on the specific type of struct ccb_* being cleared
as specified by the union ccb member being used.  The latter can
normally be seen in the nearby code.  This is friendlier for Coverity
and other static analysis tools because they will see that the
intent is to clear the trailing part of the ccb.

Wrap all of the boilerplate code in a convenient macro that only
requires a pointer to the desired union ccb member (or a pointer
to the union ccb itself) as an argument.

Reported by:	Coverity
CID:		1007578, 1008684, 1009724, 1009773, 1011304, 1011306
CID:		1011307, 1011308, 1011309, 1011310, 1011311, 1011312
CID:		1011313, 1011314, 1011315, 1011316, 1011317, 1011318
CID:		1011319, 1011320, 1011321, 1011322, 1011324, 1011325
CID:		1011326, 1011327, 1011328, 1011329, 1011330, 1011374
CID:		1011390, 1011391, 1011392, 1011393, 1011394, 1011395
CID:		1011396, 1011397, 1011398, 1011399, 1011400, 1011401
CID:		1011402, 1011403, 1011404, 1011405, 1011406, 1011408
CID:		1011409, 1011410, 1011411, 1011412, 1011413, 1011414
CID:		1017461, 1018387, 1086860, 1086874, 1194257, 1229897
CID:		1229968, 1306229, 1306234, 1331282, 1331283, 1331294
CID:		1331295, 1331535, 1331536, 1331539, 1331540, 1341623
CID:		1341624, 1341637, 1341638, 1355264, 1355324
Reviewed by:	scottl, ken, delphij, imp
MFH:		1 month
Differential Revision:	https://reviews.freebsd.org/D6496
2016-05-24 00:57:11 +00:00
Kenneth D. Merry
3f54ec85e8 Fix ada(4) trim support quirk setting.
I broke broke the quirk in the ada(4) driver disabling NCQ trim support
in revision 300207.  The support flags were set before the quirks were
loaded.

sys/cam/ata/ata_da.c:
	Call adasetflags() after loading quirks, so that we'll set the
	flags accurately.

Sponsored by:	Spectra Logic
2016-05-23 19:52:08 +00:00
Alexander Motin
96b5475b7a Pass task management response information from CTL through CAM to isp(4),
utilizing previously unused arg field of struct ccb_notify_acknowledge.

This makes new QUERY TASK, QUERY TASK SET and QUERY ASYNC EVENT requests
really functional for CAM target mode drivers.
2016-05-20 10:26:12 +00:00
Kenneth D. Merry
9a6844d55f Add support for managing Shingled Magnetic Recording (SMR) drives.
This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to
the Zoned ATA Command Set or ZAC spec) behind SAS expanders.

This includes full management support through the GEOM BIO interface, and
through a new userland utility, zonectl(8), and through camcontrol(8).

This is now ready for filesystems to use to detect and manage zoned drives.
(There is no work in progress that I know of to use this for ZFS or UFS, if
anyone is interested, let me know and I may have some suggestions.)

Also, improve ATA command passthrough and dispatch support, both via ATA
and ATA passthrough over SCSI.

Also, add support to camcontrol(8) for the ATA Extended Power Conditions
feature set.  You can now manage ATA device power states, and set various
idle time thresholds for a drive to enter lower power states.

Note that this change cannot be MFCed in full, because it depends on
changes to the struct bio API that break compatilibity.  In order to
avoid breaking the stable API, only changes that don't touch or depend on
the struct bio changes can be merged.  For example, the camcontrol(8)
changes don't depend on the new bio API, but zonectl(8) and the probe
changes to the da(4) and ada(4) drivers do depend on it.

Also note that the SMR changes have not yet been tested with an actual
SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports
ZBC to ZAC translation.  I have not yet gotten a suitable drive or SAT
layer, so any testing help would be appreciated.  These changes have been
tested with Seagate Host Aware SATA drives attached to both SAS and SATA
controllers.  Also, I do not have any SATA Host Managed devices, and I
suspect that it may take additional (hopefully minor) changes to support
them.

Thanks to Seagate for supplying the test hardware and answering questions.

sbin/camcontrol/Makefile:
	Add epc.c and zone.c.

sbin/camcontrol/camcontrol.8:
	Document the zone and epc subcommands.

sbin/camcontrol/camcontrol.c:
	Add the zone and epc subcommands.

	Add auxiliary register support to build_ata_cmd().  Make sure to
	set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA
	flags as appropriate for ATA commands.

	Add a new get_ata_status() function to parse ATA result from SCSI
	sense descriptors (for ATA passthrough over SCSI) and ATA I/O
	requests.

sbin/camcontrol/camcontrol.h:
	Update the build_ata_cmd() prototype

	Add get_ata_status(), zone(), and epc().

sbin/camcontrol/epc.c:
	Support for ATA Extended Power Conditions features.  This includes
	support for all features documented in the ACS-4 Revision 12
	specification from t13.org (dated February 18, 2016).

	The EPC feature set allows putting a drive into a power power mode
	immediately, or setting timeouts so that the drive will
	automatically enter progressively lower power states after various
	idle times.

sbin/camcontrol/fwdownload.c:
	Update the firmware download code for the new build_ata_cmd()
	arguments.

sbin/camcontrol/zone.c:
	Implement support for Shingled Magnetic Recording (SMR) drives
	via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA
	Command Set (ZAC).

	These specs were developed in concert, and are functionally
	identical.  The primary differences are due to SCSI and ATA
	differences.  (SCSI is big endian, ATA is little endian, for
	example.)

	This includes support for all commands defined in the ZBC and
	ZAC specs.

sys/cam/ata/ata_all.c:
	Decode a number of additional ATA command names in ata_op_string().

	Add a new CCB building function, ata_read_log().

	Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building
	functions.  These support both DMA and NCQ encapsulation.

sys/cam/ata/ata_all.h:
	Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and
	ata_zac_mgmt_in().

sys/cam/ata/ata_da.c:
	Revamp the ada(4) driver to support zoned devices.

	Add four new probe states to gather information needed for zone
	support.

	Add a new adasetflags() function to avoid duplication of large
	blocks of flag setting between the async handler and register
	functions.

	Add new sysctl variables that describe zone support and paramters.

	Add support for the new BIO_ZONE bio, and all of its subcommands:
	DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
	DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

sys/cam/scsi/scsi_all.c:
	Add command descriptions for the ZBC IN/OUT commands.

	Add descriptions for ZBC Host Managed devices.

	Add a new function, scsi_ata_pass() to do ATA passthrough over
	SCSI.  This will eventually replace scsi_ata_pass_16() -- it
	can create the 12, 16, and 32-byte variants of the ATA
	PASS-THROUGH command, and supports setting all of the
	registers defined as of SAT-4, Revision 5 (March 11, 2016).

	Change scsi_ata_identify() to use scsi_ata_pass() instead of
	scsi_ata_pass_16().

	Add a new scsi_ata_read_log() function to facilitate reading
	ATA logs via SCSI.

sys/cam/scsi/scsi_all.h:
	Add the new ATA PASS-THROUGH(32) command CDB.  Add extended and
	variable CDB opcodes.

	Add Zoned Block Device Characteristics VPD page.

	Add ATA Return SCSI sense descriptor.

	Add prototypes for scsi_ata_read_log() and scsi_ata_pass().

sys/cam/scsi/scsi_da.c:
	Revamp the da(4) driver to support zoned devices.

	Add five new probe states, four of which are needed for ATA
	devices.

	Add five new sysctl variables that describe zone support and
	parameters.

	The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC
	devices when they are attached via a SCSI to ATA Translation (SAT)
	layer.  Since ZBC -> ZAC translation is a new feature in the T10
	SAT-4 spec, most SATA drives will be supported via ATA commands
	sent via the SCSI ATA PASS-THROUGH command.  The da(4) driver will
	prefer the ZBC interface, if it is available, for performance
	reasons, but will use the ATA PASS-THROUGH interface to the ZAC
	command set if the SAT layer doesn't support translation yet.
	As I mentioned above, ZBC command support is untested.

	Add support for the new BIO_ZONE bio, and all of its subcommands:
	DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
	DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

	Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.

	Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB
	building functions.  Note that these have return values, unlike
	almost all other CCB building functions in CAM.  The reason is
	that they can fail, depending upon the particular combination
	of input parameters.  The primary failure case is if the user
	wants NCQ, but fails to specify additional CDB storage.  NCQ
	requires using the 32-byte version of the SCSI ATA PASS-THROUGH
	command, and the current CAM CDB size is 16 bytes.

sys/cam/scsi/scsi_da.h:
	Add ZBC IN and ZBC OUT CDBs and opcodes.

	Add SCSI Report Zones data structures.

	Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and
	scsi_ata_zac_mgmt_in() prototypes.

sys/dev/ahci/ahci.c:
	Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.

	ahci_setup_fis() previously set the top bits of the sector count
	register in the FIS to 0 for FPDMA commands.  This is okay for
	read and write, because the PRIO field is in the only thing in
	those bits, and we don't implement that further up the stack.

	But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that
	byte, so it needs to be transmitted to the drive.

	In ahci_setup_fis(), always set the the top 8 bits of the
	sector count register.  We need it in both the standard
	and NCQ / FPDMA cases.

sys/geom/eli/g_eli.c:
	Pass BIO_ZONE commands through the GELI class.

sys/geom/geom.h:
	Add g_io_zonecmd() prototype.

sys/geom/geom_dev.c:
	Add new DIOCZONECMD ioctl, which allows sending zone commands to
	disks.

sys/geom/geom_disk.c:
	Add support for BIO_ZONE commands.

sys/geom/geom_disk.h:
	Add a new flag, DISKFLAG_CANZONE, that indicates that a given
	GEOM disk client can handle BIO_ZONE commands.

sys/geom/geom_io.c:
	Add a new function, g_io_zonecmd(), that handles execution of
	BIO_ZONE commands.

	Add permissions check for BIO_ZONE commands.

	Add command decoding for BIO_ZONE commands.

sys/geom/geom_subr.c:
	Add DDB command decoding for BIO_ZONE commands.

sys/kern/subr_devstat.c:
	Record statistics for REPORT ZONES commands.  Note that the
	number of bytes transferred for REPORT ZONES won't quite match
	what is received from the harware.  This is because we're
	necessarily counting bytes coming from the da(4) / ada(4) drivers,
	which are using the disk_zone.h interface to communicate up
	the stack.  The structure sizes it uses are slightly different
	than the SCSI and ATA structure sizes.

sys/sys/ata.h:
	Add many bit and structure definitions for ZAC, NCQ, and EPC
	command support.

sys/sys/bio.h:
	Convert the bio_cmd field to a straight enumeration.  This will
	yield more space for additional commands in the future.  After
	change r297955 and other related changes, this is now possible.
	Converting to an enumeration will also prevent use as a bitmask
	in the future.

sys/sys/disk.h:
	Define the DIOCZONECMD ioctl.

sys/sys/disk_zone.h:
	Add a new API for managing zoned disks.  This is very close to
	the SCSI ZBC and ATA ZAC standards, but uses integers in native
	byte order instead of big endian (SCSI) or little endian (ATA)
	byte arrays.

	This is intended to offer to the complete feature set of the ZBC
	and ZAC disk management without requiring the application developer
	to include SCSI or ATA headers.  We also use one set of headers
	for ioctl consumers and kernel bio-level consumers.

sys/sys/param.h:
	Bump __FreeBSD_version for sys/bio.h command changes, and inclusion
	of SMR support.

usr.sbin/Makefile:
	Add the zonectl utility.

usr.sbin/diskinfo/diskinfo.c
	Add disk zoning capability to the 'diskinfo -v' output.

usr.sbin/zonectl/Makefile:
	Add zonectl makefile.

usr.sbin/zonectl/zonectl.8
	zonectl(8) man page.

usr.sbin/zonectl/zonectl.c
	The zonectl(8) utility.  This allows managing SCSI or ATA zoned
	disks via the disk_zone.h API.  You can report zones, reset write
	pointers, get parameters, etc.

Sponsored by:	Spectra Logic
Differential Revision:	https://reviews.freebsd.org/D6147
Reviewed by:	wblock (documentation)
2016-05-19 14:08:36 +00:00
Eitan Adler
cef367e6a1 Don't repeat the the word 'the'
(one manual change to fix grammar)

Confirmed With: db
Approved by: secteam (not really, but this is a comment typo fix)
2016-05-17 12:52:31 +00:00
Edward Tomasz Napierala
f41492b00f Add icl_conn_connect() ICL method, required for iSER.
Obtained from:	Mellanox Technologies (earlier version)
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-05-17 11:10:44 +00:00
Mark Johnston
565e7fd3bc opt_kdtrace.h is not needed for SDT probes as of r258541. 2016-05-15 20:04:43 +00:00
Alexander Motin
ed246e8260 Allow sleepable allocations in enclosure daemon threads.
There were at least two places where M_NOWAIT was used without NULL check.
This change should fix NULL-dereference panic there and possibly improve
operation in other ways under memory pressure.

MFC after:	2 weeks
2016-05-10 16:20:36 +00:00
Edward Tomasz Napierala
d68fae5849 Add "camcontrol reprobe" subcommand, and implement it for da(4).
This makes it possible to manually force updating capacity data
after the disk got resized. Without it it might be neccessary to
reboot before FreeBSD notices updated disk size under eg VMWare.

Discussed with:	imp@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6108
2016-05-10 15:46:33 +00:00
Alexander Motin
8fadf66094 Fix previous commit to report proper error code.
MFC after:	2 weeks
2016-05-10 08:37:41 +00:00
Alexander Motin
38618bf430 Validate XCOPY range offsets and lengths.
MFC after:	2 weeks
2016-05-10 08:28:16 +00:00
Alexander Motin
e13f4248db More XCOPY parameters validation.
MFC after:	2 weeks
2016-05-10 08:08:39 +00:00
Alexander Motin
3eb7651aad Improve validation of some POPULATE TOKEN parameters.
MFC after:	2 weeks
2016-05-10 07:14:49 +00:00
Pedro F. Giffuni
ea0ccffc39 cam/scsi: Remove mostly unused scsi_quirk_table_size.
It was only being used only once and it can be directly replaced
instead.
2016-05-06 16:51:56 +00:00
Pedro F. Giffuni
c1f403238a sys/cam/scsi: unsigned some loop indexes.
Although usually small, values produced by nitems() are unsigned.
One more case where unsgining the index may be useful.
2016-05-06 15:36:25 +00:00
Pedro F. Giffuni
2a392dd62b sys/cam/scsi: unsigned some loop indexes.
Although usually small, values produced by nitems() are unsigned.
By unsigning the corresponding indexes we avoid signed vs unsigned
comparisons. This may have some effect on performance, although given the
small sizes the effect will not be perceivable and it makes the code
clearer.
2016-05-06 15:13:44 +00:00
Scott Long
de482e7bbb Fix a memory leak in the devctl notify code.
Submitted by:	markj
MFC after:	asap
2016-05-03 14:30:26 +00:00
Pedro F. Giffuni
1ffe58516c sys/cam: spelling fixes in comments.
No functional change.
2016-04-29 21:05:48 +00:00
Pedro F. Giffuni
d63978e2b6 sys/cam: spelling fixes.
Affects a debug message.

MFC after: 2 weeks
2016-04-29 21:04:06 +00:00
Pedro F. Giffuni
e45a63ee67 Small typos. 2016-04-28 15:18:28 +00:00
Pedro F. Giffuni
167e63e394 cam: unsign some types to match their definitions and avoid overflows.
numpatterns is u_int.

ctl:
CTL_NUM_MODE_PAGES comes from sizeof().
In struct:ctl_scsiio, kern_sg_entries is uint32_t.

MFC after:	2 weeks
2016-04-27 15:35:05 +00:00
Pedro F. Giffuni
55e0987aea sys: extend use of the howmany() macro when available.
We have a howmany() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
2016-04-26 15:38:17 +00:00
Pedro F. Giffuni
a380994fff Yet more redundant parenthesis from r298431.
Mea culpa.
2016-04-21 20:30:38 +00:00
Pedro F. Giffuni
323b076e9c sys: use our nitems() macro when param.h is available.
This should cover all the remaining cases in the kernel.

Discussed in:	freebsd-current
2016-04-21 19:40:10 +00:00
Pedro F. Giffuni
8dfea46460 Remove slightly used const values that can be replaced with nitems().
Suggested by:	jhb
2016-04-21 15:38:28 +00:00
Sean Bruno
bbea316cc8 Plug memory leak in ctl(4) when ctl_copyin_args() is called with a non-
null terminated ASCII string.

PR:		207626
Submitted by:	cturt@hardenedbsd.org
MFC after:	2 days
2016-04-19 16:48:14 +00:00
Alan Somers
f94033f407 Add the ability to read a SAS device's Target Port NAA designator
sys/cam/scsi/scsi_all.h
sys/cam/scsi/scsi_all.c
	Add the scsi_devid_is_port_naa helper function

Reviewed by:	ken
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D5975
2016-04-18 15:01:59 +00:00
Xin LI
b97b6d27f2 Fix tinderbox LINT build. 2016-04-18 08:24:13 +00:00
Warner Losh
2b5c19f196 Do the intmax_t dance for debug so CAM_NETFLIX_IOSCHED builds on
i386.

Sponsored by: Netflix, Inc
2016-04-17 21:29:44 +00:00
Warner Losh
916d57dfc5 Implement Auxiliary register. Add PIM_ATA_EXT flag to flag that a SIM
can handle it, and add the code to add it to the FIS that's sent to
the drive. The mvs driver is the only other ATA driver in the system,
and its hardware doesn't appear to support setting the Auxiliary
register.

Differential Revision: https://reviews.freebsd.org/D5598
2016-04-17 05:24:36 +00:00
Warner Losh
e4cc6558b3 tag_action is not used at all in ata. It's set to 1 for ordered
transactions, but that value isn't used. It's bogusly used to report
in devstat, due to a cut and paste error from SCSI. Mark it as unused
in cam_fill_ataio. Reclaim the memory as a new ata_flags. In addition,
tag_id and init_id are completely unused, so reclaim those as 'unused'
now too. These were needlessly copied when ata was split from scsi.

This allows us, in the future, to create structures that can
communicate AUXILIARY regsiter to the SIMs, which cannot be done now.

Differential Revision: https://reviews.freebsd.org/D5598
2016-04-17 05:24:28 +00:00
Warner Losh
bf95d6a610 Dell has an OEM drive from Samsung that has issues. NCQ Trim isn't
broken on this drive, but it doesn't support it and the fallback logic
is failing. Quirk it until those issues can be resolved in a more
generic way.
2016-04-17 02:06:10 +00:00
Warner Losh
acfc9b6862 Expand CAM_IO_STATS #ifdef to logical unit. 2016-04-15 05:10:39 +00:00
Warner Losh
5ede5b8cb5 Put function only used by CAM_NETFLIX_IOSCHED under that ifdef. 2016-04-15 05:10:32 +00:00
Warner Losh
b93ecd35e7 Out of an abundance of caution treat
* Samsung 843T Series SSDs (MZ7WD*)
 * Samsung PM851 Series SSDs (MZ7TE*)
 * Samsung PM853T Series SSDs (MZ7GE*)
as known having broken NCQ TRIM support as they appear to be based on
the same controller technology as the 840 and 850 series.

I've had at least one report of the PM853 being broken, so err on the
side of caution for the above drives. The PM863/SM863 appears to be
based on a newer controller, so give it the benefit of the doubt.
2016-04-15 05:10:31 +00:00
Warner Losh
555bb680cc Add FCCT M500 to the NCQ black list. Linux added it in 4.2 (August
2015). Correct the M500 firmware versions. EU07 was the engineering
test version, not the release version with the fix. MU07 is the
release version. It's the only Micron firmware version to actually
work. Remove support for EU07.

This brings the blacklist into parity with the Linux blacklist as of
4.5, except for the Micron M500 MU07 entry. I personally tested the
MU07 firmware on 12 machines running 6 drives each with no corruption
in the past 6 months with Netflix production loads. Prior versions of
the M500 firmware wouldn't last more than a few days.

Sponsored by: Netflix, Inc.
2016-04-15 03:10:04 +00:00
Warner Losh
f3bea265e1 Use the new TUNABLE_INT64 to match the type of sbintime_t. 2016-04-15 03:09:58 +00:00
Enji Cooper
da908789ee Fix typos (intenral -> internal) in comments 2016-04-15 02:36:14 +00:00
Warner Losh
86ddf15ebd Add a comment about why the timeout for flush was lowered to 5s. 2016-04-14 22:13:46 +00:00
Warner Losh
ba6c22ce93 Add in missing files from r298002. 2016-04-14 22:13:44 +00:00
Scott Long
a30ecd42b8 Add a devctl/devd notification conduit for CAM errors that happen at the
periph level.  When a relevant error is reported to the periph, some
amplifying information is gathered, and the error and information are fed
to devctl with the attributes / keys system=CAM, subsystem=periph.  The
'type' key will be either 'error' or 'timeout', and based on this, various
other keys are also populated.

The purpose of this is to provide a concise mechanism for error reporting
that is less noisy than the system console but higher in resolution and
fidelity than simple sysctl counters.  We will be using it at Netflix to
populate a structured log and database to track errors and error trends
across our world-wide population of drives.

Submitted by:	imp, scottl
Approved by:	kenm
MFC after:	3 days
Sponsored by:	Netflix
Differential Revision:	D5943
2016-04-14 21:55:42 +00:00
Warner Losh
a6e0c5da99 New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the same
as before. The common scheduling bits have moved from inline code in
each of the CAM periph drivers into a library that implements the
default scheduling.

In addition, a number of rate-limiting and I/O preference options can
be enabled by adding CAM_IOSCHED_NETFLIX to your config file. A number
of extra stats are also maintained. CAM_IOSCHED_NETFLIX isn't on by
default because it uses a separate BIO_READ and BIO_WRITE queue, so
doesn't honor BIO_ORDERED between these two types of operations. We
already didn't honor it for BIO_DELETE, and we don't depend on
BIO_ORDERED between reads and writes anywhere in the system (it is
currently used with BIO_FLUSH in ZFS to make sure some writes are
complete before others start and as a poor-man's soft dependency in
one place in UFS where we won't be issuing READs until after the
operation completes). However, out of an abundance of caution, it
isn't enabled by default.

Plus, this also brings in NCQ TRIM support for those SSDs that support
it. A black list is also provided for known rogues that use NCQ trim
as an excuse to corrupt the drive. It was difficult to separate out
into a separate commit.

This code has run in production at Netflix for over a year now.

Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D4609
2016-04-14 21:47:58 +00:00
Scott Long
c9767ca834 Add sbuf variants ata_cmd_sbuf() and ata_res_sbuf(), and reimplement the
_string variants on top of this.  This requires a change to the function
signature of ata_res_sbuf().  Its use in the tree seems to be very limited,
and the change makes it more consistent with the rest of the API.

Reviewed by:	imp, mav, kenm
Sponsored by:	Netflix
Differential Revision:	D5940
2016-04-13 20:10:06 +00:00
Scott Long
fd369120f8 Use scsi_cdb_sbuf() inside of scsi_command_string now that the temporary
string storage is no longer needed.

MFC after:	3 days
Sponsored by:	Netflix
2016-04-13 15:57:13 +00:00
Scott Long
4b35e39c6a Add scsi_cdb_sbuf() for handling CDB strings. Reimplement scsi_cdb_string()
in terms of it.

Reviewed by:	imp, mav, ken
MFC after:	3 days
Sponsored by:	Netflix
Differential Revision:	D5934
2016-04-13 15:43:11 +00:00
Pedro F. Giffuni
74b8d63dcc Cleanup unnecessary semicolons from the kernel.
Found with devel/coccinelle.
2016-04-10 23:07:00 +00:00
Alexander Motin
4cace9ebf6 Add couple new constants from SPC5r08. 2016-04-09 18:39:57 +00:00
Pedro F. Giffuni
d8e49c71e3 chdone(): Prevent returning uninitialized scalar value.
Improve over the solution in r297527:
Instead of attempting to initialize all the possible cases, just
move the check nearer to the case where it makes sense.

CID:		1006486
Reviewed by:	ken
MFC after:	2 weeks
2016-04-07 21:33:14 +00:00
Pedro F. Giffuni
f18b18fe5f chdone(): Prevent returning uninitialized scalar value.
If there is an error different from ERESTART, there is some
chance that we may end up accessing an uninitialized value. This
doesn't seem likely/possible but initialize announce_buf[0],
just in case.

CID:	1006486
2016-04-03 16:38:15 +00:00
Pedro F. Giffuni
5a0c02d2e6 Small typo. 2016-04-02 01:14:51 +00:00
Jean-Sébastien Pédron
eae90da9b0 CAM: Generalize 4k quirk to all Samsung MZ7* SSDs
This adds Samsung PM851 to the list. It can be found in Lenovo Thinkpad
T440 for instance.

Reviewed by:	Kevin Bowling <kevin.bowling@kev009.com>,
		Jason Wolfe <j@nitrology.com>
Approved by:	Kevin Bowling <kevin.bowling@kev009.com>,
		Jason Wolfe <j@nitrology.com>
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D5753
2016-03-29 06:56:46 +00:00
Sean Bruno
844b798499 Add 4k enabled cam quirks for Samsung SM863 Series SSDs
Submitted by:	Jason (j@nitrology.com)
MFC after:	2 weeks
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D5711
2016-03-24 14:20:33 +00:00
Warner Losh
4aa947cb70 Make sure we check for CAM_CDB_POINTER for all drivers. Also, for the
drivers I've touched, filter out CAM_CDB_PHYS.

Differential Revision: https://reviews.freebsd.org/D5585
2016-03-15 05:17:29 +00:00
Warner Losh
a2531862b8 Move to new value for XPT_GET_SIM_KNOB to avoid clash with XPT_ATA_IO. 2016-03-10 06:25:05 +00:00
Alexander Motin
1a686265ee Set bhsdi_target_transfer_tag to reserved value, which is 0xffffffff.
This should be a purely cosmetic change.
2016-03-04 19:57:11 +00:00
Warner Losh
787db28adf Remove a stray else. It isn't needed (due to the return at the end of
the if statement it pairs with). While not an error today, a careless
edit in the future could cause problems (though given the nature of
this specific code, the problems quite likely would be some variation
of "most direct access SCSI storage devices won't attach," which is
unlikely to go unnoticed).

PVS-Studio: V705
2016-02-18 15:12:52 +00:00
Edward Tomasz Napierala
c5805a3deb Remove stray semicolons from the iSCSI code.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-02-10 17:55:39 +00:00
Kenneth D. Merry
07b1ab582b Fix the SCSI Extended INQUIRY probe case when an error is returned
and a retry is scheduled.

Instead of leaving the device queue frozen, unfreeze the device queue so
that the retry can happen.

Sponsored by:	Spectra Logic
MFC after:	3 days
2016-02-08 22:13:08 +00:00
Ravi Pokala
dd4637c078 Add defines for WRITE_UNCORRECTABLE ATA command, and improve command logging
Add #defines for ATA_WRITE_UNCORRECTABLE48 and its features. Update the
decoding in ATACAM to recognize the new values. Also improve command
decoding for a few other commands (SMART, NOP, SET_FEATURES). Bring the
decoding in ata(4) up to parity with ATACAM.

Reviewed by:	mav, imp
MFC after:	1 month
Sponsored by:	Panasas, Inc.
Differential Revision:	https://reviews.freebsd.org/D5181
2016-02-04 19:53:54 +00:00
Marcelo Araujo
314c683931 Clean up unused-but-set-variable spotted by gcc-4.9.
Reviewed by:	mav
Approved by:	rodrigc (mentor)
MFC after:	2 weeks
Sponsored by:	gandi.net
Differential Revision:	https://reviews.freebsd.org/D5109
2016-02-01 02:33:58 +00:00
Alexander Motin
72cc93767c Hide "soconnect() error" messages under bootverbose.
They can be too noisy.
2016-01-22 09:32:19 +00:00
Konstantin Belousov
ee1988938c Convert sys/cam to use make_dev_s().
Reviewed by:	hps, jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	3 weeks
Differential revision:	https://reviews.freebsd.org/D4746
2016-01-07 20:22:55 +00:00
Mark Johnston
3616095801 Fix style issues around existing SDT probes.
- Use SDT_PROBE<N>() instead of SDT_PROBE(). This has no functional effect
  at the moment, but will be needed for some future changes.
- Don't hardcode the module component of the probe identifier. This is
  set automatically by the SDT framework.

MFC after:	1 week
2015-12-16 23:39:27 +00:00
Alexander Motin
1e6a6c096e Set DS flag, required for LPB log page by spec.
MFC after:	1 week
2015-12-15 21:19:32 +00:00
Kenneth D. Merry
23d63288c2 The ccb_xflags enumeration was removed from FreeBSD/head in
r259397 (it contained the CAM_EXTLUN_VALID bit) and I added the
same type name with a different set of values back in r291716.

The old ccb_xflags enumeration still exists in FreeBSD stable/10.
Shift all of the new values by one bit to avoid compatibility
issues when merged to stable/10.

MFC after:	3 days
Sponsored by:	Spectra Logic
2015-12-07 21:04:27 +00:00
Warner Losh
8272782498 Improve cam tracing a little by including the function code in the
traces for xpt_action. Note up-calls (down-calls?) to the SIM as well.

Differential Review: https://reviews.freebsd.org/D4382
2015-12-07 04:14:29 +00:00
Kenneth D. Merry
a9934668aa Add asynchronous command support to the pass(4) driver, and the new
camdd(8) utility.

CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and
completed CCBs may be retrieved via the CAMIOGET ioctl.  User
processes can use poll(2) or kevent(2) to get notification when
I/O has completed.

While the existing CAMIOCOMMAND blocking ioctl interface only
supports user virtual data pointers in a CCB (generally only
one per CCB), the new CAMIOQUEUE ioctl supports user virtual and
physical address pointers, as well as user virtual and physical
scatter/gather lists.  This allows user applications to have more
flexibility in their data handling operations.

Kernel memory for data transferred via the queued interface is
allocated from the zone allocator in MAXPHYS sized chunks, and user
data is copied in and out.  This is likely faster than the
vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in
configurations with many processors (there are more TLB shootdowns
caused by the mapping/unmapping operation) but may not be as fast
as running with unmapped I/O.

The new memory handling model for user requests also allows
applications to send CCBs with request sizes that are larger than
MAXPHYS.  The pass(4) driver now limits queued requests to the I/O
size listed by the SIM driver in the maxio field in the Path
Inquiry (XPT_PATH_INQ) CCB.

There are some things things would be good to add:

1. Come up with a way to do unmapped I/O on multiple buffers.
   Currently the unmapped I/O interface operates on a struct bio,
   which includes only one address and length.  It would be nice
   to be able to send an unmapped scatter/gather list down to
   busdma.  This would allow eliminating the copy we currently do
   for data.

2. Add an ioctl to list currently outstanding CCBs in the various
   queues.

3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do
   that.

4. Test physical address support.  Virtual pointers and scatter
   gather lists have been tested, but I have not yet tested
   physical addresses or scatter/gather lists.

5. Investigate multiple queue support.  At the moment there is one
   queue of commands per pass(4) device.  If multiple processes
   open the device, they will submit I/O into the same queue and
   get events for the same completions.  This is probably the right
   model for most applications, but it is something that could be
   changed later on.

Also, add a new utility, camdd(8) that uses the asynchronous pass(4)
driver interface.

This utility is intended to be a basic data transfer/copy utility,
a simple benchmark utility, and an example of how to use the
asynchronous pass(4) interface.

It can copy data to and from pass(4) devices using any target queue
depth, starting offset and blocksize for the input and ouptut devices.
It currently only supports SCSI devices, but could be easily extended
to support ATA devices.

It can also copy data to and from regular files, block devices, tape
devices, pipes, stdin, and stdout.  It does not support queueing
multiple commands to any of those targets, since it uses the standard
read(2)/write(2)/writev(2)/readv(2) system calls.

The I/O is done by two threads, one for the reader and one for the
writer.  The reader thread sends completed read requests to the
writer thread in strictly sequential order, even if they complete
out of order.  That could be modified later on for random I/O patterns
or slightly out of order I/O.

camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from
the pass(4) driver and also to send request notifications internally.

For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR)
per CAM CCB on the reading side, and a scatter/gather list
(CAM_DATA_SG) on the writing side.  In addition to testing both
interfaces, this makes any potential reblocking of I/O easier.  No
data is copied between the reader and the writer, but rather the
reader's buffers are split into multiple I/O requests or combined
into a single I/O request depending on the input and output blocksize.

For the file I/O path, camdd(8) also uses a single buffer (read(2),
write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list
(readv(2), writev(2), preadv(2), pwritev(2)) on writes.

Things that would be nice to do for camdd(8) eventually:

1.  Add support for I/O pattern generation.  Patterns like all
    zeros, all ones, LBA-based patterns, random patterns, etc. Right
    Now you can always use /dev/zero, /dev/random, etc.

2.  Add support for a "sink" mode, so we do only reads with no
    writes.  Right now, you can use /dev/null.

3.  Add support for automatic queue depth probing, so that we can
    figure out the right queue depth on the input and output side
    for maximum throughput.  At the moment it defaults to 6.

4.  Add support for SATA device passthrough I/O.

5.  Add support for random LBAs and/or lengths on the input and
    output sides.

6.  Track average per-I/O latency and busy time.  The busy time
    and latency could also feed in to the automatic queue depth
    determination.

sys/cam/scsi/scsi_pass.h:
	Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue
	and fetch asynchronous CAM CCBs respectively.

	Although these ioctls do not have a declared argument, they
	both take a union ccb pointer.  If we declare a size here,
	the ioctl code in sys/kern/sys_generic.c will malloc and free
	a buffer for either the CCB or the CCB pointer (depending on
	how it is declared).  Since we have to keep a copy of the
	CCB (which is fairly large) anyway, having the ioctl malloc
	and free a CCB for each call is wasteful.

sys/cam/scsi/scsi_pass.c:
	Add asynchronous CCB support.

	Add two new ioctls, CAMIOQUEUE and CAMIOGET.

	CAMIOQUEUE adds a CCB to the incoming queue.  The CCB is
	executed immediately (and moved to the active queue) if it
	is an immediate CCB, but otherwise it will be executed
	in passstart() when a CCB is available from the transport layer.

	When CCBs are completed (because they are immediate or
	passdone() if they are queued), they are put on the done
	queue.

	If we get the final close on the device before all pending
	I/O is complete, all active I/O is moved to the abandoned
	queue and we increment the peripheral reference count so
	that the peripheral driver instance doesn't go away before
	all pending I/O is done.

	The new passcreatezone() function is called on the first
	call to the CAMIOQUEUE ioctl on a given device to allocate
	the UMA zones for I/O requests and S/G list buffers.  This
	may be good to move off to a taskqueue at some point.
	The new passmemsetup() function allocates memory and
	scatter/gather lists to hold the user's data, and copies
	in any data that needs to be written.  For virtual pointers
	(CAM_DATA_VADDR), the kernel buffer is malloced from the
	new pass(4) driver malloc bucket.  For virtual
	scatter/gather lists (CAM_DATA_SG), buffers are allocated
	from a new per-pass(9) UMA zone in MAXPHYS-sized chunks.
	Physical pointers are passed in unchanged.  We have support
	for up to 16 scatter/gather segments (for the user and
	kernel S/G lists) in the default struct pass_io_req, so
	requests with longer S/G lists require an extra kernel malloc.

	The new passcopysglist() function copies a user scatter/gather
	list to a kernel scatter/gather list.  The number of elements
	in each list may be different, but (obviously) the amount of data
	stored has to be identical.

	The new passmemdone() function copies data out for the
	CAM_DATA_VADDR and CAM_DATA_SG cases.

	The new passiocleanup() function restores data pointers in
	user CCBs and frees memory.

	Add new functions to support kqueue(2)/kevent(2):

	passreadfilt() tells kevent whether or not the done
	queue is empty.

	passkqfilter() adds a knote to our list.

	passreadfiltdetach() removes a knote from our list.

	Add a new function, passpoll(), for poll(2)/select(2)
	to use.

	Add devstat(9) support for the queued CCB path.

sys/cam/ata/ata_da.c:
	Add support for the BIO_VLIST bio type.

sys/cam/cam_ccb.h:
	Add a new enumeration for the xflags field in the CCB header.
	(This doesn't change the CCB header, just adds an enumeration to
	use.)

sys/cam/cam_xpt.c:
	Add a new function, xpt_setup_ccb_flags(), that allows specifying
	CCB flags.

sys/cam/cam_xpt.h:
	Add a prototype for xpt_setup_ccb_flags().

sys/cam/scsi/scsi_da.c:
	Add support for BIO_VLIST.

sys/dev/md/md.c:
	Add BIO_VLIST support to md(4).

sys/geom/geom_disk.c:
	Add BIO_VLIST support to the GEOM disk class.  Re-factor the I/O size
	limiting code in g_disk_start() a bit.

sys/kern/subr_bus_dma.c:
	Change _bus_dmamap_load_vlist() to take a starting offset and
	length.

	Add a new function, _bus_dmamap_load_pages(), that will load a list
	of physical pages starting at an offset.

	Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios.
	Allow unmapped I/O to start at an offset.

sys/kern/subr_uio.c:
	Add two new functions, physcopyin_vlist() and physcopyout_vlist().

sys/pc98/include/bus.h:
	Guard kernel-only parts of the pc98 machine/bus.h header with
	#ifdef _KERNEL.

	This allows userland programs to include <machine/bus.h> to get the
	definition of bus_addr_t and bus_size_t.

sys/sys/bio.h:
	Add a new bio flag, BIO_VLIST.

sys/sys/uio.h:
	Add prototypes for physcopyin_vlist() and physcopyout_vlist().

share/man/man4/pass.4:
	Document the CAMIOQUEUE and CAMIOGET ioctls.

usr.sbin/Makefile:
	Add camdd.

usr.sbin/camdd/Makefile:
	Add a makefile for camdd(8).

usr.sbin/camdd/camdd.8:
	Man page for camdd(8).

usr.sbin/camdd/camdd.c:
	The new camdd(8) utility.

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-12-03 20:54:55 +00:00
Alexander Motin
2d80af4f4a Fix panic when trying to sort unsupported command in OOA queue.
Handle unsupported commands as not conflicting/blocking.
2015-11-27 14:29:04 +00:00
Alexander Motin
6d2a1fbf23 Add API to obtain primary enclosure name and ID for /dev/sesX devices.
sesX device number may change between reboots, so to properly identify
the instance we need more data.  Name and ID reported here may mach ones
reported by SCSI device, but that is not really required by specs.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2015-11-21 10:22:01 +00:00
Alexander Motin
9f72f0ee52 Add NULL check to make Coverity happy. 2015-11-14 14:56:01 +00:00
Alexander Motin
6bd364b523 Modify target port groups logic in CTL.
- Introduce "ha_shared" port option, which being set to "on" moves the
port into separate port group, shared between HA nodes.  This allows to
better handle cases when iSCSI portals are bound to CARP address that can
dynamically move between nodes.  Some initiators (at least VMware) don't
detect that after iSCSI reconnect they've attached to different SCSI port
from different port group, that totally breakes ALUA status parsing.
In theory, I believe, it should be enough to have different iSCSI portal
group tags on different nodes to make initiators detect this condition,
but it seems like VMware ignores those values, and even full LUN retaste
forced by UA does not help.
 - Make CTL report up to three port groups: 1 -- non-HA mode or ports
with "ha_shared" option set, 2 -- HA node 1, 3 -- HA node 2.
 - Report Transitioning state for all port groups when HA interlink is
connected, but neither of nodes is primary for the LUN.

MFC after:	2 weeks
2015-11-11 13:18:38 +00:00
Alexander Motin
b94650a2bb Removed unused malloc types.
Submitted by:	Dmitry Luhtionov <dmitryluhtionov@gmail.com>
MFC after:	1 week
2015-11-06 18:50:01 +00:00
Alexander Motin
5b9676b17b Add two more KASSERTs. 2015-11-06 17:29:21 +00:00
Alexander Motin
c71150ce5d Extend mask of VMware virtual disks. 2015-11-05 09:07:53 +00:00
Alexander Motin
10643dd70e Don't try to replicate mode pages not present on this device.
MFC after:	3 days
2015-10-26 14:14:56 +00:00
Alexander Motin
59f063d549 Give CTL support for PIM_EXTLUNS when talking to CAM.
CTL itself still lives in flat LUN space, but it can generate extended
numbers if CAM SIM reports such capability.
2015-10-24 17:24:19 +00:00
Alexander Motin
c98d2b1f1e Add partial support for QUERY TMF to CAM and isp(4).
This change allows to decode respective functions in isp(4) in target mode
and pass them through CAM to CTL.  Unfortunately neither CAM nor isp(4)
support returning response info for those task management functions now.

On the other side I just have no initiator to test this functionality.
2015-10-23 18:34:18 +00:00
Alexander Motin
aeb1faa040 Fix LUN disable in CAM broken at r285155.
MFC after:	1 week
2015-10-23 10:39:43 +00:00
Alexander Motin
5124012aae Make some panic strings mode informative. 2015-10-21 15:31:26 +00:00
Alexander Motin
0ac03010f6 Make delete method set via kern.cam.da.X.delete_method persistent.
This allows to set delete method via tunable, before device capabilities
are known.  Also allow ZERO method for devices not reporting LBP, if user
explicitly requests it -- it may be useful if storage supports compression
and WRITE SAME, but does not support UNMAP.

MFC after:	2 weeks
2015-10-11 18:26:06 +00:00
Alexander Motin
6854699543 Remove legacy CHS geometry from dmesg and unify capacity outputs. 2015-10-11 13:48:20 +00:00
Alexander Motin
4a3760bae6 Remove compatibility shims for legacy ATA device names.
We got new ATA stack in FreeBSD 8.x, switched to it at 9.x, completely
removed old stack at 10.x, so at 11.x it is time to remove compat shims.
2015-10-11 13:01:51 +00:00
Alexander Motin
361e885315 Remove lock upgrade attempt from ctl_be_block_open_file().
I am not sure what for it was done.  Now open routine should automatically
fall back to read-only if open for writing is impossible.  In such case
attempt to upgrade to write sounds strange.

MFC after:	1 week
2015-10-11 08:28:49 +00:00
Alexander Motin
3d5cb709bd Add missing vnode lock in case of file modify request.
Submitted by:	Richard Kojedzinszky
MFC after:	1 week
2015-10-08 07:34:30 +00:00
Alexander Motin
0952a19f7d More aggressively fill WUT read pipeline.
On some tests I've measured 5% copy speedup from this.
2015-10-01 19:07:15 +00:00
Alexander Motin
6ac1446d0e Make zero WUT use WRITE SAME with recently allowed NDOB flag. 2015-10-01 16:30:20 +00:00
Alexander Motin
f24b1dc90c Implement SPC-3 exceptions to SPC-2 RESERVE and RELEASE behavior. 2015-10-01 12:57:37 +00:00
Alexander Motin
f53270c858 Unify PR variable names to reduce confusion. 2015-10-01 12:15:36 +00:00
Alexander Motin
f5a2bbe6d6 Use proper STAILQ_* macros where possible. 2015-09-30 20:38:35 +00:00
Alexander Motin
de2393124c Make pass, sg and targ drivers respect HBA's maxio.
Previous limitation of 64K (DFLTPHYS) is quite annoying.
2015-09-30 13:31:37 +00:00
Alexander Motin
66b6967686 Really implement PREVENT ALLOW MEDIUM REMOVAL command. 2015-09-29 15:12:40 +00:00
Alexander Motin
7f3d6f1fe3 Don't report SYNC_NV bit set in SYNCHRONIZE CACHE as error.
While this bit is obsolete in SBC-3, behavior controlled by it is allowed
on device discretion.
2015-09-29 13:58:27 +00:00
Alexander Motin
862aedb0d6 Fix arguments order. 2015-09-29 12:53:41 +00:00
Alexander Motin
6908da41fa Report that we can read all flavours of DVD. Why not? 2015-09-29 10:44:37 +00:00
Alexander Motin
d6e7f6e741 Add CD/DVD Capabilities and Mechanical Status Page.
This page is obsolete since MMC-4, but still used by some software.
2015-09-29 09:09:37 +00:00
Alexander Motin
648dfc1a29 Umplement media load/eject support for removable devices.
In case of block backend eject really closes the backing store, while
load tries to open it back.  Failed store open is reported as no media.
2015-09-28 20:54:18 +00:00
Alexander Motin
91be33dc78 Add to CTL initial support for CDROMs and removable devices.
Relnotes:	yes
2015-09-27 13:47:28 +00:00
Alexander Motin
2e33ae99cf Allow LOG SENSE command on non-disk devices. 2015-09-26 13:51:29 +00:00
Alexander Motin
0b33261bbe Remove concept of control device. 2015-09-26 12:53:55 +00:00
Alexander Motin
6bff2b5bff Move ioctl frontend defines where they belong. 2015-09-26 11:56:28 +00:00
Alexander Motin
d3ab449cfa Remove few more unused variables. 2015-09-26 11:39:54 +00:00
Alexander Motin
9c887a4f86 Remove some duplicate, legacy, dead and questionable code. 2015-09-26 11:28:45 +00:00
Alexander Motin
0bb9989c43 Pass to sobind() copy of the address, not the original. 2015-09-26 01:23:29 +00:00
Alexander Motin
86b7954b92 Properly lock LUN in ctl_failover_lun(). 2015-09-25 22:45:23 +00:00
Alexander Motin
f44570f617 Fix bit masks changed for the wrong commands in r288221. 2015-09-25 19:46:34 +00:00
Alexander Motin
5d4dfee032 Constify ctl_serialize_table. 2015-09-25 18:49:25 +00:00
Alexander Motin
c30a4c1871 Remove some dead code found by Clang analyzer. 2015-09-25 18:15:34 +00:00
Alexander Motin
67cc546dfc Remove stale comments and some excessive empty lines. 2015-09-25 16:34:59 +00:00
Alexander Motin
e675024a02 Switch I/O time accounting from system time to uptime.
While there, make num_dmas accounted independently of CTL_TIME_IO.
2015-09-25 10:14:39 +00:00
Alexander Motin
6c068d4bf3 Collect DMA statistics on secondary HA node. 2015-09-25 09:55:51 +00:00
Alexander Motin
116c5818ec Make HA handle datamove and done in a universal way, using port methods.
Now from primary node point of view requests transferred from secondary
node should look almost normal and always have valid port.
2015-09-25 09:14:29 +00:00
Alexander Motin
21d963e528 Remove some control_softc references. 2015-09-25 07:27:23 +00:00
Alexander Motin
6c2acea564 Allow WRITE SAME with NDOB bit set but without UNMAP.
This combination was originally forbidden, but allowed at spc4r3.
2015-09-24 15:59:08 +00:00
Alexander Motin
4ef0129a46 Add new report types to REPORT LUNS command.
This is only for completeness, since we have nothing new to report there.
2015-09-24 12:22:47 +00:00
Alexander Motin
a6daea64fd Update WRITE ATOMIC(16) support to sbc4r8 draft.
This is only a cosmetic change.  We still don't support atomic boundary
field in the CDB, but at least now we do it formally.
2015-09-24 08:04:47 +00:00
Alexander Motin
de988746be Add support for READ BUFFER(16) command. 2015-09-24 07:16:34 +00:00
Alexander Motin
ca85b7c4e9 Synchronize mode pages between HA peers.
We allow to modify only few fields in mode pages now, but still it is
not good if they unexpectedly change during failover.  Also this fixes
reporting of "Mode parameters changed" UAs on secondary node.
2015-09-23 18:33:00 +00:00
Alexander Motin
a85700a912 Make HA peers announce their parameters on connect.
HA protocol requires strict version, parameters and configuration match.
Differences there may cause full set of problems up to kernel panic.
To avoid that, validate peer parameters on connect, and abort connection
immediately if some mismatch detected.
2015-09-23 15:49:27 +00:00
Alexander Motin
c53993057b Add support for Control extension mode page. 2015-09-22 14:55:46 +00:00
Alexander Motin
4ce7a0868c Remove duplicate and incomplete code handling LUN modify.
Instead reuse code from LUN creation.  This allows most of LUN media
options to be changed live with modify request without full restart.
2015-09-22 10:45:50 +00:00
Alexander Motin
8153ec9ad5 Log iSCSI session reinstatements.
False session reinstatements can be result of misconfiguration, when
several initiators use the same initiator name and ISID.
2015-09-21 13:25:29 +00:00
Alexander Motin
218d25d00b Mark with DMA flag I/Os waiting for iSCSI write data after R2T.
Reads and immediate writes are not blocking, so don't bother.
2015-09-20 11:39:19 +00:00
Alexander Motin
b22213694e Remove couple excess SGLIST I/O flags.
Those flags duplicated respective (sg_entries > 0) values.
2015-09-20 10:40:30 +00:00
Alexander Motin
921af50782 Always execute REPORT LUNS and REQUEST SENSE commands locally.
REPORT LUNS command is more related to target rather then specific LUN.
This node may be primary for LUNs for some reason unknown to another,
and command forwarded to another node won't be able to report them.

REQUEST SENSE is related to LUN, but in our implementation it reports
only UAs and CAs, that are stored locally rather then on primary node.
2015-09-19 19:30:55 +00:00
Alexander Motin
efbf6139a4 Split two command flags with different meaning.
This is only a cosmetical change.
2015-09-19 19:11:59 +00:00
Alexander Motin
027dd0cfd0 Pack struct ctl_ha_msg_hdr by 8 bytes. 2015-09-19 18:23:06 +00:00
Alexander Motin
6861045bda Fix memory corruption when >128K transferred through HA link.
While there, do some cleaning.
2015-09-19 13:12:09 +00:00
Alexander Motin
bc8749e2ff Mark I/Os with DMA flag while moving data through the HA link. 2015-09-18 20:11:10 +00:00
Alexander Motin
75a3108e13 Relax serseq option operation for reads.
Previously, with serseq enabled, next command was unblocked only after
previous completed.  With this change, for read operations, next command
is unblocked as soon as last media read completed.  This is important
for frontends that actually wait for data move completion (like camtgt),
or when data are moved through the HA link, or especially when both.
2015-09-18 19:43:14 +00:00
Alexander Motin
59bb97a934 Kill HA link and shutdown the threads on shutdown. 2015-09-18 12:08:00 +00:00
Alexander Motin
1c69dbd098 Update list of opcodes to 5/26/15. 2015-09-18 10:44:25 +00:00
Alexander Motin
f90e68de18 Update list of ASC/ASCQ codes from 5/20/12 to 8/12/15. 2015-09-18 10:23:17 +00:00
Alexander Motin
0c05f0dc16 Replicate initiators WWPNs and names between HA peers. 2015-09-17 21:51:11 +00:00
Alexander Motin
54713bce25 Replicate port->init_devid to HA peer. 2015-09-17 20:36:34 +00:00
Alexander Motin
6213882769 When reporting TPT UA, report which of thresholds was reached. 2015-09-17 17:00:36 +00:00
Alexander Motin
042e9bdc41 Report number of failed XCOPY segment. 2015-09-17 14:22:52 +00:00
Alexander Motin
7f7bb97a0f Report proper medium error code for VERIFY commands. 2015-09-17 12:52:18 +00:00
Alexander Motin
83981e319d Fix reading after end of file for file-backed LUNs.
If backing file is smaller then the LUN size, we have to explicitly clear
the rest of the buffer to not leak some random data from previous I/Os.
2015-09-16 21:43:51 +00:00
Alexander Motin
d6043e4643 Make COMPARE AND WRITE report offset of difference. 2015-09-16 18:33:04 +00:00
Alexander Motin
723c363f7f Fix fixed sense writing when passed more data then it can fit.
MFC after:	1 week
2015-09-16 17:56:24 +00:00
Alexander Motin
5c5fb901eb Frontends don't need to set errors themselves. 2015-09-16 13:25:35 +00:00
Alexander Motin
e2c3044b13 Don't flap the HA link if sysctl is reset to the same value. 2015-09-16 09:59:05 +00:00
Alexander Motin
c39d464164 Make CAM log errors that make it wait.
Waiting can take minutes, and it would be good for user to know what is
going on.

MFC after:	2 weeks
2015-09-15 10:57:16 +00:00
Alexander Motin
18e87f95f7 Fix completion/error status reporting. 2015-09-15 10:42:53 +00:00
Alexander Motin
923863546a Close potential race between datamove and HA failover. 2015-09-15 09:36:46 +00:00
Mark Johnston
87dd1668e0 Preserve the device queue status before retrying a sense request in
chdone(). Previously, the retry could clear the CAM_DEV_QFRZN bit in the
CCB status, leaving the queue frozen.

Submitted by:	Jeff Miller <Jeff.Miller@isilon.com>
Reviewed by:	ken
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-09-15 05:09:17 +00:00
Alexander Motin
d0f25804a7 Implement data/status aggregation for secondary HA node.
For short read requests this reduces latency by 30%, reporting command
completion after two interlink roundtrips instead of full three.
2015-09-14 15:21:23 +00:00
Alexander Motin
3ed019a919 Implement data/status aggregation for camsim backend.
This is almost pointless for primary HA node, but useful for secondary,
where between fe_datamove and fe_done calls goes another link roundtrip.
2015-09-14 15:15:32 +00:00
Alexander Motin
7b6660c9aa Remove CTL_PRIV_LBA_LEN from HA messages.
Previously it was used for statistics, but now just a 16 extra bytes.
2015-09-14 09:56:01 +00:00
Alexander Motin
ceff31dc0c Implement QUERY TASK, QUERY TASK SET and QUERY ASYNC EVENT.
Now we support most of SAM-5 task management.
2015-09-14 08:01:05 +00:00
Alexander Motin
bb978da493 Map CLEAR TASK SET and I_T NEXUS RESET for iSCSI.
The last should not be called without iSCSIProtocolLevel negotiation.
2015-09-13 20:40:00 +00:00
Alexander Motin
4cbc0bc5ae Implement iSCSI TARGET COLD RESET task management function.
Implement it as CTL_TASK_TARGET_RESET plus termination of all sessions.
2015-09-13 20:22:18 +00:00
Alexander Motin
6187d4722a Improve read-only support. 2015-09-13 16:49:41 +00:00
Alexander Motin
6b56dd0cfa Reannounce port to HA peer if LUN map changed after online. 2015-09-13 15:31:55 +00:00
Alexander Motin
cfa0987386 Report INQUIRY DATA HAS CHANGED for related LUNs on port on-/offline. 2015-09-13 14:52:31 +00:00
Alexander Motin
eebde34629 Report CTL_UA_LUN_CHANGE on LUN map change. 2015-09-13 14:04:42 +00:00
Alexander Motin
844c2a3a94 Make TARGET RESET respect LUN mapping. 2015-09-13 13:00:20 +00:00
Alexander Motin
84e2fad15a Check for obsolete NUL bin in CSCD descriptor. 2015-09-12 20:45:09 +00:00
Alexander Motin
051481a24e Add HA support for CTL_TASK_I_T_NEXUS_RESET. 2015-09-12 20:06:22 +00:00
Alexander Motin
0a4c638188 Fix false CTL_UA_RES_RELEASE on secondary HA node. 2015-09-12 18:29:05 +00:00
Alexander Motin
119c9aca64 Decode WRITE ATOMIC(16) command. 2015-09-12 17:53:49 +00:00
Alexander Motin
a65a997fd9 Improve XCOPY error reporting. 2015-09-12 16:30:01 +00:00
Alexander Motin
238b6b7c75 Report that we have no limit on POPULATE TOKEN segment size. 2015-09-12 14:20:11 +00:00
Alexander Motin
eed0d596cf Correct RTPG bitmask. 2015-09-12 13:53:41 +00:00
Alexander Motin
e184d2576a Some HA polishing. 2015-09-12 12:46:04 +00:00
Alexander Motin
d36c617616 CTL documentation update, mostly for HA. 2015-09-12 10:23:23 +00:00
Alexander Motin
572846e089 Make ctld restart on primary HA node less invasive for secondary. 2015-09-11 14:47:35 +00:00
Alexander Motin
ee4ad294d2 Close races between device close and request processing.
All requests arriving for processing after OFFLINE flag set are rejected
with BUSY status.  Races around OFFLINE flag setting are closed by calling
taskqueue_drain_all().
2015-09-11 14:33:05 +00:00
Alexander Motin
3236151ea8 Reference/release devices on every I/O, rather on open/close.
While this may be slower, it allows device destruction to complete,
rather then block waiting for indefinitely long time.
2015-09-11 12:50:52 +00:00
Alexander Motin
7ac58230ea Reimplement CTL High Availability.
CTL HA functionality was originally implemented by Copan many years ago,
but large part of the sources was never published.  This change includes
clean room implementation of the missing code and fixes for many bugs.

This code supports dual-node HA with ALUA in four modes:
 - Active/Unavailable without interlink between nodes;
 - Active/Standby with second node handling only basic LUN discovery and
reservation, synchronizing with the first node through the interlink;
 - Active/Active with both nodes processing commands and accessing the
backing storage, synchronizing with the first node through the interlink;
 - Active/Active with second node working as proxy, transfering all
commands to the first node for execution through the interlink.

Unlike original Copan's implementation, depending on specific hardware,
this code uses simple custom TCP-based protocol for interlink.  It has
no authentication, so it should never be enabled on public interfaces.

The code may still need some polishing, but generally it is functional.

Relnotes:	yes
Sponsored by:	iXsystems, Inc.
2015-09-10 12:40:31 +00:00
Alexander Motin
fb606ebabc Remove unused target and initiator IDs. 2015-09-10 10:46:21 +00:00
Alexander Motin
3f326305fc Disable CTL_IO_DELAY feature.
It is too developer-oriented to be enabled by default.
2015-09-10 09:27:22 +00:00
Alexander Motin
a3977bea20 Allow LUN options modification via CTL_LUNREQ_MODIFY.
Not all changes take effect, but that is a different question.
2015-09-06 11:23:01 +00:00
Alexander Motin
0bcd4ab6ba Move setting of media parameters inside open routines.
This is preparation for possibility to open/close media several times
per LUN life cycle.  While there, rename variables to reduce confusion.
As additional bonus this allows to open read-only media, such as ZFS
snapshots.
2015-09-06 09:54:56 +00:00
Alexander Motin
bd236ba5c0 Remove some dead code. 2015-09-04 09:19:01 +00:00
Alexander Motin
420abfcecd Small UA cleanup. 2015-09-03 12:56:57 +00:00
Alexander Motin
8be6a2df51 Fix copy-paste bug introduced in r275458.
MFC after:	3 days
2015-09-03 12:15:14 +00:00
Alexander Motin
1b7849d0f2 Make most of port methods optional and remove bunch of dummies. 2015-09-01 16:28:06 +00:00
Alexander Motin
7278725be5 Remove 600 bytes of port_priv from struct ctl_io_hdr.
This field used only for camtgt frontend, and once it any way preallocates
all requests, let it preallocate this memory too, not bothering core code.
2015-08-29 15:33:31 +00:00
Alexander Motin
9202485814 Attach pass driver to LUNs is OFFLINE state.
Previously such LUNs were silently ignored.  But while they indeed unable
to process most of SCSI commands, some, like RTPG, they still can.

MFC after:	1 month
2015-08-29 11:21:20 +00:00
Alexander Motin
f6295033c1 Fix type bug introduced at r286811. 2015-08-27 21:16:24 +00:00
Alexander Motin
4beec13537 Remove some code duplication by using biofinish().
Submitted by:	imp
MFC after:	1 week
2015-08-22 15:58:35 +00:00
Kenneth D. Merry
0e358df062 Revamp camcontrol(8) fwdownload support and add the opcodes subcommand.
The significant changes and bugs fixed here are:

1. Fixed a bug in the progress display code:

   When the user's filename is too big, or his terminal width is too
   small, the progress code could wind up using a negative number for
   the length of the "stars" that it uses to indicate progress.

   This negative value was assigned to an unsigned variable, resulting
   in a very large positive value.

   The result is that we wound up writing garbage from memory to the
   user's terminal.

   With an 80 column terminal, a file name length of more than 35
   characters would generate this problem.

   To address this, we now set a minimum progress bar length, and
   truncate the user's file name as needed.

   This has been tested with large filenames and small terminals, and
   at least produces reasonable results.  If the terminal is too
   narrow, the progress display takes up an additional line with each
   update, but this is more user friendly than writing garbage to the
   tty.

2. SATA drives connected via a SATA controller didn't have SCSI Inquiry
   data populated in struct cam_device.  This meant that the code in
   fw_get_vendor() in fwdownload.c would try to match a zero-length
   vendor ID, and so return the first entry in the vendor table.  (Which
   used to be HITACHI.)  Fixed by grabbing identify data, passing the
   identify buffer into fw_get_vendor(), and matching against the model
   name.

3. SATA drives connected via a SAS controller do have Inquiry data
   populated.  The table included a couple of entries -- "ATA ST" and
   "ATA HDS", intended to handle Seagate and Hitachi SATA drives attached
   via a SAS controller.  SCSI to ATA translation layers use a vendor
   ID of "ATA" (which is standard), and then the model name from the ATA
   identify data as the SCSI product name when they are returning data on
   SATA disks.  The cam_strmatch code will match the first part of the
   string (because the length it is given is the length of the vendor,
   "ATA"), and return 0 (i.e. a match).  So all SATA drives attached to
   a SAS controller would be programmed using the Seagate method
   (WRITE BUFFER mode 7) of SCSI firmware downloading.

4. Issue  above covered up a bug in fw_download_img() -- if the
   maximum packet size in the vendor table was 0, it tried to default
   to a packet size of 32K.  But then it didn't actually succeed in
   doing that, because it set the packet size to the value that was
   in the vendor table (0).  Now that we actually have ATA attached
   drives fall use the VENDOR_ATA case, we need a reasonable default
   packet size.  So this is fixed to properly set the default packet size.

5. Add support for downloading firmware to IBM LTO drives, and add a
   firmware file validation method to make sure that the firmware
   file matches the drive type.  IBM tape drives include a Load ID and
   RU name in their vendor-specific VPD page 0x3.  Those should match
   the IDs in the header of the firmware file to insure that the
   proper firmware file is loaded.

6. This also adds a new -q option to the camcontrol fwdownload
   subcommand to suppress informational output.  When -q is used in
   combination with -y, the firmware upgrade will happen without
   prompting and without output except if an error condition occurs.

7. Re-add support for printing out SCSI inquiry information when
   asking the user to confirm that they want to download firmware, and
   add printing of ATA Identify data if it is a SATA disk.  This was
   removed in r237281 when support for flashing ATA disks was added.

8. Add a new camcontrol(8) "opcodes" subcommand, and use the
   underlying code to get recommended timeout values for drive
   firmware downloads.

   Many SCSI devices support the REPORT SUPPORTED OPERATION CODES
   command, and some support the optional timeout descriptor that
   specifies nominal and recommended timeouts for the commands
   supported by the device.

   The new camcontrol opcodes subcommand allows displaying all
   opcodes supported by a drive, information about which fields
   in a SCSI CDB are actually used by a given SCSI device, and the
   nominal and recommended timeout values for each command.

   Since firmware downloads can take a long time in some devices, and
   the time varies greatly between different types of devices, take
   advantage of the infrastructure used by the camcontrol opcodes
   subcommand to determine the best timeout to use for the WRITE
   BUFFER command in SCSI device firmware downloads.

   If the device recommends a timeout, it is likely to be more
   accurate than the default 50 second timeout used by the firmware
   download code.  If the user specifies a timeout, it will override
   the default or device recommended timeout.  If the device doesn't
   support timeout descriptors, we fall back to the default.

9. Instead of downloading firmware to SATA drives behind a SAS controller
   using WRITE BUFFER, use the SCSI ATA PASS-THROUGH command to compose
   an ATA DOWNLOAD MICROCODE command and it to the drive.  The previous
   version of this code attempted to send a SCSI WRITE BUFFER command to
   SATA drives behind a SAS controller.  Although that is part of the
   SAT-3 spec, it doesn't work with the parameters used with LSI
   controllers at least.

10.Add a new mechanism for making common ATA passthrough and
   ATA-behind-SCSI passthrough commands.

   The existing camcontrol(8) ATA command mechanism checks the device
   type on every command executed.  That works fine for individual
   commands, but is cumbersome for things like a firmware download
   that send a number of commands.

   The fwdownload code detects the device type up front, and then
   sends the appropriate commands.

11.In simulation mode (-s), if the user specifies the -v flag, print out
   the SCSI CDB or ATA registers that would be sent to the drive.  This will
   aid in debugging any firmware download issues.

sbin/camcontrol/fwdownload.c:
	Add a device type to the fw_vendor structure, so that we can
	specify different download methods for different devices from the
	same vendor.  In this case, IBM hard drives (from when they
	still made hard drives) and tape drives.

	Add a tur_status field to the fw_vendor structure so that we can
	specify whether the drive to be upgraded should be ready, not
	ready, or whether it doesn't matter.  Add the corresponding
	capability in fw_download_img().

	Add comments describing each of the vendor table fields.

	Add HGST and SmrtStor to the supported SCSI vendors list.

	In fw_get_vendor(), look at ATA identify data if we have a SATA
	device to try to identify what the drive vendor is.

	Add IBM firmware file validation.  This gets VPD page 0x3, and
	compares the Load ID and RU name in the page to the values
	included in the header.  The validation code will refuse to load
	a firmware file if the values don't match.  This does allow the
	user to attempt a downgrade; whether or not it succeeds will
	likely depend on the drive settings.

	Add a -q option, and disable all informative output
	(progress bars, etc.) when this is enabled.

	Re-add the inquiry in the confirmation dialog so the user has
	a better idea of which device he is talking to.  Add support for
	displaying ATA identify data.

	Don't automatically disable confirmation in simulation (-s) mode.
	This allows the user to see the inquiry or identify data in the
	dialog, and see exactly what they would see when the command
	actually runs.  Also, in simulation mode, if the user specifies
	the -v flag, print out the SCSI CDB or ATA registers that would
	be sent to the drive.  This will aid in debugging any firmware
	download issues.

	Add a timeout field and timeout type to the firmware download
	vendor table.  This allows specifying a default timeout and allows
	specifying whether we should attempt to probe for a recommended
	timeout from the drive.

	Add a new fuction, fw_get_timeout(), that will determine
	which timeout to use for the WRITE BUFFER command.  If the
	user specifies a timeout, we always use that.  Otherwise,
	we will use the drive recommended timeout, if available,
	and fall back to the default when a drive recommended
	timeout isn't available.

	When we prompt the user, tell him what timeout we're going
	to use, and the source of the timeout.

	Revamp the way SATA devices are handled.

	In fwdownload(), use the new get_device_type() function to
	determine what kind of device we're talking to.

	Allow firmware downloads to any SATA device, but restrict
	SCSI downloads to known devices.  (The latter is not a
	change in behavior.)

	Break out the "ready" check from fw_download_img() into a
	new subfunction, fw_check_device_ready().  This sends the
	appropriate command to the device in question -- a TEST
	UNIT READY or an IDENTIFY.  The IDENTIFY for SATA devices
 	a SAT layer is done using the SCSI ATA PASS-THROUGH
	command.

	Use the new build_ata_cmd() function to build either a SCSI or
	ATA I/O CCB to issue the DOWNLOAD MICROCODE command to SATA
	devices.  build_ata_cmd() figures looks at the devtype argument
	and fills in the correct CCB type and CDB or ATA registers.

	Revamp the vendor table to remove the previous
	vendor-specific ATA entries and use a generic ATA vendor
	placeholder.  We currently use the same method for all ATA
	drives, although we may have to add vendor-specific
	behavior once we test this with more drives.

sbin/camcontrol/progress.c:
	In progress_draw(), make barlength a signed value so that
	we can easily detect a negative value.

	If barlength (the length of the progress bar) would wind up
	negative due to a small TTY width or a large filename,
	set the bar length to the new minimum (10 stars) and
	truncate the user's filename.  We will truncate it down to
	0 characters if necessary.

	Calculate a new prefix_len variable (user's filename length)
	and use it as the precision when printing the filename.

sbin/camcontrol/camcontrol.c:
	Implement a new camcontrol(8) subcommand, "opcodes".  The
	opcodes subcommand allows displaying the entire list of
	SCSI commands supported by a device, or details on an
	individual command.  In either case, it can display
	nominal and recommended timeout values.

	Add the scsiopcodes() function, which calls the new
	scsigetopcodes() function to fetch opcode data from a
	drive.

	Add two new functions, scsiprintoneopcode() and
	scsiprintopcodes(), which print information about one
	opcode or all opcodes, respectively.

	Remove the get_disk_type() function.  It is no longer used.

	Add a new function, dev_has_vpd_page(), that fetches the
	supported INQUIRY VPD list from a device and tells the
	caller whether the requested VPD page is available.

	Add a new function, get_device_type(), that returns a more
	precise device type than the old get_disk_type() function.
	The get_disk_type() function only distinguished between
	SCSI and ATA devices, and SATA devices behind a SCSI to ATA
	translation layer were considered to be "SCSI".

	get_device_type() offers a third type, CC_DT_ATA_BEHIND_SCSI.
	We need to know this to know whether to attempt to send ATA
	passthrough commands.  If the device has the ATA
	Information VPD page (0x89), then it is an ATA device
	behind a SCSI to ATA translation layer.

	Remove the type argument from the fwdownload() subcommand.

	Add a new function, build_ata_cmd(), that will take one set
	of common arguments and build either a SCSI or ATA I/O CCB,
	depending on the device type passed in.

sbin/camcontrol/camcontrol.h:
	Add a prototype for scsigetopcodes().

	Add a new enumeration, camcontrol_devtype.

	Add prototypes for dev_has_vpd_page(), get_device_type()
	and build_ata_cmd().

	Remove the type argument from the fwdownload() subcommand.

sbin/camcontrol/camcontrol.8
	Explain that the fwdownload subcommand will use the drive
	recommended timeout if available, and that the user can
	override the timeout.

	Document the new opcodes subcommand.

	Explain that we will attempt to download firmware to any
	SATA device.

	Document supported SCSI vendors, and models tested if known.

	Explain the commands used to download firmware for the
	three different drive and controller combinations.

	Document that the -v flag in simulation mode for the fwdownload
	subcommand will print out the SCSI CDBs or ATA registers that would
	be used.

sys/cam/scsi/scsi_all.h:
	Add new bit definitions for the one opcode descriptor for
	the REPORT SUPPORTED OPCODES command.

	Add a function prototype for scsi_report_supported_opcodes().

sys/cam/scsi/scsi_all.c:
	Add a new CDB building function, scsi_report_supported_opcodes().

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-08-20 16:07:51 +00:00
Alexander Motin
a15bbf1508 Polish sizes processing. 2015-08-15 18:22:16 +00:00
Alexander Motin
67ceb24bca Move "ioctl" CAM frontend into separate file.
It has nothing to share with too huge ctl.c other then device descriptor,
but even that may be counted as design error that may be fixed later.
At some point we may even want to have several ioctl ports.
2015-08-15 15:42:21 +00:00
Alexander Motin
2f444d157b Drop "internal" CTL frontend.
Its idea was to be a simple initiator and execute several commands from
kernel level, but FreeBSD never had consumer for that functionality,
while its implementation polluted many unrelated places..
2015-08-15 13:34:38 +00:00
Alexander Motin
4ddb055594 Remove verbose CTL messages.
Reporting SCSI errors to console is often useless, pollutes logs and may
affect performance.  For debugging there is kern.cam.ctl.debug sysctl

MFC after:	1 week
2015-08-09 09:54:29 +00:00