Before this ioctl frontend always replaced tags with sequential ones.
It was done for ctladm, that can not keep track of global tag list.
But in case of virtio-scsi in bhyve we can pass provided tags as-is.
It should be on virtio-scsi initiator to provide us valid tags. It
should allow proper task management, error reporting, etc. In case
of several virtio-scsi devices, they should use different CTL ports
or initiator IDs to avoid conflicts, but this is expected by design.
PR: 267539
SAM-5 specification states maximum size of command identifier (tag),
defined by specific transports, should not be larger than 64 bits.
While most of supported transports use 32 bits or less, it was
reported that virtio-scsi uses 64 bits. Truncation to 32 bits in
bhyve code caused false tag conflict errors reported and possibly
other issues.
This changes CTL ABI and HA protocol, so CTL_HA_VERSION is bumped.
While we make HA protocol incompatible, increase default maximum
number of ports in CTL from 256 to 1024, matching number of LUNs.
There are many reports from people who need many iSCSI targets with
only one LUN each. Increased memory consumption should be less of
a problem these days.
PR: 267539
For CAM, move to the FreeBSD standard copyright rather than the 'put it
at the front' variation. This variaiton has been flagged as potentially
problematic in other contexts. Since this variation wasn't a conscious
decision on our part, use the standard license from src/COPYRIGHT.
Also, remove the -FreeBSD suffix in SPDX-License-Identifier. It's
obsolete at SPDX and even the original text didn't match it.
MFC After: 3 days
Sponsored by: Netflix
The CAM version bump 0x19 -> 0x1a changed the CAMGETPASSTHRU definition,
so applications using the old ioctl are broken. However, that version
change did not affect anything relating to the ioctl implementation for
periphs.
Fixes: 8f9be1eed1 ("cam(4): Improve XPT_DEV_MATCH")
PR: 264709
Tested by: andreas.mahling@googlemail.com
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36389
It was possible to cause kernel panic by passing too large args_len
or non-NULL result_nvl.
Though since the /dev/cam/ctl device is accessible only by root and
used only by limited number of tools it was not a big problem.
PR: 266115
PR: 266136
Reported by: Robert Morris <rtm@lcs.mit.edu>
MFC after: 1 week
Before this patch CAM periph drivers called both disk_alloc() and
disk_create() same time on periph creation. But then prevented disks
from opening until the periph probe completion with cam_periph_hold().
As result, especially if disk misbehaves during the probe, GEOM event
thread, triggered to taste the disk, got blocked on open attempt,
potentially for a long time, unable to process other events.
This patch moves disk_create() call from periph creation to the end of
the probe. To allow disk_create() calls from non-sleepable CAM contexts
some of its duties requiring memory allocations are moved either back
to disk_alloc() or forward to g_disk_create(), so now disk_alloc() and
disk_add_alias() are the only disk methods that require sleeping. If
disk fails during the probe disk_create() may just be skipped, going
directly to disk_destroy(). Other method calls during that time are
just ignored. Since GEOM may now see the disks after CAM bus scan is
already completed, introduce per-periph boot hold functions. Enclosure
driver already had such mechanism, so just generalize it.
Reviewed by: imp
MFC after: 1 month
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D35784
SES allows element descriptors to contain characters like spaces and
quotes that devfs does not allow to appear in device aliases. Since SES
element descriptors are outside of the kernel's control, we should
gracefully handle a failure to create a device physical path alias.
PR: 264513
Reported by: Yuri <yuri@aetern.org>
Reviewed by: imp, mav
Sponsored by: Axcient
MFC after: 2 weeks
I've found couple cases when CTL_FLAG_SENT_2OTHER_SC flags were not
cleared on commands return from active node or the send failure. It
created races when ctl_failover_lun() call before ctl_process_done()
could cause second ctl_done() and ctl_process_done() calls, causing
all sorts of problems.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
The physical address argument is essentially ignored by every dumper
method. In addition, the dump routines don't actually pass a real
address; every call to dump_append() passes a value of zero for
physical.
Reviewed by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35173
This printf was designed to catch misqueued bio requests. Prior to
supporting read_bias == 0, we couldn't get anything but reads and writes
in this queue. However, for read_bias == 0 we queue everything except
BIO_DELETE to this queue, so remove the printf. We don't need to update
any statistics.
Sponsored by: Netflix
In the places where we set an integer to 0 or 1 and then use it like a
boolean, replace int with bool and 0/1 with false/true. Left alone
places where this is a function argument or return value. No functional
changes intended.
Sponsored by: Netflix
Allow a global setting for the read_bias for the dynamic io
scheduler. This allows global policy to be set, in addition to the
existing per-drive policy. kern.cam.iosched.read_bias is a new tunable.
Sponsored by: Netflix
Reviewed by: chs
Differential Revision: https://reviews.freebsd.org/D34365
When we're doing dynamic I/O scheduling, don't default to a read bias of
100. Default it to 0 so turning on dynamic scheduling only does
scheduling tweaks that are requested. The other limiters are off by
default, and need no further adjustment.
Sponsored by: Netflix
Change the meaning of read bias == 0 in the dynamic I/O scheduler. Prior
to this change, a read bias of 0 would mean prefer writes. Now, when
read bias is 0, we queue all requests to the same queue removing the
bias. When it's non-zero, we still separate the queues we use so we can
bias reads vs writes for workloads that are read centric. These changes
restore the typical bias you get from disksort or ordered insertion at
the end of the list.
Sponsored by: Netflix
The AHCI and ATA SIMs will return CAM_SEL_TIMEOUT when an underlying
device has stopped responding. This is usually seen after a timeouted
out command and can be a transient event. Rather than fail the
peripheral immediately after seeing this, queue a retry. For transient
events, this allows drives to continue to provide data, though with some
added latency, just like we do when we have some other kind of retriable
error. If the error isn't transient (the drive is truly gone), then
we'll discover that eventually and fail the transaction and invalidate
the drive like we do today.
This helps us avoid a panic at the end of camperiphfree when
CAM_PERIPH_NEW_DEV_FOUND is set. However, the deferred callback should
be queued to xpt_async_td instead of being made inline there. This issue
will be solved in a different patch that does that. PR 263703.
This also helps us avoid another bug where we can drop all references to
the device (causing us to go through camperiphfree and destroy the path)
while we have an I/O pending in the ata_da state machine (usually in
state ADA_STATE_RAHEAD with ATA_SETFEATURES ATA_SF_ENAB_RCACHE
command). It's not clear why the reference that we take out to do the
reprobe isn't effective at blocking this. By retrying this condition,
though we avoid this bug (at least more often, I don't have a good
reproduction test case, I just see this panic a few times a month at
work on systems that have transient disk errors on ahci connected SATA
SSDs). PR 263704. It's too soon to know how much this helps us avoid
this bug.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D34977
When the peripheral is invalidated, no further I/O is possible. Signal
this up the stack with ENXIO now that upper layers of the stack
differentiate sometimes. In order for there to be further I/O, and new
open is required for any block device that a future periph might
instantiate for devices at this location that might return or otherwise
become available. The I/O scheduler flushes its I/O with the ENXIO error
for pending I/O that didn't make it to the device, so this makes the two
paths match.
MFC After: 3 days
Sponsored by: Netflix
Reviewed by: chs, mav
Differential Revision: https://reviews.freebsd.org/D35093
While recovery command executed, some other commands from the periph
may complete, that may affect periph_links of this CCB. So restoring
original CCB we must keep current periph_links as more up to date.
I've found this triggering assertions with debug kernel and suspect
some memory corruptions otherwise when spun down disk receives two
or sometimes more concurrent requests.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
We should call cam_periph_async() always, like SCSI does. This routine
is supposed to be more of a catch-all.
cam_periph_async() only does actions for AC_LOST_DEVICE. It ignores all
other events (today), but this may not always be true. So this is a nop
change.
Drop in a 'break' so we don't fall through unnecessarily.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D35057
We never use the cgd that we get from the XPT_GDEV_TYPE call. Prior to
9a6844d55f we used it to determine if READ AHEAD or WRITE CACHING was
supported. However, all that information was moved into adasetflags so
we no longer need to this since it's cached in the softc and updated
with the IDENTIFY data changes automatically.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D35039
Use a switch rather than a nested if to simplify the async event
processing code. No functional changes intended.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D35038
Fix a comment that was left over from the orignial
implementation. Explain how pending transactions in hardware are
completed/aborted in the SIM prior to ndacleanup being called.
Sponsored by: Netflix
cxgbei needs the ability to return different limits based on the
connection (e.g. if the connection is over a T5 adapter or a T6
adapter as well as factoring in the MTU).
This change plumbs through the changes in the ioctls without changing
any of the backends. The limits callback passed to icl_register now
accepts a second socket argument which holds the integer file
descriptor. To support ABI compatiblity for old binaries, the
callback should return "global" values if the socket fd is zero.
The CTL_ISCSI_LIMITS argument used with CTL_ISCSI by ctld(8) now
accepts the socket fd in a field that was previously part of a
reserved spare field. Old binaries zero this request which results in
passing a socket fd of 0 to the limits callback.
The ISCSIDREQUEST ioctl no longer returns limits. Instead, iscsid(8)
invokes a new ISCSIDLIMITS ioctl after establishing the connection via
connect(2). For ABI compat, if the old ISCSIDREQUEST is invoked, the
global limits are still fetched (with a socket fd of 0) and returned.
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34928
This should fix uninitialized memory reads when working with broken
HA peer, like one fixed in 1a8d8a3a90. Instead print error message
and kill the HA link.
MFC after: 3 days
Sponsored by: iXsystems, Inc.
For a compare request, the ramdisk backend allocates a temporary
buffer to hold the I/O data and then compares it against the LUN's
pages in ctl_backend_ramdisk_cmp after the data has been filled.
However, the tempory buffer was leaked when after the comparison was
complete. Fix this by freeing the buffer after the comparison.
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34316
The NVMe 1.4 spec simply says that Model and Serial numbers are
ASCII strings. Unlike SCSI, it doesn't prohibit non-printable
characters or say that the strings should be padded with spaces.
Since 2014, we have had cam_strvis_sbuf(), which gives additional
options for handling non-ASCII characters. That behavior hasn't
been available for non-sbuf consumers, so users of cam_strvis()
were left with having octal ASCII codes inserted.
So, to avoid having garbage or octal chracters in the strings, use
cam_strvis_sbuf() to create a new function, cam_strvis_flag(), and
re-implement cam_strvis() using cam_strvis_flag().
Now, for the NVMe drives, we can use cam_strvis_flag with the
CAM_STRVIS_FLAG_NONASCII_SPC flag. This transforms non-printable
characters into spaces.
sys/cam/cam.c:
Add a new function, cam_strvis_flag(), that creates an sbuf
on the stack with the user's destination buffer, and calls
cam_strvis_sbuf() with the given flag argument.
Re-implement cam_strvis() to call cam_strvis_flag with the
CAM_STRVIS_FLAG_NONASCII_ESC argument. This should be the
equivalent of the old cam_strvis() function, except for the
overhead of creating the sbuf and calling sbuf_putc/printf.
sys/cam/cam.h:
Declaration for cam_strvis_flag.
sys/cam/nvme/nvme_all.c:
In nvme_print_ident, use the NONASCII_SPC flag with
cam_strvis_flag().
sys/cam/nvme/nvme_da.c:
In ndaregister(), use cam_strvis_flag() with the
NONASCII_SPC flag for the disk description and serial
number we report to GEOM.
sys/cam/nvme/nvme_xpt.c:
In nvme_probe_done(), use cam_strvis_flag with the
NONASCII_SPC flag when storing the drive serial number
in the CAM EDT.
MFC after: 1 week
Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D33973
The dummy PDU needs to be freed before marking task abortion complete
as otherwise cfiscsi_session_terminate_tasks can return and destroy
the session in another thread before the PDU is freed.
Fixes: 2e8d1a5525 iscsi: Allocate a dummy PDU for the internal nexus reset task.
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34176
Due to variable size of struct ctl_ha_msg_mode ctl_isc_announce_mode()
sent only first 4 bytes of modified mode page to the other HA side,
that caused its corruption there, noticeable only after failover.
I've found alike bug also in ctl_isc_announce_lun(), but there it was
sending slightly more than needed, that is a smaller problem.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
When an iSCSI target session is terminated, an internal nexus reset
task is posted to abort existing tasks belonging to the session.
Previously, the ctl_io for this internal nexus reset stored a pointer
to the session in the slot that normally holds a pointer to the PDU
from the initiator that triggered the I/O request. The completion
handler then assumed that any nexus reset I/O was due to an internal
request and fetched the session pointer (instead of the PDU pointer)
from the ctl_io. However, it is possible to trigger a nexus reset via
an on-the-wire task management PDU. If such a PDU were sent to the
target, then the completion handler would incorrectly treat this
request as an internal request and treat the pointer to the received
PDU as a pointer to the session instead.
To fix, allocate a dummy PDU for the internal reset task and use an
invalid opcode to differentiate internal nexus resets from resets
requested by the initiator.
PR: 260449
Reported by: Robert Morris <rtm@lcs.mit.edu>
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34055
It does not look like there is anything in mmc_da code that actually
requires protocol 2.0 or later. dev/mmc code also does not have such a
restriction.
Tested with a very old 2GB mini-SD card. Prior to this change mmc_da
would claim the card but would not expose it to GEOM.
Without MMCCAM:
mmc0: <MMC/SD bus> on sdhci_pci0
mmc0: Probing bus
mmc0: SD probe: OK (OCR: 0x00ff8000)
mmc0: Current OCR: 0x00ff8000
mmc0: CMD8 failed, RESULT: 1
mmc0: Probing cards
mmc0: New card detected (CID 1c53565344432020100002982e007600)
mmc0: New card detected (CSD 005e00325f5a83d02db7ffbf96800000)
mmc0: Card at relative address 0xb368 added:
mmc0: card: SD SDC 1.0 SN 0002982E MFG 06/2007 by 28 SV
mmc0: quirks: 0
mmc0: bus: 4bit, 50MHz (high speed timing)
mmc0: memory: 3998720 blocks, erase sector 256 blocks
mmc0: setting transfer rate to 50.000MHz (high speed timing)
GEOM: new disk mmcsd0
mmcsd0: 2GB <SD SDC 1.0 SN 0002982E MFG 06/2007 by 28 SV> at mmc0 50.0MHz/4bit/65535-block
mmc0: setting bus width to 4 bits high speed timing
With MMCCAM and this change:
sdda0 at sdhci_slot0 bus 0 scbus2 target 0 lun 0
sdda0: Relative addr: 0000b368
Card features: <Memory>
sdda0: Serial Number 0002982E
sdda0: SD SDC 1.0 SN 0002982E MFG 06/2007 by 28 SV
GEOM: new disk sdda0
Reviewed by: manu
MFC after: 3 weeks
* Prefer variables of small scope rather than large scope
* Remove a magic number
* style(9) for return statements
* Remove the get_enc_status method, which never did anything
* Fix a variable type in the handle_string method
* Proofread some comments
MFC after: 2 weeks
Sponsored by: Spectra Logic, Axcient
Reviewed by: ken, mav
Differential Revision: https://reviews.freebsd.org/D31686
sys/cam/scsi/scsi_sa.c:
Add comments explaining the priority order of the various
sources of timeout values. Also, explain that the probe
that pulls in drive recommended timeouts via the REPORT
SUPPORTED OPERATION CODES command is in a race with the
thread that creates the sysctl variables. Because of that
race, it is important that the sysctl thread not load any
timeout values from the kernel environment.
share/man/man4/sa.4:
Use the Sy macro to emphasize thousandths of a second
instead of capitalizing it.
Requested by: Warner Losh <imp@freebsd.org>
Requested by: Daniel Ebdrup Jensen <debdrup@freebsd.org>
Sponsored by: Spectra Logic
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33883
Summary:
The sa(4) driver has historically used tape drive timeouts that
were one-size fits all, with compile-time options to adjust a few
of them.
LTO-9 drives (and presumably other tape drives in the future)
implement a tape characterization process that happens the first
time a tape is loaded. The characterization process formats the
tape to account for the temperature and humidity in the environment
it is being used in. The process for LTO-9 tapes can take from 20
minutes (I have observed 17-18 minutes) to 2 hours according to the
documentation.
As a result, LTO-9 drives have significantly longer recommended
load times than previous LTO generations.
To handle this, change the sa(4) driver over to using timeouts
supplied by the tape drive using the timeout descriptors obtained
through the REPORT SUPPORTED OPERATION CODES command. That command
was introduced in SPC-4. IBM tape drives going back to at least
LTO-5 report timeout values. Oracle/Sun/StorageTek tape drives
going back to at least the T10000C report timeout values. HP LTO-5
and newer drives report timeout values. The sa(4) driver only
queries drives that claim to support SPC-4.
This makes the timeout settings automatic and accurate for newer
tape drives.
Also, add loader tunable and sysctl support so that the user can
override individual command type timeouts for all tape drives in
the system, or only for specific drives.
The new global (these affect all tape drives) loader tunables are:
kern.cam.sa.timeout.erase
kern.cam.sa.timeout.load
kern.cam.sa.timeout.locate
kern.cam.sa.timeout.mode_select
kern.cam.sa.timeout.mode_sense
kern.cam.sa.timeout.prevent
kern.cam.sa.timeout.read
kern.cam.sa.timeout.read_position
kern.cam.sa.timeout.read_block_limits
kern.cam.sa.timeout.report_density
kern.cam.sa.timeout.reserve
kern.cam.sa.timeout.rewind
kern.cam.sa.timeout.space
kern.cam.sa.timeout.tur
kern.cam.sa.timeout.write
kern.cam.sa.timeout.write_filemarks
The new per-instance loader tunable / sysctl variables are:
kern.cam.sa.%d.timeout.erase
kern.cam.sa.%d.timeout.load
kern.cam.sa.%d.timeout.locate
kern.cam.sa.%d.timeout.mode_select
kern.cam.sa.%d.timeout.mode_sense
kern.cam.sa.%d.timeout.prevent
kern.cam.sa.%d.timeout.read
kern.cam.sa.%d.timeout.read_position
kern.cam.sa.%d.timeout.read_block_limits
kern.cam.sa.%d.timeout.report_density
kern.cam.sa.%d.timeout.reserve
kern.cam.sa.%d.timeout.rewind
kern.cam.sa.%d.timeout.space
kern.cam.sa.%d.timeout.tur
kern.cam.sa.%d.timeout.write
kern.cam.sa.%d.timeout.write_filemarks
The values are reported and set in units of thousandths of a
second.
share/man/man4/sa.4:
Document the new loader tunables in the sa(4) man page.
sys/cam/scsi/scsi_sa.c:
Add a new timeout_info array to the softc.
Add a default timeouts array, along with descriptions.
Add a new sysctl tree to the softc to handle the timeout
sysctl values.
Add a new function, saloadtotunables(), that will load
the global loader tunables first and then any per-instance
loader tunables second.
Add creation of the new timeout sysctl variables in
sasysctlinit().
Add a new, optional probe state to the sa(4) driver. We
previously didn't do any probing, but now we probe for
timeout descriptors if the drive claims to support SPC-4 or
later. In saregister(), we check the SCSI revision and
either launch the probe state machine, or announce the
device and become ready.
In sastart() and sadone(), add support for the new
SA_STATE_PROBE. If we're probing, we don't go through
saerror(), since that is currently only written to handle
I/O errors in the normal state.
Change every place in the sa(4) driver that fills in
timeout values in a CCB to use the new timeout_info[] array
in the softc.
Add a new saloadtimeouts() routine to parse the returned
timeout descriptors from a completed REPORT SUPPORTED
OPERATION CODES command, and set the values for the
commands we support.
MFC after: 1 week
Sponsored by: Spectra Logic
Test Plan:
Try this out with a variety of tape drives and make sure the timeouts that
result (sysctl kern.cam.sa to see them) are reasonable.
Reviewers: #manpages, #cam
Subscribers: imp
Differential Revision: https://reviews.freebsd.org/D33883
Since both scsi_xpt and ata_xpt use the same name for the softc, this
can lead to problems in gdb. Avoid the issue by renaming the ata
probe_softc to aprobe_softc as has been done for the aprobe in
0f280cbd0a. This was overlooked at the time.
Sponsored by: Netflix
MFC After: 2 weeks
