Commit Graph

97 Commits

Author SHA1 Message Date
Alexander Motin
f7241cceb0 Coalesce last data move and command status for read commands.
Make CTL core and block backend set success status before initiating last
data move for read commands.  Make CAM target and iSCSI frontends detect
such condition and send command status together with data.  New I/O flag
allows to skip duplicate status sending on later fe_done() call.

For Fibre Channel this change saves one of three interrupts per read command,
increasing performance from 126K to 160K IOPS.  For iSCSI this change saves
one of three PDUs per read command, increasing performance from 1M to 1.2M
IOPS.

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2014-11-25 17:53:35 +00:00
Alexander Motin
3f829b0c9c Fix LUN resize broken by r272911 commit.
MFC after:	3 days
2014-11-07 20:42:15 +00:00
Alexander Motin
c3e7ba3e6d Add to CTL support for logical block provisioning threshold notifications.
For ZVOL-backed LUNs this allows to inform initiators if storage's used or
available spaces get above/below the configured thresholds.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-11-06 00:48:36 +00:00
Alexander Motin
4fc18ff9bb Implement better handling for ENOSPC error for both CTL and CAM.
This makes VMWare VAAI Thin Provisioning Stun primitive activate, pausing
the virtual machine, when backing storage (ZFS pool) is getting overflowed.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2014-10-29 03:14:29 +00:00
Alexander Motin
20a28d6cee Report physical block size for file-backed LUNs, using vattr.va_blocksize.
MFC after:	1 week
2014-10-13 11:00:58 +00:00
Alexander Motin
19720f4113 Make ctld start even if some LUNs are unable to open backing storage.
Such LUNs will be visible to initiators, but return "not ready" status
on media access commands.  If backing storage become available later,
`ctladm modify ...` or `service ctld reload` can trigger its reopen.
2014-10-10 19:41:09 +00:00
Alexander Motin
8a41675372 Add support for WRITE ATOMIC (16) command and report SBC-4 compliance.
Atomic writes are only supported for ZVOLs in "dev" mode.  In other cases
atomicity can not be guarantied and so the command is blocked.
2014-10-08 07:48:36 +00:00
Alexander Motin
64c5167c91 Add support for "no Data-Out Buffer" (NDOB) flag of WRITE SAME (16) command. 2014-09-18 21:39:00 +00:00
Alexander Motin
71d8e97e35 When updating device media size use cached cdevsw pointer.
Using pointer from the cdev directly is dangerous since we have no reference
on it, and it may change any time.  That caused panic if device has gone.

While there, report capacity change only if it really changed.

MFC after:	3 days
2014-09-18 17:25:20 +00:00
Alexander Motin
55551d0542 Improve cache control support, including DPO/FUA flags and the mode page.
At this moment it works only for files and ZVOLs in device mode since BIOs
have no respective respective cache control flags (DPO/FUA).

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2014-09-09 11:38:29 +00:00
Alexander Motin
3406a2a083 Fix several issues and inconsistencies in UNMAP capabilities reporting.
This makes Windows 2012 to start using UNMAP on our disks.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-08-06 08:54:31 +00:00
Alexander Motin
67f586a8c1 Implement separate I/O dispatch method for ZVOLs in "dev" mode.
Unlike disk devices ZVOLs process all requests synchronously.  That makes
impossible sending multiple requests to them from single thread.  From the
other side ZVOLs have real d_read/d_write methods, which unlike d_strategy
can handle uio scatter/gather and have no strict I/O size limitations.

So, if ZVOL in "dev" mode is detected, use of d_read/d_write methods instead
of d_strategy allows to avoid pointless splitting of large requests into
MAXPHYS (128K) sized chunks.

MFC after:	1 week
2014-07-26 13:56:50 +00:00
Alexander Motin
696297ad2f Fix infinite loop, when doing WRITE SAME on file-backed LUN.
MFC after:	3 days
2014-07-26 13:43:25 +00:00
Alexander Motin
2cfbcb9b3a Improve readability of XML generated by CTL_LUN_LIST. 2014-07-05 04:10:24 +00:00
Alexander Motin
43fb3a65e3 Make options KPI more generic to allow it to be used for ports too,
not only for LUNs.
2014-07-05 03:34:52 +00:00
Hans Petter Selasky
af3b2549c4 Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
Glen Barber
37a107a407 Revert r267961, r267973:
These changes prevent sysctl(8) from returning proper output,
such as:

 1) no output from sysctl(8)
 2) erroneously returning ENOMEM with tools like truss(1)
    or uname(1)
 truss: can not get etype: Cannot allocate memory
2014-06-27 22:05:21 +00:00
Hans Petter Selasky
3da1cf1e88 Extend the meaning of the CTLFLAG_TUN flag to automatically check if
there is an environment variable which shall initialize the SYSCTL
during early boot. This works for all SYSCTL types both statically and
dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
be used in the case a tunable sysctl has a custom initialisation
function allowing the sysctl to still be marked as a tunable. The
kernel SYSCTL API is mostly the same, with a few exceptions for some
special operations like iterating childrens of a static/extern SYSCTL
node. This operation should probably be made into a factored out
common macro, hence some device drivers use this. The reason for
changing the SYSCTL API was the need for a SYSCTL parent OID pointer
and not only the SYSCTL parent OID list pointer in order to quickly
generate the sysctl path. The motivation behind this patch is to avoid
parameter loading cludges inside the OFED driver subsystem. Instead of
adding special code to the OFED driver subsystem to post-load tunables
into dynamically created sysctls, we generalize this in the kernel.

Other changes:
- Corrected a possibly incorrect sysctl name from "hw.cbb.intr_mask"
to "hw.pcic.intr_mask".
- Removed redundant TUNABLE statements throughout the kernel.
- Some minor code rewrites in connection to removing not needed
TUNABLE statements.
- Added a missing SYSCTL_DECL().
- Wrapped two very long lines.
- Avoid malloc()/free() inside sysctl string handling, in case it is
called to initialize a sysctl from a tunable, hence malloc()/free() is
not ready when sysctls from the sysctl dataset are registered.
- Bumped FreeBSD version to indicate SYSCTL API change.

MFC after:	2 weeks
Sponsored by:	Mellanox Technologies
2014-06-27 16:33:43 +00:00
Alexander Motin
75c7a1d357 Lock devstat updates in block backend to make it usable. Polish lock names.
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-06-25 17:54:36 +00:00
Alexander Motin
ead2f11724 Add some more CTL_FLAG_ABORT check points.
This should allow to abort commands doing mostly disk I/O, such as VERIFY
or WRITE SAME.  Before this change CTL_FLAG_ABORT was only checked around
data moves, which for these commands may not happen for a very long time.

MFC after:	2 weeks
2014-06-19 12:43:41 +00:00
Alexander Motin
11b569f7cb Add support for VERIFY(10/12/16) and COMPARE AND WRITE SCSI commands.
Make data_submit backends method support not only read and write requests,
but also two new ones: verify and compare.  Verify just checks readability
of the data in specified location without transferring them outside.
Compare reads the specified data and compares them to received data,
returning error if they are different.

VERIFY(10/12/16) commands request either verify or compare from backend,
depending on BYTCHK CDB field.  COMPARE AND WRITE command executed in two
stages: first it requests compare, and then, if succeesed, requests write.
Atomicity of operation is guarantied by CTL request ordering code.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-06-16 11:00:14 +00:00
Alexander Motin
e86a414238 Make backends track completion by processed number of sectors instead of
total transfer size.

Commands such as VERIFY or COMPARE AND WRITE may have transfer size not
matching directly to number of sectors.
2014-06-15 20:14:11 +00:00
Alexander Motin
66df9136e3 Remove memcpy() from ctl_private[] accesses.
That union is aligned enough to access data directly.
2014-06-15 18:16:51 +00:00
Alexander Motin
9c71cd5aae Move kern_total_len setting from backend to core code. 2014-06-15 17:14:52 +00:00
Alexander Motin
eb3687a6a6 Remove custom processing for "file" option. 2014-06-15 09:37:06 +00:00
Alexander Motin
57a5db13b7 Implement small KPI to access LUN options instead doing it by hands.
MFC after:	2 weeks
2014-06-14 17:47:44 +00:00
Alexander Motin
9e005bbcc9 Fix some leaks on LUN creation error.
MFC after:	2 weeks
2014-06-12 21:50:46 +00:00
Alexander Motin
08a7cce543 Remove limits on size of READ/WRITE operations.
Instead of allocating up to 16MB or RAM at once to handle whole I/O,
allocate up to 1MB at a time, but do multiple ctl_datamove() and storage
I/Os if needed.
2014-04-24 16:19:49 +00:00
Alexander Motin
8f5a226a3c When splitting huge unmap requests, do it on sector boundary. 2014-04-09 10:44:09 +00:00
Alexander Motin
7e0be0226f Another fix for r264274. Last moment cosmetic changes are evil! 2014-04-08 22:36:39 +00:00
Alexander Motin
ee7f31c068 Add support for SCSI UNMAP commands to CTL.
This patch adds support for three new SCSI commands: UNMAP, WRITE SAME(10)
and WRITE SAME(16).  WRITE SAME commands support both normal write mode
and UNMAP flag.  To properly report UNMAP capabilities this patch also adds
support for reporting two new VPD pages: Block limits and Logical Block
Provisioning.

UNMAP support can be enabled per-LUN by adding "-o unmap=on" to `ctladm
create` command line or "option unmap on" to lun sections of /etc/ctl.conf.

At this moment UNMAP supported for ramdisks and device-backed block LUNs.
It was tested to work great with ZFS ZVOLs.  For file-backed LUNs UNMAP
support is unfortunately missing due to absence of respective VFS KPI.

Reviewed by:	ken
MFC after:	1 month
Sponsored by:	iXsystems, Inc
2014-04-08 20:50:48 +00:00
Alexander Motin
f601272298 Report stripe size and offset of the backing device in READ CAPACITY (16)
as physical sector size and offset.

MFC after:	2 weeks
2014-04-06 10:13:14 +00:00
Edward Tomasz Napierala
a0e36aee93 Remove the homegrown ctl_be_block_io allocator, replacing it with UMA.
There is no performance difference.

Reviewed by:	mav@
Sponsored by:	The FreeBSD Foundation
2014-04-01 21:13:05 +00:00
Alexander Motin
e0c2f97550 Make CTL block backend return proper error code for operations unsupposed
by the underlying device.

MFC after:	2 weeks
2014-02-06 03:54:58 +00:00
Edward Tomasz Napierala
025a2301f8 Properly report an error instead of panicing when user tries to create
LUN backed by non-disk device, e.g. /dev/null.

Reviewed by:	ken (earlier version)
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-03 18:04:14 +00:00
Andriy Gapon
d9fae5ab88 dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE
In its stead use the Solaris / illumos approach of emulating '-' (dash)
in probe names with '__' (two consecutive underscores).

Reviewed by:	markj
MFC after:	3 weeks
2013-11-26 08:46:27 +00:00
Attilio Rao
54366c0bd7 - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
  calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
  version of the releasing functions for mutex, rwlock and sxlock.
  Failing to do so skips the lockstat_probe_func invokation for
  unlocking.
- As part of the LOCKSTAT support is inlined in mutex operation, for
  kernel compiled without lock debugging options, potentially every
  consumer must be compiled including opt_kdtrace.h.
  Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
  dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
  is linked there and it is only used as a compile-time stub [0].

[0] immediately shows some new bug as DTRACE-derived support for debug
in sfxge is broken and it was never really tested.  As it was not
including correctly opt_kdtrace.h before it was never enabled so it
was kept broken for a while.  Fix this by using a protection stub,
leaving sfxge driver authors the responsibility for fixing it
appropriately [1].

Sponsored by:	EMC / Isilon storage division
Discussed with:	rstone
[0] Reported by:	rstone
[1] Discussed with:	philip
2013-11-25 07:38:45 +00:00
Alexander Motin
eeb9405409 Remove 128KB bzero() call done for every block I/O data buffer.
On my tests this improves performance of the new iSCSI target backed by
GEOM STRIPE of SSDs from 75K to 110K IOPS.

Reviewed by:	ken
2013-10-23 17:55:35 +00:00
Edward Tomasz Napierala
81a2151d5c CTL changes required for iSCSI target, most notably LUN remapping
and a mechanism to allow CTL frontends for retrieving LUN options.

Reviewed by:	ken (earlier version)
2013-08-24 01:50:31 +00:00
Edward Tomasz Napierala
d2a0972d46 Make SYNCHRONIZE CACHE work with LUNs backed by device files (as opposed
to regular files, which already worked fine).  With this change, it's no
longer neccessary to use "ctladm realsync off" workaround.
2013-04-06 10:34:02 +00:00
Edward Tomasz Napierala
9eef3375ce Don't directly dereference userland pointer; instead use kernel pointer
copied in from userspace.  This fixes instant panic when creating CTL LUN
on sparc64.  Not a security problem, since the API is root-only.

Reviewed by:	ken
Sponsored by:	FreeBSD Foundation
2013-04-02 16:50:50 +00:00
Edward Tomasz Napierala
a492b68b27 Use M_ZERO instead of explicit memsets and bzeros. 2012-10-30 12:52:41 +00:00
Konstantin Belousov
5050aa86cf Remove the support for using non-mpsafe filesystem modules.
In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by:	attilio
Tested by:	pho
2012-10-22 17:50:54 +00:00
Edward Tomasz Napierala
a0a6ff825b Remove useless NULL checks after M_WAITOK allocations. 2012-09-27 10:51:38 +00:00
Edward Tomasz Napierala
811772950f Add LUN resizing to CTL. Also make it possible to explicitly set
size when creating file-backed or device-backed LUN.

Reviewed by:	ken (earlier version)
Sponsored by:	The FreeBSD Foundation
2012-03-06 13:43:57 +00:00
Kenneth D. Merry
2a2443d833 Quiet some clang warnings when compiling CTL.
ctl_error.c,
ctl_error.h:	Take out the ctl_sense_format enumeration, and use
		scsi_sense_data_type instead.

		Remove ctl_get_sense_format() and switch ctl_build_ua()
		over to using scsi_sense_data_type.

ctl_backend_ramdisk.c,
ctl_backend_block.c:
		Use C99 structure initializers instead of GNU initializers.

ctl.c:		Switch over to using the SCSI sense format enumeration
		instead of the CTL-specific enumeration.

Submitted by:	dim (partially)
MFC after:	1 month
2012-01-19 18:42:03 +00:00
Kenneth D. Merry
130f4520cb Add the CAM Target Layer (CTL).
CTL is a disk and processor device emulation subsystem originally written
for Copan Systems under Linux starting in 2003.  It has been shipping in
Copan (now SGI) products since 2005.

It was ported to FreeBSD in 2008, and thanks to an agreement between SGI
(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is
available under a BSD-style license.  The intent behind the agreement was
that Spectra would work to get CTL into the FreeBSD tree.

Some CTL features:

 - Disk and processor device emulation.
 - Tagged queueing
 - SCSI task attribute support (ordered, head of queue, simple tags)
 - SCSI implicit command ordering support.  (e.g. if a read follows a mode
   select, the read will be blocked until the mode select completes.)
 - Full task management support (abort, LUN reset, target reset, etc.)
 - Support for multiple ports
 - Support for multiple simultaneous initiators
 - Support for multiple simultaneous backing stores
 - Persistent reservation support
 - Mode sense/select support
 - Error injection support
 - High Availability support (1)
 - All I/O handled in-kernel, no userland context switch overhead.

(1) HA Support is just an API stub, and needs much more to be fully
    functional.

ctl.c:			The core of CTL.  Command handlers and processing,
			character driver, and HA support are here.

ctl.h:			Basic function declarations and data structures.

ctl_backend.c,
ctl_backend.h:		The basic CTL backend API.

ctl_backend_block.c,
ctl_backend_block.h:	The block and file backend.  This allows for using
			a disk or a file as the backing store for a LUN.
			Multiple threads are started to do I/O to the
			backing device, primarily because the VFS API
			requires that to get any concurrency.

ctl_backend_ramdisk.c:	A "fake" ramdisk backend.  It only allocates a
			small amount of memory to act as a source and sink
			for reads and writes from an initiator.  Therefore
			it cannot be used for any real data, but it can be
			used to test for throughput.  It can also be used
			to test initiators' support for extremely large LUNs.

ctl_cmd_table.c:	This is a table with all 256 possible SCSI opcodes,
			and command handler functions defined for supported
			opcodes.

ctl_debug.h:		Debugging support.

ctl_error.c,
ctl_error.h:		CTL-specific wrappers around the CAM sense building
			functions.

ctl_frontend.c,
ctl_frontend.h:		These files define the basic CTL frontend port API.

ctl_frontend_cam_sim.c:	This is a CTL frontend port that is also a CAM SIM.
			This frontend allows for using CTL without any
			target-capable hardware.  So any LUNs you create in
			CTL are visible in CAM via this port.

ctl_frontend_internal.c,
ctl_frontend_internal.h:
			This is a frontend port written for Copan to do
			some system-specific tasks that required sending
			commands into CTL from inside the kernel.  This
			isn't entirely relevant to FreeBSD in general,
			but can perhaps be repurposed.

ctl_ha.h:		This is a stubbed-out High Availability API.  Much
			more is needed for full HA support.  See the
			comments in the header and the description of what
			is needed in the README.ctl.txt file for more
			details.

ctl_io.h:		This defines most of the core CTL I/O structures.
			union ctl_io is conceptually very similar to CAM's
			union ccb.

ctl_ioctl.h:		This defines all ioctls available through the CTL
			character device, and the data structures needed
			for those ioctls.

ctl_mem_pool.c,
ctl_mem_pool.h:		Generic memory pool implementation used by the
			internal frontend.

ctl_private.h:		Private data structres (e.g. CTL softc) and
			function prototypes.  This also includes the SCSI
			vendor and product names used by CTL.

ctl_scsi_all.c,
ctl_scsi_all.h:		CTL wrappers around CAM sense printing functions.

ctl_ser_table.c:	Command serialization table.  This defines what
			happens when one type of command is followed by
			another type of command.

ctl_util.c,
ctl_util.h:		CTL utility functions, primarily designed to be
			used from userland.  See ctladm for the primary
			consumer of these functions.  These include CDB
			building functions.

scsi_ctl.c:		CAM target peripheral driver and CTL frontend port.
			This is the path into CTL for commands from
			target-capable hardware/SIMs.

README.ctl.txt:		CTL code features, roadmap, to-do list.

usr.sbin/Makefile:	Add ctladm.

ctladm/Makefile,
ctladm/ctladm.8,
ctladm/ctladm.c,
ctladm/ctladm.h,
ctladm/util.c:		ctladm(8) is the CTL management utility.
			It fills a role similar to camcontrol(8).
			It allow configuring LUNs, issuing commands,
			injecting errors and various other control
			functions.

usr.bin/Makefile:	Add ctlstat.

ctlstat/Makefile
ctlstat/ctlstat.8,
ctlstat/ctlstat.c:	ctlstat(8) fills a role similar to iostat(8).
			It reports I/O statistics for CTL.

sys/conf/files:		Add CTL files.

sys/conf/NOTES:		Add device ctl.

sys/cam/scsi_all.h:	To conform to more recent specs, the inquiry CDB
			length field is now 2 bytes long.

			Add several mode page definitions for CTL.

sys/cam/scsi_all.c:	Handle the new 2 byte inquiry length.

sys/dev/ciss/ciss.c,
sys/dev/ata/atapi-cam.c,
sys/cam/scsi/scsi_targ_bh.c,
scsi_target/scsi_cmds.c,
mlxcontrol/interface.c:	Update for 2 byte inquiry length field.

scsi_da.h:		Add versions of the format and rigid disk pages
			that are in a more reasonable format for CTL.

amd64/conf/GENERIC,
i386/conf/GENERIC,
ia64/conf/GENERIC,
sparc64/conf/GENERIC:	Add device ctl.

i386/conf/PAE:		The CTL frontend SIM at least does not compile
			cleanly on PAE.

Sponsored by:	Copan Systems, SGI and Spectra Logic
MFC after:	1 month
2012-01-12 00:34:33 +00:00