Commit Graph

326 Commits

Author SHA1 Message Date
Alexander Motin
7f7aacb476 Fix memory and references leak due to unfreed path in case we can't
allocate bus scan CCB.

MFC after:	2 weeks
2014-01-21 23:15:23 +00:00
Alexander Motin
d718926b6d Move xpt_run_devq() call before request completion callback where it was
originally.

I am not sure why exactly have I moved it during one of many refactorings
during camlock project, but obviously it opens race window that may cause
use after free panics during SIM (in reported cases umass(4)) detach.

MFC after:	2 weeks
2014-01-11 16:52:09 +00:00
Alexander Motin
c33e40291b Take additional reference on SCSI probe periph to cover its freeze count.
Otherwise periph may be invalidated and freed before single-stepping freeze
is dropped, causing use after free panic.
2014-01-11 13:35:36 +00:00
Alexander Motin
431d3a5bfc Replace several instances of -1 with appropriate CAM_*_WILDCARD and types.
It was equal before r259397, but for good or bad, not any more for LUNs.

This change fixes at least CAM debugging.
2014-01-10 12:18:05 +00:00
Alexander Motin
daa5487f36 Some CAM locks polishing:
- Fix LOR and possible lock recursion when handling high-power commands.
Introduce new lock to protect left power quota and list of frozen devices.
 - Correct locking around xpt periph creation.
 - Remove seems never used XPT_FLAG_OPEN xpt periph flag.
2013-11-10 12:16:09 +00:00
Alexander Motin
9813c936c4 Fix lock recursion, triggered by smartctl -a /dev/adaX. 2013-11-01 00:14:15 +00:00
Nathan Whitehorn
abe8350519 printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t.
MFC after:	2 weeks
2013-10-30 14:13:15 +00:00
Alexander Motin
ec08c07e8c Minor (mostly cosmetical) addition to r256960. 2013-10-23 14:58:09 +00:00
Alexander Motin
f1486b5163 Move CAM_UNQUEUED_INDEX setting to the last moment and under the periph lock.
This fixes race condition with cam_periph_ccbwait(), causing use-after-free.
2013-10-23 12:53:05 +00:00
Alexander Motin
8ec5ab3f16 Unconditionally acquire periph reference on CCB allocation failure.
cam_periph_acquire() can return error if periph already invalidated, but
that may be unacceptable and cause deadlock if the invalidated periph can't
be destroyed without "executing" the scheduled request.

Coverity CID:	1109822
MFC after:	2 months
2013-10-22 12:58:22 +00:00
Alexander Motin
227d67aa54 Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
 - per-LUN locks to protect device and peripheral drivers state;
 - per-target locks to protect list of LUNs on target;
 - per-bus locks to protect reference counting;
 - per-send queue locks to protect queue of CCBs to be sent;
 - per-done queue locks to protect queue of completed CCBs;
 - remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock.  The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded.  Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads.  Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by:	iXsystems, Inc.
MFC after:	2 months
2013-10-21 12:00:26 +00:00
Alexander Motin
8d36a71b76 Unify periph invalidation and destruction reporting.
Print message containing device model and serial number on invalidation.

Requested by:   glebius
MFC after:	1 week
2013-10-15 17:59:41 +00:00
Alexander Motin
aa93041d2c Unhide "Serial Number" lines from bootverbose. That information may be
useful for system administration to have in hard copy (in logs) if one of
several devices suddenly dies.

Requested by:	glebius
MFC after:	1 week
2013-10-15 12:59:40 +00:00
Alexander Motin
6dfc67e379 Close the race on path ID allocation in xpt_bus_register() if two buses are
registered simultaneously. Due to topology unlock between the ID allocation
and the bus registration there is a chance that two buses may get the
same IDs. That is supposed reason of lock assertion panic in CAM during
initial bus scanning after new iscsid initiates two sessions same time.

Reported by:	trasz
Approved by:	re (glebus, marius)
MFC after:	2 weeks
2013-10-09 12:09:01 +00:00
Nathan Whitehorn
b559575358 Make sure the CCB xflags field is initialized to zero so that
CAM_EXTLUN_VALID is not erroneously set. Also add an XPORT_SRP
identifier to the known SCSI transports for the SCSI RDMA protocol, as
used, for example with Infiniband storage.

Reviewed by:	scottl
Approved by:	re (marius)
2013-09-27 16:02:40 +00:00
Scott Long
f564de00f7 Re-do r255853. Along with adding back the API/ABI changes from the
original, this hides the contents of cam_compat.h from ktrace/kdump/truss,
avoiding problems there.  There are no user-servicable parts in there, so
no need for those tools to be groping around in there.

Approved by:	re
2013-09-25 15:55:56 +00:00
Glen Barber
0082e54e9d Revert r255853 pending fixes to build errors in usr.bin/kdump
Approved by:	re (implicit)
2013-09-25 01:48:45 +00:00
Scott Long
185884259b Update the CAM API for FreeBSD 10:
- Remove the timeout_ch field.  It's been deprecated since FreeBSD 7.0;
  MPSAFE drivers should be managing their own timeout storage.  The
  remaining non-MPSAFE drivers have been modified to also manage their own
  storage, and should be considered for updating to MPSAFE (or removal)
  during the FreeBSD 10.x lifecycle.

- Add fields related to soft timeouts and quality of service, to be used
  in upcoming work.

- Add room for more flags in the CCB header and path_inq structures.

- Begin support for extended 64-bit LUNs.

- Bump the CAM version number to 0x18, but add compat shims.  Tested with
  camcontrol and smartctl.

Reviewed by:    nathanw, ken, kib
Approved by:    re
Obtained from:  Netflix
2013-09-24 16:50:53 +00:00
Alexander Motin
0d4f3c316e Add debug trace points for freeze/release device queue. 2013-09-01 17:37:19 +00:00
Alexander Motin
40f27d7cf6 Add new attribute lunname to report only textual LUN-specific device IDs.
While lunid attribute prefers to report numeric ones, having both may be
useful in some situations.
2013-08-24 09:42:14 +00:00
Alexander Motin
a29779e865 Remove droping topology mutex after iterating 100 periphs in CAMGETPASSTHRU.
That is not so slow and so often operation to handle unneeded otherwise
xsoftc.xpt_generation and respective locking complications.
2013-08-07 11:34:20 +00:00
Alexander Motin
39fe6d85ee MFprojects/camlock r249006:
Pass SIM pointer as an argument to camisr_runqueue() instead of doneq
pointer.
2013-08-05 12:15:53 +00:00
Alexander Motin
ea541bfdaa MFprojects/camlock r249505:
Change CCB queue resize logic to be able safely handle overallocations:
 - (re)allocate queue space in power of 2 chunks with 64 elements minimum
and never shrink it; with only 4/8 bytes per element size is insignificant.
 - automatically reallocate the queue to double size if it is overflowed.
 - if queue reallocation failed, store extra CCBs in unsorted TAILQ,
fetching them back as soon as some queue element is freed.

To free space in CCB for TAILQ linking, change highpowerq from keeping
high-power CCBs to keeping devices frozen due to high-power CCBs.

This encloses all pieces of queue resize logic inside of cam_queue.[ch],
removing some not obvious duties from xpt_release_ccb().
2013-08-05 11:48:40 +00:00
Kenneth D. Merry
b01773b0f1 CAM and mps(4) driver scanning changes.
Add a PIM_NOSCAN flag to the CAM path inquiry CCB.  This tells CAM
not to perform a rescan on a bus when it is registered.

We now use this flag in the mps(4) driver.  Since it knows what
devices it has attached, it is more efficient for it to just issue
a target rescan on the targets that are attached.

Also, remove the private rescan thread from the mps(4) driver in
favor of the rescan thread already built into CAM.  Without this
change, but with the change above, the MPS scanner could run before
or during CAM's initial setup, which would cause duplicate device
reprobes and announcements.

sys/param.h:
	Bump __FreeBSD_version to 1000039 for the inclusion of the
	PIM_RESCAN CAM path inquiry flag.

sys/cam/cam_ccb.h:
sys/cam/cam_xpt.c:
	Added a PIM_NOSCAN flag.  If a SIM sets this in the path
	inquiry ccb, then CAM won't rescan the bus in
	xpt_bus_regsister.

sys/dev/mps/mps_sas.c
	For versions of FreeBSD that have the PIM_NOSCAN path
	inquiry flag, don't freeze the sim queue during scanning,
	because CAM won't be scanning this bus.  Instead, hold
	up the boot.  Don't call mpssas_rescan_target in
	mpssas_startup_decrement; it's redundant and I don't
	know why it was in there.

	Set PIM_NOSCAN in path inquiry CCBs.

	Remove methods related to the internal rescan daemon.

	Always use async events to trigger a probe for EEDP support.
	In older versions of FreeBSD where AC_ADVINFO_CHANGED is
	not available, use AC_FOUND_DEVICE and issue the
	necessary READ CAPACITY manually.

	Provide a path to xpt_register_async() so that we only
	receive events for our own SCSI domain.

	Improve error reporting in cases where setup for EEDP
	detection fails.

sys/dev/mps/mps_sas.h:
	Remove softc flags and data related to the scanner thread.

sys/dev/mps/mps_sas_lsi.c:
	Unconditionally rescan the target whenever a device is added.

Sponsored by:	Spectra Logic
MFC after:	1 week
2013-07-22 18:37:07 +00:00
Alexander Motin
e5736ac88c Make some improvements to r253322 to really rescan target, not a bus.
Add there and in two more places checks for NULL on xpt_alloc_ccb_nowait().
2013-07-15 18:17:31 +00:00
Scott Long
25a2902c04 Add infrastructure for doing compatibility shims, as has been sorely
needed for the last 10 years.  Far too much of the internal API is
exposed, and every small adjustment causes applications to stop working.
To kick this off, bump the API version to 0x17 as should have been done
with r246713, but add shims to compensate.  Thanks to the shims, there
should be no visible change in application behavior.

I have plans to do a significant overhaul of the API to harnen it for
the future, but until then, I welcome others to add shims for older
versions of the API.

Obtained from:	Netflix
2013-06-17 08:57:09 +00:00
Alexander Motin
ccba710262 Make CAM return and GEOM DISK pass through new GEOM::lunid attribute.
SPC-4 specification states that serial number may be property of device,
but not a specific logical unit.  People reported about FC storages using
serial number in that way, making it unusable for purposes of LUN multipath
detection.  SPC-4 states that designators associated with logical unit from
the VPD page 83h "Device Identification" should be used for that purpose.
Report first of them in the new attribute in such preference order: NAA,
EUI-64, T10 and SCSI name string.

While there, make GEOM DISK properly report GEOM::ident in XML output also
using d_getattr() method, if available.  This fixes serial numbers reporting
for SCSI disks in `geom disk list` output and confxml.

Discussed with:	gibbs, ken
Sponsored by:	iXsystems, Inc.
MFC after:	2 weeks
2013-06-12 13:36:20 +00:00
Steven Hartland
6fb5c84ea2 Added output of device QUIRKS for CAM and AHCI devices during boot.
Reviewed by:	mav
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-05-18 23:36:21 +00:00
Sean Bruno
6b156f6137 Expose CAM_BOOT_DELAY as a kernel conf item now.
This allows users who boot without loader to adjust their environments
around slightly buggy or slow hardware.

PR:	kern/161809
Submitted by:	rozhuk.im@gmail.com
MFC after:	2 weeks
2013-04-20 00:33:37 +00:00
Alexander Motin
d442caf633 Remove owner field from struct cam_ed, unused at least since FreeBSD 7. 2013-04-14 10:14:26 +00:00
Alexander Motin
e5dfa058da MFprojects/camlock r248982:
Stop abusing xpt_periph in random plases that really have no periph related
to CCB, for example, bus scanning.  NULL value is fine in such cases and it
is correctly logged in debug messages as "noperiph".  If at some point we
need some real XPT periphs (alike to pmpX now), quite likely they will be
per-bus, and not a single global instance as xpt_periph now.
2013-04-14 09:55:48 +00:00
Alexander Motin
cccf422080 MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests.  Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by:	gibbs
2013-04-14 09:28:14 +00:00
Alexander Motin
dcdf6e7418 MFprojects/camlock:
r249017:
Some cosmetic things:
 - Unify device to target insertion inside xpt_alloc_device() instead of
duplicating it three times.
 - Remove extra checks for empty lists of devices and targets on release
since zero refcount check also implies it.
 - Reformat code to reduce indentation.

r249103:
 - Add lock assertions to every point where reference counters are modified.
 - When reference counters are reaching zero, add assertions that there are
no children items left.
 - Add a bit more locking to the xptpdperiphtraverse().
2013-04-04 20:31:40 +00:00
Alexander Motin
de9ebb6812 MFprojects/camlock r248928:
Move CAM_DEBUG_CDB messages from the point of queuing to the point of
sending to SIM.  That allows to inspect real requests execution order,
respecting priorities, freezing, etc.

MFC after:	2 weeks
2013-04-04 18:59:29 +00:00
Alexander Motin
4373522abf Remove xpt_sim_opened(), the only consumer of which was atapicam, which is
now gone.
2013-04-04 17:08:49 +00:00
Alexander Motin
9a7c269693 Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock)
to unify the code.
2013-04-04 10:19:05 +00:00
Konstantin Belousov
dd0b4fb6d5 Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c.  It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code.  The MD busdma is then given a chance to do any final processing
in the complete() callback.

The cam changes unify the bus_dmamap_load* handling in cam drivers.

The arm and mips implementations are updated to track virtual
addresses for sync().  Previously this was done in a type specific
way.  Now it is done in a generic way by recording the list of
virtuals in the map.

Submitted by:	jeff (sponsored by EMC/Isilon)
Reviewed by:	kan (previous version), scottl,
	mjacob (isp(4), no objections for target mode changes)
Discussed with:	     ian (arm changes)
Tested by:	marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
	amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)
2013-02-12 16:57:20 +00:00
Kenneth D. Merry
33a38f7453 Fix a panic during CAM EDT traversal.
The problem was a race condition between the EDT traversal used by
things like 'camcontrol devlist', and CAM peripheral driver
removal.

The EDT traversal code holds the CAM topology lock, and wants
to show devices that have been invalidated.  It acquires a
reference to the peripheral to make sure the peripheral it is
examining doesn't go away.

However, because the peripheral removal code in camperiphfree()
drops the CAM topology lock to call the peripheral's destructor
routine, we can run into a situation where the EDT traversal
increments the peripheral reference count after free process is
already in progress.  At that point, the reference count is
ignored, because it was 0 when we started the process.

Fix this race by setting a flag, CAM_PERIPH_FREE, that I previously
added and checked in xptperiphtraverse() and xptpdperiphtravsere(),
but failed to use.  If the EDT traversal code sees that flag,
it will know that the peripheral free process has already started,
and that it should not access that peripheral.

Also, fix an inconsistency in the locking between
xptpdperiphtraverse() and xptperiphtraverse().  They now both
hold the CAM topology lock while calling the peripheral traversal
function.

cam_xpt.c:	Change xptperiphtraverse() to hold the CAM topology
		lock across calls to the traversal function.

		Take out the comment in xptpdperiphtraverse() that
		referenced the locking inconsistency.

cam_periph.c:	Set the CAM_PERIPH_FREE flag when we are in the
		process of freeing a peripheral driver.

Sponsored by:	Spectra Logic Corporation
MFC after:	1 week
2012-12-07 23:48:54 +00:00
Alexander Motin
8cff7eb82f Remove priority enforcement from xpt_ation(). It is not good and even not
safe in some cases to reduce CCB priority after it was scheduled with high
priority.  This fixes reproducible deadlock when command sent through the
pass interface while ATA XPT recovers from command timeout.

Instead of that enforce priority at passioctl().  libcam provides no obvious
interface to specify CCB priority and so much (all?) code specifies zero
(highest) priority.  This change limits pass CCBs priority to NORMAL run
level, allowing XPT to complete bus and device recovery after reset before
running any payload.
2012-10-27 10:14:12 +00:00
Alexander Motin
9bccc7a939 Don't exclude XPT SIM from locking in xpt_create_path_unlocked().
We don't want xpt periph, device, target or bus disappeared because of
incorrect reference counting.
2012-10-13 18:24:52 +00:00
Alexander Motin
8cdef07f7c Use xpt_create_path_unlocked() for initial debug path compilation because
we are not holding respective SIM lock.
2012-10-13 18:11:50 +00:00
Alexander Motin
80d6987c5a Fix XPT_DEBUG paths operations locking:
- Extend the lock to cover xpt_path_release() for the new path.
 - While xpt_action() is called while holding right SIM lock for the new
   bus, the old path release may require different SIM lock. So we have
   to temporary drop the new lock and get the old one.
2012-10-13 11:23:16 +00:00
Alexander Motin
92c40f40fd XPT_DEV_MATCH is probably the only xpt_action() method that is called
without holding SIM lock. It really doesn't need that lock, but adding it
removes that specific exception, allowing to assert locking there later.

Submitted by:	ken@ (earlier version)
2012-10-13 10:18:36 +00:00
Alexander Motin
6884b66275 Protect xpt_getattr() calls with the SIM lock and assert that.
Submitted by:	ken@ (earlier version)
2012-10-12 17:18:24 +00:00
Alexander Motin
596ee08f65 Use separate malloc buckets for CAM devices, CCBs and paths. This will
make it easier to track down the source of any use after free problems.

Submitted by:	ken@
2012-10-11 20:14:11 +00:00
Alexander Motin
32aa80a6bd Don't duplicate path/ccb allocation code, use existing functions. 2012-10-11 19:57:11 +00:00
Alexander Motin
627995dcde Really handle xpt_compile_path() error in xpt_bus_register() instead of
print error message and probably crash just after it on NULL dereference.

Found by:	Clang Static Analyzer
2012-10-10 18:34:15 +00:00
Alexander Motin
3631c6382f Implement media change notification for DA and CD removable media devices.
It includes three parts:
 1) Modifications to CAM to detect media media changes and report them to
disk(9) layer. For modern SATA (and potentially UAS) devices it utilizes
Asynchronous Notification mechanism to receive events from hardware.
Active polling with TEST UNIT READY commands with 3 seconds period is used
for incapable hardware. After that both CD and DA drivers work the same way,
detecting two conditions: "NOT READY: Medium not present" after medium was
detected previously, and "UNIT ATTENTION: Not ready to ready change, medium
may have changed". First one reported to disk(9) as media removal, second
as media insert/change. To reliably receive second event new
AC_UNIT_ATTENTION async added to make UAs broadcasted to all periphs by
generic error handling code in cam_periph_error().
 2) Modifications to GEOM core to handle media remove and change events.
Media removal handled by spoiling all consumers attached to the provider.
Media change event also schedules provider retaste after spoiling to probe
new media. New flag G_CF_ORPHAN was added to consumers to reflect that
consumer is in process of destruction. It allows retaste to create new
geom instance of the same class, while previous one is still dying.
 3) Modifications to some GEOM classes: DEV -- to report media change
events to devd; VFS -- to handle spoiling same as orphan to prevent
accessing replaced media. PART class already handles spoiling alike to
orphan.

Reviewed by:	silence on geom@ and scsi@
Tested by:	avg
Sponsored by:	iXsystems, Inc. / PC-BSD
MFC after:	2 months
2012-07-29 11:51:48 +00:00
Alexander Motin
10284c8b26 In camisr() clear CAM_SIM_ON_DONEQ flag after camisr_runqueue() purged SIM
done queue. Clearing it before caused extra SIM queueing in some cases.
It was invisible during normal operation, but during USB device unplug and
respective SIM destruction it could keep pointer on SIM without having
counted reference and as result crash the system by use afer free.

Reported by:	hselasky
MFC after:	1 week
2012-06-21 14:35:46 +00:00
Kenneth D. Merry
ea37f51942 Fix several reference counting and object lifetime issues between
the pass(4) and enc(4) drivers and devfs.

The pass(4) driver uses the destroy_dev_sched() routine to
schedule its device node for destruction in a separate thread
context.  It does this because the passcleanup() routine can get
called indirectly from the passclose() routine, and that would
cause a deadlock if the close routine tried to destroy its own
device node.

In any case, once a particular passthrough driver number, e.g.
pass3, is destroyed, CAM considers that unit number (3 in this
case) available for reuse.

The problem is that devfs may not be done cleaning up the previous
instance of pass3, and will panic if isn't done cleaning up the
previous instance.

The solution is to get a callback from devfs when the device node
is removed, and make sure we hold a reference to the peripheral
until that happens.

Testing exposed some other cases where we have reference counting
issues, and those were also fixed in the pass(4) driver.

cam_periph.c:	In camperiphfree(), reorder some of the operations.

		The peripheral destructor needs to be called before
		the peripheral is removed from the peripheral is
		removed from the list.  This is because once we
		remove the peripheral from the list, and drop the
		topology lock, the peripheral number may be reused.
		But if the destructor hasn't been called yet, there
		may still be resources hanging around (like devfs
		nodes) that haven't been fully cleaned up.

cam_xpt.c:	Add an argument to xpt_remove_periph() to indicate
		whether the topology lock is already held.

scsi_enc.c:	Acquire an extra reference to the peripheral during
		registration, and release it once we get a callback
		from devfs indicating that the device node is gone.

		Call destroy_dev_sched_cb() in enc_oninvalidate()
		instead of calling destroy_dev() in the cleanup
		routine.

scsi_pass.c:	Add reference counting to handle peripheral and
		devfs object lifetime issues.

		Add a reference to the peripheral and the devfs
		node in the peripheral registration.

		Don't attempt to add a physical path alias if the
		peripheral has been marked invalid.

		Release the devfs reference once the initial
		physical path alias taskqueue run has completed.

		Schedule devfs node destruction in the
		passoninvalidate(), and release our peripheral
		reference in a new routine, passdevgonecb() once
		the devfs node is gone.  This allows the peripheral
		to fully go away, and the peripheral destructor,
		passcleanup(), will get called.

MFC after:	3 days
Sponsored by:	Spectra Logic
2012-06-20 17:08:00 +00:00