Commit Graph

20 Commits

Author SHA1 Message Date
Mark Johnston
f30cad3355 CAM ccbq sanity: checks on insert and remove
KASSERT in cam_ccbq_insert_ccb that only XPT_FC_QUEUED ops are queued,
and XPT_FC_USER_CCB ops are not. Otherwise cam_ccbq_ccb_done may be
skipped.

Bounds check the index used for camq_remove in order to panic instead
of scribble on removal of an out-of-bounds index (e.g. consider the
effect of camq_remove of CAM_UNQUEUED_INDEX).

KASSERT in cam_ccbq_remove_ccb that the ccb removed by index was the
one sought.

Submitted by:	Ryan Libby <rlibby@gmail.com>
Reviewed by:	imp, mav
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D8151
2016-10-05 17:18:24 +00:00
Pedro F. Giffuni
d63978e2b6 sys/cam: spelling fixes.
Affects a debug message.

MFC after: 2 weeks
2016-04-29 21:04:06 +00:00
Alexander Motin
959ec2581b Update CAM CCB accounting for the new status quo.
devq_openings counter lost its meaning after allocation queues has gone.
held counter is still meaningful, but problematic to update due to separate
locking of CCB allocation and queuing.

To fix that replace devq_openings counter with allocated counter.  held is
now calculated on request as difference between number of allocated, queued
and active CCBs.

MFC after:	1 month
2014-09-14 11:59:49 +00:00
Alexander Motin
227d67aa54 Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
 - per-LUN locks to protect device and peripheral drivers state;
 - per-target locks to protect list of LUNs on target;
 - per-bus locks to protect reference counting;
 - per-send queue locks to protect queue of CCBs to be sent;
 - per-done queue locks to protect queue of completed CCBs;
 - remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock.  The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded.  Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads.  Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by:	iXsystems, Inc.
MFC after:	2 months
2013-10-21 12:00:26 +00:00
Alexander Motin
ea541bfdaa MFprojects/camlock r249505:
Change CCB queue resize logic to be able safely handle overallocations:
 - (re)allocate queue space in power of 2 chunks with 64 elements minimum
and never shrink it; with only 4/8 bytes per element size is insignificant.
 - automatically reallocate the queue to double size if it is overflowed.
 - if queue reallocation failed, store extra CCBs in unsorted TAILQ,
fetching them back as soon as some queue element is freed.

To free space in CCB for TAILQ linking, change highpowerq from keeping
high-power CCBs to keeping devices frozen due to high-power CCBs.

This encloses all pieces of queue resize logic inside of cam_queue.[ch],
removing some not obvious duties from xpt_release_ccb().
2013-08-05 11:48:40 +00:00
Alexander Motin
cccf422080 MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests.  Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by:	gibbs
2013-04-14 09:28:14 +00:00
Alexander Motin
8cc6481cd5 Increase device CCB queue array size by CAM_RL_VALUES - 1 (4) elements.
It is required to store extra recovery requests in case of bus resets.
On ATA/SATA this fixes assertion panics on HEAD with INVARIANTS enabled or
possible memory corruptions otherwise if timeout/reset happens when device
CCB queue is already full.

Reported by:	gibbs@
MFC after:	1 week
2012-10-11 15:21:07 +00:00
Alexander Motin
d6e285946d Change queue overflow checks from DIAGNOSTIC+panic() to KASSERT() to make
them enabled on HEAD by default. It is probably better to do single compare
then hunt for unexpected memory corruption.
2012-09-28 12:13:34 +00:00
Ed Schouten
d745c852be Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.
This means that their use is restricted to a single C file.
2011-11-07 06:44:47 +00:00
Alexander Motin
1f1158b28d MFp4:
Some more missed parts from previous commits.
2009-11-14 20:30:42 +00:00
Alexander Motin
20a7933f53 After thinking again, implement cam_ccbq_fini().
This is effectively NULL change, but makes this API a bit more consistent.
2009-10-22 21:07:32 +00:00
Tai-hwa Liang
362abc449c - Providing fine-grained malloc statistic by replacing M_DEVBUF with
module-specific malloc types.  These should help us to pinpoint the
  possible memory leakage in the future.
- Implementing xpt_alloc_ccb_nowait() and replacing all malloc/free based
  CCB memory management with xpt_alloc_ccb[_nowait]/xpt_free_ccb.  Hopefully
  this would be helpful if someday we move the CCB allocator to use UMA
  instead of malloc().

Encouraged by:	jeffr, rwatson
Reviewed by:	gibbs, scottl
Approved by:	re (scottl)
2005-07-01 15:21:30 +00:00
Warner Losh
898b0535b7 Start each of the license/copyright comments with /*- 2005-01-05 22:34:37 +00:00
David E. O'Brien
9c963d873e Use __FBSDID(). 2003-06-10 17:50:20 +00:00
Jens Schweikhardt
d64ada501a Fix typos, mostly s/ an / a / where appropriate and a few s/an/and/
Add FreeBSD Id tag where missing.
2002-12-30 21:18:15 +00:00
Peter Wemm
c3aac50f28 $Id$ -> $FreeBSD$ 1999-08-28 01:08:13 +00:00
Justin T. Gibbs
f22adc7f6d Fix a long standing bug in the camq teardown code. 1999-08-16 02:23:42 +00:00
Justin T. Gibbs
5a526431f8 cam_periph.c:
Move handling of CAM_AUTOSENSE_FAIL into block dealing with
	all other scsi status errors.

cam_queue.c:
cam_queue.h:
	Fix 'off by one' heap bug in a more efficient manner.  Since
	heap algorithms like to deal with indexes started from 1,
	offset our heap array pointer at allocation time to make this
	so for a C environment.  This makes the implementation of the
	algorithm a bit more efficient.

cam_xpt.c:
	Use macros for accessing the head of the heap so that code
	is isolated from implementation details of the heap.
1999-04-19 21:26:08 +00:00
Justin T. Gibbs
8bad620d54 Remove camq_regen(). We already perform modular comparisons
for generation counts, so no further steps to deal with generation
count wrap are required.

Fix an off by one problem in the camq heap code.
1999-04-07 22:57:48 +00:00
Justin T. Gibbs
8b8a9b1d3e CAM Transport Layer (XPT).
Submitted by:	The CAM Team
1998-09-15 06:33:23 +00:00