Commit Graph

15 Commits

Author SHA1 Message Date
Alexander Motin
227d67aa54 Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
 - per-LUN locks to protect device and peripheral drivers state;
 - per-target locks to protect list of LUNs on target;
 - per-bus locks to protect reference counting;
 - per-send queue locks to protect queue of CCBs to be sent;
 - per-done queue locks to protect queue of completed CCBs;
 - remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock.  The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded.  Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads.  Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by:	iXsystems, Inc.
MFC after:	2 months
2013-10-21 12:00:26 +00:00
Alexander Motin
cccf422080 MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests.  Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by:	gibbs
2013-04-14 09:28:14 +00:00
Alexander Motin
4373522abf Remove xpt_sim_opened(), the only consumer of which was atapicam, which is
now gone.
2013-04-04 17:08:49 +00:00
Alexander Motin
711f661393 Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI
to allow drivers to handle request completion directly without passing
them to the CAM SWI thread removing extra context switch.
Modify all ATA/SATA drivers to use them.

Reviewed by:	gibbs, ken
MFC after:	2 weeks
2012-05-12 13:55:36 +00:00
Alexander Motin
83c5d981ac MFp4: Large set of CAM inprovements.
- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
Edward Tomasz Napierala
02caf36ee1 Make "kldunload atapicam" return EBUSY instead of deadlocking when a device
created by atapicam is being kept opened or mounted.  This is probably just
a temporary solution until we invent something better.

Reviewed by:	scottl
Approved by:	rwatson (mentor)
Sponsored by:	FreeBSD Foundation
Reported by:	Jaakko Heinonen
2009-01-08 17:26:51 +00:00
Scott Long
b50569b71d Prepare for future integration between CAM and newbus. xpt_bus_register
now takes a device_t to be the parent of the bus that is being created.
Most SIMs have been updated with a reasonable argument, but a few exceptions
just pass NULL for now.  This argument isn't used yet and the newbus
integration likely won't be ready until after 7.0-RELEASE.
2007-06-17 05:55:54 +00:00
Warner Losh
898b0535b7 Start each of the license/copyright comments with /*- 2005-01-05 22:34:37 +00:00
Justin T. Gibbs
434bbf6e63 Clean up the xpt_sim creation to mirror how HBA drivers perform this task.
Clean up node creation in the EDT so that initialization follows member
declaration.

Sort registered paths by pathid so that we probe busses in order of
ascending pathid.  This makes hardwiring of busses without wiring
individual targets do what the user expects. (submitted by tegge@FreeBSD.org)

Fix an EDT node leak.  Target nodes would never go away.

Implement xpt_bus_deregister().
(prodded by some patches from T. Ichinoseki, but implemented differently.)
2000-01-14 23:08:46 +00:00
Peter Wemm
c447342094 Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL"
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot).  This is consistant with the other
BSD's who made this change quite some time ago.  More commits to come.
1999-12-29 05:07:58 +00:00
Peter Wemm
c3aac50f28 $Id$ -> $FreeBSD$ 1999-08-28 01:08:13 +00:00
Justin T. Gibbs
e5f5a0d8b1 xpt_release_devq now takes a count. 1999-08-16 22:23:01 +00:00
Justin T. Gibbs
d1dc9ab13a xpt_done has no reason to take a volatile ccb *. 1999-03-05 23:18:56 +00:00
Matthew Dillon
b4e36adf1c Fix warnings preparing for -Wall -Wcast-qual
Also disable one usb module in LINT due to fatal compilation errors,
    temporary.
1999-01-27 20:09:21 +00:00
Justin T. Gibbs
8b8a9b1d3e CAM Transport Layer (XPT).
Submitted by:	The CAM Team
1998-09-15 06:33:23 +00:00