Commit Graph

54 Commits

Author SHA1 Message Date
Alexander Motin
227d67aa54 Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
 - per-LUN locks to protect device and peripheral drivers state;
 - per-target locks to protect list of LUNs on target;
 - per-bus locks to protect reference counting;
 - per-send queue locks to protect queue of CCBs to be sent;
 - per-done queue locks to protect queue of completed CCBs;
 - remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock.  The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded.  Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads.  Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by:	iXsystems, Inc.
MFC after:	2 months
2013-10-21 12:00:26 +00:00
Alexander Motin
ca11419237 Make siis(4) and mvs(4) send bus_get_dma_tag() requests to parent buses
passing real bus' child pointers instead of grandchilds.

Requested by:	kib
2013-04-18 12:43:06 +00:00
Konstantin Belousov
129c6621f7 ahci(4) and siis(4) are ready to process the unmapped i/o requests
Sponsored by:	The FreeBSD Foundation
Tested by:	pho
Submitted by:	bf (siis patch)
2013-03-19 15:09:32 +00:00
Konstantin Belousov
dd0b4fb6d5 Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c.  It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code.  The MD busdma is then given a chance to do any final processing
in the complete() callback.

The cam changes unify the bus_dmamap_load* handling in cam drivers.

The arm and mips implementations are updated to track virtual
addresses for sync().  Previously this was done in a type specific
way.  Now it is done in a generic way by recording the list of
virtuals in the map.

Submitted by:	jeff (sponsored by EMC/Isilon)
Reviewed by:	kan (previous version), scottl,
	mjacob (isp(4), no objections for target mode changes)
Discussed with:	     ian (arm changes)
Tested by:	marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
	amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)
2013-02-12 16:57:20 +00:00
Alexander Motin
eb586bd9ee Partially revert r236666:
Return PROTO_ATA protocol in response to XPT_PATH_INQ.

smartmontools uses it to identify ATA devices and I don't know any other
place now where it is important. It could probably use XPT_GDEV_TYPE
instead for more accurate protocol information, but let it live for now.

Reported by:	matthew
MFC after:	3 days
2012-06-10 11:17:14 +00:00
Alexander Motin
bc1bf6e842 ATA/SATA controllers have no idea about protocol of the connected device
until transport will do some probe actions (at least soft reset).
Make ATA/SATA SIMs to not report bogus and confusing PROTO_ATA protocol.
Make ATA/SATA transport to fill that gap by reporting protocol to SIM with
XPT_SET_TRAN_SETTINGS and patching XPT_GET_TRAN_SETTINGS results if needed.
2012-06-06 06:52:51 +00:00
Alexander Motin
711f661393 Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI
to allow drivers to handle request completion directly without passing
them to the CAM SWI thread removing extra context switch.
Modify all ATA/SATA drivers to use them.

Reviewed by:	gibbs, ken
MFC after:	2 weeks
2012-05-12 13:55:36 +00:00
Hans Petter Selasky
3b12bdb58f Rename device_delete_all_children() into device_delete_children().
Suggested by:	jhb @ and marius @
MFC after:	1 week
2011-11-22 21:56:55 +00:00
Hans Petter Selasky
11bcf702f4 Move the device_delete_all_children() function from usb_util.c
to kern/subr_bus.c. Simplify this function so that it no longer
depends on malloc() to execute. Identify a few other places where
it makes sense to use device_delete_all_children().

MFC after:	1 week
2011-11-19 10:11:50 +00:00
Ed Schouten
d745c852be Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.
This means that their use is restricted to a single C file.
2011-11-07 06:44:47 +00:00
Alexander Motin
08c8fde007 According to SATA specification, when Serial ATA Enclosure Management Bridge
(SEMB) is unable to communicate to Storage Enclosure Processor (SEP), in
response to hard and soft resets it should among other things return value
0x7F in Status register. The weird side is that it means DRQ bit set, which
tells that reset request is not completed. It would be fine if SEMB was the
only device on port. But if SEMB connected to PMP or built into it, it may
block access to other devices sharing same SATA port.

Make some tunings/fixes to soft-reset handling to workaround the issue:
 - ahci(4): request CLO on the port after soft reset to ignore DRQ bit;
 - siis(4): gracefully reinitialize port after soft reset timeout (hardware
doesn't detect reset request completion in this case);
 - mvs(4): if PMP is used, send dummy soft-reset to the PMP port to make it
clear DRQ bit for us.

For now this makes quirks in ata_pmp.c, hiding SEMB ports of SiI3726/SiI4726
PMPs, less important. Further, if hardware permit, I hope to implement real
SEMB support.
2011-05-25 13:55:49 +00:00
Alexander Motin
7bcc595738 Fix some English grammar. 2011-04-19 10:57:40 +00:00
Alexander Motin
1f145eafea According to specification. device should respond to COMRESET with COMINIT
in no more then 10ms. If we detected no device presence within that time,
there is no reason to wait longer.
2011-04-19 10:51:19 +00:00
Alexander Motin
6ac0befde2 Properly handle memory allocation errors during error recovery. 2011-04-19 08:01:17 +00:00
Alexander Motin
8d169381e4 Improve SATA Asynchronous Notification feature support in CAM:
- make SATA SIMs announce capabilities to handle SDB with Notification bit;
 - make PMP driver honor this SIMs capability;
 - make SATA XPT to negotiate and enable this feature for ATAPI devices.

This feature allows supporting SATA ATAPI devices to inform system about
some events happened, that may require attention. In my case this allows
LG GH22LS50 SATA DVR-RW drive to report tray open/close events. Events
reported to CAM in form of AC_SCSI_AEN async. Further they could be used
as a hints for checking device status and reporting media change to upper
layers, for example, via spoiling mechanism of GEOM.
2011-04-13 16:20:54 +00:00
Alexander Motin
54fc853acf As soon as siis_reset() doesn't waits for device readiness, but only for
controller port readiness (that should set just after PHY ready signal),
reduce wait time from 10s to 1s before trying more aggressive reset method.

This should improve system responsibility in some failure conditions.
2011-04-13 06:36:22 +00:00
Alexander Motin
b8b7a9027b Implement automatic SCSI sense fetching for siis(4).
Fix device freeze leak on recovery request (READ LOG, REQUEST SENSE)
failure.
2011-04-12 11:29:15 +00:00
Alexander Motin
7f63cad89e Add missing part of r217877. 2011-01-26 13:10:07 +00:00
Alexander Motin
a59641a90d Hardware supported by siis(4) allows software control over activity LEDs.
Expose that functionality to led(4) OR-ing it with regular LED activity.
2011-01-26 08:54:10 +00:00
Alexander Motin
ba3a999598 Teach ahci(4), siis(4) and ATA_CAM ata(4) wrapper report to CAM residual
I/O length on underruns, that often happens for some SCSI commands.
2010-11-08 15:36:15 +00:00
Alexander Motin
5b9392e840 Add missing mtx_destroy() on channel attach failure. 2010-10-25 07:41:21 +00:00
Alexander Motin
bf12976c76 Fix panic, when due to some kind of congestion on FIS-based switching
port multiplier some command triggers false positive timeout, but then
completes normally.

MFC after:	2 weeks
2010-09-16 12:39:50 +00:00
Alexander Motin
8edcf69406 Export PCI IDs of ATA/SATA controllers through CAM and ata(4) layers to
GEOM. This information needed for proper soft-RAID's on-disk metadata
reading and writing.
2010-07-25 15:43:52 +00:00
Alexander Motin
8d659f349e Plug memory leak to silent Coverity. Error is still not really handled.
Found with:   Coverity Prevent(tm)
CID:          4196
2010-06-05 08:16:21 +00:00
Alexander Motin
b136465250 Fix attach errors handling.
Found with:   Coverity Prevent(tm)
CID:          3477
2010-06-05 08:14:19 +00:00
Alexander Motin
cc6b610bea Fill rman range start/end values. It makes devinfo output more readable. 2010-05-22 08:30:47 +00:00
Alexander Motin
445cc79ca9 Report ATA/SATA channel number to NewBus at location string. 2010-05-22 07:32:47 +00:00
Alexander Motin
243e0fb9a0 Improve suspend/resume support. Make sure controller is idle on suspend
and reset it on resume.
2010-05-21 17:26:16 +00:00
Alexander Motin
da6808c111 Make SATA XPT negotiate and enable some additional SATA features, such as:
- device initiated power management (some devices support only this way);
 - Automatic Partial to Slumber Transition (more power saving);
 - DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.
2010-05-02 12:07:47 +00:00
Alexander Motin
81a75ae365 Add Target/LUN ID checks and deny access to targets 1-14 when PMP absent. 2010-04-30 08:22:47 +00:00
Alexander Motin
94410af774 Explicitly enable PCI busmastering on attach.
Now SiI3124 with siis(4) successfully works on sparc64 (SunBlade 100).

H/W donated by:	Gheorghe Ardelean
2010-04-15 11:17:33 +00:00
Alexander Motin
f81100fd11 Enable MSI by default for SiI3124. 2010-03-20 04:40:15 +00:00
Alexander Motin
301c88be98 MFp4:
After last running command completed, give commands in timeout state
second time.
2010-02-14 12:27:49 +00:00
Alexander Motin
2622a559a7 Use new helper functions to set PCIe max read request size. 2010-02-05 17:20:48 +00:00
Alexander Motin
4cca153030 - Give ATA/SATA SIMs info about ATAPI packet size, supported by device.
- Make ATA XPT to reject longer SCSI CDBs then supported by device, or
any SCSI CDBs, if device doesn't support ATAPI.
2010-02-02 11:09:28 +00:00
Alexander Motin
ed70cffd35 Reset port on disconnect event, to abort any running requests. 2010-01-29 12:47:04 +00:00
Alexander Motin
83c5d981ac MFp4: Large set of CAM inprovements.
- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.
2010-01-28 08:41:30 +00:00
Alexander Motin
723bd8c6b5 Usually these controllers are able to automatically decode command code to
get required command protocol. But they have no idea about new commands,
such as DATA SET MANAGEMENT (TRIM). As soon as this info any way provided
by CAM, give controller specific instructions.
2009-12-29 21:49:11 +00:00
Alexander Motin
1535d59d7c Increase Max Read Request Size for PCIe chips from 512 to 1024 bytes.
It gives those beasts additional 10% of write bandwidth.
2009-12-09 13:10:10 +00:00
Alexander Motin
3005368137 Explicitly acknowledge MSI completion, as required by SiI3124 datasheet.
It makes MSI working there. Later (and cheaper) PCIe chips (3132/3531)
still randomly crashing system in few seconds of high MSI rates, generating
something inaporopriate, like NMI or "Fatal trap 30".
2009-12-07 18:37:50 +00:00
Alexander Motin
d095fa045c SiI3124 has no SNotification register. Handle Asynchronous Notifications
there without it as good as possible.
2009-12-07 16:10:48 +00:00
Alexander Motin
b447e682d6 MFp4:
Improve ATA mode/SATA revision control.
2009-11-26 08:49:46 +00:00
Alexander Motin
c8039fc667 MFp4:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.
2009-11-24 12:47:58 +00:00
Alexander Motin
5319fb62a8 Do not require payload data to be aligned. It is not mentioned in datasheet
and works fine in practice.
2009-11-16 20:54:47 +00:00
Alexander Motin
30a4094f86 MFp4:
- Move tagged queueing control from ADA to ATA XPT. It allows to control
  device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.
2009-11-11 11:10:36 +00:00
Alexander Motin
03a6387ec3 MFp4:
Organize device IDs and add some more of them.
2009-11-10 09:46:52 +00:00
Alexander Motin
c82b245af9 Do not unarm callout on request completion and change slot selection
algorithm as done in ahci(4). This saves some CPU time on high request
rates.
2009-11-04 15:10:46 +00:00
Alexander Motin
6f9a51c735 MFp4:
- Rework timeout handling, to make it more graceful for devices sharing
controller port (with PMP). Wait for other commands completion/timeout
before initiating recovery.
- Handle timeouts and fatal errors with port hard-reset. The rest of
recovery will be done by XPT on receiving async event. More gracefull
per-device soft-reset recovery can be implemented later.
2009-11-03 12:03:13 +00:00
Alexander Motin
27acd6ea1f Reimplement device reset sequence in more controller-specific way. 2009-10-23 21:33:26 +00:00
Alexander Motin
8e7cccb3bd MFp4:
Report real max_target = 15. SIM doesn't need to know that target 15 is PMP.
It is XPT business.
2009-10-21 13:00:01 +00:00