Previously ahci_attach returned a hard coded ENXIO instead of the value
from ahci_setup_interrupt. This is effectively a NOOP change as currently
ahci_setup_interrupt only ever returns 0 or ENXIO, so just there to protect
against any future changes to that.
Differential Revision: D838
MFC after: 2 weeks
Sponsored by: Multiplay
This prevents the possiblity of any overruns on the statically allocated
struct irqs field.
Differential Revision: D838
MFC after: 2 weeks
X-MFC-With: r276012
Sponsored by: Multiplay
Previously, any timeout value for which (timeout * hz) will overflow the
signed integer, will give weird results, since callout(9) routines will
convert negative values of ticks to '1'. For unsigned integer overflow we
will get sufficiently smaller timeout values than expected.
Switch from callout_reset, which requires conversion to int based ticks
to callout_reset_sbt to avoid this.
Also correct isci to correctly resolve ccb timeout.
This was based on the original work done by Eygene Ryabinkin
<rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid
the overlow.
Differential Revision: https://reviews.freebsd.org/D1157
Reviewed by: mav, davide
MFC after: 1 month
Sponsored by: Multiplay
It was found that VirtualBox' AHCI does not allow nterrupt to be cleared
before the interrupt status register is read, causing interrupt storm.
AHCI specification allows to skip this register use when multi-vector MSI
is enabled and so interrupting port is known. For single-vector MSI that
is not stated explicitly, but if the port is only one, it is obviously
known too.
Previously it was possible for issues e.g. use after free, to result
from processing the done queue while not holding the channel lock.
While this should never happen in practice, unexpected code flows
which result in two threads processing from the same queue may
be possible.
We now use a local STAILQ to prevent this ever being an issue.
Sponsored by: Multiplay
For controllers with only one port (like PCIe or M.2 SSDs) interrupt can
come from only one source, and skipping read saves few percents of CPU time.
MFC after: 1 month
H/W donated by: I/O Switch
I gave up to update list of Marvell chips that require this quirk.
The final nail was growing number of PCIe/M.2 SSDs where Marvell chips
have PCI IDs of different vendors.
MFC after: 1 week
H/W donated by: I/O Switch
PCI IDs into quirks, which mostly fit (though you'd get no argument
from me that AHCI_Q_SATA1_UNIT0 is oddly specific). Set these quirks
in the PCI attachment. Make some shared functions public so that PCI
and possibly other bus attachments can use them.
The split isn't perfect yet, but it is functional. The split will be
perfected as other bus attachments for AHCI are written.
Sponsored by: Netflix
Reviewed by: kan, mav
Differential Revision: https://reviews.freebsd.org/D699
initializing 16. Initialize all 20 so we don't send garbage in the
Auxiliary register. The SATA standard mandates a 5 dword length for
the Host to Device FIS.
Sponsored by: Netflix
Using unmapped BIOs causes failure inside bus_dmamap_sync, since
this function requires valid MVA address, which is not present
if mapping is not set up.
Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf
shifts into the sign bit. Instead use (1U << 31) which gets the
expected result.
This fix is not ideal as it assumes a 32 bit int, but does fix the issue
for most cases.
A similar change was made in OpenBSD.
Discussed with: -arch, rdivacky
Reviewed by: cperciva
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.
Replace big per-SIM locks with bunch of smaller ones:
- per-LUN locks to protect device and peripheral drivers state;
- per-target locks to protect list of LUNs on target;
- per-bus locks to protect reference counting;
- per-send queue locks to protect queue of CCBs to be sent;
- per-done queue locks to protect queue of completed CCBs;
- remaining per-SIM locks now protect only HBA driver internals.
While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock. The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded. Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.
To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.
Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads. Load balanced
between them using "hash" of the device B:T:L address.
HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.
Sponsored by: iXsystems, Inc.
MFC after: 2 months
extra read from PxCI/PxSACT registers. If only NCQ commands are running, we
don't really need PxCI. If only non-NCQ commands are running we don't need
PxSACT. Mixed set may happen only on controllers with FIS-based switching
when port multiplier is attached, and then we have to read both registers.
MFC after: 1 month
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.
The cam changes unify the bus_dmamap_load* handling in cam drivers.
The arm and mips implementations are updated to track virtual
addresses for sync(). Previously this was done in a type specific
way. Now it is done in a generic way by recording the list of
virtuals in the map.
Submitted by: jeff (sponsored by EMC/Isilon)
Reviewed by: kan (previous version), scottl,
mjacob (isp(4), no objections for target mode changes)
Discussed with: ian (arm changes)
Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)
I am not exactly sure about the naming due to lack of specs on AMD site,
but it is better to have some identification then none at all.
MFC after: 1 month
subdevice ahciem. Emulate SEMB SES device from AHCI LED interface to expose
it to users in form of ses(4) CAM device. If we ever see AHCI controllers
supporting SES of SAF-TE over I2C as described by specification, they should
fit well into this new picture.
Sponsored by: iXsystems, Inc.
Return PROTO_ATA protocol in response to XPT_PATH_INQ.
smartmontools uses it to identify ATA devices and I don't know any other
place now where it is important. It could probably use XPT_GDEV_TYPE
instead for more accurate protocol information, but let it live for now.
Reported by: matthew
MFC after: 3 days
until transport will do some probe actions (at least soft reset).
Make ATA/SATA SIMs to not report bogus and confusing PROTO_ATA protocol.
Make ATA/SATA transport to fill that gap by reporting protocol to SIM with
XPT_SET_TRAN_SETTINGS and patching XPT_GET_TRAN_SETTINGS results if needed.
to allow drivers to handle request completion directly without passing
them to the CAM SWI thread removing extra context switch.
Modify all ATA/SATA drivers to use them.
Reviewed by: gibbs, ken
MFC after: 2 weeks
interface supported by mvs(4) are 88SX, while AHCI-like chips are 88SE.
PR: kern/165271
Submitted by: Jia-Shiun Li <jiashiun@gmail.com>
MFC after: 1 week
to known AHCI-capable chips (AMD/NVIDIA), configured for legacy emulation.
Enabled by default to get additional performance and functionality of AHCI
when it can't be enabled by BIOS. Can be disabled to honor BIOS settings if
needed for some reason.
MFC after: 1 month
to kern/subr_bus.c. Simplify this function so that it no longer
depends on malloc() to execute. Identify a few other places where
it makes sense to use device_delete_all_children().
MFC after: 1 week
completely skipping them, create ahcich devices for them to allocate unit
numbers, but mark them as disabled to prevent driver probe and attach.
Last time some BIOSes tend to report unused channels as "not implemented".
This change makes ahcichX devices numbering consistent, independently of
connected disks. It makes per-channel driver hints usable and CAM devices
wiring possible on such systems.
Mac with this chipset does not initialize AHCI mode unless it is started
from EFI loader. However, legacy ATA mode works.
Submitted by: jkim@ (original version)
Approved by: re (kib)
MFC after: 1 week
Slot field of the PxCMD register may point to an empty command slot.
That breaks command timeout detection logic, making impossible to find
what command actually caused timeout, and leading to infinite wait.
Workaround that by checking whether pointed command slot is really used
and can timeout in its time. And if not, fallback to the dumb algorithm
used with FBS -- let all commands to time out and then fail all of them.
Approved by: re (kib)
MFC after: 1 week
without waiting for device readiness (or at least not updating FIS receive
area in time). To workaround that, special quirk was added earlier to wait
for the FIS receive area update. But it was found that under same PCI ID
0x91231b4b and revision 0x11 there are two completely different chip
versions (firmware?): HBA and RAID. The problem is that RAID version in
some cases, such as hot-plug, does not update FIS receive area at all!
To workaround that, differentiate the chip versions by their capabilities,
and, if RAID version found, skip FIS receive area update waiting and read
device signature from the PxSIG register instead. This method doesn't work
for HBA version when PMP attached, so keep using previous workaround there.
(SEMB) is unable to communicate to Storage Enclosure Processor (SEP), in
response to hard and soft resets it should among other things return value
0x7F in Status register. The weird side is that it means DRQ bit set, which
tells that reset request is not completed. It would be fine if SEMB was the
only device on port. But if SEMB connected to PMP or built into it, it may
block access to other devices sharing same SATA port.
Make some tunings/fixes to soft-reset handling to workaround the issue:
- ahci(4): request CLO on the port after soft reset to ignore DRQ bit;
- siis(4): gracefully reinitialize port after soft reset timeout (hardware
doesn't detect reset request completion in this case);
- mvs(4): if PMP is used, send dummy soft-reset to the PMP port to make it
clear DRQ bit for us.
For now this makes quirks in ata_pmp.c, hiding SEMB ports of SiI3726/SiI4726
PMPs, less important. Further, if hardware permit, I hope to implement real
SEMB support.
When supported by hardware, this allows to control per-port activity, locate
and fault LEDs via the led(4) API for localization and status reporting
purposes. Supporting AHCI controllers may transmit that information to the
backplane controllers via SGPIO interface. Backplane controllers interpret
received statuses in some way (IBPI standard) to report them using present
indicators.
- use ATA_SE_EXCHANGED (SError.DIAG.X) bit to detect hot-plug events when
power-management enabled and ATA_SE_PHY_CHANGED (SError.DIAG.N) can't be
trusted;
- on controllers supporting staggered spin-up (SS) put unused channels
into Listen state instead of Off. It should still save some power, but
allow plug-in events to be detected;
- on controllers supporting cold presence detection (CPD), when power
management enabled, use CPD events to detect hot-plug in addition to PHY
events.
- make SATA SIMs announce capabilities to handle SDB with Notification bit;
- make PMP driver honor this SIMs capability;
- make SATA XPT to negotiate and enable this feature for ATAPI devices.
This feature allows supporting SATA ATAPI devices to inform system about
some events happened, that may require attention. In my case this allows
LG GH22LS50 SATA DVR-RW drive to report tray open/close events. Events
reported to CAM in form of AC_SCSI_AEN async. Further they could be used
as a hints for checking device status and reporting media change to upper
layers, for example, via spoiling mechanism of GEOM.
Instead of spinning in a tight loop for up to 15 seconds, polling for device
readiness while it spins up, return reset completion just after PHY reports
"connect well" or 100ms connection timeout. If device was found, use callout
for checking device readiness with 100ms period up to full 31 second timeout.
This fixes system freeze for 5-10 seconds on drives hot plug-in.
active I/O to several disks (copying large file on ZFS) causes timeout after
just a few seconds of run. Single port 88SX6111 seems like not affected.
Skip reading transferred bytes count for these controllers. It works for
88SX6111, but 88SX6145 always returns zero there. Haven't tested others,
but better to be safe.
- SMBus Controller
- SATA Controller
- HD Audio Controller
- Watchdog Controller
Thanks to Seth Heasley (seth.heasley@intel.com) for providing us code.
MFC after 3 days
These controllers consist of two Marvell 88SE9128 6Gbps SATA chips and
PLX PCIe bridge. As result, they seem to be agree to work with ahci(4)
as usual HBAs. The only noticed issue is that RAID BIOS disables all
drive caches during boot, though `camcontrol cmd ...` is able to fix that.
Those who wants RAID functionality can still use closed proprietary driver
from HighPoint site.
MFC after: 1 week
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.
With FBS enabled, we have no idea what command caused timeout.
Implement same logic as in siis(4) - wait for other commands
complete or timeout and then give some more time.
- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.
Limit early revisions from 6Gb/s to 3Gb/s by default, or they negotiate
only 1.5Gbps, when 3Gb/s devices connected.
Add dummy driver for PATA part of these controllers, preventing generic
driver attach them. It causes system freeze when SATA controller used after
PATA was touched.
register, nVidia chipsets have different oppinion, requiring every interrupt
to be acknowledged there.
While there, add interrupt descriptions in multi-vector MSI mode.
causes additional MSIs messages sent if several ports asked for attention
same time. Time window before clearing is not important, as these interrupts
are level triggered by interrupt source.
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.
and Marvell handled. Instead of trying to attach two different drivers to
single device, wrapping each call, make one of them (atajmicron, atamarvell)
attach do device solely, but create child device for AHCI driver,
passing it all required resources. It is quite easy, as none of
resources are shared, except IRQ.
As result, it:
- makes drivers operation more independent and straitforward,
- allows to use new ahci(4) driver with such devices, adding support for
new features, such as PMP and NCQ, same time keeping legacy PATA support,
- will allow to just drop old ataahci driver, when it's time come.
- Move tagged queueing control from ADA to ATA XPT. It allows to control
device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.
- Handle timeouts and fatal errors with port hard-reset. The rest of
recovery will be done by XPT on receiving async event. More gracefull
per-device soft-reset recovery can be implemented later.
- Add workaround for ATI SB600/SB700 PMP probe related bug, to speedup boot.
On error, freeze device queue, to allow periph driver to do proper recovery.
Freeze SIM queue only in some cases, when it is needed to protect SIM.
Implement better command timeout detection logic for non-queued commands.
This fixes false positives when command with short timeout waiting for the
long one. For example, when hald tastes CD during burning process.
Read and clear SERR register on interrupt.
modularize it so that new transports can be created.
Add a transport for SATA
Add a periph+protocol layer for ATA
Add a driver for AHCI-compliant hardware.
Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.
The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.
ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.
This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.
Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.
The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.
Submitted by: scottl, mav
Approved by: re
