Add the BIO_ORDERED flag for struct bio and update bio clients to use it.
The barrier semantics of bioq_insert_tail() were broken in two ways:
o In bioq_disksort(), an added bio could be inserted at the head of
the queue, even when a barrier was present, if the sort key for
the new entry was less than that of the last queued barrier bio.
o The last_offset used to generate the sort key for newly queued bios
did not stay at the position of the barrier until either the
barrier was de-queued, or a new barrier (which updates last_offset)
was queued. When a barrier is in effect, we know that the disk
will pass through the barrier position just before the
"blocked bios" are released, so using the barrier's offset for
last_offset is the optimal choice.
sys/geom/sched/subr_disk.c:
sys/kern/subr_disk.c:
o Update last_offset in bioq_insert_tail().
o Only update last_offset in bioq_remove() if the removed bio is
at the head of the queue (typically due to a call via
bioq_takefirst()) and no barrier is active.
o In bioq_disksort(), if we have a barrier (insert_point is non-NULL),
set prev to the barrier and cur to it's next element. Now that
last_offset is kept at the barrier position, this change isn't
strictly necessary, but since we have to take a decision branch
anyway, it does avoid one, no-op, loop iteration in the while
loop that immediately follows.
o In bioq_disksort(), bypass the normal sort for bios with the
BIO_ORDERED attribute and instead insert them into the queue
with bioq_insert_tail(). bioq_insert_tail() not only gives
the desired command order during insertion, but also provides
barrier semantics so that commands disksorted in the future
cannot pass the just enqueued transaction.
sys/sys/bio.h:
Add BIO_ORDERED as bit 4 of the bio_flags field in struct bio.
sys/cam/ata/ata_da.c:
sys/cam/scsi/scsi_da.c
Use an ordered command for SCSI/ATA-NCQ commands issued in
response to bios with the BIO_ORDERED flag set.
sys/cam/scsi/scsi_da.c
Use an ordered tag when issuing a synchronize cache command.
Wrap some lines to 80 columns.
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
sys/geom/geom_io.c
Mark bios with the BIO_FLUSH command as BIO_ORDERED.
Sponsored by: Spectra Logic Corporation
MFC after: 1 month
by timeout/error of one of probe commands, process may continue infinitely.
Make CAM ATA more robust to faulty devices and false positive detections,
abort probe after two restarts on timeouts or ten on other errors.
This makes things easier for target implementations to calculate how many
elements they need to allocate.
Discussed with: mjacob, gibbs
MFC after: 1 week
the case of immediate unconfigure after configure. Hold the periph an
extra count while we have the task to create sysctl context outstanding
so that the periph doesn't go away unexpectedly.
Sponsored by: Panasas
Reviewed by: scsi@
MFC after: 1 month
better devices. This can be disabled on a per-device basis using quirks as
well.
This also handles the case where there is actually no connected LUN 0
(which can definitely be the case for storage arrays).
Reviewed by: scsi@
MFC after: 1 month
for REPORT and SET TARGET PORT GROUP commands (foundations for future work).
Regularize opcodes to be upper case hex.
Pick *one* of tab or space after #define (tab) and stick with that.
MFC after: 2 weeks
whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN).
It's less resource comsumptive than scanning a whole bus when the
caller knows only one target has changes.
Reviewed by: scsi@
Sponsored by: Panasas
MFC after: 1 month
hook it up to ada(4) also. While at it, rename *ad_firmware_geom_adjust()
to *ata_disk_firmware_geom_adjust() etc now that these are no longer
limited to ad(4).
Reviewed by: mav
MFC after: 3 days
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.
As scottl@ noticed, max_target/max_lun was intended to be only a hint for
existing bus scanner. Some FC/SAS SIMs report fake values there, that are
smaller then maximum supported IDs. In that case this check makes impossible
manual scan outside hinted range.
For ATA/SATA SIMs respective check was instead implemented at SIM level.
Newer SCSI SIMs expected to have these checks at driver or firmware level.
Some older SCSI SIMs have no this check and the issues will get back there.
d_mediasize
CD_FLAG_VALID_TOC is not required for setting those media properties.
PR: kern/145385
Submitted by: Juergen Lock <nox@jelal.kn-bremen.de>
a slightly different version
Tested by: Pavel Sukhoy <sukhoy@ripn.net>,
Markus Wild <m.wild@cybernet.ch>,
Juergen Lock <nox@jelal.kn-bremen.de>,
uqs
MFC after: 1 week
xpt_done for queued requests. This solves the problem of
indefinite hangs for unspecified transports when XPT_SCAN_BUS
is called.
A few minor cosmetics elsewhere.
MFC after: 1 week
Add Power Up In Stand-by feature support. Device with PUIS enabled
require explicit command to do initial spin-up. Mark that command
with CAM_HIGH_POWER flag, to allow CAM manage staggered spin-up.
Make CAM to stop all attached devices on system shutdown.
It allows devices to park heads, reducing stress on power loss.
Add `kern.cam.power_down` tunable and sysctl to controll it.
- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.
- For SSDs use TRIM feature of DATA SET MANAGEMENT command, as defined by
ACS-2 specification working draft.
- For CompactFlash use CFA ERASE command, same as ad(4) does.
With this patch, `newfs -E /dev/ada1` was able to restore write speed of
my heavily weared OCZ Vertex SSD (firmware 1.4) up to the initial level
for the most part of it's capacity. Previous 1.3 firmware, even reportiong
TRIM capabilty bit set, was not working, reporting ABORT error for every
DSM command.
I have no idea whether it is normal, but for some reason it takes 200ms
to handle any TRIM command on this drive, that was making delete extremely
slow. But TRIM command is able to accept long list of LBAs and the length of
that list seems doesn't affect it's execution time. Implemented request
clusting algorithm allowed me to rise delete rate up to reasonable numbers,
when many parallel DELETE requests running.
- Cleanup kernel messages, mostly PMP.
- Took references on devices, while PMP reinitializes them, to not let them
go and distort freeze reference counting.
Introduce ATA_CAM kernel option, turning ata(4) controller drivers into
cam(4) interface modules. When enabled, this options deprecates all ata(4)
peripheral drivers (ad, acd, ...) and interfaces and allows cam(4) drivers
(ada, cd, ...) and interfaces to be natively used instead.
As side effect of this, ata(4) mode setting code was completely rewritten
to make controller API more strict and permit above change. While doing
this, SATA revision was separated from PATA mode. It allows DMA-incapable
SATA devices to operate and makes hw.ata.atapi_dma tunable work again.
Also allow ata(4) controller drivers (except some specific or broken ones)
to handle larger data transfers. Previous constraint of 64K was artificial
and is not really required by PCI ATA BM specification or hardware.
Submitted by: nwitehorn (powerpc part)
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.
Remove code that years ago was closing race between request submission
to SIM and device/SIM freeze. That race become impossible after moving from
spl to mutex locking, while this workaround causes some unexpected effects.
- Move tagged queueing control from ADA to ATA XPT. It allows to control
device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.
- Remove CAM_PERIPH_POLLED flag. It is broken by design. Polling can't be
periph flag. May be SIM, may be CCB, but now it works fine just without it.
- Remove check unused for at least five years. If we will ever have non-BIO
devices in CAM, this check is smallest of what we will need.
- If several controllers complete requests same time, call swi_sched()
only once.
- Add support for sector size > 512 bytes and physical sector of several
logical sectors, introduced by ATA-7 specification.
- Remove some obsoleted code.
disk is first probed. dagetcapacity is called whenever the disk is opened from
geom via d_open(), a zero sector size will cause geom to panic later on.
Fix reference counting bug, when device unreferenced before then
invalidated. To do it, do not handle validity flag as another
reference, but explicitly modify reference count each time flag is
modified.
Discovered by: thompsa
- Reduce code duplication in ATA XPT and PMP driver.
- Move PIO size setting from ada driver to ATA XPT. It is XPT business
to negotiate transfer details. ada driver is now stateless.
- Report PIO size to SIM. It is required for correct PATA SIM operation.
- Tune PMP scan timings. It workarounds some problems with SiI.
- If reset hapens during PMP initialization - restart it.
- Introduce early-initialized periph drivers, which are used during initial
scan process. Use it for xpt, probe, aprobe and pmp. It gives pmp chance
to finish scan before mountroot and numerate devices in right order.
Ensure target/lun passed from user-level supported on this bus.
Scanning unsupported IDs causes different issues from duplicate
devices to system crash.
Separate CAM_DEV_IDENTIFY_DATA_VALID flag from CAM_DEV_INQUIRY_DATA_VALID.
Add workaround for very old devices without support for mode setting.
Add some PATA bus scanning support.
Remove some SCSIsms.
Add support for PIO-only devices.
Fix maxio values and 256 sectors transactions for 28bits commands.
Implement periodic ordered commands insertion, sames as da driver does.
Remove some SCSIsms.
allocations M_NOWAIT so that we don't try and sleep with a nested non-sleepable
lock.
This makes the userland scsi_target begin to function again.
Obtained from: Sean Bruno
MFC after: 1 month
for ATA_SETFEATURES/ATA_SF_SETXFER command which by definition transfers no
data. Most of controllers are irrelevant to this bug, but some nVidia's
doesn't.
Tested on: current@
Approved by: re (kib)
firmware loading bugs.
Target mode support has received some serious attention to make it
more usable and stable.
Some backward compatible additions to CAM have been made that make
target mode async events easier to deal with have also been put
into place.
Further refinement and better support for NP-IV (N-port Virtualization)
is now in place.
Code for release prior to RELENG_7 has been stripped away for code clarity.
Sponsored by: Copan Systems
Reviewed by: scottl, ken, jung-uk kim
Approved by: re
page_list using the matching malloc type for the allocation.
Approved by: re
Reviewed by: scottl [1]
MFC after: 1 week
[1] Original patch was against xpt_cam.c, prior to the cam refactoring.
modularize it so that new transports can be created.
Add a transport for SATA
Add a periph+protocol layer for ATA
Add a driver for AHCI-compliant hardware.
Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.
The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.
ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.
This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.
Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.
The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.
Submitted by: scottl, mav
Approved by: re
Note that this does not actually enable full-range i/o requests for
64 architectures, and is done now to update KBI only.
Tested by: pho
Reviewed by: jhb, bde (as part of the review of the bigger patch)
int. All of its callers pass in cmd as a u_long, so this has
always been a dangerous type demotion. It was spooted by clang/llvm
trying to do a type promotion and sign extension within
cam_periph_ioctl.
Submitted by: rdivacky
vnode interlock to protect the knote fields [1]. The locking assumes
that shared vnode lock is held, thus we get exclusive access to knote
either by exclusive vnode lock protection, or by shared vnode lock +
vnode interlock.
Do not use kl_locked() method to assert either lock ownership or the
fact that curthread does not own the lock. For shared locks, ownership
is not recorded, e.g. VOP_ISLOCKED can return LK_SHARED for the shared
lock not owned by curthread, causing false positives in kqueue subsystem
assertions about knlist lock.
Remove kl_locked method from knlist lock vector, and add two separate
assertion methods kl_assert_locked and kl_assert_unlocked, that are
supposed to use proper asserts. Change knlist_init accordingly.
Add convenience function knlist_init_mtx to reduce number of arguments
for typical knlist initialization.
Submitted by: jhb [1]
Noted by: jhb [2]
Reviewed by: jhb
Tested by: rnoland
The device index number stored in the unit number of sa(4) devices is
only used to print debug messages. Get rid of this index number and use
devtoname() to just print the entire device name.
a serial number, fall through to the next case so that initial negotiation
still happens. Without this, devices were showing up with only 1 available
tag opening, leading to observations of very poor I/O performance.
This should fix problems reported with VMWare Fusion and ESX. Early
generation MPT-SAS controllers with SATA disks might also be affected.
HP CISS controllers are also likely affected, as are many other
pseudo-scsi disk subsystems.
things around so the periph destructors look alike. Based on a patch
by Jaakko Heinonen.
Submitted by: Jaakko Heinonen
Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
created by atapicam is being kept opened or mounted. This is probably just
a temporary solution until we invent something better.
Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
Reported by: Jaakko Heinonen
This fixes problems with discovering some USB devices that are very slow to
respond during initialisation.
When a USB device is inserted, CAM performs the sequence:
1) INQUIRY
2) INQUIRY (second time with other parameters)
3) TEST UNIT READY
4) READ CAPACITY
Before this change CAM didn't check if TEST UNIT READY was successful and went
on blindly to the next state and sent READ CAPACITY. If the device was still
not ready by then, CAM ended with error message. This patch adds checking for the
status of TEST UNIT READY command and retrying up to 10 times with 0.5 sec
interval.
Submitted by: Grzegorz Bernacki gjb ! semihalf dot com
Reviewed by: scottl
When trying to read scratched or damaged CDs and DVDs, the default
mechanism is sub-optimal. Programs like ddrescue do much better if
you turn off retries entirely, since their algorithms are designed
scan big areas fast, then winnow the areas down. Turning off retries
speeds these programs up by as much as 20x, since the drive is able to
'stream past' many small errors...
The sysctl/tunable kern.cam.cd.retry_count controls this. That
defaults to '4' (for a total of 5 attempts). Setting to 0 turns off
all retry attempts.
Reviewed by: scottl@
does - in DragonFly, it's cam_sim_release() what actually frees the
SIM; cam_sim_free does nothing more than calling cam_sim_release().
Here, we drain in cam_sim_free, waiting for refcount to drop to zero.
We cannot do the same think DragonFly does, because after cam_sim_free
returns, client would destroy the sim->mtx, and CAM would trip over
an initialized mutex.
Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
to actually use it would panic on mtx operation, as dead_sim doesn't
have a proper mutex. Even if it had a properly initialized mutex,
it wouldn't have properly locked and owned one.
Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
After I removed all the unit2minor()/minor2unit() calls from the kernel
yesterday, I realised calling minor() everywhere is quite confusing.
Character devices now only have the ability to store a unit number, not
a minor number. Remove the confusion by using dev2unit() everywhere.
This commit could also be considered as a bug fix. A lot of drivers call
minor(), while they should actually be calling dev2unit(). In -CURRENT
this isn't a problem, but it turns out we never had any problem reports
related to that issue in the past. I suspect not many people connect
more than 256 pieces of the same hardware.
Reviewed by: kib
When I changed kern_conf.c three months ago I made device unit numbers
equal to (unneeded) device minor numbers. We used to require
bitshifting, because there were eight bits in the middle that were
reserved for a device major number. Not very long after I turned
dev2unit(), minor(), unit2minor() and minor2unit() into macro's.
The unit2minor() and minor2unit() macro's were no-ops.
We'd better not remove these four macro's from the kernel, because there
is a lot of (external) code that may still depend on them. For now it's
harmless to remove all invocations of unit2minor() and minor2unit().
Reviewed by: kib
because the media was removed, the periph would get its refcount dropped
and ultimately freed before getting unlocked. This created a dangling
pointer that was easy to trip over. This fixes a common source of
crashes with removaable media, but problems remain and will get tracked
down.
http://www.t10.org/lists/1spc-lst.htm
Note opcodes for scanner and communication devices are taken from the previous
revision because they are not listed in the files any more.
Also, note newly added ASCs are all marked with 'XXX TBD' and take SS_RDEF action
for now. Some ASCs need SS_TUR for error recovery or SS_FATAL to prevent further
retrials. We should deal with them later.
Reviewed by: scottl, ken
the referenced data is only obtained/changed in the device open handler,
and the ioctl handler can only run after the open handler. Also fix a
few nearby style issues.
Submitted by: Matt Jacob
device supports retrieving a serial number. Instead, first query the
list of VPD pages it does support, and only query the serial number if
it's supported, else silently move on. This eliminates a lot of noise
during verbose booting, and will likely eliminate the need for most
NOSERIAL quirks.
for that argument. This will allow DDB to detect the broad category of
reason why the debugger has been entered, which it can use for the
purposes of deciding which DDB script to run.
Assign approximate why values to all current consumers of the
kdb_enter() interface.
to kproc_xxx as they actually make whole processes.
Thos makes way for us to add REAL kthread_create() and friends
that actually make theads. it turns out that most of these
calls actually end up being moved back to the thread version
when it's added. but we need to make this cosmetic change first.
I'd LOVE to do this rename in 7.0 so that we can eventually MFC the
new kthread_xxx() calls.
now takes a device_t to be the parent of the bus that is being created.
Most SIMs have been updated with a reasonable argument, but a few exceptions
just pass NULL for now. This argument isn't used yet and the newbus
integration likely won't be ready until after 7.0-RELEASE.
sysctl_handle_int is not sizeof the int type you want to export.
The type must always be an int or an unsigned int.
Remove the instances where a sizeof(variable) is passed to stop
people accidently cut and pasting these examples.
In a few places this was sysctl_handle_int was being used on 64 bit
types, which would truncate the value to be exported. In these
cases use sysctl_handle_quad to export them and change the format
to Q so that sysctl(1) can still print them.
use to synchornize and protect all data objects that are used for that
SIM. Drivers that are not yet MPSAFE register Giant and operate as
usual. RIght now, no drivers are MPSAFE, though a few will be changed
in the coming week as this work settles down.
The driver API has changed, so all CAM drivers will need to be recompiled.
The userland API has not changed, so tools like camcontrol do not need to
be recompiled.
Linux SCSI SG passthrough device API. The intention is to allow for both
running of Linux apps that want to talk to /dev/sg* nodes, and to facilitate
porting of apps from Linux to FreeBSD. As such, both native and linuxolator
entry points and definitions are provided.
Caveats:
- This does not support the procfs and sysfs nodes that the Linux SG
driver provides. Some Linux apps may rely on these for operation,
others may only use them for informational purposes.
- More ioctls need to be implemented.
- Linux uses a naming scheme of "sg[a-z]" for devices, while FreeBSD uses a
scheme of "sg[0-9]". Devfs aliasis (symlinks) are automatically created
to link the two together. However, tools like camcontrol only see the
native names.
- Some operations were originally designed to return byte counts or other
data directly as the syscall return value. The linuxolator doesn't appear
to support this well, so this driver just punts for these cases.
Now that the driver is in place, others are welcome to add missing
functionality. Thanks to Roman Divacky for pushing this work along.
rescan requests. The purpose of this is to allow a SIM
(or other entities) to request a bus rescan and have it
then fielded in a different (process) context from the
caller.
There are probably better ways to accomplish this, but
it's a very small change that helps solve a number of
problems.
Reviewed by: Justin, Ken and Scott.
MFC after: 2 weeks
flash card reader.
Also remove an 'Opened da0 -> <random number>' which is not needed on a daily
basis (available through bootverbose).
Reviewed by: phk, ken
MFC after: 1 week
author can't remember why it was there.
The CTS_SCSI_FLAGS_TAG_ENB remains in place, and makes sense, and is
checked all over the place.
The CTS_SPI_FLAGS_TAG_ENB was probably an attempt to distinguish
protocol and transport tag capabilities. At the very least this can
be confusing and prone to many bugs, so let's just assume that the
transport tag case just flows from the protocol (and vice versa)
for now.
the ORDERED tag. This recoups significant performance gains for many
arrays.
The default is still to send out the ORDERED tag periodically.
Reviewed by: scsi (justin+timeout)
