Commit Graph

86 Commits

Author SHA1 Message Date
Justin T. Gibbs
f03f7a0ca3 Correct bioq_disksort so that bioq_insert_tail() offers barrier semantic.
Add the BIO_ORDERED flag for struct bio and update bio clients to use it.

The barrier semantics of bioq_insert_tail() were broken in two ways:

 o In bioq_disksort(), an added bio could be inserted at the head of
   the queue, even when a barrier was present, if the sort key for
   the new entry was less than that of the last queued barrier bio.

 o The last_offset used to generate the sort key for newly queued bios
   did not stay at the position of the barrier until either the
   barrier was de-queued, or a new barrier (which updates last_offset)
   was queued.  When a barrier is in effect, we know that the disk
   will pass through the barrier position just before the
   "blocked bios" are released, so using the barrier's offset for
   last_offset is the optimal choice.

sys/geom/sched/subr_disk.c:
sys/kern/subr_disk.c:
	o Update last_offset in bioq_insert_tail().

	o Only update last_offset in bioq_remove() if the removed bio is
	  at the head of the queue (typically due to a call via
	  bioq_takefirst()) and no barrier is active.

	o In bioq_disksort(), if we have a barrier (insert_point is non-NULL),
	  set prev to the barrier and cur to it's next element.  Now that
	  last_offset is kept at the barrier position, this change isn't
	  strictly necessary, but since we have to take a decision branch
	  anyway, it does avoid one, no-op, loop iteration in the while
	  loop that immediately follows.

	o In bioq_disksort(), bypass the normal sort for bios with the
	  BIO_ORDERED attribute and instead insert them into the queue
	  with bioq_insert_tail().  bioq_insert_tail() not only gives
	  the desired command order during insertion, but also provides
	  barrier semantics so that commands disksorted in the future
	  cannot pass the just enqueued transaction.

sys/sys/bio.h:
	Add BIO_ORDERED as bit 4 of the bio_flags field in struct bio.

sys/cam/ata/ata_da.c:
sys/cam/scsi/scsi_da.c
	Use an ordered command for SCSI/ATA-NCQ commands issued in
	response to bios with the BIO_ORDERED flag set.

sys/cam/scsi/scsi_da.c
	Use an ordered tag when issuing a synchronize cache command.

	Wrap some lines to 80 columns.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
sys/geom/geom_io.c
	Mark bios with the BIO_FLUSH command as BIO_ORDERED.

Sponsored by:	Spectra Logic Corporation
MFC after:	1 month
2010-09-02 19:40:28 +00:00
Edward Tomasz Napierala
7ce513a52a Untangle g_print_bio(), silencing Coverity.
Found with:	Coverity Prevent
CID:		3566, 3567
2010-06-10 17:49:36 +00:00
Andriy Gapon
2a842317eb g_io_check: respond to zero pp->mediasize with ENXIO
Previsouly this condition was reported with EIO by bio_offset > mediasize
check.
Perhaps that check should be extended to bio_offset+bio_length > mediasize.

MFC after:	1 week
2010-04-15 08:39:56 +00:00
Alexander Motin
a5be8eb530 Do not fetch precise time of request start when stats collection disabled.
Reviewed by:	pjd, phk
2010-03-24 18:04:25 +00:00
Alexander Motin
0d883b11e3 Call wakeup() only for the first request on the queue. 2009-12-30 17:23:27 +00:00
Alexander Motin
7fc019af65 MFp4:
Remove msleep() timeout from g_io_schedule_up/down(). It works fine
without it, saving few percents of CPU on high request rates without
need to rearm callout twice per request.
2009-09-06 19:33:13 +00:00
Edward Tomasz Napierala
fb231f3627 Make gjournal work with kernel compiled with "options DIAGNOSTIC".
Previously, it would panic immediately.

Reviewed by:	pjd
Approved by:	re (kib)
2009-06-30 14:34:06 +00:00
Luigi Rizzo
6231f75bcf As discussed in the devsummit, introduce two fields in the
struct bio to store classification information, and a hook
for classifier functions that can be called by g_io_request().

This code is from Fabio Checconi as part of his GSOC work.
2009-06-11 09:55:26 +00:00
Sean Bruno
c4901b6798 Just a fixup for a KTRACE message I stumbled upon many moons ago.
Reviewed by:	Scott Long
MFC after:	2 days
2008-09-18 15:02:19 +00:00
Poul-Henning Kamp
eed6cda966 Don't limit BIO_DELETE requests to MAXPHYS, they perform no data
transfers, so they are not subject to the VM system limitation.
2007-12-16 18:03:31 +00:00
Pawel Jakub Dawidek
b656c1b836 Save stack only when KTR_GEOM is both compiled into the kernel and enabled
in debug.ktr.mask. Because saving stack is very expensive, it's better only
to do it when one really wants to.

Reported by:	Dan Nelson
2007-10-26 06:55:00 +00:00
Pawel Jakub Dawidek
2b17fb9514 Implement g_delete_data() similar to g_read_data() and g_write_data().
OK'ed by:	phk
2007-05-05 16:35:22 +00:00
John Baldwin
4d70511ac3 Use pause() rather than tsleep() on stack variables and function pointers. 2007-02-27 17:23:29 +00:00
John Baldwin
6e50e38fcc Use tsleep() rather than msleep() with a NULL mtx parameter. 2007-02-23 23:06:10 +00:00
Pawel Jakub Dawidek
1ded77b222 We expect 'bio_data != NULL' for BIO_{READ,WRITE,GETATTR}, but for
BIO_{DELETE,FLUSH} we expect 'bio_data == NULL'.

Reviewed by:	phk
2007-01-28 23:36:07 +00:00
Pawel Jakub Dawidek
c3618c657a Add a new I/O request - BIO_FLUSH, which basically tells providers below to
flush their caches. For now will mostly be used by disks to flush their
write cache.

Sponsored by:	home.pl
2006-10-31 21:11:21 +00:00
Pawel Jakub Dawidek
4bec0ff1c4 Add g_duplicate_bio() function which does the same thing what g_clone_bio()
is doing, but g_duplicate_bio() allocates new bio with M_WAITOK flag.
2006-06-05 21:13:22 +00:00
Ruslan Ermilov
ad5722357f Fix a typo. 2006-03-13 14:59:57 +00:00
Pawel Jakub Dawidek
92ee312dd4 Assert proper use of bio_caller1, bio_caller2, bio_cflags, bio_driver1,
bio_driver2 and bio_pflags fields.

Reviewed by:	phk
2006-03-01 19:01:58 +00:00
John Baldwin
51460da87f - Add a new simple facility for marking the current thread as being in a
state where sleeping on a sleep queue is not allowed.  The facility
  doesn't support recursion but uses a simple private per-thread flag
  (TDP_NOSLEEPING).  The sleepq_add() function will panic if the flag is
  set and INVARIANTS is enabled.
- Use this new facility to replace the g_xup and g_xdown mutexes that were
  (ab)used to achieve similar behavior.
- Disallow sleeping in interrupt threads when invoking interrupt handlers.

MFC after:	1 week
Reviewed by:	phk
2005-09-15 19:05:37 +00:00
Pawel Jakub Dawidek
3b37814794 Use KTR to log allocations and destructions of bios.
This should hopefully allow to track down "duplicate free of g_bio" panics.
2005-08-29 11:39:24 +00:00
Poul-Henning Kamp
8827c821de By design I left a tiny race in updating the I/O statistics based on
the assumption that performance was more important that beancounter
quality statistics.

As it transpires the microoptimization is not measurable in the
real world and the inconsistent statistics confuse users, so revert
the decision.

MT6 candidate:	possibly
MT5 candidate:	possibly
2005-07-25 21:12:54 +00:00
Robert Watson
49dbb61dfc Add KTR_GEOM, which allows tracing of basic GEOM I/O events occuring
in the g_up and g_down threads.  Each time a bio is propelled up and
down the stack, an event is generating showing the provider, offset,
and length, as well as thread wakeup and work status information.
2004-10-21 18:35:24 +00:00
Stephan Uphoff
f7717523a2 Trace information about a buffer while we still control it.
Reviewed by:    phk
Approved by:    sam (mentor)
2004-10-11 21:22:59 +00:00
Poul-Henning Kamp
276f72c550 Don't set the BIO_ONQUEUE debugging flag until we actually put the bio
onto a queue.  This made the ENOMEM handling an instant panic.
2004-10-06 20:59:59 +00:00
Poul-Henning Kamp
19fa21aa50 Protect the start/end counts on consumers and providers with the up/down
mutexes.

Make it possible to also protect the disk statistics (at a minor cost in
performance) by setting bit 2 of kern.geom.collectstats.
2004-09-28 11:56:37 +00:00
Pawel Jakub Dawidek
8dd5480d29 - Set maximum request size to MAXPHYS (128kB), instead of DFLPHYS (64kB).
- Set minimum request size to sectorsize, instead of 512 bytes.

Approved by:	phk (some time ago)
2004-09-28 08:34:27 +00:00
Poul-Henning Kamp
dcbd0fe5aa Add more KASSERTS and checks. 2004-08-30 09:33:06 +00:00
Poul-Henning Kamp
a2033c9615 Introduce g_alloc_bio() as a waiting variant of g_new_bio().
Use in places where we can sleep and where we previously failed to check
for a NULL pointer.

MT5 candidate.
2004-08-27 14:43:11 +00:00
Pawel Jakub Dawidek
1b949c05a3 When sending request once again because of ENOMEM, reset bio_children
and bio_inbed fields to 0. Without this change we can end up with
I/O leakage in some rare situations.
I tested this change by putting failure probability mechanism simlar
to this used in NOP class into g_clone_bio(9) function, so it was
able to return NULL with the given probability.

Discussed with:	phk
2004-08-11 12:04:35 +00:00
Robert Watson
5706472c3a The g_up and g_down threads use a local 'mymutex' mutex to allow WITNESS
to warn about attempts to sleep in the I/O path.  This change pushes the
definition and use of 'mymutex' behind #ifdef WITNESS to avoid the cost
in non-debugging cases.  This results in a clear .22% performance win for
512 byte and 1k I/O tests on my SMP test box.  Not much, but every bit
counts.
2004-06-26 23:27:42 +00:00
Poul-Henning Kamp
cf4572847a Make the sysctl kern.geom.collectstats more granular.
Bit 0 controls statistics collection on GEOM providers.
Bit 1 controls statistics collection on GEOM consumers.

Default value is 1.

Prodded by:	scottl
2004-06-09 19:44:44 +00:00
Pawel Jakub Dawidek
46aeebec57 Calculate bio_completed properly or die!
Approved by:	phk
2004-04-04 20:37:28 +00:00
Pawel Jakub Dawidek
72e330954e Added g_print_bio() function to print informations about given bio.
Approved by:	phk, scottl (mentor)
2004-02-11 18:21:32 +00:00
Poul-Henning Kamp
5fcf4e4398 Bring back the geom_bioqueues, they _are_ a good idea.
ATA will uses these RSN.
2004-01-28 08:39:18 +00:00
Don Lewis
2cf0d8a6ef Correct usage of mtx_init() API. This is not a functional change since
the code happened to work because MTX_DEF and NULL are both defined as 0.

Reviewed by:	phk
2003-12-07 23:20:53 +00:00
Poul-Henning Kamp
43bff1a7ab Forgotten commit: If a provider has zero sectorsize, it is an
indication of lack of media.

Tripped up:	peter
2003-10-22 06:32:20 +00:00
Poul-Henning Kamp
d1b8bf476c Remove KASSERT check for negative bio_offsets, add "normal" EIO
error return for same.
2003-10-19 19:06:54 +00:00
Poul-Henning Kamp
f7eeab176c Allow our bio tools to be used for local bio-chopping by not mandating
a bio_from value.  bio_to is still mandated (mostly for debuggign) and
shall be copied from the parent bio.
2003-10-06 09:07:35 +00:00
Poul-Henning Kamp
3eb6ffdf43 Add more KASSERTS(). 2003-09-26 20:52:46 +00:00
Poul-Henning Kamp
e060b6bd03 Reorder a couple of KASSERTS to give more sensible messages.
Found by:	GEOM 101 class of '03
2003-09-11 00:49:02 +00:00
Poul-Henning Kamp
f0ffd81bc2 In case we encounter a zero sectorsize provider in g_io_check(), fail
the request with a printf rather than a divide by zero error.
2003-08-13 06:42:56 +00:00
Poul-Henning Kamp
44be139bbb Sleep on "-" in our normal state to simplify debugging. 2003-06-18 10:33:09 +00:00
David E. O'Brien
50b1faef38 Use __FBSDID().
Approved by:	phk
2003-06-11 06:49:16 +00:00
Poul-Henning Kamp
2cc9686e52 Hide the "ENOMEM" notice messages behind bootverbose. They are still
a valuable debugging tool for certain kinds of problems.

Approved by:	re/scottl
2003-05-07 05:37:31 +00:00
Poul-Henning Kamp
5ffb2c8bfd Use an uma-zone for allocation bio requests. 2003-05-02 12:36:12 +00:00
Poul-Henning Kamp
c4da4e46b2 Back out all the stuff that didn't belong in the last commit. 2003-05-02 06:42:59 +00:00
Poul-Henning Kamp
e65ab0f83f Use g_slice_spoiled() rather than g_std_spoiled().
Remember to free the buffer we got from g_read_data().
2003-05-02 06:36:14 +00:00
Poul-Henning Kamp
3924ad705e Time has run from the "run GEOM in userland" harness, and the new regression
test is built to test GEOM as running in the kernel.

This commit is basically "unifdef -D_KERNEL" to remove the mainly #include
related code to support the userland-harness.
2003-04-13 09:02:06 +00:00
Poul-Henning Kamp
5f5a9022b2 Retire the experimental bio_taskqueue(), it was not quite as usable as
hoped.  It can be revived from here, should other drivers be able to
use it.
2003-04-12 09:13:01 +00:00