Commit Graph

67 Commits

Author SHA1 Message Date
John Baldwin
51460da87f - Add a new simple facility for marking the current thread as being in a
state where sleeping on a sleep queue is not allowed.  The facility
  doesn't support recursion but uses a simple private per-thread flag
  (TDP_NOSLEEPING).  The sleepq_add() function will panic if the flag is
  set and INVARIANTS is enabled.
- Use this new facility to replace the g_xup and g_xdown mutexes that were
  (ab)used to achieve similar behavior.
- Disallow sleeping in interrupt threads when invoking interrupt handlers.

MFC after:	1 week
Reviewed by:	phk
2005-09-15 19:05:37 +00:00
Pawel Jakub Dawidek
3b37814794 Use KTR to log allocations and destructions of bios.
This should hopefully allow to track down "duplicate free of g_bio" panics.
2005-08-29 11:39:24 +00:00
Poul-Henning Kamp
8827c821de By design I left a tiny race in updating the I/O statistics based on
the assumption that performance was more important that beancounter
quality statistics.

As it transpires the microoptimization is not measurable in the
real world and the inconsistent statistics confuse users, so revert
the decision.

MT6 candidate:	possibly
MT5 candidate:	possibly
2005-07-25 21:12:54 +00:00
Robert Watson
49dbb61dfc Add KTR_GEOM, which allows tracing of basic GEOM I/O events occuring
in the g_up and g_down threads.  Each time a bio is propelled up and
down the stack, an event is generating showing the provider, offset,
and length, as well as thread wakeup and work status information.
2004-10-21 18:35:24 +00:00
Stephan Uphoff
f7717523a2 Trace information about a buffer while we still control it.
Reviewed by:    phk
Approved by:    sam (mentor)
2004-10-11 21:22:59 +00:00
Poul-Henning Kamp
276f72c550 Don't set the BIO_ONQUEUE debugging flag until we actually put the bio
onto a queue.  This made the ENOMEM handling an instant panic.
2004-10-06 20:59:59 +00:00
Poul-Henning Kamp
19fa21aa50 Protect the start/end counts on consumers and providers with the up/down
mutexes.

Make it possible to also protect the disk statistics (at a minor cost in
performance) by setting bit 2 of kern.geom.collectstats.
2004-09-28 11:56:37 +00:00
Pawel Jakub Dawidek
8dd5480d29 - Set maximum request size to MAXPHYS (128kB), instead of DFLPHYS (64kB).
- Set minimum request size to sectorsize, instead of 512 bytes.

Approved by:	phk (some time ago)
2004-09-28 08:34:27 +00:00
Poul-Henning Kamp
dcbd0fe5aa Add more KASSERTS and checks. 2004-08-30 09:33:06 +00:00
Poul-Henning Kamp
a2033c9615 Introduce g_alloc_bio() as a waiting variant of g_new_bio().
Use in places where we can sleep and where we previously failed to check
for a NULL pointer.

MT5 candidate.
2004-08-27 14:43:11 +00:00
Pawel Jakub Dawidek
1b949c05a3 When sending request once again because of ENOMEM, reset bio_children
and bio_inbed fields to 0. Without this change we can end up with
I/O leakage in some rare situations.
I tested this change by putting failure probability mechanism simlar
to this used in NOP class into g_clone_bio(9) function, so it was
able to return NULL with the given probability.

Discussed with:	phk
2004-08-11 12:04:35 +00:00
Robert Watson
5706472c3a The g_up and g_down threads use a local 'mymutex' mutex to allow WITNESS
to warn about attempts to sleep in the I/O path.  This change pushes the
definition and use of 'mymutex' behind #ifdef WITNESS to avoid the cost
in non-debugging cases.  This results in a clear .22% performance win for
512 byte and 1k I/O tests on my SMP test box.  Not much, but every bit
counts.
2004-06-26 23:27:42 +00:00
Poul-Henning Kamp
cf4572847a Make the sysctl kern.geom.collectstats more granular.
Bit 0 controls statistics collection on GEOM providers.
Bit 1 controls statistics collection on GEOM consumers.

Default value is 1.

Prodded by:	scottl
2004-06-09 19:44:44 +00:00
Pawel Jakub Dawidek
46aeebec57 Calculate bio_completed properly or die!
Approved by:	phk
2004-04-04 20:37:28 +00:00
Pawel Jakub Dawidek
72e330954e Added g_print_bio() function to print informations about given bio.
Approved by:	phk, scottl (mentor)
2004-02-11 18:21:32 +00:00
Poul-Henning Kamp
5fcf4e4398 Bring back the geom_bioqueues, they _are_ a good idea.
ATA will uses these RSN.
2004-01-28 08:39:18 +00:00
Don Lewis
2cf0d8a6ef Correct usage of mtx_init() API. This is not a functional change since
the code happened to work because MTX_DEF and NULL are both defined as 0.

Reviewed by:	phk
2003-12-07 23:20:53 +00:00
Poul-Henning Kamp
43bff1a7ab Forgotten commit: If a provider has zero sectorsize, it is an
indication of lack of media.

Tripped up:	peter
2003-10-22 06:32:20 +00:00
Poul-Henning Kamp
d1b8bf476c Remove KASSERT check for negative bio_offsets, add "normal" EIO
error return for same.
2003-10-19 19:06:54 +00:00
Poul-Henning Kamp
f7eeab176c Allow our bio tools to be used for local bio-chopping by not mandating
a bio_from value.  bio_to is still mandated (mostly for debuggign) and
shall be copied from the parent bio.
2003-10-06 09:07:35 +00:00
Poul-Henning Kamp
3eb6ffdf43 Add more KASSERTS(). 2003-09-26 20:52:46 +00:00
Poul-Henning Kamp
e060b6bd03 Reorder a couple of KASSERTS to give more sensible messages.
Found by:	GEOM 101 class of '03
2003-09-11 00:49:02 +00:00
Poul-Henning Kamp
f0ffd81bc2 In case we encounter a zero sectorsize provider in g_io_check(), fail
the request with a printf rather than a divide by zero error.
2003-08-13 06:42:56 +00:00
Poul-Henning Kamp
44be139bbb Sleep on "-" in our normal state to simplify debugging. 2003-06-18 10:33:09 +00:00
David E. O'Brien
50b1faef38 Use __FBSDID().
Approved by:	phk
2003-06-11 06:49:16 +00:00
Poul-Henning Kamp
2cc9686e52 Hide the "ENOMEM" notice messages behind bootverbose. They are still
a valuable debugging tool for certain kinds of problems.

Approved by:	re/scottl
2003-05-07 05:37:31 +00:00
Poul-Henning Kamp
5ffb2c8bfd Use an uma-zone for allocation bio requests. 2003-05-02 12:36:12 +00:00
Poul-Henning Kamp
c4da4e46b2 Back out all the stuff that didn't belong in the last commit. 2003-05-02 06:42:59 +00:00
Poul-Henning Kamp
e65ab0f83f Use g_slice_spoiled() rather than g_std_spoiled().
Remember to free the buffer we got from g_read_data().
2003-05-02 06:36:14 +00:00
Poul-Henning Kamp
3924ad705e Time has run from the "run GEOM in userland" harness, and the new regression
test is built to test GEOM as running in the kernel.

This commit is basically "unifdef -D_KERNEL" to remove the mainly #include
related code to support the userland-harness.
2003-04-13 09:02:06 +00:00
Poul-Henning Kamp
5f5a9022b2 Retire the experimental bio_taskqueue(), it was not quite as usable as
hoped.  It can be revived from here, should other drivers be able to
use it.
2003-04-12 09:13:01 +00:00
Poul-Henning Kamp
4eba52a2d2 Remove all references to BIO_SETATTR. We will not be using it. 2003-04-03 19:19:36 +00:00
Poul-Henning Kamp
376ceb799f Fix a bug in the ENOMEM pacing code which probably made it panic systems
after a lot of ENOMEM errors.
2003-03-29 22:34:37 +00:00
Poul-Henning Kamp
e24cbd9017 Retire the GEOM private statistics code and use devstat instead. 2003-03-18 09:42:33 +00:00
Poul-Henning Kamp
b4b138c27f Including <sys/stdint.h> is (almost?) universally only to be able to use
%j in printfs, so put a newsted include in <sys/systm.h> where the printf
prototype lives and save everybody else the trouble.
2003-03-18 08:45:25 +00:00
Poul-Henning Kamp
c6ae9b5fd2 Don't abuse the statistics counters for detecting if we have outstanding
I/O requests, instead use the new dedicated fields in the consumer and
provider to track this.
2003-03-09 09:59:48 +00:00
Warner Losh
a163d034fa Back out M_* changes, per decision of the TRB.
Approved by: trb
2003-02-19 05:47:46 +00:00
Poul-Henning Kamp
f0e185d705 Implement a bio-taskqueue to reduce number of context switches in
disk I/O processing.

        The intent is that the disk driver in its hardware interrupt
        routine will simply schedule the bio on the task queue with
        a routine to finish off whatever needs done.

        The g_up thread will then schedule this routine, the likely
        outcome of which is a biodone() which queues the bio on
        g_up's regular queue where it will be picked up and processed.

        Compared to the using the regular taskqueue, this saves one
        contextswitch.

Change our scheduling of the g_up and g_down queues to be water-tight,
at the cost of breaking the userland regression test-shims.

Input and ideas from:   scottl
2003-02-11 22:30:26 +00:00
Poul-Henning Kamp
392d56b4bd Don't short-circuit zero-length requests of they are BIO_[SG]ETATTR. 2003-02-11 13:13:10 +00:00
Poul-Henning Kamp
81e9eba33a Turn the "updating" flag (back) into two sequence number fields at
either ends of the structure so we have a way to determine if a
snapshot is consistent.
2003-02-11 11:01:26 +00:00
Poul-Henning Kamp
cce7303af3 Update the statistics collection code to track busy time instead of
idle time.

Statistics now default to "on" and can be turned off with
        sysctl kern.geom.collectstats=0

Performance impact of statistics collection is on the order of
800 nsec per consumer/provider set on a 700MHz Athlon.
2003-02-09 17:04:57 +00:00
Poul-Henning Kamp
4ec353005c Move the g_stat struct to its own .h file, we will export it to other code.
Insted of embedding a struct g_stat in consumers and providers, merely
include a pointer.

Remove a couple of <sys/time.h> includes now unneeded.

Add a special allocator for struct g_stat.  This allocator will allocate
entire pages and hand out g_stat functions from there.  The "id" field
indicates free/used status.

Add "/dev/geom.stats" device driver whic exports the pages from the
allocator to userland with mmap(2) in read-only mode.

This mmap(2) interface should be considered a non-public interface and
the functions in libgeom (not yet committed) should be used to access
the statistics data.
2003-02-08 13:03:57 +00:00
Poul-Henning Kamp
801bb689ca Commit the correct copy of the g_stat structure.
Add debug.sizeof.g_stat sysctl.

Set the id field of the g_stat when we create consumers and providers.

Remove biocount from consumer, we will use the counters in the g_stat
structure instead.  Replace one field which will need to be atomically
manipulated with two fields which will not (stat.nop and stat.nend).

Change add companion field to bio_children: bio_inbed for the exact
same reason.

Don't output the biocount in the confdot output.

Fix KASSERT in g_io_request().

Add sysctl kern.geom.collectstats defaulting to off.

Collect the following raw statistics conditioned on this sysctl:

    for each consumer and provider {
        total number of operations started.
        total number of operations completed.
        time last operation completed.
        sum of idle-time.
        for each of BIO_READ, BIO_WRITE and BIO_DELETE {
            number of operations completed.
            number of bytes completed.
            number of ENOMEM errors.
            number of other errors.
            sum of transaction time.
        }
    }

API for getting hold of these statistics data not included yet.
2003-02-07 23:08:24 +00:00
Poul-Henning Kamp
936cc4614b Rename bio_linkage to the more obvious bio_parent.
Add bio_t0 timestamp, and include <sys/time.h> where needed
2003-02-07 21:09:51 +00:00
Poul-Henning Kamp
e39d70d4fd Put the checks we perform on a bio before calling ::start in their
own function, handle all validation and truncation at the time we
process the bio instead of when it gets scheduled.
2003-02-06 21:01:36 +00:00
Alfred Perlstein
44956c9863 Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.
2003-01-21 08:56:16 +00:00
Poul-Henning Kamp
5ab413bf6e white-space changes 2002-12-26 21:02:50 +00:00
Poul-Henning Kamp
0ae8896ed2 Balk at unaligned requests.
MFC candidate.
2002-12-18 19:53:59 +00:00
Poul-Henning Kamp
3432e4fd03 malloc(9) with M_NOWAIT seems to return NULL a lot more than I would have
expected under -current.  This is a problem for GEOM because the up/down
threads cannot sleep waiting for memory to become free.  The reason they
cannot sleep is that paging things out to disk may be the only way we can
clear up some RAM.  Nice catch-22 there.

Implement a rudimentary ENOMEM recovery strategy:  If an I/O request
fails with an error code of ENOMEM, schedule it for a retry, and
tell the down-thread to sleep hz/10 to get other parts of the system
a chance to free up some memory, in particular the up-path in GEOM.

All caches should probably start to monitor malloc(9) failures using the new
malloc_last_fail() function, and release when it indicates congestion.

Sponsored by:	DARPA & NAI Labs.
2002-11-02 11:08:07 +00:00
Poul-Henning Kamp
0355b86e33 Don't track bio allocation in debug output.
Sponsored by:	DARPA & NAI Labs.
2002-10-20 08:45:17 +00:00