d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
b038594e17
freebsd-skq/sys/kern/kern_condvar.c

416 lines
10 KiB
C
Raw Normal View History

Implement condition variables.
2001-01-16 01:00:43 +00:00
/*-
* Copyright (c) 2000 Jake Burkholder <jake@freebsd.org>.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
Use __FBSDID().
2003-06-11 00:56:59 +00:00
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
Implement condition variables.
2001-01-16 01:00:43 +00:00
#include "opt_ktrace.h"
#include <sys/param.h>
#include <sys/systm.h>
Undo part of the tangle of having sys/lock.h and sys/mutex.h included in other "system" header files. Also help the deprecation of lockmgr.h by making it a sub-include of sys/lock.h and removing sys/lockmgr.h form kernel .c files. Sort sys/*.h includes where possible in affected files. OK'ed by: bde (with reservations)
2001-05-01 08:13:21 +00:00
#include <sys/lock.h>
#include <sys/mutex.h>
Implement condition variables.
2001-01-16 01:00:43 +00:00
#include <sys/proc.h>
#include <sys/kernel.h>
#include <sys/ktr.h>
#include <sys/condvar.h>
- Call sched_sleep() instead of rolling our own in cv_waitq_add().
2003-01-26 04:00:39 +00:00
#include <sys/sched.h>
Implement condition variables.
2001-01-16 01:00:43 +00:00
#include <sys/signalvar.h>
Switch the sleep/wakeup and condition variable implementations to use the sleep queue interface: - Sleep queues attempt to merge some of the benefits of both sleep queues and condition variables. Having sleep qeueus in a hash table avoids having to allocate a queue head for each wait channel. Thus, struct cv has shrunk down to just a single char * pointer now. However, the hash table does not hold threads directly, but queue heads. This means that once you have located a queue in the hash bucket, you no longer have to walk the rest of the hash chain looking for threads. Instead, you have a list of all the threads sleeping on that wait channel. - Outside of the sleepq code and the sleep/cv code the kernel no longer differentiates between cv's and sleep/wakeup. For example, calls to abortsleep() and cv_abort() are replaced with a call to sleepq_abort(). Thus, the TDF_CVWAITQ flag is removed. Also, calls to unsleep() and cv_waitq_remove() have been replaced with calls to sleepq_remove(). - The sched_sleep() function no longer accepts a priority argument as sleep's no longer inherently bump the priority. Instead, this is soley a propery of msleep() which explicitly calls sched_prio() before blocking. - The TDF_ONSLEEPQ flag has been dropped as it was never used. The associated TDF_SET_ONSLEEPQ and TDF_CLR_ON_SLEEPQ macros have also been dropped and replaced with a single explicit clearing of td_wchan. TD_SET_ONSLEEPQ() would really have only made sense if it had taken the wait channel and message as arguments anyway. Now that that only happens in one place, a macro would be overkill.
2004-02-27 18:52:44 +00:00
#include <sys/sleepqueue.h>
Implement condition variables.
2001-01-16 01:00:43 +00:00
#include <sys/resourcevar.h>
#ifdef KTRACE
#include <sys/uio.h>
#include <sys/ktrace.h>
#endif
/*
* Common sanity checks for cv_wait* functions.
*/
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
#define CV_ASSERT(cvp, lock, td) do { \
Update to C99, s/__FUNCTION__/__func__/.
2001-12-10 05:51:45 +00:00
KASSERT((td) != NULL, ("%s: curthread NULL", __func__)); \
Completely redo thread states. Reviewed by: davidxu@freebsd.org
2002-09-11 08:13:56 +00:00
KASSERT(TD_IS_RUNNING(td), ("%s: not TDS_RUNNING", __func__)); \
Update to C99, s/__FUNCTION__/__func__/.
2001-12-10 05:51:45 +00:00
KASSERT((cvp) != NULL, ("%s: cvp NULL", __func__)); \
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
KASSERT((lock) != NULL, ("%s: lock NULL", __func__)); \
Implement condition variables.
2001-01-16 01:00:43 +00:00
} while (0)
/*
* Initialize a condition variable. Must be called before use.
*/
void
cv_init(struct cv *cvp, const char *desc)
{
cvp->cv_description = desc;
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
cvp->cv_waiters = 0;
Implement condition variables.
2001-01-16 01:00:43 +00:00
}
/*
* Destroy a condition variable. The condition variable must be re-initialized
* in order to be re-used.
*/
void
cv_destroy(struct cv *cvp)
{
Switch the sleep/wakeup and condition variable implementations to use the sleep queue interface: - Sleep queues attempt to merge some of the benefits of both sleep queues and condition variables. Having sleep qeueus in a hash table avoids having to allocate a queue head for each wait channel. Thus, struct cv has shrunk down to just a single char * pointer now. However, the hash table does not hold threads directly, but queue heads. This means that once you have located a queue in the hash bucket, you no longer have to walk the rest of the hash chain looking for threads. Instead, you have a list of all the threads sleeping on that wait channel. - Outside of the sleepq code and the sleep/cv code the kernel no longer differentiates between cv's and sleep/wakeup. For example, calls to abortsleep() and cv_abort() are replaced with a call to sleepq_abort(). Thus, the TDF_CVWAITQ flag is removed. Also, calls to unsleep() and cv_waitq_remove() have been replaced with calls to sleepq_remove(). - The sched_sleep() function no longer accepts a priority argument as sleep's no longer inherently bump the priority. Instead, this is soley a propery of msleep() which explicitly calls sched_prio() before blocking. - The TDF_ONSLEEPQ flag has been dropped as it was never used. The associated TDF_SET_ONSLEEPQ and TDF_CLR_ON_SLEEPQ macros have also been dropped and replaced with a single explicit clearing of td_wchan. TD_SET_ONSLEEPQ() would really have only made sense if it had taken the wait channel and message as arguments anyway. Now that that only happens in one place, a macro would be overkill.
2004-02-27 18:52:44 +00:00
#ifdef INVARIANTS
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
struct sleepqueue *sq;
Implement condition variables.
2001-01-16 01:00:43 +00:00
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
Switch the sleep/wakeup and condition variable implementations to use the sleep queue interface: - Sleep queues attempt to merge some of the benefits of both sleep queues and condition variables. Having sleep qeueus in a hash table avoids having to allocate a queue head for each wait channel. Thus, struct cv has shrunk down to just a single char * pointer now. However, the hash table does not hold threads directly, but queue heads. This means that once you have located a queue in the hash bucket, you no longer have to walk the rest of the hash chain looking for threads. Instead, you have a list of all the threads sleeping on that wait channel. - Outside of the sleepq code and the sleep/cv code the kernel no longer differentiates between cv's and sleep/wakeup. For example, calls to abortsleep() and cv_abort() are replaced with a call to sleepq_abort(). Thus, the TDF_CVWAITQ flag is removed. Also, calls to unsleep() and cv_waitq_remove() have been replaced with calls to sleepq_remove(). - The sched_sleep() function no longer accepts a priority argument as sleep's no longer inherently bump the priority. Instead, this is soley a propery of msleep() which explicitly calls sched_prio() before blocking. - The TDF_ONSLEEPQ flag has been dropped as it was never used. The associated TDF_SET_ONSLEEPQ and TDF_CLR_ON_SLEEPQ macros have also been dropped and replaced with a single explicit clearing of td_wchan. TD_SET_ONSLEEPQ() would really have only made sense if it had taken the wait channel and message as arguments anyway. Now that that only happens in one place, a macro would be overkill.
2004-02-27 18:52:44 +00:00
sq = sleepq_lookup(cvp);
sleepq_release(cvp);
KASSERT(sq == NULL, ("%s: associated sleep queue non-empty", __func__));
#endif
Implement condition variables.
2001-01-16 01:00:43 +00:00
}
/*
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* Wait on a condition variable. The current thread is placed on the condition
Implement condition variables.
2001-01-16 01:00:43 +00:00
* variable's wait queue and suspended. A cv_signal or cv_broadcast on the same
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* condition variable will resume the thread. The mutex is released before
Implement condition variables.
2001-01-16 01:00:43 +00:00
* sleeping and will be held on return. It is recommended that the mutex be
* held when cv_signal or cv_broadcast are called.
*/
void
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
_cv_wait(struct cv *cvp, struct lock_object *lock)
Implement condition variables.
2001-01-16 01:00:43 +00:00
{
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE_DECL(lock_witness);
struct lock_class *class;
Don't use cv_wait_unlock() to implement cv_wait(). Instead, implement cv_wait() fully and add missing KTRACE context switch traces.
2007-03-21 20:46:26 +00:00
struct thread *td;
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
int lock_state;
Implement condition variables.
2001-01-16 01:00:43 +00:00
Don't use cv_wait_unlock() to implement cv_wait(). Instead, implement cv_wait() fully and add missing KTRACE context switch traces.
2007-03-21 20:46:26 +00:00
td = curthread;
#ifdef KTRACE
if (KTRPOINT(td, KTR_CSW))
ktrcsw(1, 0);
#endif
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
CV_ASSERT(cvp, lock, td);
WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, lock,
Don't use cv_wait_unlock() to implement cv_wait(). Instead, implement cv_wait() fully and add missing KTRACE context switch traces.
2007-03-21 20:46:26 +00:00
"Waiting on \"%s\"", cvp->cv_description);
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE(lock, lock_witness);
class = LOCK_CLASS(lock);
Contributions from XFS for FreeBSD project: - Implement cv_wait_unlock() method which has semantics compatible with the sv_wait() method in IRIX. For cv_wait_unlock(), the lock must be held before entering the function, but is not held when the function is exited. - Implement the existing cv_wait() function in terms of cv_wait_unlock(). Submitted by: kan Feedback from: jhb, trhodes, Christoph Hellwig <hch at infradead dot org>
2005-12-12 00:02:22 +00:00
if (cold || panicstr) {
/*
* During autoconfiguration, just give interrupts
* a chance, then just return. Don't run any other
* thread or panic below, in case this is the idle
* process and already asleep.
*/
return;
}
Don't use cv_wait_unlock() to implement cv_wait(). Instead, implement cv_wait() fully and add missing KTRACE context switch traces.
2007-03-21 20:46:26 +00:00
sleepq_lock(cvp);
cvp->cv_waiters++;
DROP_GIANT();
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
sleepq_add(cvp, lock, cvp->cv_description, SLEEPQ_CONDVAR, 0);
Fix a potential LOR with sx_sleep() and cv_wait() with sx locks by 1) adding the thread to the sleepq via sleepq_add() before dropping the lock, and 2) dropping the sleepq lock around calls to lc_unlock() for sleepable locks (i.e. locks that use sleepq's in their implementation).
2007-05-08 21:49:59 +00:00
if (class->lc_flags & LC_SLEEPABLE)
sleepq_release(cvp);
lock_state = class->lc_unlock(lock);
if (class->lc_flags & LC_SLEEPABLE)
sleepq_lock(cvp);
Don't use cv_wait_unlock() to implement cv_wait(). Instead, implement cv_wait() fully and add missing KTRACE context switch traces.
2007-03-21 20:46:26 +00:00
sleepq_wait(cvp);
#ifdef KTRACE
if (KTRPOINT(td, KTR_CSW))
ktrcsw(0, 0);
#endif
PICKUP_GIANT();
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
class->lc_lock(lock, lock_state);
WITNESS_RESTORE(lock, lock_witness);
Contributions from XFS for FreeBSD project: - Implement cv_wait_unlock() method which has semantics compatible with the sv_wait() method in IRIX. For cv_wait_unlock(), the lock must be held before entering the function, but is not held when the function is exited. - Implement the existing cv_wait() function in terms of cv_wait_unlock(). Submitted by: kan Feedback from: jhb, trhodes, Christoph Hellwig <hch at infradead dot org>
2005-12-12 00:02:22 +00:00
}
/*
* Wait on a condition variable. This function differs from cv_wait by
* not aquiring the mutex after condition variable was signaled.
*/
void
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
_cv_wait_unlock(struct cv *cvp, struct lock_object *lock)
Contributions from XFS for FreeBSD project: - Implement cv_wait_unlock() method which has semantics compatible with the sv_wait() method in IRIX. For cv_wait_unlock(), the lock must be held before entering the function, but is not held when the function is exited. - Implement the existing cv_wait() function in terms of cv_wait_unlock(). Submitted by: kan Feedback from: jhb, trhodes, Christoph Hellwig <hch at infradead dot org>
2005-12-12 00:02:22 +00:00
{
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
struct lock_class *class;
Contributions from XFS for FreeBSD project: - Implement cv_wait_unlock() method which has semantics compatible with the sv_wait() method in IRIX. For cv_wait_unlock(), the lock must be held before entering the function, but is not held when the function is exited. - Implement the existing cv_wait() function in terms of cv_wait_unlock(). Submitted by: kan Feedback from: jhb, trhodes, Christoph Hellwig <hch at infradead dot org>
2005-12-12 00:02:22 +00:00
struct thread *td;
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
td = curthread;
Implement condition variables.
2001-01-16 01:00:43 +00:00
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(1, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
CV_ASSERT(cvp, lock, td);
WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, lock,
Replace calls to WITNESS_SLEEP() and witness_list() with equivalent calls to WITNESS_WARN().
2003-03-04 21:03:05 +00:00
"Waiting on \"%s\"", cvp->cv_description);
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
class = LOCK_CLASS(lock);
Implement condition variables.
2001-01-16 01:00:43 +00:00
Switch the sleep/wakeup and condition variable implementations to use the sleep queue interface: - Sleep queues attempt to merge some of the benefits of both sleep queues and condition variables. Having sleep qeueus in a hash table avoids having to allocate a queue head for each wait channel. Thus, struct cv has shrunk down to just a single char * pointer now. However, the hash table does not hold threads directly, but queue heads. This means that once you have located a queue in the hash bucket, you no longer have to walk the rest of the hash chain looking for threads. Instead, you have a list of all the threads sleeping on that wait channel. - Outside of the sleepq code and the sleep/cv code the kernel no longer differentiates between cv's and sleep/wakeup. For example, calls to abortsleep() and cv_abort() are replaced with a call to sleepq_abort(). Thus, the TDF_CVWAITQ flag is removed. Also, calls to unsleep() and cv_waitq_remove() have been replaced with calls to sleepq_remove(). - The sched_sleep() function no longer accepts a priority argument as sleep's no longer inherently bump the priority. Instead, this is soley a propery of msleep() which explicitly calls sched_prio() before blocking. - The TDF_ONSLEEPQ flag has been dropped as it was never used. The associated TDF_SET_ONSLEEPQ and TDF_CLR_ON_SLEEPQ macros have also been dropped and replaced with a single explicit clearing of td_wchan. TD_SET_ONSLEEPQ() would really have only made sense if it had taken the wait channel and message as arguments anyway. Now that that only happens in one place, a macro would be overkill.
2004-02-27 18:52:44 +00:00
if (cold || panicstr) {
Implement condition variables.
2001-01-16 01:00:43 +00:00
/*
Allow alphas to do crashdumps: Refuse to run anything in choosethread() after a panic which is not an interrupt thread, or the thread which caused the panic. Also, remove panicstr checks from msleep() and from cv_wait() in order to allow threads to go to sleep and yeild the cpu to the panicing thread, or to an interrupt thread which might be doing the crashdump. Reviewed by: jhb (and it was mostly his idea too)
2002-07-17 02:23:44 +00:00
* During autoconfiguration, just give interrupts
* a chance, then just return. Don't run any other
* thread or panic below, in case this is the idle
* process and already asleep.
Implement condition variables.
2001-01-16 01:00:43 +00:00
*/
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
class->lc_unlock(lock);
Implement condition variables.
2001-01-16 01:00:43 +00:00
return;
}
The cold and panicstr variables do not need to be protected by sched_lock. Submitted by: Jennifer Yang (yangjihui@yahoo.com) Reviewed by: jake & jhb in principle
2002-04-23 19:50:22 +00:00
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
cvp->cv_waiters++;
Change the preemption code for software interrupt thread schedules and mutex releases to not require flags for the cases when preemption is not allowed: The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent switching to a higher priority thread on mutex releease and swi schedule, respectively when that switch is not safe. Now that the critical section API maintains a per-thread nesting count, the kernel can easily check whether or not it should switch without relying on flags from the programmer. This fixes a few bugs in that all current callers of swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from fast interrupt handlers and the swi_sched of softclock needed this flag. Note that to ensure that swi_sched()'s in clock and fast interrupt handlers do not switch, these handlers have to be explicitly wrapped in critical_enter/exit pairs. Presently, just wrapping the handlers is sufficient, but in the future with the fully preemptive kernel, the interrupt must be EOI'd before critical_exit() is called. (critical_exit() can switch due to a deferred preemption in a fully preemptive kernel.) I've tested the changes to the interrupt code on i386 and alpha. I have not tested ia64, but the interrupt code is almost identical to the alpha code, so I expect it will work fine. PowerPC and ARM do not yet have interrupt code in the tree so they shouldn't be broken. Sparc64 is broken, but that's been ok'd by jake and tmm who will be fixing the interrupt code for sparc64 shortly. Reviewed by: peter Tested on: i386, alpha
2002-01-05 08:47:13 +00:00
DROP_GIANT();
Implement condition variables.
2001-01-16 01:00:43 +00:00
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
sleepq_add(cvp, lock, cvp->cv_description, SLEEPQ_CONDVAR, 0);
Fix a potential LOR with sx_sleep() and cv_wait() with sx locks by 1) adding the thread to the sleepq via sleepq_add() before dropping the lock, and 2) dropping the sleepq lock around calls to lc_unlock() for sleepable locks (i.e. locks that use sleepq's in their implementation).
2007-05-08 21:49:59 +00:00
if (class->lc_flags & LC_SLEEPABLE)
sleepq_release(cvp);
class->lc_unlock(lock);
if (class->lc_flags & LC_SLEEPABLE)
sleepq_lock(cvp);
Switch the sleep/wakeup and condition variable implementations to use the sleep queue interface: - Sleep queues attempt to merge some of the benefits of both sleep queues and condition variables. Having sleep qeueus in a hash table avoids having to allocate a queue head for each wait channel. Thus, struct cv has shrunk down to just a single char * pointer now. However, the hash table does not hold threads directly, but queue heads. This means that once you have located a queue in the hash bucket, you no longer have to walk the rest of the hash chain looking for threads. Instead, you have a list of all the threads sleeping on that wait channel. - Outside of the sleepq code and the sleep/cv code the kernel no longer differentiates between cv's and sleep/wakeup. For example, calls to abortsleep() and cv_abort() are replaced with a call to sleepq_abort(). Thus, the TDF_CVWAITQ flag is removed. Also, calls to unsleep() and cv_waitq_remove() have been replaced with calls to sleepq_remove(). - The sched_sleep() function no longer accepts a priority argument as sleep's no longer inherently bump the priority. Instead, this is soley a propery of msleep() which explicitly calls sched_prio() before blocking. - The TDF_ONSLEEPQ flag has been dropped as it was never used. The associated TDF_SET_ONSLEEPQ and TDF_CLR_ON_SLEEPQ macros have also been dropped and replaced with a single explicit clearing of td_wchan. TD_SET_ONSLEEPQ() would really have only made sense if it had taken the wait channel and message as arguments anyway. Now that that only happens in one place, a macro would be overkill.
2004-02-27 18:52:44 +00:00
sleepq_wait(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
Don't use cv_wait_unlock() to implement cv_wait(). Instead, implement cv_wait() fully and add missing KTRACE context switch traces.
2007-03-21 20:46:26 +00:00
#ifdef KTRACE
if (KTRPOINT(td, KTR_CSW))
ktrcsw(0, 0);
#endif
Implement condition variables.
2001-01-16 01:00:43 +00:00
PICKUP_GIANT();
}
/*
* Wait on a condition variable, allowing interruption by signals. Return 0 if
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* the thread was resumed with cv_signal or cv_broadcast, EINTR or ERESTART if
Implement condition variables.
2001-01-16 01:00:43 +00:00
* a signal was caught. If ERESTART is returned the system call should be
* restarted if possible.
*/
int
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
_cv_wait_sig(struct cv *cvp, struct lock_object *lock)
Implement condition variables.
2001-01-16 01:00:43 +00:00
{
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE_DECL(lock_witness);
struct lock_class *class;
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct thread *td;
Add missing ; in last commit Pointy-hat-to: jhb
2001-09-19 02:53:59 +00:00
struct proc *p;
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
int lock_state, rval;
Implement condition variables.
2001-01-16 01:00:43 +00:00
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
td = curthread;
Use a 'p' variable instead of repetitively indirecting td->td_proc for signal things that are still per-process and won't be per-thread.
2001-09-18 23:27:06 +00:00
p = td->td_proc;
Implement condition variables.
2001-01-16 01:00:43 +00:00
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(1, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
CV_ASSERT(cvp, lock, td);
WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, lock,
Replace calls to WITNESS_SLEEP() and witness_list() with equivalent calls to WITNESS_WARN().
2003-03-04 21:03:05 +00:00
"Waiting on \"%s\"", cvp->cv_description);
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE(lock, lock_witness);
class = LOCK_CLASS(lock);
Implement condition variables.
2001-01-16 01:00:43 +00:00
if (cold || panicstr) {
/*
* After a panic, or during autoconfiguration, just give
* interrupts a chance, then just return; don't run any other
* procs or panic below, in case this is the idle process and
* already asleep.
*/
Synchronize the extra SA threading checks and return value handling of condition variables with that of msleep(). Reviewed by: davidxu
2004-08-10 17:42:59 +00:00
return (0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
}
The cold and panicstr variables do not need to be protected by sched_lock. Submitted by: Jennifer Yang (yangjihui@yahoo.com) Reviewed by: jake & jhb in principle
2002-04-23 19:50:22 +00:00
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
The cold and panicstr variables do not need to be protected by sched_lock. Submitted by: Jennifer Yang (yangjihui@yahoo.com) Reviewed by: jake & jhb in principle
2002-04-23 19:50:22 +00:00
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
cvp->cv_waiters++;
Change the preemption code for software interrupt thread schedules and mutex releases to not require flags for the cases when preemption is not allowed: The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent switching to a higher priority thread on mutex releease and swi schedule, respectively when that switch is not safe. Now that the critical section API maintains a per-thread nesting count, the kernel can easily check whether or not it should switch without relying on flags from the programmer. This fixes a few bugs in that all current callers of swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from fast interrupt handlers and the swi_sched of softclock needed this flag. Note that to ensure that swi_sched()'s in clock and fast interrupt handlers do not switch, these handlers have to be explicitly wrapped in critical_enter/exit pairs. Presently, just wrapping the handlers is sufficient, but in the future with the fully preemptive kernel, the interrupt must be EOI'd before critical_exit() is called. (critical_exit() can switch due to a deferred preemption in a fully preemptive kernel.) I've tested the changes to the interrupt code on i386 and alpha. I have not tested ia64, but the interrupt code is almost identical to the alpha code, so I expect it will work fine. PowerPC and ARM do not yet have interrupt code in the tree so they shouldn't be broken. Sparc64 is broken, but that's been ok'd by jake and tmm who will be fixing the interrupt code for sparc64 shortly. Reviewed by: peter Tested on: i386, alpha
2002-01-05 08:47:13 +00:00
DROP_GIANT();
Implement condition variables.
2001-01-16 01:00:43 +00:00
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
sleepq_add(cvp, lock, cvp->cv_description, SLEEPQ_CONDVAR |
Add second sleep queue so that sx and lockmgr can have separate sleep queues for shared and exclusive acquisitions Submitted by: Attilio Rao Approved by: jhb
2006-12-16 06:54:09 +00:00
SLEEPQ_INTERRUPTIBLE, 0);
Fix a potential LOR with sx_sleep() and cv_wait() with sx locks by 1) adding the thread to the sleepq via sleepq_add() before dropping the lock, and 2) dropping the sleepq lock around calls to lc_unlock() for sleepable locks (i.e. locks that use sleepq's in their implementation).
2007-05-08 21:49:59 +00:00
if (class->lc_flags & LC_SLEEPABLE)
sleepq_release(cvp);
lock_state = class->lc_unlock(lock);
if (class->lc_flags & LC_SLEEPABLE)
sleepq_lock(cvp);
Now that the return value semantics of cv's for multithreaded processes have been unified with that of msleep(9), further refine the sleepq interface and consolidate some duplicated code: - Move the pre-sleep checks for theaded processes into a thread_sleep_check() function in kern_thread.c. - Move all handling of TDF_SINTR to be internal to subr_sleepqueue.c. Specifically, if a thread is awakened by something other than a signal while checking for signals before going to sleep, clear TDF_SINTR in sleepq_catch_signals(). This removes a sched_lock lock/unlock combo in that edge case during an interruptible sleep. Also, fix sleepq_check_signals() to properly handle the condition if TDF_SINTR is clear rather than requiring the callers of the sleepq API to notice this edge case and call a non-_sig variant of sleepq_wait(). - Clarify the flags arguments to sleepq_add(), sleepq_signal() and sleepq_broadcast() by creating an explicit submask for sleepq types. Also, add an explicit SLEEPQ_MSLEEP type rather than a magic number of 0. Also, add a SLEEPQ_INTERRUPTIBLE flag for use with sleepq_add() and move the setting of TDF_SINTR to sleepq_add() if this flag is set rather than sleepq_catch_signals(). Note that it is the caller's responsibility to ensure that sleepq_catch_signals() is called if and only if this flag is passed to the preceeding sleepq_add(). Note that this also removes a sched_lock lock/unlock pair from sleepq_catch_signals(). It also ensures that for an interruptible sleep, TDF_SINTR is always set when TD_ON_SLEEPQ() is true.
2004-08-19 11:31:42 +00:00
rval = sleepq_wait_sig(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(0, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
PICKUP_GIANT();
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
class->lc_lock(lock, lock_state);
WITNESS_RESTORE(lock, lock_witness);
Implement condition variables.
2001-01-16 01:00:43 +00:00
return (rval);
}
/*
* Wait on a condition variable for at most timo/hz seconds. Returns 0 if the
* process was resumed by cv_signal or cv_broadcast, EWOULDBLOCK if the timeout
* expires.
*/
int
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
_cv_timedwait(struct cv *cvp, struct lock_object *lock, int timo)
Implement condition variables.
2001-01-16 01:00:43 +00:00
{
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE_DECL(lock_witness);
struct lock_class *class;
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct thread *td;
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
int lock_state, rval;
Implement condition variables.
2001-01-16 01:00:43 +00:00
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
td = curthread;
Implement condition variables.
2001-01-16 01:00:43 +00:00
rval = 0;
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(1, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
CV_ASSERT(cvp, lock, td);
WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, lock,
Replace calls to WITNESS_SLEEP() and witness_list() with equivalent calls to WITNESS_WARN().
2003-03-04 21:03:05 +00:00
"Waiting on \"%s\"", cvp->cv_description);
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE(lock, lock_witness);
class = LOCK_CLASS(lock);
Implement condition variables.
2001-01-16 01:00:43 +00:00
if (cold || panicstr) {
/*
* After a panic, or during autoconfiguration, just give
* interrupts a chance, then just return; don't run any other
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* thread or panic below, in case this is the idle process and
Implement condition variables.
2001-01-16 01:00:43 +00:00
* already asleep.
*/
return 0;
}
The cold and panicstr variables do not need to be protected by sched_lock. Submitted by: Jennifer Yang (yangjihui@yahoo.com) Reviewed by: jake & jhb in principle
2002-04-23 19:50:22 +00:00
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
cvp->cv_waiters++;
Change the preemption code for software interrupt thread schedules and mutex releases to not require flags for the cases when preemption is not allowed: The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent switching to a higher priority thread on mutex releease and swi schedule, respectively when that switch is not safe. Now that the critical section API maintains a per-thread nesting count, the kernel can easily check whether or not it should switch without relying on flags from the programmer. This fixes a few bugs in that all current callers of swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from fast interrupt handlers and the swi_sched of softclock needed this flag. Note that to ensure that swi_sched()'s in clock and fast interrupt handlers do not switch, these handlers have to be explicitly wrapped in critical_enter/exit pairs. Presently, just wrapping the handlers is sufficient, but in the future with the fully preemptive kernel, the interrupt must be EOI'd before critical_exit() is called. (critical_exit() can switch due to a deferred preemption in a fully preemptive kernel.) I've tested the changes to the interrupt code on i386 and alpha. I have not tested ia64, but the interrupt code is almost identical to the alpha code, so I expect it will work fine. PowerPC and ARM do not yet have interrupt code in the tree so they shouldn't be broken. Sparc64 is broken, but that's been ok'd by jake and tmm who will be fixing the interrupt code for sparc64 shortly. Reviewed by: peter Tested on: i386, alpha
2002-01-05 08:47:13 +00:00
DROP_GIANT();
Implement condition variables.
2001-01-16 01:00:43 +00:00
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
sleepq_add(cvp, lock, cvp->cv_description, SLEEPQ_CONDVAR, 0);
- Remove old sleep queues. - Remove sleepqueue argument from sleepq_set_timeout() since it is not used.
2004-03-12 19:06:18 +00:00
sleepq_set_timeout(cvp, timo);
Fix a potential LOR with sx_sleep() and cv_wait() with sx locks by 1) adding the thread to the sleepq via sleepq_add() before dropping the lock, and 2) dropping the sleepq lock around calls to lc_unlock() for sleepable locks (i.e. locks that use sleepq's in their implementation).
2007-05-08 21:49:59 +00:00
if (class->lc_flags & LC_SLEEPABLE)
sleepq_release(cvp);
lock_state = class->lc_unlock(lock);
if (class->lc_flags & LC_SLEEPABLE)
sleepq_lock(cvp);
Remove the signal_caught argument from sleepq_timedwait() as it was effectively always zero.
2004-06-28 18:57:06 +00:00
rval = sleepq_timedwait(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(0, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
PICKUP_GIANT();
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
class->lc_lock(lock, lock_state);
WITNESS_RESTORE(lock, lock_witness);
Implement condition variables.
2001-01-16 01:00:43 +00:00
return (rval);
}
/*
* Wait on a condition variable for at most timo/hz seconds, allowing
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* interruption by signals. Returns 0 if the thread was resumed by cv_signal
Implement condition variables.
2001-01-16 01:00:43 +00:00
* or cv_broadcast, EWOULDBLOCK if the timeout expires, and EINTR or ERESTART if
* a signal was caught.
*/
int
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
_cv_timedwait_sig(struct cv *cvp, struct lock_object *lock, int timo)
Implement condition variables.
2001-01-16 01:00:43 +00:00
{
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE_DECL(lock_witness);
struct lock_class *class;
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct thread *td;
Use a 'p' variable instead of repetitively indirecting td->td_proc for signal things that are still per-process and won't be per-thread.
2001-09-18 23:27:06 +00:00
struct proc *p;
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
int lock_state, rval;
Implement condition variables.
2001-01-16 01:00:43 +00:00
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
td = curthread;
Use a 'p' variable instead of repetitively indirecting td->td_proc for signal things that are still per-process and won't be per-thread.
2001-09-18 23:27:06 +00:00
p = td->td_proc;
Implement condition variables.
2001-01-16 01:00:43 +00:00
rval = 0;
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(1, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
CV_ASSERT(cvp, lock, td);
WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, lock,
Replace calls to WITNESS_SLEEP() and witness_list() with equivalent calls to WITNESS_WARN().
2003-03-04 21:03:05 +00:00
"Waiting on \"%s\"", cvp->cv_description);
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
WITNESS_SAVE(lock, lock_witness);
class = LOCK_CLASS(lock);
Implement condition variables.
2001-01-16 01:00:43 +00:00
if (cold || panicstr) {
/*
* After a panic, or during autoconfiguration, just give
* interrupts a chance, then just return; don't run any other
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* thread or panic below, in case this is the idle process and
Implement condition variables.
2001-01-16 01:00:43 +00:00
* already asleep.
*/
return 0;
}
The cold and panicstr variables do not need to be protected by sched_lock. Submitted by: Jennifer Yang (yangjihui@yahoo.com) Reviewed by: jake & jhb in principle
2002-04-23 19:50:22 +00:00
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
cvp->cv_waiters++;
Change the preemption code for software interrupt thread schedules and mutex releases to not require flags for the cases when preemption is not allowed: The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent switching to a higher priority thread on mutex releease and swi schedule, respectively when that switch is not safe. Now that the critical section API maintains a per-thread nesting count, the kernel can easily check whether or not it should switch without relying on flags from the programmer. This fixes a few bugs in that all current callers of swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from fast interrupt handlers and the swi_sched of softclock needed this flag. Note that to ensure that swi_sched()'s in clock and fast interrupt handlers do not switch, these handlers have to be explicitly wrapped in critical_enter/exit pairs. Presently, just wrapping the handlers is sufficient, but in the future with the fully preemptive kernel, the interrupt must be EOI'd before critical_exit() is called. (critical_exit() can switch due to a deferred preemption in a fully preemptive kernel.) I've tested the changes to the interrupt code on i386 and alpha. I have not tested ia64, but the interrupt code is almost identical to the alpha code, so I expect it will work fine. PowerPC and ARM do not yet have interrupt code in the tree so they shouldn't be broken. Sparc64 is broken, but that's been ok'd by jake and tmm who will be fixing the interrupt code for sparc64 shortly. Reviewed by: peter Tested on: i386, alpha
2002-01-05 08:47:13 +00:00
DROP_GIANT();
Implement condition variables.
2001-01-16 01:00:43 +00:00
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
sleepq_add(cvp, lock, cvp->cv_description, SLEEPQ_CONDVAR |
Add second sleep queue so that sx and lockmgr can have separate sleep queues for shared and exclusive acquisitions Submitted by: Attilio Rao Approved by: jhb
2006-12-16 06:54:09 +00:00
SLEEPQ_INTERRUPTIBLE, 0);
- Remove old sleep queues. - Remove sleepqueue argument from sleepq_set_timeout() since it is not used.
2004-03-12 19:06:18 +00:00
sleepq_set_timeout(cvp, timo);
Fix a potential LOR with sx_sleep() and cv_wait() with sx locks by 1) adding the thread to the sleepq via sleepq_add() before dropping the lock, and 2) dropping the sleepq lock around calls to lc_unlock() for sleepable locks (i.e. locks that use sleepq's in their implementation).
2007-05-08 21:49:59 +00:00
if (class->lc_flags & LC_SLEEPABLE)
sleepq_release(cvp);
lock_state = class->lc_unlock(lock);
if (class->lc_flags & LC_SLEEPABLE)
sleepq_lock(cvp);
Fix a long standing race between sleep queue and thread suspension code. When a thread A is going to sleep, it calls sleepq_catch_signals() to detect any pending signals or thread suspension request, if nothing happens, it returns without holding process lock or scheduler lock, this opens a race window which allows thread B to come in and do process suspension work, however since A is still at running state, thread B can do nothing to A, thread A continues, and puts itself into actually sleeping state, but B has never seen it, and it sits there forever until B is woken up by other threads sometimes later(this can be very long delay or never happen). Fix this bug by forcing sleepq_catch_signals to return with scheduler lock held. Fix sleepq_abort() by passing it an interrupted code, previously, it worked as wakeup_one(), and the interruption can not be identified correctly by sleep queue code when the sleeping thread is resumed. Let thread_suspend_check() returns EINTR or ERESTART, so sleep queue no longer has to use SIGSTOP as a hack to build a return value. Reviewed by: jhb MFC after: 1 week
2006-02-15 23:52:01 +00:00
rval = sleepq_timedwait_sig(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#ifdef KTRACE
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
if (KTRPOINT(td, KTR_CSW))
ktrcsw(0, 0);
Implement condition variables.
2001-01-16 01:00:43 +00:00
#endif
- Catch up to new ktrace API. - ktrace trace points in msleep() and cv_wait() no longer need Giant.
2002-06-07 05:39:16 +00:00
PICKUP_GIANT();
Rename the cv_*wait*() functions to _cv_*wait*() and change their second argument from a mutex to a lock_object. Add cv_*wait*() wrapper macros that accept either a mutex, rwlock, or sx lock as the second argument and convert it to a lock_object and then call _cv_*wait*(). Basically, the visible difference is that you can now use rwlocks and sx locks with condition variables using the same API as with mutexes.
2007-03-21 22:22:13 +00:00
class->lc_lock(lock, lock_state);
WITNESS_RESTORE(lock, lock_witness);
Implement condition variables.
2001-01-16 01:00:43 +00:00
return (rval);
}
/*
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* Signal a condition variable, wakes up one waiting thread. Will also wakeup
Implement condition variables.
2001-01-16 01:00:43 +00:00
* the swapper if the process is not in memory, so that it can bring the
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* sleeping process in. Note that this may also result in additional threads
Implement condition variables.
2001-01-16 01:00:43 +00:00
* being made runnable. Should be called with the same mutex as was passed to
* cv_wait held.
*/
void
cv_signal(struct cv *cvp)
{
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
if (cvp->cv_waiters > 0) {
cvp->cv_waiters--;
Add second sleep queue so that sx and lockmgr can have separate sleep queues for shared and exclusive acquisitions Submitted by: Attilio Rao Approved by: jhb
2006-12-16 06:54:09 +00:00
sleepq_signal(cvp, SLEEPQ_CONDVAR, -1, 0);
Commit 2/14 of sched_lock decomposition. - Adapt sleepqueues to the new thread_lock() mechanism. - Delay assigning the sleep queue spinlock as the thread lock until after we've checked for signals. It is illegal for a thread to return in mi_switch() with any lock assigned to td_lock other than the scheduler locks. - Change sleepq_catch_signals() to do the switch if necessary to simplify the callers. - Simplify timeout handling now that locking a sleeping thread has the side-effect of locking the sleepqueue. Some previous races are no longer possible. Tested by: kris, current@ Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc. Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each)
2007-06-04 23:50:56 +00:00
}
sleepq_release(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
}
/*
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
* Broadcast a signal to a condition variable. Wakes up all waiting threads.
Implement condition variables.
2001-01-16 01:00:43 +00:00
* Should be called with the same mutex as was passed to cv_wait held.
*/
void
- Implement selwakeuppri() which allows raising the priority of a thread being waken up. The thread waken up can run at a priority as high as after tsleep(). - Replace selwakeup()s with selwakeuppri()s and pass appropriate priorities. - Add cv_broadcastpri() which raises the priority of the broadcast threads. Used by selwakeuppri() if collision occurs. Not objected in: -arch, -current
2003-11-09 09:17:26 +00:00
cv_broadcastpri(struct cv *cvp, int pri)
Implement condition variables.
2001-01-16 01:00:43 +00:00
{
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
sleepq_lock(cvp);
Associate a simple count of waiters with each condition variable. The count is protected by the mutex that protects the condition, so the count does not require any extra locking or atomic operations. It serves as an optimization to avoid calling into the sleepqueue code at all if there are no waiters. Note that the count can get temporarily out of sync when threads sleeping on a condition variable time out or are aborted. However, it doesn't hurt to call the sleepqueue code for either a signal or a broadcast when there are no waiters, and the count is never out of sync in the opposite direction unless we have more than INT_MAX sleeping threads.
2004-04-06 19:17:46 +00:00
if (cvp->cv_waiters > 0) {
cvp->cv_waiters = 0;
Add second sleep queue so that sx and lockmgr can have separate sleep queues for shared and exclusive acquisitions Submitted by: Attilio Rao Approved by: jhb
2006-12-16 06:54:09 +00:00
sleepq_broadcast(cvp, SLEEPQ_CONDVAR, pri, 0);
Refine the turnstile and sleep queue interfaces just a bit: - Add a new _lock() call to each API that locks the associated chain lock for a lock_object pointer or wait channel. The _lookup() functions now require that the chain lock be locked via _lock() when they are called. - Change sleepq_add(), turnstile_wait() and turnstile_claim() to lookup the associated queue structure internally via _lookup() rather than accepting a pointer from the caller. For turnstiles, this means that the actual lookup of the turnstile in the hash table is only done when the thread actually blocks rather than being done on each loop iteration in _mtx_lock_sleep(). For sleep queues, this means that sleepq_lookup() is no longer used outside of the sleep queue code except to implement an assertion in cv_destroy(). - Change sleepq_broadcast() and sleepq_signal() to require that the chain lock is already required. For condition variables, this lets the cv_broadcast() and cv_signal() functions lock the sleep queue chain lock while testing the waiters count. This means that the waiters count internal to condition variables is no longer protected by the interlock mutex and cv_broadcast() and cv_signal() now no longer require that the interlock be held when they are called. This lets consumers of condition variables drop the lock before waking other threads which can result in fewer context switches. MFC after: 1 month
2004-10-12 18:36:20 +00:00
} else
sleepq_release(cvp);
Implement condition variables.
2001-01-16 01:00:43 +00:00
}
Reference in New Issue Copy Permalink