82 Commits

Author SHA1 Message Date
davidxu
72ce06de36 MFp4:
Further decreases unexpected context switches by defering mutex wakeup
until internal sleep queue lock is released.
2012-08-11 23:17:02 +00:00
davidxu
03c118fef9 Return EBUSY for PTHREAD_MUTEX_ADAPTIVE_NP too when the mutex could not
be acquired.

PR:	168317
MFC after:	3 days
2012-05-27 01:24:51 +00:00
davidxu
bbfe229093 Return 0 instead of garbage value.
Found by:	clang static analyzer
2011-01-06 08:13:30 +00:00
davidxu
1d89f14a5c Always clear flag PMUTEX_FLAG_DEFERED when unlocking, as it is only
significant for lock owner.
2010-12-24 07:41:39 +00:00
davidxu
437ad27f9c MFp4:
- Add flags CVWAIT_ABSTIME and CVWAIT_CLOCKID for umtx kernel based
  condition variable, this should eliminate an extra system call to get
  current time.

- Add sub-function UMTX_OP_NWAKE_PRIVATE to wake up N channels in single
  system call. Create userland sleep queue for condition variable, in most
  cases, thread will wait in the queue, the pthread_cond_signal will defer
  thread wakeup until the mutex is unlocked, it tries to avoid an extra
  system call and a extra context switch in time window of pthread_cond_signal
  and pthread_mutex_unlock.

The changes are part of process-shared mutex project.
2010-12-22 05:01:52 +00:00
davidxu
d60836560b Remove locking and unlock in pthread_mutex_destroy, because
it can not fix race condition in application code, as a result,
the problem described in PR threads/151767 is avoided.
2010-10-27 04:19:07 +00:00
davidxu
df6acfd7c4 Check invalid mutex in _mutex_cv_unlock. 2010-09-29 06:06:58 +00:00
davidxu
f329bc965c In current code, statically initialized and destroyed object have
same null value, the code can not distinguish between them, to
fix the problem, now a destroyed object is assigned to a non-null
value, and it will be rejected by some pthread functions.
PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP is changed to number 1, so that
adaptive mutex can be statically initialized correctly.
2010-09-28 04:57:56 +00:00
davidxu
5f00b957ae Change atfork lock from mutex to rwlock, also make mutexes used by malloc()
module private type, when private type mutex is locked/unlocked, thread
critical region is entered or leaved. These changes makes fork()
async-signal safe which required by POSIX. Note that user's atfork handler
still needs to be async-signal safe, but it is not problem of libthr, it
is user's responsiblity.
2010-09-01 03:11:21 +00:00
davidxu
70dd244f26 Add two commands to _umtx_op system call to allow a simple mutex to be
locked and unlocked completely in userland. by locking and unlocking mutex
in userland, it reduces the total time a mutex is locked by a thread,
in some application code, a mutex only protects a small piece of code, the
code's execution time is less than a simple system call, if a lock contention
happens, however in current implemenation, the lock holder has to extend its
locking time and enter kernel to unlock it, the change avoids this disadvantage,
it first sets mutex to free state and then enters kernel and wake one waiter
up. This improves performance dramatically in some sysbench mutex tests.

Tested by: kris
Sounds great: jeff
2008-06-24 07:32:12 +00:00
davidxu
c0f6b35a3a - Reduce function call overhead for uncontended case.
- Remove unused flags MUTEX_FLAGS_* and their code.
- Check validity of the timeout parameter in mutex_self_lock().
2008-05-29 07:57:33 +00:00
kris
4a87c82b19 Increase the default MUTEX_ADAPTIVE_SPINS to 2000, after further
testing it turns out 200 was too short to give good adaptive
performance.

Reviewed by:   jeff
MFC after:     1 week
2008-04-26 13:19:07 +00:00
ru
e2f2131976 Fixed mis-implementation of pthread_mutex_get{spin,yield}loops_np().
Reviewed by:	davidxu
2008-03-25 09:48:10 +00:00
des
c5334cac08 _pthread_mutex_isowned_np(): use a more reliable method; the current code
will work in simple cases, but may fail in more complicated ones.

Reviewed by:	davidxu
2008-02-14 12:37:58 +00:00
des
f006d1f25a Remove unnecessary prototype. 2008-02-06 20:43:19 +00:00
des
053e111aba Per discussion on -threads, rename _islocked_np() to _isowned_np(). 2008-02-06 19:34:31 +00:00
des
c089534891 After careful consideration (and a brief discussion with attilio@), change
the semantics of pthread_mutex_islocked_np() to return true if and only if
the mutex is held by the current thread.

Obviously, change the regression test to match.

MFC after:	2 weeks
2008-02-04 12:35:23 +00:00
des
72d185548f Add pthread_mutex_islocked_np(), a cheap way to verify that a mutex is
locked.  This is intended primarily to support the userland equivalent
of the various *_ASSERT_LOCKED() macros we have in the kernel.

MFC after:	2 weeks
2008-02-03 22:38:10 +00:00
davidxu
42e5c27a87 Add function prototypes. 2007-12-17 02:53:11 +00:00
davidxu
5340a5f502 1. Add function pthread_mutex_setspinloops_np to turn a mutex's spin
loop count.
2. Add function pthread_mutex_setyieldloops_np to turn a mutex's yield
   loop count.
3. Make environment variables PTHREAD_SPINLOOPS and PTHREAD_YIELDLOOPS
   to be only used for turnning PTHREAD_MUTEX_ADAPTIVE_NP mutex.
2007-12-14 06:25:57 +00:00
davidxu
50439b0991 Enclose all code for macro ENQUEUE_MUTEX in do while statement, and
add missing brackets.

MFC: after 1 day
2007-12-11 08:00:58 +00:00
jasone
600513aa8d Fix pointer dereferencing problems in _pthread_mutex_init_calloc_cb() that
were obscured by pseudo-opaque pthreads API pointer casting.
2007-11-28 00:16:24 +00:00
jasone
21bb948195 Add _pthread_mutex_init_calloc_cb() to libthr and libkse, so that malloc(3)
(part of libc) can use pthreads mutexes without causing infinite recursion
during initialization.
2007-11-27 03:16:44 +00:00
davidxu
a0922e4b91 Convert ceiling type to unsigned integer before comparing, fix compiler
warnings.
2007-11-21 05:25:27 +00:00
davidxu
c01b764192 Avoid doing adaptive spinning for priority protected mutex, current
implementation always does lock in kernel.
2007-10-31 01:50:48 +00:00
davidxu
674cdbbcee Don't do adaptive spinning if it is running on UP kernel. 2007-10-31 01:44:50 +00:00
davidxu
e199852bb6 Restore revision 1.55, the kris's adaptive mutex type. 2007-10-31 01:37:13 +00:00
kris
9e91eb96b8 Adaptive mutexes should have the same deadlock detection properties that
default (errorcheck) mutexes do.

Noticed by:          davidxu
2007-10-30 09:24:23 +00:00
davidxu
97a20b1db7 Add my recent work of adaptive spin mutex code. Use two environments variable
to tune pthread mutex performance:
1. LIBPTHREAD_SPINLOOPS
	If a pthread mutex is being locked by another thread, this environment
	variable sets total number of spin loops before the current thread
	sleeps in kernel, this saves a syscall overhead if the mutex will be
	unlocked very soon (well written application code).
2. LIBPTHREAD_YIELDLOOPS
	If a pthread mutex is being locked by other threads, this environment
	variable sets total number of sched_yield() loops before the currrent
	thread sleeps in kernel. if a pthread mutex is locked, the current thread
	gives up cpu, but will not sleep in kernel, this means, current thread
	does not set contention bit in mutex, but let lock owner to run again
	if the owner is on kernel's run queue, and when lock owner unlocks the
	mutex, it does not need to enter kernel and do lots of work to resume
	mutex waiters, in some cases, this saves lots of syscall overheads for
	mutex owner.

In my practice, sometimes LIBPTHREAD_YIELDLOOPS can massively improve performance
than LIBPTHREAD_SPINLOOPS, this depends on application. These two environments
are global to all pthread mutex, there is no interface to set them for each
pthread mutex, the default values are zero, this means spinning is turned off
by default.
2007-10-30 05:57:37 +00:00
kris
bbfd76f872 Add a new "non-portable" mutex type, PTHREAD_MUTEX_ADAPTIVE_NP. This
is also implemented in glibc and is used by a number of existing
applications (mysql, firefox, etc).

This mutex type is a default mutex with the additional property that
it spins briefly when attempting to acquire a contested lock, doing
trylock operations in userland before entering the kernel to block if
eventually unsuccessful.

The expectation is that applications requesting this mutex type know
that the mutex is likely to be only held for very brief periods, so it
is faster to spin in userland and probably succeed in acquiring the
mutex, than to enter the kernel and sleep, only to be woken up almost
immediately.  This can help significantly in certain cases when
pthread mutexes are heavily contended and held for brief durations
(such as mysql).

Spin up to 200 times before entering the kernel, which represents only
a few us on modern CPUs.  No performance degradation was observed with
this value and it is sufficient to avoid a large performance drop in
mysql performance in the heavily contended pthread mutex case.

The libkse implementation is a NOP.

Reviewed by:      jeff
MFC after:        3 days
2007-10-29 21:01:47 +00:00
davidxu
d97c4f1e52 backout experimental adaptive spinning mutex for product use. 2007-05-09 08:39:33 +00:00
davidxu
9770c4c640 Insert mutex at tail if it has highest ceiling. 2007-01-05 03:57:11 +00:00
davidxu
dec2b546dd Oops, don't corrupt the list. 2007-01-05 03:33:47 +00:00
davidxu
190109deab Check if the PP mutex is recursive, if we have already locked it, place the
mutex in right order sorted by priority ceiling.
2007-01-05 03:29:15 +00:00
davidxu
e034ab54f2 Check environment variable PTHREAD_ADAPTIVE_SPIN, if it is set, use
it as a default spin cycle count.
2006-12-20 04:43:34 +00:00
davidxu
26cbb63b3f Create inline function _thr_umutex_trylock2 to only try one atomic
operation, if it is failed, we call syscall directly, this saves
one atomic operation per lock contention.
2006-12-14 13:22:02 +00:00
davidxu
cc0840138e Move code calculating new inherited priority into single function. 2006-11-11 13:33:47 +00:00
davidxu
e83ab88356 Use return value of _thr_umutex_lock instead of using zero. 2006-09-08 09:29:14 +00:00
davidxu
58fc7458af Use umutex APIs to implement pthread_mutex, member pp_mutexq is added
into pthread structure to keep track of locked PTHREAD_PRIO_PROTECT mutex,
no real mutex code is changed, the mutex locking and unlocking code should
has same performance as before.
2006-08-28 04:52:50 +00:00
davidxu
c8a98491d0 Axe unused member field. 2006-08-08 05:04:43 +00:00
delphij
36c285d128 Unexpand two TAILQ_FOREACH_SAFE cases.
Ok'ed by:	davidxu
2006-07-17 09:23:44 +00:00
davidxu
523fcb36a8 Remove unused member field m_queue. 2006-06-02 08:37:01 +00:00
davidxu
7d0c23506c Do not check validity of timeout if a mutex can be acquired immediately.
Completly drop recursive mutex in pthread_cond_wait and restore recursive
after resumption. Reorganize code to make gcc to generate better code.
2006-04-08 13:24:44 +00:00
davidxu
31f2b819c6 WARNS level 4 cleanup. 2006-04-04 02:57:49 +00:00
davidxu
255936645e Remove priority mutex code because it does not work correctly,
to make it work, turnstile like mechanism to support priority
propagating and other realtime scheduling options in kernel
should be available to userland mutex, for the moment, I just
want to make libthr be simple and efficient thread library.

Discussed with: deischen, julian
2006-03-27 23:50:21 +00:00
davidxu
6fdf2b7d81 Reimplement mutex_init to get rid of compile warning. 2006-02-28 06:06:19 +00:00
davidxu
97a1cdabb8 Eliminate unused code. 2006-01-16 05:33:48 +00:00
davidxu
aedf24a684 Enable mutex inheritance code in mutex_fork, I forgot to turn on it.
while here, add some comments about process shared mutex.
2006-01-14 11:33:43 +00:00
davidxu
2d34e45c94 Let _mutex_cv_lock call internal functiona mutex_lock_common. 2005-12-21 05:14:07 +00:00
davidxu
7bdde27518 Remove unused _get_curthread() call. 2005-12-12 07:14:57 +00:00