Commit Graph

294 Commits

Author SHA1 Message Date
davidxu
674cdbbcee Don't do adaptive spinning if it is running on UP kernel. 2007-10-31 01:44:50 +00:00
davidxu
e199852bb6 Restore revision 1.55, the kris's adaptive mutex type. 2007-10-31 01:37:13 +00:00
kris
9e91eb96b8 Adaptive mutexes should have the same deadlock detection properties that
default (errorcheck) mutexes do.

Noticed by:          davidxu
2007-10-30 09:24:23 +00:00
davidxu
97a20b1db7 Add my recent work of adaptive spin mutex code. Use two environments variable
to tune pthread mutex performance:
1. LIBPTHREAD_SPINLOOPS
	If a pthread mutex is being locked by another thread, this environment
	variable sets total number of spin loops before the current thread
	sleeps in kernel, this saves a syscall overhead if the mutex will be
	unlocked very soon (well written application code).
2. LIBPTHREAD_YIELDLOOPS
	If a pthread mutex is being locked by other threads, this environment
	variable sets total number of sched_yield() loops before the currrent
	thread sleeps in kernel. if a pthread mutex is locked, the current thread
	gives up cpu, but will not sleep in kernel, this means, current thread
	does not set contention bit in mutex, but let lock owner to run again
	if the owner is on kernel's run queue, and when lock owner unlocks the
	mutex, it does not need to enter kernel and do lots of work to resume
	mutex waiters, in some cases, this saves lots of syscall overheads for
	mutex owner.

In my practice, sometimes LIBPTHREAD_YIELDLOOPS can massively improve performance
than LIBPTHREAD_SPINLOOPS, this depends on application. These two environments
are global to all pthread mutex, there is no interface to set them for each
pthread mutex, the default values are zero, this means spinning is turned off
by default.
2007-10-30 05:57:37 +00:00
kris
bbfd76f872 Add a new "non-portable" mutex type, PTHREAD_MUTEX_ADAPTIVE_NP. This
is also implemented in glibc and is used by a number of existing
applications (mysql, firefox, etc).

This mutex type is a default mutex with the additional property that
it spins briefly when attempting to acquire a contested lock, doing
trylock operations in userland before entering the kernel to block if
eventually unsuccessful.

The expectation is that applications requesting this mutex type know
that the mutex is likely to be only held for very brief periods, so it
is faster to spin in userland and probably succeed in acquiring the
mutex, than to enter the kernel and sleep, only to be woken up almost
immediately.  This can help significantly in certain cases when
pthread mutexes are heavily contended and held for brief durations
(such as mysql).

Spin up to 200 times before entering the kernel, which represents only
a few us on modern CPUs.  No performance degradation was observed with
this value and it is sufficient to avoid a large performance drop in
mysql performance in the heavily contended pthread mutex case.

The libkse implementation is a NOP.

Reviewed by:      jeff
MFC after:        3 days
2007-10-29 21:01:47 +00:00
davidxu
5523a9bb34 Use macro THR_CLEANUP_PUSH/POP, they are cheaper than pthread_cleanup_push/pop. 2007-10-16 07:46:15 +00:00
davidxu
38b01da7d2 Reverse the logic of UP and SMP.
Submitted by: jasone
2007-10-16 07:36:02 +00:00
davidxu
52bd1282cd Output error message to STDERR_FILENO.
Approved by: re (bmah)
2007-08-07 04:50:14 +00:00
davidxu
d97c4f1e52 backout experimental adaptive spinning mutex for product use. 2007-05-09 08:39:33 +00:00
davidxu
2dd85eafce If a thread who's name is being set is not the current thread, use macros
THR_THREAD_LOCK and THR_THREAD_UNLOCK instead, this should fix wrong
lock level problem.

Bug reported by: ed dot maste at gmail dot com
2007-04-05 07:20:31 +00:00
imp
9109b1ceb8 Remove 3rd clause, renumber, ok per email 2007-01-12 07:26:21 +00:00
davidxu
9770c4c640 Insert mutex at tail if it has highest ceiling. 2007-01-05 03:57:11 +00:00
davidxu
dec2b546dd Oops, don't corrupt the list. 2007-01-05 03:33:47 +00:00
davidxu
190109deab Check if the PP mutex is recursive, if we have already locked it, place the
mutex in right order sorted by priority ceiling.
2007-01-05 03:29:15 +00:00
davidxu
d305995d66 get LIBPTHREAD_ADAPTIVE_SPIN early, so it can be used for some global
mutexes.
2006-12-20 05:05:44 +00:00
davidxu
e034ab54f2 Check environment variable PTHREAD_ADAPTIVE_SPIN, if it is set, use
it as a default spin cycle count.
2006-12-20 04:43:34 +00:00
davidxu
ca27871833 - Remove variable _thr_scope_system, all threads are system scope.
- Rename _thr_smp_cpus to boolean variable _thr_is_smp.
- Define CPU_SPINWAIT macro for each arch, only X86 supports it.
2006-12-15 11:52:01 +00:00
davidxu
26cbb63b3f Create inline function _thr_umutex_trylock2 to only try one atomic
operation, if it is failed, we call syscall directly, this saves
one atomic operation per lock contention.
2006-12-14 13:22:02 +00:00
davidxu
229aca4634 Correctly check failed syscall. 2006-12-12 05:26:39 +00:00
davidxu
fcda4340a4 Move checking for c_has_waiters into low level _thr_ucond_signal and
_thr_ucond_broadcast, clear condition variable pointer in cancellation
info after returing from _thr_ucond_wait, since kernel has already
dropped the internal lock, so we don't need to unlock it in cancellation
handler again.
2006-12-12 03:08:49 +00:00
davidxu
33168bd24f test cancel_pending to save a thr_wake call in some specical cases. 2006-12-06 00:15:35 +00:00
davidxu
b01e86cf0a _thr_ucond_wait drops lock, we should pick it up again. 2006-12-05 23:46:11 +00:00
davidxu
3f73376426 the c_has_waiters is lazily updated, temporarily disable the false
alarm code.
2006-12-05 07:23:58 +00:00
davidxu
f26900460b Use ucond to implement barrier. 2006-12-05 06:54:25 +00:00
davidxu
f1f5293365 Add _thr_ucond_init(). 2006-12-05 06:53:44 +00:00
davidxu
19c999e75f Tweak _thr_cancel_leave_defer a bit to fix a possible race. 2006-12-05 05:01:57 +00:00
davidxu
5ad129a10a Fix typo, I was using a wrong header file, and the typo is not detected
by compiler.
2006-12-04 14:27:42 +00:00
davidxu
cbb0fd8174 Use kernel provided userspace condition variable to implement pthread
condition variable.
2006-12-04 14:20:41 +00:00
davidxu
61e0842016 If a thread was detached, return EINVAL instead, the error code
is also returned by pthread_detach() if a thread was already
detached, the error code was already documented:

>    [EINVAL]	The implementation has detected that the value speci-
>		fied by thread does not refer to a joinable thread.
2006-11-28 11:05:31 +00:00
davidxu
add205129c Eliminate atomic operations in thread cancellation functions, it should
reduce overheads of cancellation points.
2006-11-24 09:57:38 +00:00
davidxu
cc0840138e Move code calculating new inherited priority into single function. 2006-11-11 13:33:47 +00:00
davidxu
e8a2970aaf Don't inherit THR_FLAGS_NEED_SUSPEND for child process, child process
only has one thread, setting the flag can cause the thread to be
suspended and no another thread will resume it.
2006-10-14 13:40:08 +00:00
davidxu
653013bedc o Make _thr_umutex_init a function.
o Eliminate unused parameter for some functions.
o Convert type of first parameter to void * for _thr_umtx_wait
  and _thr_umtx_wake.
2006-10-13 22:31:00 +00:00
davidxu
4fdad8dba6 Use type pthread_state for thread state. 2006-10-13 12:45:21 +00:00
davidxu
d2c57b7fad use rtprio_thread system call to get or set thread priority. 2006-09-21 04:21:30 +00:00
davidxu
e83ab88356 Use return value of _thr_umutex_lock instead of using zero. 2006-09-08 09:29:14 +00:00
davidxu
21e4536026 Replace internal usage of struct umtx with umutex which can supports
real-time if we want, no functionality is changed.
2006-09-06 04:04:10 +00:00
davidxu
567ba06917 Same as pthread_setschedparam, use sizeof(struct sched_param) instead. 2006-09-05 14:39:06 +00:00
davidxu
5b28602728 Pass correct parameter size. 2006-09-05 14:37:22 +00:00
davidxu
37473cc569 Remove unused file. 2006-08-29 13:01:23 +00:00
davidxu
faa99f1910 pthread_sigmask is in thr_sig.c, remove this file. 2006-08-28 12:29:54 +00:00
davidxu
b7dde83f82 Kill unused files. 2006-08-28 05:01:31 +00:00
davidxu
58fc7458af Use umutex APIs to implement pthread_mutex, member pp_mutexq is added
into pthread structure to keep track of locked PTHREAD_PRIO_PROTECT mutex,
no real mutex code is changed, the mutex locking and unlocking code should
has same performance as before.
2006-08-28 04:52:50 +00:00
davidxu
77e7cda2cf Add umutex APIs. 2006-08-28 04:47:27 +00:00
davidxu
c8a98491d0 Axe unused member field. 2006-08-08 05:04:43 +00:00
davidxu
cfa46376c7 Get number of CPUs and ignore spin count on single processor machine. 2006-08-08 04:42:41 +00:00
davidxu
5193e44c40 1. Don't override underscore version of aio_suspend(), system(),
wait(), waitpid() and usleep(), they are internal versions and
   should not be cancellation points.
2. Make wait3() as a cancellation point.
3. Move raise() and pause() into file thr_sig.c.
4. Add functions _sigsuspend, _sigwait, _sigtimedwait and _sigwaitinfo,
   remove SIGCANCEL bit in wait-set for those functions, the signal is
   used internally to implement thread cancellation.
2006-07-25 12:50:05 +00:00
delphij
36c285d128 Unexpand two TAILQ_FOREACH_SAFE cases.
Ok'ed by:	davidxu
2006-07-17 09:23:44 +00:00
davidxu
2b1dbc0acb Caching scheduling policy and priority in userland, a critical but baddly
written application is frequently changing thread priority for SCHED_OTHER
policy.
2006-07-13 22:45:19 +00:00
davidxu
19b8418903 Use thr_setscheduler, thr_getscheduler and thr_setschedparam to implement
pthread functions.
2006-07-13 06:35:43 +00:00
davidxu
ecacf536b0 Use kernel facilities to support real-time scheduling. 2006-07-12 06:13:18 +00:00
davidxu
be54fdbbff Remove unused member. 2006-06-03 00:19:40 +00:00
davidxu
523fcb36a8 Remove unused member field m_queue. 2006-06-02 08:37:01 +00:00
delphij
6e4a34202a Explicitly request pre-zeroed memory instead of memset'ing our
own.

Ok'ed by:	davidxu
2006-05-31 00:31:38 +00:00
davidxu
6a9d879cfe s/long/int. 2006-04-27 08:25:13 +00:00
davidxu
66d0fee031 - Use same priority range returned by kernel's sched_get_priority_min()
and sched_get_priority_max() syscalls.
- Remove unused fields from structure pthread_attr.
2006-04-27 08:18:23 +00:00
davidxu
8001df6f90 Remove multiple _get_curthread() calls. 2006-04-23 11:23:37 +00:00
davidxu
7d0c23506c Do not check validity of timeout if a mutex can be acquired immediately.
Completly drop recursive mutex in pthread_cond_wait and restore recursive
after resumption. Reorganize code to make gcc to generate better code.
2006-04-08 13:24:44 +00:00
davidxu
31f2b819c6 WARNS level 4 cleanup. 2006-04-04 02:57:49 +00:00
des
5438ed2ce6 Fix prototype mismatch. 2006-03-28 21:46:55 +00:00
davidxu
255936645e Remove priority mutex code because it does not work correctly,
to make it work, turnstile like mechanism to support priority
propagating and other realtime scheduling options in kernel
should be available to userland mutex, for the moment, I just
want to make libthr be simple and efficient thread library.

Discussed with: deischen, julian
2006-03-27 23:50:21 +00:00
davidxu
32673f0256 Check cancellation state carefully to see we really need to call
_pthread_testcancel(). Preserve errno in _thr_suspend_check().
2006-03-26 01:57:03 +00:00
davidxu
2926ddf5d1 Don't cancel thread if it is in critical region. 2006-03-25 07:03:13 +00:00
davidxu
679e2338c0 Only wake up writer if the lock is free. 2006-03-25 05:14:21 +00:00
davidxu
cba4d932c1 Compile thr_rtld.c 2006-03-25 05:00:54 +00:00
davidxu
6ee36bbb6d Add locking support for rtld. 2006-03-25 04:49:07 +00:00
davidxu
eb9bdb4259 set an upper limit for thread structures which can be allocated. 2006-03-24 04:34:06 +00:00
davidxu
020594940d Set default contention scope to system. 2006-03-20 03:14:14 +00:00
davidxu
d119bdaa7a Remove mqueue and timer, now they are in librt. 2006-03-08 23:47:04 +00:00
deischen
e586a0042e Add some more pthread stubs so that librt can use them.
The thread jump table has been resorted, so you need to
keep libc, libpthread, and libthr in sync.

Submitted by:	xu
2006-03-05 18:10:28 +00:00
davidxu
6fdf2b7d81 Reimplement mutex_init to get rid of compile warning. 2006-02-28 06:06:19 +00:00
davidxu
bb43155dbb Disable POSIX SIGEV_THREAD notification support, soon we will have a common
library which can be reused both for libthr and libpthread.
2006-02-22 02:52:22 +00:00
davidxu
91fda076fb Read cycle count before sending signal, this fixes a userland thread
suspension bug.

MFC after: 3 days
2006-02-20 09:02:40 +00:00
davidxu
70ea4e1323 Rework last change of pthread_once, create a function _thr_once_init to
reinitialize its internal locks.
2006-02-15 23:05:03 +00:00
davidxu
44a8ce5006 After fork(), reinitialize internal locks for pthread_once(). 2006-02-15 13:41:02 +00:00
davidxu
08b1c5c928 Now, thread name is stored in kernel, userland no longer has to keep it. 2006-02-05 03:04:54 +00:00
davidxu
118990614c use syscall thr_set_name to implement pthread_set_name_np. 2006-02-05 02:26:17 +00:00
davidxu
3172ccca0f Add missing symbol pthread_set_name_np. 2006-01-23 06:11:23 +00:00
davidxu
97a1cdabb8 Eliminate unused code. 2006-01-16 05:33:48 +00:00
davidxu
aedf24a684 Enable mutex inheritance code in mutex_fork, I forgot to turn on it.
while here, add some comments about process shared mutex.
2006-01-14 11:33:43 +00:00
jasone
e30d92806f Increase the number of spinlocks, since libc's malloc implementation is
about to significantly increase the number of spinlocks used.

Approved by:	markm (mentor)
2006-01-13 06:14:04 +00:00
jasone
3668a2e494 In preparation for a new malloc implementation:
* Add posix_memalign().

  * Move calloc() from calloc.c to malloc.c.  Add a calloc() implementation in
    rtld-elf in order to make the loader happy (even though calloc() isn't
    used in rtld-elf).

  * Add _malloc_prefork() and _malloc_postfork(), and use them instead of
    directly manipulating __malloc_lock.

Approved by:	phk, markm (mentor)
2006-01-12 07:28:21 +00:00
davidxu
1218aa5627 Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into
static binary.
2006-01-10 04:53:03 +00:00
davidxu
36bb818343 Rescue pthread_set_name_np for compatible reason, remove unused code. 2006-01-09 08:07:22 +00:00
davidxu
8798f9bcec Tweak macro THR_LOCK_RELEASE a bit for non-PTHREAD_INVARIANTS case. 2006-01-09 07:32:22 +00:00
davidxu
db9594be6e Return real detached state. 2006-01-09 03:59:51 +00:00
davidxu
6f27a53b6f Fix a bug recently introduced, the _thread_active_count should be
decreased if thread can not be created.
2006-01-08 10:13:18 +00:00
davidxu
d1f7b6d605 Allow background threads to be suspended. 2006-01-08 01:49:31 +00:00
davidxu
4c4f339838 Try to reduce total time needed for suspending all threads,
first broadcast signals to all threads, then enter a wait loop.
2006-01-08 01:48:51 +00:00
davidxu
d6c88c0f27 Refine thread suspension code, now thread suspension is a blockable
operation, the caller is blocked util target threads are really
suspended, also avoid suspending a thread when it is holding a
critical lock.
Fix a bug in _thr_ref_delete which tests a never set flag.
2006-01-05 13:51:22 +00:00
davidxu
45180066f9 1. Add SIGEV_THREAD notification for mq_notify.
2. Reuse current timer code and abstract some common code to
   to support both timer and mqueue.
2006-01-04 11:48:02 +00:00
davidxu
46ad9344c2 Remove in-progress wait code to sync with libpthread's behavior. 2006-01-03 13:30:23 +00:00
davidxu
2d34e45c94 Let _mutex_cv_lock call internal functiona mutex_lock_common. 2005-12-21 05:14:07 +00:00
davidxu
5ac55e9c46 Hide umtx API symbols as well. 2005-12-21 03:53:29 +00:00
davidxu
8d9e23fb51 1. Retire macro SCLASS, instead simply use language keyword and
put variables in thr_init.c.
2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
davidxu
a8cc7e07c2 Clear return code to zero if joiner successfully waited joinee.
Bug reported by: jasone at connonware when using ports lang/onyx
MFC after: 3 days
2005-12-19 03:20:55 +00:00
davidxu
272f64aa70 Update copyright. 2005-12-17 09:42:45 +00:00
davidxu
7bdde27518 Remove unused _get_curthread() call. 2005-12-12 07:14:57 +00:00
davidxu
ae161ac239 Fix name compatible problem with POSIX standard. the sigval_ptr and
sigval_int really should be sival_ptr and sival_int.
Also sigev_notify_function accepts a union sigval value but not a
pointer.
2005-11-04 09:41:00 +00:00
davidxu
3e97a4d0bd Remove a redundant _get_curthread() call. 2005-11-02 14:06:29 +00:00