Commit Graph

404 Commits

Author SHA1 Message Date
Peter Wemm
70ba1e8fc1 When libthr and rtld start up, there are a number of magic spells cast
in order to get the symbol binding state "just so".  This is to allow
locking to be activated and not run into recursion problems later.

However, one of the magic bits involves an explicit call to _umtx_op()
to force symbol resolution.  It does a wakeup operation on a fake,
uninitialized (ie: random contents) umtx.  Since libthr isn't active, this
is harmless.  Nothing can match the random wakeup.

However, valgrind finds this and is not amused.  Normally I'd just
write a suppression record for it, but the idea of passing random
args to syscalls (on purpose) just doesn't feel right.
2008-12-07 02:32:49 +00:00
Konstantin Belousov
10b4034657 Provide custom simple allocator for rtld locks in libthr. The allocator
does not use any external symbols, thus avoiding possible recursion into
rtld to resolve symbols, when called.

Reviewed by:	kan, davidxu
Tested by:	rink
MFC after:	1 month
2008-12-02 11:58:31 +00:00
Alexander Kabaev
97df383415 Invoke _rtld_atfork_post earlier, before we reinitialize rtld locks
by switching into single-thread mode.

libthr ignores broken use of lock bitmaps used by default rtld locking
implementation, this in turn turns lock handoff in _rtld_thread_init
into NOP. This in turn makes child processes of forked multi-threaded
programs to run with _thr_signal_block still in effect, with most
signals blocked.

Reported by: phk, kib
2008-12-01 21:00:25 +00:00
Konstantin Belousov
e711c6f0d1 Unlock the malloc() locks in the child process after fork(). This gives
us working malloc in the fork child of the multithreaded process.

Although POSIX requires that only async-signal safe functions shall be
operable after fork in multithreaded process, not having malloc lower
the quality of our implementation.

Tested by:	rink
Discussed with:	kan, davidxu
Reviewed by:	kan
MFC after:	1 month
2008-11-29 21:46:28 +00:00
Konstantin Belousov
cb5c4b10ba Add two rtld exported symbols, _rtld_atfork_pre and _rtld_atfork_post.
Threading library calls _pre before the fork, allowing the rtld to
lock itself to ensure that other threads of the process are out of
dynamic linker. _post releases the locks.

This allows the rtld to have consistent state in the child. Although
child may legitimately call only async-safe functions, the call may
need plt relocation resolution, and this requires working rtld.

Reported and debugging help by:	rink
Reviewed by:	kan, davidxu
MFC after:	1 month (anyway, not before 7.1 is out)
2008-11-27 11:27:59 +00:00
Marcel Moolenaar
03fad2ad5f Allow psaddr_t to be widened by using thr_pread_{int,long,ptr},
where critical. Some places still use ps_pread/ps_pwrite directly,
but only need changed when byte-order comes into the picture.
Also, change th_p in td_event_msg_t from a pointer type to
psaddr_t, so that events also work when psaddr_t is widened.
2008-09-14 16:07:21 +00:00
Jason Evans
5b3842aefa Move call to _malloc_thread_cleanup() so that if this is the last thread,
the call never happens.  This is necessary because malloc may be used
during exit handler processing.

Submitted by:	davidxu
2008-09-09 17:14:32 +00:00
Jason Evans
d6742bfbd3 Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes.  The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes.  Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.
2008-08-27 02:00:53 +00:00
David Xu
fc45432be6 In function pthread_condattr_getpshared, store result correctly.
PR:		kern/126128
2008-08-01 01:21:49 +00:00
Ruslan Ermilov
042df2e2da Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
  turned opt-in for stable branches depending on the consensus.  You
  can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
  It is harmless to steal the knob as SSP symbols have been provided
  by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
  (sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
  libc will be automatically downgraded to -fstack-protector because it
  breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by:	Jeremie Le Hen <jeremie@le-hen.org>
2008-06-25 21:33:28 +00:00
David Xu
7de1ecef2d Add two commands to _umtx_op system call to allow a simple mutex to be
locked and unlocked completely in userland. by locking and unlocking mutex
in userland, it reduces the total time a mutex is locked by a thread,
in some application code, a mutex only protects a small piece of code, the
code's execution time is less than a simple system call, if a lock contention
happens, however in current implemenation, the lock holder has to extend its
locking time and enter kernel to unlock it, the change avoids this disadvantage,
it first sets mutex to free state and then enters kernel and wake one waiter
up. This improves performance dramatically in some sysbench mutex tests.

Tested by: kris
Sounds great: jeff
2008-06-24 07:32:12 +00:00
David Xu
83a0758789 Make pthread_cleanup_push() and pthread_cleanup_pop() as a pair of macros,
use stack space to keep cleanup information, this eliminates overhead of
calling malloc() and free() in thread library.

Discussed on: thread@
2008-06-09 01:14:10 +00:00
Doug Rabson
cd7d66a21f Call the fcntl compatiblity wrapper from the thread library fcntl wrappers
so that they get the benefit of the (limited) forward ABI compatibility.

MFC after: 1 week
2008-05-30 14:47:42 +00:00
David Xu
1b3418b2dc Eliminate global mutex by using pthread_once's state field as
a semaphore.
2008-05-30 00:02:59 +00:00
David Xu
850f4d66cb - Reduce function call overhead for uncontended case.
- Remove unused flags MUTEX_FLAGS_* and their code.
- Check validity of the timeout parameter in mutex_self_lock().
2008-05-29 07:57:33 +00:00
Warner Losh
13d2e92b70 Commit missing mips libthr support that I thought I'd committed earlier 2008-05-11 05:54:52 +00:00
David Xu
cf181aee60 Remove libc_r's remnant code. 2008-05-06 07:27:11 +00:00
David Xu
8d6a11a070 Use UMTX_OP_WAIT_UINT_PRIVATE and UMTX_OP_WAKE_PRIVATE to save
time in kernel(avoid VM lookup).
2008-04-29 03:58:18 +00:00
Kris Kennaway
dd77f9f7f2 Increase the default MUTEX_ADAPTIVE_SPINS to 2000, after further
testing it turns out 200 was too short to give good adaptive
performance.

Reviewed by:   jeff
MFC after:     1 week
2008-04-26 13:19:07 +00:00
Warner Losh
416e6cfbf0 Bring in mips threads support from perforce mips2-jnpr branch. 2008-04-26 12:17:57 +00:00
Xin LI
d0aa4fd3ca Avoid various shadowed variables. libthr is now almost WARNS=4 clean except
for some const dequalifiers that needs more careful investigation.

Ok'ed by:	davidxu
2008-04-23 21:06:51 +00:00
David Xu
fb2641d9b1 Use native rwlock. 2008-04-22 06:44:11 +00:00
David Xu
6d9517bc9f _vfork is not in libthr, remove the reference. 2008-04-16 03:19:11 +00:00
David Xu
fa4b421a7a don't include pthread_np.h, it is not used. 2008-04-14 08:08:40 +00:00
David Xu
caad30a422 put THR_CRITICAL_LEAVE into do .. while statement. 2008-04-03 02:47:35 +00:00
David Xu
a6cba9400a add __hidden suffix to _umtx_op_err, this eliminates PLT. 2008-04-03 02:13:51 +00:00
David Xu
7abb97dcd8 Non-portable functions are in pthread_np.h, fix compiling problem. 2008-04-02 11:41:12 +00:00
David Xu
7a30bcf04b Add pthread_setaffinity_np and pthread_getaffinity_np to libc namespace. 2008-04-02 08:53:18 +00:00
David Xu
8b873a2328 Remove unused functions. 2008-04-02 08:33:42 +00:00
David Xu
d6e0eb0a48 Replace function _umtx_op with _umtx_op_err, the later function directly
returns errno, because errno can be mucked by user's signal handler and
most of pthread api heavily depends on errno to be correct, this change
should improve stability of the thread library.
2008-04-02 07:41:25 +00:00
David Xu
8bf1a48cb3 Replace userland rwlock with a pure kernel based rwlock, the new
implementation does not switch pointers when it resumes waiters.

Asked by: jeff
2008-04-02 04:32:31 +00:00
David Xu
18967c1918 Restore normal pthread_cond_signal path to avoid some obscure races. 2008-04-01 06:23:08 +00:00
David Xu
f5bc4f9930 return EAGAIN early rather than running bunch of code later, micro optimize
static branch prediction.
2008-04-01 00:21:49 +00:00
David Xu
5ab512bb8e Rewrite rwlock to user atomic operations to change rwlock state, this
eliminates internal mutex lock contention when most rwlock operations
are read.

Orignal patch provided by: jeff
2008-03-31 02:55:49 +00:00
Ruslan Ermilov
dbdb679c6f Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse.  This should fix WITHOUT_LIBTHR builds as a side effect.
2008-03-29 17:44:40 +00:00
Ruslan Ermilov
e03efb02bc Compile libthr with warnings. 2008-03-25 13:28:12 +00:00
Ruslan Ermilov
7e0e78248e Fixed mis-implementation of pthread_mutex_get{spin,yield}loops_np().
Reviewed by:	davidxu
2008-03-25 09:48:10 +00:00
David Xu
9939a13667 Add POSIX pthread API pthread_getcpuclockid() to get a thread's cpu
time clock id.
2008-03-22 09:59:20 +00:00
David Xu
04a57d2c83 Resolve __error()'s PLT early so that it needs not to be resolved again,
otherwise rwlock is recursivly called when signal happens and the __error
was never resolved before.
2008-03-21 02:31:55 +00:00
Ruslan Ermilov
a1292a02d3 pthread_mutexattr_destroy() was accidentally broken in last revision,
unbreak it.  We should really start compiling this with warnings.
2008-03-20 11:47:08 +00:00
David Xu
8c38215f50 Preserve application code's errno in rtld locking code, it attemps to keep
any case safe.
2008-03-20 09:35:44 +00:00
David Xu
48ebe2ebc4 Make pthread_mutexattr_settype to return error number directly and
conformant to POSIX specification.

Bug reported by: modelnine at modelnine dt org
2008-03-20 08:27:14 +00:00
David Xu
c8a4eae56f don't reduce new thread's refcount if current thread can not set cpuset
for it, since the new thread will reduce it by itself.
2008-03-19 09:33:07 +00:00
David Xu
519e8d87bb - Trim trailing spaces.
- Use a different sigmask variable name to avoid confusing.
2008-03-19 08:13:04 +00:00
David Xu
86a06c6000 if passed thread pointer is equal to current thread, pass -1 to kernel
to speed up searching.
2008-03-19 06:38:21 +00:00
David Xu
2ea1f90a18 - Copy signal mask out before THR_UNLOCK(), because THR_UNLOCK() may call
_thr_suspend_check() which messes sigmask saved in thread structure.
- Don't suspend a thread has force_exit set.
- In pthread_exit(), if there is a suspension flag set, wake up waiting-
  thread after setting PS_DEAD, this causes waiting-thread to break loop
  in suspend_common().
2008-03-18 02:06:51 +00:00
David Xu
a9a11568ff Actually delete SIGCANCEL mask for suspended thread, so the signal will not
be masked when it is resumed.
2008-03-16 03:22:38 +00:00
David Xu
150b71918c If a thread is cancelled, it may have already consumed a umtx_wake,
check waiter and semphore counter to see if we may wake up next thread.
2008-03-11 03:26:47 +00:00
David Xu
8a18c0d3c8 Fix a bug when calculating remnant size. 2008-03-06 03:24:03 +00:00
David Xu
697b4b49be Don't report death event to debugger if it is a forced exit. 2008-03-06 02:07:18 +00:00