Commit Graph

398 Commits

Author SHA1 Message Date
jasone
a734052e9c Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes.  The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes.  Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.
2008-08-27 02:00:53 +00:00
davidxu
0cc238e339 In function pthread_condattr_getpshared, store result correctly.
PR:		kern/126128
2008-08-01 01:21:49 +00:00
ru
8735fdbd4c Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
  turned opt-in for stable branches depending on the consensus.  You
  can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
  It is harmless to steal the knob as SSP symbols have been provided
  by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
  (sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
  libc will be automatically downgraded to -fstack-protector because it
  breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by:	Jeremie Le Hen <jeremie@le-hen.org>
2008-06-25 21:33:28 +00:00
davidxu
70dd244f26 Add two commands to _umtx_op system call to allow a simple mutex to be
locked and unlocked completely in userland. by locking and unlocking mutex
in userland, it reduces the total time a mutex is locked by a thread,
in some application code, a mutex only protects a small piece of code, the
code's execution time is less than a simple system call, if a lock contention
happens, however in current implemenation, the lock holder has to extend its
locking time and enter kernel to unlock it, the change avoids this disadvantage,
it first sets mutex to free state and then enters kernel and wake one waiter
up. This improves performance dramatically in some sysbench mutex tests.

Tested by: kris
Sounds great: jeff
2008-06-24 07:32:12 +00:00
davidxu
f4d6ff9c5e Make pthread_cleanup_push() and pthread_cleanup_pop() as a pair of macros,
use stack space to keep cleanup information, this eliminates overhead of
calling malloc() and free() in thread library.

Discussed on: thread@
2008-06-09 01:14:10 +00:00
dfr
a6bd1d1955 Call the fcntl compatiblity wrapper from the thread library fcntl wrappers
so that they get the benefit of the (limited) forward ABI compatibility.

MFC after: 1 week
2008-05-30 14:47:42 +00:00
davidxu
8951bcd14c Eliminate global mutex by using pthread_once's state field as
a semaphore.
2008-05-30 00:02:59 +00:00
davidxu
c0f6b35a3a - Reduce function call overhead for uncontended case.
- Remove unused flags MUTEX_FLAGS_* and their code.
- Check validity of the timeout parameter in mutex_self_lock().
2008-05-29 07:57:33 +00:00
imp
b9242ed45b Commit missing mips libthr support that I thought I'd committed earlier 2008-05-11 05:54:52 +00:00
davidxu
fc58e99cef Remove libc_r's remnant code. 2008-05-06 07:27:11 +00:00
davidxu
0e9d39ae8f Use UMTX_OP_WAIT_UINT_PRIVATE and UMTX_OP_WAKE_PRIVATE to save
time in kernel(avoid VM lookup).
2008-04-29 03:58:18 +00:00
kris
4a87c82b19 Increase the default MUTEX_ADAPTIVE_SPINS to 2000, after further
testing it turns out 200 was too short to give good adaptive
performance.

Reviewed by:   jeff
MFC after:     1 week
2008-04-26 13:19:07 +00:00
imp
92f18b23d5 Bring in mips threads support from perforce mips2-jnpr branch. 2008-04-26 12:17:57 +00:00
delphij
6b7d752076 Avoid various shadowed variables. libthr is now almost WARNS=4 clean except
for some const dequalifiers that needs more careful investigation.

Ok'ed by:	davidxu
2008-04-23 21:06:51 +00:00
davidxu
b794176bc5 Use native rwlock. 2008-04-22 06:44:11 +00:00
davidxu
6a349c6771 _vfork is not in libthr, remove the reference. 2008-04-16 03:19:11 +00:00
davidxu
a1371575f4 don't include pthread_np.h, it is not used. 2008-04-14 08:08:40 +00:00
davidxu
8d9f007088 put THR_CRITICAL_LEAVE into do .. while statement. 2008-04-03 02:47:35 +00:00
davidxu
2d5bf7e6fc add __hidden suffix to _umtx_op_err, this eliminates PLT. 2008-04-03 02:13:51 +00:00
davidxu
f1df18eb48 Non-portable functions are in pthread_np.h, fix compiling problem. 2008-04-02 11:41:12 +00:00
davidxu
d00a98a4e8 Add pthread_setaffinity_np and pthread_getaffinity_np to libc namespace. 2008-04-02 08:53:18 +00:00
davidxu
aa2019ec00 Remove unused functions. 2008-04-02 08:33:42 +00:00
davidxu
570834290d Replace function _umtx_op with _umtx_op_err, the later function directly
returns errno, because errno can be mucked by user's signal handler and
most of pthread api heavily depends on errno to be correct, this change
should improve stability of the thread library.
2008-04-02 07:41:25 +00:00
davidxu
8d50c29c72 Replace userland rwlock with a pure kernel based rwlock, the new
implementation does not switch pointers when it resumes waiters.

Asked by: jeff
2008-04-02 04:32:31 +00:00
davidxu
111802a962 Restore normal pthread_cond_signal path to avoid some obscure races. 2008-04-01 06:23:08 +00:00
davidxu
664ef9f365 return EAGAIN early rather than running bunch of code later, micro optimize
static branch prediction.
2008-04-01 00:21:49 +00:00
davidxu
220584ad61 Rewrite rwlock to user atomic operations to change rwlock state, this
eliminates internal mutex lock contention when most rwlock operations
are read.

Orignal patch provided by: jeff
2008-03-31 02:55:49 +00:00
ru
0f0375e36a Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse.  This should fix WITHOUT_LIBTHR builds as a side effect.
2008-03-29 17:44:40 +00:00
ru
e8df07e5aa Compile libthr with warnings. 2008-03-25 13:28:12 +00:00
ru
e2f2131976 Fixed mis-implementation of pthread_mutex_get{spin,yield}loops_np().
Reviewed by:	davidxu
2008-03-25 09:48:10 +00:00
davidxu
1423f22a4c Add POSIX pthread API pthread_getcpuclockid() to get a thread's cpu
time clock id.
2008-03-22 09:59:20 +00:00
davidxu
8420f5505a Resolve __error()'s PLT early so that it needs not to be resolved again,
otherwise rwlock is recursivly called when signal happens and the __error
was never resolved before.
2008-03-21 02:31:55 +00:00
ru
735c62ce43 pthread_mutexattr_destroy() was accidentally broken in last revision,
unbreak it.  We should really start compiling this with warnings.
2008-03-20 11:47:08 +00:00
davidxu
8326c06222 Preserve application code's errno in rtld locking code, it attemps to keep
any case safe.
2008-03-20 09:35:44 +00:00
davidxu
1bc96ca9dc Make pthread_mutexattr_settype to return error number directly and
conformant to POSIX specification.

Bug reported by: modelnine at modelnine dt org
2008-03-20 08:27:14 +00:00
davidxu
5e11ba1ac4 don't reduce new thread's refcount if current thread can not set cpuset
for it, since the new thread will reduce it by itself.
2008-03-19 09:33:07 +00:00
davidxu
a6c50de140 - Trim trailing spaces.
- Use a different sigmask variable name to avoid confusing.
2008-03-19 08:13:04 +00:00
davidxu
e846a1612b if passed thread pointer is equal to current thread, pass -1 to kernel
to speed up searching.
2008-03-19 06:38:21 +00:00
davidxu
9dbfb036ab - Copy signal mask out before THR_UNLOCK(), because THR_UNLOCK() may call
_thr_suspend_check() which messes sigmask saved in thread structure.
- Don't suspend a thread has force_exit set.
- In pthread_exit(), if there is a suspension flag set, wake up waiting-
  thread after setting PS_DEAD, this causes waiting-thread to break loop
  in suspend_common().
2008-03-18 02:06:51 +00:00
davidxu
3da38e35b4 Actually delete SIGCANCEL mask for suspended thread, so the signal will not
be masked when it is resumed.
2008-03-16 03:22:38 +00:00
davidxu
19ffe8108d If a thread is cancelled, it may have already consumed a umtx_wake,
check waiter and semphore counter to see if we may wake up next thread.
2008-03-11 03:26:47 +00:00
davidxu
cd00bbaa4b Fix a bug when calculating remnant size. 2008-03-06 03:24:03 +00:00
davidxu
6b41341850 Don't report death event to debugger if it is a forced exit. 2008-03-06 02:07:18 +00:00
davidxu
5f0ffdddf1 Restore code setting new thread's scheduler parameters, I was thinking
that there might be starvations, but because we have already locked the
thread, the cpuset settings will always be done before the new thread
does real-world work.
2008-03-06 01:59:08 +00:00
davidxu
1efa6566c1 Increase and decrease in_sigcancel_handler accordingly to avoid possible
error caused by nested SIGCANCEL stack, it is a bit complex.
2008-03-05 07:04:55 +00:00
davidxu
d6c532fa79 Use cpuset defined in pthread_attr for newly created thread, for now,
we set scheduling parameters and cpu binding fully in userland, and
because default scheduling policy is SCHED_RR (time-sharing), we set
default sched_inherit to PTHREAD_SCHED_INHERIT, this saves a system
call.
2008-03-05 07:01:20 +00:00
davidxu
b118d117f4 Add more cpu affinity function's symbols. 2008-03-05 06:56:35 +00:00
davidxu
adf8d28a8f Check actual size of cpuset kernel is using and define underscore version
of API.
2008-03-05 06:55:48 +00:00
davidxu
8ead1ed2f9 If a new thread is created, it inherits current thread's signal masks,
however if current thread is executing cancellation handler, signal
SIGCANCEL may have already been blocked, this is unexpected, unblock the
signal in new thread if this happens.

MFC after: 1 week
2008-03-04 04:28:59 +00:00
davidxu
e0d98325b4 Include cpuset.h, unbreak compiling. 2008-03-04 03:45:11 +00:00