d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
95a74ab4fb
freebsd-dev/lib/libthr/thread/thr_init.c

499 lines
17 KiB
C
Raw Normal View History

spdx: initial adoption of licensing ID tags. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point. Initially, only tag files that use BSD 4-Clause "Original" license. RelNotes: yes Differential Revision: https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
/*-
* SPDX-License-Identifier: BSD-4-Clause
*
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
* Copyright (c) 2003 Daniel M. Eischen <deischen@freebsd.org>
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
* Copyright (c) 1995-1998 John Birrell <jb@cimlogic.com.au>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by John Birrell.
* 4. Neither the name of the author nor the names of any co-contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY JOHN BIRRELL AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
Use __FBSDID() for .c files from lib/libthr/thread. Sponsored by: The FreeBSD Foundation
2016-04-08 11:15:26 +00:00
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include "namespace.h"
#include <sys/types.h>
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
#include <sys/signalvar.h>
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include <sys/ioctl.h>
Fix known issues which blow up the process after dlopen("libthr.so") (or loading a dso linked to libthr.so into process which was not linked against threading library). - Remove libthr interposers of the libc functions, including __error(). Instead, functions calls are indirected through the interposing table, similar to how pthread stubs in libc are already done. Libc by default points either to syscall trampolines or to existing libc implementations. On libthr load, libthr rewrites the pointers to the cancellable implementations already in libthr. The interposition table is separate from pthreads stubs indirection table to not pull pthreads stubs into static binaries. - Postpone the malloc(3) internal mutexes initialization until libthr is loaded. This avoids recursion between calloc(3) and static pthread_mutex_t initialization. - Reinstall signal handlers with wrapper on libthr load. The _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2) when libthr is statically referenced from the main binary. In the process, fix openat(2), swapcontext(2) and setcontext(2) interposing. The libc symbols were exported at different versions than libthr interposers. Export both libc and libthr versions from libc now, with default set to the higher version from libthr. Remove unused and disconnected swapcontext(3) userspace implementation from libc/gen. No objections from: deischen Tested by: pho, antoine (exp-run) (previous versions) Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-01-03 18:38:46 +00:00
#include <sys/link_elf.h>
Add a knob LIBPTHREAD_BIGSTACK_MAIN, which instructs libthr to leave the whole RLIMIT_STACK-sized region of the kernel-allocated stack as the stack of main thread. By default, the main thread stack is clamped at 2MB (4MB on 64bit ABIs) and the rest is used for other threads stack allocation. Since there is no programmatic way to adjust the size of the main thread stack, pthread_attr_setstacksize() is too late, the knob allows user to manage the main stack size both for single-threaded and multi-threaded processes with the rlimit. Reported by: "Ivan A. Kosarev" <ivan@ivan-labs.com> Tested by: dim Sponsored by: The FreeBSD Foundation MFC after: 3 days
2014-08-13 05:53:41 +00:00
#include <sys/resource.h>
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include <sys/sysctl.h>
#include <sys/ttycom.h>
#include <sys/mman.h>
Use kernel facilities to support real-time scheduling.
2006-07-12 06:13:18 +00:00
#include <sys/rtprio.h>
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include <errno.h>
#include <fcntl.h>
#include <paths.h>
#include <pthread.h>
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
#include <pthread_np.h>
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include <signal.h>
#include <stdlib.h>
#include <string.h>
Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into static binary.
2006-01-10 04:53:03 +00:00
#include <time.h>
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include <unistd.h>
#include "un-namespace.h"
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
#include "libc_private.h"
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
#include "thr_private.h"
WARNS level 4 cleanup.
2006-04-04 02:57:49 +00:00
char *_usrstack;
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
struct pthread *_thr_initial;
int _libthr_debug;
int _thread_event_mask;
struct pthread *_thread_last_event;
pthreadlist _thread_list = TAILQ_HEAD_INITIALIZER(_thread_list);
pthreadlist _thread_gc_list = TAILQ_HEAD_INITIALIZER(_thread_gc_list);
int _thread_active_threads = 1;
atfork_head _thr_atfork_list = TAILQ_HEAD_INITIALIZER(_thr_atfork_list);
Change atfork lock from mutex to rwlock, also make mutexes used by malloc() module private type, when private type mutex is locked/unlocked, thread critical region is entered or leaved. These changes makes fork() async-signal safe which required by POSIX. Note that user's atfork handler still needs to be async-signal safe, but it is not problem of libthr, it is user's responsiblity.
2010-09-01 03:11:21 +00:00
struct urwlock _thr_atfork_lock = DEFAULT_URWLOCK;
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
- Use same priority range returned by kernel's sched_get_priority_min() and sched_get_priority_max() syscalls. - Remove unused fields from structure pthread_attr.
2006-04-27 08:18:23 +00:00
struct pthread_prio _thr_priorities[3] = {
Use kernel facilities to support real-time scheduling.
2006-07-12 06:13:18 +00:00
{RTP_PRIO_MIN, RTP_PRIO_MAX, 0}, /* FIFO */
{0, 0, 63}, /* OTHER */
{RTP_PRIO_MIN, RTP_PRIO_MAX, 0} /* RR */
- Use same priority range returned by kernel's sched_get_priority_min() and sched_get_priority_max() syscalls. - Remove unused fields from structure pthread_attr.
2006-04-27 08:18:23 +00:00
};
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
struct pthread_attr _pthread_attr_default = {
Remove priority mutex code because it does not work correctly, to make it work, turnstile like mechanism to support priority propagating and other realtime scheduling options in kernel should be available to userland mutex, for the moment, I just want to make libthr be simple and efficient thread library. Discussed with: deischen, julian
2006-03-27 23:50:21 +00:00
.sched_policy = SCHED_OTHER,
Use cpuset defined in pthread_attr for newly created thread, for now, we set scheduling parameters and cpu binding fully in userland, and because default scheduling policy is SCHED_RR (time-sharing), we set default sched_inherit to PTHREAD_SCHED_INHERIT, this saves a system call.
2008-03-05 07:01:20 +00:00
.sched_inherit = PTHREAD_INHERIT_SCHED,
- Use same priority range returned by kernel's sched_get_priority_min() and sched_get_priority_max() syscalls. - Remove unused fields from structure pthread_attr.
2006-04-27 08:18:23 +00:00
.prio = 0,
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
.suspend = THR_CREATE_RUNNING,
Set default contention scope to system.
2006-03-20 03:14:14 +00:00
.flags = PTHREAD_SCOPE_SYSTEM,
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
.stackaddr_attr = NULL,
.stacksize_attr = THR_STACK_DEFAULT,
implement pthread_attr_getaffinity_np and pthread_attr_setaffinity_np.
2008-03-04 03:03:24 +00:00
.guardsize_attr = 0,
.cpusetsize = 0,
.cpuset = NULL
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
};
struct pthread_mutex_attr _pthread_mutexattr_default = {
.m_type = PTHREAD_MUTEX_DEFAULT,
.m_protocol = PTHREAD_PRIO_NONE,
Implement process-shared locks support for libthr.so.3, without breaking the ABI. Special value is stored in the lock pointer to indicate shared lock, and offline page in the shared memory is allocated to store the actual lock. Reviewed by: vangyzen (previous version) Discussed with: deischen, emaste, jhb, rwatson, Martin Simmons <martin@lispworks.com> Tested by: pho Sponsored by: The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
.m_ceiling = 0,
.m_pshared = PTHREAD_PROCESS_PRIVATE,
Add implementation of robust mutexes, hopefully close enough to the intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013. A robust mutex is guaranteed to be cleared by the system upon either thread or process owner termination while the mutex is held. The next mutex locker is then notified about inconsistent mutex state and can execute (or abandon) corrective actions. The patch mostly consists of small changes here and there, adding neccessary checks for the inconsistent and abandoned conditions into existing paths. Additionally, the thread exit handler was extended to iterate over the userspace-maintained list of owned robust mutexes, unlocking and marking as terminated each of them. The list of owned robust mutexes cannot be maintained atomically synchronous with the mutex lock state (it is possible in kernel, but is too expensive). Instead, for the duration of lock or unlock operation, the current mutex is remembered in a special slot that is also checked by the kernel at thread termination. Kernel must be aware about the per-thread location of the heads of robust mutex lists and the current active mutex slot. When a thread touches a robust mutex for the first time, a new umtx op syscall is issued which informs about location of lists heads. The umtx sleep queues for PP and PI mutexes are split between non-robust and robust. Somewhat unrelated changes in the patch: 1. Style. 2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared pi mutexes. 3. Removal of the userspace struct pthread_mutex m_owner field. 4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls the lifetime of the shared mutex associated with a vnode' page. Reviewed by: jilles (previous version, supposedly the objection was fixed) Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects) Tested by: pho Sponsored by: The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
.m_robust = PTHREAD_MUTEX_STALLED,
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
};
In current code, statically initialized and destroyed object have same null value, the code can not distinguish between them, to fix the problem, now a destroyed object is assigned to a non-null value, and it will be rejected by some pthread functions. PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP is changed to number 1, so that adaptive mutex can be statically initialized correctly.
2010-09-28 04:57:56 +00:00
struct pthread_mutex_attr _pthread_mutexattr_adaptive_default = {
.m_type = PTHREAD_MUTEX_ADAPTIVE_NP,
.m_protocol = PTHREAD_PRIO_NONE,
Implement process-shared locks support for libthr.so.3, without breaking the ABI. Special value is stored in the lock pointer to indicate shared lock, and offline page in the shared memory is allocated to store the actual lock. Reviewed by: vangyzen (previous version) Discussed with: deischen, emaste, jhb, rwatson, Martin Simmons <martin@lispworks.com> Tested by: pho Sponsored by: The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
.m_ceiling = 0,
.m_pshared = PTHREAD_PROCESS_PRIVATE,
Add implementation of robust mutexes, hopefully close enough to the intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013. A robust mutex is guaranteed to be cleared by the system upon either thread or process owner termination while the mutex is held. The next mutex locker is then notified about inconsistent mutex state and can execute (or abandon) corrective actions. The patch mostly consists of small changes here and there, adding neccessary checks for the inconsistent and abandoned conditions into existing paths. Additionally, the thread exit handler was extended to iterate over the userspace-maintained list of owned robust mutexes, unlocking and marking as terminated each of them. The list of owned robust mutexes cannot be maintained atomically synchronous with the mutex lock state (it is possible in kernel, but is too expensive). Instead, for the duration of lock or unlock operation, the current mutex is remembered in a special slot that is also checked by the kernel at thread termination. Kernel must be aware about the per-thread location of the heads of robust mutex lists and the current active mutex slot. When a thread touches a robust mutex for the first time, a new umtx op syscall is issued which informs about location of lists heads. The umtx sleep queues for PP and PI mutexes are split between non-robust and robust. Somewhat unrelated changes in the patch: 1. Style. 2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared pi mutexes. 3. Removal of the userspace struct pthread_mutex m_owner field. 4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls the lifetime of the shared mutex associated with a vnode' page. Reviewed by: jilles (previous version, supposedly the objection was fixed) Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects) Tested by: pho Sponsored by: The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
.m_robust = PTHREAD_MUTEX_STALLED,
In current code, statically initialized and destroyed object have same null value, the code can not distinguish between them, to fix the problem, now a destroyed object is assigned to a non-null value, and it will be rejected by some pthread functions. PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP is changed to number 1, so that adaptive mutex can be statically initialized correctly.
2010-09-28 04:57:56 +00:00
};
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
/* Default condition variable attributes: */
struct pthread_cond_attr _pthread_condattr_default = {
.c_pshared = PTHREAD_PROCESS_PRIVATE,
.c_clockid = CLOCK_REALTIME
};
- Remove variable _thr_scope_system, all threads are system scope. - Rename _thr_smp_cpus to boolean variable _thr_is_smp. - Define CPU_SPINWAIT macro for each arch, only X86 supports it.
2006-12-15 11:52:01 +00:00
int _thr_is_smp = 0;
WARNS level 4 cleanup.
2006-04-04 02:57:49 +00:00
size_t _thr_guard_default;
size_t _thr_stack_default = THR_STACK_DEFAULT;
size_t _thr_stack_initial = THR_STACK_INITIAL;
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
int _thr_page_size;
Add my recent work of adaptive spin mutex code. Use two environments variable to tune pthread mutex performance: 1. LIBPTHREAD_SPINLOOPS If a pthread mutex is being locked by another thread, this environment variable sets total number of spin loops before the current thread sleeps in kernel, this saves a syscall overhead if the mutex will be unlocked very soon (well written application code). 2. LIBPTHREAD_YIELDLOOPS If a pthread mutex is being locked by other threads, this environment variable sets total number of sched_yield() loops before the currrent thread sleeps in kernel. if a pthread mutex is locked, the current thread gives up cpu, but will not sleep in kernel, this means, current thread does not set contention bit in mutex, but let lock owner to run again if the owner is on kernel's run queue, and when lock owner unlocks the mutex, it does not need to enter kernel and do lots of work to resume mutex waiters, in some cases, this saves lots of syscall overheads for mutex owner. In my practice, sometimes LIBPTHREAD_YIELDLOOPS can massively improve performance than LIBPTHREAD_SPINLOOPS, this depends on application. These two environments are global to all pthread mutex, there is no interface to set them for each pthread mutex, the default values are zero, this means spinning is turned off by default.
2007-10-30 05:57:37 +00:00
int _thr_spinloops;
int _thr_yieldloops;
MFp4: Enqueue thread in LIFO, this can cause starvation, but it gives better performance. Use _thr_queuefifo to control the frequency of FIFO vs LIFO, you can use environment string LIBPTHREAD_QUEUE_FIFO to configure the variable.
2012-05-03 09:17:31 +00:00
int _thr_queuefifo = 4;
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
int _gc_count;
Replace internal usage of struct umtx with umutex which can supports real-time if we want, no functionality is changed.
2006-09-06 04:04:10 +00:00
struct umutex _mutex_static_lock = DEFAULT_UMUTEX;
struct umutex _cond_static_lock = DEFAULT_UMUTEX;
struct umutex _rwlock_static_lock = DEFAULT_UMUTEX;
struct umutex _keytable_lock = DEFAULT_UMUTEX;
Convert thread list lock from mutex to rwlock.
2010-09-13 07:03:01 +00:00
struct urwlock _thr_list_lock = DEFAULT_URWLOCK;
Replace internal usage of struct umtx with umutex which can supports real-time if we want, no functionality is changed.
2006-09-06 04:04:10 +00:00
struct umutex _thr_event_lock = DEFAULT_UMUTEX;
In suspend_common(), don't wait for a thread which is in creation, because pthread_suspend_all_np() may have already suspended its parent thread. Add locking code in pthread_suspend_all_np() to only allow one thread to suspend other threads, this eliminates a deadlock where two or more threads try to suspend each others.
2012-08-27 03:09:39 +00:00
struct umutex _suspend_all_lock = DEFAULT_UMUTEX;
struct pthread *_single_thread;
int _suspend_all_cycle;
int _suspend_all_waiters;
1. Retire macro SCLASS, instead simply use language keyword and put variables in thr_init.c. 2. Hide all global symbols which won't be exported.
2005-12-21 03:14:06 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
int __pthread_cond_wait(pthread_cond_t *, pthread_mutex_t *);
int __pthread_mutex_lock(pthread_mutex_t *);
int __pthread_mutex_trylock(pthread_mutex_t *);
void _thread_init_hack(void) __attribute__ ((constructor));
static void init_private(void);
static void init_main_thread(struct pthread *thread);
Make C applications statically compiled with libthr work. Previously, an application compiled -static with libthr would dump core in malloc(3) because the stub thread initialization routine in libc would be used instead of the libthr supplied one.
2003-06-04 08:23:05 +00:00
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
/*
* All weak references used within libc should be in this table.
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
* This is so that static libraries will work.
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
*/
1. Don't override underscore version of aio_suspend(), system(), wait(), waitpid() and usleep(), they are internal versions and should not be cancellation points. 2. Make wait3() as a cancellation point. 3. Move raise() and pause() into file thr_sig.c. 4. Add functions _sigsuspend, _sigwait, _sigtimedwait and _sigwaitinfo, remove SIGCANCEL bit in wait-set for those functions, the signal is used internally to implement thread cancellation.
2006-07-25 12:50:05 +00:00
STATIC_LIB_REQUIRE(_fork);
Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into static binary.
2006-01-10 04:53:03 +00:00
STATIC_LIB_REQUIRE(_pthread_getspecific);
STATIC_LIB_REQUIRE(_pthread_key_create);
STATIC_LIB_REQUIRE(_pthread_key_delete);
STATIC_LIB_REQUIRE(_pthread_mutex_destroy);
STATIC_LIB_REQUIRE(_pthread_mutex_init);
STATIC_LIB_REQUIRE(_pthread_mutex_lock);
STATIC_LIB_REQUIRE(_pthread_mutex_trylock);
STATIC_LIB_REQUIRE(_pthread_mutex_unlock);
STATIC_LIB_REQUIRE(_pthread_mutexattr_init);
STATIC_LIB_REQUIRE(_pthread_mutexattr_destroy);
STATIC_LIB_REQUIRE(_pthread_mutexattr_settype);
STATIC_LIB_REQUIRE(_pthread_once);
STATIC_LIB_REQUIRE(_pthread_setspecific);
1. Don't override underscore version of aio_suspend(), system(), wait(), waitpid() and usleep(), they are internal versions and should not be cancellation points. 2. Make wait3() as a cancellation point. 3. Move raise() and pause() into file thr_sig.c. 4. Add functions _sigsuspend, _sigwait, _sigtimedwait and _sigwaitinfo, remove SIGCANCEL bit in wait-set for those functions, the signal is used internally to implement thread cancellation.
2006-07-25 12:50:05 +00:00
STATIC_LIB_REQUIRE(_raise);
STATIC_LIB_REQUIRE(_sem_destroy);
STATIC_LIB_REQUIRE(_sem_getvalue);
STATIC_LIB_REQUIRE(_sem_init);
STATIC_LIB_REQUIRE(_sem_post);
STATIC_LIB_REQUIRE(_sem_timedwait);
STATIC_LIB_REQUIRE(_sem_trywait);
STATIC_LIB_REQUIRE(_sem_wait);
Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into static binary.
2006-01-10 04:53:03 +00:00
STATIC_LIB_REQUIRE(_sigaction);
STATIC_LIB_REQUIRE(_sigprocmask);
STATIC_LIB_REQUIRE(_sigsuspend);
1. Don't override underscore version of aio_suspend(), system(), wait(), waitpid() and usleep(), they are internal versions and should not be cancellation points. 2. Make wait3() as a cancellation point. 3. Move raise() and pause() into file thr_sig.c. 4. Add functions _sigsuspend, _sigwait, _sigtimedwait and _sigwaitinfo, remove SIGCANCEL bit in wait-set for those functions, the signal is used internally to implement thread cancellation.
2006-07-25 12:50:05 +00:00
STATIC_LIB_REQUIRE(_sigtimedwait);
STATIC_LIB_REQUIRE(_sigwait);
STATIC_LIB_REQUIRE(_sigwaitinfo);
STATIC_LIB_REQUIRE(_spinlock);
STATIC_LIB_REQUIRE(_spinunlock);
Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into static binary.
2006-01-10 04:53:03 +00:00
STATIC_LIB_REQUIRE(_thread_init_hack);
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
/*
* These are needed when linking statically. All references within
* libgcc (and in the future libc) to these routines are weak, but
* if they are not (strongly) referenced by the application or other
* libraries, then the actual functions will not be loaded.
*/
Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into static binary.
2006-01-10 04:53:03 +00:00
STATIC_LIB_REQUIRE(_pthread_once);
STATIC_LIB_REQUIRE(_pthread_key_create);
STATIC_LIB_REQUIRE(_pthread_key_delete);
STATIC_LIB_REQUIRE(_pthread_getspecific);
STATIC_LIB_REQUIRE(_pthread_setspecific);
STATIC_LIB_REQUIRE(_pthread_mutex_init);
STATIC_LIB_REQUIRE(_pthread_mutex_destroy);
STATIC_LIB_REQUIRE(_pthread_mutex_lock);
STATIC_LIB_REQUIRE(_pthread_mutex_trylock);
STATIC_LIB_REQUIRE(_pthread_mutex_unlock);
STATIC_LIB_REQUIRE(_pthread_create);
/* Pull in all symbols required by libthread_db */
STATIC_LIB_REQUIRE(_thread_state_running);
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
#define DUAL_ENTRY(entry) \
(pthread_func_t)entry, (pthread_func_t)entry
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
static pthread_func_t jmp_table[][2] = {
Avoid conflicts with libc symbols in libthr jump table. In some corner cases of static linking and unexpected libraries order on the linker command line, libc symbol might preempt the same libthr symbol, in which case libthr jump table points back to libc causing either infinite recursion or loop. Handle all of such symbols by using private libthr names for them, ensuring that the right pointers are installed into the table. In collaboration with: arichardson PR: 239475 Tested by: pho MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D21088
2019-07-31 19:27:20 +00:00
[PJT_ATFORK] = {DUAL_ENTRY(_thr_atfork)},
[PJT_ATTR_DESTROY] = {DUAL_ENTRY(_thr_attr_destroy)},
[PJT_ATTR_GETDETACHSTATE] = {DUAL_ENTRY(_thr_attr_getdetachstate)},
[PJT_ATTR_GETGUARDSIZE] = {DUAL_ENTRY(_thr_attr_getguardsize)},
[PJT_ATTR_GETINHERITSCHED] = {DUAL_ENTRY(_thr_attr_getinheritsched)},
[PJT_ATTR_GETSCHEDPARAM] = {DUAL_ENTRY(_thr_attr_getschedparam)},
[PJT_ATTR_GETSCHEDPOLICY] = {DUAL_ENTRY(_thr_attr_getschedpolicy)},
[PJT_ATTR_GETSCOPE] = {DUAL_ENTRY(_thr_attr_getscope)},
[PJT_ATTR_GETSTACKADDR] = {DUAL_ENTRY(_thr_attr_getstackaddr)},
[PJT_ATTR_GETSTACKSIZE] = {DUAL_ENTRY(_thr_attr_getstacksize)},
[PJT_ATTR_INIT] = {DUAL_ENTRY(_thr_attr_init)},
[PJT_ATTR_SETDETACHSTATE] = {DUAL_ENTRY(_thr_attr_setdetachstate)},
[PJT_ATTR_SETGUARDSIZE] = {DUAL_ENTRY(_thr_attr_setguardsize)},
[PJT_ATTR_SETINHERITSCHED] = {DUAL_ENTRY(_thr_attr_setinheritsched)},
[PJT_ATTR_SETSCHEDPARAM] = {DUAL_ENTRY(_thr_attr_setschedparam)},
[PJT_ATTR_SETSCHEDPOLICY] = {DUAL_ENTRY(_thr_attr_setschedpolicy)},
[PJT_ATTR_SETSCOPE] = {DUAL_ENTRY(_thr_attr_setscope)},
[PJT_ATTR_SETSTACKADDR] = {DUAL_ENTRY(_thr_attr_setstackaddr)},
[PJT_ATTR_SETSTACKSIZE] = {DUAL_ENTRY(_thr_attr_setstacksize)},
[PJT_CANCEL] = {DUAL_ENTRY(_thr_cancel)},
[PJT_CLEANUP_POP] = {DUAL_ENTRY(_thr_cleanup_pop)},
[PJT_CLEANUP_PUSH] = {DUAL_ENTRY(_thr_cleanup_push)},
[PJT_COND_BROADCAST] = {DUAL_ENTRY(_thr_cond_broadcast)},
[PJT_COND_DESTROY] = {DUAL_ENTRY(_thr_cond_destroy)},
[PJT_COND_INIT] = {DUAL_ENTRY(_thr_cond_init)},
[PJT_COND_SIGNAL] = {DUAL_ENTRY(_thr_cond_signal)},
[PJT_COND_TIMEDWAIT] = {DUAL_ENTRY(_thr_cond_timedwait)},
[PJT_COND_WAIT] = {(pthread_func_t)__thr_cond_wait,
(pthread_func_t)_thr_cond_wait},
[PJT_DETACH] = {DUAL_ENTRY(_thr_detach)},
[PJT_EQUAL] = {DUAL_ENTRY(_thr_equal)},
[PJT_EXIT] = {DUAL_ENTRY(_Tthr_exit)},
[PJT_GETSPECIFIC] = {DUAL_ENTRY(_thr_getspecific)},
[PJT_JOIN] = {DUAL_ENTRY(_thr_join)},
[PJT_KEY_CREATE] = {DUAL_ENTRY(_thr_key_create)},
[PJT_KEY_DELETE] = {DUAL_ENTRY(_thr_key_delete)},
[PJT_KILL] = {DUAL_ENTRY(_Tthr_kill)},
[PJT_MAIN_NP] = {DUAL_ENTRY(_thr_main_np)},
[PJT_MUTEXATTR_DESTROY] = {DUAL_ENTRY(_thr_mutexattr_destroy)},
[PJT_MUTEXATTR_INIT] = {DUAL_ENTRY(_thr_mutexattr_init)},
[PJT_MUTEXATTR_SETTYPE] = {DUAL_ENTRY(_thr_mutexattr_settype)},
[PJT_MUTEX_DESTROY] = {DUAL_ENTRY(_thr_mutex_destroy)},
[PJT_MUTEX_INIT] = {DUAL_ENTRY(__Tthr_mutex_init)},
[PJT_MUTEX_LOCK] = {DUAL_ENTRY(__Tthr_mutex_lock)},
[PJT_MUTEX_TRYLOCK] = {DUAL_ENTRY(__Tthr_mutex_trylock)},
[PJT_MUTEX_UNLOCK] = {DUAL_ENTRY(_thr_mutex_unlock)},
[PJT_ONCE] = {DUAL_ENTRY(_thr_once)},
[PJT_RWLOCK_DESTROY] = {DUAL_ENTRY(_thr_rwlock_destroy)},
[PJT_RWLOCK_INIT] = {DUAL_ENTRY(_thr_rwlock_init)},
[PJT_RWLOCK_RDLOCK] = {DUAL_ENTRY(_Tthr_rwlock_rdlock)},
[PJT_RWLOCK_TRYRDLOCK] = {DUAL_ENTRY(_Tthr_rwlock_tryrdlock)},
[PJT_RWLOCK_TRYWRLOCK] = {DUAL_ENTRY(_Tthr_rwlock_trywrlock)},
[PJT_RWLOCK_UNLOCK] = {DUAL_ENTRY(_Tthr_rwlock_unlock)},
[PJT_RWLOCK_WRLOCK] = {DUAL_ENTRY(_Tthr_rwlock_wrlock)},
[PJT_SELF] = {DUAL_ENTRY(_Tthr_self)},
[PJT_SETCANCELSTATE] = {DUAL_ENTRY(_thr_setcancelstate)},
[PJT_SETCANCELTYPE] = {DUAL_ENTRY(_thr_setcanceltype)},
[PJT_SETSPECIFIC] = {DUAL_ENTRY(_thr_setspecific)},
[PJT_SIGMASK] = {DUAL_ENTRY(_thr_sigmask)},
[PJT_TESTCANCEL] = {DUAL_ENTRY(_Tthr_testcancel)},
[PJT_CLEANUP_POP_IMP] = {DUAL_ENTRY(__thr_cleanup_pop_imp)},
[PJT_CLEANUP_PUSH_IMP] = {DUAL_ENTRY(__thr_cleanup_push_imp)},
Fix _pthread_cancel_enter() and _pthread_cancel_leave() jmptable entries. PR: 240022 Reported by: Andrew Gierth <andrew@tao11.riddles.org.uk> Sponsored by: The FreeBSD Foundation MFC after: 3 days
2019-08-21 19:53:50 +00:00
[PJT_CANCEL_ENTER] = {DUAL_ENTRY(_Tthr_cancel_enter)},
[PJT_CANCEL_LEAVE] = {DUAL_ENTRY(_Tthr_cancel_leave)},
Avoid conflicts with libc symbols in libthr jump table. In some corner cases of static linking and unexpected libraries order on the linker command line, libc symbol might preempt the same libthr symbol, in which case libthr jump table points back to libc causing either infinite recursion or loop. Handle all of such symbols by using private libthr names for them, ensuring that the right pointers are installed into the table. In collaboration with: arichardson PR: 239475 Tested by: pho MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D21088
2019-07-31 19:27:20 +00:00
[PJT_MUTEX_CONSISTENT] = {DUAL_ENTRY(_Tthr_mutex_consistent)},
[PJT_MUTEXATTR_GETROBUST] = {DUAL_ENTRY(_thr_mutexattr_getrobust)},
[PJT_MUTEXATTR_SETROBUST] = {DUAL_ENTRY(_thr_mutexattr_setrobust)},
[PJT_GETTHREADID_NP] = {DUAL_ENTRY(_thr_getthreadid_np)},
libc: Add pthread_attr_get_np(3) stub, reporting ESRCH. This seems to be required by recent clang asan. I do not see other way than put the symbol under FBSD_1.0 version. PR: 251112 Reported by: Andrew Stitcher <astitcher@apache.org> Reviewed by: emaste Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D27389
2020-11-28 12:19:20 +00:00
[PJT_ATTR_GET_NP] = {DUAL_ENTRY(_thr_attr_get_np)},
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
};
static int init_once = 0;
Preparations to make libthr work in multi-threaded fork()ing applications. o Remove some code duplication between _thread_init(), which is run once to initialize libthr and the intitial thread, and pthread_create(), which initializes newly created threads, into a new function called from both places: init_td_common() o Move initialization of certain parts of libthr into a separate function. These include: - Active threads list and it's lock - Dead threads list and it's lock & condition variable - Naming and insertion of the initial thread into the active threads list.
2003-12-26 08:16:17 +00:00
/*
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
* For the shared version of the threads library, the above is sufficient.
* But for the archive version of the library, we need a little bit more.
* Namely, we must arrange for this particular module to be pulled in from
* the archive library at link time. To accomplish that, we define and
* initialize a variable, "_thread_autoinit_dummy_decl". This variable is
* referenced (as an extern) from libc/stdlib/exit.c. This will always
* create a need for this module, ensuring that it is present in the
* executable.
Preparations to make libthr work in multi-threaded fork()ing applications. o Remove some code duplication between _thread_init(), which is run once to initialize libthr and the intitial thread, and pthread_create(), which initializes newly created threads, into a new function called from both places: init_td_common() o Move initialization of certain parts of libthr into a separate function. These include: - Active threads list and it's lock - Dead threads list and it's lock & condition variable - Naming and insertion of the initial thread into the active threads list.
2003-12-26 08:16:17 +00:00
*/
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
extern int _thread_autoinit_dummy_decl;
int _thread_autoinit_dummy_decl = 0;
Preparations to make libthr work in multi-threaded fork()ing applications. o Remove some code duplication between _thread_init(), which is run once to initialize libthr and the intitial thread, and pthread_create(), which initializes newly created threads, into a new function called from both places: init_td_common() o Move initialization of certain parts of libthr into a separate function. These include: - Active threads list and it's lock - Dead threads list and it's lock & condition variable - Naming and insertion of the initial thread into the active threads list.
2003-12-26 08:16:17 +00:00
void
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
_thread_init_hack(void)
Preparations to make libthr work in multi-threaded fork()ing applications. o Remove some code duplication between _thread_init(), which is run once to initialize libthr and the intitial thread, and pthread_create(), which initializes newly created threads, into a new function called from both places: init_td_common() o Move initialization of certain parts of libthr into a separate function. These include: - Active threads list and it's lock - Dead threads list and it's lock & condition variable - Naming and insertion of the initial thread into the active threads list.
2003-12-26 08:16:17 +00:00
{
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
_libpthread_init(NULL);
Preparations to make libthr work in multi-threaded fork()ing applications. o Remove some code duplication between _thread_init(), which is run once to initialize libthr and the intitial thread, and pthread_create(), which initializes newly created threads, into a new function called from both places: init_td_common() o Move initialization of certain parts of libthr into a separate function. These include: - Active threads list and it's lock - Dead threads list and it's lock & condition variable - Naming and insertion of the initial thread into the active threads list.
2003-12-26 08:16:17 +00:00
}
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
/*
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
* Threaded process initialization.
*
* This is only called under two conditions:
*
* 1) Some thread routines have detected that the library hasn't yet
* been initialized (_thr_initial == NULL && curthread == NULL), or
*
* 2) An explicit call to reinitialize after a fork (indicated
* by curthread != NULL)
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
*/
void
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
_libpthread_init(struct pthread *curthread)
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
{
From libthr, remove special and strange code to set up session and control terminal, activated when running with pid 1. It is application duty to handle this, and unsuspecting init replacements which are linked with libthr would be broken by this. The pre-resolving of getpid() is restored, just in case. Reviewed by: jilles Sponsored by: The FreeBSD Foundation MFC after: 2 weeks
2016-03-21 06:46:16 +00:00
int first, dlopened;
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
/* Check if this function has already been called: */
Add implementation of robust mutexes, hopefully close enough to the intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013. A robust mutex is guaranteed to be cleared by the system upon either thread or process owner termination while the mutex is held. The next mutex locker is then notified about inconsistent mutex state and can execute (or abandon) corrective actions. The patch mostly consists of small changes here and there, adding neccessary checks for the inconsistent and abandoned conditions into existing paths. Additionally, the thread exit handler was extended to iterate over the userspace-maintained list of owned robust mutexes, unlocking and marking as terminated each of them. The list of owned robust mutexes cannot be maintained atomically synchronous with the mutex lock state (it is possible in kernel, but is too expensive). Instead, for the duration of lock or unlock operation, the current mutex is remembered in a special slot that is also checked by the kernel at thread termination. Kernel must be aware about the per-thread location of the heads of robust mutex lists and the current active mutex slot. When a thread touches a robust mutex for the first time, a new umtx op syscall is issued which informs about location of lists heads. The umtx sleep queues for PP and PI mutexes are split between non-robust and robust. Somewhat unrelated changes in the patch: 1. Style. 2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared pi mutexes. 3. Removal of the userspace struct pthread_mutex m_owner field. 4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls the lifetime of the shared mutex associated with a vnode' page. Reviewed by: jilles (previous version, supposedly the objection was fixed) Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects) Tested by: pho Sponsored by: The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
if (_thr_initial != NULL && curthread == NULL)
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/* Only initialize the threaded application once. */
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
return;
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/*
* Check the size of the jump table to make sure it is preset
* with the correct number of entries.
*/
Add implementation of robust mutexes, hopefully close enough to the intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013. A robust mutex is guaranteed to be cleared by the system upon either thread or process owner termination while the mutex is held. The next mutex locker is then notified about inconsistent mutex state and can execute (or abandon) corrective actions. The patch mostly consists of small changes here and there, adding neccessary checks for the inconsistent and abandoned conditions into existing paths. Additionally, the thread exit handler was extended to iterate over the userspace-maintained list of owned robust mutexes, unlocking and marking as terminated each of them. The list of owned robust mutexes cannot be maintained atomically synchronous with the mutex lock state (it is possible in kernel, but is too expensive). Instead, for the duration of lock or unlock operation, the current mutex is remembered in a special slot that is also checked by the kernel at thread termination. Kernel must be aware about the per-thread location of the heads of robust mutex lists and the current active mutex slot. When a thread touches a robust mutex for the first time, a new umtx op syscall is issued which informs about location of lists heads. The umtx sleep queues for PP and PI mutexes are split between non-robust and robust. Somewhat unrelated changes in the patch: 1. Style. 2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared pi mutexes. 3. Removal of the userspace struct pthread_mutex m_owner field. 4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls the lifetime of the shared mutex associated with a vnode' page. Reviewed by: jilles (previous version, supposedly the objection was fixed) Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects) Tested by: pho Sponsored by: The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
if (sizeof(jmp_table) != sizeof(pthread_func_t) * PJT_MAX * 2)
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
PANIC("Thread jump table not properly initialized");
memcpy(__thr_jtable, jmp_table, sizeof(jmp_table));
Fix known issues which blow up the process after dlopen("libthr.so") (or loading a dso linked to libthr.so into process which was not linked against threading library). - Remove libthr interposers of the libc functions, including __error(). Instead, functions calls are indirected through the interposing table, similar to how pthread stubs in libc are already done. Libc by default points either to syscall trampolines or to existing libc implementations. On libthr load, libthr rewrites the pointers to the cancellable implementations already in libthr. The interposition table is separate from pthreads stubs indirection table to not pull pthreads stubs into static binaries. - Postpone the malloc(3) internal mutexes initialization until libthr is loaded. This avoids recursion between calloc(3) and static pthread_mutex_t initialization. - Reinstall signal handlers with wrapper on libthr load. The _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2) when libthr is statically referenced from the main binary. In the process, fix openat(2), swapcontext(2) and setcontext(2) interposing. The libc symbols were exported at different versions than libthr interposers. Export both libc and libthr versions from libc now, with default set to the higher version from libthr. Remove unused and disconnected swapcontext(3) userspace implementation from libc/gen. No objections from: deischen Tested by: pho, antoine (exp-run) (previous versions) Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-01-03 18:38:46 +00:00
__thr_interpose_libc();
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/* Initialize pthread private data. */
init_private();
/* Set the initial thread. */
if (curthread == NULL) {
first = 1;
/* Create and initialize the initial thread. */
curthread = _thr_alloc(NULL);
if (curthread == NULL)
PANIC("Can't allocate initial thread");
init_main_thread(curthread);
Fix known issues which blow up the process after dlopen("libthr.so") (or loading a dso linked to libthr.so into process which was not linked against threading library). - Remove libthr interposers of the libc functions, including __error(). Instead, functions calls are indirected through the interposing table, similar to how pthread stubs in libc are already done. Libc by default points either to syscall trampolines or to existing libc implementations. On libthr load, libthr rewrites the pointers to the cancellable implementations already in libthr. The interposition table is separate from pthreads stubs indirection table to not pull pthreads stubs into static binaries. - Postpone the malloc(3) internal mutexes initialization until libthr is loaded. This avoids recursion between calloc(3) and static pthread_mutex_t initialization. - Reinstall signal handlers with wrapper on libthr load. The _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2) when libthr is statically referenced from the main binary. In the process, fix openat(2), swapcontext(2) and setcontext(2) interposing. The libc symbols were exported at different versions than libthr interposers. Export both libc and libthr versions from libc now, with default set to the higher version from libthr. Remove unused and disconnected swapcontext(3) userspace implementation from libc/gen. No objections from: deischen Tested by: pho, antoine (exp-run) (previous versions) Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-01-03 18:38:46 +00:00
} else {
first = 0;
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
}
Fix known issues which blow up the process after dlopen("libthr.so") (or loading a dso linked to libthr.so into process which was not linked against threading library). - Remove libthr interposers of the libc functions, including __error(). Instead, functions calls are indirected through the interposing table, similar to how pthread stubs in libc are already done. Libc by default points either to syscall trampolines or to existing libc implementations. On libthr load, libthr rewrites the pointers to the cancellable implementations already in libthr. The interposition table is separate from pthreads stubs indirection table to not pull pthreads stubs into static binaries. - Postpone the malloc(3) internal mutexes initialization until libthr is loaded. This avoids recursion between calloc(3) and static pthread_mutex_t initialization. - Reinstall signal handlers with wrapper on libthr load. The _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2) when libthr is statically referenced from the main binary. In the process, fix openat(2), swapcontext(2) and setcontext(2) interposing. The libc symbols were exported at different versions than libthr interposers. Export both libc and libthr versions from libc now, with default set to the higher version from libthr. Remove unused and disconnected swapcontext(3) userspace implementation from libc/gen. No objections from: deischen Tested by: pho, antoine (exp-run) (previous versions) Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-01-03 18:38:46 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/*
* Add the thread to the thread list queue.
*/
THR_LIST_ADD(curthread);
_thread_active_threads = 1;
The thread id was being set *before* zeroing out the thread. Reverse the order. Approved by: markm/mentor, re/blanket libthr
2003-05-21 03:22:36 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/* Setup the thread specific data */
_tcb_set(curthread->tcb);
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
if (first) {
_thr_initial = curthread;
Fix known issues which blow up the process after dlopen("libthr.so") (or loading a dso linked to libthr.so into process which was not linked against threading library). - Remove libthr interposers of the libc functions, including __error(). Instead, functions calls are indirected through the interposing table, similar to how pthread stubs in libc are already done. Libc by default points either to syscall trampolines or to existing libc implementations. On libthr load, libthr rewrites the pointers to the cancellable implementations already in libthr. The interposition table is separate from pthreads stubs indirection table to not pull pthreads stubs into static binaries. - Postpone the malloc(3) internal mutexes initialization until libthr is loaded. This avoids recursion between calloc(3) and static pthread_mutex_t initialization. - Reinstall signal handlers with wrapper on libthr load. The _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2) when libthr is statically referenced from the main binary. In the process, fix openat(2), swapcontext(2) and setcontext(2) interposing. The libc symbols were exported at different versions than libthr interposers. Export both libc and libthr versions from libc now, with default set to the higher version from libthr. Remove unused and disconnected swapcontext(3) userspace implementation from libc/gen. No objections from: deischen Tested by: pho, antoine (exp-run) (previous versions) Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-01-03 18:38:46 +00:00
dlopened = _rtld_is_dlopened(&_thread_autoinit_dummy_decl) != 0;
_thr_signal_init(dlopened);
Conditionally report initial thread event.
2005-04-12 03:13:49 +00:00
if (_thread_event_mask & TD_CREATE)
_thr_report_creation(curthread, curthread);
libthr: Always use the threaded rtld lock implementation. The threaded rtld lock implementation is faster even in the single-threaded case because it postpones signal handlers via THR_CRITICAL_ENTER and THR_CRITICAL_LEAVE instead of calling sigprocmask(2). As a result, exception handling becomes faster in single-threaded applications linked with libthr. Reviewed by: kib
2013-01-18 23:08:40 +00:00
/*
* Always use our rtld lock implementation.
* It is faster because it postpones signal handlers
* instead of calling sigprocmask(2).
*/
_thr_rtld_init();
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
}
}
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/*
* This function and pthread_create() do a lot of the same things.
* It'd be nice to consolidate the common stuff in one place.
*/
static void
init_main_thread(struct pthread *thread)
{
Caching scheduling policy and priority in userland, a critical but baddly written application is frequently changing thread priority for SCHED_OTHER policy.
2006-07-13 22:45:19 +00:00
struct sched_param sched_param;
Implement process-shared locks support for libthr.so.3, without breaking the ABI. Special value is stored in the lock pointer to indicate shared lock, and offline page in the shared memory is allocated to store the actual lock. Reviewed by: vangyzen (previous version) Discussed with: deischen, emaste, jhb, rwatson, Martin Simmons <martin@lispworks.com> Tested by: pho Sponsored by: The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
int i;
Caching scheduling policy and priority in userland, a critical but baddly written application is frequently changing thread priority for SCHED_OTHER policy.
2006-07-13 22:45:19 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/* Setup the thread attributes. */
thr_self(&thread->tid);
thread->attr = _pthread_attr_default;
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
/*
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
* Set up the thread stack.
*
* Create a red zone below the main stack. All other stacks
* are constrained to a maximum size by the parameters
* passed to mmap(), but this stack is only limited by
* resource limits, so this stack needs an explicitly mapped
* red zone to protect the thread stack that is just beyond.
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
*/
WARNS level 4 cleanup.
2006-04-04 02:57:49 +00:00
if (mmap(_usrstack - _thr_stack_initial -
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
_thr_guard_default, _thr_guard_default, 0, MAP_ANON,
-1, 0) == MAP_FAILED)
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
PANIC("Cannot allocate red zone for initial thread");
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/*
* Mark the stack as an application supplied stack so that it
* isn't deallocated.
*
* XXX - I'm not sure it would hurt anything to deallocate
* the main thread stack because deallocation doesn't
* actually free() it; it just puts it in the free
* stack queue for later reuse.
*/
WARNS level 4 cleanup.
2006-04-04 02:57:49 +00:00
thread->attr.stackaddr_attr = _usrstack - _thr_stack_initial;
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
thread->attr.stacksize_attr = _thr_stack_initial;
thread->attr.guardsize_attr = _thr_guard_default;
thread->attr.flags |= THR_STACK_USER;
/*
* Write a magic value to the thread structure
* to help identify valid ones:
*/
thread->magic = THR_MAGIC;
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Eliminate atomic operations in thread cancellation functions, it should reduce overheads of cancellation points.
2006-11-24 09:57:38 +00:00
thread->cancel_enable = 1;
thread->cancel_async = 0;
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Implement process-shared locks support for libthr.so.3, without breaking the ABI. Special value is stored in the lock pointer to indicate shared lock, and offline page in the shared memory is allocated to store the actual lock. Reviewed by: vangyzen (previous version) Discussed with: deischen, emaste, jhb, rwatson, Martin Simmons <martin@lispworks.com> Tested by: pho Sponsored by: The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
/* Initialize the mutex queues */
for (i = 0; i < TMQ_NITEMS; i++)
TAILQ_INIT(&thread->mq[i]);
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
thread->state = PS_RUNNING;
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
use rtprio_thread system call to get or set thread priority.
2006-09-21 04:21:30 +00:00
_thr_getscheduler(thread->tid, &thread->attr.sched_policy,
&sched_param);
Caching scheduling policy and priority in userland, a critical but baddly written application is frequently changing thread priority for SCHED_OTHER policy.
2006-07-13 22:45:19 +00:00
thread->attr.prio = sched_param.sched_priority;
add code to support stack unwinding when thread exits. note that only defer-mode cancellation works, asynchrnous mode does not work because it lacks of libuwind's support. stack unwinding is not enabled unless LIBTHR_UNWIND_STACK is defined in Makefile.
2010-09-15 02:56:32 +00:00
#ifdef _PTHREAD_FORCED_UNWIND
thread->unwind_stackend = _usrstack;
#endif
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/* Others cleared to zero by thr_alloc() */
- Add libthr but don't hook it up to the regular build yet. This is an adaptation of libc_r for the thr system call interface. This is beta quality code.
2003-04-01 03:46:29 +00:00
}
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
static void
init_private(void)
{
Add a knob LIBPTHREAD_BIGSTACK_MAIN, which instructs libthr to leave the whole RLIMIT_STACK-sized region of the kernel-allocated stack as the stack of main thread. By default, the main thread stack is clamped at 2MB (4MB on 64bit ABIs) and the rest is used for other threads stack allocation. Since there is no programmatic way to adjust the size of the main thread stack, pthread_attr_setstacksize() is too late, the knob allows user to manage the main stack size both for single-threaded and multi-threaded processes with the rlimit. Reported by: "Ivan A. Kosarev" <ivan@ivan-labs.com> Tested by: dim Sponsored by: The FreeBSD Foundation MFC after: 3 days
2014-08-13 05:53:41 +00:00
struct rlimit rlim;
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
size_t len;
int mib[2];
Switch the defaults to not split the RLIMIT_STACK-sized initial thread stack into the stacks of the created threads. Add knob LIBPTHREAD_SPLITSTACK_MAIN to restore the older behaviour. Sponsored by: The FreeBSD Foundation MFC after: 3 weeks
2014-09-24 12:39:12 +00:00
char *env, *env_bigstack, *env_splitstack;
get LIBPTHREAD_ADAPTIVE_SPIN early, so it can be used for some global mutexes.
2006-12-20 05:05:44 +00:00
Replace internal usage of struct umtx with umutex which can supports real-time if we want, no functionality is changed.
2006-09-06 04:04:10 +00:00
_thr_umutex_init(&_mutex_static_lock);
_thr_umutex_init(&_cond_static_lock);
_thr_umutex_init(&_rwlock_static_lock);
_thr_umutex_init(&_keytable_lock);
Change atfork lock from mutex to rwlock, also make mutexes used by malloc() module private type, when private type mutex is locked/unlocked, thread critical region is entered or leaved. These changes makes fork() async-signal safe which required by POSIX. Note that user's atfork handler still needs to be async-signal safe, but it is not problem of libthr, it is user's responsiblity.
2010-09-01 03:11:21 +00:00
_thr_urwlock_init(&_thr_atfork_lock);
Replace internal usage of struct umtx with umutex which can supports real-time if we want, no functionality is changed.
2006-09-06 04:04:10 +00:00
_thr_umutex_init(&_thr_event_lock);
In suspend_common(), don't wait for a thread which is in creation, because pthread_suspend_all_np() may have already suspended its parent thread. Add locking code in pthread_suspend_all_np() to only allow one thread to suspend other threads, this eliminates a deadlock where two or more threads try to suspend each others.
2012-08-27 03:09:39 +00:00
_thr_umutex_init(&_suspend_all_lock);
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
_thr_spinlock_init();
_thr_list_init();
MFp4: - Add flags CVWAIT_ABSTIME and CVWAIT_CLOCKID for umtx kernel based condition variable, this should eliminate an extra system call to get current time. - Add sub-function UMTX_OP_NWAKE_PRIVATE to wake up N channels in single system call. Create userland sleep queue for condition variable, in most cases, thread will wait in the queue, the pthread_cond_signal will defer thread wakeup until the mutex is unlocked, it tries to avoid an extra system call and a extra context switch in time window of pthread_cond_signal and pthread_mutex_unlock. The changes are part of process-shared mutex project.
2010-12-22 05:01:52 +00:00
_thr_wake_addr_init();
_sleepq_init();
In suspend_common(), don't wait for a thread which is in creation, because pthread_suspend_all_np() may have already suspended its parent thread. Add locking code in pthread_suspend_all_np() to only allow one thread to suspend other threads, this eliminates a deadlock where two or more threads try to suspend each others.
2012-08-27 03:09:39 +00:00
_single_thread = NULL;
_suspend_all_waiters = 0;
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/*
* Avoid reinitializing some things if they don't need to be,
* e.g. after a fork().
*/
if (init_once == 0) {
Lock pshared_lock shared around fork, to ensure that the COW snapshot of the pshared hash in child is consistent and can be safely used. Reported and tested by: "Oleg V. Nauman" <oleg@opentransfer.com> Sponsored by: The FreeBSD Foundation
2016-03-21 06:52:35 +00:00
__thr_pshared_init();
Untangle jemalloc and mutexes initialization. The need to use libc malloc(3) from some places in libthr always caused issues. For instance, per-thread key allocation was switched to use plain mmap(2) to get storage, because some third party mallocs used keys for implementation of calloc(3). Even more important, libthr calls calloc(3) during initialization of pthread mutexes, and jemalloc uses pthread mutexes. Jemalloc provides some way to both postpone the initialization, and to make initialization to use specialized allocator, but this is very fragile and often breaks. See the referenced PR for another example. Add the small malloc implementation used by rtld, to libthr. Use it in thr_spec.c and for mutexes initialization. This avoids the issues with mutual dependencies between malloc and libthr in principle. The drawback is that some more allocations are not interceptable for alternate malloc implementations. There should be not too much memory use from this allocator, and the alternative, direct use of mmap(2) is obviously worse. PR: 235211 MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D18988
2019-01-29 22:46:44 +00:00
__thr_malloc_init();
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
/* Find the stack top */
mib[0] = CTL_KERN;
mib[1] = KERN_USRSTACK;
len = sizeof (_usrstack);
if (sysctl(mib, 2, &_usrstack, &len, NULL, 0) == -1)
PANIC("Cannot get kern.usrstack from sysctl");
Switch the defaults to not split the RLIMIT_STACK-sized initial thread stack into the stacks of the created threads. Add knob LIBPTHREAD_SPLITSTACK_MAIN to restore the older behaviour. Sponsored by: The FreeBSD Foundation MFC after: 3 weeks
2014-09-24 12:39:12 +00:00
env_bigstack = getenv("LIBPTHREAD_BIGSTACK_MAIN");
env_splitstack = getenv("LIBPTHREAD_SPLITSTACK_MAIN");
if (env_bigstack != NULL || env_splitstack == NULL) {
Add a knob LIBPTHREAD_BIGSTACK_MAIN, which instructs libthr to leave the whole RLIMIT_STACK-sized region of the kernel-allocated stack as the stack of main thread. By default, the main thread stack is clamped at 2MB (4MB on 64bit ABIs) and the rest is used for other threads stack allocation. Since there is no programmatic way to adjust the size of the main thread stack, pthread_attr_setstacksize() is too late, the knob allows user to manage the main stack size both for single-threaded and multi-threaded processes with the rlimit. Reported by: "Ivan A. Kosarev" <ivan@ivan-labs.com> Tested by: dim Sponsored by: The FreeBSD Foundation MFC after: 3 days
2014-08-13 05:53:41 +00:00
if (getrlimit(RLIMIT_STACK, &rlim) == -1)
PANIC("Cannot get stack rlimit");
_thr_stack_initial = rlim.rlim_cur;
}
Make libthr(3) use sysconf(_SC_NPROCESSORS_CONF); this shaves off two calls to sysctl(2) from the binary startup. Reviewed by: kib MFC after: 2 weeks Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18046
2018-11-19 18:24:08 +00:00
_thr_is_smp = sysconf(_SC_NPROCESSORS_CONF);
if (_thr_is_smp == -1)
PANIC("Cannot get _SC_NPROCESSORS_CONF");
- Remove variable _thr_scope_system, all threads are system scope. - Rename _thr_smp_cpus to boolean variable _thr_is_smp. - Define CPU_SPINWAIT macro for each arch, only X86 supports it.
2006-12-15 11:52:01 +00:00
_thr_is_smp = (_thr_is_smp > 1);
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
_thr_page_size = getpagesize();
_thr_guard_default = _thr_page_size;
_pthread_attr_default.guardsize_attr = _thr_guard_default;
_pthread_attr_default.stacksize_attr = _thr_stack_default;
Add my recent work of adaptive spin mutex code. Use two environments variable to tune pthread mutex performance: 1. LIBPTHREAD_SPINLOOPS If a pthread mutex is being locked by another thread, this environment variable sets total number of spin loops before the current thread sleeps in kernel, this saves a syscall overhead if the mutex will be unlocked very soon (well written application code). 2. LIBPTHREAD_YIELDLOOPS If a pthread mutex is being locked by other threads, this environment variable sets total number of sched_yield() loops before the currrent thread sleeps in kernel. if a pthread mutex is locked, the current thread gives up cpu, but will not sleep in kernel, this means, current thread does not set contention bit in mutex, but let lock owner to run again if the owner is on kernel's run queue, and when lock owner unlocks the mutex, it does not need to enter kernel and do lots of work to resume mutex waiters, in some cases, this saves lots of syscall overheads for mutex owner. In my practice, sometimes LIBPTHREAD_YIELDLOOPS can massively improve performance than LIBPTHREAD_SPINLOOPS, this depends on application. These two environments are global to all pthread mutex, there is no interface to set them for each pthread mutex, the default values are zero, this means spinning is turned off by default.
2007-10-30 05:57:37 +00:00
env = getenv("LIBPTHREAD_SPINLOOPS");
if (env)
_thr_spinloops = atoi(env);
env = getenv("LIBPTHREAD_YIELDLOOPS");
if (env)
_thr_yieldloops = atoi(env);
MFp4: Enqueue thread in LIFO, this can cause starvation, but it gives better performance. Use _thr_queuefifo to control the frequency of FIFO vs LIFO, you can use environment string LIBPTHREAD_QUEUE_FIFO to configure the variable.
2012-05-03 09:17:31 +00:00
env = getenv("LIBPTHREAD_QUEUE_FIFO");
if (env)
_thr_queuefifo = atoi(env);
Import my recent 1:1 threading working. some features improved includes: 1. fast simple type mutex. 2. __thread tls works. 3. asynchronous cancellation works ( using signal ). 4. thread synchronization is fully based on umtx, mainly, condition variable and other synchronization objects were rewritten by using umtx directly. those objects can be shared between processes via shared memory, it has to change ABI which does not happen yet. 5. default stack size is increased to 1M on 32 bits platform, 2M for 64 bits platform. As the result, some mysql super-smack benchmarks show performance is improved massivly. Okayed by: jeff, mtm, rwatson, scottl
2005-04-02 01:20:00 +00:00
TAILQ_INIT(&_thr_atfork_list);
}
init_once = 1;
}
Reference in New Issue Copy Permalink