Commit Graph

152 Commits

Author SHA1 Message Date
Pedro F. Giffuni
5e53a4f90f lib: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-26 02:00:33 +00:00
Michael Zhilin
37a6f46190 [libthr] revert change of visibility of _thread_keytable to unbreak debugger
Fix regression by r318539. The sysutils/pstack uses library libthread_db to
read information about threads state. The function pt_ta_new makes lookup of
several key symbols including _thread_keytable. But r318539 mades this field
static. It causes silent ignore of libthr library by pstack and as result
sysutils/pstack doesn't output any thread information.

This fix changes this field back to non-static.

MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D11738
2017-10-02 20:33:16 +00:00
Eric van Gyzen
718fb5ba5b libthr: fix warnings at WARNS=6
Fix more warnings about redundant declarations.

Reviewed by:	kib emaste
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10932
2017-05-26 15:56:28 +00:00
Eric van Gyzen
5a6d7b723f libthr: fix warnings from GCC when WARNS=6
Fix warnings about:
- redundant declarations
- a local variable shadowing a global function (dlinfo)
- an old-style function definition (with an empty parameter list)
- a variable that is possibly used uninitialized

"make tinderbox" passes this time, except for a few unrelated
kernel failures.

Reviewed by:	kib
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10870
2017-05-23 16:12:50 +00:00
Eric van Gyzen
3f8455b090 Add clock_nanosleep()
Add a clock_nanosleep() syscall, as specified by POSIX.
Make nanosleep() a wrapper around it.

Attach the clock_nanosleep test from NetBSD. Adjust it for the
FreeBSD behavior of updating rmtp only when interrupted by a signal.
I believe this to be POSIX-compliant, since POSIX mentions the rmtp
parameter only in the paragraph about EINTR. This is also what
Linux does. (NetBSD updates rmtp unconditionally.)

Copy the whole nanosleep.2 man page from NetBSD because it is complete
and closely resembles the POSIX description. Edit, polish, and reword it
a bit, being sure to keep any relevant text from the FreeBSD page.

Reviewed by:	kib, ngie, jilles
MFC after:	3 weeks
Relnotes:	yes
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10020
2017-03-19 00:51:12 +00:00
Pedro F. Giffuni
649702c5a3 Make use of clang nullability attributes.
Replace uses of the GCC __nonnull__ attribute with the clang nullability
qualifiers. The replacement should be transparent for clang developers as
the new qualifiers will produce the same warnings and will be useful for
static checkers but will not cause aggressive optimizations.

GCC will not produce such warnings and developers will have to use
upgraded GCC ports built with the system headers from r312538.

Hinted by:	Apple's Libc-1158.20.4, Bionic libc
MFC after:	11.1 Release

Differential Revision:	https://reviews.freebsd.org/D9004
2017-01-28 20:54:43 +00:00
Konstantin Belousov
bf890e4877 Remove empty initializer for the once facility. It was not needed
since r179417.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-07-27 15:14:11 +00:00
Conrad Meyer
3a7d122f96 libthr: Add vprintf variant of _thread_printf, formatted PANIC()
No ABI change.

Reviewed by:	kib
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D6672
2016-06-01 16:09:56 +00:00
Konstantin Belousov
6180f50bbb Stop inlining the struct ucond definition into struct pthread_cond.
This avoids unneccessary casts and make the calls to _thr_ucond_*()
functions less questionable.

The c_spare field was not included into struct pthread_cond, so the
change modifies libthr ABI for shared condvars.  But since an off-page
does not legitimately contains any other data past the struct
pthread_cond, the change keeps shared condvars from pre- and post-
changed libthr compatible.  Also note that the whole struct ucond was
never copied in or out by kernel.

For private condvars, the privately allocated memory was never exposed
outside libthr.

Sponsored by:	The FreeBSD Foundation
2016-05-29 19:35:55 +00:00
Konstantin Belousov
2a339d9e3d Add implementation of robust mutexes, hopefully close enough to the
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.

A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held.  The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.

The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths.  Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.

The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive).  Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.

Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot.  When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.

The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.

Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
   pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
   the lifetime of the shared mutex associated with a vnode' page.

Reviewed by:	jilles (previous version, supposedly the objection was fixed)
Discussed with:	brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
Konstantin Belousov
841ecd471a Remove unused variable. It was write-only before r297139.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-04-04 06:58:59 +00:00
Konstantin Belousov
6044c03a37 Apparently there are some popular programs around which assume that it
is safe to call pthread_mutex_init() on the same shared mutex several
times.  POSIX claims that the behaviour in this case is undefined.

Make this working by only allowing one caller to initialize the mutex.
Other callers either see already completed initialization and do
nothing, or busy-loop yielding while designated initializer finishes.
Also make the API requirements loose by initializing mutexes on other
pthread_mutex*() calls if they see uninitialized shared mutex.

Only mutexes provide the hack for now, but it could be also
implemented for other process shared primitives from libthr.

Reported and tested by:	"Oleg V. Nauman" <oleg@opentransfer.com>
Sponsored by:	The FreeBSD Foundation
2016-03-22 10:51:42 +00:00
Konstantin Belousov
53fd961f05 Lock pshared_lock shared around fork, to ensure that the COW snapshot
of the pshared hash in child is consistent and can be safely used.

Reported and tested by:	"Oleg V. Nauman" <oleg@opentransfer.com>
Sponsored by:	The FreeBSD Foundation
2016-03-21 06:52:35 +00:00
Konstantin Belousov
1bdbd70599 Implement process-shared locks support for libthr.so.3, without
breaking the ABI.  Special value is stored in the lock pointer to
indicate shared lock, and offline page in the shared memory is
allocated to store the actual lock.

Reviewed by:	vangyzen (previous version)
Discussed with:	deischen, emaste, jhb, rwatson,
	Martin Simmons <martin@lispworks.com>
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
Konstantin Belousov
bd43f0691c If libthr.so is dlopened without RTLD_GLOBAL flag, the libthr symbols
do not participate in the global symbols namespace, but rtld locks are
still replaced and functions are interposed.  In particular,
__pthread_map_stacks_exec is resolved to the libc version.  If a
library is loaded later, which requires adjustment of the stack
protection mode, rtld calls into libc __pthread_map_stacks_exec due to
the symbols scope.  The libc version might recurse into binder and
recursively acquire rtld bind lock, causing the hang.

Make libc __pthread_map_stacks_exec() interposed, which synchronizes
rtld locks and version of the stack exec hook when libthr loaded,
regardless of the symbol scope control or symbol resolution order.

The __pthread_map_stacks_exec() symbol is removed from the private
version in libthr since libc symbol now operates correctly in presence
of libthr.

Reported and tested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2016-02-08 19:24:13 +00:00
Jilles Tjoelker
17981398bd libthr: Don't use both __sys_open() and __sys_openat(). 2015-12-20 16:33:56 +00:00
Jung-uk Kim
be070eb896 Fix a typo in comment and explain the reason. 2015-03-09 20:26:42 +00:00
Konstantin Belousov
45468c5356 Properly interpose libc spinlocks, was missed in r276630. In
particular, stdio locking was affected.

Reported and tested by:	"Matthew D. Fuller" <fullermd@over-yonder.net>
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2015-02-14 11:47:40 +00:00
Andrew Turner
20fe2c9465 Merge all the copies of _tcb_ctor and _tcb_dtor.
The amd64, i386, and sparc64 versions were identical, with the one
difference where the former two used inline asm instead of _tcb_get. I
have compared the function before and after replacing the asm with _tcb_get
and found the object files to be identical.

The arm, mips, and powerpc versions were almost identical. The only
difference was the powerpc version used an alignment of 1 where arm and
mips used 16. As this is an increase in alignment is will be safe.

Along with this arm, mips, and powerpc all passed, when initial was true,
the value returned from _tcb_get as the first argument to
_rtld_allocate_tls. This would then return this pointer back to the caller.
We can remove these extra calls by checking if initial is set and setting
the thread control block directly. As this is what the sparc64 code does
we can use it directly.

As after these observations all the architectures can now have identical
code we can merge them into a common file.

Differential Revision:	https://reviews.freebsd.org/D1556
Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
2015-01-21 16:41:05 +00:00
Konstantin Belousov
397d851d66 Reduce the size of the interposing table and amount of
cancellation-handling code in the libthr.  Translate some syscalls
into their more generic counterpart, and remove translated syscalls
from the table.

List of the affected syscalls:
creat, open -> openat
raise -> thr_kill
sleep, usleep -> nanosleep
pause -> sigsuspend
wait, wait3, waitpid -> wait4

Suggested and reviewed by:	jilles (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-01-11 22:16:31 +00:00
Konstantin Belousov
1a744fefc2 Avoid calling internal libc function through PLT or accessing data
though GOT, by staticizing and hiding.  Add setter for
__error_selector to hide it as well.

Suggested and reviewed by:	jilles
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-01-05 01:06:54 +00:00
Konstantin Belousov
8495e8b1e9 Fix known issues which blow up the process after dlopen("libthr.so")
(or loading a dso linked to libthr.so into process which was not
linked against threading library).

- Remove libthr interposers of the libc functions, including
  __error(). Instead, functions calls are indirected through the
  interposing table, similar to how pthread stubs in libc are already
  done.  Libc by default points either to syscall trampolines or to
  existing libc implementations.  On libthr load, libthr rewrites the
  pointers to the cancellable implementations already in libthr.  The
  interposition table is separate from pthreads stubs indirection
  table to not pull pthreads stubs into static binaries.

- Postpone the malloc(3) internal mutexes initialization until libthr
  is loaded.  This avoids recursion between calloc(3) and static
  pthread_mutex_t initialization.

- Reinstall signal handlers with wrapper on libthr load.  The
  _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2)
  when libthr is statically referenced from the main binary.

In the process, fix openat(2), swapcontext(2) and setcontext(2)
interposing.  The libc symbols were exported at different versions
than libthr interposers.  Export both libc and libthr versions from
libc now, with default set to the higher version from libthr.

Remove unused and disconnected swapcontext(3) userspace implementation
from libc/gen.

No objections from:	deischen
Tested by:	pho, antoine (exp-run) (previous versions)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-01-03 18:38:46 +00:00
Ed Maste
294246bb7d Revert r274772: it is not valid on MIPS
Reported by:	sbruno
2014-11-25 03:50:31 +00:00
Ed Maste
688fd61ae8 Use canonical __PIC__ flag
It is automatically set when -fPIC is passed to the compiler.

Reviewed by:	dim, kib
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1179
2014-11-21 02:05:48 +00:00
Konstantin Belousov
0a9655a082 If check_deferred_signal() execution needs binding of PLT symbol,
unlocking the rtld bind lock results in the processing of ast and
recursing into the check_deferred_signal().  Nested execution of
check_deferred_signal() delivers the signal to user code and clears
si_signo.  On return, top-level check_deferred_signal() frame
continues delivering the same signal one more time, but now with zero
si_signo.

Fix this by adding a flag to indicate that deferred delivery is
running, so check_deferred_signal() should avoid doing anything. Since
user signal handler is allowed to modify the passed machine context to
make return from the signal handler to cause arbitrary jump, or do
longjmp(). For this case, also clear the flag in thr_sighandler(),
since kernel signal delivery means that nested delivery code should
not run right now.

Reported by:	Vitaly Magerya <vmagerya@gmail.com>
Reviewed by:	davidxu, jilles
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2013-11-23 15:48:17 +00:00
Jilles Tjoelker
706b04b66f libthr: Remove _thr_rtld_fini(), unused since r245630. 2013-04-12 19:47:32 +00:00
David Xu
a7b84c6512 In suspend_common(), don't wait for a thread which is in creation, because
pthread_suspend_all_np() may have already suspended its parent thread.
Add locking code in pthread_suspend_all_np() to only allow one thread
to suspend other threads, this eliminates a deadlock where two or more
threads try to suspend each others.
2012-08-27 03:09:39 +00:00
David Xu
e220a13ab9 MFp4:
Further decreases unexpected context switches by defering mutex wakeup
until internal sleep queue lock is released.
2012-08-11 23:17:02 +00:00
David Xu
84ac0fb8ca MFp4:
Enqueue thread in LIFO, this can cause starvation, but it gives better
performance. Use _thr_queuefifo to control the frequency of FIFO vs LIFO,
you can use environment string LIBPTHREAD_QUEUE_FIFO to configure the
variable.
2012-05-03 09:17:31 +00:00
George V. Neville-Neil
6e047a2426 Set SIGCANCEL to SIGTHR as part of some cleanup of DTrace code.
Reviewed by:	davidxu@
MFC after:	1 week
2012-04-18 16:29:55 +00:00
David Xu
17ce606321 umtx operation UMTX_OP_MUTEX_WAKE has a side-effect that it accesses
a mutex after a thread has unlocked it, it event writes data to the mutex
memory to clear contention bit, there is a race that other threads
can lock it and unlock it, then destroy it, so it should not write
data to the mutex memory if there isn't any waiter.
The new operation UMTX_OP_MUTEX_WAKE2 try to fix the problem. It
requires thread library to clear the lock word entirely, then
call the WAKE2 operation to check if there is any waiter in kernel,
and try to wake up a thread, if necessary, the contention bit is set again
by the operation. This also mitgates the chance that other threads find
the contention bit and try to enter kernel to compete with each other
to wake up sleeping thread, this is unnecessary. With this change, the
mutex owner is no longer holding the mutex until it reaches a point
where kernel umtx queue is locked, it releases the mutex as soon as
possible.
Performance is improved when the mutex is contensted heavily.  On Intel
i3-2310M, the runtime of a benchmark program is reduced from 26.87 seconds
to 2.39 seconds, it even is better than UMTX_OP_MUTEX_WAKE which is
deprecated now. http://people.freebsd.org/~davidxu/bench/mutex_perf.c
2012-04-05 02:24:08 +00:00
David Xu
e70bf9d5eb When destroying a barrier, waiting all threads exit the barrier,
this makes it possible a thread received PTHREAD_BARRIER_SERIAL_THREAD
immediately free memory area of the barrier.
2012-03-16 04:35:52 +00:00
David Xu
e7004bf44d Plug a memory leak. When a cached thread is reused, don't clear sleep
queue pointers, just reuse it.

PR:		164828
MFC after:	1 week
2012-02-07 02:57:36 +00:00
Konstantin Belousov
da2fcff746 Implement the __pthread_map_stacks_exec() for libthr.
Stack creation code is changed to call _rtld_get_stack_prot() to get
the stack protection right. There is a race where thread is created
during dlopen() of dso that requires executable stacks. Then,
_rtld_get_stack_prot() may return PROT_READ | PROT_WRITE, but thread
is still not linked into the thread list. In this case, the callback
misses the thread stack, and rechecks the required protection
afterward.

Reviewed by:	davidxu
2011-01-09 12:38:40 +00:00
David Xu
d1078b0b03 MFp4:
- Add flags CVWAIT_ABSTIME and CVWAIT_CLOCKID for umtx kernel based
  condition variable, this should eliminate an extra system call to get
  current time.

- Add sub-function UMTX_OP_NWAKE_PRIVATE to wake up N channels in single
  system call. Create userland sleep queue for condition variable, in most
  cases, thread will wait in the queue, the pthread_cond_signal will defer
  thread wakeup until the mutex is unlocked, it tries to avoid an extra
  system call and a extra context switch in time window of pthread_cond_signal
  and pthread_mutex_unlock.

The changes are part of process-shared mutex project.
2010-12-22 05:01:52 +00:00
David Xu
7f25f6c72d Get cpuset in pthread_attr_get_np() and free it in pthread_attr_destroy().
MFC after:	7 days
2010-10-25 09:16:04 +00:00
David Xu
de1e74c6a5 Revert revision 214007, I realized that MySQL wants to resolve
a silly rwlock deadlock problem, the deadlock is caused by writer
waiters, if a thread has already locked a reader lock, and wants to
acquire another reader lock, it will be blocked by writer waiters,
but we had already fixed it years ago.
2010-10-20 02:34:02 +00:00
David Xu
a6b9b59e04 Add pthread_rwlockattr_setkind_np and pthread_rwlockattr_getkind_np, the
functions set or get pthread_rwlock type, current supported types are:
   PTHREAD_RWLOCK_PREFER_READER_NP,
   PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP,
   PTHREAD_RWLOCK_PREFER_WRITER_NP,
default is PTHREAD_RWLOCK_PREFER_WRITER_NONCECURSIVE_NP, this maintains
binary compatible with old code.
2010-10-18 05:09:22 +00:00
David Xu
722488013d change code to use unwind.h. 2010-09-30 12:59:56 +00:00
David Xu
bbb64c2143 In current code, statically initialized and destroyed object have
same null value, the code can not distinguish between them, to
fix the problem, now a destroyed object is assigned to a non-null
value, and it will be rejected by some pthread functions.
PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP is changed to number 1, so that
adaptive mutex can be statically initialized correctly.
2010-09-28 04:57:56 +00:00
David Xu
9f1dc4c107 Add missing field. 2010-09-25 08:36:46 +00:00
David Xu
f4213b9006 To support stack unwinding for cancellation points, add -fexceptions flag
for them, two functions _pthread_cancel_enter and _pthread_cancel_leave
are added to let thread enter and leave a cancellation point, it also
makes it possible that other functions can be cancellation points in
libraries without having to be rewritten in libthr.
2010-09-25 01:57:47 +00:00
David Xu
93ea4a71bf In most cases, cancel_point and cancel_async needn't be checked again,
because cancellation is almostly checked at cancellation points.
2010-09-24 07:52:07 +00:00
David Xu
4173ebef4f Because atfork lock is held while forking, a thread cancellation triggered
by atfork handler is unsafe, use intenal flag no_cancel to disable it.
2010-09-19 09:03:11 +00:00
David Xu
3832fd24f1 add code to support stack unwinding when thread exits. note that only
defer-mode cancellation works, asynchrnous mode does not work because
it lacks of libuwind's support. stack unwinding is not enabled unless
LIBTHR_UNWIND_STACK is defined in Makefile.
2010-09-15 02:56:32 +00:00
David Xu
707ee8154d Move back IN_GCLIST flag into field tlflags, since thread list and gc list
still share same lock.
2010-09-15 01:21:30 +00:00
David Xu
cbadc1d7ad Fix copy&paste problem. 2010-09-13 11:57:46 +00:00
David Xu
a9b764e218 Convert thread list lock from mutex to rwlock. 2010-09-13 07:03:01 +00:00
David Xu
ada33a6e36 Change atfork lock from mutex to rwlock, also make mutexes used by malloc()
module private type, when private type mutex is locked/unlocked, thread
critical region is entered or leaved. These changes makes fork()
async-signal safe which required by POSIX. Note that user's atfork handler
still needs to be async-signal safe, but it is not problem of libthr, it
is user's responsiblity.
2010-09-01 03:11:21 +00:00
David Xu
02c3c85869 Add signal handler wrapper, the reason to add it becauses there are
some cases we want to improve:
  1) if a thread signal got a signal while in cancellation point,
     it is possible the TDP_WAKEUP may be eaten by signal handler
     if the handler called some interruptibly system calls.
  2) In signal handler, we want to disable cancellation.
  3) When thread holding some low level locks, it is better to
     disable signal, those code need not to worry reentrancy,
     sigprocmask system call is avoided because it is a bit expensive.
The signal handler wrapper works in this way:
  1) libthr installs its signal handler if user code invokes sigaction
     to install its handler, the user handler is recorded in internal
     array.
  2) when a signal is delivered, libthr's signal handler is invoke,
     libthr checks if thread holds some low level lock or is in critical
     region, if it is true, the signal is buffered, and all signals are
     masked, once the thread leaves critical region, correct signal
     mask is restored and buffered signal is processed.
  3) before user signal handler is invoked, cancellation is temporarily
     disabled, after user signal handler is returned, cancellation state
     is restored, and pending cancellation is rescheduled.
2010-09-01 02:18:33 +00:00