for the same object. This can happen when object is a dependency of the
dlopen()ed dso. When called several times, we waste time due to unneeded
processing, and memory, because obj->vertab is allocated anew on each
iteration.
Reviewed by: kan
MFC after: 2 weeks
are assumed to not fail.
Make the xcalloc() calling conventions follow the calloc(3) calling
conventions and replace unchecked calls to calloc() with calls to
xcalloc().
Remove redundand declarations from xmalloc.c, which are already
present in rtld.h.
Reviewed by: kan
Discussed with: bde
MFC after: 2 weeks
Do not relocate twice an object which happens to be needed by loaded
binary (or dso) and some filtee opened due to symbol resolution when
relocating need objects. Record the state of the relocation
processing in Obj_Entry and short-circuit relocate_objects() if
current object already processed.
Do not call constructors for filtees loaded during the early
relocation processing before image is initialized enough to run
user-provided code. Filtees are loaded using dlopen_object(), which
normally performs relocation and initialization. If filtee is
lazy-loaded during the relocation of dso needed by the main object,
dlopen_object() runs too earlier, when most runtime services are not
yet ready.
Postpone the constructors call to the time when main binary and
depended libraries constructors are run, passing the new flag
RTLD_LO_EARLY to dlopen_object(). Symbol lookups callers inform
symlook_* functions about early stage of initialization with
SYMLOOK_EARLY. Pass flags through all functions participating in
object relocation.
Use the opportunity and fix flags argument to find_symdef() in
arch-specific reloc.c to use proper name SYMLOOK_IN_PLT instead of
true, which happen to have the same numeric value.
Reported and tested by: theraven
Reviewed by: kan
MFC after: 2 weeks
Stop using strerror(3) in rtld, which brings in msgcat and stdio.
Directly access sys_errlist array of errno messages with private
rtld_strerror() function.
Now,
$ size /libexec/ld-elf.so.1
text data bss dec hex filename
96983 2480 8744 108207 1a6af /libexec/ld-elf.so.1
Reviewed by: dim, kan
MFC after: 2 weeks
Provide rtld-private implementations of __stack_chk_guard,
__stack_chk_fail() and __chk_fail() symbols, to be used by functions
linked from libc_pic.a. This avoids use of libc stack_protector.c,
which pulls in syslog(3) and stdio as dependency.
Also, do initialize rtld-private copy __stack_chk_guard, previously
libc-provided one was not initialized, since we do not call rtld
object _init() methods.
Reviewed by: kan
MFC after: 3 weeks
yet, and object segments are not yet mapped. Only parse the notes that
appear in the first page of the dso (as it should be anyway), and use
the preloaded page content.
Reported and tested by: stass
MFC after: 20 days
particular on ARM, do require working init arrays.
Traditional FreeBSD crt1 calls _init and _fini of the binary, instead
of allowing runtime linker to arrange the calls. This was probably
done to have the same crt code serve both statically and dynamically
linked binaries. Since ABI mandates that first is called preinit
array functions, then init, and then init array functions, the init
have to be called from rtld now.
To provide binary compatibility to old FreeBSD crt1, which calls _init
itself, rtld only calls intializers and finalizers for main binary if
binary has a note indicating that new crt was used for linking. Add
parsing of ELF notes to rtld, and cache p_osrel value since we parsed
it anyway.
The patch is inspired by init_array support for DragonflyBSD, written
by John Marino.
Reviewed by: kan
Tested by: andrew (arm, previous version), flo (sparc64, previous version)
MFC after: 3 weeks
for TLS microbenchmark using global-dynamic TLS model on amd64 (which is
default for PIC dso objects).
Split the slow path into tls_get_addr_slow(), for which inlining is
disabled. This prevents the registers spill on tls_get_addr_common()
entry.
Provide static branch hint to the compiler, indicating that slow path
is not likely to be taken.
While there, do some minimal style adjustments.
Reported and tested by: davidxu
MFC after: 1 week
Since after r232498 the ctype macros require working access to
thread-local variables, rtld crashes when libmap.conf is present.
Use hand-made isspace1() macro which is enough to detect spaces in
libmap.conf.
Reported by: alc, lme, many on current@
Tested by: lme
Reviewed by: dim, kan
MFC after: 1 week
relocations until tls is initialized and stacks permissions correctly
set. This allows the ifunc to call malloc(3) and some other heavy
services.
Add debug banner.
MFC after: 3 days
from the dispatcher would also acquire bind lock in read mode, which
is the supported operation. plt is explicitely designed to allow safe
multithreaded updates, so the shared lock do not cause problems.
The error in r228435 is that it allows read lock acquisition after the
write lock for the bind block. If we dlopened the shared object that
contains IRELATIVE or jump slot which target is STT_GNU_IFUNC, then
possible recursive plt resolve from the dispatcher would cause it.
Postpone the resolution for irelative/ifunc right before initializers
are called, and drop bind lock around calls to dispatcher. Use
initlist to iterate over the objects instead of the ->next, due to
drop of the bind lock in iteration.
For i386/reloc.c:reloc_iresolve(), fix calculation of the dispatch
function address for dso, by taking into account possible non-zero
relocbase.
MFC after: 3 weeks
rtld on 386 and amd64. This adds runtime bits neccessary for the use
of the dispatch functions from the dynamically-linked executables and
shared libraries.
To allow use of external references from the dispatch function, resolution
of the R_MACHINE_IRESOLVE relocations in PLT is postponed until GOT entries
for PLT are prepared, and normal resolution of the GOT entries is finished.
Similar to how it is done by GNU, IRELATIVE relocations are resolved in
advance, instead of normal lazy handling for PLT.
Move the init_pltgot() call before the relocations for the object are
processed.
MFC after: 3 weeks
relocations are processed, since tls initialization section might be
itself subject for relocations. Only set up of the block is postponed,
the tls block offsets are allocated before relocation processing, since
TLS-related relocations may need offsets ready.
Reported by: ale
PR: threads/161344
Reviewed by: kan
MFC after: 1 week
executable) after r190885. The whole region for the dso is mmaped with
MAP_NOCORE flag, doing only mprotect(2) over .bss prevented it from
writing .bss to core files.
Revert the optimization of using mprotect(2) to establish .bss, overlap
the section with mmap(2).
Reported by: attilio
Reviewed by: attilio, emaste
Approved by: re (bz)
MFC after: 2 weeks
functions. The _rtld_error() function might be called early during the rtld
bootstrap, in which case function pointers are not yet functional on ia64
due to required relocations not yet performed.
Reported, reviewed and tested by: marcel
Approved by: re (bz)
function (a hook necessary for gdb support), is inlined, but since the
function contains no code, no calls to it are generated. When gdb is
debugging a dynamically linked program, this causes backtraces to be
corrupted.
Fix it by marking the function __noinline, and inserting an empty asm
statement, that pretends to clobber memory. This forces the compiler to
emit calls to r_debug_state() throughout rtld.c.
Approved by: re (kib)
C runtime services, like printf(). Unfortunately, the multithread-safeness
measures in the libc do not work in rtld environment.
Rip the kernel printf() implementation and use it in the rtld instead of
libc version. This printf does not require any shared global data and thus
is mt-safe. Systematically use rtld_printf() and related functions, remove
the calls to err(3).
Note that stdio is still pulled from libc due to libmap implementaion using
fopen(). This is safe but unoptimal, and can be changed later.
Reported and tested by: pgj
Diagnosed and reviewed by: kan (previous version)
Approved by: re (bz)
The second close(2) call resulted in heisenbugs in some multi-threaded
applications where e.g. dlopen(3) call in one thread could close a file
descriptor for a file having been opened in other thread concurrently.
My litmus test for this issue was an openoffice.org build.
Reviewed by: jhb
MFC after: 2 weeks
- Emitt an error when encountering an unsupported and in case of the
kernel also for unaligned relocations.
- Fix R_SPARC_LOX10 relocations. Apparently these are hardly ever used.
- Add the _RF_X committed in r212998 also to the tables in the sparc64
reloc.c in order reduce differences between the kernel and the userland
source. This results in no functional change though.
- Fix further inconsistencies in the abbreviations of the names of the
relocations.
- Further whitespace fixes.
Obtained from: NetBSD [1]
values for resolved symbols relative to relocbase instead of sections
so detect this case and handle as appropriate, which allows using
kernel modules linked with affected versions of binutils. Actually I
think this is a bug in binutils but given that apparently nobody
complained for nearly six years and powerpc has basically the same
workaround I decided to put it in for the sparc64 kernel, too.
- Fix R_SPARC_HIX22 relocations. Apparently these are hardly ever used.
Instead of aborting in locate_dependency(), propagate the error to
caller. The rtld startup function does the right thing with an error
from rtld_verify_versions(), depending on the mode of operation.
Reported by: maho
In collaboration with: kan
MFC after: 1 week
Place elements on DAG lists in breadth-first order. This allows us to
walk pre-built list in all cases where breadth-first dependency chain
enumeration is required.
Fix dlsym on special handle obtained by dlopen(NULL, ...) to do what
comment claims it does. Take advantage of recently added symlook_global
function to iterate over main objects and global DAGs lists properly in
search of a symbol. Since rtld itself provides part of the global
namespace, search rtld_obj too.
Remove recursion from init_dag and symlook_needed functions. Use
symlook_needed for ELF filtee processing only and change lookup order
used in the function to match the order used by Solaris runtime linker
under same circumstances. While there, fix weak symbol handling in the
loop so that we return the first weak symbol definition if no strong one
was found, instead of the last one.
Reviewed by: kib
MFC after: 1 month
one. Search global objects, together with main object and
dependencies, for the requested symbol.
Move the common code from symlook_default() into new helper
symlook_global(), and use it both in symlook_global() and
get_program_var_addr().
Supply lock state to get_program_var_addr().
Reviewed by: kan
Tested by: Mykola Dzham <i levsha me>
by kernel, and parse PT_GNU_STACK phdr from linked and loaded dsos.
If the loaded dso requires executable stack, as specified by PF_X bit
of p_flags of PT_GNU_STACK phdr, but current stack protection does not
permit execution, the __pthread_map_stacks_exec symbol is looked up
and called. It should be implemented in libc or threading library and
change the protection mode of all thread stacks to be executable.
Provide a private interface _rtld_get_stack_prot() to export the stack
access mode as calculated by rtld.
Reviewed by: kan
3DNow, MMX and floating point instructions in rtld-elf.
Otherwise, _rtld_bind() (and whatever it calls) could possibly clobber
function arguments that are passed in SSE/3DNow/MMX/FP registers,
usually floating point values. This can happen, for example, when clang
generates SSE code for memset() or memcpy() calls.
One symptom of this is sshd dying early on amd64 with "PRNG not seeded",
which is ultimately caused by libcrypto.so.6 calling RAND_add() with a
double parameter. That parameter is passed via %xmm0, which gets wiped
out by an SSE memset() in _rtld_bind().
Reviewed by: kib, kan
C function, the caller's stack frame must have room to store all of the
arguments to that function. While here, fix stack frame alignment issues.
Without this change, the compiler will save r3 and r4 into the caller's
stack frame before calling setjmp() in _rtld_bind(). These would then
overwrite arguments to the newly-bound function, causing eventual failures.
filters are implemented.
Filtees are loaded on demand, unless LD_LOADFLTR environment variable
is set or -z loadfltr was specified during the linking. This forces
rtld to upgrade read-locked rtld_bind_lock to write lock when it
encounters an object with filter during symbol lookup.
Consolidate common arguments of the symbol lookup functions in the
SymLook structure. Track the state of the rtld locks in the
RtldLockState structure. Pass local RtldLockState through the rtld
symbol lookup calls to allow lock upgrades.
Reviewed by: kan
Tested by: Mykola Dzham <i levsha me>, nwhitehorn (powerpc)
dlclose() call may unload the object of the outer call prematurely
because objects are unreferenced before _fini() calls.
Fix this by unreferencing objects after calling objlist_call_fini() in
dlclose(). Therefore objlist_call_fini() now calls the fini function if
the reference count of an object is 1. In addition we must restart the
list_fini traversal after every _fini() call because another dlclose()
call might have modified the reference counts.
Add an XXX comment to objlist_call_fini() about possible race with
dlopen().
PR: 133246, 149464
Reviewed by: kan, kib
loaded as a dependency and marked -z nodlopen, object' DAG is already
initialized by load_needed_objects(). Due to this, the init_dag() call
from dlopen() does not increment refcount for the object [1].
Change init_dag() to not increment DAG refcount. Require explicit calls
to ref_dag() to increment, and assert that ref_dag() and unref_dag()
are called for root that has dag initialized. To fix the noted issue,
unconditionally call both init_dag() and ref_dag() in dlopen() for the
case when the object was already loaded, making it similar to the case
of newly loaded object.
Noted by: jh [1]
Reviewed by: jh, kan
MFC after: 6 days
dependency, then the dso never has its DAG initialized. Empty DAG
makes ref_dag() call in dlopen() a nop, and the dso refcount is off
by one.
Initialize the DAG on the first dlopen() call, using a boolean flag
to prevent double initialization.
From the PR (edited):
Assume we have a library liba.so, containing a function a(), and a
library libb.so, containing function b(). liba.so needs functionality
from libb.so, so liba.so links in libb.so.
An application doesn't know about the relation between these libraries,
but needs to call a() and b(). It dlopen()s liba.so and obtains a
pointer to a(), then it dlopen()s libb.so and obtains a pointer to b().
As soon as the application doesn't need a() anymore, it dlclose()s liba.so.
Expected result: the pointer to b() is still valid and can be called
Actual result: the pointer to b() has become invalid, even though the
application did not dlclose() the handle to libb.so. On calling b(), the
application crashes with a segmentation fault.
PR: misc/151861
Based on patch by: jh
Reviewed by: kan
Tested by: Arjan van Leeuwen <freebsd-maintainer opera com>
MFC after: 1 week
With r169630 I disabled symbol versioning because it broke rtld. With
r211706 rtld got broken for ia64 & powerpc64. It was fixed for powerpc64
with r212497. In between, r211749 removed the exports table because the
version script handled the exports. But wait, symbol versioning was
disabled on ia64.
With exports controlled by the version script and symbol versioning
disabled, all symbols are exported and too many symbols bind to the
definition in rtld. Let's just say that waird things happen.
So, enable symbol versioning on ia64 and apply a work-around for the
SIGSEGV that triggered r169630 to begin with: when rtld relocates
itself, it comes across r_debug_state and for some reason can't find the
definition. This causes a failure, relocation aborts and null pointers
galore. The work-around is to ignore the missing definition when rtld
is relocating itself and keep going.
Maybe with the next binutils this will all go away. Maybe not, in
which case I still need to figure out why r_debug_state cannot be found.
BTW: r_debug_state is in the symbol map -- I don't think any other rtld
symbols that rtld references are in the symbol map...
where long PLT calls in multi-threaded environments could end up with
incorrect jmptab values. The second is that, after the addition of extended
PLT support, I forgot to update the PLT icache synchronization code to cover
the extended PLT instead of just the basic PLT.
MFC after: 10 days
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.
Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.
Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.
The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.
Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks
Use local version of getpagesize(), rtld_getpagesize() in private allocator.
Override the __getosreldate() previously fetched from libc_pic.a with
local version that uses aux value if present. Note that __getosreldate()
is used by rtld indirectly, by mmap(2) libc wrapper.
To be able to utilize aux, split digest_dynamic() for use by init_rtld()
into two parts, where the first one does not call malloc(), and the
second part uses it. init_rtld() is able to initialize global variables
before digest_dynamic2() calls. In particular, pagesize and osreldate are
set up from the aux values.
Now, rtld avoids (two) sysctl calls in startup.
Tested by: marius (sparc64)
MFC after: 1 month
