Commit Graph

73 Commits

Author SHA1 Message Date
Konstantin Belousov
e5c3405ce8 Align initial-exec TLS segments to the p_vaddr % align.
This is continuation of D21163/r359634, which handled the alignment
for global mode.

Non-x86 arches are not handled, maintainers are welcomed.

Tested by:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D24366
2020-04-19 09:28:59 +00:00
Konstantin Belousov
c5ca0d1132 Handle non-plt IRELATIVE relocations, at least for x86.
lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by:	dim (amd64, previous version)
Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D23652
2020-02-13 23:42:09 +00:00
Konstantin Belousov
7e3300e505 rtld: clean up Makefile.
Move all MD statements into $MACHINE_ARCH/Makefile.inc.
Unconditionally apply version script to rtld, the interpreter is not
functional without it for long time.

Reviewed by:	brooks, emaste
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D23083
2020-01-11 09:18:58 +00:00
Konstantin Belousov
4105901933 rtld-elf: Remove x86 elf_rtld.x linker scripts.
First, amd64 version of the script cannot work at least due to the
wrong architecture specification.  Second, kernel can activate shared
objects for long time, due to PIE support.

It seems the intent was to allow ld-elf.so.1 to be build and used as
an executable.  Since we have direct exec mode implemented for dso
ld-elf.so.1, the non-functional and commented out scripts can be
finally removed.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-04 21:43:34 +00:00
Michal Meloun
4849c3a570 Improve R_AARCH64_TLSDESC relocation.
The original code did not support dynamically loaded libraries and used
suboptimal access to TLS variables.
New implementation removes lazy resolving of TLS relocation - due to flaw
in TLSDESC design is impossible to switch resolver function at runtime
without expensive locking.

Due to this, 3 specialized resolvers are implemented:
 - load time resolver for TLS relocation from libraries loaded with main
   executable (thus with known TLS offset).
 - resolver for undefined thread weak symbols.
 - slower lazy resolver for dynamically loaded libraries with fast path for
   already resolved symbols.

PR:		228892, 232149, 233204, 232311
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D18417
2018-12-15 10:38:07 +00:00
Konstantin Belousov
bac111cc74 Silence gcc warnings.
Reported by:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2018-11-23 18:15:23 +00:00
Konstantin Belousov
ca2b9726c3 Remove rtld use of libc amd64_set_fsbase().
One less non-trivial dependency of rtld on libc.  Also,
amd64_set_fsbase() is to be converted to ifunc, which I do not want to
support inside rtld.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-10-29 23:59:26 +00:00
Alex Richardson
903e0ffd07 rtld-elf: compile with WANRS=4 warnings other than -Wcast-align
Reviewed By:	kib
Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17153
2018-10-29 21:08:19 +00:00
Marius Strobl
41fc6f680b o Let rtld(1) set up psABI user trap handlers prior to executing the
objects' init functions instead of doing the setup via a constructor
  in libc as the init functions may already depend on these handlers
  to be in place. This gets us rid of:
  - the undefined order in which libc constructors as __guard_setup()
    and jemalloc_constructor() are executed WRT __sparc_utrap_setup(),
  - the requirement to link libc last so __sparc_utrap_setup() gets
    called prior to constructors in other libraries (see r122883).
  For static binaries, crt1.o still sets up the user trap handlers.
o Move misplaced prototypes for MD functions in to the MD prototype
  section of rtld.h.
o Sprinkle nitems().
2018-02-03 23:14:11 +00:00
Pedro F. Giffuni
e6209940de libexec: adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:25:02 +00:00
Konstantin Belousov
e35ddbe448 Implement LD_BIND_NOT knob for rtld.
From the manpage:
When set to a nonempty string, prevents modifications of the PLT slots
when doing bindings.  As result, each call of the PLT-resolved
function is resolved.  In combination with debug output, this provides
complete account of all bind actions at runtime.

Same feature exists on Linux and Solaris.

Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2017-03-15 21:11:57 +00:00
Ed Maste
ebf8934652 rtld: do not rely on a populated GOT on amd64
On rela architectures GNU BFD ld and gold store the relocation addend
in GOT entries (in addition to the relocation's r_addend field).
rtld previously relied on this to access its own _DYNAMIC symbol in
order to apply its own relocations.

However, recording addends in the GOT is not specified by the ABI,
and some versions of LLVM's LLD linker leave the GOT uninitialized on
rela architectures.

BFD ld does not populate the GOT on sparc64, and sparc64 rtld has a
machine-dependent rtld_dynamic_addr() function that returns the
_DYNAMIC address. Use the same approach on amd64, obtaining the %rip-
relative _DYNAMIC address following a suggestion from Rafael Espíndola.

Architectures other than amd64 should be addressed in future work.

PR:		214972
Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9180
2017-01-16 14:49:29 +00:00
Konstantin Belousov
d27078f990 Adjust r308689 to make rtld compilable with either in-tree or
(hopefully) stock gcc 4.2.1 on i386 and other arches.

In particular:
- Do not use %ebx in the asm constraints on i386, since rtld is
  compiled with -fPIC and gcc cannot handle GOT-base register reload
  (clang and newer gcc can).
- Avoid direct use of [static N] construct in the function
  declaration/definion.  In-tree gcc was patched to support this, but
  stock 4.2.1 cannot handle the feature.

Requested by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-11-21 14:13:57 +00:00
Konstantin Belousov
4352999e0e Pass CPUID[1] %edx (cpu_feature), %ecx (cpu_feature2) and
CPUID[7].%ebx (cpu_stdext_feature), %ecx (cpu_stdext_feature2) to the
ifunc resolvers on x86.

It is much more clean to use CPUID instruction in usermode to retrieve
this information than to pass AT_HWCAP aux vector from kernel, on
x86.  Still, the change does allow for use of AT_HWCAP on arches where it is
needed, by passing aux array to ifunc_init() initializer which should
prepare arguments for ifunc resolvers.

Current signature for resolvers on x86 is
	func_t iresolve(uint32_t cpu_feature, uint32_t cpu_feature2,
	    uint32_t cpu_stdext_feature, uint32_t cpu_stdext_feature2);
where arguments have identical meaning as the kernel variables of the
same name.  The ABIs allow to use resolvers with the void or shortened
list of arguments.

Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D8448
2016-11-15 09:43:26 +00:00
Konstantin Belousov
9fee0541f2 Do not call callbacks for dl_iterate_phdr(3) with the rtld bind and
phdr locks locked.  This allows to call rtld services from the
callback, which is only reasonable for dlopen(path, RTLD_NOLOAD) to
test existence of the library in the image, and for dlsym().  The
later might still be not quite safe, due to the lazy resolution of
filters.

To allow dropping the locks around iteration in dl_iterate_phdr(3), we
insert markers to track current position between relocks.  The global
objects list is converted to tailq and all iterators skip markers,
globallist_next() and globallist_curr() helpers are added.

Reported and tested by:	davide
Reviewed by:	kan
Sponsored by:	The FreeBSD Foundation
MFC after:	3 weeks
2016-01-20 07:21:33 +00:00
Ed Maste
d88def534e rtld: wrap a comment to 80 columns 2016-01-05 02:21:57 +00:00
Warner Losh
8fd53f4577 Create a generalized exec hook that different architectures can hook
into if they need to, but default to no action.

Differential Review: https://reviews.freebsd.org/D2718
2016-01-03 04:32:02 +00:00
Eric van Gyzen
ddab052725 Disable SSE in libthr
Clang emits SSE instructions on amd64 in the common path of
pthread_mutex_unlock.  If the thread does not otherwise use SSE,
this usage incurs a context-switch of the FPU/SSE state, which
reduces the performance of multiple real-world applications by a
non-trivial amount (3-5% in one application).

Instead of this change, I experimented with eagerly switching the
FPU state at context-switch time.  This did not help.  Most of the
cost seems to be in the read/write of memory--as kib@ stated--and
not in the #NM handling.  I tested on machines with and without
XSAVEOPT.

One counter-argument to this change is that most applications already
use SIMD, and the number of applications and amount of SIMD usage
are only increasing.  This is absolutely true.  I agree that--in
general and in principle--this change is in the wrong direction.
However, there are applications that do not use enough SSE to offset
the extra context-switch cost.  SSE does not provide a clear benefit
in the current libthr code with the current compiler, but it does
provide a clear loss in some cases.  Therefore, disabling SSE in
libthr is a non-loss for most, and a gain for some.

I refrained from disabling SSE in libc--as was suggested--because
I can't make the above argument for libc.  It provides a wide variety
of code; each case should be analyzed separately.

https://lists.freebsd.org/pipermail/freebsd-current/2015-March/055193.html

Suggestions from:	dim, jmg, rpaulo
Approved by:	kib (mentor)
MFC after:	2 weeks
Sponsored by:	Dell Inc.
2015-08-05 12:53:55 +00:00
Konstantin Belousov
0c4f9ecde3 Change compiler setting to make default visibility of the symbols for
rtld on x86 to be hidden.  This is a micro-optimization, which allows
intrinsic references inside rtld to be handled without indirection
through PLT.  The visibility of rtld symbols for other objects in the
symbol namespace is controlled by a version script.

Reviewed by:	kan, jilles
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2015-03-29 18:53:21 +00:00
Konstantin Belousov
74b0daf4f9 Optimize r270798, only do the second pass over non-plt relocations
when the first pass found IFUNCs.

Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-08-29 10:43:56 +00:00
Konstantin Belousov
14c3564759 IFUNC symbol type shall be processed for non-PLT relocations,
e.g. when a global variable is initialized with a pointer to ifunc.
Add symbol type check and call resolver for STT_GNU_IFUNC symbol types
when processing non-PLT relocations, but only after non-IFUNC
relocations are done.  The two-phase proceessing is required since
resolvers may reference other symbols, which must be ready to use when
resolver calls are done.

Restructure reloc_non_plt() on x86 to call find_symdef() and handle
IFUNC in single place.

For non-x86 reloc_non_plt(), check for call for IFUNC relocation and
do nothing, to avoid processing relocs twice.

PR:	193048
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-08-29 09:29:10 +00:00
Konstantin Belousov
8cc5663495 Add dwarf annotations to the amd64 _rtld_bind_start to allow debuggers
to unwind around the calls from PLT to binder.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-04-14 22:44:50 +00:00
Konstantin Belousov
f62651920d Add GNU hash support for rtld.
Based on dragonflybsd support for GNU hash by John Marino <draco marino st>
Reviewed by:	kan
Tested by:	bapt
MFC after:	2 weeks
2012-04-30 13:31:10 +00:00
Konstantin Belousov
082f959ac8 Fix several problems with our ELF filters implementation.
Do not relocate twice an object which happens to be needed by loaded
binary (or dso) and some filtee opened due to symbol resolution when
relocating need objects.  Record the state of the relocation
processing in Obj_Entry and short-circuit relocate_objects() if
current object already processed.

Do not call constructors for filtees loaded during the early
relocation processing before image is initialized enough to run
user-provided code.  Filtees are loaded using dlopen_object(), which
normally performs relocation and initialization.  If filtee is
lazy-loaded during the relocation of dso needed by the main object,
dlopen_object() runs too earlier, when most runtime services are not
yet ready.

Postpone the constructors call to the time when main binary and
depended libraries constructors are run, passing the new flag
RTLD_LO_EARLY to dlopen_object().  Symbol lookups callers inform
symlook_* functions about early stage of initialization with
SYMLOOK_EARLY.  Pass flags through all functions participating in
object relocation.

Use the opportunity and fix flags argument to find_symdef() in
arch-specific reloc.c to use proper name SYMLOOK_IN_PLT instead of
true, which happen to have the same numeric value.

Reported and tested by:	theraven
Reviewed by:	kan
MFC after:	2 weeks
2012-03-20 13:20:49 +00:00
Konstantin Belousov
83aa9cc00c Add support for preinit, init and fini arrays. Some ABIs, in
particular on ARM, do require working init arrays.

Traditional FreeBSD crt1 calls _init and _fini of the binary, instead
of allowing runtime linker to arrange the calls.  This was probably
done to have the same crt code serve both statically and dynamically
linked binaries.  Since ABI mandates that first is called preinit
array functions, then init, and then init array functions, the init
have to be called from rtld now.

To provide binary compatibility to old FreeBSD crt1, which calls _init
itself, rtld only calls intializers and finalizers for main binary if
binary has a note indicating that new crt was used for linking.  Add
parsing of ELF notes to rtld, and cache p_osrel value since we parsed
it anyway.

The patch is inspired by init_array support for DragonflyBSD, written
by John Marino.

Reviewed by:	kan
Tested by:	andrew (arm, previous version), flo (sparc64, previous version)
MFC after:	3 weeks
2012-03-11 20:03:09 +00:00
Ed Schouten
581f58e7a3 Remove unneeded dtv variable.
It is only assigned and not used at all. The object files stay identical
when the variables are removed.

Approved by:	kib
2012-01-17 21:55:20 +00:00
Konstantin Belousov
5734c46c68 _rtld_bind() read-locks the bind lock, and possible plt resolution
from the dispatcher would also acquire bind lock in read mode, which
is the supported operation. plt is explicitely designed to allow safe
multithreaded updates, so the shared lock do not cause problems.

The error in r228435 is that it allows read lock acquisition after the
write lock for the bind block.  If we dlopened the shared object that
contains IRELATIVE or jump slot which target is STT_GNU_IFUNC, then
possible recursive plt resolve from the dispatcher would cause it.

Postpone the resolution for irelative/ifunc right before initializers
are called, and drop bind lock around calls to dispatcher.  Use
initlist to iterate over the objects instead of the ->next, due to
drop of the bind lock in iteration.

For i386/reloc.c:reloc_iresolve(), fix calculation of the dispatch
function address for dso, by taking into account possible non-zero
relocbase.

MFC after:	3 weeks
2011-12-14 16:47:53 +00:00
Konstantin Belousov
6be4b69715 Add support for STT_GNU_IFUNC and R_MACHINE_IRELATIVE GNU extensions to
rtld on 386 and amd64. This adds runtime bits neccessary for the use
of the dispatch functions from the dynamically-linked executables and
shared libraries.

To allow use of external references from the dispatch function, resolution
of the R_MACHINE_IRESOLVE relocations in PLT is postponed until GOT entries
for PLT are prepared, and normal resolution of the GOT entries is finished.
Similar to how it is done by GNU, IRELATIVE relocations are resolved in
advance, instead of normal lazy handling for PLT.

Move the init_pltgot() call before the relocations for the object are
processed.

MFC after:	3 weeks
2011-12-12 11:03:14 +00:00
Eitan Adler
36daf0495a - change "is is" to "is" or "it is"
- change "the the" to "the"

Approved by:	lstewart
Approved by:	sahil (mentor)
MFC after:	3 days
2011-10-16 14:30:28 +00:00
Konstantin Belousov
cb38d4941c When loading dso without PT_GNU_STACK phdr, only call
__pthread_map_stacks_exec() on architectures that allow executable
stacks.

Reported and tested by:	marcel (ia64)
2011-01-25 21:12:31 +00:00
Konstantin Belousov
3ad6376e56 Add section .note.GNU-stack for assembly files used by 386 and amd64. 2011-01-07 16:07:05 +00:00
Dimitry Andric
9a17b89ccf Sort -mno-(mmx|3dnow|sse|sse2|sse3) options consistently throughout the
tree.

Submitted by:	arundel
2011-01-05 21:23:26 +00:00
Dimitry Andric
e172464728 On amd64 and i386, tell the compiler to refrain from generating SSE,
3DNow, MMX and floating point instructions in rtld-elf.

Otherwise, _rtld_bind() (and whatever it calls) could possibly clobber
function arguments that are passed in SSE/3DNow/MMX/FP registers,
usually floating point values.  This can happen, for example, when clang
generates SSE code for memset() or memcpy() calls.

One symptom of this is sshd dying early on amd64 with "PRNG not seeded",
which is ultimately caused by libcrypto.so.6 calling RAND_add() with a
double parameter.  That parameter is passed via %xmm0, which gets wiped
out by an SSE memset() in _rtld_bind().

Reviewed by:	kib, kan
2011-01-04 20:51:28 +00:00
Dimitry Andric
7606ddab28 Remove '-elf' from build flags for libexec/rtld-elf for amd64 and i386.
ELF has been the default format for almost 12 years now.
2011-01-04 20:26:41 +00:00
Konstantin Belousov
8569deaf1c Implement support for ELF filters in rtld. Both normal and auxillary
filters are implemented.

Filtees are loaded on demand, unless LD_LOADFLTR environment variable
is set or -z loadfltr was specified during the linking. This forces
rtld to upgrade read-locked rtld_bind_lock to write lock when it
encounters an object with filter during symbol lookup.

Consolidate common arguments of the symbol lookup functions in the
SymLook structure.  Track the state of the rtld locks in the
RtldLockState structure. Pass local RtldLockState through the rtld
symbol lookup calls to allow lock upgrades.

Reviewed by:	kan
Tested by:	Mykola Dzham <i levsha me>, nwhitehorn (powerpc)
2010-12-25 08:51:20 +00:00
Warner Losh
25faff346c MFtbemd:
Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.
2010-08-23 22:24:11 +00:00
Roman Divacky
1dfdc15bb0 Only use the cache after the early stage of loading. This is
because calling mmap() etc. may use GOT which is not set up
yet. Use calloc() instead of mmap() in cases where this
was the case before (sparc64, powerpc, arm).

Submitted by:	Dimitry Andric (dimitry andric com)
Reviewed by:	kan
Approved by:	ed (mentor)
2010-05-18 08:55:23 +00:00
Robert Watson
d1f2f1c3f3 Now that the kernel defines CACHE_LINE_SIZE in machine/param.h, use
that definition in the custom locking code for the run-time linker
rather than local definitions.

Pointed out by:	tinderbox
MFC after:	2 weeks
2009-04-19 23:02:50 +00:00
Dag-Erling Smørgrav
4421d895a9 *thwack*! all the world's not i386.
Pointy hat to:	des
2006-03-29 12:29:01 +00:00
David Xu
c0d2338cdd Allocate space for thread pointer, this allows thread library to access
its pointer from begin, and simplifies _get_curthread() in libthr.
2006-03-28 06:09:24 +00:00
Alexander Kabaev
0eb88f2029 Implement ELF symbol versioning using GNU semantics. This code aims
to be compatible with symbol versioning support as implemented by
GNU libc and documented by http://people.redhat.com/~drepper/symbol-versioning
and LSB 3.0.

Implement dlvsym() function to allow lookups for a specific version of
a given symbol.
2005-12-18 19:43:33 +00:00
Marcel Moolenaar
55dfaa9163 Explicitly cast ELF_R_TYPE() to the right type. 2005-12-18 01:38:26 +00:00
John Baldwin
2939195e46 Remove these unused files before any other archs include the same bogus
file.
2004-11-12 18:05:30 +00:00
Doug Rabson
017246d02f Add support for Thread Local Storage. 2004-08-03 08:51:00 +00:00
Peter Wemm
c707fea10b More stack alignment fixes. Arrange so we call _rtld() in ld-elf.so.1
with the correct alignment.  This is important because this calls to
library static constructors are made from here.  The bug in the old crt*.s
files hid this because in this case, two wrongs do indeed make a right.
Also, call _rtld_bind() with the correct alignment, because it calls back
into the pthread library locking functions.  If things happen just
the wrong way, we get a SIG10 due to the broken stack alignment.
2004-03-21 01:43:39 +00:00
Peter Wemm
6143d8ba5f Fix dynamic linking a bit more.. enough that mozilla-firebird works if you
dig up the patches for amd64 support for it.

Note to self: do not put a 64 bit value in a 32 bit space.
2003-12-12 01:12:41 +00:00
Peter Wemm
080f5381b7 Revert last change. ../rtld.c uses CACHE_LINE_SIZE too.
Change it to 64 while here.

Reported by:  ps
2003-12-11 18:42:51 +00:00
Peter Wemm
165d50f626 Only define CACHE_LINE_SIZE in one place.. 2003-12-11 04:49:37 +00:00
Peter Wemm
40a7c81112 CACHE_LINE_SIZE is 64 on athlon and amd64 chips, not 32. This should
probably be 128 since that is what the hardware prefetch fill size is
on both the p3, p4 and athlon* cpus.
2003-12-11 04:47:53 +00:00
Alexander Kabaev
6d5d786f80 Allow threading libraries to register their own locking
implementation in case default one provided by rtld is
not suitable.

Consolidate various identical MD lock implementation into
a single file using appropriate machine/atomic.h.

Approved by:	re (scottl)
2003-05-29 22:58:26 +00:00