Fix error in extent_try_coalesce_impl(), which could cause abort
to happen when trying to coalesce extents backwards. The error could
happen because of how extent_before_get() function works. This function
gets address of previous extent, by subtracting page size from current
extent address. If current extent is located at PAGE_SIZE offset, this
address resolved to 0x0000. An assertion in rtree_leaf_elm_lookup
then caused the running program to abort.
This problem was discovered when trying to build world on 32-bit
machines with ASLR and PIE enabled. The problem was encountered
on armv7 and i386 machines, but most likely other 32-bit
architectures are affected as well.
While this patch fixes one problem with buildworld on 32-bit platforms
with ASLR, the build still fails, however it happens much later
and due to lack of memory.
The change is aligned with accepted fix in the upstream Jemalloc
repository (https://github.com/jemalloc/jemalloc/pull/1973).
As it doesn't apply on top of Jemalloc tree, its updated version
was eventually merged: https://github.com/jemalloc/jemalloc/pull/2003
PR: 249937
Submitted by: Dawid Gorecki <dgr@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D27025
For historical reasons, defining MALLOC_PRODUCTION in /etc/make.conf has
been used to turn off potentially expensive debug checks and statistics
gathering in the implementation of malloc(3).
It seems more consistent to turn this into a regular src.conf(5) option,
e.g. WITH_MALLOC_PRODUCTION / WITHOUT_MALLOC_PRODUCTION. This can then
be toggled similar to any other source build option, and turned on or
off by default for e.g. stable branches.
Reviewed by: imp, #manpages
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26337
The Sv48 PTE format is the largest currently defined address space for
RISC-V. It makes no sense to define a larger size and doing so (at
least for 64-bits) forces rtrees down a slow path.
Reviewed by: vangyzen, jhb, mhorne
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24658
Revert r354606 to restore r354605.
Apply one line from jemalloc commit d01b425e5d1e1 in hash_x86_128()
to fix the build with gcc, which only allows a fallthrough attribute
to appear before a case or default label.
Submitted by: jasone in r354605
Discussed with: jasone
Reviewed by: bdrewery
MFC after: never, due to gcc 4.2.1
Relnotes: yes
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D24522
like it already does on Linux and OSX. This results in significantly
fewer calls to mmap(2). This should result in a small reduction
in system CPU time and improved superpage usage.
Reviewed by: markj
Tested by: markj
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23874
Pick 2d6d099fed05b1509e81e54458516528bfbbf38d from upstream jemalloc:
Fix GCC-9.1 warning with macro GET_ARG_NUMERIC
GCC-9.1 reports following error when trying to compile file
src/malloc_io.c and with CFLAGS='-Werror' :
src/malloc_io.c: In function ‘malloc_vsnprintf’:
src/malloc_io.c:369:2: error: case label value exceeds maximum value for type [-Werror]
369 | case '?' | 0x80: \
| ^~~~
src/malloc_io.c:581:5: note: in expansion of macro ‘GET_ARG_NUMERIC’
581 | GET_ARG_NUMERIC(val, 'p');
| ^~~~~~~~~~~~~~~
...
<snip>
cc1: all warnings being treated as errors
make: *** [Makefile:388: src/malloc_io.sym.o] Error 1
The warning is reported as by default the type 'char' is 'signed char'
and or-ing 0x80 will turn the case label char negative which will be
beyond the printable ascii range (0 - 127).
The patch fixes this by explicitly casting the 'len' variable as
unsigned char' inside the 'switch' statement so that value of
expression " '?' | 0x80 " falls within the legal values of the
variable 'len'.
Discussed with: jasone (maintainer)
Sponsored by: Dell EMC Isilon
Avoid touching all pages in extent_recycle for debug build.
We may have a large number of pages with *zero set (since they are populated on
demand). Only check the first page to avoid paging in all of them.
This makes it easy to compare performance with and without 'retain:true'.
Discussed with: jasone
Obtained from: Qi Wang <interwq at gwu dot edu>
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Set commit properly for FreeBSD w/ overcommit.
When overcommit is enabled, commit needs to be set when doing mmap(). The
regression was introduced in f80c97e.
This fixes 'retain:true'.
Discussed with: jasone
Obtained from: Qi Wang <interwq at gwu dot edu>
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Rework the way jemalloc uses mmap(2) on FreeBSD.
This makes it directly use MAP_EXCL and MAP_ALIGNED() instead
of weird workarounds involving mapping at random places and then
unmapping parts of them.
Discussed with: jasone
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Disable runtime detection of lazy purging support on FreeBSD.
The check doesn't seem to serve any purpose here, and this shaves
off three syscalls on binary startup.
Discussed by: jasone
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
The problem has been reported to upstream and similar change will
be included in next jemalloc release.
Submitted by: David Goldblatt <davidtgoldblatt@gmail.com>
MFC after: 2 weeks
The problem has been reported to upstream and similar change will
be included in next jemalloc release.
Submitted by: David Goldblatt <davidtgoldblatt@gmail.com>
MFC after: 2 weeks
linked with AddressSanitizer (even those linked on earlier versions of
FreeBSD, or with external versions of clang) started failing with errors
similar to:
==14688==AddressSanitizer CHECK failed:
/usr/src/contrib/compiler-rt/lib/asan/asan_poisoning.cc:36
"((AddrIsAlignedByGranularity(addr))) != (0)" (0x0, 0x0)
This is because AddressSanitizer expects all the TLS data in the program
to be aligned to at least 8 bytes.
Before the jemalloc 5.0.0 update, all the TLS data in the i386 version
of libc.so added up to 80 bytes (a multiple of 8), but 5.0.0 made this
grow to 2404 bytes (not a multiple of 8). This is due to added caching
data in jemalloc's internal struct tsd_s.
To fix AddressSanitizer, ensure this struct is aligned to at least 16
bytes, which can be done unconditionally for all architectures. (An
earlier version of the fix aligned the struct to 8 bytes, but only for
ILP32 architectures. This was deemed unnecessarily complicated.)
PR: 221337
X-MFC-With: r319971
Apply the changes from upstream jemalloc 048c6679. This is actually not
quite a cherry pick due to makefile difference and because FreeBSD does
not carry the msvc project files which were also modified in that
commit.
Approved by: jasone (maintainer), markj (mentor)
Sponsored by: Dell EMC Isilon
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)
This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.
RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):
__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen
Reviewed by: ngie
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D11901
Add missing Symbol.map entry for __aligned_alloc.
Add weak-->strong symbol binding for
{malloc_stats_print,mallctl,mallctlnametomib,mallctlbymib} -->
{__malloc_stats_print,__mallctl,__mallctlnametomib,__mallctlbymib}. These
bindings complete the set necessary to allow applications to replace all
malloc-related symbols.
- Use _Bool rather than bool to resolve missing type errors in malloc_np.h.
- Fix malloc manual page #include documentation.
- Add *allocm manual pages to obsolete files.
Submitted by: jbeich
(or loading a dso linked to libthr.so into process which was not
linked against threading library).
- Remove libthr interposers of the libc functions, including
__error(). Instead, functions calls are indirected through the
interposing table, similar to how pthread stubs in libc are already
done. Libc by default points either to syscall trampolines or to
existing libc implementations. On libthr load, libthr rewrites the
pointers to the cancellable implementations already in libthr. The
interposition table is separate from pthreads stubs indirection
table to not pull pthreads stubs into static binaries.
- Postpone the malloc(3) internal mutexes initialization until libthr
is loaded. This avoids recursion between calloc(3) and static
pthread_mutex_t initialization.
- Reinstall signal handlers with wrapper on libthr load. The
_rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2)
when libthr is statically referenced from the main binary.
In the process, fix openat(2), swapcontext(2) and setcontext(2)
interposing. The libc symbols were exported at different versions
than libthr interposers. Export both libc and libthr versions from
libc now, with default set to the higher version from libthr.
Remove unused and disconnected swapcontext(3) userspace implementation
from libc/gen.
No objections from: deischen
Tested by: pho, antoine (exp-run) (previous versions)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
