Commit Graph

56 Commits

Author SHA1 Message Date
Edward Tomasz Napierala
d5aef6d6ca Pick 57553c3b1a5592dc4c03f3c6831d9b794e523865 from upstream:
Avoid touching all pages in extent_recycle for debug build.

    We may have a large number of pages with *zero set (since they are populated on
    demand).  Only check the first page to avoid paging in all of them.

This makes it easy to compare performance with and without 'retain:true'.

Discussed with:	jasone
Obtained from:	Qi Wang <interwq at gwu dot edu>
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-11-14 13:06:48 +00:00
Edward Tomasz Napierala
086165ecb5 Pick 50b473c8839f5408df179bdf6f2b3fd2cf5c3b2f from upstream:
Set commit properly for FreeBSD w/ overcommit.

    When overcommit is enabled, commit needs to be set when doing mmap().  The
    regression was introduced in f80c97e.

This fixes 'retain:true'.

Discussed with:	jasone
Obtained from:	Qi Wang <interwq at gwu dot edu>
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-11-06 12:05:46 +00:00
Edward Tomasz Napierala
0c885e274a Pick f80c97e477d1b3fe7778c65d9439d673738b4131 from upstream:
Rework the way jemalloc uses mmap(2) on FreeBSD.

    This makes it directly use MAP_EXCL and MAP_ALIGNED() instead
    of weird workarounds involving mapping at random places and then
    unmapping parts of them.

Discussed with:	jasone
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-10-23 14:11:35 +00:00
Edward Tomasz Napierala
bff19560ee Pick 676cdd66792ccb629a978837ea2a066d5db342cc from upstream:
Disable runtime detection of lazy purging support on FreeBSD.

    The check doesn't seem to serve any purpose here, and this shaves
    off three syscalls on binary startup.

Discussed by:	jasone
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-10-23 13:54:54 +00:00
Jason Evans
0ef50b4ec8 Update jemalloc to version 5.1.0. 2018-05-11 00:32:31 +00:00
Marius Strobl
c2a2eeff5c Account for the fact that jemalloc 5.0.0 dropped STATIC_PAGE_SHIFT
in favor for using LG_PAGE directly and, thus, for the fact that
host and target don't necessarily use pages of the same sizes.

Approved by:	jasone
2018-01-31 21:56:23 +00:00
Michal Meloun
d65a63e3aa Relax too restrictive assert.
The problem has been reported to upstream and similar change will
be included in next jemalloc release.

Submitted by:	David Goldblatt <davidtgoldblatt@gmail.com>
MFC after: 	2 weeks
2017-12-10 09:17:21 +00:00
Michal Meloun
e7dad90fe5 Revert r326740. I committed wrong diff. 2017-12-10 09:15:37 +00:00
Michal Meloun
3ee2fe0199 Relax too restrictive assert.
The problem has been reported to upstream and similar change will
be included in next jemalloc release.

Submitted by:	David Goldblatt <davidtgoldblatt@gmail.com>
MFC after: 	2 weeks
2017-12-10 08:55:48 +00:00
Dimitry Andric
cc718496a9 After jemalloc was updated to version 5.0.0 in r319971, i386 executables
linked with AddressSanitizer (even those linked on earlier versions of
FreeBSD, or with external versions of clang) started failing with errors
similar to:

  ==14688==AddressSanitizer CHECK failed:
  /usr/src/contrib/compiler-rt/lib/asan/asan_poisoning.cc:36
  "((AddrIsAlignedByGranularity(addr))) != (0)" (0x0, 0x0)

This is because AddressSanitizer expects all the TLS data in the program
to be aligned to at least 8 bytes.

Before the jemalloc 5.0.0 update, all the TLS data in the i386 version
of libc.so added up to 80 bytes (a multiple of 8), but 5.0.0 made this
grow to 2404 bytes (not a multiple of 8).  This is due to added caching
data in jemalloc's internal struct tsd_s.

To fix AddressSanitizer, ensure this struct is aligned to at least 16
bytes, which can be done unconditionally for all architectures.  (An
earlier version of the fix aligned the struct to 8 bytes, but only for
ILP32 architectures.  This was deemed unnecessarily complicated.)

PR:		221337
X-MFC-With:	r319971
2017-10-23 21:31:04 +00:00
Ryan Libby
20100e740e Pick 'Remove external linkage for spin_adaptive' from upstream jemalloc
Apply the changes from upstream jemalloc 048c6679.  This is actually not
quite a cherry pick due to makefile difference and because FreeBSD does
not carry the msvc project files which were also modified in that
commit.

Approved by:	jasone (maintainer), markj (mentor)
Sponsored by:	Dell EMC Isilon
2017-08-09 22:58:42 +00:00
Ruslan Bukin
ca20f8ec29 o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by:	ngie
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D11901
2017-08-07 14:09:57 +00:00
Jason Evans
8b2f5aaf29 Update jemalloc to 5.0.1. 2017-07-03 23:27:57 +00:00
Jason Evans
1a78ec8941 Default the abort_conf malloc option to false.
This avoids troublesome backward compatibility issues.
2017-06-30 17:45:51 +00:00
Jason Evans
b7eaed250f Update jemalloc to 5.0.0. 2017-06-15 07:15:05 +00:00
Jason Evans
8244f2aa7c Update jemalloc to 4.5.0. 2017-03-02 01:14:48 +00:00
Jason Evans
7fa7f12ff8 Update jemalloc to 4.4.0. 2016-12-04 21:13:26 +00:00
Jason Evans
bde951447f Update jemalloc to 4.3.1. 2016-11-09 18:42:30 +00:00
Jason Evans
62b2691e48 Update jemalloc to 4.2.1. 2016-06-09 06:10:20 +00:00
Jason Evans
fbb1d85eb3 Work around invalid gcc warning (explicit cast apparently lost). 2016-05-13 21:18:10 +00:00
Jason Evans
1f0a49e863 Update jemalloc to 4.2.0. 2016-05-13 04:03:20 +00:00
Andrew Turner
4b6f23dfaf Stop using sbrk(2) with malloc. This helps reduce the number of places
within the tree where we call this legacy interface.

Reviewed by:	jasone
Obtained from:	brooks
Sponsored by:	ABT Systems Ltd
2016-04-28 12:24:58 +00:00
Jason Evans
cbc3697dda Restore support for decay time of -1 (no decay). 2016-03-03 01:43:36 +00:00
Jason Evans
cb2ababcfd Add a cast to prevent a compiler warning. 2016-03-03 01:30:28 +00:00
Ruslan Bukin
ac8f9de31a Restore configuration for RISC-V. 2016-03-01 17:43:06 +00:00
Jason Evans
df0d881d94 Update jemalloc to 4.1.0.
Add missing Symbol.map entry for __aligned_alloc.

Add weak-->strong symbol binding for
{malloc_stats_print,mallctl,mallctlnametomib,mallctlbymib} -->
{__malloc_stats_print,__mallctl,__mallctlnametomib,__mallctlbymib}.  These
bindings complete the set necessary to allow applications to replace all
malloc-related symbols.
2016-02-29 19:10:32 +00:00
Ruslan Bukin
47f9f6eb3a Add configuration for RISC-V ISA.
Reviewed by:	emaste
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D5020
2016-01-22 16:37:26 +00:00
Jason Evans
ba4f5cc0a6 Update jemalloc to version 4.0.4. 2015-10-24 23:18:05 +00:00
Jason Evans
536b3538e0 Update jemalloc to 4.0.2. 2015-09-22 03:02:18 +00:00
Jason Evans
c13244b92e Fix minor malloc regressions.
- Use _Bool rather than bool to resolve missing type errors in malloc_np.h.
- Fix malloc manual page #include documentation.
- Add *allocm manual pages to obsolete files.

Submitted by:	jbeich
2015-08-19 00:06:46 +00:00
Jason Evans
337776f858 Define CPU_SPINWAIT as cpu_spinwait().
Submitted by:	cem
2015-08-18 20:42:08 +00:00
Jason Evans
d8e39d2d4f Re-add LG_SIZEOF_PTR definition for __aarch64__.
This definition was erroneously removed during the 4.0.0 import.
2015-08-18 09:09:27 +00:00
Jason Evans
a4cee496bc Fix build failure due to missing CPU_SPINWAIT definition. 2015-08-18 08:10:46 +00:00
Jason Evans
8d13fecf5f Add jemalloc support for 8 KiB, 16 KiB, and 64 KiB pages. 2015-08-18 06:28:37 +00:00
Jason Evans
d0e79aa362 Update jemalloc to version 4.0.0. 2015-08-18 00:21:25 +00:00
Ed Maste
552f44af8b Add aarch64 (arm64) #define for jemalloc
Sponsored by:	The FreeBSD Foundation
2015-03-12 08:52:00 +00:00
Konstantin Belousov
8495e8b1e9 Fix known issues which blow up the process after dlopen("libthr.so")
(or loading a dso linked to libthr.so into process which was not
linked against threading library).

- Remove libthr interposers of the libc functions, including
  __error(). Instead, functions calls are indirected through the
  interposing table, similar to how pthread stubs in libc are already
  done.  Libc by default points either to syscall trampolines or to
  existing libc implementations.  On libthr load, libthr rewrites the
  pointers to the cancellable implementations already in libthr.  The
  interposition table is separate from pthreads stubs indirection
  table to not pull pthreads stubs into static binaries.

- Postpone the malloc(3) internal mutexes initialization until libthr
  is loaded.  This avoids recursion between calloc(3) and static
  pthread_mutex_t initialization.

- Reinstall signal handlers with wrapper on libthr load.  The
  _rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2)
  when libthr is statically referenced from the main binary.

In the process, fix openat(2), swapcontext(2) and setcontext(2)
interposing.  The libc symbols were exported at different versions
than libthr interposers.  Export both libc and libthr versions from
libc now, with default set to the higher version from libthr.

Remove unused and disconnected swapcontext(3) userspace implementation
from libc/gen.

No objections from:	deischen
Tested by:	pho, antoine (exp-run) (previous versions)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-01-03 18:38:46 +00:00
Jason Evans
2fff27f84c Update jemalloc to version 3.6.0. 2014-03-31 17:04:04 +00:00
Jason Evans
706d9bd159 Update jemalloc to version 3.5.1. 2014-02-26 02:36:59 +00:00
Jason Evans
f921d10f48 Update jemalloc to version 3.5.0. 2014-01-23 02:47:36 +00:00
Jason Evans
2b06b2013c Update jemalloc to version 3.4.1. 2013-10-21 05:10:46 +00:00
Jason Evans
f8ca2db1f8 Update jemalloc to version 3.4.0. 2013-06-03 14:36:28 +00:00
Marcel Moolenaar
9dfba391d6 Make the malloc(3) family of functions weak and make their non-weak
implementations visible for use by applications. The functions $F that
are now weak symbols are:
	allocm, calloc, dallocm, free, malloc, malloc_usable_size,
	nallocm, posix_memalign, rallocm, realloc, sallocm

The non-weak implementations of $F are exported as __$F.

Submitted by:	stevek@juniper.net
Reviewed by:	jasone@, kib@
Approved by:	jasone@ (jemalloc)
Obtained from:	juniper Networks, Inc
2013-05-25 18:59:11 +00:00
Jason Evans
70fabd840d Update list of ports required for importing jemalloc. 2013-01-24 03:33:15 +00:00
Jason Evans
88ad2f8dcc Import jemalloc 3.3.0. This reduces zeroed memory validation overhead for
non-MALLOC_PRODUCTION builds.
2013-01-24 03:08:37 +00:00
Jason Evans
82872ac086 Import jemalloc 3.2.0. 2012-11-10 01:46:13 +00:00
Dimitry Andric
5948d40700 Since our clang now supports the tls_model attribute, remove the
workaround for it in jemalloc_FreeBSD.h.

Reviewed by:	jasone
2012-08-23 17:58:22 +00:00
Jason Evans
edaa25bdae Fix config_lazy_lock so that thread caching isn't used for
single-threaded applications.
2012-05-13 01:33:28 +00:00
Jason Evans
35dad07381 Import jemalloc 3.0.0. This fixes memory zeroing bugs that manifested as
jemalloc assertion failures for debug builds, or as calloc() sometimes
returning non-zeroed memory for production builds.
2012-05-12 05:26:49 +00:00
Jason Evans
e722f8f8ac Import jemalloc 37b6f95dcd866f51c91488531a2efc3ed4c2b754 (dev branch,
prior to 3.0.0 release).  This version is likely very close to what will be
3.0.0.
2012-05-10 18:29:40 +00:00