load of _l suffixed versions of various standard library functions that use
the global locale, making them take an explicit locale parameter. Also
adds support for per-thread locales. This work was funded by the FreeBSD
Foundation.
Please test any code you have that uses the C standard locale functions!
Reviewed by: das (gdtoa changes)
Approved by: dim (mentor)
The size passed to strlcat() must depend on the input length, not the
output length. Because the input and output buffers are equal in size,
the resulting binary does not change at all.
-g, by reverting r219139. The LLVM PR referenced in that revision was
fixed in the mean time, and we imported a clang snapshot soon
afterwards, so the temporary workaround of disabling clang's integrated
assembler is no longer needed.
In this particular case, using e.g. DEBUG_FLAGS=-g causes clang to
output certain directives into assembly that our version of GNU as
chokes on.
Reported by: dougb
Approved by: re (kib)
support for it. Note that while sparc64 also supports the static TLS
model and thus tls_model("initial-exec"), using the default model
turned out to yield slightly better buildstone performance.
kernel.debug (or possibly other files), when WITH_CTF is active.
This is caused by a bug in clang's integrated assembler, causing malloc
to sometimes hang during initialization in statically linked executables
that use threading, such as the copy of ctfmerge that is built during
the bootstrap stage of buildworld. The bug has been submitted upstream:
http://llvm.org/bugs/show_bug.cgi?id=9352
Note that you might have to rebuild and install libc first, to get your
kernel build to finish, because the ctfmerge binary built during
bootstrap is linked with your base system's copy of libc.a, which might
already contain a bad copy of malloc.o.
add a wrapper for it in libc and rework the code in libthr, the
system call still can return EINTR, we keep this feature.
Discussed on: thread
Reviewed by: jilles
their implementations aren't in the same files. Introduce LIBC_ARCH
and use that in preference to MACHINE_CPUARCH. Tested by amd64 and
powerpc64 builds (thanks nathanw@)
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.
Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.
Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.
The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.
Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks
number of host CPUs and osreldate.
This eliminates the last sysctl(2) calls from the dynamically linked image
startup.
No objections from: kan
Tested by: marius (sparc64)
MFC after: 1 month
finished using it. This allows the mutex's allocated memory to be
freed.
This is one sense a rather silly change, since at this point we're
less than a microsecond away from calling _exit; but fixing this
memory leak is likely to make life easier for anyone trying to
track down other memory leaks.
bottom of the manpages and order them consistently.
GNU groff doesn't care about the ordering, and doesn't even mention
CAVEATS and SECURITY CONSIDERATIONS as common sections and where to put
them.
Found by: mdocml lint run
Reviewed by: ru
SUSv4 requires that implementation returns EINVAL if supplied path is NULL,
and ENOENT if path is empty string [1].
Bring prototype in conformance with SUSv4, adding restrict keywords.
Allow the resolved path buffer pointer be NULL, in which case realpath(3)
allocates storage with malloc().
PR: kern/121897 [1]
MFC after: 2 weeks
Although groff_mdoc(7) gives another impression, this is the ordering
most widely used and also required by mdocml/mandoc.
Reviewed by: ru
Approved by: philip, ed (mentors)
the static TLS model, which is fundamentally different from the dynamic
TLS model. The consequence was data corruption. Limit the attribute to
i386 and amd64.
http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/143350
Empty string test gone wrong.
Testing this requires that you have a locale that has the sign string
unset but has int_n_sign_posn set (the default locale falls through to
use "()" around negative numbers which is probably another bug).
I created that setup by hand and indeed without this fix negative
numbers are put out as positive numbers (doesn't fall through to use
"-" as default indicator).
Unfixed example in nl_NL.ISO8859-1 with lc->negative_sign set to empty
string:
strfmon(buf, sizeof(buf), "%-8i", -42.0);
==>
example2: 'EUR 42,00' 'Eu 42,00'
Fixed:
example2: 'EUR 42,00-' 'Eu 42,00-'
This file and suggested fix are identical in at least freebsd-8.
Backport might be appropriate but some expert on locales should
probably have a look at us defaulting to negative numbers in
parenthesis when LC_* is default. That doesn't look right and is not
what other OSes are doing.
PR: 143350
Submitted by: Corinna Vinschen
Reviewed by: bug reporter submitted, tested by me
* Fix a race in chunk_dealloc_dss().
* Check for allocation failure before zeroing memory in base_calloc().
Merge enhancements from a divergent version of jemalloc:
* Convert thread-specific caching from magazines to an algorithm that is
more tunable, and implement incremental GC.
* Add support for medium size classes, [4KiB..32KiB], 2KiB apart by
default.
* Add dirty page tracking for pages within active small/medium object
runs. This allows malloc to track precisely which pages are in active
use, which makes dirty page purging more effective.
* Base maximum dirty page count on proportion of active memory.
* Use optional zeroing in arena_chunk_alloc() to avoid needless zeroing
of chunks. This is useful in the context of DSS allocation, since a
long-lived application may commonly recycle chunks.
* Increase the default chunk size from 1MiB to 4MiB.
Remove feature:
* Remove the dynamic rebalancing code, since thread caching reduces its
utility.
of setenv(), putenv() and unsetenv() when dealing with corrupt entries in
environ. They now output a warning and complete their task without error.
MFC after: 1 week
instead of returning an error if a corrupt (not a "name=value" string) entry
in the environ array is detected when (re)-building the internal
environment. This should prevent applications or libraries from
experiencing issues arising from the expectation that these calls will
complete even with corrupt entries. The behavior is now as it was prior to
7.0.
Reviewed by: jilles
MFC after: 1 week
find a variable. Include a note that it must not cause the internal
environment to be generated since malloc() depends upon getenv(). To call
malloc() would create a circular dependency.
Recommended by: green
Approved by: jilles
MFC after: 1 week
**environ entries. This puts non-getenv(3) operations in line with
getenv(3) in that bad environ entries do not cause all operations to
fail. There is still some inconsistency in that getenv(3) in the
absence of any environment-modifying operation does not emit corrupt
environ entry warnings.
I also fixed another inconsistency in getenv(3) where updating the
global environ pointer would not be reflected in the return values.
It would have taken an intermediary setenv(3)/putenv(3)/unsetenv(3)
in order to see the change.
a large page size that is greater than malloc(3)'s default chunk size but
less than or equal to 4 MB, then increase the chunk size to match the large
page size.
Most often, using a chunk size that is less than the large page size is not
a problem. However, consider a long-running application that allocates and
frees significant amounts of memory. In particular, it frees enough memory
at times that some of that memory is munmap()ed. Up until the first
munmap(), a 1MB chunk size is just fine; it's not a problem for the virtual
memory system. Two adjacent 1MB chunks that are aligned on a 2MB boundary
will be promoted automatically to a superpage even though they were
allocated at different times. The trouble begins with the munmap(),
releasing a 1MB chunk will trigger the demotion of the containing superpage,
leaving behind a half-used 2MB reservation. Now comes the real problem.
Unfortunately, when the application needs to allocate more memory, and it
recycles the previously munmap()ed address range, the implementation of
mmap() won't be able to reuse the reservation. Basically, the coalescing
rules in the virtual memory system don't allow this new range to combine
with its neighbor. The effect being that superpage promotion will not
reoccur for this range of addresses until both 1MB chunks are freed at some
point in the future.
Reviewed by: jasone
MFC after: 3 weeks
When I wrote the pseudo-terminal driver for the MPSAFE TTY code, Robert
Watson and I agreed the best way to implement this, would be to let
posix_openpt() create a pseudo-terminal with proper permissions in place
and let grantpt() and unlockpt() be no-ops.
This isn't valid behaviour when looking at the spec. Because I thought
it was an elegant solution, I filed a bug report at the Austin Group
about this. In their last teleconference, they agreed on this subject.
This means that future revisions of POSIX may allow grantpt() and
unlockpt() to be no-ops if an open() on /dev/ptmx (if the implementation
has such a device) and posix_openpt() already do the right thing.
I'd rather put this in the manpage, because simply mentioning we don't
comply to any standard makes it look worse than it is. Right now we
don't, but at least we took care of it.
Approved by: re (kib)
MFC after: 3 days
A more elegant way of obtaining a name of a character device by its file
descriptor on FreeBSD, is to use the FIODGNAME ioctl. Because a valid
file descriptor implies a file descriptor is visible in /dev, it will
always resolve a valid device name.
I'm adding a more friendly wrapper for this ioctl, called fdevname(). It
is a lot easier to use than devname() and also has better error
handling. When a device name cannot be resolved, it will just return
NULL instead of a generated device name that makes no sense.
Discussed with: kib
stating that in FreeBSD the atol() and atoll() functions affect
errno in the same way as strtol() and stroll().
PR: docs/126487
Submitted by: edwin
Reviewed by: trhodes, gabor
MFC after: 1 week
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.
Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.
Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes. The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.
Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.
Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes. Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.
Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.
The routines in grantpt.c have been moved to ptsname.c in the MPSAFE TTY
layer, because grantpt() is now effectively a no-op. I forgot to remove
the corresponding source file from libc.
The last half year I've been working on a replacement TTY layer for the
FreeBSD kernel. The new TTY layer was designed to improve the following:
- Improved driver model:
The old TTY layer has a driver model that is not abstract enough to
make it friendly to use. A good example is the output path, where the
device drivers directly access the output buffers. This means that an
in-kernel PPP implementation must always convert network buffers into
TTY buffers.
If a PPP implementation would be built on top of the new TTY layer
(still needs a hooks layer, though), it would allow the PPP
implementation to directly hand the data to the TTY driver.
- Improved hotplugging:
With the old TTY layer, it isn't entirely safe to destroy TTY's from
the system. This implementation has a two-step destructing design,
where the driver first abandons the TTY. After all threads have left
the TTY, the TTY layer calls a routine in the driver, which can be
used to free resources (unit numbers, etc).
The pts(4) driver also implements this feature, which means
posix_openpt() will now return PTY's that are created on the fly.
- Improved performance:
One of the major improvements is the per-TTY mutex, which is expected
to improve scalability when compared to the old Giant locking.
Another change is the unbuffered copying to userspace, which is both
used on TTY device nodes and PTY masters.
Upgrading should be quite straightforward. Unlike previous versions,
existing kernel configuration files do not need to be changed, except
when they reference device drivers that are listed in UPDATING.
Obtained from: //depot/projects/mpsafetty/...
Approved by: philip (ex-mentor)
Discussed: on the lists, at BSDCan, at the DevSummit
Sponsored by: Snow B.V., the Netherlands
dcons(4) fixed by: kan
detect whether the integer division table is large enough to handle the
divisor. Before this change, the last two table elements were never used,
thus causing the slow path to be used for those divisors.
environ[0] to be more obvious that environ is not NULL before environ[0]
is tested. Although I believe the previous code worked, this change
improves code maintainability.
Reviewed by: ache
MFC after: 3 days
the first value (environ[0]) to NULL. This is in addition to the
current detection of environ being replaced, which includes being set to
NULL. Without this fix, the environment is not truly wiped, but appears
to be by getenv() until an *env() call is made to alter the enviroment.
This change is necessary to support those applications that use this
method for clearing environ such as Dovecot and Postfix. Applications
such as Sendmail and the base system's env replace environ (already
detected). While neither of these methods are defined by SUSv3, it is
best to support them due to historic reasons and in lieu of a clean,
defined method.
Add extra units tests for clearing environ using four different methods:
1. Set environ to NULL pointer.
2. Set environ[0] to NULL pointer.
3. Set environ to calloc()'d NULL-terminated array.
4. Set environ to static NULL-terminated array.
Noticed by: Timo Sirainen
MFC after: 3 days
the chunk map instead of red-black trees where possible. Remove the
red-black trees and node objects that are obsoleted by this change. The
net result is a ~1-2% memory savings, and a substantial allocation speed
improvement.
statement. Add the one from the current NetBSD version.
- Also bump a date to reflect my content changes I have done in previous
revision
Approved by: imp
MFC after: 3 days
The __use_pts() routine was once probably used by libutil to determine
if we are using BSD or UNIX98 style PTY device names. It doesn't seem to
be used outside grantpt.c, which means we can make it static and remove
it from the Symbol.map.
Reviewed by: cognet, kib
Approved by: philip (mentor)
This substantially improves worst case allocation performance, since
O(lg n) tree search can be used instead of O(n) tree iteration.
Use rb_wrap() instead of directly calling rb_*() macros.
macros.
Add rb_foreach_next() and rb_foreach_reverse_prev(), which make it
possible to re-synchronize tree iteration after the tree has been
modified.
Rename rb_tree_new() to rb_new().
color bit in the least significant bit of the right child pointer, in
order to reduce red-black tree linkage overhead by ~2X as compared to
sys/tree.h.
Use the new red-black tree implementation in malloc, which drops
memory usage by ~0.5 or ~1%, for 32- and 64-bit systems, respectively.
There were no checks for left and right precisions at all, and
a check for field width had integer overflow bug.
Reported by: Maksymilian Arciemowicz
Security: http://securityreason.com/achievement_securityalert/53
Submitted by: Maxim Dounin <mdounin@mdounin.ru>
MFC after: 3 days
This reduces the size of a statically-linked binary by approximately 100KB
in a trivial "return (0)" test application. readelf -S was used to verify
that the .text section was reduced and that using strlen() saved a few
more bytes over using sizeof(). Since the section of code is only called
when environ is corrupt (program bug), I went with fewer bytes over fewer
cycles.
I made minor edits to the submitted patch to make the output resemble
warnx().
Submitted by: kib bz
Approved by: wes (mentor)
MFC after: 5 days
reallocation, when junk filling is enabled. Junk filling must occur
prior to shrinking, since any deallocated trailing pages are immediately
available for use by other threads.
Reported by: Mats Palmgren <mats.palmgren@bredband.net>
allocation patterns, number of CPUs, and MALLOC_OPTIONS settings indicate
that lazy deallocation has the potential to worsen throughput dramatically.
Performance degradation occurs when multiple threads try to clear the lazy
free cache simultaneously. Various experiments to avoid this bottleneck
failed to completely solve this problem, while adding yet more complexity.
