Implement a small enhancement to the original qsort implementation:
If the data is 32 bit aligned we can side-step the long type
version and use int instead.
The change brings a modest but significant improvement in
32 bit workloads.
Relnotes: yes
PR: 135718
Taken from: ache
any applications which need unpredictable random numbers, not merely those
which are cryptographic in nature.
If you work for a lottery and you're using random(3) to select the winning
numbers, please let me know.
(or loading a dso linked to libthr.so into process which was not
linked against threading library).
- Remove libthr interposers of the libc functions, including
__error(). Instead, functions calls are indirected through the
interposing table, similar to how pthread stubs in libc are already
done. Libc by default points either to syscall trampolines or to
existing libc implementations. On libthr load, libthr rewrites the
pointers to the cancellable implementations already in libthr. The
interposition table is separate from pthreads stubs indirection
table to not pull pthreads stubs into static binaries.
- Postpone the malloc(3) internal mutexes initialization until libthr
is loaded. This avoids recursion between calloc(3) and static
pthread_mutex_t initialization.
- Reinstall signal handlers with wrapper on libthr load. The
_rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2)
when libthr is statically referenced from the main binary.
In the process, fix openat(2), swapcontext(2) and setcontext(2)
interposing. The libc symbols were exported at different versions
than libthr interposers. Export both libc and libthr versions from
libc now, with default set to the higher version from libthr.
Remove unused and disconnected swapcontext(3) userspace implementation
from libc/gen.
No objections from: deischen
Tested by: pho, antoine (exp-run) (previous versions)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
bsearch_b is the Apple blocks enabled version of bsearch(3).
This was added to libc in Revision 264042 but the commit
missed the declaration required to make use of it.
While here move some other block-related functions to the
BSD_VISIBLE block as these are non-standard.
Phabric: D638
Reviewed by: theraven, wollman
The hcreate(3) implementation and related functions we inherited
from NetBSD used to free() the key value, something that is not
supported by the standard implementation.
This would cause a segmentation fault when attempting to run
the examples from the opengroup and linux manpages. NetBSD
has added non-standard calls to provide the previous
behaviour but hdestroy is not very commonly used so at this
time it seems excessive to bring those to FreeBSD.
Bump the __FreeBSD_version as this is an ABI change.
Reference:
http://bugs.dragonflybsd.org/issues/1398
MFC after: 2 weeks
While testing this I found a conformance issue in hdestroy()
that will be fixed in a subsequent commit.
Obtained from: NetBSD (hcreate.c, CVS Rev. 1.7)
Also ANSIfy a function declaration.
While here update the OpenBSD patch level in getopt_long.c as we
already have the corresponding change.
Obtained from: NetBSD
MFC after: 2 weeks
If realpath() is called on pathnames like "/dev/null/." or "/dev/null/..",
it should fail with [ENOTDIR]. Pathnames like "/dev/null/" already failed as
they should.
Also, put the check for non-directories after lstatting the previous
component instead of when the empty component (consecutive or trailing
slashes) is detected, saving an lstat() call and some lines of code.
PR: kern/82980
MFC after: 2 weeks
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.
A typical use-case is to build a standard ILP32 version and a COMPAT32
version in a single iteration by building the COMPAT32 version using a
reach-over makefile.
Obtained from: Juniper Networks, Inc.
owning the handle passed to __cxa_finalize() but which are registered
by other dso, when the process is inside exit(3).
Running them makes the destruction order wrong, and there is hope that
such destructors would not call dlclose(3), since it is pointless at
this stage of the process existence.
The change effectively disables the r211706 after the exit(3) is
called.
Reported and tested by: Michael Gmelin <freebsd@grem.de>
Analyzed by: dim
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
As mentioned in r16117 and the book "Advanced Programming in the Unix
Environment" by W. Richard Stevens, we should ignore SIGINT and SIGQUIT
before forking, since it is not guaranteed that the parent process starts
running soon enough.
To avoid calling sigaction() in the vforked child, instead block SIGINT and
SIGQUIT before vfork() and keep the sigaction() to ignore after vfork(). The
FreeBSD kernel discards ignored signals, even if they are blocked;
therefore, it is not necessary to unblock SIGINT and SIGQUIT earlier.
reports of its impending demise were removed in 2009 (r199257).
However, in 1996 (r16117) system(3) was switched from vfork(2) to
fork(2) based partly on this. Switch back to vfork(2). This has a
dramatic effect in cases of extreme mmap use - such as excessive
abuse (500+) of shared libraries.
popen(3) has used vfork(2) for a while. vfork(2) isn't going anywhere.
This should a regression introduced in r253380 if malloc'ed memory
happens to have '=' at the right place.
Reported by: ache
Pointyhat to: me (avg)
MFC after: 1 day
X-MFC with: r253380
but ACM formula we use have internal state (and return value) in the
[1, 0x7ffffffe] range, so our RAND_MAX (0x7fffffff) is never reached
because it is off by one, zero is not reached too.
Correct both RAND_MAX and rand(3) return value, shifting last one
to the 0 by 1 subtracted, resulting POSIXed [0, 0x7ffffffd(=new RAND_MAX)]
range.
2) Add a checks for not overflowing on too big seeds. It may happens on
the machines, where sizeof(unsigned int) > 32 bits.
Reviewed by: bde [1]
MFC after: 2 weeks
implementations visible for use by applications. The functions $F that
are now weak symbols are:
allocm, calloc, dallocm, free, malloc, malloc_usable_size,
nallocm, posix_memalign, rallocm, realloc, sallocm
The non-weak implementations of $F are exported as __$F.
Submitted by: stevek@juniper.net
Reviewed by: jasone@, kib@
Approved by: jasone@ (jemalloc)
Obtained from: juniper Networks, Inc
example from bsearch(3) too, so that we don't have to duplicate
the example code in both places.
PR: docs/176197
Reviewed by: stefanf
Approved by: remko (mentor), gjb (mentor)
MFC after: 1 week
- Remove unused #include.
- Do not cast away const.
- Use the canonical idiom to compare two numbers.
- Use proper type for sizes, i.e. size_t instead of int.
- Correct indentation.
- Simplify printf("\n") to puts("").
- Use return instead of exit() in main().
Submitted by: Christoph Mallon, christoph.mallon at gmx.de
Approved by: gjb (mentor)
Reviewed by: stefanf
MFC after: 1 week
qsort(3) can work together to sort an array of integers.
PR: docs/176197
Submitted by: Fernando, fapesteguia at opensistemas.com
Approved by: gjb (mentor)
MFC after: 1 week
srandomdev(). This doesn't actually work
with any modern C compiler:
In particular, both clang and modern gcc
verisons silently elide any xor operation
with 'junk'.
Approved by: secteam
MFC after: 3 days
1) Don't iterate the loop from the environment array beginning each time,
iterate it under the last place we deactivate instead.
2) Call __rebuild_environ() not on each iteration but once, only at the end
of whole loop (of course, only in case if something is changed).
MFC after: 1 week
to craft environment variables with similar names like that:
a=1
a=2
...
unsetenv("a") should remove them all to make later getenv("a") impossible.
Fix it to do so (this is GNU autoconf test #3 failure too).
PR: 172273
MFC after: 1 week
This fixes a race condition where another thread may fork() before CLOEXEC
is set, unintentionally passing the descriptor to the child process.
This commit only adds O_CLOEXEC flags to open() or openat() calls where no
fcntl(fd, F_SETFD, FD_CLOEXEC) follows. The separate fcntl() call still
leaves a race window so it should be fixed later.
The ENOTDIR mapping was introduced in r235266 for kern/128933 based on
an interpretation of the somewhat ambiguous language in the POSIX realpath
specification. The interpretation is inconsistent with Solaris and Linux,
a regression from 9.0, and does not appear to be permitted by the
description of ENOTDIR:
20 ENOTDIR Not a directory. A component of the specified pathname
existed, but it was not a directory, when a directory was
expected.
PR: standards/171577
MFC after: 3 days
http://austingroupbugs.net/view.php?id=385#c713
(Resolved state) recommend this way for the current standard (called
"earlier" in the text)
"However, earlier versions of this standard did not require this, and the
same example had to be written as:
// buf was obtained by malloc(buflen)
ret = write(fd, buf, buflen);
if (ret < 0) {
int save = errno;
free(buf);
errno = save;
return ret;
}
"
from feedback I have for previous commit it seems that many people prefer
to avoid mass code change needed for current standard compliance
and prefer to track unpublished standard instead, which requires now
that free() itself must save errno, not its usage code.
So, I back out "save errno across free()" part of previous commit,
and will fill PR for changing free() isntead.
2) Remove now unused serrno.
MFC after: 1 week
"The setting of errno after a successful call to a function is
unspecified unless the description of that function specifies that
errno shall not be modified."
However, free() in IEEE Std 1003.1-2008 does not mention its interaction
with errno, so MAY modify it after successful call
(it depends on particular free() implementation, OS-specific, etc.).
So, save errno across free() calls to make code portable and
POSIX-conformant.
2) Remove unused serrno assignment.
MFC after: 1 week
[ENOENT] A component of file_name does not name an existing file or
file_name points to an empty string.
[ENOTDIR] A component of the path prefix is not a directory, or the
file_name argument contains at least one non- <slash> character
and ends with one or more trailing <slash> characters and the last
pathname component names an existing file that is neither a
directory nor a symbolic link to a directory.
Add checks for the listed conditions, and set errno accordingly.
Update the realpath(3) manpage to mention SUS behaviour. Remove the
requirement to include sys/param.h before stdlib.h.
PR: 128933
MFC after: 3 weeks
prior to 3.0.0 release) as contrib/jemalloc, and integrate it into libc.
The code being imported by this commit diverged from
lib/libc/stdlib/malloc.c in March 2010, which means that a portion of
the jemalloc 1.0.0 ChangeLog entries are relevant, as are the entries
for all subsequent releases.
The C11 folks reinvented the wheel by introducing an aligned version of
malloc(3) called aligned_alloc(3), instead of posix_memalign(3). Instead
of returning the allocation by reference, it returns the address, just
like malloc(3).
Reviewed by: jasone@
yet (see LLVM PR 9788), and warns about it, rub it out for now. When
clang grows support for this attribute, I will revert this again.
MFC after: 1 week
__noreturn macro and modify the other exiting functions to use it.
The __noreturn macro, unlike __dead2, must be used BEFORE the function.
This is in line with the C and C++ specifications that place _Noreturn (c1x)
and [[noreturn]] (C++11) in front of the functions. As with __dead2, this
macro falls back to using the GCC attribute.
Unfortunately, clang currently sets the same value for the C version macro
in C99 and C1x modes, so these functions are hidden by default. At some
point before 10.0, I need to go through the headers and clean up the C1x /
C++11 visibility.
Reviewed by: brooks (mentor)
load of _l suffixed versions of various standard library functions that use
the global locale, making them take an explicit locale parameter. Also
adds support for per-thread locales. This work was funded by the FreeBSD
Foundation.
Please test any code you have that uses the C standard locale functions!
Reviewed by: das (gdtoa changes)
Approved by: dim (mentor)
The size passed to strlcat() must depend on the input length, not the
output length. Because the input and output buffers are equal in size,
the resulting binary does not change at all.
-g, by reverting r219139. The LLVM PR referenced in that revision was
fixed in the mean time, and we imported a clang snapshot soon
afterwards, so the temporary workaround of disabling clang's integrated
assembler is no longer needed.
In this particular case, using e.g. DEBUG_FLAGS=-g causes clang to
output certain directives into assembly that our version of GNU as
chokes on.
Reported by: dougb
Approved by: re (kib)
support for it. Note that while sparc64 also supports the static TLS
model and thus tls_model("initial-exec"), using the default model
turned out to yield slightly better buildstone performance.
kernel.debug (or possibly other files), when WITH_CTF is active.
This is caused by a bug in clang's integrated assembler, causing malloc
to sometimes hang during initialization in statically linked executables
that use threading, such as the copy of ctfmerge that is built during
the bootstrap stage of buildworld. The bug has been submitted upstream:
http://llvm.org/bugs/show_bug.cgi?id=9352
Note that you might have to rebuild and install libc first, to get your
kernel build to finish, because the ctfmerge binary built during
bootstrap is linked with your base system's copy of libc.a, which might
already contain a bad copy of malloc.o.
add a wrapper for it in libc and rework the code in libthr, the
system call still can return EINTR, we keep this feature.
Discussed on: thread
Reviewed by: jilles
their implementations aren't in the same files. Introduce LIBC_ARCH
and use that in preference to MACHINE_CPUARCH. Tested by amd64 and
powerpc64 builds (thanks nathanw@)
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.
Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.
Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.
The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.
Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks
number of host CPUs and osreldate.
This eliminates the last sysctl(2) calls from the dynamically linked image
startup.
No objections from: kan
Tested by: marius (sparc64)
MFC after: 1 month
finished using it. This allows the mutex's allocated memory to be
freed.
This is one sense a rather silly change, since at this point we're
less than a microsecond away from calling _exit; but fixing this
memory leak is likely to make life easier for anyone trying to
track down other memory leaks.
bottom of the manpages and order them consistently.
GNU groff doesn't care about the ordering, and doesn't even mention
CAVEATS and SECURITY CONSIDERATIONS as common sections and where to put
them.
Found by: mdocml lint run
Reviewed by: ru
SUSv4 requires that implementation returns EINVAL if supplied path is NULL,
and ENOENT if path is empty string [1].
Bring prototype in conformance with SUSv4, adding restrict keywords.
Allow the resolved path buffer pointer be NULL, in which case realpath(3)
allocates storage with malloc().
PR: kern/121897 [1]
MFC after: 2 weeks
Although groff_mdoc(7) gives another impression, this is the ordering
most widely used and also required by mdocml/mandoc.
Reviewed by: ru
Approved by: philip, ed (mentors)
the static TLS model, which is fundamentally different from the dynamic
TLS model. The consequence was data corruption. Limit the attribute to
i386 and amd64.
http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/143350
Empty string test gone wrong.
Testing this requires that you have a locale that has the sign string
unset but has int_n_sign_posn set (the default locale falls through to
use "()" around negative numbers which is probably another bug).
I created that setup by hand and indeed without this fix negative
numbers are put out as positive numbers (doesn't fall through to use
"-" as default indicator).
Unfixed example in nl_NL.ISO8859-1 with lc->negative_sign set to empty
string:
strfmon(buf, sizeof(buf), "%-8i", -42.0);
==>
example2: 'EUR 42,00' 'Eu 42,00'
Fixed:
example2: 'EUR 42,00-' 'Eu 42,00-'
This file and suggested fix are identical in at least freebsd-8.
Backport might be appropriate but some expert on locales should
probably have a look at us defaulting to negative numbers in
parenthesis when LC_* is default. That doesn't look right and is not
what other OSes are doing.
PR: 143350
Submitted by: Corinna Vinschen
Reviewed by: bug reporter submitted, tested by me
* Fix a race in chunk_dealloc_dss().
* Check for allocation failure before zeroing memory in base_calloc().
Merge enhancements from a divergent version of jemalloc:
* Convert thread-specific caching from magazines to an algorithm that is
more tunable, and implement incremental GC.
* Add support for medium size classes, [4KiB..32KiB], 2KiB apart by
default.
* Add dirty page tracking for pages within active small/medium object
runs. This allows malloc to track precisely which pages are in active
use, which makes dirty page purging more effective.
* Base maximum dirty page count on proportion of active memory.
* Use optional zeroing in arena_chunk_alloc() to avoid needless zeroing
of chunks. This is useful in the context of DSS allocation, since a
long-lived application may commonly recycle chunks.
* Increase the default chunk size from 1MiB to 4MiB.
Remove feature:
* Remove the dynamic rebalancing code, since thread caching reduces its
utility.
of setenv(), putenv() and unsetenv() when dealing with corrupt entries in
environ. They now output a warning and complete their task without error.
MFC after: 1 week
instead of returning an error if a corrupt (not a "name=value" string) entry
in the environ array is detected when (re)-building the internal
environment. This should prevent applications or libraries from
experiencing issues arising from the expectation that these calls will
complete even with corrupt entries. The behavior is now as it was prior to
7.0.
Reviewed by: jilles
MFC after: 1 week
find a variable. Include a note that it must not cause the internal
environment to be generated since malloc() depends upon getenv(). To call
malloc() would create a circular dependency.
Recommended by: green
Approved by: jilles
MFC after: 1 week
**environ entries. This puts non-getenv(3) operations in line with
getenv(3) in that bad environ entries do not cause all operations to
fail. There is still some inconsistency in that getenv(3) in the
absence of any environment-modifying operation does not emit corrupt
environ entry warnings.
I also fixed another inconsistency in getenv(3) where updating the
global environ pointer would not be reflected in the return values.
It would have taken an intermediary setenv(3)/putenv(3)/unsetenv(3)
in order to see the change.
a large page size that is greater than malloc(3)'s default chunk size but
less than or equal to 4 MB, then increase the chunk size to match the large
page size.
Most often, using a chunk size that is less than the large page size is not
a problem. However, consider a long-running application that allocates and
frees significant amounts of memory. In particular, it frees enough memory
at times that some of that memory is munmap()ed. Up until the first
munmap(), a 1MB chunk size is just fine; it's not a problem for the virtual
memory system. Two adjacent 1MB chunks that are aligned on a 2MB boundary
will be promoted automatically to a superpage even though they were
allocated at different times. The trouble begins with the munmap(),
releasing a 1MB chunk will trigger the demotion of the containing superpage,
leaving behind a half-used 2MB reservation. Now comes the real problem.
Unfortunately, when the application needs to allocate more memory, and it
recycles the previously munmap()ed address range, the implementation of
mmap() won't be able to reuse the reservation. Basically, the coalescing
rules in the virtual memory system don't allow this new range to combine
with its neighbor. The effect being that superpage promotion will not
reoccur for this range of addresses until both 1MB chunks are freed at some
point in the future.
Reviewed by: jasone
MFC after: 3 weeks
When I wrote the pseudo-terminal driver for the MPSAFE TTY code, Robert
Watson and I agreed the best way to implement this, would be to let
posix_openpt() create a pseudo-terminal with proper permissions in place
and let grantpt() and unlockpt() be no-ops.
This isn't valid behaviour when looking at the spec. Because I thought
it was an elegant solution, I filed a bug report at the Austin Group
about this. In their last teleconference, they agreed on this subject.
This means that future revisions of POSIX may allow grantpt() and
unlockpt() to be no-ops if an open() on /dev/ptmx (if the implementation
has such a device) and posix_openpt() already do the right thing.
I'd rather put this in the manpage, because simply mentioning we don't
comply to any standard makes it look worse than it is. Right now we
don't, but at least we took care of it.
Approved by: re (kib)
MFC after: 3 days
A more elegant way of obtaining a name of a character device by its file
descriptor on FreeBSD, is to use the FIODGNAME ioctl. Because a valid
file descriptor implies a file descriptor is visible in /dev, it will
always resolve a valid device name.
I'm adding a more friendly wrapper for this ioctl, called fdevname(). It
is a lot easier to use than devname() and also has better error
handling. When a device name cannot be resolved, it will just return
NULL instead of a generated device name that makes no sense.
Discussed with: kib
stating that in FreeBSD the atol() and atoll() functions affect
errno in the same way as strtol() and stroll().
PR: docs/126487
Submitted by: edwin
Reviewed by: trhodes, gabor
MFC after: 1 week
