Commit Graph

633 Commits

Author SHA1 Message Date
Jason Evans
5f2b1ed91b Simplify arena_run_reg_dalloc(), and remove a bug that was due to incorrect
initialization of ssize_invs.
2009-12-10 02:51:40 +00:00
Jason Evans
2354bdcf94 Fix the posix_memalign() changes in r196861 to actually return a NULL pointer
as intended.

PR:		standards/138307
2009-12-10 00:16:11 +00:00
Sean Farley
f959b43fd0 Revert behavior change to setenv(), unsetenv() and putenv() until a more
thorough security review has been completed.
2009-12-07 00:22:10 +00:00
Sean Farley
ef6ccfb52a Update the getenv(3) man page to reflect the recent change to the behavior
of setenv(), putenv() and unsetenv() when dealing with corrupt entries in
environ.  They now output a warning and complete their task without error.

MFC after:	1 week
2009-12-06 23:51:27 +00:00
Sean Farley
ca7520fe57 Change the behavior of setenv(), putenv() and unsetenv() to continue parsing
instead of returning an error if a corrupt (not a "name=value" string) entry
in the environ array is detected when (re)-building the internal
environment.  This should prevent applications or libraries from
experiencing issues arising from the expectation that these calls will
complete even with corrupt entries.  The behavior is now as it was prior to
7.0.

Reviewed by:	jilles
MFC after:	1 week
2009-12-06 23:27:24 +00:00
Sean Farley
6da7f71cd4 Improve the comment within getenv() explaining the search order it takes to
find a variable.  Include a note that it must not cause the internal
environment to be generated since malloc() depends upon getenv().  To call
malloc() would create a circular dependency.

Recommended by:	green
Approved by:	jilles
MFC after:	1 week
2009-12-06 23:05:17 +00:00
Ed Schouten
2c201a9afe Fix many "function declaration isn't a prototype" warnings in libc.
I've only fixed code that seems to be written by `us'. There are still
many warnings like this present in resolv/, rpc/, stdtime/ and yp/.
2009-12-05 19:31:38 +00:00
Brian Feldman
56a3273e0b Temporarily revert the previous change because the linker has been
modified so that it will abort when the environment is bad.
2009-12-01 06:42:47 +00:00
Brian Feldman
20f492f0eb Do not gratuitously fail *env(3) operations due to corrupt ('='-less)
**environ entries.  This puts non-getenv(3) operations in line with
getenv(3) in that bad environ entries do not cause all operations to
fail.  There is still some inconsistency in that getenv(3) in the
absence of any environment-modifying operation does not emit corrupt
environ entry warnings.

I also fixed another inconsistency in getenv(3) where updating the
global environ pointer would not be reflected in the return values.
It would have taken an intermediary setenv(3)/putenv(3)/unsetenv(3)
in order to see the change.
2009-12-01 05:04:31 +00:00
Colin Percival
8a7f1847b7 Change the utrace log entry for malloc_init from (0, 0, 0) to (-1, 0, 0)
in order to distinguish it from free(NULL), which is logged as (0, 0, 0).

Reviewed by:	jhb
2009-11-14 09:31:47 +00:00
Alan Cox
b8947edcb6 Make malloc(3) superpage aware. Specifically, if getpagesizes(3) returns
a large page size that is greater than malloc(3)'s default chunk size but
less than or equal to 4 MB, then increase the chunk size to match the large
page size.

Most often, using a chunk size that is less than the large page size is not
a problem.  However, consider a long-running application that allocates and
frees significant amounts of memory.  In particular, it frees enough memory
at times that some of that memory is munmap()ed.  Up until the first
munmap(), a 1MB chunk size is just fine; it's not a problem for the virtual
memory system.  Two adjacent 1MB chunks that are aligned on a 2MB boundary
will be promoted automatically to a superpage even though they were
allocated at different times.  The trouble begins with the munmap(),
releasing a 1MB chunk will trigger the demotion of the containing superpage,
leaving behind a half-used 2MB reservation.  Now comes the real problem.
Unfortunately, when the application needs to allocate more memory, and it
recycles the previously munmap()ed address range, the implementation of
mmap() won't be able to reuse the reservation.  Basically, the coalescing
rules in the virtual memory system don't allow this new range to combine
with its neighbor.  The effect being that superpage promotion will not
reoccur for this range of addresses until both 1MB chunks are freed at some
point in the future.

Reviewed by:	jasone
MFC after:	3 weeks
2009-09-26 18:20:40 +00:00
Konstantin Belousov
1ecc75dfe3 Handle zero size for posix_memalign. Return NULL or unique address
according to the 'V' option.

PR:	standards/138307
MFC after:	1 week
2009-09-05 13:32:05 +00:00
Ed Schouten
00ee13a0af Our implementation of granpt(3) could be valid in the future.
When I wrote the pseudo-terminal driver for the MPSAFE TTY code, Robert
Watson and I agreed the best way to implement this, would be to let
posix_openpt() create a pseudo-terminal with proper permissions in place
and let grantpt() and unlockpt() be no-ops.

This isn't valid behaviour when looking at the spec. Because I thought
it was an elegant solution, I filed a bug report at the Austin Group
about this. In their last teleconference, they agreed on this subject.
This means that future revisions of POSIX may allow grantpt() and
unlockpt() to be no-ops if an open() on /dev/ptmx (if the implementation
has such a device) and posix_openpt() already do the right thing.

I'd rather put this in the manpage, because simply mentioning we don't
comply to any standard makes it look worse than it is. Right now we
don't, but at least we took care of it.

Approved by:	re (kib)
MFC after:	3 days
2009-08-24 11:16:44 +00:00
Ed Schouten
87fbd35f4f Simplify. We can just use .sinclude here.
Submitted by:	Christoph Mallon
2009-06-23 14:10:46 +00:00
Ed Schouten
98669c791d Our grantpt(3) and unlockpt(3) don't comply with POSIX. 2009-05-04 18:14:45 +00:00
Xin LI
00d949789a "-isoC-99" should be spelled without 'c'. 2009-03-01 05:44:28 +00:00
Ed Schouten
26d4f5e969 Add two new routines: fdevname() and fdevname_r().
A more elegant way of obtaining a name of a character device by its file
descriptor on FreeBSD, is to use the FIODGNAME ioctl. Because a valid
file descriptor implies a file descriptor is visible in /dev, it will
always resolve a valid device name.

I'm adding a more friendly wrapper for this ioctl, called fdevname(). It
is a lot easier to use than devname() and also has better error
handling. When a device name cannot be resolved, it will just return
NULL instead of a generated device name that makes no sense.

Discussed with:	kib
2009-02-11 20:24:59 +00:00
Giorgos Keramidas
0bc4c01d56 Fix language on atol(3) manpage. Add a COMPATIBILITY section
stating that in FreeBSD the atol() and atoll() functions affect
errno in the same way as strtol() and stroll().

PR:		docs/126487
Submitted by:	edwin
Reviewed by:	trhodes, gabor
MFC after:	1 week
2009-02-08 08:26:58 +00:00
Jason Evans
d7ba3e423a Fix a lock order reversal bug that could cause deadlock during fork(2).
Reported by:	kib
2008-12-01 10:20:59 +00:00
Jason Evans
17daa728ae Adjust an assertion to handle the case where a lock is contested, but
spinning is avoided due to running on a single-CPU system.

Reported by:	stefanf
2008-11-30 19:30:31 +00:00
Jason Evans
93e34865fa Do not spin when trying to lock on a single-CPU system.
Reported by:	davidxu
2008-11-30 05:55:24 +00:00
Jason Evans
b74d3e0c37 Revert to preferring mmap(2) over sbrk(2) when mapping memory, due to
potential extreme contention in the kernel for multi-threaded applications
on SMP systems.

Reported by:	kris
2008-11-03 21:17:18 +00:00
Alexander Leidinger
1455fd2638 MTC r183949:
Allow to define MALLOC_PRODUCTION with a make variable instead of polluting
 the global CFLAGS.

Reviewed by:	jasone
2008-10-17 08:30:20 +00:00
Jason Evans
bf5b19279d Use PAGE_{SIZE,MASK,SHIFT} from machine/param.h rather than hard-coding
page size and using sysconf(3).

Suggested by:	marcel
2008-09-10 14:27:34 +00:00
Marcel Moolenaar
93bf4a8436 Unbreak ia64: pges are 8KB. 2008-09-06 05:26:31 +00:00
Jason Evans
d6742bfbd3 Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes.  The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes.  Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.
2008-08-27 02:00:53 +00:00
Ed Schouten
f0c96ff802 Remove grantpt.c, which should have been deleted in the MPSAFE TTY commit.
The routines in grantpt.c have been moved to ptsname.c in the MPSAFE TTY
layer, because grantpt() is now effectively a no-op. I forgot to remove
the corresponding source file from libc.
2008-08-20 09:43:46 +00:00
Ed Schouten
bc093719ca Integrate the new MPSAFE TTY layer to the FreeBSD operating system.
The last half year I've been working on a replacement TTY layer for the
FreeBSD kernel. The new TTY layer was designed to improve the following:

- Improved driver model:

  The old TTY layer has a driver model that is not abstract enough to
  make it friendly to use. A good example is the output path, where the
  device drivers directly access the output buffers. This means that an
  in-kernel PPP implementation must always convert network buffers into
  TTY buffers.

  If a PPP implementation would be built on top of the new TTY layer
  (still needs a hooks layer, though), it would allow the PPP
  implementation to directly hand the data to the TTY driver.

- Improved hotplugging:

  With the old TTY layer, it isn't entirely safe to destroy TTY's from
  the system. This implementation has a two-step destructing design,
  where the driver first abandons the TTY. After all threads have left
  the TTY, the TTY layer calls a routine in the driver, which can be
  used to free resources (unit numbers, etc).

  The pts(4) driver also implements this feature, which means
  posix_openpt() will now return PTY's that are created on the fly.

- Improved performance:

  One of the major improvements is the per-TTY mutex, which is expected
  to improve scalability when compared to the old Giant locking.
  Another change is the unbuffered copying to userspace, which is both
  used on TTY device nodes and PTY masters.

Upgrading should be quite straightforward. Unlike previous versions,
existing kernel configuration files do not need to be changed, except
when they reference device drivers that are listed in UPDATING.

Obtained from:		//depot/projects/mpsafetty/...
Approved by:		philip (ex-mentor)
Discussed:		on the lists, at BSDCan, at the DevSummit
Sponsored by:		Snow B.V., the Netherlands
dcons(4) fixed by:	kan
2008-08-20 08:31:58 +00:00
Jason Evans
6f14f9b656 Move CPU_SPINWAIT into the innermost spin loop, in order to allow faster
preemption while busy-waiting.

Submitted by:	Mike Schuster <schuster@adobe.com>
2008-08-14 17:31:42 +00:00
Jason Evans
52d7a117c0 Re-order the terms of an expression in arena_run_reg_dalloc() to correctly
detect whether the integer division table is large enough to handle the
divisor.  Before this change, the last two table elements were never used,
thus causing the slow path to be used for those divisors.
2008-08-14 17:03:29 +00:00
Colin Percival
c123de30b6 Remove variables which are assigned values and never used thereafter.
Found by:	LLVM/Clang Static Checker
Approved by:	jasone
2008-08-08 20:42:42 +00:00
Sean Farley
ee2889cb98 Restructure and use different variables in the tests that involve
environ[0] to be more obvious that environ is not NULL before environ[0]
is tested.  Although I believe the previous code worked, this change
improves code maintainability.

Reviewed by:	ache
MFC after:	3 days
2008-08-03 22:47:23 +00:00
Sean Farley
3522c38bbe Detect if the application has cleared the environ variable by setting
the first value (environ[0]) to NULL.  This is in addition to the
current detection of environ being replaced, which includes being set to
NULL.  Without this fix, the environment is not truly wiped, but appears
to be by getenv() until an *env() call is made to alter the enviroment.

This change is necessary to support those applications that use this
method for clearing environ such as Dovecot and Postfix.  Applications
such as Sendmail and the base system's env replace environ (already
detected).  While neither of these methods are defined by SUSv3, it is
best to support them due to historic reasons and in lieu of a clean,
defined method.

Add extra units tests for clearing environ using four different methods:
1. Set environ to NULL pointer.
2. Set environ[0] to NULL pointer.
3. Set environ to calloc()'d NULL-terminated array.
4. Set environ to static NULL-terminated array.

Noticed by:	Timo Sirainen

MFC after:	3 days
2008-08-02 02:34:35 +00:00
Jason Evans
2bb0f7ba54 Enhance arena_chunk_map_t to directly support run coalescing, and use
the chunk map instead of red-black trees where possible.  Remove the
red-black trees and node objects that are obsoleted by this change.  The
net result is a ~1-2% memory savings, and a substantial allocation speed
improvement.
2008-07-18 19:35:44 +00:00
Daniel Gerzo
5fd5badfa9 - This code was intially obtained from NetBSD, but it's missing licence
statement. Add the one from the current NetBSD version.
- Also bump a date to reflect my content changes I have done in previous
  revision

Approved by:	imp
MFC after:	3 days
2008-07-06 17:03:37 +00:00
Daniel Gerzo
6d05da1dc9 - Add description about a missing return value
PR:		docs/75995
Submitted by:	Tarc <tarc@po.cs.msu.su>
MFC after:	3 days
2008-07-06 12:17:53 +00:00
Daniel Gerzo
408425ce37 - remove superfluous word
- remove contractions

MFC after:	3 days
2008-07-06 11:31:20 +00:00
Daniel Gerzo
91bc389e54 Mark the section describing return values with an appropriate section flag.
PR:		docs/122818
MFC after:	3 days
2008-06-26 08:24:59 +00:00
Ed Schouten
e3580e9d91 Don't export the unused __use_pts() routine.
The __use_pts() routine was once probably used by libutil to determine
if we are using BSD or UNIX98 style PTY device names. It doesn't seem to
be used outside grantpt.c, which means we can make it static and remove
it from the Symbol.map.

Reviewed by:	cognet, kib
Approved by:	philip (mentor)
2008-06-17 14:05:03 +00:00
Jason Evans
b1c8b30f55 In the error path through base_alloc(), release base_mtx [1].
Fix bit vector initialization for run headers.

Submitted by:	[1] Mike Schuster <schuster@adobe.com>
2008-06-10 15:46:18 +00:00
Jason Evans
2e78350530 Clean up cpp logic and comments. 2008-05-14 18:33:13 +00:00
Jason Evans
4788234366 Fix a comment. 2008-05-03 17:49:16 +00:00
Jason Evans
9007109030 Add a separate tree to track arena chunks that contain dirty pages.
This substantially improves worst case allocation performance, since
O(lg n) tree search can be used instead of O(n) tree iteration.

Use rb_wrap() instead of directly calling rb_*() macros.
2008-05-01 17:25:55 +00:00
Jason Evans
21162484ae Add rb_wrap(), which creates C function wrappers for most rb_*()
macros.

Add rb_foreach_next() and rb_foreach_reverse_prev(), which make it
possible to re-synchronize tree iteration after the tree has been
modified.

Rename rb_tree_new() to rb_new().
2008-05-01 17:24:37 +00:00
Oleksandr Tymoshenko
00fb5362ba Set QUANTUM_2POW_MIN and SIZEOF_PTR_2POW parameters for MIPS
Approved by: imp
2008-04-29 22:56:05 +00:00
Jason Evans
e3085308be Check for integer overflow before calling sbrk(2), since it uses a
signed increment argument, but the size is an unsigned integer.
2008-04-29 01:32:42 +00:00
Ruslan Ermilov
eff93c8073 Stricter check for integer overflow. 2008-04-24 07:49:00 +00:00
Jason Evans
e5bf0d71c9 Implement red-black trees without using parent pointers, and store the
color bit in the least significant bit of the right child pointer, in
order to reduce red-black tree linkage overhead by ~2X as compared to
sys/tree.h.

Use the new red-black tree implementation in malloc, which drops
memory usage by ~0.5 or ~1%, for 32- and 64-bit systems, respectively.
2008-04-23 16:09:18 +00:00
Ruslan Ermilov
5b30d6ca77 Don't forget to free() currency_symbol and asciivalue when multiple
conversion specifiers for them are present.

Submitted by:	Maxim Dounin <mdounin@mdounin.ru>
Obtained from:	NetBSD (partially)
MFC after:	3 days
2008-04-19 07:22:58 +00:00
Ruslan Ermilov
3890416f9c Better strfmon(3) conversion specifiers sanity checking.
There were no checks for left and right precisions at all, and
a check for field width had integer overflow bug.

Reported by:	Maksymilian Arciemowicz
Security:	http://securityreason.com/achievement_securityalert/53
Submitted by:	Maxim Dounin <mdounin@mdounin.ru>
MFC after:	3 days
2008-04-19 07:18:22 +00:00