Commit Graph

10862 Commits

Author SHA1 Message Date
Tom Rhodes
98cea79298 Back out my previous commit to this area, there are differences between
our implementation and OpenBSD's.

Requested by:	des
2007-03-30 06:12:45 +00:00
Jason Evans
d33f4690ba Use size_t instead of unsigned for pagesize-related values, in order to
avoid downcasting issues.  In particular, this change fixes
posix_memalign(3) for alignments greater than 2^31 on LP64 systems.

Make sure that NDEBUG is always set to be compatible with MALLOC_DEBUG. [1]

Reported by:	[1] Lee Hyo geol <hyogeollee@gmail.com>
2007-03-29 21:07:17 +00:00
Jason Evans
eaf8d73212 Remove the run promotion/demotion machinery. Replace it with red-black
trees that track all non-full runs for each bin.  Use the red-black
trees to be able to guarantee that each new allocation is placed in the
lowest address available in any non-full run.  This change completes the
transition to allocating from low addresses in order to reduce the
retention of sparsely used chunks.

If the run in current use by a bin becomes empty, deallocate the run
rather than retaining it for later use.  The previous behavior had the
tendency to spread empty runs across multiple chunks, thus preventing
the release of chunks that were completely unused.

Generalize base_chunk_alloc() (and rename it to base_pages_alloc()) to
handle allocation sizes larger than the chunk size, so that it is
possible to support chunk sizes that are smaller than an arena object.

Reduce the minimum chunk size from 64kB to 8kB.

Optimize tracking of addresses for deleted chunks.

Fix a statistics bug for huge allocations.
2007-03-28 19:55:07 +00:00
Jason Evans
2872a3cdfe Change macro in order to refer to FreeBSD 7.0 instead of 7.0BSD.
Reported by:	Michal Mertl <mime@traveller.cz>
2007-03-28 18:55:44 +00:00
Jason Evans
0a19939042 Update the IMPLEMENTATION NOTES section to reflect recent malloc
enhancements.
2007-03-28 04:34:19 +00:00
Jason Evans
1d06bfeccf Remove some stray roff formatting that caused incorrect rendering. 2007-03-28 04:33:34 +00:00
Jason Evans
cec95ce278 Add a HISTORY section. 2007-03-28 04:32:51 +00:00
Yaroslav Tykhiy
cf21ead53b In account management, verify whether the account has been locked
with `pw lock', so that it's impossible to log into a locked account
using an alternative authentication mechanism, such as an ssh key.
This change affects only accounts locked with pw(8), i.e., having a
`*LOCKED*' prefix in their password hash field, so people still can
use a different pattern to disable password authentication only.

Mention all account management criteria in the manpage.

Approved by:	maintainer (timeout)
PR:		bin/71147
MFC after:	1 month
2007-03-27 09:59:15 +00:00
Joseph Koshy
a77c710a25 Describe the contents of the "ar_name" and "ar_rawname" fields of
Elf_Arhdr structures better.
2007-03-27 04:47:50 +00:00
Joseph Koshy
7a3d57d27a Bug fixes to ar(1) archive handling:
- Correctly retrieve the initial (special) members of an archive after
   an archive descriptor is rewound using elf_rand(SARMAG).
 - Do not strip trailing white space from the 'raw' names retrieved
   using elf_getarhdr().

Reported by:	"Hyo geol, Lee" <hyogeollee at gmail dot com>
2007-03-27 04:40:57 +00:00
Joseph Koshy
7aa963e5e5 Clarify memory management rules for pmc_cpuinfo().
Suggested by:	"Harald Servat" <redcrash at gmail dot com>
2007-03-26 17:33:41 +00:00
Joseph Koshy
77d1ae043e Document the return type of elf_rand(3) correctly. 2007-03-26 16:31:42 +00:00
Jason Evans
12fbf47cfb Fix some subtle bugs for posix_memalign() having to do with integer
rounding and overflow.  Carefully document what the various overflow
tests actually detect.

The bugs mostly canceled out, such that the worst possible failure
cases resulted in non-fatal over-allocations.
2007-03-24 20:44:06 +00:00
Tim Kientzle
e2d97e54d0 Notice when mkdir() fails.
Don't change permissions on an existing dir unless _EXTRACT_PERM
is requested.

In particular, bsdtar -x should not edit mode of existing dirs
now; bsdtar -xp will.
2007-03-24 05:02:16 +00:00
Tim Kientzle
7fa9cd7871 Fix compile error when libbz2 is unavailable. 2007-03-24 03:23:26 +00:00
Jason Evans
e3da012f00 Fix posix_memalign() for large objects. Now that runs are extents rather
than binary buddies, the alignment guarantees are weaker, which requires
a more complex aligned allocation algorithm, similar to that used for
alignment greater than the chunk size.

Reported by:	matteo
2007-03-23 22:58:15 +00:00
Jason Evans
bb99793a2b Use extents rather than binary buddies to track free pages within
chunks.  This allows runs to be any multiple of the page size.  The
primary advantage is that large objects are no longer constrained to be
2^n pages, which can dramatically decrease internal fragmentation for
large objects.  This also allows the sizes for runs that back small
objects to be more finely tuned.

Free runs are searched for linearly using the chunk page map (with the
help of some heuristic optimizations).  This changes the allocation
policy from "first best fit" to "first fit".  A prototype red-black tree
implementation for tracking free runs that implemented "first best fit"
did not cause a measurable speed or memory usage difference for
realistic chunk sizes (though of course it is possible to construct
benchmarks that favor one allocation policy over another).

Refine the handling of fullness constraints for small runs to be more
tunable.

Restructure the per chunk page map to contain only two fields per entry,
rather than four.  Also, increase each entry from 4 to 8 bytes, since it
allows for 32-bit integers, without increasing the number of chunk
header pages.

Relax the maximum chunk size constraint.  This is of no practical
interest; it is merely fallout from the chunk page map restructuring.

Revamp statistics gathering and reporting to be faster, clearer and more
informative.  Statistics gathering is fast enough now to have little
to no impact on application speed, but it still requires approximately
two extra pages of memory per arena (per process).  This memory overhead
may be acceptable for most systems, but we still need to leave
statistics gathering disabled by default in RELENG branches.

Rename NO_MALLOC_EXTRAS to MALLOC_PRODUCTION in order to make its intent
clearer (i.e. it should be defined in RELENG branches).
2007-03-23 05:05:48 +00:00
Max Laier
02023242ea Let people in on the magic of INET[6]_ADDRSTRLEN which comes quite handy in
combination with inet_ntop().

Reviewed by:	trhodes "works for me"
2007-03-21 05:46:18 +00:00
Jason Evans
c9f0c8fd74 Avoid using vsnprintf(3) unless MALLOC_STATS is defined, in order to
avoid substantial potential bloat for static binaries that do not
otherwise use any printf(3)-family functions. [1]

Rearrange arena_run_t so that the region bitmask can be minimally sized
according to constraints related to each bin's size class.  Previously,
the region bitmask was the same size for all run headers, which wasted
a measurable amount of memory.

Rather than making runs for small objects as large as possible, make
runs as small as possible such that header overhead stays below a
certain bound.  There are two exceptions that override the header
overhead bound:

	1) If the bound is impossible to honor, it is relaxed on a
	   per-size-class basis.  Since there is one bit of header
	   overhead per object (plus a constant), it is impossible to
	   achieve a header overhead less than or equal to 1/(# of bits
	   per object).  For the current setting of maximum 0.5% header
	   overhead, this relaxation comes into play for {2, 4, 8,
	   16}-byte objects, for which header overhead is (on 64-bit
	   systems) {7.1, 4.3, 2.2, 1.2}%, respectively.

	2) There is still a cap on small run size, still set to 64kB.
	   This comes into play for {1024, 2048}-byte objects, for which
	   header overhead is {1.6, 3.1}%, respectively.

In practice, this reduces the run sizes, which makes worst case
low-water memory usage due to fragmentation less bad.  It also reduces
worst case high-water run fragmentation due to non-full runs, but this
is only a constant improvement (most important to small short-lived
processes).

Reduce the default chunk size from 2MB to 1MB.  Benchmarks indicate that
the external fragmentation reduction makes 1MB the new sweet spot (as
small as possible without adversely affecting performance).

Reported by:	[1] kientzle
2007-03-20 03:44:10 +00:00
Joseph Koshy
4864e6c230 Correct a typo.
Submitted by:	 Kai Wang <kaiw27 at gmail dot com>
2007-03-19 03:52:20 +00:00
Colin Percival
f2d18f3112 Don't forget to increment the raw_position (bytes written) counter, even
when operating in non-buffered mode.

Pointy hat to:	cperciva
MFC after:	3 days
2007-03-17 14:59:04 +00:00
Maxim Konovalov
a0490929c3 o Add ENVIRONMENT section and mention there that TMPDIR is ignored
when issetugid(3) is true.

PR:		docs/108346
Obtained from:	OpenBSD
MFC after:	1 week
2007-03-16 21:46:24 +00:00
Pawel Jakub Dawidek
fa1abc314f The ufs_disk_fillout(3) can take special device name (with or without /dev/
prefix) as an argument and mount point path. At the end it has to find
device name file system is stored on, which means when mount point path is
given, it tries to look into /etc/fstab and find special device
corresponding to the given mount point. This is not perfect, because it
doesn't handle the case when file system is mounted by hand and mount point
is given as an argument.

I found this problem while trying to use snapinfo(8), which passes mount
points to the ufs_disk_fillout(3) function, but I had file system mounted
manually, so snapinfo(8) was exiting with the error below:

	ufs_disk_fillout: No such file or directory

I modified libufs(3) to handle those arguments (the order is important):

1. special device with /dev/ prefix
2. special device without /dev/ prefix
3. mount point listed in /etc/fstab, directory exists
4. mount point listed in /etc/fstab, directory doesn't exist
5. mount point of a file system mounted by hand
2007-03-16 03:13:28 +00:00
Randall Stewart
42551e993f - Sysctl's move to seperate file
- moved away from ifn/ifa access to sctp_ifa/sctp_ifn
  built and managed by the add-ip code.
- cleaned up add-ip code to use the iterator
- made iterator be a thread, which enables auto-asconf now.
- rewrote and cleaned up source address selection (also
  made it use new structures).
- Fixed a couple of memory leaks.
- DACK now settable as to how many packets to delay as
  well as time.
- connectx() to latest socket API, new associd arg.
- Fixed issue with revoking and loosing potential to
  send when we inflate the flight size. We now inflate
  the cwnd too and deflate it later when the revoked
  chunk is sent or acked.
- Got rid of some temp debug code
- src addr selection moved to a common file (sctp_output.c)
- Support for simple VRF's (we have support for multi-vfr
  via compile switch that is scrubbed from BSD but we won't
  need multi-vrf until we first get VRF :-D)
- Rest of mib work for address information now done
- Limit number of addresses in INIT/INIT-ACK to
  a #def (30).

Reviewed by:	gnn
2007-03-15 11:27:14 +00:00
Robert Watson
e6a4641532 Fix a comment in memstat.h: errors are associated with memory type lists,
not individual types.

Submitted by:	Bryan Venteicher <bryanv at daemoninthecloset dot org>
MFC after:	3 days
2007-03-15 10:44:18 +00:00
Tim Kientzle
22976ba40a When ARCHIVE_EXTRACT_UNLINK is requested:
* Only try to remove the existing item if we're not restoring a directory.
  * If unlink fails, try rmdir next.
This should fix the broken --unlink option in bsdtar.

Thanks again to: Kris Kennaway, for beating up bsdtar on pointyhat.
2007-03-13 06:04:24 +00:00
Stefan Farfeleder
f9ff2f8ffb Merge the following changes from NetBSD:
history.c 1.32:
  # Fix memory leak found by valgrind (Julien Torres)
map.c 1.24:
  # fix debugging printf format.
read.c 1.40:
  # Fix bug with multiple pending el_pushes. Reported by Julien Torres.
tty.c 1.24:
  # Coverity CID 1216: Prevent negative index use.

MFC after:	3 weeks
2007-03-11 21:47:40 +00:00
Stefan Farfeleder
e6de94e677 Merge the following changes from NetBSD:
chared.h 1.17, common.c 1.19, emacs.c 1.21, key.c 1.18, key.h 1.9, map.c 1.23,
term.c 1.42, term.h 1.17, vi.c 1.25:
  # Print the actual eofc, instead of ^D\b\b.
  # Change internal character decoding to prevent buffer oveflows.
key.c 1.19, key.h 1.10:
  # move declaration to header file.
term.c 1.43:
  # Coverity CID 806: Prevent NULL deref
term.c 1.44:
  # Coverity CID 1668: Plug memory leak.
term.c 1.45:
  # Fix compilation.

MFC after:	3 weeks
2007-03-11 18:30:22 +00:00
Tim Kientzle
2304493774 Libarchive 2.0.23:
* The ACL formatter was mis-formatting entries which had a
     user/group ID but no name.  Make the parser tolerant of
     these, so that old archives can be correctly restored;
     fix the formatter to generate correct entries.
   * Fix overwrite detection by introducing a new "FAILED" return
     code that indicates the current entry cannot be continued
     but the archive as a whole is still sound.
   * Header cleanup:  Remove some unused headers, add some that
     are required with new Linux systems.
2007-03-11 10:29:52 +00:00
Stefan Farfeleder
370bf6653c Merge changes to the NetBSD copyright (advertising clause removal). 2007-03-11 08:41:01 +00:00
Bruce M Simpson
fd46d76ecf Wordsmithery.
Pointed out by:	ru
2007-03-09 19:43:42 +00:00
Rong-En Fan
31b4134f7b Enable ncurses wide character support
Approved by:	delphij (mentor)
Tested by:	kris on pointyhat (early version), current@
2007-03-09 12:11:58 +00:00
Rong-En Fan
557158de64 - style.Makefile(9) fix
- first line is $FreeBSD$
  - Reorder special variables: DPADD, LPADD, CFLAGS
  - Use = instead of += for variables that are initially empty
  - Use space instead of tab after :
  - Use one tab after =
- Use .SUFFIXES for section 3 manual page which simplifies Makefile a lot
- Use SHAREDIR instead of /usr/share
- Remove SRCDIR in INCS since we set .PATH properly
- Use plural in variable name when it stands for more that one source file

Reviewed by:	ru
Approved by:	delphij (mentor)
2007-03-09 09:54:07 +00:00
Bruce M Simpson
7b7b32179e Document SO_ACCEPTCONN.
Submitted by:	Vlad GALU (with changes)
MFC after:	3 days
2007-03-08 12:57:12 +00:00
Tim Kientzle
d4c6c416dc New tests for system-independent ACL support.
These tests verify that archive_entry objects can store and return
ACL data and that pax format archives can read and write ACL
information.  These do not (yet) test that ACL data is read or
written to disk correctly.  (And hence would not have caught the
recent snafu about ACL read-from-disk being turned off.)
2007-03-08 06:09:27 +00:00
Tim Kientzle
19dd89364c Distinguish between the end of ACL data and an error in pulling
ACL data from the archive entry.  This doesn't impact
archive_read_extract or archive_write_disk since they only
check for != ARCHIVE_OK when calling this function.  (Though
they should be more careful.)
2007-03-08 06:07:07 +00:00
Joseph Koshy
dad626cccb Add GELF_* accessor macros.
Prodded by:	Sam Arun Raj <samarunraj at gmail dot com>
2007-03-08 04:01:30 +00:00
Bruce M Simpson
9dba3fec85 Fix license. Clause 4 is still required (UCB materiel).
Submitted by:	rwatson
Pointy hat to:	bms
2007-03-07 13:38:11 +00:00
Bruce M Simpson
13f35fd9be use 2-clause BSD license as per hoskins strike-off july 22 1999.
use wording of FreeBSD License.
2007-03-07 11:06:46 +00:00
John Baldwin
5e41bd2cd2 Only reject file descriptors higher than FD_SETSIZE if we are not using
poll(2) or kqueue(2).  Previously we rejected fd's higher than FD_SETSIZE
for kevent(2), and larger than sysconf(_SC_OPEN_MAX) for poll(2).  However,
the check for poll(2) wasn't really needed.  open(2) and socket(2) won't
return an fd you can't pass to either poll(2) or kevent(2).  This fixes
a but where gethostbyname() would fail if you had more than 1023 files
open in a process.

MFC after:	1 week
Reviewed by:	ume
Found by:	ps
2007-03-05 19:39:51 +00:00
Brian Somers
d5d9eb5095 Oops, fix a typo in the last commit :-/ 2007-03-05 19:00:49 +00:00
Brian Somers
ee12856f1d In the NOTYET code path when a process forks, the remaining
child thread goes back to system scope rather than process
scope.  This allows an ensuing exec() to actually work.

This change was made a year ago here, but I "forgot" to
commit it :(

Approved by:	deischen
MFC after:	3 weeks
2007-03-05 17:47:27 +00:00
Ruslan Ermilov
ff91121ada Apply my patch properly. 2007-03-05 15:44:00 +00:00
Bruce M Simpson
693099c53f Fix markup.
Submitted by:	ru
MFC after:	2 days
2007-03-05 13:52:01 +00:00
Bruce M Simpson
491deb49c4 .Xr nit.
Submitted by:	brueffer
2007-03-05 12:54:03 +00:00
Bruce M Simpson
4802040ebf Update shutdown() manual page to reflect actual behaviour of code.
Add IMPLEMENTATION NOTES section explaining in detail the effect this
system call has in common use cases involving PF_INET and PF_INET6 sockets.

PR:		kern/84761
MFC after:	2 days
2007-03-05 12:39:53 +00:00
Xin LI
2f1b6e8bb5 Test cases for back references.
Obtained from:	OpenBSD
2007-03-05 09:44:41 +00:00
Xin LI
082063a051 Only stop evaluation of a back reference if the match length is
zero and the recursion level is too deep.

Obtained from:	OpenBSD
2007-03-05 09:43:55 +00:00
Xin LI
0f4481c5e4 Avoid infinite recursion on:
echo "foo foo bar bar bar baz" | sed 's/\([^ ]*\)\( *\1\)*/\1/g'

Obtained from:	OpenBSD via NetBSD (rev. 1.18)
2007-03-05 03:07:36 +00:00
Simon L. B. Nielsen
3dd425f70b Disable RPC exponential back-off for FreeBSD.org systems (IE. hidden
behind _FREEFALL_CONFIG).  This is done mainly to make NIS even more
resistant to packet loss.

This is not enabled by default for "normal" FreeBSD since it might cause
the server providing the RPC service to be hit heavily with RPC traffic
in case of problems.  freefall.FreeBSD.org and hub.FreeBSD.org have been
running with a patch similar to this for a couple of weeks.

MFC after:	1 week
Discussed with:	peter
2007-03-04 12:25:03 +00:00