Commit Graph

10910 Commits

Author SHA1 Message Date
kientzle
c25369d5e8 Style: bare "unsigned" is deprecated, use "unsigned int" instead.
Thanks to: Joerg Sonnenberger
2007-04-02 00:15:45 +00:00
kientzle
53ce615be3 Remove some unused fields from archive_read internal structure.
(Left over from when read and write used to share this structure.)
2007-04-02 00:11:54 +00:00
cperciva
5ca8f701f1 Provide a dummy compression-layer skip function which just reads data and
discards it, for use when the compression layer code doesn't know how to
skip data (e.g., everything other than the "none" compressor).  This makes
format level code simpler because that code can now assume that the
compression layer always knows how to skip and will always skip exactly
the requested number of bytes.

Discussed with:	kientzle (3 months ago)
2007-03-31 22:59:43 +00:00
trhodes
d74e4b9429 Back out my previous commit to this area, there are differences between
our implementation and OpenBSD's.

Requested by:	des
2007-03-30 06:12:45 +00:00
jasone
26e0e354b0 Use size_t instead of unsigned for pagesize-related values, in order to
avoid downcasting issues.  In particular, this change fixes
posix_memalign(3) for alignments greater than 2^31 on LP64 systems.

Make sure that NDEBUG is always set to be compatible with MALLOC_DEBUG. [1]

Reported by:	[1] Lee Hyo geol <hyogeollee@gmail.com>
2007-03-29 21:07:17 +00:00
jasone
ca1ed8272e Remove the run promotion/demotion machinery. Replace it with red-black
trees that track all non-full runs for each bin.  Use the red-black
trees to be able to guarantee that each new allocation is placed in the
lowest address available in any non-full run.  This change completes the
transition to allocating from low addresses in order to reduce the
retention of sparsely used chunks.

If the run in current use by a bin becomes empty, deallocate the run
rather than retaining it for later use.  The previous behavior had the
tendency to spread empty runs across multiple chunks, thus preventing
the release of chunks that were completely unused.

Generalize base_chunk_alloc() (and rename it to base_pages_alloc()) to
handle allocation sizes larger than the chunk size, so that it is
possible to support chunk sizes that are smaller than an arena object.

Reduce the minimum chunk size from 64kB to 8kB.

Optimize tracking of addresses for deleted chunks.

Fix a statistics bug for huge allocations.
2007-03-28 19:55:07 +00:00
jasone
ed54eea1f9 Change macro in order to refer to FreeBSD 7.0 instead of 7.0BSD.
Reported by:	Michal Mertl <mime@traveller.cz>
2007-03-28 18:55:44 +00:00
jasone
07d14320a3 Update the IMPLEMENTATION NOTES section to reflect recent malloc
enhancements.
2007-03-28 04:34:19 +00:00
jasone
064edc9b9b Remove some stray roff formatting that caused incorrect rendering. 2007-03-28 04:33:34 +00:00
jasone
a011d43e23 Add a HISTORY section. 2007-03-28 04:32:51 +00:00
yar
e8ee34e142 In account management, verify whether the account has been locked
with `pw lock', so that it's impossible to log into a locked account
using an alternative authentication mechanism, such as an ssh key.
This change affects only accounts locked with pw(8), i.e., having a
`*LOCKED*' prefix in their password hash field, so people still can
use a different pattern to disable password authentication only.

Mention all account management criteria in the manpage.

Approved by:	maintainer (timeout)
PR:		bin/71147
MFC after:	1 month
2007-03-27 09:59:15 +00:00
jkoshy
9645b900be Describe the contents of the "ar_name" and "ar_rawname" fields of
Elf_Arhdr structures better.
2007-03-27 04:47:50 +00:00
jkoshy
e13fe525e7 Bug fixes to ar(1) archive handling:
- Correctly retrieve the initial (special) members of an archive after
   an archive descriptor is rewound using elf_rand(SARMAG).
 - Do not strip trailing white space from the 'raw' names retrieved
   using elf_getarhdr().

Reported by:	"Hyo geol, Lee" <hyogeollee at gmail dot com>
2007-03-27 04:40:57 +00:00
jkoshy
1fe075e853 Clarify memory management rules for pmc_cpuinfo().
Suggested by:	"Harald Servat" <redcrash at gmail dot com>
2007-03-26 17:33:41 +00:00
jkoshy
359932b592 Document the return type of elf_rand(3) correctly. 2007-03-26 16:31:42 +00:00
jasone
8561f36aa3 Fix some subtle bugs for posix_memalign() having to do with integer
rounding and overflow.  Carefully document what the various overflow
tests actually detect.

The bugs mostly canceled out, such that the worst possible failure
cases resulted in non-fatal over-allocations.
2007-03-24 20:44:06 +00:00
kientzle
c9ae05df5c Notice when mkdir() fails.
Don't change permissions on an existing dir unless _EXTRACT_PERM
is requested.

In particular, bsdtar -x should not edit mode of existing dirs
now; bsdtar -xp will.
2007-03-24 05:02:16 +00:00
kientzle
bc64e87ca7 Fix compile error when libbz2 is unavailable. 2007-03-24 03:23:26 +00:00
jasone
b63dd57bc9 Fix posix_memalign() for large objects. Now that runs are extents rather
than binary buddies, the alignment guarantees are weaker, which requires
a more complex aligned allocation algorithm, similar to that used for
alignment greater than the chunk size.

Reported by:	matteo
2007-03-23 22:58:15 +00:00
jasone
a1325e04ba Use extents rather than binary buddies to track free pages within
chunks.  This allows runs to be any multiple of the page size.  The
primary advantage is that large objects are no longer constrained to be
2^n pages, which can dramatically decrease internal fragmentation for
large objects.  This also allows the sizes for runs that back small
objects to be more finely tuned.

Free runs are searched for linearly using the chunk page map (with the
help of some heuristic optimizations).  This changes the allocation
policy from "first best fit" to "first fit".  A prototype red-black tree
implementation for tracking free runs that implemented "first best fit"
did not cause a measurable speed or memory usage difference for
realistic chunk sizes (though of course it is possible to construct
benchmarks that favor one allocation policy over another).

Refine the handling of fullness constraints for small runs to be more
tunable.

Restructure the per chunk page map to contain only two fields per entry,
rather than four.  Also, increase each entry from 4 to 8 bytes, since it
allows for 32-bit integers, without increasing the number of chunk
header pages.

Relax the maximum chunk size constraint.  This is of no practical
interest; it is merely fallout from the chunk page map restructuring.

Revamp statistics gathering and reporting to be faster, clearer and more
informative.  Statistics gathering is fast enough now to have little
to no impact on application speed, but it still requires approximately
two extra pages of memory per arena (per process).  This memory overhead
may be acceptable for most systems, but we still need to leave
statistics gathering disabled by default in RELENG branches.

Rename NO_MALLOC_EXTRAS to MALLOC_PRODUCTION in order to make its intent
clearer (i.e. it should be defined in RELENG branches).
2007-03-23 05:05:48 +00:00
mlaier
4cb223184a Let people in on the magic of INET[6]_ADDRSTRLEN which comes quite handy in
combination with inet_ntop().

Reviewed by:	trhodes "works for me"
2007-03-21 05:46:18 +00:00
jasone
ef1856bb83 Avoid using vsnprintf(3) unless MALLOC_STATS is defined, in order to
avoid substantial potential bloat for static binaries that do not
otherwise use any printf(3)-family functions. [1]

Rearrange arena_run_t so that the region bitmask can be minimally sized
according to constraints related to each bin's size class.  Previously,
the region bitmask was the same size for all run headers, which wasted
a measurable amount of memory.

Rather than making runs for small objects as large as possible, make
runs as small as possible such that header overhead stays below a
certain bound.  There are two exceptions that override the header
overhead bound:

	1) If the bound is impossible to honor, it is relaxed on a
	   per-size-class basis.  Since there is one bit of header
	   overhead per object (plus a constant), it is impossible to
	   achieve a header overhead less than or equal to 1/(# of bits
	   per object).  For the current setting of maximum 0.5% header
	   overhead, this relaxation comes into play for {2, 4, 8,
	   16}-byte objects, for which header overhead is (on 64-bit
	   systems) {7.1, 4.3, 2.2, 1.2}%, respectively.

	2) There is still a cap on small run size, still set to 64kB.
	   This comes into play for {1024, 2048}-byte objects, for which
	   header overhead is {1.6, 3.1}%, respectively.

In practice, this reduces the run sizes, which makes worst case
low-water memory usage due to fragmentation less bad.  It also reduces
worst case high-water run fragmentation due to non-full runs, but this
is only a constant improvement (most important to small short-lived
processes).

Reduce the default chunk size from 2MB to 1MB.  Benchmarks indicate that
the external fragmentation reduction makes 1MB the new sweet spot (as
small as possible without adversely affecting performance).

Reported by:	[1] kientzle
2007-03-20 03:44:10 +00:00
jkoshy
b5f826c9a6 Correct a typo.
Submitted by:	 Kai Wang <kaiw27 at gmail dot com>
2007-03-19 03:52:20 +00:00
cperciva
820da80807 Don't forget to increment the raw_position (bytes written) counter, even
when operating in non-buffered mode.

Pointy hat to:	cperciva
MFC after:	3 days
2007-03-17 14:59:04 +00:00
maxim
47a8f834c2 o Add ENVIRONMENT section and mention there that TMPDIR is ignored
when issetugid(3) is true.

PR:		docs/108346
Obtained from:	OpenBSD
MFC after:	1 week
2007-03-16 21:46:24 +00:00
pjd
781b031d07 The ufs_disk_fillout(3) can take special device name (with or without /dev/
prefix) as an argument and mount point path. At the end it has to find
device name file system is stored on, which means when mount point path is
given, it tries to look into /etc/fstab and find special device
corresponding to the given mount point. This is not perfect, because it
doesn't handle the case when file system is mounted by hand and mount point
is given as an argument.

I found this problem while trying to use snapinfo(8), which passes mount
points to the ufs_disk_fillout(3) function, but I had file system mounted
manually, so snapinfo(8) was exiting with the error below:

	ufs_disk_fillout: No such file or directory

I modified libufs(3) to handle those arguments (the order is important):

1. special device with /dev/ prefix
2. special device without /dev/ prefix
3. mount point listed in /etc/fstab, directory exists
4. mount point listed in /etc/fstab, directory doesn't exist
5. mount point of a file system mounted by hand
2007-03-16 03:13:28 +00:00
rrs
bd8786ed77 - Sysctl's move to seperate file
- moved away from ifn/ifa access to sctp_ifa/sctp_ifn
  built and managed by the add-ip code.
- cleaned up add-ip code to use the iterator
- made iterator be a thread, which enables auto-asconf now.
- rewrote and cleaned up source address selection (also
  made it use new structures).
- Fixed a couple of memory leaks.
- DACK now settable as to how many packets to delay as
  well as time.
- connectx() to latest socket API, new associd arg.
- Fixed issue with revoking and loosing potential to
  send when we inflate the flight size. We now inflate
  the cwnd too and deflate it later when the revoked
  chunk is sent or acked.
- Got rid of some temp debug code
- src addr selection moved to a common file (sctp_output.c)
- Support for simple VRF's (we have support for multi-vfr
  via compile switch that is scrubbed from BSD but we won't
  need multi-vrf until we first get VRF :-D)
- Rest of mib work for address information now done
- Limit number of addresses in INIT/INIT-ACK to
  a #def (30).

Reviewed by:	gnn
2007-03-15 11:27:14 +00:00
rwatson
7f1d3da162 Fix a comment in memstat.h: errors are associated with memory type lists,
not individual types.

Submitted by:	Bryan Venteicher <bryanv at daemoninthecloset dot org>
MFC after:	3 days
2007-03-15 10:44:18 +00:00
kientzle
847bf0fdc9 When ARCHIVE_EXTRACT_UNLINK is requested:
* Only try to remove the existing item if we're not restoring a directory.
  * If unlink fails, try rmdir next.
This should fix the broken --unlink option in bsdtar.

Thanks again to: Kris Kennaway, for beating up bsdtar on pointyhat.
2007-03-13 06:04:24 +00:00
stefanf
176735320e Merge the following changes from NetBSD:
history.c 1.32:
  # Fix memory leak found by valgrind (Julien Torres)
map.c 1.24:
  # fix debugging printf format.
read.c 1.40:
  # Fix bug with multiple pending el_pushes. Reported by Julien Torres.
tty.c 1.24:
  # Coverity CID 1216: Prevent negative index use.

MFC after:	3 weeks
2007-03-11 21:47:40 +00:00
stefanf
11fa4c4751 Merge the following changes from NetBSD:
chared.h 1.17, common.c 1.19, emacs.c 1.21, key.c 1.18, key.h 1.9, map.c 1.23,
term.c 1.42, term.h 1.17, vi.c 1.25:
  # Print the actual eofc, instead of ^D\b\b.
  # Change internal character decoding to prevent buffer oveflows.
key.c 1.19, key.h 1.10:
  # move declaration to header file.
term.c 1.43:
  # Coverity CID 806: Prevent NULL deref
term.c 1.44:
  # Coverity CID 1668: Plug memory leak.
term.c 1.45:
  # Fix compilation.

MFC after:	3 weeks
2007-03-11 18:30:22 +00:00
kientzle
c533ace5de Libarchive 2.0.23:
* The ACL formatter was mis-formatting entries which had a
     user/group ID but no name.  Make the parser tolerant of
     these, so that old archives can be correctly restored;
     fix the formatter to generate correct entries.
   * Fix overwrite detection by introducing a new "FAILED" return
     code that indicates the current entry cannot be continued
     but the archive as a whole is still sound.
   * Header cleanup:  Remove some unused headers, add some that
     are required with new Linux systems.
2007-03-11 10:29:52 +00:00
stefanf
0aafd373d5 Merge changes to the NetBSD copyright (advertising clause removal). 2007-03-11 08:41:01 +00:00
bms
b4f28cba28 Wordsmithery.
Pointed out by:	ru
2007-03-09 19:43:42 +00:00
rafan
fa9f87addc Enable ncurses wide character support
Approved by:	delphij (mentor)
Tested by:	kris on pointyhat (early version), current@
2007-03-09 12:11:58 +00:00
rafan
00dd163071 - style.Makefile(9) fix
- first line is $FreeBSD$
  - Reorder special variables: DPADD, LPADD, CFLAGS
  - Use = instead of += for variables that are initially empty
  - Use space instead of tab after :
  - Use one tab after =
- Use .SUFFIXES for section 3 manual page which simplifies Makefile a lot
- Use SHAREDIR instead of /usr/share
- Remove SRCDIR in INCS since we set .PATH properly
- Use plural in variable name when it stands for more that one source file

Reviewed by:	ru
Approved by:	delphij (mentor)
2007-03-09 09:54:07 +00:00
bms
e8eac47f5e Document SO_ACCEPTCONN.
Submitted by:	Vlad GALU (with changes)
MFC after:	3 days
2007-03-08 12:57:12 +00:00
kientzle
0d9d66867b New tests for system-independent ACL support.
These tests verify that archive_entry objects can store and return
ACL data and that pax format archives can read and write ACL
information.  These do not (yet) test that ACL data is read or
written to disk correctly.  (And hence would not have caught the
recent snafu about ACL read-from-disk being turned off.)
2007-03-08 06:09:27 +00:00
kientzle
4e920aa7c0 Distinguish between the end of ACL data and an error in pulling
ACL data from the archive entry.  This doesn't impact
archive_read_extract or archive_write_disk since they only
check for != ARCHIVE_OK when calling this function.  (Though
they should be more careful.)
2007-03-08 06:07:07 +00:00
jkoshy
cb00361cdd Add GELF_* accessor macros.
Prodded by:	Sam Arun Raj <samarunraj at gmail dot com>
2007-03-08 04:01:30 +00:00
bms
bbe2ee3543 Fix license. Clause 4 is still required (UCB materiel).
Submitted by:	rwatson
Pointy hat to:	bms
2007-03-07 13:38:11 +00:00
bms
769f429126 use 2-clause BSD license as per hoskins strike-off july 22 1999.
use wording of FreeBSD License.
2007-03-07 11:06:46 +00:00
jhb
881404945f Only reject file descriptors higher than FD_SETSIZE if we are not using
poll(2) or kqueue(2).  Previously we rejected fd's higher than FD_SETSIZE
for kevent(2), and larger than sysconf(_SC_OPEN_MAX) for poll(2).  However,
the check for poll(2) wasn't really needed.  open(2) and socket(2) won't
return an fd you can't pass to either poll(2) or kevent(2).  This fixes
a but where gethostbyname() would fail if you had more than 1023 files
open in a process.

MFC after:	1 week
Reviewed by:	ume
Found by:	ps
2007-03-05 19:39:51 +00:00
brian
b30fdbdb81 Oops, fix a typo in the last commit :-/ 2007-03-05 19:00:49 +00:00
brian
cdf4f0e426 In the NOTYET code path when a process forks, the remaining
child thread goes back to system scope rather than process
scope.  This allows an ensuing exec() to actually work.

This change was made a year ago here, but I "forgot" to
commit it :(

Approved by:	deischen
MFC after:	3 weeks
2007-03-05 17:47:27 +00:00
ru
873b029d16 Apply my patch properly. 2007-03-05 15:44:00 +00:00
bms
e97dc5692f Fix markup.
Submitted by:	ru
MFC after:	2 days
2007-03-05 13:52:01 +00:00
bms
bc98d02991 .Xr nit.
Submitted by:	brueffer
2007-03-05 12:54:03 +00:00
bms
d623026ad1 Update shutdown() manual page to reflect actual behaviour of code.
Add IMPLEMENTATION NOTES section explaining in detail the effect this
system call has in common use cases involving PF_INET and PF_INET6 sockets.

PR:		kern/84761
MFC after:	2 days
2007-03-05 12:39:53 +00:00
delphij
b444fd4080 Test cases for back references.
Obtained from:	OpenBSD
2007-03-05 09:44:41 +00:00