Commit Graph

47 Commits

Author SHA1 Message Date
Tim Kientzle
3381df89e8 Merge r714,r715 from libarchive.googlecode.com: Fix Debian bug #516577.
Don't crash if client does not provide a skip function.  Extend one
of the test cases to use archive_read_open2() with only a read callback.
2009-03-07 01:21:46 +00:00
Tim Kientzle
83829cd8ea Merge r608 from libarchive.googlecode.com: Clear a newly-allocated
bidder object.
2009-03-07 01:18:30 +00:00
Tim Kientzle
98768f0df8 Merge r564,r566 from libarchive.googlecode.com: Fix segfault when
specifying an option and the current format doesn't have an
options handler.
2009-03-07 01:17:13 +00:00
Tim Kientzle
14289cd993 Merge r659 from libarchive.googlecode.com: Correctly report "none"
as the compression name when no other read filter bid.  Add some
assertions to various tests to verify that read filters are properly
setting the textual name as well as the compression code.
2009-03-07 00:25:33 +00:00
Tim Kientzle
634fb9dd48 Merge r491,493,500,507,510,530,543 from libarchive.googlecode.com:
This implements the new generic options framework that provides a way
to override format- and compression-specific parameters.
2009-03-06 05:58:56 +00:00
Tim Kientzle
341f2011d8 Merge r390,r391,r392,r397 from libarchive.googlecode.com: Virtualize
"close" and "finish" across both read and write interfaces.
(Someday, "finish" should be renamed to "free" to better reflect
what it actually does...)
2009-03-05 21:18:10 +00:00
Tim Kientzle
facbbae9f9 Merge r364, r378, r379, r393, and r539 from libarchive.googlecode.com:
This is the last phase of the "big decompression refactor" that
puts a lazy reblocking layer between each pair of read filters.
I've also changed the terminology for this area---the two kinds
of objects are now called "read filters" and "read filter bidders"---and
moved ownership of these objects to the archive_read core.

This greatly simplifies implementing new read filters, which
can now use peek/consume I/O semantics both for bidding (arbitrary
look-ahead!) and for reading streams (look-ahead simplifies handling
concatenated streams, for instance).

The first merge here is the overhaul proper; the remainder are small
fixes to correct errors in the initial implementation.
2009-03-05 02:19:42 +00:00
Tim Kientzle
bc14277c79 Merge r282 from libarchive.googlecode.com: Close multiple filters
by walking the filter list in archive_read_close().
2009-03-03 03:33:25 +00:00
Tim Kientzle
b1ff9c25b8 MfP4: Big read filter refactoring.
This is an attempt to eliminate a lot of redundant
code from the read ("decompression") filters by
changing them to juggle arbitrary-sized blocks
and consolidate reblocking code at a single point
in archive_read.c.

Along the way, I've changed the internal read/consume
API used by the format handlers to a slightly
different style originally suggested by des@.  It
does seem to simplify a lot of common cases.

The most dramatic change is, of course, to
archive_read_support_compression_none(), which
has just evaporated into a no-op as the blocking
code this used to hold has all been moved up
a level.

There's at least one more big round of refactoring
yet to come before the individual filters are as
straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
Tim Kientzle
0b4793efb7 Remove some unused fields from the private archive_read structure
(left over from when the unified read/write structure was copied
to form separate read and write structures) and eliminate the
pointless initialization of a couple of the unused fields.
2008-03-12 04:58:32 +00:00
Dag-Erling Smørgrav
4823b3de93 Add an internal utility function to simplify the many, many places where
the number of bytes read is actually not important as long as we have at
least what we ask for.  Illustrate its benefits by using it throughout
the ZIP support code, except for the few cases where it doesn't apply.

Approved by:	kientzle
2008-01-03 17:54:26 +00:00
Tim Kientzle
9dd49f960f Update libarchive to 2.4.10. This includes a number of improvements
that I've been working on but put off committing until after the
RELENG_7 branch, including:

* New manpages: cpio.5 mtree.5
* New archive_entry_strmode()
* New archive_entry_link_resolver()
* New read support: mtree format
* Internal API change:  read format auction only runs once
* Running the auction only once allowed simplifying a lot of bid logic.
* Cpio robustness:  search for next header after a sync error
* Support device nodes on ISO9660 images
* Eliminate a lot of unnecessary copies for uncompressed archives
* Corrected handling of new GNU --sparse --posix formats
* Correctly handle a zero-byte write to a compressed archive
* Fixed memory leaks

Many of these improvements were motivated by the upcoming bsdcpio
front-end.

There have also been extensive improvements to the libarchive_test
test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
Tim Kientzle
b48b40f1f8 libarchive 2.2.3
* "compression_program" support uses an external program
  * Portability: no longer uses "struct stat" as a primary
    data interchange structure internally
  * Part of the above: refactor archive_entry to separate
    out copy_stat() and stat() functions
  * More complete tests for archive_entry
  * Finish archive_entry_clone()
  * Isolate major()/minor()/makedev() in archive_entry; remove
    these from everywhere else.
  * Bug fix: properly handle decompression look-ahead at end-of-data
  * Bug fixes to 'ar' support
  * Fix memory leak in ZIP reader
  * Portability: better timegm() emulation in iso9660 reader
  * New write_disk flags to suppress auto dir creation and not
    overwrite newer files (for future cpio front-end)
  * Simplify trailing-'/' fixup when writing tar and pax
  * Test enhancements:  fix various compiler warnings, improve
    portability, add lots of new tests.
  * Documentation: document new functions, first draft of
    libarchive_internals.3

MFC after: 14 days
Thanks to: Joerg Sonnenberger (compression_program)
Thanks to: Kai Wang (ar)
Thanks to: Colin Percival (many small fixes)
Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
Colin Percival
ceb38a3142 Move archive_read_data_into_buffer into archive_read.c, simplify its
implementation, and mark it as deprecated.  It will be removed entirely
in libarchive 3.0 (in FreeBSD 8.0?) but there's no reason for anyone to
use it instead of archive_read_data.

Approved by:	kientzle
2007-04-05 15:51:19 +00:00
Tim Kientzle
8c338614ac More corrections from Joerg Sonnenberger. 2007-04-05 05:22:13 +00:00
Tim Kientzle
7045ea1dde Fix type-punned pointer, minor style fixes.
Thanks to: Joerg Sonnenberger
2007-04-02 00:41:37 +00:00
Colin Percival
5998aba99e Provide a dummy compression-layer skip function which just reads data and
discards it, for use when the compression layer code doesn't know how to
skip data (e.g., everything other than the "none" compressor).  This makes
format level code simpler because that code can now assume that the
compression layer always knows how to skip and will always skip exactly
the requested number of bytes.

Discussed with:	kientzle (3 months ago)
2007-03-31 22:59:43 +00:00
Tim Kientzle
f81da3e584 libarchive 2.0
* libarchive_test program exercises many of the core features
  * Refactored old "read_extract" into new "archive_write_disk", which
    uses archive_write methods to put entries onto disk.  In particular,
    you can now use archive_write_disk to create objects on disk
    without having an archive available.
  * Pushed some security checks from bsdtar down into libarchive, where
    they can be better optimized.
  * Rearchitected the logic for creating objects on disk to reduce
    the number of system calls.  Several common cases now use a
    minimum number of system calls.
  * Virtualized some internal interfaces to provide a clearer separation
    of read and write handling and make it simpler to override key
    methods.
  * New "empty" format reader.
  * Corrected return types (this ABI breakage required the "2.0" version bump)
  * Many bug fixes.
2007-03-03 07:37:37 +00:00
Colin Percival
a16b1c1fd9 If (a == NULL), don't dereference (a) to record an error message. [1]
Fallout from changing the skip API to use off_t instead of size_t: Print
the skip length using %jd and cast to (intmax_t) instead of %d / (int),
and if ARCHIVE_API_VERSION >= 2, allow the client skipper to be called
for requests longer than SSIZE_MAX. [2]

Approved by:	kientzle
Pointy hats to:	kientzle [1], cperciva [2]
MFC after:	3 days
2007-02-05 16:30:40 +00:00
Tim Kientzle
a9490c597b Add support for a new archive format "empty" that reads empty files. 2007-02-01 06:18:17 +00:00
Tim Kientzle
63165a380d Fix the copyright notice; it was always intended to be
a vanilla 2-clause BSD license, but somehow some confusing
extra verbage get copied from somewhere.

Also, update the copyright dates to 2007 for all of the files.

Prompted by: several questions about what those extra words really mean
2007-01-09 08:05:56 +00:00
Colin Percival
29d7c2fce5 Quiet a compiler warning where (off_t + size_t) has a different sign
from (off_t).

Submitted by:	delphij
2007-01-03 20:01:44 +00:00
Colin Percival
dbc258608e Insert zero-padding between sparse blocks in archive_read_data(). This
fixes "tar -c @foo.tar" where "foo.tar" contains sparse entries.

MFC after:	1 week
2007-01-03 13:16:59 +00:00
Colin Percival
9d8f7dccb3 Correct spelling of "accommodate", "guarantee", "guaranteed", "guard",
"return", "ridiculous", and "success".

MFC after:	3 days
2006-12-28 05:21:48 +00:00
Tim Kientzle
aa1eeda578 Portability and style fixes:
* Actually use the HAVE_<header>_H macros to conditionally include
    system headers.  They've been defined for a long time, but only
    used in a few places.  Now they're used pretty consistently
    throughout.
  * Fill in a lot of missing casts for conversions from void*.
    Although Standard C doesn't require this, some people have been
    trying to use C++ compilers with this code, and they do require it.

Bit-for-bit, the compiled object files are identical, except for
one assert() whose line number changed, so I'm pretty confident I
didn't break anything.  ;-)
2006-11-10 06:39:46 +00:00
Tim Kientzle
c12a9d810e Some minor corrections:
* Expose functions for setting the "skip file" dev/ino information
  * Expose functions for setting/querying the block size on reads
  * Correctly propagate errors out of archive_read_close/archive_write_close
  * Update manpage with information about new functions
2006-09-05 05:59:46 +00:00
Tim Kientzle
693285bc87 Use 'skip' when ignoring data in tar archives. This dramatically
increases performance when extracting a single entry from a large
uncompressed archive, especially on slow devices such as USB hard
drives.

Requires a number of changes:
   * New archive_read_open2() supports a 'skip' client function
   * Old archive_read_open() is implemented as a wrapper now, to
     continue supporting the old API/ABI.
   * _read_open_fd and _read_open_file sprout new 'skip' functions.
   * compression layer gets a new 'skip' operation.
   * compression_none passes skip requests through to client.
   * compression_{gzip,bzip2,compress} simply ignore skip requests.

Thanks to: Benjamin Lutz, who designed and implemented the whole thing.
   I'm just committing it.  ;-)

TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
Tim Kientzle
752ede3058 If the attempt to open the archive fails (either the client open
routine fails or the first read fails), invoke the client close
routine immediately so the client can clean up.  Also, don't store the
client pointers in this case, so that the client close routine can't
accidentally get called more than once.

A minor style fix to archive_read_open_fd.c while I'm here.

PR: 86453
Thanks to: Andrew Turner for reporting this and suggesting a fix.
2006-01-17 04:49:04 +00:00
Tim Kientzle
c4e21983bc signed/unsigned fixes (thanks to GCC4) and a few related minor style corrections. 2005-09-24 21:15:00 +00:00
Tim Kientzle
3be7274eed In archive_read_open(), do not set the internal archive state to
"HEADER" unless the open is successful.  Instead, leave the state as
"NEW."  In particular, if archive_read_open() fails, a subsequent call
to archive_read_next_header() will now cause an explicit assertion
failure instead of a silent segmentation fault.

This may need a little more work to fully realize the intention: If
archive_read_open() fails, you should be able to call it again on the
same archive handle to open a different archive (or the same archive
using a different mechanism).
2005-09-21 04:48:52 +00:00
Tim Kientzle
8aaa8fe733 Add a lot of error checks, based on the patches provided by Dan Lukes.
Also fixes a memory leak reported by Andrew Turner.

PR: bin/83476
Thanks to: Dan Lukes, Andrew Turner
2005-09-21 04:25:06 +00:00
Tim Kientzle
9b98b2d5d1 Fix some errors in archive_read_data that caused failures in bsdtar's
pass-through filtering.

Thanks to: Bjoern Koenigönig
PR: bin/82878
2005-09-10 18:05:54 +00:00
Tim Kientzle
0f412b97da Remove the C99-specific __func__ that is one of the few barrier to
compiling on IRIX and Solaris.  Remove the "archive_check_magic" macro
that existed only to provide __func__ to the underlying __archive_check_magic
function.

Thanks to: Darin Broady
MFC after: 14 days
2005-06-01 15:52:39 +00:00
Tim Kientzle
81a4ac6ddb A number of improvements to ZIP support.
* Handles entries with compressed size >2GB (signed/unsigned cleanup)
  * Handles entries with compressed size >4GB ("ZIP64" extension)
  * Handles Unix extensions (ctime, atime, mtime, mode, uid, etc)
  * Format-specific "skip data" override allows ZIP reader to skip
    entries without decompressing them, which makes "tar -t"
    a lot faster.
  * Handles "length-at-end" entries generated by, e.g., "zip -r - foo"

Many thanks to: Dan Nelson, who contributed the code and test files for
   the first three items above and suggested the fourth.
2005-04-06 04:19:30 +00:00
Tim Kientzle
49f9f01a7f Clear the error buffer on entry
to archive_read_next_header so the
next error doesn't just get appended.

MFC after: 7 days
2005-01-08 18:28:11 +00:00
Tim Kientzle
57b665990a Eliminate reliance on non-portable <err.h> by implementing a very
simple errx() function.
Improve behavior when bzlib/zlib are missing by detecting and
issuing an error message on attempts to read gzip/bzip2 compressed
archives.
2004-08-14 03:45:45 +00:00
Tim Kientzle
91ce8f27d8 Split archive_{read,write}_finish into separate "close" (finish the archive
and close it) and "finish" (destroy the object) functions.   For backwards
compat and simplicity, have "finish" invoke "close" transparently if needed.
This allows clients to close the archive and check end-of-operation
statistics before destroying the object.
2004-08-07 19:22:50 +00:00
Tim Kientzle
2b68caa0f2 Correctly reset archive_read_data state everytime a header is read. 2004-06-04 23:25:20 +00:00
Tim Kientzle
1a74b99db7 Be more careful about the initial read (used for "tasting" the compression):
* Check for and return input errors
  * Treat empty file (zero-length read) as a fatal error
2004-06-04 01:36:10 +00:00
Tim Kientzle
a8659f8468 Refactor the extraction code somewhat. In particular,
push extract data down into archive_read_extract.c and out
of the library-global archive_private.h; push dir-specific
mode/time fixup down into dir restore function; now that the
fixup list is file-local, I can use somewhat more natural
naming.

Oh, yeah, update a bunch of comments to match current reality.
2004-06-03 23:29:47 +00:00
Tim Kientzle
e250dd4fad Refactor read_data:
* New read_data_block is both sparse-file aware and uses zero-copy semantics
 * Push read_data_block down into specific formats (opens door to
   various encoded entry bodies, such as zip or gtar -S)
 * Reimplement read_data, read_data_skip, read_data_into_fd in terms
   of new read_data_block.
 * Update documentation
It's unfortunate that I couldn't just call the new interface
archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
Tim Kientzle
d911e48507 * Plug a buffer overrun in ACL parsing. (archive_entry.c)
* Re-use a single buffer for shar output formatting rather
   than hammering the heap. (archive_write_set_format_shar.c)
 * Fix a handful of minor memory leaks and clean up some of the
   memory-management code.
2004-04-13 23:45:37 +00:00
Tim Kientzle
aee47dd7c8 More work on ACLs: fix error in archive_entry's ACL parsing code,
try to set ACLs even if fflag restore fails, first cut at reading
  Solaris tar ACLs

Code improvement: merge gnu tar read support into main tar reader;
  this eliminates a lot of duplicate code and generalizes the tar
  reader to handle formats with GNU-like extensions.

Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype'
  from archive_entry (this makes archive_entry more format-agnostic)

Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
Tim Kientzle
71b44796d9 Overhauled ACL support. This makes us compatible
with 'star' ACL handling, though there's still a
bit more work needed in this area.

Added 'write_open_fd' and 'read_open_fd' to simplify, e.g.,
tar's u and r modes.  Eliminated old 'write_open_file_position'
as a bad idea.  (It required closing/reopening files to
do updates, which led to unpleasant implications.)

Various other minor fixes, API tweaks, etc.
2004-04-05 21:12:29 +00:00
Tim Kientzle
44a3d34206 Many fixes:
* Disabled shared-library building, as some API breakage is
  still likely.  (I didn't realize it was turned on by default.)  If
  you have an existing /usr/lib/libarchive.so.2, I recommend deleting it.
* Pax interchange format now correctly stores and reads UTF8
  for extended attributes.  In particular, pax format can portably
  handle arbitrarily long pathnames containing arbitrary characters.
* Library compiles cleanly at -O2, -O3, and WARNS=6 on all
  FreeBSD-CURRENT platforms.
* Minor portability improvements inspired by Juergen Lock
  and Greg Lewis.  (Less reliance on stdint.h, isolating of
  various portability-challenged constructs.)
* archive_entry transparently converts multi-byte <-> wide character
  strings, allowing clients and format handlers to deal with either
  one, as appropriate.
* Support for reading 'L' and 'K' entries in standard tar archives
  for star compatibility.
* Recognize (but don't yet handle) ACL entries from Solaris tar.
* Pushed format-specific data for format readers down into
  format-specific storage and out of library-global storage.  This
  should make it easier to maintain individual formats without mucking
  with the core library management.
* Documentation updates to track the above changes.
* Updates to tar.5 to correct a few mistakes and add some additional
  information about GNU tar and Solaris tar formats.

Notes:
* The basic 'tar' reader is getting more general; there's not much
  point in keeping the 'gnutar' reader separate.  Merging the two
  would lose a bunch of duplicate code.
* The libc ACL support is looking increasingly inadequate for my needs
  here.  I might need to assemble some fairly significant code for
  parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Tim Kientzle
df3c1316b0 Many fixes.
Portability: Thanks to Juergen Lock, libarchive now compiles cleanly
on Linux.  Along the way, I cleaned up a lot of error return codes and
reorganized some code to simplify conditional compilation of certain
sections.

Bug fixes:
  * pax format now actually stores filenames that are 101-154
    characters long.
  * pax format now allows newline characters in extended attributes
    (this fixes a long-standing bug in ACL handling)
  * mtime/atime are now restored for directories
  * directory list is now sorted prior to fix-up to permit
    correct restore of non-writable dir heirarchies
2004-03-09 19:50:41 +00:00
Tim Kientzle
2710e4d1ef Initial import of libarchive.
What it is:
   A library for reading and writing various streaming archive
   formats, especially tar and cpio.  Being a library, it should
   be easy to incorporate into pkg_* tools, sysinstall, and any
   other place that needs to read or write such archives.

Features:
  * Full automatic detection of both compression and archive format.
  * Extensible internal architecture to make it easy to add new formats.
  * Support for "pax interchange format," a new POSIX-standard tar format
    that eliminates essentially all of the restrictions of historic formats.
  * BSD license

Thanks to: jkh for pushing me to start this work, gordon for
  encouraging me to commit it, bde for answering endless style
  questions, and many others for feedback and encouragement.

Status: Pretty good overall, though there are still a few rough edges and
  the library could always use more testing.  Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00