Commit Graph

34 Commits

Author SHA1 Message Date
Tim Kientzle
5e9641ba0a Merge r658 from libarchive.googlecode.com: Only flush and close the
file if it was actually opened.  Test for this case.
2009-03-07 02:09:21 +00:00
Tim Kientzle
ce077a6fd8 Merge r416 from libarchive.googlecode.com:
Restoring POSIX.1e Extended Attributes on FreeBSD, part 1

This implements the basic ability to restore extended attributes
on FreeBSD, including a test suite.
2009-03-06 04:55:51 +00:00
Tim Kientzle
43a8e5f098 Merge r399,401,402,405,415,430,440,452,453,458,506,533,536,538,544,590
from libarchive.googlecode.com:  Add a new "archive_read_disk" API
that provides the important service of reading metadata from the
disk.  In particular, this will make it possible to remove all
knowledge of extended attributes, ACLs, etc, from clients such
as bsdtar and bsdcpio.

Closely related, this API also provides pluggable uid->uname
and gid->gname lookup and caching services similar to
the uname->uid and gname->gid services provided by archive_write_disk.
Remember this is also required for correct ACL management.

Documentation is still pending...
2009-03-06 04:35:31 +00:00
Tim Kientzle
7376ba1329 Fix ARCHIVE_EXTRACT_SPARSE handling in libarchive.
Add a test to exercise this feature.
This should fix --sparse/-S support in tar.

Thanks to:      Daichi GOTO
MFC after:	1 week
2009-01-26 05:44:40 +00:00
Tim Kientzle
53272655f6 Merge-from-Googlecode r419
In archive_write_disk:  If archive_write_header() fails to create
the file, that's a failure and should return ARCHIVE_FAILED.
Metadata restore failures still return ARCHIVE_WARN, because
that's non-critical.  Fix test_write_disk_secure test to
verify the correct return code in one case; add test_write_disk_failures
to do another very simple test of restore failure.

This should fix cpio coredumping when it tries to restore to
a write-protected directory.

Thanks to: Giorgos Keramidas
MFC after: 30 days
2009-01-21 06:55:27 +00:00
Tim Kientzle
dff5b8bc1f New tests:
* support for bzip2 file with multiple concatenated bzip2 streams
 * support for bzip2 file with junk after bzip2 stream
 * support for gzip file with junk after gzip stream
 * "fuzz" tester randomly modifies a bunch of input files in order to try
   to crash libarchive (this found an amusing hang in the ISO9660 code
   when trying to read images that advertised a zero blocksize).

This test is implemented, but commented out for now:
 * support for gzip file with multiple concatenated gzip streams
2008-12-06 07:08:08 +00:00
Tim Kientzle
758dfd70db Add test to verify ACL restores on FreeBSD, verify kern/128203 (mtime
restore fails if ACL provided), apply patch from kern/128203, and verify fix.

PR:		kern/128203
Submitted by:	Udo Schweigert
MFC after:	3 days
2008-10-19 00:18:44 +00:00
Tim Kientzle
f70105fb18 MfP4: restore birth time data to disk + more thorough tests for
time restore to disk.

MFC after:	30 days
2008-09-30 04:02:36 +00:00
Tim Kientzle
155524db13 MfP4: Store/read birthtime data in pax format.
Submitted by:	Pedro Giffuni
MFC after:	30 days
2008-09-30 03:57:07 +00:00
Tim Kientzle
a2a72fd640 Fix a type-aliasing problem in the libarchive test suite,
restore WARNS=6.
2008-09-08 00:58:12 +00:00
Tim Kientzle
39056f55e3 Alphabetize the list of tests. 2008-08-25 06:08:22 +00:00
Dag-Erling Smørgrav
c7dd6fa2c9 Some tests won't build at WARNS level 6 due to aliasing violations.
Add missing -I. so the tests will build when ${.OBJDIR} != ${.CURDIR}.
${.OBJDIR} does not need to be spelled out.
2008-06-26 11:58:26 +00:00
Dag-Erling Smørgrav
e2157b51de Allow the tests to build without libdmalloc. 2008-06-26 10:53:05 +00:00
Tim Kientzle
f9fe0a0abb archive.h is no longer constructed from archive.h.in,
so we can rename it and drop some no-longer-necessary
build magic from the Makefile.
2008-06-15 05:05:53 +00:00
Tim Kientzle
a212de8851 Until the old archive.h.in gets renamed to archive.h in the repository,
we still need some Makefile trickery to ensure archive.h is
correctly built for the test harness.
2008-05-27 04:12:17 +00:00
Tim Kientzle
fa07de5eeb MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more
feedback, but the 2.5 branch is shaping up nicely.)

In addition to many small bug fixes and code improvements:
 * Another iteration of versioning; I think I've got it right now.
 * Portability:  A lot of progress on Windows support (though I'm
   not committing all of the Windows support files to FreeBSD CVS)
 * Explicit tracking of MBS, WCS, and UTF-8 versions of strings
   in archive_entry; the archive_entry routines now correctly return
   NULL only when something is unset, setting NULL properly clears
   string values.  Most charset conversions have been pushed down to
   archive_string.
 * Better handling of charset conversion failure when writing or
   reading UTF-8 headers in pax archives
 * archive_entry_linkify() provides multiple strategies for
   hardlink matching to suit different format expectations
 * More accurate bzip2 format detection
 * Joerg Sonnenberger's extensive improvements to mtree support
 * Rough support for self-extracting ZIP archives.  Not an ideal
   approach, but it works for the archives I've tried.
 * New "sparsify" option in archive_write_disk converts blocks of nulls
   into seeks.
 * Better default behavior for the test harness; it now reports
   all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
Tim Kientzle
cde1a05218 GC a reference to the defunct TESTFILES variable. 2008-03-15 02:22:08 +00:00
Tim Kientzle
60617bf578 A subtle point: "pax interchange format" mandates that all strings
(including pathname, gname, uname) be stored in UTF-8.  This usually
doesn't cause problems on FreeBSD because the "C" locale on FreeBSD
can convert any byte to Unicode/wchar_t and from there to UTF-8.  In
other locales (including the "C" locale on Linux which is really
ASCII), you can get into trouble with pathnames that cannot be
converted to UTF-8.

Libarchive's pax writer truncated pathnames and other strings at the
first nonconvertible character.  (ouch!)  Other archivers have worked
around this by storing unconvertible pathnames as raw binary, a
practice which has been sanctioned by the Austin group.  However,
libarchive's pax reader would segfault reading headers that weren't
proper UTF-8.  (ouch!)  Since bsdtar defaults to pax format, this
affects bsdtar rather heavily.

To correctly support the new "hdrcharset" header that is going into
SUS and to handle conversion failures in general, libarchive's pax reader
and writer have been overhauled fairly extensively.  They used to do
most of the pax header processing using wchar_t (Unicode); they now do
most of it using char so that common logic applies to either UTF-8 or
"binary" strings.

As a bonus, a number of extraneous conversions to/from wchar_t have
been eliminated, which should speed things up just a tad.

Thanks to: Bjoern Jacke for originally reporting this to me
Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this
Thanks to: Gunnar Ritter for getting the standard fixed
MFC after: 5 days
2008-03-15 01:43:59 +00:00
Tim Kientzle
7c5b1173a5 Support for writing 'compress' format, thanks to Joerg Sonnenberger. 2008-03-14 20:35:38 +00:00
Tim Kientzle
df4691b984 Portability: Eliminate the need for uudecode by incorporating
uudecode into the main test driver and invoking it just-in-time
within the various tests.

Also, incorporate a number of improvements to the main test support
code that have proven useful on other projects where I've used this
framework.
2008-03-12 05:12:23 +00:00
Tim Kientzle
fae7e96d82 Add a test to verify compatibility with archives with
odd hardlinks.  I need to extend this to test pax extended
archives with bodies attached to hardlinks and other less-common cases.
2008-01-31 07:47:38 +00:00
Tim Kientzle
2adbd7ee43 Issues with hardlinks in newc-format files prompted me to
write a new test to exercise the hardlink strategies used
by different archive formats (tar, old cpio, new cpio).
This uncovered two problems, both fixed by this commit:

1) Enforce file size when writing files to disk.

2) When restoring hardlink entries, if they have data associated, go
   ahead and open the file so we can write the data.

In particular, this fixes bsdtar/bsdcpio extraction of new cpio
formats where the "original" is empty and the subsequent "hardlink"
entry actually carries the data.  It also provides correct behavior
for old cpio archives where hardlinked entries have their bodies
stored multiple times in the archive; the last body should always be
the one that ends up in the final file.  The new pax format also
permits (but does not require) hardlinks to carry file data; again,
the last contents should always win.

Note that with any of these, a size of zero on a hardlink simply means
that the hardlink carries no data; it does not mean that the file has
zero size.  A non-zero size on a hardlink does provide the file size.

Thanks to: John Baldwin, for reminding me about this long-standing bug
    and sending me a simple example archive that prompted this test case
2008-01-18 05:05:58 +00:00
Tim Kientzle
2a5e8d812c Extensive improvements to the libarchive_test test program that
exercises and verifies the libarchive APIs:

* Improved error reporting; hexdumps are now provided for
  many file/memory content differences.
* Overall status more clearly counts "tests" and "assertions"
* Reference files can now be stored on disk instead of having
  to be compiled into the test program itself.  A couple of
  tests have been converted to this more natural structure.
* Several memory leaks corrected so that leaks within libarchive
  itself can be more easily detected and diagnosed.
* New test: GNU tar compatibility
* New test: Zip compatibility
* New test: Zero-byte writes to a compressed archive entry
* New test: archive_entry_strmode() format verification
* New test: mtree reader
* New test: write/read of large (2G - 1TB) entries to tar archives
  (thanks to recent performance work, this test only requires a few seconds)
* New test: detailed format verification of cpio odc and newc writers
* Many minor additions/improvements to existing tests as well.
2008-01-01 22:28:04 +00:00
Tim Kientzle
ceab112df6 New test suite test_read_pax_truncated probes libarchive
behavior with truncated or damaged pax archives.  This
tests most of the cases covered by the recent security advisory.

Approved by: re (blanket, libarchive test suite)
2007-07-13 15:14:35 +00:00
Tim Kientzle
8ed6656123 New file "read_open_memory.c" is a custom variant of
archive_read_open_memory.c that tries to test border
cases.  In particular, it copies over each returned block
so that formats or decompressors that read past the end
of a returned block will break.

Approved by: re (blanket, libarchive test suite)
2007-07-13 15:12:52 +00:00
Tim Kientzle
734f944a06 Ouch. I partially screwed up the last commit by
enabling a test that's not ready yet. <sigh>

Pointy hat: /me
Approved by: re@
2007-06-22 05:49:13 +00:00
Tim Kientzle
17e60e6230 Support for writing the 'newc' cpio format, plus a minimal test harness
for the cpio formats.

Thanks to: Rudolf Marek
Approved by: re@
2007-06-22 05:47:00 +00:00
Tim Kientzle
0ddfde5d16 Read support for the new GNU tar sparse formats added in gtar 1.15 and
gtar 1.16.
2007-06-13 03:35:37 +00:00
Stephane E. Potvin
04031e9ae2 Options spring cleanup:
- Add and document the KVM and KVM_SUPPORT options that
are needed for the ifmcstats(3) makefile
- Garbage collect unused variables
- Add missing inclusion of bsd.own.mk where needed

Approved by: kan (mentor)
Reviewed by: ru
2007-06-13 02:08:04 +00:00
Tim Kientzle
b48b40f1f8 libarchive 2.2.3
* "compression_program" support uses an external program
  * Portability: no longer uses "struct stat" as a primary
    data interchange structure internally
  * Part of the above: refactor archive_entry to separate
    out copy_stat() and stat() functions
  * More complete tests for archive_entry
  * Finish archive_entry_clone()
  * Isolate major()/minor()/makedev() in archive_entry; remove
    these from everywhere else.
  * Bug fix: properly handle decompression look-ahead at end-of-data
  * Bug fixes to 'ar' support
  * Fix memory leak in ZIP reader
  * Portability: better timegm() emulation in iso9660 reader
  * New write_disk flags to suppress auto dir creation and not
    overwrite newer files (for future cpio front-end)
  * Simplify trailing-'/' fixup when writing tar and pax
  * Test enhancements:  fix various compiler warnings, improve
    portability, add lots of new tests.
  * Documentation: document new functions, first draft of
    libarchive_internals.3

MFC after: 14 days
Thanks to: Joerg Sonnenberger (compression_program)
Thanks to: Kai Wang (ar)
Thanks to: Colin Percival (many small fixes)
Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
Tim Kientzle
015f35775b Conventionally, tar archives have always included a trailing '/'
for directories.  bsdtar used to add this, but that recently got
lost somehow.  So now I'm adding it back in libarchive.
The only odd part of doing this in libarchive:  Adding a directory to
a tar archive and then reading it back again can yield a different name.

Add a test case to exercise some boundary conditions with
tar filenames and ensure that trailing slashes are added to
dir names only as necessary.

Thanks to: Oliver Lehmann for bringing this regression to my attention.
2007-04-14 08:20:31 +00:00
Tim Kientzle
94451fa8a0 Enable 'ar' support; hook it up to the build and
enable it with _read_support_format_all().
2007-04-07 05:54:23 +00:00
Tim Kientzle
d4c6c416dc New tests for system-independent ACL support.
These tests verify that archive_entry objects can store and return
ACL data and that pax format archives can read and write ACL
information.  These do not (yet) test that ACL data is read or
written to disk correctly.  (And hence would not have caught the
recent snafu about ACL read-from-disk being turned off.)
2007-03-08 06:09:27 +00:00
Tim Kientzle
f81da3e584 libarchive 2.0
* libarchive_test program exercises many of the core features
  * Refactored old "read_extract" into new "archive_write_disk", which
    uses archive_write methods to put entries onto disk.  In particular,
    you can now use archive_write_disk to create objects on disk
    without having an archive available.
  * Pushed some security checks from bsdtar down into libarchive, where
    they can be better optimized.
  * Rearchitected the logic for creating objects on disk to reduce
    the number of system calls.  Several common cases now use a
    minimum number of system calls.
  * Virtualized some internal interfaces to provide a clearer separation
    of read and write handling and make it simpler to override key
    methods.
  * New "empty" format reader.
  * Corrected return types (this ABI breakage required the "2.0" version bump)
  * Many bug fixes.
2007-03-03 07:37:37 +00:00