Commit Graph

798 Commits

Author SHA1 Message Date
Tim Kientzle
a86d4001dd Merge r420,r494 from libarchive.googlecode.com: Prettify the test
harness a bit:  remove a dead comment, tweak the wording of the
summary report.
2009-03-06 05:07:03 +00:00
Tim Kientzle
6a7c3e70e3 Merge r435,r443 from libarchive.googlecode.com: Read config files from
include path; this makes it easier to support multiple build frameworks.
2009-03-06 05:04:15 +00:00
Tim Kientzle
ce077a6fd8 Merge r416 from libarchive.googlecode.com:
Restoring POSIX.1e Extended Attributes on FreeBSD, part 1

This implements the basic ability to restore extended attributes
on FreeBSD, including a test suite.
2009-03-06 04:55:51 +00:00
Tim Kientzle
17fcda1a0b Merge r407,r508 from libarchive.googlecode.com: Correctly handle
Zip entries that are zero length but stored with deflate.  This
is arguably a silly thing to do (deflating a zero-length file actually
makes it bigger) but apparently quite a few Zip writers do this.
This was broken in two places:  archive_write_disk disliked being asked
to write data to zero-length files (even if the write was zero-length)
and zip_read_file_header tripped over itself when non-regular files
had compressed bodies.
2009-03-06 04:50:39 +00:00
Tim Kientzle
43a8e5f098 Merge r399,401,402,405,415,430,440,452,453,458,506,533,536,538,544,590
from libarchive.googlecode.com:  Add a new "archive_read_disk" API
that provides the important service of reading metadata from the
disk.  In particular, this will make it possible to remove all
knowledge of extended attributes, ACLs, etc, from clients such
as bsdtar and bsdcpio.

Closely related, this API also provides pluggable uid->uname
and gid->gname lookup and caching services similar to
the uname->uid and gname->gid services provided by archive_write_disk.
Remember this is also required for correct ACL management.

Documentation is still pending...
2009-03-06 04:35:31 +00:00
Tim Kientzle
74aced1051 Merge r398 from libarchive.googlecode.com: Check that bidder
object was allocated before we try to use it.
2009-03-06 04:22:34 +00:00
Tim Kientzle
c6b235629a Merge r394,r396 from libarchive.googlecode.com: Plug some memory
leaks in the ACL test, correctly mark that FreeBSD has acl_get_perm_np().
2009-03-06 04:21:23 +00:00
Tim Kientzle
341f2011d8 Merge r390,r391,r392,r397 from libarchive.googlecode.com: Virtualize
"close" and "finish" across both read and write interfaces.
(Someday, "finish" should be renamed to "free" to better reflect
what it actually does...)
2009-03-05 21:18:10 +00:00
Tim Kientzle
90e7486664 Merge r389 from libarchive.googlecode.com: Fix a memory
leak in ISO9660 handler structure whenever a file entry
has a nonsensical CE offset.
2009-03-05 18:38:36 +00:00
Tim Kientzle
e80e6f7018 Merge r386,r395,r451 from libarchive.googlecode.com: On Windows, break
into the debugger on test setup failures (otherwise, the console window
just goes away and you can't see what went wrong).  On all platforms,
clean up a stray buffer before exiting.
2009-03-05 18:30:50 +00:00
Tim Kientzle
ffd201719e Argh. r189389 was supposed to include r539 from libarchive.googlecode.com
but those compile fixes somehow got lost.  This should fix the build.
2009-03-05 06:26:08 +00:00
Tim Kientzle
241028d43a Merge r551,r561 from libarchive.googlecode.com: Update gzip read filter
to fully take advantage of the new peek/consume I/O support.
In particular, this now properly handles concatenated gzip streams.
2009-03-05 02:37:05 +00:00
Tim Kientzle
facbbae9f9 Merge r364, r378, r379, r393, and r539 from libarchive.googlecode.com:
This is the last phase of the "big decompression refactor" that
puts a lazy reblocking layer between each pair of read filters.
I've also changed the terminology for this area---the two kinds
of objects are now called "read filters" and "read filter bidders"---and
moved ownership of these objects to the archive_read core.

This greatly simplifies implementing new read filters, which
can now use peek/consume I/O semantics both for bidding (arbitrary
look-ahead!) and for reading streams (look-ahead simplifies handling
concatenated streams, for instance).

The first merge here is the overhaul proper; the remainder are small
fixes to correct errors in the initial implementation.
2009-03-05 02:19:42 +00:00
Tim Kientzle
a48ae5111e Correct r189383, which mis-merged a change from libarchive.googlecode.com. 2009-03-05 01:59:49 +00:00
Tim Kientzle
30b7287aff Merge r356 and r358 from libarchive.googlecode.com: Remove a Windows
special case from archive_entry.c, add one to archive_check_magic.c.
2009-03-05 00:44:12 +00:00
Tim Kientzle
a60167b66c Merge r357 from libarchive.googlecode.com: bzip2 compression
support can always be enabled even if bzlib doesn't exist on
this platform; don't give up until we fail to open the file.
2009-03-05 00:42:50 +00:00
Tim Kientzle
24b4dd3387 Merge r362 from libarchive.googlecode.com: Minor fix to
custom argument parser.
2009-03-05 00:41:02 +00:00
Tim Kientzle
32baf20434 Merge r342 from libarchive.googlecode.com: Remove some
Windows special casing.
2009-03-05 00:36:13 +00:00
Tim Kientzle
faeada5e26 Merge r344 from libarchive.googlecode.com: __LA_INT64_T and __LA_SSIZE_T
are part of the public API and therefore need to be exposed.  This is
ugly; I'd like to find a better solution for this.
2009-03-05 00:35:21 +00:00
Tim Kientzle
ba2a6a7201 Merge r341,r345,r346,347 from libarchive.googlecode.com: Style
fixes to test harness and a few extra guards to detect tests
that can't succeed on certain platforms.
2009-03-05 00:31:48 +00:00
Tim Kientzle
9c886c4181 Merge r340 from libarchive.googlecode.com: If zlib/bzlib aren't available,
we can still detect gzip/bzip2 compressed streams, we just can't
decompress them.
2009-03-03 17:07:27 +00:00
Tim Kientzle
690f818afd Merge r294:337,r348:350 from libarchive.googlecode.com: A lot
of work to make libarchive work on Windows.
2009-03-03 17:02:51 +00:00
Tim Kientzle
ed466ea1cf Merge r294 from libarchive.googlecode.com: Skip testing for
locale-based failures on systems where the "C" locale is so permissive
that it cannot possibly fail.  In particular, this fixes a test
problem on Cygwin.
2009-03-03 07:01:57 +00:00
Tim Kientzle
bc14277c79 Merge r282 from libarchive.googlecode.com: Close multiple filters
by walking the filter list in archive_read_close().
2009-03-03 03:33:25 +00:00
Tim Kientzle
80bb6bf259 Merge r272 from libarchive.googlecode.com: Fix building on MSVC6. 2009-03-03 03:28:09 +00:00
Tim Kientzle
56b5addbbd Permit ` as a pad character in the filename table.
This seems to fix the devel/zziplib port, which distributes
its man pages in an ar archive.
2009-02-01 02:33:02 +00:00
Tim Kientzle
7376ba1329 Fix ARCHIVE_EXTRACT_SPARSE handling in libarchive.
Add a test to exercise this feature.
This should fix --sparse/-S support in tar.

Thanks to:      Daichi GOTO
MFC after:	1 week
2009-01-26 05:44:40 +00:00
Tim Kientzle
53272655f6 Merge-from-Googlecode r419
In archive_write_disk:  If archive_write_header() fails to create
the file, that's a failure and should return ARCHIVE_FAILED.
Metadata restore failures still return ARCHIVE_WARN, because
that's non-critical.  Fix test_write_disk_secure test to
verify the correct return code in one case; add test_write_disk_failures
to do another very simple test of restore failure.

This should fix cpio coredumping when it tries to restore to
a write-protected directory.

Thanks to: Giorgos Keramidas
MFC after: 30 days
2009-01-21 06:55:27 +00:00
Tim Kientzle
3342e45402 "The first part is just to give more info, the latter part fixes
an error to read files past the 32bit byte offset, for instance
on DVDs."

Submitted by:	phk@
MFC after:	10 days
2009-01-13 04:56:41 +00:00
Tim Kientzle
a612d4d7ae Don't try to read the next Gzip header after we reach the
end of the compressed stream.  This is desirable behavior,
but the implementation here is very broken and causes strange
problems, so disable it for now.

Thanks to Simon L. Nielsen for reporting this problem.
2009-01-01 02:29:57 +00:00
Tim Kientzle
929b60ba42 If conversion from UTF8 fails, don't mark Unicode text as available.
Submitted by:	Michihiro NAKAJIMA
MFC after:	30 days
2008-12-23 05:01:43 +00:00
Tim Kientzle
fec90b6c9b Teach get_refdir() about FreeBSD's /usr/obj convention.
In development, I run libarchive_test frequently by hand
and it gets tedious having to specify a suitable -r path
all of the time.
2008-12-21 00:13:50 +00:00
Tim Kientzle
514c68e92c Fill in a missing const 2008-12-17 19:05:58 +00:00
Tim Kientzle
eef2284331 Exit earlier on failure in this test. 2008-12-17 19:05:25 +00:00
Tim Kientzle
9515da8712 Once the test has failed, exit before the segfault. 2008-12-17 19:05:00 +00:00
Tim Kientzle
da9157722d Update the ar write test to give more detailed information about failures. 2008-12-17 19:03:44 +00:00
Tim Kientzle
6ebec34c8d Clarify an 'ar' error message. 2008-12-17 19:02:42 +00:00
Tim Kientzle
18f815b95c Obey the TMPDIR, TMP, TEMP, or TEMPDIR environment variables
when choosing a scratch directory for the tests.  Fallback
to "/tmp", of course.
2008-12-08 17:22:44 +00:00
Tim Kientzle
dff5b8bc1f New tests:
* support for bzip2 file with multiple concatenated bzip2 streams
 * support for bzip2 file with junk after bzip2 stream
 * support for gzip file with junk after gzip stream
 * "fuzz" tester randomly modifies a bunch of input files in order to try
   to crash libarchive (this found an amusing hang in the ISO9660 code
   when trying to read images that advertised a zero blocksize).

This test is implemented, but commented out for now:
 * support for gzip file with multiple concatenated gzip streams
2008-12-06 07:08:08 +00:00
Tim Kientzle
dc8cb157dd Strip ";1" and trailing "." from ISO9660 entries.
This seems a better match for people's expectations.
2008-12-06 06:57:45 +00:00
Tim Kientzle
08da6f539c General improvements to Rockridge parsing and ISO9660 format detection. 2008-12-06 06:55:07 +00:00
Tim Kientzle
ebcb29a003 Conditionalize a bunch of debugging messages; this also
eliminates what should be the only remaining stdio dependency.
2008-12-06 06:50:09 +00:00
Tim Kientzle
b1ff9c25b8 MfP4: Big read filter refactoring.
This is an attempt to eliminate a lot of redundant
code from the read ("decompression") filters by
changing them to juggle arbitrary-sized blocks
and consolidate reblocking code at a single point
in archive_read.c.

Along the way, I've changed the internal read/consume
API used by the format handlers to a slightly
different style originally suggested by des@.  It
does seem to simplify a lot of common cases.

The most dramatic change is, of course, to
archive_read_support_compression_none(), which
has just evaporated into a no-op as the blocking
code this used to hold has all been moved up
a level.

There's at least one more big round of refactoring
yet to come before the individual filters are as
straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
Tim Kientzle
72a4c788ff Style: tabs after #define 2008-12-06 06:23:37 +00:00
Tim Kientzle
da28fde4f9 Use GCC-specific dead function markers only when using GCC. 2008-12-06 06:20:21 +00:00
Tim Kientzle
9d822fe4b1 Structure declarations can't be dllexported. 2008-12-06 06:18:46 +00:00
Tim Kientzle
485698c170 Style fixes:
* Wrap long declarations to fit 80 chars
 * #undef macros that shouldn't be exported
 * Organize the version-dependent conditionals a
   bit more consistently

Speculative:
 * libarchive 3.0 will (eventually) use int64_t
   instead of off_t.  This is an attempt to avoid
   some the headaches caused by Linux LFS.  (I'll
   still have to do ugly things for the struct stat
   references in archive_entry.h, of course.)
2008-12-06 06:17:18 +00:00
Tim Kientzle
0373bf6531 A couple of portability fixes from Joerg Sonnenberger 2008-12-06 06:12:24 +00:00
Tim Kientzle
605fc11462 Recover after test failure. 2008-12-06 06:02:26 +00:00
Tim Kientzle
796b668980 Style: skipping() function correctly counts this as a skipped test. 2008-12-06 06:01:50 +00:00
Tim Kientzle
f1bcf83200 Style improvements. 2008-12-06 06:00:52 +00:00
Tim Kientzle
39bb5e0dd0 Better reporting of test failures. 2008-12-06 05:59:46 +00:00
Tim Kientzle
23fb0d7a7c Recover from a test failure here. 2008-12-06 05:58:24 +00:00
Tim Kientzle
0a68927a88 New internal archive_string_concat utility function. 2008-12-06 05:56:43 +00:00
Tim Kientzle
fe7350cb22 Minor portability: Declare 'r' only within the conditional
block where it's used; S_ISLNK is more portable than S_ISLINK.
2008-12-06 05:55:46 +00:00
Tim Kientzle
b55ef4a75a Windows build now uses PLATFORM_CONFIG_H macro, so we can eliminate the
special case for it.
2008-12-06 05:53:05 +00:00
Tim Kientzle
a8d0992f23 2 is more portable than STDERR_FILENO 2008-12-06 05:52:01 +00:00
Tim Kientzle
9fe9800cb5 When running on a filesystem that lacks ACL support,
just SKIP the test, don't report a test failure.
2008-11-17 21:06:17 +00:00
Tim Kientzle
92ab0af8a3 Correct the documented declaration of the archive_write_callback to
match the code.

PR:		docs/128089
Submitted by:	Mel
MFC after:	3 days
2008-11-01 19:11:21 +00:00
Dag-Erling Smørgrav
c1f91a825f Move va_end() up 2008-10-21 12:10:30 +00:00
Tim Kientzle
ee8a341ef4 If we don't know the file size, leave the file size unset.
If it's not a regular file, don't return any data, even if the size is unknown.

Update the Zip test with a hand-tweaked Zip archive that has a
directory (with length-at-end set), a regular file without
length-at-end set, and a regular file with length-at-end set and a bad
CRC.  Update the test code to verify that the file size is unset
for the regular file with length-at-end.

MFC after:	7 days
2008-10-21 05:08:35 +00:00
Tim Kientzle
758dfd70db Add test to verify ACL restores on FreeBSD, verify kern/128203 (mtime
restore fails if ACL provided), apply patch from kern/128203, and verify fix.

PR:		kern/128203
Submitted by:	Udo Schweigert
MFC after:	3 days
2008-10-19 00:18:44 +00:00
Tim Kientzle
d34dca65fa Style: Mark private data 'static', correct an old comment.
MFC after:	3 days
2008-10-19 00:13:57 +00:00
Tim Kientzle
b2d4ddad82 Verify high-res birthtime support on FreeBSD.
MFC after:	30 days
2008-09-30 04:13:21 +00:00
Tim Kientzle
f70105fb18 MfP4: restore birth time data to disk + more thorough tests for
time restore to disk.

MFC after:	30 days
2008-09-30 04:02:36 +00:00
Tim Kientzle
155524db13 MfP4: Store/read birthtime data in pax format.
Submitted by:	Pedro Giffuni
MFC after:	30 days
2008-09-30 03:57:07 +00:00
Tim Kientzle
8b18fa965d MfP4: Support for storing birthtime in archive_entry objects.
Submitted by:	Pedro Giffuni
MFC after:	30 days
2008-09-30 03:53:03 +00:00
Tim Kientzle
c42f5c226a MfP4: Generalize int64_t in the public headers to support compilers
which have a 64-bit integer type but don't provide standard C99 definitions.
2008-09-18 04:19:06 +00:00
Tim Kientzle
a370f3ca73 Trying to distinguish different "versions" of mtree files was a silly idea.
Submitted by:	Joerg Sonnenberger
2008-09-18 04:13:36 +00:00
Tim Kientzle
29a23c776a Choose a flag-handling strategy based on platform capabilities
rather than on platform.
2008-09-14 05:51:25 +00:00
Tim Kientzle
1074aa240c Portability: Don't use symbolic constants when the values are more
standard than the names are.  Remove some trailing whitespace.
2008-09-12 05:33:00 +00:00
Tim Kientzle
a2854491c4 Portability: Not everyone is lucky enough to have ftruncate() 2008-09-12 04:08:11 +00:00
Tim Kientzle
220265229d Portability: Support platforms that lack (struct stat).st_blksize or SSIZE_MAX
Of course, FreeBSD has both.
2008-09-12 04:03:34 +00:00
Tim Kientzle
6d35d9a6e9 Windows compatibility: The stub replacement for compression_program
(which always returns an error when invoked) needs its prototype.
2008-09-12 03:53:54 +00:00
Tim Kientzle
a2a72fd640 Fix a type-aliasing problem in the libarchive test suite,
restore WARNS=6.
2008-09-08 00:58:12 +00:00
Tim Kientzle
fb7ec956eb Correct a leaking archive_entry in the test suite. 2008-09-07 23:59:27 +00:00
Tim Kientzle
7480311f87 Correct the comments here.
PR:		bin/126849
MFC after:	3 days
2008-09-07 06:15:45 +00:00
Tim Kientzle
11663004bd When restoring a directory, allow symlinks to be followed. The full
logic here gets a little complex, but the net effect is that the
SECURE_SYMLINKS flag will prevent us from ever following a symlink.
Without it, we'll only follow symlinks to dirs.  bsdtar specifies
SECURE_SYMLINKS by default, suppresses it for -P.

I've also beefed up the write_disk_secure test to verify this
behavior.

PR:		bin/126849
2008-09-07 05:22:33 +00:00
Tim Kientzle
11b4b8458c The link resolver now unsets the size to mark a hardlink
with no body instead of setting it to zero.
2008-09-05 06:15:25 +00:00
Tim Kientzle
a0bfb8ae3e MfP4: Handle entries with unset size properly: Regular files with
unspecified size are "unlimited" (required by Zip reader, which
sometimes does not know the uncompressed size of an entry until it
gets to the end).  Also, hardlinks with unspecified (or zero) size do
not overwrite the data on disk nor do they set metadata.  This is
compatible with GNU tar and NetBSD pax behavior.
2008-09-05 06:13:11 +00:00
Tim Kientzle
e20d272f0a MfP4: Use the set/unset tracking to determine when mtime or atime
is unavailable.  Use start_time as a substitute when one is missing;
if both are missing, skip the call to {f,l,}utime{s,}() entirely.
2008-09-05 05:55:49 +00:00
Tim Kientzle
1b42ff4cd6 Markup fix.
MFC after:	3 days
2008-09-04 05:22:00 +00:00
Tim Kientzle
0c35edadaa Style: Use ARCHIVE_VERSION_NUMBER to conditionalize tests on
the particular libarchive version being tested instead of the
deprecated ARCHIVE_API_VERSION and ARCHIVE_VERSION_STAMP macros.
2008-09-01 05:38:33 +00:00
Tim Kientzle
32364a7ddb MfP4: set/unset tracking for atime, ctime, mtime, and size fields.
This generalizes the existing set/unset tracking for hardlink/symlink
fields and extends it to cover non-string fields.  Eventually, this
will be further extended to cover most fields.

In particular, this is needed to correctly detect when time fields
are missing (for example, reading ustar archives doesn't set atime or
ctime) for proper time restore and is helpful when trying to determine
whether to overwrite data when restoring hardlinks.

This commit updates the tests but not the docs.
2008-09-01 04:54:29 +00:00
Tim Kientzle
c7715560f1 Sort the list of format names. 2008-09-01 02:50:53 +00:00
Tim Kientzle
e32c65cb17 Use the "start_time" (time at which the restore process began)
to fill in a missing atime instead of substituting mtime.

PR:		bin/124915
2008-09-01 02:50:24 +00:00
Tim Kientzle
9ea6421090 New mtree writer, thanks to Joerg Sonnenberger.
Obtained from:	Joerg Sonnenberger
2008-08-31 07:21:46 +00:00
Tim Kientzle
7584fa8aa4 Joerg Sonnenberger's improvements to the shar writer.
Significant performance improvements, better quoting of file names, etc.
2008-08-31 07:10:40 +00:00
Tim Kientzle
5dc843b27b Make libarchive_test a little smarter about trying to find
its reference files when you don't specify -r.  It now checks
a couple of likely nearby directories to see if any of them
have a particular known file.
2008-08-31 07:06:02 +00:00
Tim Kientzle
cec048ddf0 When atime wasn't specified, it should get set == mtime.
This is much stronger than just asserting that it's not zero.
2008-08-30 05:31:23 +00:00
Tim Kientzle
9cfec77e7e If no atime was specified (for example, when extracting from ustar
archives), set atime == mtime.  Before this, atime would get restored
to 0.
2008-08-28 06:40:22 +00:00
Tim Kientzle
39056f55e3 Alphabetize the list of tests. 2008-08-25 06:08:22 +00:00
Tim Kientzle
3269dc16fd Update the total archive byte counters when writing entries to disk using
archive_write_disk.
Update cpio to use this to emit block counts in -p mode.
Update cpio tests to verify these block counts.
2008-08-24 06:21:00 +00:00
Tim Kientzle
a705ee2bcd Remove the stillborn attempt to cleanup tests as well as the build dir. 2008-08-24 05:54:28 +00:00
Tim Kientzle
b9364a48e5 Fail copying path/.. only if SECURE_NODOTDOT was specified.
Since we already warn for any '..' elements in that case,
the extra "lastdotdot" tracking turns out to be unnecessary.

PR:		bin/124924
2008-08-24 05:42:22 +00:00
Tim Kientzle
8d14bd8cc2 Fix the cpio bug tested for by r182092 of usr.bin/cpio/test.
Since various 'find' incantations can emit container directories
in various orders, we cannot refuse to update a dir because it's
apparently the same age.

MFC after:	3 days
2008-08-24 05:01:01 +00:00
Tim Kientzle
85c04a017f Fix up test suite so it works again with the libarchive 1.x API.
In particular, FreeBSD 6 still uses the libarchive 1.x API and
this correction will permit MFCing new libarchive features back
to FreeBSD 6.
2008-08-11 01:19:36 +00:00
Tim Kientzle
438e9b4a9e Correct test_tar_large so it works correctly with the libarchive 1.x API. 2008-08-10 21:14:26 +00:00
Tim Kientzle
dde4785e50 Explain why two apparently-identical archive_read_data() calls
really should have different return values.
2008-08-10 03:13:42 +00:00
Tim Kientzle
07fe09e227 MfP4: Joerg Sonnenberger's fixes to make 'ar' and 'pax' writers
robust against multiple calls to their destroy() functions.
2008-08-10 02:06:28 +00:00
Colin Percival
f9bcf9cabf Mark functions as __dead2 in order to help the LLVM static checker
understand which code paths aren't possible.

This commit eliminates 117 false positive bug reports of the form
"allocate memory; error out if pointer is NULL; use pointer".
2008-08-04 01:25:48 +00:00
Colin Percival
b4d3a08be1 Garbage collect a variable which is assigned a value once but otherwise
is never used.

Found by:	LLVM/Clang Static Analyzer
2008-07-10 09:50:55 +00:00
Tim Kientzle
1ad4959f57 libarchive in FreeBSD-CURRENT is now synched with libarchive-portable 2.5.5. 2008-07-05 01:50:40 +00:00
Tim Kientzle
f37f4b11e4 Remove a couple of capability definitions that are never used. 2008-07-05 01:50:07 +00:00
Tim Kientzle
f5a5cfb411 When ARCHIVE_EXTRACT_PERM is requested (e.g., by "tar -p"), always
schedule a chmod() fixup for directories.  In particular, this fixes
sgid handling on systems where the sgid bit is inherited from the
parent directory (which means that the actual mode of the dir
does not match the mode used in the mkdir() system call.

It may be possible to tighten this condition a bit.  In
working through this, I also found a few other places where
it looks like we can avoid a redundant syscall or two.  I've
commented those here but not yet tried to address them.
2008-07-05 01:48:33 +00:00
Tim Kientzle
4e8f10a152 Don't use the deprecated linux/ext2_fs.h include file. 2008-07-05 01:45:31 +00:00
Peter Wemm
a11c72cfcb Add $FreeBSD$ to the uuencoded test files that didn't have them already. 2008-07-03 03:26:30 +00:00
Dag-Erling Smørgrav
d8c5a84b61 More explicit error message. 2008-06-30 16:19:26 +00:00
Dag-Erling Smørgrav
24cbd9936b Add missing $FreeBSD$ 2008-06-30 15:49:12 +00:00
Dag-Erling Smørgrav
c7dd6fa2c9 Some tests won't build at WARNS level 6 due to aliasing violations.
Add missing -I. so the tests will build when ${.OBJDIR} != ${.CURDIR}.
${.OBJDIR} does not need to be spelled out.
2008-06-26 11:58:26 +00:00
Dag-Erling Smørgrav
f9145f3547 Add regression test for CRC32 check. The test file has been modified to
include an invalid checksum for file2.

Approved by:	kientzle
2008-06-26 11:50:11 +00:00
Dag-Erling Smørgrav
c7d703c46a Implement CRC32 verification. Note that you have to read until EOF to
trigger the check.

Requested by:	ache
Approved by:	kientzle
2008-06-26 11:48:19 +00:00
Dag-Erling Smørgrav
e2157b51de Allow the tests to build without libdmalloc. 2008-06-26 10:53:05 +00:00
Tim Kientzle
8b88e9591a Split out the reference zip file for ease of maintenance. 2008-06-26 04:48:42 +00:00
Tim Kientzle
80a6a0328a Refactor one of the ISO extraction tests: Move the reference
file into a separate file (instead of embedding it in the C code)
and use later timestamps (timestamps too close to the Epoch fail
predictably on systems that lack timegm(), whose mktime() doesn't
support dates before the Epoch and which are running in timezones
with negative offsets from GMT).  The goal here is to test the ISO
extraction, not the local platform's time support.
2008-06-21 19:11:51 +00:00
Tim Kientzle
8a4ce0fb38 MfP4: Joerg Sonnenberg's extensions to the mtree support for more
complete quoting.  In particular, this handles escaped newlines
and common C-style escapes.
2008-06-21 19:06:37 +00:00
Tim Kientzle
be5eb9821c A security-check failure here should be ARCHIVE_FAILED (cannot continue this
operation) and not ARCHIVE_WARN, since we don't actually open the file.
Both bsdtar and bsdcpio will try to copy file contents after an ARCHIVE_WARN,
which will fail loudly.
2008-06-21 19:05:29 +00:00
Tim Kientzle
d984a6e000 Rework the my_mbtowc_utf8() support function to fully match the mbtowc()
calling convention, not the mbrtowc() convention.
2008-06-15 11:28:56 +00:00
Tim Kientzle
4a9c863c87 Since wctomb() returns int, temporaries should be int, not size_t. 2008-06-15 11:15:12 +00:00
Tim Kientzle
936d273c3a A number of minor corrections to the support for external compression
programs:
  * Support platforms that have fork() but not vfork()
  * Don't write(), select(), or poll() against closed file descriptors
2008-06-15 10:45:57 +00:00
Tim Kientzle
dff2641a67 Be a little more careful about closing file descriptors that may not exist. 2008-06-15 10:43:59 +00:00
Tim Kientzle
c8390967f2 MfP4: test harness improvements. 2008-06-15 10:35:22 +00:00
Tim Kientzle
8d86da5715 Fix a bad cast. 2008-06-15 10:26:08 +00:00
Tim Kientzle
bfc29cb4d3 Fix reading TOC from zip archives with unsupported
compression.  We can't read the body, but we
shouldn't try to skip the body twice.
2008-06-15 05:15:53 +00:00
Tim Kientzle
693f6cf97c Mark hardlinks that lack any other interesting filetype information
with an 'h'.
2008-06-15 05:14:01 +00:00
Tim Kientzle
a64ca54afc Portability: We can get away with the older and better-supported
wctomb() here; we don't need wcrtomb().  In particular, this fixes
libarchive building on FreeBSD 4.
2008-06-15 05:12:47 +00:00
Tim Kientzle
8dd058b03d Portability: wchar_t is defined in stdlib.h on some systems. 2008-06-15 05:11:08 +00:00
Tim Kientzle
f9fe0a0abb archive.h is no longer constructed from archive.h.in,
so we can rename it and drop some no-longer-necessary
build magic from the Makefile.
2008-06-15 05:05:53 +00:00
Tim Kientzle
546c9253a3 Fix the new generic link resolver in libarchive to never match
dirs as hardlinks.  In particular, this fixes some recent ports
build failures.

Thanks to: Kris Kennaway
2008-06-15 04:31:43 +00:00
Colin Percival
7254457b50 Don't store a pointer in "next" if we're never going to use it.
Fix typo in checking results of strchr.

Found by:	Coverity Prevent
2008-06-09 14:44:56 +00:00
Tim Kientzle
40715dc446 Minor code hardening: Verify the final bytes of the string
are actually accessible before trying to use them.
2008-05-27 04:46:12 +00:00
Tim Kientzle
a212de8851 Until the old archive.h.in gets renamed to archive.h in the repository,
we still need some Makefile trickery to ensure archive.h is
correctly built for the test harness.
2008-05-27 04:12:17 +00:00
Tim Kientzle
fa07de5eeb MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more
feedback, but the 2.5 branch is shaping up nicely.)

In addition to many small bug fixes and code improvements:
 * Another iteration of versioning; I think I've got it right now.
 * Portability:  A lot of progress on Windows support (though I'm
   not committing all of the Windows support files to FreeBSD CVS)
 * Explicit tracking of MBS, WCS, and UTF-8 versions of strings
   in archive_entry; the archive_entry routines now correctly return
   NULL only when something is unset, setting NULL properly clears
   string values.  Most charset conversions have been pushed down to
   archive_string.
 * Better handling of charset conversion failure when writing or
   reading UTF-8 headers in pax archives
 * archive_entry_linkify() provides multiple strategies for
   hardlink matching to suit different format expectations
 * More accurate bzip2 format detection
 * Joerg Sonnenberger's extensive improvements to mtree support
 * Rough support for self-extracting ZIP archives.  Not an ideal
   approach, but it works for the archives I've tried.
 * New "sparsify" option in archive_write_disk converts blocks of nulls
   into seeks.
 * Better default behavior for the test harness; it now reports
   all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
Colin Percival
b011a14a0a Check that lseek(2) succeeds and puts us where we expect. [1]
While we're here, fix a long-standing bug in the handling of write(2)
errors: The API changed from "return # of bytes written" to "return
status code" almost 4 years ago, so instead of returning (-1) we need
to return ARCHIVE_FATAL.

Found by:	Coverity Prevent [1]
2008-05-23 05:01:29 +00:00
Colin Percival
ca42a8e225 We only use the string "?rwxrwxrwx " once, so inline it rather than
declaring a variable which points to it.  Aside from eliminating a
line of code and one level of unnecessary indirection, this eliminates
a false positive in Coverity.
2008-05-23 04:57:28 +00:00
Colin Percival
ee99deabac Return ARCHIVE_FATAL if we can't allocate memory instead of going ahead and
dereferencing NULL.

Found by:	Coverity Prevent
2008-05-19 18:06:48 +00:00
Tim Kientzle
4b7d286a5b Include an extra byte for the trailing NUL. <sigh>
Pointy hat: Me
2008-03-31 06:24:39 +00:00
Kai Wang
7a36fb79f9 Add MLINK for archive_write_close.
Approved by:	jkoshy(mentor), kientzle
2008-03-21 11:10:20 +00:00
Tim Kientzle
409e319377 Update a comment: the format bid only runs once per archive; it no
longer runs once per entry.
2008-03-15 11:09:16 +00:00
Tim Kientzle
845aa4ab0a Free up the entry objects allocated during this test. 2008-03-15 11:06:15 +00:00
Tim Kientzle
adfb462fea Release the buffers used for exercising the compress code. 2008-03-15 11:05:49 +00:00
Tim Kientzle
0b315cd9ae Remove the duplicate "archive_format" and "archive_format_name" fields
from the private archive_write structure and fix up all writers to use
the format fields in the base "archive" structure.  This error made it
impossible to query the format after setting up a writer because the
write format was stored in an inaccessible place.
2008-03-15 11:04:45 +00:00
Tim Kientzle
c43d294189 Correct a sign mismatch that only showed up on 64-bit systems.
Pointy hat: me
2008-03-15 11:02:47 +00:00
Tim Kientzle
3010219939 Refactor the mtree code a bit to make the layering clearer: Each
"file" is described by multiple "lines" each possibly containing
multiple "keywords."  Incorporate some additions from Joerg Sonnenberger
to handle linked files and correctly deal with backing files on disk.
2008-03-15 07:10:24 +00:00
Tim Kientzle
d7740aea75 FreeBSD does have fstat().
Correct the nasty typo this uncovers.
2008-03-15 04:20:50 +00:00
Tim Kientzle
eb971f9524 Testability is more important than standards conformance.
Disable the use of PaxHeader.<pid> for the fake pax extension pathname
until I can make the name here settable.  Otherwise, tests that try
to compare output to static pre-generated reference files break.
2008-03-15 03:49:18 +00:00
Tim Kientzle
24f55a5963 Ignore a few more common files. 2008-03-15 02:31:28 +00:00
Tim Kientzle
80334b7d22 Resolve a minor nit in SUS compliance by including the PID in the
fake directory name used for pax extended headers.
2008-03-15 02:30:42 +00:00
Tim Kientzle
cde1a05218 GC a reference to the defunct TESTFILES variable. 2008-03-15 02:22:08 +00:00
Tim Kientzle
60617bf578 A subtle point: "pax interchange format" mandates that all strings
(including pathname, gname, uname) be stored in UTF-8.  This usually
doesn't cause problems on FreeBSD because the "C" locale on FreeBSD
can convert any byte to Unicode/wchar_t and from there to UTF-8.  In
other locales (including the "C" locale on Linux which is really
ASCII), you can get into trouble with pathnames that cannot be
converted to UTF-8.

Libarchive's pax writer truncated pathnames and other strings at the
first nonconvertible character.  (ouch!)  Other archivers have worked
around this by storing unconvertible pathnames as raw binary, a
practice which has been sanctioned by the Austin group.  However,
libarchive's pax reader would segfault reading headers that weren't
proper UTF-8.  (ouch!)  Since bsdtar defaults to pax format, this
affects bsdtar rather heavily.

To correctly support the new "hdrcharset" header that is going into
SUS and to handle conversion failures in general, libarchive's pax reader
and writer have been overhauled fairly extensively.  They used to do
most of the pax header processing using wchar_t (Unicode); they now do
most of it using char so that common logic applies to either UTF-8 or
"binary" strings.

As a bonus, a number of extraneous conversions to/from wchar_t have
been eliminated, which should speed things up just a tad.

Thanks to: Bjoern Jacke for originally reporting this to me
Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this
Thanks to: Gunnar Ritter for getting the standard fixed
MFC after: 5 days
2008-03-15 01:43:59 +00:00
Tim Kientzle
3a6aaff135 Ignore some built files. 2008-03-15 00:52:22 +00:00
Tim Kientzle
408a822432 Don't lie. If a string can't be converted to a wide (Unicode) string,
return a NULL instead of an incomplete string.  Expand the test coverage
to verify the correct behavior here.
2008-03-14 23:19:46 +00:00
Tim Kientzle
6c8f54e991 Don't advertise the default block size as a constant; don't
rely on a deprecated value to set the default.  This is also
related to a longer-term goal of setting the default block
size based on format and possibly other factors, which makes
it a bad idea to tie this to a published constant.
2008-03-14 23:09:02 +00:00
Tim Kientzle
8e4bc81237 New public functions archive_entry_copy_link() and archive_entry_copy_link_w()
override the currently set link value, whether that's a hardlink
or a symlink.  Plus documentation update and tests.
2008-03-14 23:00:53 +00:00
Tim Kientzle
1051e364aa Update some comments, comment out argument names to guard against
namespace problems.
2008-03-14 22:47:38 +00:00
Tim Kientzle
871e5c0326 Since "length" computes the length of a string and is used as an
argument to malloc(3), it should be size_t, not int.
2008-03-14 22:44:07 +00:00
Tim Kientzle
d6f37be734 Let archive_entry_clear() accept a NULL pointer and simply do nothing.
In particular, this allows archive_entry_free() to work correctly
for a NULL pointer, which makes it parallel with free(3).
2008-03-14 22:40:36 +00:00
Tim Kientzle
42d1f7b4ba Rework the versioning implementation and test to match the
new interface.  Mark the functions that are going away in
libarchive 3.0.

In particular, archive_version_string() now computes the
string rather than assuming that it will be created by the
build infrastructure.  Eventually, this will allow some
simplification of the build infrastructure.
2008-03-14 22:31:57 +00:00
Tim Kientzle
0349d719b1 Rework the versioning information, hopefully for the last time.
* There are now only two public version identifiers:  "number" is
   a single integer that combines Major/minor/release in a single
   value of the form Mmmmrrr.  This is easy to compare against for
   checking feature support.  "string" is a displayable text string
   of the form "libarchive M.mm.rr".
 * The number is present both as a macro (version of the installed header)
   and a function (version of the shared library).  The string form
   is available only as a function.
 * Retain the older version definitions for now, but mark them all
   as deprecated, to disappear in libarchive 3.0 (whenever that happens).
 * Rework the various deprecation conditionals to use ARCHIVE_VERSION_NUMBER.

An ancillary goal is to reduce the number of @...@ substitutions that
are required.  Someday, I might even be able to avoid build-time
processing of archive.h entirely.
2008-03-14 22:19:50 +00:00
Tim Kientzle
45943bfd93 Add a useful sprintf()-style wrapper around
archive_string_vsprintf().  (Which is built
on top of libarchive's internal resizable string
support.)
2008-03-14 22:00:09 +00:00
Tim Kientzle
7c5b1173a5 Support for writing 'compress' format, thanks to Joerg Sonnenberger. 2008-03-14 20:35:38 +00:00
Tim Kientzle
20347f62e6 A block in a tar file is 512 bytes. Period.
Remove the entirely pointless symbolic constant
and sizeof(unsigned char).  (The constant
here is doubly wrong, since not only does
it obscure a basic format constant, it was
never intended to be a tar-specific value,
so could conceivably be changed at some point
in the future.)
2008-03-14 20:32:20 +00:00
Kai Wang
a739eb8374 Current 'ar' read support in libarchive can only handle a GNU/SVR4
filename table whose size is less than 65536 bytes.

The original intention was to not consume the filename table, so the
client will have a chance to look at it. To achieve that, the library
call decompressor->read_ahead to read(look ahead) but do not call
decompressor->consume to consume the data, thus a limit was raised
since read_ahead call can only look ahead at most BUFFER_SIZE(65536)
bytes at the moment, and you can not "look any further" before you
consume what you already "saw".

This commit will turn GNU/SVR4 filename table into "archive format
data", i.e., filename table will be consumed by libarchive, so the
65536-bytes limit will be gone, but client can no longer have access
to the content of filename table.

'ar' support test suite is changed accordingly. BSD ar(1) is not
affected by this change since it doesn't look at the filename table.

Reported by:	erwin
Discussed with:	jkoshy, kientzle
Reviewed by:	jkoshy, kientzle
Approved by:	jkoshy(mentor), kientzle
2008-03-12 21:10:26 +00:00
Tim Kientzle
df4691b984 Portability: Eliminate the need for uudecode by incorporating
uudecode into the main test driver and invoking it just-in-time
within the various tests.

Also, incorporate a number of improvements to the main test support
code that have proven useful on other projects where I've used this
framework.
2008-03-12 05:12:23 +00:00
Tim Kientzle
0b4793efb7 Remove some unused fields from the private archive_read structure
(left over from when the unified read/write structure was copied
to form separate read and write structures) and eliminate the
pointless initialization of a couple of the unused fields.
2008-03-12 04:58:32 +00:00
Tim Kientzle
c2247d3995 Tighten up the semantics of acl_next() and xattr_next() when you
hit the end of the ACL or xattr list.

Thanks to: Jeff Johnson for pointing out the obvious typo
2008-03-12 04:47:37 +00:00
Tim Kientzle
826055b6a8 Typo, thanks to: Jeff Johnson.
MFC after: 3 days
2008-03-12 04:26:44 +00:00
Joseph Koshy
c7f03ab040 Use .Fo/.Fc and .Xo/.Xc to bring the line widths below 79 columns.
Correct a typo [a misplaced comma].

Reviewed by:		ru
2008-03-10 14:45:29 +00:00
Joseph Koshy
80c4d6eba3 Use .Fo/.Fc and .Xo/.Xc to bring the line widths below 79 columns.
Reviewed by:		ru
2008-03-10 14:44:41 +00:00
Tim Kientzle
e29c664a4c Spelling correction, thanks to Joerg Sonnenberger. 2008-02-27 06:16:41 +00:00
Tim Kientzle
a26e9253f6 Optimize skipping over Zip entries.
Thanks to: Dan Nelson, who sent me the patch
MFC after: 7 days
2008-02-27 06:05:59 +00:00
Tim Kientzle
35f4ae0981 Rename the archive_endian.h functions to avoid name clashes
with NetBSD's sys/endian.h file.

Pointed out by: Joerg Sonnenberger
2008-02-26 07:17:47 +00:00
Tim Kientzle
5b7a04161d Sanity-check the block size.
Thanks to: Joerg Sonnenberger
MFC after: 7 days
2008-02-21 03:21:50 +00:00
Tim Kientzle
b3fa7a9568 Include O_BINARY in open() calls on platforms that support it. 2008-02-19 06:10:48 +00:00
Tim Kientzle
dc4a55fdfc Another tiny, tiny step towards Windows support. No, I don't plan to
ever commit the Windows support files to FreeBSD CVS.  That would just
be wrong.
2008-02-19 06:06:13 +00:00
Tim Kientzle
54c845efb9 Someday I might forgive the standards bodies for omitting timegm().
Maybe.  In the meantime, my workarounds for trying to coax UTC without
timegm() are getting uglier and uglier.  Apparently, some systems
don't support setenv()/unsetenv(), so you can't set the TZ env var and
hope thereby to coax mktime() into generating UTC.  Without that, I
don't see a really good alternative to just giving up and converting to
localtime with mktime().  (I suppose I should research the Perl library
approach for computing an inverse function to gmtime(); that might
actually be simpler than this growing list of hacks.)
2008-02-19 06:02:01 +00:00
Tim Kientzle
334a6ee707 Simplify file type setting. 2008-02-19 05:54:24 +00:00
Tim Kientzle
4d9cfd1eb7 The test_assert() function that backs my custom assert() macro
now returns a value, which supports such convenient
constructs as:
   if (assert(NULL != foo())) {
   }

Also be careful to setlocale("C") for each new test to
avoid locale pollution.

Also a couple of minor portability enhancements.
2008-02-19 05:52:30 +00:00
Tim Kientzle
5c5430972a Portability: Since the values are fixed and the symbolic names
are only present on some platforms, just use the values directly.
2008-02-19 05:49:02 +00:00
Tim Kientzle
98ef1f2ddb Portability: Include O_BINARY if the local platform defines it. 2008-02-19 05:46:58 +00:00
Tim Kientzle
f167d4f9c3 Correct a compile error when libbz2/zlib are unavailable. 2008-02-19 05:44:59 +00:00
Tim Kientzle
ee10f0feb0 Mark a few additional functions that are/are not available on FreeBSD. 2008-02-19 05:40:28 +00:00
Tim Kientzle
75018fc592 Portability improvements:
* If the platform can't restore char nodes, block nodes, or fifos,
don't try and just return error.
  * Include O_BINARY in most open() calls (define O_BINARY to 0 if the
platform doesn't provide a definition already)
  * Refactor the ownership restore to more cleanly support platforms
that don't have any form of {l,f,}chown() call.
  * Comment a lingering issue with older Unix-like systems that allow
root to hose the filesystem.  I don't (yet) have a good solution for
this, but I expect it will require adding more redundant stat()
calls. <sigh>

MFC after: 14 days
2008-02-19 05:39:35 +00:00
Kai Wang
5f1f828a63 Add hook routine archive_write_ar_finish() which writes the 'ar'
global header if nothing else has been written before the closing of
the archive. This will change the behaviour when creating archives
without members, i.e., instead of generating a 0-size archive file, an
archive with just the global header (8 bytes in total) will be created
and it is indeed a valid archive by the definition of libarchive, thus
subsequent operation on this archive will be accepted. This especially
solves the failure caused by following sequence: (several ports do)

% ar cru libfoo.a  	    # without specifying obj files
% ranlib libfoo.a

Reviewed by:	kientzle, jkoshy
Approved by:	kientzle
Approved by:	jkoshy	(mentor)
Reported by:	erwin
MFC after:	1 month
2008-01-31 08:11:01 +00:00
Tim Kientzle
fae7e96d82 Add a test to verify compatibility with archives with
odd hardlinks.  I need to extend this to test pax extended
archives with bodies attached to hardlinks and other less-common cases.
2008-01-31 07:47:38 +00:00
Tim Kientzle
6f6dfc16c2 Tighten up the heuristic that decides whether or not we should
obey or ignore the size field on a hardlink entry.  In particular,
if we're reading a non-POSIX archive, we should always ignore
the size field.

This should fix both the audio/xmcd port and the math/unixstat port.

Thanks to: Pav Lucistnik for pointing these two ports out to me.
MFC after: 7 days
2008-01-31 07:41:45 +00:00
Tim Kientzle
9695c83ff5 Track version # from the portable release. 2008-01-23 05:48:07 +00:00
Tim Kientzle
991a64e41e Explain a subtle API change that was made recently.
Even though I believe this is a good change, it does
have the potential to break certain clients, so it's
good to document the reasoning behind the change.
2008-01-23 05:47:08 +00:00
Tim Kientzle
1466a90ac6 Properly pad symlinks when writing cpio "newc" format.
Thanks to: Jesse Barker for reporting this.
MFC after: 7 days
2008-01-23 05:43:26 +00:00
Tim Kientzle
91c3a77c57 I misread the Tinderbox error; this should really unbreak 64-bit builds.
Pointy hats, yep, keep 'em coming.  ;-/
2008-01-18 06:16:08 +00:00
Tim Kientzle
c19af48f40 Fix 64-bit build after my last commit. <sigh> 2008-01-18 06:08:39 +00:00
Tim Kientzle
22177dd3e6 The previous commit caused the archive_write_disk interface to
start obeying filesize limits; this test wasn't properly setting
file sizes before trying to write file data.
2008-01-18 05:48:50 +00:00
Tim Kientzle
2adbd7ee43 Issues with hardlinks in newc-format files prompted me to
write a new test to exercise the hardlink strategies used
by different archive formats (tar, old cpio, new cpio).
This uncovered two problems, both fixed by this commit:

1) Enforce file size when writing files to disk.

2) When restoring hardlink entries, if they have data associated, go
   ahead and open the file so we can write the data.

In particular, this fixes bsdtar/bsdcpio extraction of new cpio
formats where the "original" is empty and the subsequent "hardlink"
entry actually carries the data.  It also provides correct behavior
for old cpio archives where hardlinked entries have their bodies
stored multiple times in the archive; the last body should always be
the one that ends up in the final file.  The new pax format also
permits (but does not require) hardlinks to carry file data; again,
the last contents should always win.

Note that with any of these, a size of zero on a hardlink simply means
that the hardlink carries no data; it does not mean that the file has
zero size.  A non-zero size on a hardlink does provide the file size.

Thanks to: John Baldwin, for reminding me about this long-standing bug
    and sending me a simple example archive that prompted this test case
2008-01-18 05:05:58 +00:00
Tim Kientzle
23b11f239a Reconnect the progress callback. It may not get called
as often as you might expect, but at least it will get called
now.

Thanks to: David Topham for asking how this got disconnected.
2008-01-18 04:53:45 +00:00
Tim Kientzle
f432a1c5bc Handle Zip archives that are "multi-part archives with only
one part" by simply ignoring the marker at the beginning
of the file.  (Zip archivers reserve four bytes at the beginning
of each part of a multi-part archive, if it happens to only
require one part, those four bytes get filled with a placeholder
that can be ignored.)

Thanks to: Marius Nuennerich,
	 for pointing me to a Zip archive that libarchive couldn't handle
MFC after: 7 days
2008-01-15 16:27:15 +00:00
Tim Kientzle
a8f2d755d0 Support uppercase hex digits in cpio archives.
Thanks to: Joshua Kwan
MFC after: 7 days
2008-01-15 04:56:48 +00:00
Tim Kientzle
a0751a90e6 Since the tar bidder can never get called more than once, it
doesn't need to compensate for this situation.

While here, fix a minor longstanding bug that empty tar archives
(which begin with at least 512 zero bytes) never properly reported
their format.  In particular, this fixes the output of:
   bsdtar tvvf /dev/zero

And, of course, a new test to verify that libarchive correctly
recognizes the format of such files.
2008-01-13 23:50:30 +00:00
Dag-Erling Smørgrav
aa1b83ec5a Crib {be,le}{16,32,64}{dec,enc} from src/sys/sys/endian.h and use it instead
of home-rolled [iu][248] in the ZIP support code.

Approved by:	kientzle
2008-01-03 18:30:37 +00:00
Dag-Erling Smørgrav
4823b3de93 Add an internal utility function to simplify the many, many places where
the number of bytes read is actually not important as long as we have at
least what we ask for.  Illustrate its benefits by using it throughout
the ZIP support code, except for the few cases where it doesn't apply.

Approved by:	kientzle
2008-01-03 17:54:26 +00:00
Tim Kientzle
2a5e8d812c Extensive improvements to the libarchive_test test program that
exercises and verifies the libarchive APIs:

* Improved error reporting; hexdumps are now provided for
  many file/memory content differences.
* Overall status more clearly counts "tests" and "assertions"
* Reference files can now be stored on disk instead of having
  to be compiled into the test program itself.  A couple of
  tests have been converted to this more natural structure.
* Several memory leaks corrected so that leaks within libarchive
  itself can be more easily detected and diagnosed.
* New test: GNU tar compatibility
* New test: Zip compatibility
* New test: Zero-byte writes to a compressed archive entry
* New test: archive_entry_strmode() format verification
* New test: mtree reader
* New test: write/read of large (2G - 1TB) entries to tar archives
  (thanks to recent performance work, this test only requires a few seconds)
* New test: detailed format verification of cpio odc and newc writers
* Many minor additions/improvements to existing tests as well.
2008-01-01 22:28:04 +00:00
Tim Kientzle
db267e0b5d The mtree.5 file has been moved to src/usr.sbin/mtree. 2008-01-01 06:17:05 +00:00
Tim Kientzle
9dd49f960f Update libarchive to 2.4.10. This includes a number of improvements
that I've been working on but put off committing until after the
RELENG_7 branch, including:

* New manpages: cpio.5 mtree.5
* New archive_entry_strmode()
* New archive_entry_link_resolver()
* New read support: mtree format
* Internal API change:  read format auction only runs once
* Running the auction only once allowed simplifying a lot of bid logic.
* Cpio robustness:  search for next header after a sync error
* Support device nodes on ISO9660 images
* Eliminate a lot of unnecessary copies for uncompressed archives
* Corrected handling of new GNU --sparse --posix formats
* Correctly handle a zero-byte write to a compressed archive
* Fixed memory leaks

Many of these improvements were motivated by the upcoming bsdcpio
front-end.

There have also been extensive improvements to the libarchive_test
test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
Tim Kientzle
f4c8548d52 If a Zip entry has the "length at end" flag set, then just ignore
a length field of zero; it does not mean the body is empty.

Thanks to: Lapo Luchini for sending me a JAR archive that demonstrated this bug
MFC after: 3 days
2007-12-04 06:32:12 +00:00
Tim Kientzle
c383d82e4d When skipping input data, don't overflow a 32-bit size_t.
This can only happen on 32-bit systems when you're reading
an uncompressed archive and the skip request is an exact
multiple of 4G (e.g., skipping a tar entry with an 8G body).

The symptom is that the read_ahead() ends up returning zero
bytes, and the extraction stops with a premature end-of-file.

Using '1' here is more correct anyway, as it allows read_ahead()
to function opportunistically and minimize copying.

MFC after: 5 days
2007-10-27 22:45:40 +00:00
Tim Kientzle
6fa30d2b87 Fix reading of files that use pax 'size' attribute to store size.
In particular, bsdtar uses the pax 'size' attribute for any file
over 8G.

MFC after: 3 days
2007-10-24 04:01:31 +00:00
Tim Kientzle
a3ac45275d Correct the cpio writers to not accept data for non-regular files.
In particular, the previous code led to archives that had
non-empty bodies following directory entries.  Not a fatal
problem, as bsdtar and GNU cpio are both happy to just skip
this bogus data, but it still shouldn't be there.

MFC after: 3 days
2007-10-12 04:11:31 +00:00
Tim Kientzle
04a832627b Correct the return values of the final zero-length block at EOF.
Return EOF immediately if an entry in a ZIP archive has no body.
In particular, the latter issue was causing bsdtar to emit spurious
warnings when extracting directory entries from ZIP archives.

MFC after: 3 days
2007-10-12 04:08:28 +00:00
Tim Kientzle
016e328d16 Fix the archive_write_data() function so it always returns
number of bytes written, even when used to write files to
disk.  Extend the test suite to verify the correct return
values for archive_write_data() and archive_write_data_block().

Thanks to: Bruce Mah, for stepping in promptly to back out the
   earlier broken version of this fix
Thanks to: Colin Percival, for pointing out the correct fix
MFC after: 5 days
Approved by: re (ksmith)
Pointy hat: \me
2007-09-21 04:52:43 +00:00
Tim Kientzle
d2d0f66a30 Fill in a missing 'e'
Thanks to: Kai Wang, for pointing this out
Approved by: re (bmah)
MFC after: 3 days
2007-09-19 16:37:45 +00:00
Bruce A. Mah
7b7b893394 Revert the last commit to libarchive. It introduced some regresssions,
most noticably the incorrect extraction of files by bsdtar.

This commit reverts:

	src/lib/libarchive/archive_write_disk.c 1.15
	src/lib/libarchive/test/test_write_disk.c 1.4

Approved by:	re (implicitly)
2007-09-18 20:20:37 +00:00
Tim Kientzle
8f3ba4ccf4 Correct the return value from archive_write_data()
(when used to restore files to disk) to match:
  * The documentation
  * The return values of this function when used
    to write files into an archive.

Approved by: re (bmah)
Pointy hat: \me
MFC after: 5 days
2007-09-18 04:20:21 +00:00
Tim Kientzle
68f0154dcf This commit updates libarchive to be compatible with
GNU tar 1.17's implementation of --posix --sparse,
at the cost of losing compatibility with GNU tar 1.16.
Fortunately, the 1.17 implementation actually makes sense,
so the libarchive code is now a bit more straightforward
than before.

Background:  GNU tar 1.16 defined a new way to store
sparse files in --posix archives.  Unfortunately,
the implementation incorrectly inserted several
blocks of null padding after each such entry.
As a result, non-GNU tar implementations saw the
archive as truncated after any sparse entry.
This was fixed in GNU tar 1.17 at the cost of
losing compatibility with GNU tar 1.16 for this
new format (which is not the default, so hopefully
rarely used).  Libarchive recently gained support
for reading the GNU tar 1.16 formats; this commit
updates it to read the GNU tar 1.17 variant instead.

Approved by: re (ksmith for libarchive portion)
Approved by: re (blanket for libarchive_test portion)
MFC after: 5 days
2007-08-18 21:53:25 +00:00
Tim Kientzle
9136384dc2 Make suid/sgid restore be "opportunistic" if
owner restore is not requested.  If you ask
for permissions to be restored but not owner,
you will now get no error if suid/sgid bits
cannot be set.  (It's a security hole to restore
suid/sgid bits if the owner/group aren't restored.)

This fixes an obscure problem where a simple
"tar -xf" with no other options will sometimes
fail gratuitously because of suid/sgid bits.
This is causing occasional problems for people
using bsdtar as a drop-in replacement for
"that other tar program." ;-)

Note: If you do ask for owner restore, then suid/sgid
restore failures still issue an error.  This
only suppresses the error in the case where an
suid/sgid bit restore fails because of an owner
mismatch and owner restore was not requested.

Approved by: re (bmah)
MFC after: 7 days
2007-08-12 17:35:05 +00:00
Tim Kientzle
f6d19aee04 Update the tests for reading the various GNU tar sparse formats.
In particular:
  * Include a second entry in all of the test archives (to catch errors
    with intermediate padding)
  * Test the GNU tar 1.17 version of "posix sparse format 1.0"
    instead of the GNU tar 1.16 version (the latter is no longer
    supported by GNU tar).

Right now, libarchive fails this test because I originally
implemented the GNU tar 1.16 version of "posix sparse format 1.0".
I'll fix libarchive shortly.

Approved by: re (blanket, libarchive testing)
2007-08-12 01:16:19 +00:00
Tim Kientzle
9e3fd30bac Two minor nits:
* Allow libarchive_test to compile on Interix again.
  * Track the test name (not just line number) when counting skipped tests.

Thanks to: Joerg Sonnenberger
Approved by: re (blanket; libarchive testing)
2007-07-31 05:03:27 +00:00
Tim Kientzle
8ea9716c0c Fix a strict aliasing warning from GCC 4.1.
Thanks to: Joerg Sonnenberger
Approved by: re (hrs)
MFC after: 3 days
2007-07-20 01:28:50 +00:00
Tim Kientzle
d3bb697513 archive_string_ensure() used to call exit(3) if it
couldn't allocate more memory for a string.  Change
this so it returns NULL in that case, and update
all of its callers to handle the error.  Some of
those callers can now return errors back to the
client instead of calling exit(3).

Approved by: re (bmah)
2007-07-15 19:13:59 +00:00
Tim Kientzle
75d0856ca5 Add archive_entry_copy_gname() and archive_entry_copy_uname()
functions.

Approved by: re (bmah)
MFC after: 3 days
2007-07-15 19:10:34 +00:00
Tim Kientzle
58bdc3275c Clarify one test.
Approved by: re (blanket, libarchive testing)
2007-07-15 17:16:42 +00:00
Tim Kientzle
a902b04074 Use an uncompressed test archive for gtar sparse format 1.0 format,
as that better exercises some internal read-combining logic than the
compressed archive.

Approved by: re (blanket, libarchive testing)
2007-07-14 17:54:14 +00:00
Tim Kientzle
2e4e46b540 Report each failed test once, but keep a count and report that count
if there was more than one.  In particular, this simplifies
test_tar_filenames.c, which has a tendency to be very noisy otherwise.

Approved by: re (blanket, libarchive testing)
2007-07-14 17:52:01 +00:00
Tim Kientzle
46dd1e6ee7 Restore the 'break' that was inadvertently removed in 1.57 of this file.
Without this, hardlinks get returned as symlinks.

Approved by: re (Ken Smith)
MFC after: 2 days
2007-07-14 05:53:51 +00:00
Tim Kientzle
f7f698bb22 Extend the basic tar reading test to exercise most types of
entries.  This doesn't cover everything yet, but it's a big improvement.

Approved by:  re (blanket, libarchive testing)
2007-07-14 05:35:17 +00:00
Tim Kientzle
99d4bb3999 Make the test for reading gtar sparse entries more robust;
it now verifies that the returned blocks have the correct data
at the correct file offsets, ignoring any null padding that
may exist.

Approved by: re (blanket, libarchive test suite)
2007-07-13 15:16:26 +00:00
Tim Kientzle
ceab112df6 New test suite test_read_pax_truncated probes libarchive
behavior with truncated or damaged pax archives.  This
tests most of the cases covered by the recent security advisory.

Approved by: re (blanket, libarchive test suite)
2007-07-13 15:14:35 +00:00
Tim Kientzle
8ed6656123 New file "read_open_memory.c" is a custom variant of
archive_read_open_memory.c that tries to test border
cases.  In particular, it copies over each returned block
so that formats or decompressors that read past the end
of a returned block will break.

Approved by: re (blanket, libarchive test suite)
2007-07-13 15:12:52 +00:00
Tim Kientzle
4b0489dd31 Fix running individual tests via "libarchive_test <number> <number> ..."
Approved by: re (blanket)
2007-07-13 15:09:07 +00:00
Colin Percival
612c3e7724 Correct multiple security issues in how libarchive handles corrupt
tar archives, including a potentially exploitable buffer overflow.

Approved by:	re (kensmith, security blanket)
Reviewed by:	kientzle
Security:	FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
Tim Kientzle
ed3ba42250 Fix installworld: /usr/bin/printf isn't available then,
so use awk's printf for the formatting here instead.

Pointy hat: Yours Truly
Approved by: re
2007-07-07 16:55:42 +00:00
Tim Kientzle
dbb4eb7d9c Make test suite work with libarchive 1.3.1: Take advantage of
ARCHIVE_VERSION_STAMP to selectively disable tests that don't
apply to that version; new "skipping()" function reports skipped
tests; modify final summary to report component test failures and
skips.

Note:  I don't currently intend to MFC the test suite itself;
anyone interested should just checkout and use this version
of the test suite, which should work for any library version.

Approved by: re (Ken Smith, blanket)
2007-07-06 15:43:11 +00:00
Tim Kientzle
ab16ac785a New "version stamp" simplifies determining the exact version
of libarchive being used.  I've been taking advantage of this
with a recent round of updates to libarchive_test so that it
can test older and newer versions of the library.

Approved by: re (Ken Smith)
2007-07-06 15:36:38 +00:00
Tim Kientzle
3f6c3bcd84 Fix 'bsdtar -t' on tape drives. Libarchive uses the
skip() callback to skip over data when reading uncompressed
archives.  This gets invoked, for example, during tar -t
or tar -x with a filename argument.  The revised code
only calls [lf]seek() on regular files, instead of depending
on the kernel to return an error.

Thanks to: bde for explaining the implementation of lseek()
Thanks to: Daniel O'Connor for testing
Approved by: re (Ken Smith)
MFC after: 5 days
2007-06-26 03:06:48 +00:00
Tim Kientzle
734f944a06 Ouch. I partially screwed up the last commit by
enabling a test that's not ready yet. <sigh>

Pointy hat: /me
Approved by: re@
2007-06-22 05:49:13 +00:00
Tim Kientzle
17e60e6230 Support for writing the 'newc' cpio format, plus a minimal test harness
for the cpio formats.

Thanks to: Rudolf Marek
Approved by: re@
2007-06-22 05:47:00 +00:00
Tim Kientzle
7faa61e65d Track whether the current read stream supports seek(). For now, we
assume yes unless seek has previously failed, but I fear I'll have to
avoid seeks under other circumstances.  (For instance, tape drives on
FreeBSD seem to return garbage from lseek().)  Also, optimize away
zero-byte skips.
2007-06-18 00:36:54 +00:00
Tim Kientzle
d02e3c38ab Make 'ar' write test a tad more portable. 2007-06-13 03:40:02 +00:00
Tim Kientzle
0ddfde5d16 Read support for the new GNU tar sparse formats added in gtar 1.15 and
gtar 1.16.
2007-06-13 03:35:37 +00:00
Tim Kientzle
147e16377c Add some options to libarchive_test:
-k: like make -k, try to keep going after errors.
   -q: quiet
2007-06-13 03:30:46 +00:00
Tim Kientzle
e09eb83674 Fix a broken function declaration. 2007-06-13 03:29:15 +00:00
Stephane E. Potvin
04031e9ae2 Options spring cleanup:
- Add and document the KVM and KVM_SUPPORT options that
are needed for the ifmcstats(3) makefile
- Garbage collect unused variables
- Add missing inclusion of bsd.own.mk where needed

Approved by: kan (mentor)
Reviewed by: ru
2007-06-13 02:08:04 +00:00
Tim Kientzle
8b63161182 Don't lose leading '/' for pathnames exactly 101 bytes long.
Also, update the test harness to exercise this case.
2007-06-11 05:17:30 +00:00
Tim Kientzle
b48b40f1f8 libarchive 2.2.3
* "compression_program" support uses an external program
  * Portability: no longer uses "struct stat" as a primary
    data interchange structure internally
  * Part of the above: refactor archive_entry to separate
    out copy_stat() and stat() functions
  * More complete tests for archive_entry
  * Finish archive_entry_clone()
  * Isolate major()/minor()/makedev() in archive_entry; remove
    these from everywhere else.
  * Bug fix: properly handle decompression look-ahead at end-of-data
  * Bug fixes to 'ar' support
  * Fix memory leak in ZIP reader
  * Portability: better timegm() emulation in iso9660 reader
  * New write_disk flags to suppress auto dir creation and not
    overwrite newer files (for future cpio front-end)
  * Simplify trailing-'/' fixup when writing tar and pax
  * Test enhancements:  fix various compiler warnings, improve
    portability, add lots of new tests.
  * Documentation: document new functions, first draft of
    libarchive_internals.3

MFC after: 14 days
Thanks to: Joerg Sonnenberger (compression_program)
Thanks to: Kai Wang (ar)
Thanks to: Colin Percival (many small fixes)
Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
Colin Percival
a30eadfc15 Replace "sizeof(struct bucket[cache_size])" with
"cache_size * sizeof(struct bucket)".  The former is valid in C99 but can
confuse earlier compilers, while the latter is a standard idiom which all
C compilers understand.

Approved by:	kientzle
2007-05-22 15:37:58 +00:00
Colin Percival
3662c7b8ad Don't test for NULL when it is both unnecessary (the pointer is checked
against NULL when it is first allocated) and pointless (we've already
dereferenced the pointer several times).

Found by:	Coverity Prevent(tm)
CID:		3204
2007-05-21 04:45:24 +00:00
Colin Percival
51bbfbd15c Remove pointless code: Don't assign a value to a variable when we're
going to overwrite it with a new value a few lines later.

Visual inspection of the surrounding code indicates that the code does
what it's supposed to do; i.e., the pointless code wasn't supposed to
be doing something other than what it was doing.

CID:		3323
Found by:	Coverity Prevent(tm)
2007-05-21 04:22:38 +00:00
Colin Percival
a7e2f06359 Don't forget to free a string allocated by strdup where necessary.
Found by:	Coverity Prevent
Coverity ID:	CID 3132
2007-05-19 05:09:09 +00:00
Colin Percival
9fbf190fc5 s/@VERSION@/@ARCHIVE_VERSION@/
This is a no-op as far as FreeBSD is concerned, but makes libarchive
more autoconf-friendly.

Approved by:	kientzle
2007-05-02 05:29:55 +00:00
Tim Kientzle
f1a870981d Fix a memory leak in the uname/gname lookup cache.
Thanks to: VMiklos
2007-04-20 15:32:13 +00:00
Colin Percival
6fc0ea8474 In libarchive: Downgrade ARCHIVE_FATAL and ARCHIVE_FAILED errors which
occur on the write side of extracting a file to ARCHIVE_WARN errors
when returning them from archive_read_extract.

In bsdtar: Use the return code from archive_read_data_into_fd and
archive_read_extract to determine whether we should continue trying to
extract an archive after one of the entries fails.

This commit makes extracting a truncated tarball complain once about
the archive being truncated, instead of complaining twice (once when
trying to extract an entry, and once when trying to seek to the next
entry).

Discussed with:	kientzle
2007-04-16 04:04:50 +00:00
Tim Kientzle
2a5449a7ca Properly cleanup the UID/GID lookup data. 2007-04-15 04:43:12 +00:00