Commit Graph

794 Commits

Author SHA1 Message Date
Tim Kientzle
b54f125538 Libarchive in FreeBSD is now synced up with 2.7.901a from
libarchive.googlecode.com
2009-12-30 06:25:19 +00:00
Tim Kientzle
8088bef24c UU decoder. Now that libarchive can recursively taste input streams,
you can do things like this:  tar xvf archive.tar.gz.uu
2009-12-30 06:12:03 +00:00
Tim Kientzle
0a1d787383 A raft of test changes and improvements from the Googlecode repository.
In particular, this includes tests for the new features I've merged
over the last few days.
2009-12-30 05:59:21 +00:00
Tim Kientzle
f124826e0d Merge Michihiro NAKAJIMA's significant work on the ISO9660 reader
from googlecode:
 * Support for zisofs compressed entries
 * Support for relocated deep directories
 * Direct calculation of link counts for accurate nlink values
   even on images that lack Rockridge extensions
 * Faster handling of the internal file lists.
 * Better detection of ISO variants
2009-12-30 05:30:35 +00:00
Tim Kientzle
24b76fe47f Rationalize the support for cryptographic hashes. 2009-12-29 06:39:07 +00:00
Tim Kientzle
87a571f88a Essentially eliminate problems with false hardlinks in POSIX cpio
archives by generating synthetic ino values and mapping values
from disk into the new values.
2009-12-29 06:34:23 +00:00
Tim Kientzle
9d7dd12525 Clear extraneous error messages. 2009-12-29 06:27:19 +00:00
Tim Kientzle
b1657eb07c Zip write support for libarchive.
The initial implementation was developed by Anselm Strauss as part of
Google Summer of Code 2008, then completed by Joerg Sonnenberger.
2009-12-29 06:15:32 +00:00
Tim Kientzle
84b1d75646 If we're linking against liblzma, allow the decompressor to use up to
1<<30 bytes of memory, which is required for some streams.

Also, try to make the taster more discriminating about raw lzma streams.
The detection here is still really weak, though; please use xz instead
of lzma.
2009-12-29 06:06:20 +00:00
Tim Kientzle
25557ee5f4 Minor style fixes plus a fix for an obscure crash on certain malformed
mtree files.
2009-12-29 05:52:13 +00:00
Tim Kientzle
7f16f13149 Be a little more skeptical of dev/ino matches when reading cpio files.
This eliminates some false-positives in the hardlink detection logic.
2009-12-29 05:50:34 +00:00
Tim Kientzle
023a748ef3 Style and portability fixes for pax writer. Mostly very routine, except for:
* Use ino64 where appropriate
 * Don't pass atime or nsec to ustar formatter
2009-12-29 05:47:46 +00:00
Tim Kientzle
1f8bbec2a4 Style fixes to tar reader:
For portability, prefer int64_t to off_t.
  Improve numeric overflow handling when parsing.
  Fix some variable types.
  Eliminate some unused results.
2009-12-29 05:44:39 +00:00
Tim Kientzle
88f5df0d39 Various style fixes to the 'newc' cpio writer:
* Warn about truncation of ino
 * Use a macro to hide the somewhat obscure pad-to-multiple-of-4 calculation
 * Eliminate some unused variables
2009-12-29 05:41:57 +00:00
Tim Kientzle
bb8a41af51 Various fixes when creating objects on disk:
* Write xattrs last instead of first (required on platforms that use
   system xattrs for security attributes)
 * Better handling of chdir() failures
 * Don't bother trying to shorten files via seek()/write()
 * Fix build on systems that lack link()/symlink()/mknod()
 * Prefer futimens()/utimensat() when they're present
2009-12-29 05:35:40 +00:00
Tim Kientzle
dee0ce0cbd Catch decompression failures earlier. In particular, this gives
immediate feedback if we fail to fork an external decompression program.
2009-12-29 05:30:23 +00:00
Tim Kientzle
e2196d3e85 Various style and portability fixes, including:
* Enforce option interface can only be used before the archive is opened
 * Correctly handle large skips on platforms with 32-bit off_t
 * Use int64_t instead of off_t
2009-12-29 05:28:28 +00:00
Tim Kientzle
081e9a983c 64-bit ino support and discourage bad use of this header. 2009-12-29 05:20:12 +00:00
Tim Kientzle
989f6ccc3f Portability. 2009-12-28 06:59:35 +00:00
Tim Kientzle
0d722d6394 Remove some dead assignments, fix some declarations. 2009-12-28 03:33:05 +00:00
Tim Kientzle
e15da0400a Clarify an example so Linux folks won't keep getting confused by this point. 2009-12-28 03:31:29 +00:00
Tim Kientzle
456bd00333 Compatibility with old systems with non-POSIX getpwuid_r/getgrgid_r. 2009-12-28 03:30:31 +00:00
Tim Kientzle
f773eba566 Fix some unused variables and dead assignments. 2009-12-28 03:28:21 +00:00
Tim Kientzle
108f9d8324 Remove an unused variable and an unreachable statement. 2009-12-28 03:25:33 +00:00
Tim Kientzle
409cc2477d Fix line endings. 2009-12-28 03:24:56 +00:00
Tim Kientzle
354c05f65d Discourage external use of this header. 2009-12-28 03:20:54 +00:00
Tim Kientzle
4ffba49a6f Remove unused variable. 2009-12-28 03:14:30 +00:00
Tim Kientzle
6f1d1a5084 O_BINARY for Windows. 2009-12-28 03:13:49 +00:00
Tim Kientzle
966062dd11 Handle Zip CRC validation for uncompressed Zip archives even on
platforms that lack zlib.
2009-12-28 03:11:36 +00:00
Tim Kientzle
10f117ebc2 Include limits.h; remove unused variable. 2009-12-28 03:06:27 +00:00
Tim Kientzle
09bf59d4b9 Use ino64 interface. 2009-12-28 03:05:31 +00:00
Tim Kientzle
2b1f175936 Use new ino64 interface. 2009-12-28 03:03:00 +00:00
Tim Kientzle
9adedf9679 New archive_file_count() utility. 2009-12-28 02:58:14 +00:00
Tim Kientzle
042c735c81 Portability fixes. 2009-12-28 02:45:56 +00:00
Tim Kientzle
65d5426feb Various portability fixes, plus:
* New "ino64" field.
 * New UTF8 interfaces for hardlink/symlink updates
2009-12-28 02:41:27 +00:00
Tim Kientzle
13f322aec3 Remove an unused variable and an unnecessary increment. 2009-12-28 02:33:22 +00:00
Tim Kientzle
cc1e3ebe54 Eliminate an unused assignment. 2009-12-28 02:29:21 +00:00
Tim Kientzle
2866e9d5d0 Portability: Set the file descriptor to binary mode on Win32 2009-12-28 02:28:44 +00:00
Tim Kientzle
c0840e91d3 Discourage outside use of this internal header. 2009-12-28 02:26:06 +00:00
Tim Kientzle
c17fd585bb Remove dead assignment. 2009-12-28 02:22:41 +00:00
Tim Kientzle
a5a14ed059 Portability:
* some non-FreeBSD platforms do actually define __FBSDID() in sys/cdefs.h
 * Borland C <sigh>
2009-12-28 02:22:04 +00:00
Tim Kientzle
e9cb23e15d Portability: terminate abnormally via abort() instead of segfault,
watch the return value from write(), and avoid signed arithmetic on
unsigned values.
2009-12-28 02:20:23 +00:00
Tim Kientzle
06225a399b Allow more formats to be active at a time; discourage outside use. 2009-12-28 02:18:55 +00:00
Tim Kientzle
8ead7c052b Enforce this as an internal-only header. 2009-12-28 02:18:26 +00:00
Tim Kientzle
ee674fd0b9 Discourage people from using this header outside of libarchive. 2009-12-28 02:17:53 +00:00
Tim Kientzle
1bb8e82188 Compatibility: I found some more compilers that don't like 'inline' keyword.
Protection:  Discourage people from using this header outside libarchive.
2009-12-28 02:17:15 +00:00
Tim Kientzle
f1bfc23207 Sync with googlecode some changes that have no impact on FreeBSD. 2009-12-28 02:14:09 +00:00
Tim Kientzle
96de948146 Compatibility fix for some older systems with non-POSIX getgrnam_r/getpwnam_r
and a minor style fix for the hash function.
2009-12-28 02:09:57 +00:00
Tim Kientzle
94cb976997 Catch certain gzip failures at close time. 2009-12-28 02:05:28 +00:00
Tim Kientzle
153ee32724 Remove a dead assignment. 2009-12-28 02:04:42 +00:00
Tim Kientzle
b4a4198e53 Remove some unused variables and dead assignments. 2009-12-28 02:03:54 +00:00
Tim Kientzle
23e37d63dc Update the hard-coded configuration for libarchive. 2009-12-28 02:01:42 +00:00
Tim Kientzle
ff30341ebf Update format manpages for libarchive. 2009-12-28 01:50:23 +00:00
Tim Kientzle
30a9ca8e26 Merge two cpio fixes from libarchive.googlecode.com:
1) Avoid an infinite loop in the header resync for certain malformed
    archives.
 2) Don't try to match hardlinks if the nlinks count is < 2.   This
    reduces the likelihood of a false hardlink match due to ino truncation.

MFC after:	7 days
2009-12-10 06:42:28 +00:00
Tim Kientzle
466017cf76 Fiz /usr/bin/unzip: A bug deep in libarchive's read-ahead logic
(incorrect handling of zero-length reads before the copy buffer is
allocated) is masked by the iso9660 taster.  Tar and cpio both enable
that taster so were protected from the bug; unzip is susceptible.

This both fixes the bug and updates the test harness to exercise
this case.

Submitted by: Ed Schouten diagnosed the bug and drafted a patch
MFC after: 7 days
2009-09-08 05:02:41 +00:00
Tim Kientzle
26baaf0169 Update tests to match r195873, which corrected how hardlinked files
on iso9660 images were returned.  While I'm poking around, update
some comments around this area to try to clarify what's going on and
what still remains to be improved.
2009-09-08 04:52:12 +00:00
Tim Kientzle
33fe28bbcd The parser for Rockridge symlinks tended to insert
extra slashes at the beginning of absolute targets.

Thanks to Jung-uk Kim for pointing this out to me.

Approved by:	re (kib)
2009-07-26 18:11:44 +00:00
Tim Kientzle
e2f1f1fb00 Libarchive recognizes hardlinked files on ISO images,
but returned them incorrectly, causing tar to actually
erase the resulting file while trying to restore the
link.  This one-line fix corrects the hardlink descriptions
to avoid this problem.

Thanks to Jung-uk Kim for pointing this out.

Approved by:	re (kib)
2009-07-25 18:11:55 +00:00
Ken Smith
3ca3047aee Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE.  Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by:    kib
Approved by:    re (rwatson)
2009-07-19 17:25:24 +00:00
Tim Kientzle
e8d60260d7 Fix "tar --options=iso9660:joliet" and other uses
of format-specific options.
2009-06-20 06:02:21 +00:00
Tim Kientzle
0b3daf486d Include libmd and libcrypto in DPADD 2009-05-30 17:26:55 +00:00
Tim Kientzle
65b45c166f Link libarchive against -lmd and -lcrypto.
Thanks to Ed Schouten for the clue.
2009-05-30 07:08:16 +00:00
Tim Kientzle
9f1994b5e6 Include the 2 byte length field for the optional "extra data"
field when computing the length of the gzip header.

Thanks to Dag-Erling for pointing me to the OpenSSH tarballs,
which are the first files I've seen that actually used this field.
2009-05-23 04:31:05 +00:00
Tim Kientzle
c5136006b2 Eliminate duplicate error messages from "tar c".
Reported by:	pav@
2009-05-13 00:04:08 +00:00
Tim Kientzle
fb1db28b6d Partially revert r191171, which went too far in trying
to eliminate some duplicated code.  In particular,
archive_read_open_filename() has different close
handling than archive_read_open_fd(), so delegating
the former to the latter in the degenerate case
(a NULL filename is treated as stdin) broke reading
from pipelines.  In particular, this fixes occasional
port failures that were seen when using "gunzip | tar"
pipelines under /bin/csh.

Thanks to Alexey Shuvaev for reporting this failure and
patiently helping me to track down the cause.
2009-05-07 23:01:03 +00:00
Tim Kientzle
04e8ac36ab Document the liblzma support.
Unfortunately, liblzma itself is GPLed, so unlikely to become part of
the FreeBSD base system.
However, the core lzma compression/decompression code is public
domain, so it should be feasible for someone to create a compatible
library without the GPL strings.
2009-04-27 22:39:43 +00:00
Tim Kientzle
59c7a951b5 Symlink some additional man page entries. 2009-04-27 20:23:22 +00:00
Tim Kientzle
4d24dc3687 Merge r991 from libarchive.googlecode.com: Various updates
to archive_read.3 manpage, including documentation for the
new "raw" handler.
2009-04-27 20:13:13 +00:00
Tim Kientzle
690f5ebdc0 Merge r990,r1044 from libarchive.googlecode.com:
read_support_format_raw() allows people to exploit libarchive's
automatic decompression support by simply stubbing out the
archive format handler.
The raw handler is not enabled by support_format_all(), of course.
It bids 1 on any non-empty input and always returns a single
entry named "data" with no properties set.
2009-04-27 20:09:05 +00:00
Tim Kientzle
edca00c7b2 Merge r1061,r1062,r1063 from libarchive.googlecode.com:
Fix reading big-endian binary cpio archives, and add a test.
While I'm here, add a note about Solaris ACL extension for cpio,
which should be relatively straightforward to support.

Thanks to: Edward Napierala, who sent me a big-endian cpio archive
from a Solaris system he's been playing with.
Pointy hat: me
2009-04-27 19:30:09 +00:00
Tim Kientzle
80658f77a3 Merge r1034 from libarchive.googlecode.com:
Put a much larger file on the reference ISO for this test.
2009-04-27 19:23:53 +00:00
Tim Kientzle
0cf3aa1afe Merge r1058 from libarchive.googlecode.com: Require that each
test source file has exactly one DEFINE_TEST().
2009-04-27 19:20:25 +00:00
Tim Kientzle
3da657c177 ino_t varies across platforms; casting (int) here avoids
various pointless complaints.
2009-04-27 19:14:43 +00:00
Tim Kientzle
546cb91d54 Merge r1032 from libarchive.googlecode.com:
Make test_fuzz a bit more sensitive by actually reading the body
of each entry instead of skipping it.
While I'm here, move the "UnsupportedCompress" macro into the
only file that still uses it.
2009-04-27 18:55:22 +00:00
Tim Kientzle
525ed69972 Merge r1054,r1060 from libarchive.googlecode.com:
* assertEqualMem() now takes void * arguments
 * Be a little smarter about what we hexdump when assertEqualMem() fails
2009-04-27 18:39:55 +00:00
Tim Kientzle
d906209e53 Merge r1021 from libarchive.googlecode.com:
If we know it's a socket, say so.
2009-04-27 18:35:03 +00:00
Tim Kientzle
7ad21ca8c3 Merge r1052,r1055 from libarchive.googlecode.com:
Clear the error flag on entry to a few more API functions.
2009-04-27 18:33:08 +00:00
Tim Kientzle
01a94543e9 Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com:
* Fix parsing of POSIX.1e ACLs from Solaris tar archives
 * Test the above
 * Preserve the order of POSIX.1e ACL entries
 * Update tests whose results depended on the order of ACL entries
 * Identify NFSv4 ACLs in Solaris tar archives and warn that
   they're not yet supported. (In particular, don't try to parse
   them as POSIX.1e ACLs.)

Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
Tim Kientzle
3257d55783 Merge r988,r1064 from libarchive.googlecode.com:
* Split whiny skip function to create a new best-effort skip_lenient()
 * Correctly increment the top-level file position only for the top filter
 * Simulate skip by reading against the current filter, not the top filter

The latter two bugs aren't currently visible because no existing
filter delegates skip operations.
2009-04-27 17:42:02 +00:00
Tim Kientzle
d21d99e07e Reading an mtree file is supposed to provide
access to the file data (if the file exists on
disk).  This was broken for the first regular
file; fix it and add a test so it won't break again.

In particular, this fixes the following idiom for creating
a tar archive in which every file is owned by root:

tar cf - --format=mtree . 	\
    | sed -e 's/uname=[a-z]*/uname=root/' -e 's/uid=[0-9]*/uid=0/' \
    | tar cf - @-
2009-04-26 18:57:50 +00:00
Tim Kientzle
9a4ac3e81e Various improvements to the tar.5 manpage, including
descriptions of the GNU tar "posix-style" sparse format,
clarification of the Solaris tar ACL storage,
and a few comments about Mac OS X tar's resource storage.
2009-04-26 18:46:40 +00:00
Tim Kientzle
6388433b62 Exit with ARCHIVE_FATAL if the ISO image is truncated. 2009-04-26 18:43:49 +00:00
Tim Kientzle
c49388ad34 Remove an unused variable.
Thanks to:	Christoph Mallon
2009-04-26 18:24:14 +00:00
Tim Kientzle
f4f5e3f5b9 Rely on OpenSSL bits only if we're building a system with OpenSSL.
Also, adjust the MD5 calls to rely on libmd instead of libcrypto,
so we keep MD5 support even in the !OpenSSL case.
2009-04-18 06:06:47 +00:00
Tim Kientzle
84cdfa5374 Correct and update the manpage to include
more details about some of the formats and
to briefly describe the mtree writing capability.
2009-04-18 03:47:29 +00:00
Tim Kientzle
400970912f Disabling the crypto bits should quiet tinderbox while I
track down the library dependencies that are screwing up the /rescue build.
2009-04-17 07:28:49 +00:00
Tim Kientzle
59fd3980b7 Publicize a handful of new functions and bump the version number
to indicate that this is synchronized with the recent libarchive 2.7.0
release.
2009-04-17 01:07:37 +00:00
Tim Kientzle
24a4c12895 Merge lots of test suite updates from libarchive.googlecode.com. 2009-04-17 01:06:31 +00:00
Tim Kientzle
7523935ae4 FreeBSD has a lot of crypto functions used by the recent mtree writer updates. 2009-04-17 01:04:23 +00:00
Tim Kientzle
50cc11da04 Ensure that the option setters return OK (option used) even
for options that don't change the list of keywords.
2009-04-17 01:03:52 +00:00
Tim Kientzle
c190973e9d Fix a memory leak: Release the current list of /set keywords
even when we exit on an error.
2009-04-17 01:02:56 +00:00
Tim Kientzle
7ea8a85dbc Document the new read options interface and the new read_header2() call. 2009-04-17 01:02:12 +00:00
Tim Kientzle
a5e75fbb6c Dynamically size the buffer we pass to getgrgid_r() and getpwuid_r().
Keep the buffer in the cache object so we don't have to keep doing this.
2009-04-17 01:01:15 +00:00
Tim Kientzle
82ea3751d6 Don't match an empty file on a read error. 2009-04-17 01:00:11 +00:00
Tim Kientzle
b6a1f92f24 When pulling metadata from disk, lookup the user and group name at the same time. 2009-04-17 00:59:34 +00:00
Tim Kientzle
bfe2732de8 LZW bugfix: when we hit end-of-file, return an invalid code. 2009-04-17 00:58:44 +00:00
Tim Kientzle
bf89aeae5e Minor fix: some platforms require both inttypes.h and stdint.h. 2009-04-17 00:57:11 +00:00
Tim Kientzle
06f39cd345 Implement command-line fallbacks for gzip and bzip2 decompression as well.
Not an issue for FreeBSD, since the base system has the necessary libraries.
Since all decompressors are always available now, we can unconditionally
enable them in archive_read_support_compression_all().
2009-04-17 00:55:52 +00:00
Tim Kientzle
c9f05bc8a1 Merge new xz/lzma support from libarchive.googlecode.com.
Since FreeBSD doesn't have liblzma in the base system, the
read side will always fall back to the unxz/unlzma commands for now.
(Which will in turn fail if those commands are not currently
installed.)  The write side does not yet have a fallback, so
that will just fail.
2009-04-17 00:54:35 +00:00
Tim Kientzle
a9ba4adfa5 Don't use the open callback, which is deprecated (because it's
never necessary).  Also, simplify just a tad by delegating
to read_open_fd() when we know the file descriptor, instead
of duplicating that logic.
2009-04-17 00:50:00 +00:00