d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
efa7c2b36c
freebsd-dev/lib/libarchive/archive_read_support_format_tar.c

2419 lines
67 KiB
C
Raw Normal View History

Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*-
Fix the copyright notice; it was always intended to be a vanilla 2-clause BSD license, but somehow some confusing extra verbage get copied from somewhere. Also, update the copyright dates to 2007 for all of the files. Prompted by: several questions about what those extra words really mean
2007-01-09 08:05:56 +00:00
* Copyright (c) 2003-2007 Tim Kientzle
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
Fix the copyright notice; it was always intended to be a vanilla 2-clause BSD license, but somehow some confusing extra verbage get copied from somewhere. Also, update the copyright dates to 2007 for all of the files. Prompted by: several questions about what those extra words really mean
2007-01-09 08:05:56 +00:00
* notice, this list of conditions and the following disclaimer.
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
Many fixes. Portability: Thanks to Juergen Lock, libarchive now compiles cleanly on Linux. Along the way, I cleaned up a lot of error return codes and reorganized some code to simplify conditional compilation of certain sections. Bug fixes: * pax format now actually stores filenames that are 101-154 characters long. * pax format now allows newline characters in extended attributes (this fixes a long-standing bug in ACL handling) * mtime/atime are now restored for directories * directory list is now sorted prior to fix-up to permit correct restore of non-writable dir heirarchies
2004-03-09 19:50:41 +00:00
#include "archive_platform.h"
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
__FBSDID("$FreeBSD$");
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
#ifdef HAVE_ERRNO_H
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
#include <errno.h>
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
#endif
Style issue: Don't include <wchar.h> where it is not actually needed. (wchar_t is defined in stddef.h, and only two files need more than that.) Portability: Since the wchar requirements are really quite modest, it's easy to define basic replacements for wcslen, wcscmp, wcscpy, etc, for use on systems that lack <wchar.h>. In particular, this allows libarchive to be used on older OpenBSD systems.
2005-09-10 22:58:06 +00:00
#include <stddef.h>
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/* #include <stdint.h> */ /* See archive_platform.h */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
#ifdef HAVE_STDLIB_H
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
#include <stdlib.h>
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
#endif
#ifdef HAVE_STRING_H
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
#include <string.h>
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
#endif
Style issue: Don't include <wchar.h> where it is not actually needed. (wchar_t is defined in stddef.h, and only two files need more than that.) Portability: Since the wchar requirements are really quite modest, it's easy to define basic replacements for wcslen, wcscmp, wcscpy, etc, for use on systems that lack <wchar.h>. In particular, this allows libarchive to be used on older OpenBSD systems.
2005-09-10 22:58:06 +00:00
/* Obtain suitable wide-character manipulation functions. */
#ifdef HAVE_WCHAR_H
Include wchar.h to improve our chances of finding WCHAR_MAX. This might fix a portability problem on HP_UX. Thanks to: Susan Kim
2004-12-22 06:40:28 +00:00
#include <wchar.h>
Style issue: Don't include <wchar.h> where it is not actually needed. (wchar_t is defined in stddef.h, and only two files need more than that.) Portability: Since the wchar requirements are really quite modest, it's easy to define basic replacements for wcslen, wcscmp, wcscpy, etc, for use on systems that lack <wchar.h>. In particular, this allows libarchive to be used on older OpenBSD systems.
2005-09-10 22:58:06 +00:00
#else
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
/* Good enough for equality testing, which is all we need. */
Style issue: Don't include <wchar.h> where it is not actually needed. (wchar_t is defined in stddef.h, and only two files need more than that.) Portability: Since the wchar requirements are really quite modest, it's easy to define basic replacements for wcslen, wcscmp, wcscpy, etc, for use on systems that lack <wchar.h>. In particular, this allows libarchive to be used on older OpenBSD systems.
2005-09-10 22:58:06 +00:00
static int wcscmp(const wchar_t *s1, const wchar_t *s2)
{
int diff = *s1 - *s2;
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
while (*s1 && diff == 0)
Style issue: Don't include <wchar.h> where it is not actually needed. (wchar_t is defined in stddef.h, and only two files need more than that.) Portability: Since the wchar requirements are really quite modest, it's easy to define basic replacements for wcslen, wcscmp, wcscpy, etc, for use on systems that lack <wchar.h>. In particular, this allows libarchive to be used on older OpenBSD systems.
2005-09-10 22:58:06 +00:00
diff = (int)*++s1 - (int)*++s2;
return diff;
}
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
/* Good enough for equality testing, which is all we need. */
static int wcsncmp(const wchar_t *s1, const wchar_t *s2, size_t n)
{
int diff = *s1 - *s2;
while (*s1 && diff == 0 && n-- > 0)
diff = (int)*++s1 - (int)*++s2;
return diff;
}
Style issue: Don't include <wchar.h> where it is not actually needed. (wchar_t is defined in stddef.h, and only two files need more than that.) Portability: Since the wchar requirements are really quite modest, it's easy to define basic replacements for wcslen, wcscmp, wcscpy, etc, for use on systems that lack <wchar.h>. In particular, this allows libarchive to be used on older OpenBSD systems.
2005-09-10 22:58:06 +00:00
static size_t wcslen(const wchar_t *s)
{
const wchar_t *p = s;
while (*p)
p++;
return p - s;
}
#endif
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
#include "archive.h"
#include "archive_entry.h"
#include "archive_private.h"
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
#include "archive_read_private.h"
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
#define tar_min(a,b) ((a) < (b) ? (a) : (b))
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*
* Layout of POSIX 'ustar' tar header.
*/
struct archive_entry_header_ustar {
char name[100];
char mode[8];
char uid[8];
char gid[8];
char size[12];
char mtime[12];
char checksum[8];
char typeflag[1];
char linkname[100]; /* "old format" header ends here */
char magic[6]; /* For POSIX: "ustar\0" */
char version[2]; /* For POSIX: "00" */
char uname[32];
char gname[32];
Clean up some consistent confusion between "dev" and "rdev." Mostly, these were being used correctly even though a lot of variables and function names were mis-named. In the process, I found and fixed a couple of latent bugs and added a guard against adding an archive to itself.
2004-06-27 18:38:13 +00:00
char rdevmajor[8];
char rdevminor[8];
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
char prefix[155];
};
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
/*
* Structure of GNU tar header
*/
struct gnu_sparse {
char offset[12];
char numbytes[12];
};
struct archive_entry_header_gnutar {
char name[100];
char mode[8];
char uid[8];
char gid[8];
char size[12];
char mtime[12];
char checksum[8];
char typeflag[1];
char linkname[100];
char magic[8]; /* "ustar \0" (note blank/blank/null at end) */
char uname[32];
char gname[32];
Clean up some consistent confusion between "dev" and "rdev." Mostly, these were being used correctly even though a lot of variables and function names were mis-named. In the process, I found and fixed a couple of latent bugs and added a guard against adding an archive to itself.
2004-06-27 18:38:13 +00:00
char rdevmajor[8];
char rdevminor[8];
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
char atime[12];
char ctime[12];
char offset[12];
char longnames[4];
char unused[1];
struct gnu_sparse sparse[4];
char isextended[1];
char realsize[12];
/*
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
* Old GNU format doesn't use POSIX 'prefix' field; they use
* the 'L' (longname) entry instead.
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
*/
};
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/*
* Data specific to this format.
*/
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
struct sparse_block {
struct sparse_block *next;
off_t offset;
off_t remaining;
};
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
struct tar {
struct archive_string acl_text;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
struct archive_string entry_pathname;
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
/* For "GNU.sparse.name" and other similar path extensions. */
struct archive_string entry_pathname_override;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
struct archive_string entry_linkpath;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
struct archive_string entry_uname;
struct archive_string entry_gname;
struct archive_string longlink;
struct archive_string longname;
struct archive_string pax_header;
struct archive_string pax_global;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
struct archive_string line;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
int pax_hdrcharset_binary;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
wchar_t *pax_entry;
size_t pax_entry_length;
int header_recursion_depth;
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
int64_t entry_bytes_remaining;
int64_t entry_offset;
int64_t entry_padding;
int64_t realsize;
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
struct sparse_block *sparse_list;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
struct sparse_block *sparse_last;
int64_t sparse_offset;
int64_t sparse_numbytes;
int sparse_gnu_major;
int sparse_gnu_minor;
char sparse_gnu_pending;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
};
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static ssize_t UTF8_mbrtowc(wchar_t *pwc, const char *s, size_t n);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
static int archive_block_is_null(const unsigned char *p);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static char *base64_decode(const char *, size_t, size_t *);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static void gnu_add_sparse_entry(struct tar *,
off_t offset, off_t remaining);
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
static void gnu_clear_sparse_list(struct tar *);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static int gnu_sparse_old_read(struct archive_read *, struct tar *,
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
const struct archive_entry_header_gnutar *header);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static void gnu_sparse_old_parse(struct tar *,
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
const struct gnu_sparse *sparse, int length);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static int gnu_sparse_01_parse(struct tar *, const char *);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static ssize_t gnu_sparse_10_read(struct archive_read *, struct tar *);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_Solaris_ACL(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_common(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_old_tar(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_pax_extensions(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_pax_global(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *h);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_longlink(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *h);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_longname(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *h);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_volume(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *h);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_ustar(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *h);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int header_gnutar(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, const void *h);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int archive_read_format_tar_bid(struct archive_read *);
static int archive_read_format_tar_cleanup(struct archive_read *);
static int archive_read_format_tar_read_data(struct archive_read *a,
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
const void **buff, size_t *size, off_t *offset);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int archive_read_format_tar_skip(struct archive_read *a);
static int archive_read_format_tar_read_header(struct archive_read *,
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
struct archive_entry *);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int checksum(struct archive_read *, const void *);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static int pax_attribute(struct tar *, struct archive_entry *,
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
char *key, char *value);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int pax_header(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *, char *attr);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static void pax_time(const char *, int64_t *sec, long *nanos);
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
static ssize_t readline(struct archive_read *, struct tar *, const char **,
ssize_t limit);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int read_body_to_string(struct archive_read *, struct tar *,
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
struct archive_string *, const void *h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
static int64_t tar_atol(const char *, unsigned);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static int64_t tar_atol10(const char *, unsigned);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
static int64_t tar_atol256(const char *, unsigned);
static int64_t tar_atol8(const char *, unsigned);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
static int tar_read_header(struct archive_read *, struct tar *,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *);
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
static int tohex(int c);
static char *url_decode(const char *);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static wchar_t *utf8_decode(struct tar *, const char *, size_t length);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
int
archive_read_support_format_gnutar(struct archive *a)
{
return (archive_read_support_format_tar(a));
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_read_support_format_tar(struct archive *_a)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
struct archive_read *a = (struct archive_read *)_a;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
struct tar *tar;
* Plug a buffer overrun in ACL parsing. (archive_entry.c) * Re-use a single buffer for shar output formatting rather than hammering the heap. (archive_write_set_format_shar.c) * Fix a handful of minor memory leaks and clean up some of the memory-management code.
2004-04-13 23:45:37 +00:00
int r;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
tar = (struct tar *)malloc(sizeof(*tar));
Add a lot of error checks, based on the patches provided by Dan Lukes. Also fixes a memory leak reported by Andrew Turner. PR: bin/83476 Thanks to: Dan Lukes, Andrew Turner
2005-09-21 04:25:06 +00:00
if (tar == NULL) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, ENOMEM,
"Can't allocate tar data");
Add a lot of error checks, based on the patches provided by Dan Lukes. Also fixes a memory leak reported by Andrew Turner. PR: bin/83476 Thanks to: Dan Lukes, Andrew Turner
2005-09-21 04:25:06 +00:00
return (ARCHIVE_FATAL);
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
memset(tar, 0, sizeof(*tar));
Merge r491,493,500,507,510,530,543 from libarchive.googlecode.com: This implements the new generic options framework that provides a way to override format- and compression-specific parameters.
2009-03-06 05:58:56 +00:00
r = __archive_read_register_format(a, tar, "tar",
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
archive_read_format_tar_bid,
Merge r491,493,500,507,510,530,543 from libarchive.googlecode.com: This implements the new generic options framework that provides a way to override format- and compression-specific parameters.
2009-03-06 05:58:56 +00:00
NULL,
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
archive_read_format_tar_read_header,
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
archive_read_format_tar_read_data,
Use 'skip' when ignoring data in tar archives. This dramatically increases performance when extracting a single entry from a large uncompressed archive, especially on slow devices such as USB hard drives. Requires a number of changes: * New archive_read_open2() supports a 'skip' client function * Old archive_read_open() is implemented as a wrapper now, to continue supporting the old API/ABI. * _read_open_fd and _read_open_file sprout new 'skip' functions. * compression layer gets a new 'skip' operation. * compression_none passes skip requests through to client. * compression_{gzip,bzip2,compress} simply ignore skip requests. Thanks to: Benjamin Lutz, who designed and implemented the whole thing. I'm just committing it. ;-) TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
archive_read_format_tar_skip,
* Plug a buffer overrun in ACL parsing. (archive_entry.c) * Re-use a single buffer for shar output formatting rather than hammering the heap. (archive_write_set_format_shar.c) * Fix a handful of minor memory leaks and clean up some of the memory-management code.
2004-04-13 23:45:37 +00:00
archive_read_format_tar_cleanup);
if (r != ARCHIVE_OK)
free(tar);
return (ARCHIVE_OK);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_read_format_tar_cleanup(struct archive_read *a)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
struct tar *tar;
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
tar = (struct tar *)(a->format->data);
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
gnu_clear_sparse_list(tar);
* Plug a buffer overrun in ACL parsing. (archive_entry.c) * Re-use a single buffer for shar output formatting rather than hammering the heap. (archive_write_set_format_shar.c) * Fix a handful of minor memory leaks and clean up some of the memory-management code.
2004-04-13 23:45:37 +00:00
archive_string_free(&tar->acl_text);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_string_free(&tar->entry_pathname);
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
archive_string_free(&tar->entry_pathname_override);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_string_free(&tar->entry_linkpath);
* Plug a buffer overrun in ACL parsing. (archive_entry.c) * Re-use a single buffer for shar output formatting rather than hammering the heap. (archive_write_set_format_shar.c) * Fix a handful of minor memory leaks and clean up some of the memory-management code.
2004-04-13 23:45:37 +00:00
archive_string_free(&tar->entry_uname);
archive_string_free(&tar->entry_gname);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
archive_string_free(&tar->line);
* Plug a buffer overrun in ACL parsing. (archive_entry.c) * Re-use a single buffer for shar output formatting rather than hammering the heap. (archive_write_set_format_shar.c) * Fix a handful of minor memory leaks and clean up some of the memory-management code.
2004-04-13 23:45:37 +00:00
archive_string_free(&tar->pax_global);
archive_string_free(&tar->pax_header);
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
archive_string_free(&tar->longname);
archive_string_free(&tar->longlink);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
free(tar->pax_entry);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
free(tar);
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
(a->format->data) = NULL;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
return (ARCHIVE_OK);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_read_format_tar_bid(struct archive_read *a)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
int bid;
const void *h;
const struct archive_entry_header_ustar *header;
bid = 0;
/* Now let's look at the actual header and see if it matches. */
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
h = __archive_read_ahead(a, 512, NULL);
if (h == NULL)
return (-1);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* If it's an end-of-archive mark, we can handle it. */
Since the tar bidder can never get called more than once, it doesn't need to compensate for this situation. While here, fix a minor longstanding bug that empty tar archives (which begin with at least 512 zero bytes) never properly reported their format. In particular, this fixes the output of: bsdtar tvvf /dev/zero And, of course, a new test to verify that libarchive correctly recognizes the format of such files.
2008-01-13 23:50:30 +00:00
if ((*(const char *)h) == 0
&& archive_block_is_null((const unsigned char *)h)) {
/*
* Usually, I bid the number of bits verified, but
* in this case, 4096 seems excessive so I picked 10 as
* an arbitrary but reasonable-seeming value.
*/
return (10);
Tune the bidding for tar archives. This improves the recognition of hardlink entries with/without bodies (which is implemented through a look-ahead that uses the bid function). MFC after: 7 days
2004-12-22 00:49:16 +00:00
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* If it's not an end-of-archive mark, it must have a valid checksum.*/
if (!checksum(a, h))
return (0);
bid += 48; /* Checksum is usually 6 octal digits. */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_ustar *)h;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Be smarter about hardlink sizes: some tar programs write a non-zero size but no body, some write a non-zero size and include a body. To distinguish these cases, look for a valid tar header immediately following a hardlink header with non-zero size.
2004-05-18 18:16:30 +00:00
/* Recognize POSIX formats. */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
if ((memcmp(header->magic, "ustar\0", 6) == 0)
&&(memcmp(header->version, "00", 2)==0))
bid += 56;
Be smarter about hardlink sizes: some tar programs write a non-zero size but no body, some write a non-zero size and include a body. To distinguish these cases, look for a valid tar header immediately following a hardlink header with non-zero size.
2004-05-18 18:16:30 +00:00
/* Recognize GNU tar format. */
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
if ((memcmp(header->magic, "ustar ", 6) == 0)
&&(memcmp(header->version, " \0", 2)==0))
bid += 56;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Type flag must be null, digit or A-Z, a-z. */
if (header->typeflag[0] != 0 &&
!( header->typeflag[0] >= '0' && header->typeflag[0] <= '9') &&
!( header->typeflag[0] >= 'A' && header->typeflag[0] <= 'Z') &&
!( header->typeflag[0] >= 'a' && header->typeflag[0] <= 'z') )
return (0);
Tune the bidding for tar archives. This improves the recognition of hardlink entries with/without bodies (which is implemented through a look-ahead that uses the bid function). MFC after: 7 days
2004-12-22 00:49:16 +00:00
bid += 2; /* 6 bits of variation in an 8-bit field leaves 2 bits. */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Sanity check: Look at first byte of mode field. */
switch (255 & (unsigned)header->mode[0]) {
case 0: case 255:
/* Base-256 value: No further verification possible! */
break;
case ' ': /* Not recommended, but not illegal, either. */
break;
case '0': case '1': case '2': case '3':
case '4': case '5': case '6': case '7':
/* Octal Value. */
/* TODO: Check format of remainder of this field. */
break;
default:
/* Not a valid mode; bail out here. */
return (0);
}
Clean up some consistent confusion between "dev" and "rdev." Mostly, these were being used correctly even though a lot of variables and function names were mis-named. In the process, I found and fixed a couple of latent bugs and added a guard against adding an archive to itself.
2004-06-27 18:38:13 +00:00
/* TODO: Sanity test uid/gid/size/mtime/rdevmajor/rdevminor fields. */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (bid);
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/*
* The function invoked by archive_read_header(). This
* just sets up a few things and then calls the internal
* tar_read_header() function below.
*/
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_read_format_tar_read_header(struct archive_read *a,
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
struct archive_entry *entry)
{
Generate default fake "device" and "inode" numbers for entries extracted from tar archives. Otherwise, converting tar archives to cpio format (with "bsdtar -cf out.cpio @in.tar") convert every entry into a hard link to a single file. This simple logic breaks hard links, but that's better than the alternative. MFC after: 7 days
2005-08-02 03:17:57 +00:00
/*
* When converting tar archives to cpio archives, it is
* essential that each distinct file have a distinct inode
* number. To simplify this, we keep a static count here to
* assign fake dev/inode numbers to each tar entry. Note that
* pax format archives may overwrite this with something more
* useful.
*
* Ideally, we would track every file read from the archive so
* that we could assign the same dev/ino pair to hardlinks,
* but the memory required to store a complete lookup table is
* probably not worthwhile just to support the relatively
* obscure tar->cpio conversion case.
*/
static int default_inode;
static int default_dev;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
struct tar *tar;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
struct sparse_block *sp;
Some old tar archives rely on "regular-file-plus-trailing-slash" to denote a directory. Unfortunately, in the presence of GNU or POSIX extensions, this code was checking the truncated filename stored in the regular header rather than the full filename stored in the extended attribute. As a result, long filenames with '/' in just the right position would trigger this check and be erroneously marked as directories. Move the check so it only considers the full filename. Note: the check can't simply be disabled for archives that contain these extensions because there are some very broken archivers out there. Thanks to: Will Froning MFC after: 3 days
2004-09-04 21:49:42 +00:00
const char *p;
int r;
size_t l;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Generate default fake "device" and "inode" numbers for entries extracted from tar archives. Otherwise, converting tar archives to cpio format (with "bsdtar -cf out.cpio @in.tar") convert every entry into a hard link to a single file. This simple logic breaks hard links, but that's better than the alternative. MFC after: 7 days
2005-08-02 03:17:57 +00:00
/* Assign default device/inode values. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_dev(entry, 1 + default_dev); /* Don't use zero. */
archive_entry_set_ino(entry, ++default_inode); /* Don't use zero. */
Generate default fake "device" and "inode" numbers for entries extracted from tar archives. Otherwise, converting tar archives to cpio format (with "bsdtar -cf out.cpio @in.tar") convert every entry into a hard link to a single file. This simple logic breaks hard links, but that's better than the alternative. MFC after: 7 days
2005-08-02 03:17:57 +00:00
/* Limit generated st_ino number to 16 bits. */
if (default_inode >= 0xffff) {
++default_dev;
default_inode = 0;
}
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
tar = (struct tar *)(a->format->data);
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
tar->entry_offset = 0;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
while (tar->sparse_list != NULL) {
sp = tar->sparse_list;
tar->sparse_list = sp->next;
free(sp);
}
tar->sparse_last = NULL;
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
tar->realsize = -1; /* Mark this as "unset" */
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
r = tar_read_header(a, tar, entry);
Some old tar archives rely on "regular-file-plus-trailing-slash" to denote a directory. Unfortunately, in the presence of GNU or POSIX extensions, this code was checking the truncated filename stored in the regular header rather than the full filename stored in the extended attribute. As a result, long filenames with '/' in just the right position would trigger this check and be erroneously marked as directories. Move the check so it only considers the full filename. Note: the check can't simply be disabled for archives that contain these extensions because there are some very broken archivers out there. Thanks to: Will Froning MFC after: 3 days
2004-09-04 21:49:42 +00:00
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/*
* "non-sparse" files are really just sparse files with
* a single block.
*/
if (tar->sparse_list == NULL)
gnu_add_sparse_entry(tar, 0, tar->entry_bytes_remaining);
Some old tar archives rely on "regular-file-plus-trailing-slash" to denote a directory. Unfortunately, in the presence of GNU or POSIX extensions, this code was checking the truncated filename stored in the regular header rather than the full filename stored in the extended attribute. As a result, long filenames with '/' in just the right position would trigger this check and be erroneously marked as directories. Move the check so it only considers the full filename. Note: the check can't simply be disabled for archives that contain these extensions because there are some very broken archivers out there. Thanks to: Will Froning MFC after: 3 days
2004-09-04 21:49:42 +00:00
if (r == ARCHIVE_OK) {
/*
* "Regular" entry with trailing '/' is really
* directory: This is needed for certain old tar
* variants and even for some broken newer ones.
*/
p = archive_entry_pathname(entry);
l = strlen(p);
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
if (archive_entry_filetype(entry) == AE_IFREG
&& p[l-1] == '/')
archive_entry_set_filetype(entry, AE_IFDIR);
Some old tar archives rely on "regular-file-plus-trailing-slash" to denote a directory. Unfortunately, in the presence of GNU or POSIX extensions, this code was checking the truncated filename stored in the regular header rather than the full filename stored in the extended attribute. As a result, long filenames with '/' in just the right position would trigger this check and be erroneously marked as directories. Move the check so it only considers the full filename. Note: the check can't simply be disabled for archives that contain these extensions because there are some very broken archivers out there. Thanks to: Will Froning MFC after: 3 days
2004-09-04 21:49:42 +00:00
}
return (r);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_read_format_tar_read_data(struct archive_read *a,
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
const void **buff, size_t *size, off_t *offset)
{
ssize_t bytes_read;
struct tar *tar;
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
struct sparse_block *p;
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
tar = (struct tar *)(a->format->data);
When reading GNU-style sparse archive entries, handle the first sparse block correctly (we used to assume that the first sparse block was always at offset zero).
2005-10-12 03:27:46 +00:00
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (tar->sparse_gnu_pending) {
if (tar->sparse_gnu_major == 1 && tar->sparse_gnu_minor == 0) {
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
tar->sparse_gnu_pending = 0;
/* Read initial sparse map. */
bytes_read = gnu_sparse_10_read(a, tar);
tar->entry_bytes_remaining -= bytes_read;
if (bytes_read < 0)
return (bytes_read);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
} else {
*size = 0;
*offset = 0;
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
"Unrecognized GNU sparse file format");
return (ARCHIVE_WARN);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->sparse_gnu_pending = 0;
}
/* Remove exhausted entries from sparse list. */
while (tar->sparse_list != NULL &&
tar->sparse_list->remaining == 0) {
p = tar->sparse_list;
tar->sparse_list = p->next;
free(p);
}
/* If we're at end of file, return EOF. */
if (tar->sparse_list == NULL || tar->entry_bytes_remaining == 0) {
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
if (__archive_read_skip(a, tar->entry_padding) < 0)
Now that there is always a compression-layer skip function available, skip over the end-of-entry padding instead of reading and discarding it. Considering that tar files normally have a block size of 10kB, this isn't likely to avoid reading any data, but at least it makes the code simpler and clearer.
2007-04-02 04:21:22 +00:00
return (ARCHIVE_FATAL);
tar->entry_padding = 0;
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
*buff = NULL;
*size = 0;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
*offset = tar->realsize;
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
return (ARCHIVE_EOF);
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
*buff = __archive_read_ahead(a, 1, &bytes_read);
if (bytes_read < 0)
return (ARCHIVE_FATAL);
if (*buff == NULL) {
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Truncated tar archive");
return (ARCHIVE_FATAL);
}
if (bytes_read > tar->entry_bytes_remaining)
bytes_read = tar->entry_bytes_remaining;
/* Don't read more than is available in the
* current sparse block. */
if (tar->sparse_list->remaining < bytes_read)
bytes_read = tar->sparse_list->remaining;
*size = bytes_read;
*offset = tar->sparse_list->offset;
tar->sparse_list->remaining -= bytes_read;
tar->sparse_list->offset += bytes_read;
tar->entry_bytes_remaining -= bytes_read;
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
__archive_read_consume(a, bytes_read);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
return (ARCHIVE_OK);
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
}
Use 'skip' when ignoring data in tar archives. This dramatically increases performance when extracting a single entry from a large uncompressed archive, especially on slow devices such as USB hard drives. Requires a number of changes: * New archive_read_open2() supports a 'skip' client function * Old archive_read_open() is implemented as a wrapper now, to continue supporting the old API/ABI. * _read_open_fd and _read_open_file sprout new 'skip' functions. * compression layer gets a new 'skip' operation. * compression_none passes skip requests through to client. * compression_{gzip,bzip2,compress} simply ignore skip requests. Thanks to: Benjamin Lutz, who designed and implemented the whole thing. I'm just committing it. ;-) TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_read_format_tar_skip(struct archive_read *a)
Use 'skip' when ignoring data in tar archives. This dramatically increases performance when extracting a single entry from a large uncompressed archive, especially on slow devices such as USB hard drives. Requires a number of changes: * New archive_read_open2() supports a 'skip' client function * Old archive_read_open() is implemented as a wrapper now, to continue supporting the old API/ABI. * _read_open_fd and _read_open_file sprout new 'skip' functions. * compression layer gets a new 'skip' operation. * compression_none passes skip requests through to client. * compression_{gzip,bzip2,compress} simply ignore skip requests. Thanks to: Benjamin Lutz, who designed and implemented the whole thing. I'm just committing it. ;-) TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
{
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
int64_t bytes_skipped;
Use 'skip' when ignoring data in tar archives. This dramatically increases performance when extracting a single entry from a large uncompressed archive, especially on slow devices such as USB hard drives. Requires a number of changes: * New archive_read_open2() supports a 'skip' client function * Old archive_read_open() is implemented as a wrapper now, to continue supporting the old API/ABI. * _read_open_fd and _read_open_file sprout new 'skip' functions. * compression layer gets a new 'skip' operation. * compression_none passes skip requests through to client. * compression_{gzip,bzip2,compress} simply ignore skip requests. Thanks to: Benjamin Lutz, who designed and implemented the whole thing. I'm just committing it. ;-) TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
struct tar* tar;
Some minor corrections: * Expose functions for setting the "skip file" dev/ino information * Expose functions for setting/querying the block size on reads * Correctly propagate errors out of archive_read_close/archive_write_close * Update manpage with information about new functions
2006-09-05 05:59:46 +00:00
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
tar = (struct tar *)(a->format->data);
Rewrite and simplify archive_read_format_tar_skip. Compression-layer skip functions are required to skip the requested distance, so we can avoid lots of bookkeeping which would otherwise be necessary. Reviewed by: kientzle MFC after: 1 week
2007-01-03 21:47:35 +00:00
/*
* Compression layer skip functions are required to either skip the
* length requested or fail, so we can rely upon the entire entry
* plus padding being skipped.
*/
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
bytes_skipped = __archive_read_skip(a,
tar->entry_bytes_remaining + tar->entry_padding);
Use 'skip' when ignoring data in tar archives. This dramatically increases performance when extracting a single entry from a large uncompressed archive, especially on slow devices such as USB hard drives. Requires a number of changes: * New archive_read_open2() supports a 'skip' client function * Old archive_read_open() is implemented as a wrapper now, to continue supporting the old API/ABI. * _read_open_fd and _read_open_file sprout new 'skip' functions. * compression layer gets a new 'skip' operation. * compression_none passes skip requests through to client. * compression_{gzip,bzip2,compress} simply ignore skip requests. Thanks to: Benjamin Lutz, who designed and implemented the whole thing. I'm just committing it. ;-) TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
if (bytes_skipped < 0)
return (ARCHIVE_FATAL);
Rewrite and simplify archive_read_format_tar_skip. Compression-layer skip functions are required to skip the requested distance, so we can avoid lots of bookkeeping which would otherwise be necessary. Reviewed by: kientzle MFC after: 1 week
2007-01-03 21:47:35 +00:00
tar->entry_bytes_remaining = 0;
tar->entry_padding = 0;
/* Free the sparse list. */
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
gnu_clear_sparse_list(tar);
Rewrite and simplify archive_read_format_tar_skip. Compression-layer skip functions are required to skip the requested distance, so we can avoid lots of bookkeeping which would otherwise be necessary. Reviewed by: kientzle MFC after: 1 week
2007-01-03 21:47:35 +00:00
return (ARCHIVE_OK);
Use 'skip' when ignoring data in tar archives. This dramatically increases performance when extracting a single entry from a large uncompressed archive, especially on slow devices such as USB hard drives. Requires a number of changes: * New archive_read_open2() supports a 'skip' client function * Old archive_read_open() is implemented as a wrapper now, to continue supporting the old API/ABI. * _read_open_fd and _read_open_file sprout new 'skip' functions. * compression layer gets a new 'skip' operation. * compression_none passes skip requests through to client. * compression_{gzip,bzip2,compress} simply ignore skip requests. Thanks to: Benjamin Lutz, who designed and implemented the whole thing. I'm just committing it. ;-) TODO: Need to update the documentation a little bit.
2006-07-30 00:29:01 +00:00
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/*
* This function recursively interprets all of the headers associated
* with a single entry.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
tar_read_header(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
ssize_t bytes;
int err;
const void *h;
const struct archive_entry_header_ustar *header;
/* Read 512-byte header record */
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
h = __archive_read_ahead(a, 512, &bytes);
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
if (bytes < 0)
return (bytes);
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
if (bytes < 512) { /* Short read or EOF. */
/* Try requesting just one byte and see what happens. */
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
(void)__archive_read_ahead(a, 1, &bytes);
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
if (bytes == 0) {
/*
* The archive ends at a 512-byte boundary but
* without a proper end-of-archive marker.
* Yes, there are tar writers that do this;
* hold our nose and accept it.
*/
return (ARCHIVE_EOF);
}
/* Archive ends with a partial block; this is bad. */
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
archive_set_error(&a->archive, ARCHIVE_ERRNO_FILE_FORMAT,
"Truncated tar archive");
return (ARCHIVE_FATAL);
}
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
__archive_read_consume(a, 512);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Check for end-of-archive mark. */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
if (((*(const char *)h)==0) && archive_block_is_null((const unsigned char *)h)) {
When we go to read the next tar header, if we get zero bytes, accept that as end-of-archive. Otherwise, a short read at this point generates an error. This accomodates broken tar writers (such as the one apparently in use at AT&T Labs) that don't even write a single end-of-archive block. Note that both star and pdtar behave this way as well. In contrast, gtar doesn't complain in either case, and as a result, will generate no warning for a lot of trashed archives. Pointed out by: shells/ksh93 port (Thanks to Kris Kennaway)
2004-06-04 10:27:23 +00:00
/* Try to consume a second all-null record, as well. */
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
h = __archive_read_ahead(a, 512, NULL);
if (h != NULL)
__archive_read_consume(a, 512);
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, 0, NULL);
Since the tar bidder can never get called more than once, it doesn't need to compensate for this situation. While here, fix a minor longstanding bug that empty tar archives (which begin with at least 512 zero bytes) never properly reported their format. In particular, this fixes the output of: bsdtar tvvf /dev/zero And, of course, a new test to verify that libarchive correctly recognizes the format of such files.
2008-01-13 23:50:30 +00:00
if (a->archive.archive_format_name == NULL) {
a->archive.archive_format = ARCHIVE_FORMAT_TAR;
a->archive.archive_format_name = "tar";
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (ARCHIVE_EOF);
}
/*
* Note: If the checksum fails and we return ARCHIVE_RETRY,
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
* then the client is likely to just retry. This is a very
* crude way to search for the next valid header!
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
*
* TODO: Improve this by implementing a real header scan.
*/
if (!checksum(a, h)) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, EINVAL, "Damaged tar archive");
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (ARCHIVE_RETRY); /* Retryable: Invalid header */
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
if (++tar->header_recursion_depth > 32) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, EINVAL, "Too many special headers");
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
return (ARCHIVE_WARN);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Determine the format variant. */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_ustar *)h;
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
switch(header->typeflag[0]) {
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
case 'A': /* Solaris tar ACL */
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
a->archive.archive_format = ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE;
a->archive.archive_format_name = "Solaris tar";
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_Solaris_ACL(a, tar, entry, h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
case 'g': /* POSIX-standard 'g' header. */
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
a->archive.archive_format = ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE;
a->archive.archive_format_name = "POSIX pax interchange format";
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_pax_global(a, tar, entry, h);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
break;
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
case 'K': /* Long link name (GNU tar, others) */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_longlink(a, tar, entry, h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
case 'L': /* Long filename (GNU tar, others) */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_longname(a, tar, entry, h);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
break;
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
case 'V': /* GNU volume header */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_volume(a, tar, entry, h);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
break;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
case 'X': /* Used by SUN tar; same as 'x'. */
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
a->archive.archive_format = ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE;
a->archive.archive_format_name =
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
"POSIX pax interchange format (Sun variant)";
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_pax_extensions(a, tar, entry, h);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
break;
case 'x': /* POSIX-standard 'x' header. */
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
a->archive.archive_format = ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE;
a->archive.archive_format_name = "POSIX pax interchange format";
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_pax_extensions(a, tar, entry, h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
default:
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
if (memcmp(header->magic, "ustar \0", 8) == 0) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
a->archive.archive_format = ARCHIVE_FORMAT_TAR_GNUTAR;
a->archive.archive_format_name = "GNU tar format";
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_gnutar(a, tar, entry, h);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
} else if (memcmp(header->magic, "ustar", 5) == 0) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
if (a->archive.archive_format != ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE) {
a->archive.archive_format = ARCHIVE_FORMAT_TAR_USTAR;
a->archive.archive_format_name = "POSIX ustar format";
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
}
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_ustar(a, tar, entry, h);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
} else {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
a->archive.archive_format = ARCHIVE_FORMAT_TAR;
a->archive.archive_format_name = "tar (non-POSIX)";
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = header_old_tar(a, tar, entry, h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
--tar->header_recursion_depth;
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
/* We return warnings or success as-is. Anything else is fatal. */
if (err == ARCHIVE_WARN || err == ARCHIVE_OK)
return (err);
if (err == ARCHIVE_EOF)
/* EOF when recursively reading a header is bad. */
archive_set_error(&a->archive, EINVAL, "Damaged tar archive");
return (ARCHIVE_FATAL);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
/*
* Return true if block checksum is correct.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
checksum(struct archive_read *a, const void *h)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
const unsigned char *bytes;
const struct archive_entry_header_ustar *header;
int check, i, sum;
(void)a; /* UNUSED */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
bytes = (const unsigned char *)h;
header = (const struct archive_entry_header_ustar *)h;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*
* Test the checksum. Note that POSIX specifies _unsigned_
* bytes for this calculation.
*/
sum = tar_atol(header->checksum, sizeof(header->checksum));
check = 0;
for (i = 0; i < 148; i++)
check += (unsigned char)bytes[i];
for (; i < 156; i++)
check += 32;
for (; i < 512; i++)
check += (unsigned char)bytes[i];
if (sum == check)
return (1);
/*
* Repeat test with _signed_ bytes, just in case this archive
* was created by an old BSD, Solaris, or HP-UX tar with a
* broken checksum calculation.
*/
check = 0;
for (i = 0; i < 148; i++)
check += (signed char)bytes[i];
for (; i < 156; i++)
check += 32;
for (; i < 512; i++)
check += (signed char)bytes[i];
if (sum == check)
return (1);
return (0);
}
/*
* Return true if this block contains only nulls.
*/
static int
archive_block_is_null(const unsigned char *p)
{
unsigned i;
A block in a tar file is 512 bytes. Period. Remove the entirely pointless symbolic constant and sizeof(unsigned char). (The constant here is doubly wrong, since not only does it obscure a basic format constant, it was never intended to be a tar-specific value, so could conceivably be changed at some point in the future.)
2008-03-14 20:32:20 +00:00
for (i = 0; i < 512; i++)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
if (*p++)
return (0);
return (1);
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/*
* Interpret 'A' Solaris ACL header
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
header_Solaris_ACL(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry, const void *h)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
const struct archive_entry_header_ustar *header;
size_t size;
int err;
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
int64_t type;
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
char *acl, *p;
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
wchar_t *wp;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
/*
* read_body_to_string adds a NUL terminator, but we need a little
* more to make sure that we don't overrun acl_text later.
*/
header = (const struct archive_entry_header_ustar *)h;
size = tar_atol(header->size, sizeof(header->size));
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
err = read_body_to_string(a, tar, &(tar->acl_text), h);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if (err != ARCHIVE_OK)
return (err);
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
/* Recursively read next header */
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
err = tar_read_header(a, tar, entry);
if ((err != ARCHIVE_OK) && (err != ARCHIVE_WARN))
return (err);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
/* TODO: Examine the first characters to see if this
* is an AIX ACL descriptor. We'll likely never support
* them, but it would be polite to recognize and warn when
* we do see them. */
/* Leading octal number indicates ACL type and number of entries. */
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
p = acl = tar->acl_text.s;
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
type = 0;
while (*p != '\0' && p < acl + size) {
if (*p < '0' || *p > '7') {
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Malformed Solaris ACL attribute (invalid digit)");
return(ARCHIVE_WARN);
}
type <<= 3;
type += *p - '0';
if (type > 077777777) {
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Malformed Solaris ACL attribute (count too large)");
return (ARCHIVE_WARN);
}
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
p++;
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
}
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
switch ((int)type & ~0777777) {
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
case 01000000:
/* POSIX.1e ACL */
break;
case 03000000:
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Solaris NFSv4 ACLs not supported");
return (ARCHIVE_WARN);
default:
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Malformed Solaris ACL attribute (unsupported type %o)",
(int)type);
return (ARCHIVE_WARN);
}
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
p++;
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if (p >= acl + size) {
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
"Malformed Solaris ACL attribute (body overflow)");
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
return(ARCHIVE_WARN);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
/* ACL text is null-terminated; find the end. */
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
size -= (p - acl);
acl = p;
while (*p != '\0' && p < acl + size)
p++;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
wp = utf8_decode(tar, acl, p - acl);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
err = __archive_entry_acl_parse_w(entry, wp,
ARCHIVE_ENTRY_ACL_TYPE_ACCESS);
Merge r1053,r1055,r1056,r1057,r1065 from libarchive.googlecode.com: * Fix parsing of POSIX.1e ACLs from Solaris tar archives * Test the above * Preserve the order of POSIX.1e ACL entries * Update tests whose results depended on the order of ACL entries * Identify NFSv4 ACLs in Solaris tar archives and warn that they're not yet supported. (In particular, don't try to parse them as POSIX.1e ACLs.) Thanks to: Edward Napierala sent me some Solaris 10 tar archives to test
2009-04-27 18:27:54 +00:00
if (err != ARCHIVE_OK)
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Malformed Solaris ACL attribute (unparsable)");
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
return (err);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
/*
* Interpret 'K' long linkname header.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
header_longlink(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry, const void *h)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
int err;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
err = read_body_to_string(a, tar, &(tar->longlink), h);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if (err != ARCHIVE_OK)
return (err);
err = tar_read_header(a, tar, entry);
if ((err != ARCHIVE_OK) && (err != ARCHIVE_WARN))
return (err);
/* Set symlink if symlink already set, else hardlink. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_link(entry, tar->longlink.s);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
return (ARCHIVE_OK);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
/*
* Interpret 'L' long filename header.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
header_longname(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry, const void *h)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
int err;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
err = read_body_to_string(a, tar, &(tar->longname), h);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if (err != ARCHIVE_OK)
return (err);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/* Read and parse "real" header, then override name. */
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
err = tar_read_header(a, tar, entry);
if ((err != ARCHIVE_OK) && (err != ARCHIVE_WARN))
return (err);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_pathname(entry, tar->longname.s);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
return (ARCHIVE_OK);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
/*
* Interpret 'V' GNU tar volume header.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
header_volume(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry, const void *h)
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
{
(void)h;
/* Just skip this and read the next header. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
return (tar_read_header(a, tar, entry));
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/*
* Read body of an archive entry into an archive_string object.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
read_body_to_string(struct archive_read *a, struct tar *tar,
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
struct archive_string *as, const void *h)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
Correct the layering violation in read_body_to_string. The previous version called the higher-level archive_read_data and archive_read_data_skip functions, which screwed up state management of those functions. This bit of mis-design has existed for a long time, but became a serious issue with the recent changes to the archive_read_data APIs, which added more internal state to the high-level archive_read_data function. Most common symptom was a failure to correctly read 'L' entries (long filename) from GNU-style archives, causing the message ": Can't open: No such file or directory" with an empty filename. Pointed out by: Numerous port build failures Thanks to: Kris Kennaway
2004-06-04 23:24:21 +00:00
off_t size, padded_size;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
const struct archive_entry_header_ustar *header;
Correct the layering violation in read_body_to_string. The previous version called the higher-level archive_read_data and archive_read_data_skip functions, which screwed up state management of those functions. This bit of mis-design has existed for a long time, but became a serious issue with the recent changes to the archive_read_data APIs, which added more internal state to the high-level archive_read_data function. Most common symptom was a failure to correctly read 'L' entries (long filename) from GNU-style archives, causing the message ": Can't open: No such file or directory" with an empty filename. Pointed out by: Numerous port build failures Thanks to: Kris Kennaway
2004-06-04 23:24:21 +00:00
const void *src;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Correct the layering violation in read_body_to_string. The previous version called the higher-level archive_read_data and archive_read_data_skip functions, which screwed up state management of those functions. This bit of mis-design has existed for a long time, but became a serious issue with the recent changes to the archive_read_data APIs, which added more internal state to the high-level archive_read_data function. Most common symptom was a failure to correctly read 'L' entries (long filename) from GNU-style archives, causing the message ": Can't open: No such file or directory" with an empty filename. Pointed out by: Numerous port build failures Thanks to: Kris Kennaway
2004-06-04 23:24:21 +00:00
(void)tar; /* UNUSED */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_ustar *)h;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
size = tar_atol(header->size, sizeof(header->size));
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if ((size > 1048576) || (size < 0)) {
archive_set_error(&a->archive, EINVAL,
"Special header too large");
return (ARCHIVE_FATAL);
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
archive_string_ensure() used to call exit(3) if it couldn't allocate more memory for a string. Change this so it returns NULL in that case, and update all of its callers to handle the error. Some of those callers can now return errors back to the client instead of calling exit(3). Approved by: re (bmah)
2007-07-15 19:13:59 +00:00
/* Fail if we can't make our buffer big enough. */
if (archive_string_ensure(as, size+1) == NULL) {
archive_set_error(&a->archive, ENOMEM,
"No memory");
return (ARCHIVE_FATAL);
}
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
/* Read the body into the string. */
Correct the layering violation in read_body_to_string. The previous version called the higher-level archive_read_data and archive_read_data_skip functions, which screwed up state management of those functions. This bit of mis-design has existed for a long time, but became a serious issue with the recent changes to the archive_read_data APIs, which added more internal state to the high-level archive_read_data function. Most common symptom was a failure to correctly read 'L' entries (long filename) from GNU-style archives, causing the message ": Can't open: No such file or directory" with an empty filename. Pointed out by: Numerous port build failures Thanks to: Kris Kennaway
2004-06-04 23:24:21 +00:00
padded_size = (size + 511) & ~ 511;
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
src = __archive_read_ahead(a, padded_size, NULL);
if (src == NULL)
return (ARCHIVE_FATAL);
memcpy(as->s, src, size);
__archive_read_consume(a, padded_size);
as->s[size] = '\0';
Correct the layering violation in read_body_to_string. The previous version called the higher-level archive_read_data and archive_read_data_skip functions, which screwed up state management of those functions. This bit of mis-design has existed for a long time, but became a serious issue with the recent changes to the archive_read_data APIs, which added more internal state to the high-level archive_read_data function. Most common symptom was a failure to correctly read 'L' entries (long filename) from GNU-style archives, causing the message ": Can't open: No such file or directory" with an empty filename. Pointed out by: Numerous port build failures Thanks to: Kris Kennaway
2004-06-04 23:24:21 +00:00
return (ARCHIVE_OK);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*
* Parse out common header elements.
*
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
* This would be the same as header_old_tar, except that the
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
* filename is handled slightly differently for old and POSIX
* entries (POSIX entries support a 'prefix'). This factoring
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
* allows header_old_tar and header_ustar
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
* to handle filenames differently, while still putting most of the
* common parsing into one place.
*/
static int
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_common(struct archive_read *a, struct tar *tar,
struct archive_entry *entry, const void *h)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
const struct archive_entry_header_ustar *header;
char tartype;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
(void)a; /* UNUSED */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_ustar *)h;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
if (header->linkname[0])
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_strncpy(&(tar->entry_linkpath), header->linkname,
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
sizeof(header->linkname));
else
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_string_empty(&(tar->entry_linkpath));
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Parse out the numeric fields (all are octal) */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_mode(entry, tar_atol(header->mode, sizeof(header->mode)));
archive_entry_set_uid(entry, tar_atol(header->uid, sizeof(header->uid)));
archive_entry_set_gid(entry, tar_atol(header->gid, sizeof(header->gid)));
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = tar_atol(header->size, sizeof(header->size));
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
tar->realsize = tar->entry_bytes_remaining;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
archive_entry_set_size(entry, tar->entry_bytes_remaining);
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_mtime(entry, tar_atol(header->mtime, sizeof(header->mtime)), 0);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Handle the tar type flag appropriately. */
tartype = header->typeflag[0];
switch (tartype) {
case '1': /* Hard link */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_hardlink(entry, tar->entry_linkpath.s);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*
* The following may seem odd, but: Technically, tar
* does not store the file type for a "hard link"
* entry, only the fact that it is a hard link. So, I
* leave the type zero normally. But, pax interchange
* format allows hard links to have data, which
* implies that the underlying entry is a regular
* file.
*/
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
if (archive_entry_size(entry) > 0)
archive_entry_set_filetype(entry, AE_IFREG);
Refine the heuristic used to determine whether or not to obey the size field for a hardlink entry. Specifically, ensure that we do obey the size field for archives that we know are pax interchange format archives, as required by POSIX. Also, clarify the comment explaining why this is necessary and explain the (very unusual) conditions under which it might fail.
2004-05-19 06:35:47 +00:00
Be smarter about hardlink sizes: some tar programs write a non-zero size but no body, some write a non-zero size and include a body. To distinguish these cases, look for a valid tar header immediately following a hardlink header with non-zero size.
2004-05-18 18:16:30 +00:00
/*
Refine the heuristic used to determine whether or not to obey the size field for a hardlink entry. Specifically, ensure that we do obey the size field for archives that we know are pax interchange format archives, as required by POSIX. Also, clarify the comment explaining why this is necessary and explain the (very unusual) conditions under which it might fail.
2004-05-19 06:35:47 +00:00
* A tricky point: Traditionally, tar readers have
Be smarter about hardlink sizes: some tar programs write a non-zero size but no body, some write a non-zero size and include a body. To distinguish these cases, look for a valid tar header immediately following a hardlink header with non-zero size.
2004-05-18 18:16:30 +00:00
* ignored the size field when reading hardlink
Refine the heuristic used to determine whether or not to obey the size field for a hardlink entry. Specifically, ensure that we do obey the size field for archives that we know are pax interchange format archives, as required by POSIX. Also, clarify the comment explaining why this is necessary and explain the (very unusual) conditions under which it might fail.
2004-05-19 06:35:47 +00:00
* entries, and some writers put non-zero sizes even
Tighten up the heuristic that decides whether or not we should obey or ignore the size field on a hardlink entry. In particular, if we're reading a non-POSIX archive, we should always ignore the size field. This should fix both the audio/xmcd port and the math/unixstat port. Thanks to: Pav Lucistnik for pointing these two ports out to me. MFC after: 7 days
2008-01-31 07:41:45 +00:00
* though the body is empty. POSIX blessed this
* convention in the 1988 standard, but broke with
* this tradition in 2001 by permitting hardlink
* entries to store valid bodies in pax interchange
* format, but not in ustar format. Since there is no
* hard and fast way to distinguish pax interchange
* from earlier archives (the 'x' and 'g' entries are
* optional, after all), we need a heuristic.
Be smarter about hardlink sizes: some tar programs write a non-zero size but no body, some write a non-zero size and include a body. To distinguish these cases, look for a valid tar header immediately following a hardlink header with non-zero size.
2004-05-18 18:16:30 +00:00
*/
Tighten up the heuristic that decides whether or not we should obey or ignore the size field on a hardlink entry. In particular, if we're reading a non-POSIX archive, we should always ignore the size field. This should fix both the audio/xmcd port and the math/unixstat port. Thanks to: Pav Lucistnik for pointing these two ports out to me. MFC after: 7 days
2008-01-31 07:41:45 +00:00
if (archive_entry_size(entry) == 0) {
/* If the size is already zero, we're done. */
} else if (a->archive.archive_format
== ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE) {
/* Definitely pax extended; must obey hardlink size. */
} else if (a->archive.archive_format == ARCHIVE_FORMAT_TAR
|| a->archive.archive_format == ARCHIVE_FORMAT_TAR_GNUTAR)
{
/* Old-style or GNU tar: we must ignore the size. */
archive_entry_set_size(entry, 0);
tar->entry_bytes_remaining = 0;
} else if (archive_read_format_tar_bid(a) > 50) {
/*
* We don't know if it's pax: If the bid
* function sees a valid ustar header
* immediately following, then let's ignore
* the hardlink size.
*/
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_size(entry, 0);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = 0;
}
Tighten up the heuristic that decides whether or not we should obey or ignore the size field on a hardlink entry. In particular, if we're reading a non-POSIX archive, we should always ignore the size field. This should fix both the audio/xmcd port and the math/unixstat port. Thanks to: Pav Lucistnik for pointing these two ports out to me. MFC after: 7 days
2008-01-31 07:41:45 +00:00
/*
* TODO: There are still two cases I'd like to handle:
* = a ustar non-pax archive with a hardlink entry at
* end-of-archive. (Look for block of nulls following?)
* = a pax archive that has not seen any pax headers
* and has an entry which is a hardlink entry storing
* a body containing an uncompressed tar archive.
* The first is worth addressing; I don't see any reliable
* way to deal with the second possibility.
*/
Restore the 'break' that was inadvertently removed in 1.57 of this file. Without this, hardlinks get returned as symlinks. Approved by: re (Ken Smith) MFC after: 2 days
2007-07-14 05:53:51 +00:00
break;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
case '2': /* Symlink */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFLNK);
archive_entry_set_size(entry, 0);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = 0;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_symlink(entry, tar->entry_linkpath.s);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case '3': /* Character device */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFCHR);
archive_entry_set_size(entry, 0);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = 0;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case '4': /* Block device */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFBLK);
archive_entry_set_size(entry, 0);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = 0;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case '5': /* Dir */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFDIR);
archive_entry_set_size(entry, 0);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = 0;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case '6': /* FIFO device */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFIFO);
archive_entry_set_size(entry, 0);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->entry_bytes_remaining = 0;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
case 'D': /* GNU incremental directory type */
/*
* No special handling is actually required here.
* It might be nice someday to preprocess the file list and
* provide it to the client, though.
*/
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFDIR);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
break;
case 'M': /* GNU "Multi-volume" (remainder of file from last archive)*/
/*
* As far as I can tell, this is just like a regular file
* entry, except that the contents should be _appended_ to
* the indicated file at the indicated offset. This may
* require some API work to fully support.
*/
break;
case 'N': /* Old GNU "long filename" entry. */
/* The body of this entry is a script for renaming
* previously-extracted entries. Ugh. It will never
* be supported by libarchive. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFREG);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
break;
case 'S': /* GNU sparse files */
Allow tar format to read and accept an empty (or non-existent) file. In particular, this allows bsdtar to append (-r) to an empty file. Thanks to: Ryan Sommers While I'm here, straighten out a misleading comment about GNU-compatible sparse file handling.
2004-10-27 05:15:23 +00:00
/*
* Sparse files are really just regular files with
* sparse information in the extended area.
*/
Make Lint happier.
2007-04-12 04:42:57 +00:00
/* FALLTHROUGH */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
default: /* Regular file and non-standard types */
/*
* Per POSIX: non-recognized types should always be
* treated as regular files.
*/
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_filetype(entry, AE_IFREG);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
}
return (0);
}
/*
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
* Parse out header elements for "old-style" tar archives.
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
*/
static int
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_old_tar(struct archive_read *a, struct tar *tar,
struct archive_entry *entry, const void *h)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
const struct archive_entry_header_ustar *header;
/* Copy filename over (to ensure null termination). */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_ustar *)h;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_strncpy(&(tar->entry_pathname), header->name, sizeof(header->name));
archive_entry_copy_pathname(entry, tar->entry_pathname.s);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Grab rest of common fields */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_common(a, tar, entry, h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
tar->entry_padding = 0x1ff & (-tar->entry_bytes_remaining);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (0);
}
/*
* Parse a file header for a pax extended archive entry.
*/
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
header_pax_global(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry, const void *h)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
int err;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
err = read_body_to_string(a, tar, &(tar->pax_global), h);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if (err != ARCHIVE_OK)
return (err);
err = tar_read_header(a, tar, entry);
return (err);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
static int
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
header_pax_extensions(struct archive_read *a, struct tar *tar,
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
struct archive_entry *entry, const void *h)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
int err, err2;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
err = read_body_to_string(a, tar, &(tar->pax_header), h);
if (err != ARCHIVE_OK)
return (err);
Bug: Standard C still requires declarations to precede statements. <sigh> Portability: Eliminate an accidental __unused, accomodate systems with non-POSIX strerror_r
2004-03-20 22:35:33 +00:00
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/* Parse the next header. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err = tar_read_header(a, tar, entry);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if ((err != ARCHIVE_OK) && (err != ARCHIVE_WARN))
return (err);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*
* TODO: Parse global/default options into 'entry' struct here
* before handling file-specific options.
*
* This design (parse standard header, then overwrite with pax
* extended attribute data) usually works well, but isn't ideal;
* it would be better to parse the pax extended attributes first
* and then skip any fields in the standard header that were
* defined in the pax header.
*/
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
err2 = pax_header(a, tar, entry, tar->pax_header.s);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
err = err_combine(err, err2);
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
tar->entry_padding = 0x1ff & (-tar->entry_bytes_remaining);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (err);
}
/*
* Parse a file header for a Posix "ustar" archive entry. This also
* handles "pax" or "extended ustar" entries.
*/
static int
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_ustar(struct archive_read *a, struct tar *tar,
struct archive_entry *entry, const void *h)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
const struct archive_entry_header_ustar *header;
When combining ustar prefix and name fields, check before adding a '/' character, as some tar implementations incorrectly include a '/' with the prefix. Thanks to: Divacky Roman for the UnixWare 7 tarfile that demonstrated this issue.
2004-05-19 17:09:24 +00:00
struct archive_string *as;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_ustar *)h;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Copy name into an internal buffer to ensure null-termination. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
as = &(tar->entry_pathname);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
if (header->prefix[0]) {
When combining ustar prefix and name fields, check before adding a '/' character, as some tar implementations incorrectly include a '/' with the prefix. Thanks to: Divacky Roman for the UnixWare 7 tarfile that demonstrated this issue.
2004-05-19 17:09:24 +00:00
archive_strncpy(as, header->prefix, sizeof(header->prefix));
if (as->s[archive_strlen(as) - 1] != '/')
archive_strappend_char(as, '/');
archive_strncat(as, header->name, sizeof(header->name));
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
} else
When combining ustar prefix and name fields, check before adding a '/' character, as some tar implementations incorrectly include a '/' with the prefix. Thanks to: Divacky Roman for the UnixWare 7 tarfile that demonstrated this issue.
2004-05-19 17:09:24 +00:00
archive_strncpy(as, header->name, sizeof(header->name));
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_pathname(entry, as->s);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Handle rest of common fields. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_common(a, tar, entry, h);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Handle POSIX ustar fields. */
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
archive_strncpy(&(tar->entry_uname), header->uname,
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
sizeof(header->uname));
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_uname(entry, tar->entry_uname.s);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
archive_strncpy(&(tar->entry_gname), header->gname,
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
sizeof(header->gname));
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_gname(entry, tar->entry_gname.s);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Parse out device numbers only for char and block specials. */
if (header->typeflag[0] == '3' || header->typeflag[0] == '4') {
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_rdevmajor(entry,
tar_atol(header->rdevmajor, sizeof(header->rdevmajor)));
archive_entry_set_rdevminor(entry,
Clean up some consistent confusion between "dev" and "rdev." Mostly, these were being used correctly even though a lot of variables and function names were mis-named. In the process, I found and fixed a couple of latent bugs and added a guard against adding an archive to itself.
2004-06-27 18:38:13 +00:00
tar_atol(header->rdevminor, sizeof(header->rdevminor)));
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
Refactor read_data: * New read_data_block is both sparse-file aware and uses zero-copy semantics * Push read_data_block down into specific formats (opens door to various encoded entry bodies, such as zip or gtar -S) * Reimplement read_data, read_data_skip, read_data_into_fd in terms of new read_data_block. * Update documentation It's unfortunate that I couldn't just call the new interface archive_read_data, but didn't want to upset the API that much.
2004-06-02 08:14:43 +00:00
tar->entry_padding = 0x1ff & (-tar->entry_bytes_remaining);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (0);
}
/*
* Parse the pax extended attributes record.
*
* Returns non-zero if there's an error in the data.
*/
static int
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
pax_header(struct archive_read *a, struct tar *tar,
struct archive_entry *entry, char *attr)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
size_t attr_length, l, line_length;
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
char *p;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
char *key, *value;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
int err, err2;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
attr_length = strlen(attr);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
tar->pax_hdrcharset_binary = 0;
archive_string_empty(&(tar->entry_gname));
archive_string_empty(&(tar->entry_linkpath));
archive_string_empty(&(tar->entry_pathname));
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
archive_string_empty(&(tar->entry_pathname_override));
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_string_empty(&(tar->entry_uname));
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
err = ARCHIVE_OK;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
while (attr_length > 0) {
/* Parse decimal length field at start of line. */
line_length = 0;
l = attr_length;
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
p = attr; /* Record start of line. */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
while (l>0) {
if (*p == ' ') {
p++;
l--;
break;
}
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
if (*p < '0' || *p > '9') {
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Ignoring malformed pax extended attributes");
return (ARCHIVE_WARN);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
line_length *= 10;
line_length += *p - '0';
When rejecting rediculously large pax attributes (such as pathnames over 1MB), issue a warning instead of forcing an internal assertion failure.
2005-03-13 02:35:52 +00:00
if (line_length > 999999) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
When rejecting rediculously large pax attributes (such as pathnames over 1MB), issue a warning instead of forcing an internal assertion failure.
2005-03-13 02:35:52 +00:00
"Rejecting pax extended attribute > 1MB");
return (ARCHIVE_WARN);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
p++;
l--;
}
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
/*
* Parsed length must be no bigger than available data,
* at least 1, and the last character of the line must
* be '\n'.
*/
if (line_length > attr_length
|| line_length < 1
|| attr[line_length - 1] != '\n')
{
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
"Ignoring malformed pax extended attribute");
return (ARCHIVE_WARN);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
/* Null-terminate the line. */
attr[line_length - 1] = '\0';
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
/* Find end of key and null terminate it. */
key = p;
if (key[0] == '=')
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
return (-1);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
while (*p && *p != '=')
++p;
if (*p == '\0') {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, ARCHIVE_ERRNO_MISC,
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
"Invalid pax extended attributes");
return (ARCHIVE_WARN);
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
*p = '\0';
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/* Identify null-terminated 'value' portion. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
value = p + 1;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/* Identify this attribute and set it in the entry. */
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
err2 = pax_attribute(tar, entry, key, value);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
err = err_combine(err, err2);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Skip to next line */
attr += line_length;
attr_length -= line_length;
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (archive_strlen(&(tar->entry_gname)) > 0) {
value = tar->entry_gname.s;
if (tar->pax_hdrcharset_binary)
archive_entry_copy_gname(entry, value);
else {
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
if (!archive_entry_update_gname_utf8(entry, value)) {
err = ARCHIVE_WARN;
archive_set_error(&a->archive,
ARCHIVE_ERRNO_FILE_FORMAT,
"Gname in pax header can't "
"be converted to current locale.");
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
}
}
if (archive_strlen(&(tar->entry_linkpath)) > 0) {
value = tar->entry_linkpath.s;
if (tar->pax_hdrcharset_binary)
archive_entry_copy_link(entry, value);
else {
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
if (!archive_entry_update_link_utf8(entry, value)) {
err = ARCHIVE_WARN;
archive_set_error(&a->archive,
ARCHIVE_ERRNO_FILE_FORMAT,
"Linkname in pax header can't "
"be converted to current locale.");
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
}
}
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
/*
* Some extensions (such as the GNU sparse file extensions)
* deliberately store a synthetic name under the regular 'path'
* attribute and the real file name under a different attribute.
* Since we're supposed to not care about the order, we
* have no choice but to store all of the various filenames
* we find and figure it all out afterwards. This is the
* figuring out part.
*/
value = NULL;
if (archive_strlen(&(tar->entry_pathname_override)) > 0)
value = tar->entry_pathname_override.s;
else if (archive_strlen(&(tar->entry_pathname)) > 0)
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
value = tar->entry_pathname.s;
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
if (value != NULL) {
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (tar->pax_hdrcharset_binary)
archive_entry_copy_pathname(entry, value);
else {
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
if (!archive_entry_update_pathname_utf8(entry, value)) {
err = ARCHIVE_WARN;
archive_set_error(&a->archive,
ARCHIVE_ERRNO_FILE_FORMAT,
"Pathname in pax header can't be "
"converted to current locale.");
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
}
}
if (archive_strlen(&(tar->entry_uname)) > 0) {
value = tar->entry_uname.s;
if (tar->pax_hdrcharset_binary)
archive_entry_copy_uname(entry, value);
else {
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
if (!archive_entry_update_uname_utf8(entry, value)) {
err = ARCHIVE_WARN;
archive_set_error(&a->archive,
ARCHIVE_ERRNO_FILE_FORMAT,
"Uname in pax header can't "
"be converted to current locale.");
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
}
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
return (err);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
static int
pax_attribute_xattr(struct archive_entry *entry,
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
char *name, char *value)
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
{
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
char *name_decoded;
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
void *value_decoded;
size_t value_len;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strlen(name) < 18 || (strncmp(name, "LIBARCHIVE.xattr.", 17)) != 0)
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
return 3;
name += 17;
/* URL-decode name */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
name_decoded = url_decode(name);
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
if (name_decoded == NULL)
return 2;
/* Base-64 decode value */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
value_decoded = base64_decode(value, strlen(value), &value_len);
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
if (value_decoded == NULL) {
free(name_decoded);
return 1;
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
archive_entry_xattr_add_entry(entry, name_decoded,
value_decoded, value_len);
free(name_decoded);
free(value_decoded);
return 0;
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*
* Parse a single key=value attribute. key/value pointers are
* assumed to point into reasonably long-lived storage.
*
* Note that POSIX reserves all-lowercase keywords. Vendor-specific
* extensions should always have keywords of the form "VENDOR.attribute"
* In particular, it's quite feasible to support many different
* vendor extensions here. I'm using "LIBARCHIVE" for extensions
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
* unique to this library.
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
*
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
* Investigate other vendor-specific extensions and see if
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
* any of them look useful.
*/
static int
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
pax_attribute(struct tar *tar, struct archive_entry *entry,
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
char *key, char *value)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
int64_t s;
long n;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
wchar_t *wp;
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
switch (key[0]) {
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
case 'G':
/* GNU "0.0" sparse pax format. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.numblocks") == 0) {
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->sparse_offset = -1;
tar->sparse_numbytes = -1;
tar->sparse_gnu_major = 0;
tar->sparse_gnu_minor = 0;
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.offset") == 0) {
tar->sparse_offset = tar_atol10(value, strlen(value));
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (tar->sparse_numbytes != -1) {
gnu_add_sparse_entry(tar,
tar->sparse_offset, tar->sparse_numbytes);
tar->sparse_offset = -1;
tar->sparse_numbytes = -1;
}
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.numbytes") == 0) {
tar->sparse_numbytes = tar_atol10(value, strlen(value));
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (tar->sparse_numbytes != -1) {
gnu_add_sparse_entry(tar,
tar->sparse_offset, tar->sparse_numbytes);
tar->sparse_offset = -1;
tar->sparse_numbytes = -1;
}
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.size") == 0) {
tar->realsize = tar_atol10(value, strlen(value));
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
archive_entry_set_size(entry, tar->realsize);
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/* GNU "0.1" sparse pax format. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.map") == 0) {
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->sparse_gnu_major = 0;
tar->sparse_gnu_minor = 1;
if (gnu_sparse_01_parse(tar, value) != ARCHIVE_OK)
return (ARCHIVE_WARN);
}
/* GNU "1.0" sparse pax format */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.major") == 0) {
tar->sparse_gnu_major = tar_atol10(value, strlen(value));
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->sparse_gnu_pending = 1;
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.minor") == 0) {
tar->sparse_gnu_minor = tar_atol10(value, strlen(value));
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
tar->sparse_gnu_pending = 1;
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "GNU.sparse.name") == 0) {
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
/*
* The real filename; when storing sparse
* files, GNU tar puts a synthesized name into
* the regular 'path' attribute in an attempt
* to limit confusion. ;-)
*/
archive_strcpy(&(tar->entry_pathname_override), value);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
}
if (strcmp(key, "GNU.sparse.realsize") == 0) {
tar->realsize = tar_atol10(value, strlen(value));
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
archive_entry_set_size(entry, tar->realsize);
Fix reading of files that use pax 'size' attribute to store size. In particular, bsdtar uses the pax 'size' attribute for any file over 8G. MFC after: 3 days
2007-10-24 04:01:31 +00:00
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
break;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
case 'L':
/* Our extensions */
/* TODO: Handle arbitrary extended attributes... */
/*
if (strcmp(key, "LIBARCHIVE.xxxxxxx")==0)
archive_entry_set_xxxxxx(entry, value);
*/
MfP4: Store/read birthtime data in pax format. Submitted by: Pedro Giffuni MFC after: 30 days
2008-09-30 03:57:07 +00:00
if (strcmp(key, "LIBARCHIVE.creationtime")==0) {
pax_time(value, &s, &n);
archive_entry_set_birthtime(entry, s, n);
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strncmp(key, "LIBARCHIVE.xattr.", 17)==0)
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
pax_attribute_xattr(entry, key, value);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'S':
/* We support some keys used by the "star" archiver */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "SCHILY.acl.access")==0) {
wp = utf8_decode(tar, value, strlen(value));
/* TODO: if (wp == NULL) */
__archive_entry_acl_parse_w(entry, wp,
Overhauled ACL support. This makes us compatible with 'star' ACL handling, though there's still a bit more work needed in this area. Added 'write_open_fd' and 'read_open_fd' to simplify, e.g., tar's u and r modes. Eliminated old 'write_open_file_position' as a bad idea. (It required closing/reopening files to do updates, which led to unpleasant implications.) Various other minor fixes, API tweaks, etc.
2004-04-05 21:12:29 +00:00
ARCHIVE_ENTRY_ACL_TYPE_ACCESS);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
} else if (strcmp(key, "SCHILY.acl.default")==0) {
wp = utf8_decode(tar, value, strlen(value));
/* TODO: if (wp == NULL) */
__archive_entry_acl_parse_w(entry, wp,
Overhauled ACL support. This makes us compatible with 'star' ACL handling, though there's still a bit more work needed in this area. Added 'write_open_fd' and 'read_open_fd' to simplify, e.g., tar's u and r modes. Eliminated old 'write_open_file_position' as a bad idea. (It required closing/reopening files to do updates, which led to unpleasant implications.) Various other minor fixes, API tweaks, etc.
2004-04-05 21:12:29 +00:00
ARCHIVE_ENTRY_ACL_TYPE_DEFAULT);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
} else if (strcmp(key, "SCHILY.devmajor")==0) {
archive_entry_set_rdevmajor(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "SCHILY.devminor")==0) {
archive_entry_set_rdevminor(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "SCHILY.fflags")==0) {
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
archive_entry_copy_fflags_text(entry, value);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
} else if (strcmp(key, "SCHILY.dev")==0) {
archive_entry_set_dev(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "SCHILY.ino")==0) {
archive_entry_set_ino(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "SCHILY.nlink")==0) {
archive_entry_set_nlink(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "SCHILY.realsize")==0) {
tar->realsize = tar_atol10(value, strlen(value));
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
archive_entry_set_size(entry, tar->realsize);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'a':
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "atime")==0) {
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
pax_time(value, &s, &n);
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_atime(entry, s, n);
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'c':
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "ctime")==0) {
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
pax_time(value, &s, &n);
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_ctime(entry, s, n);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
} else if (strcmp(key, "charset")==0) {
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* TODO: Publish charset information in entry. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
} else if (strcmp(key, "comment")==0) {
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* TODO: Publish comment in entry. */
}
break;
case 'g':
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "gid")==0) {
archive_entry_set_gid(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "gname")==0) {
archive_strcpy(&(tar->entry_gname), value);
}
break;
case 'h':
if (strcmp(key, "hdrcharset") == 0) {
if (strcmp(value, "BINARY") == 0)
tar->pax_hdrcharset_binary = 1;
else if (strcmp(value, "ISO-IR 10646 2000 UTF-8") == 0)
tar->pax_hdrcharset_binary = 0;
else {
/* TODO: Warn about unsupported hdrcharset */
}
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'l':
/* pax interchange doesn't distinguish hardlink vs. symlink. */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "linkpath")==0) {
archive_strcpy(&(tar->entry_linkpath), value);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
break;
case 'm':
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "mtime")==0) {
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
pax_time(value, &s, &n);
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_mtime(entry, s, n);
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'p':
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "path")==0) {
archive_strcpy(&(tar->entry_pathname), value);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'r':
/* POSIX has reserved 'realtime.*' */
break;
case 's':
/* POSIX has reserved 'security.*' */
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
/* Someday: if (strcmp(key, "security.acl")==0) { ... } */
if (strcmp(key, "size")==0) {
Fix reading of files that use pax 'size' attribute to store size. In particular, bsdtar uses the pax 'size' attribute for any file over 8G. MFC after: 3 days
2007-10-24 04:01:31 +00:00
/* "size" is the size of the data in the entry. */
tar->entry_bytes_remaining
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
= tar_atol10(value, strlen(value));
Fix reading of files that use pax 'size' attribute to store size. In particular, bsdtar uses the pax 'size' attribute for any file over 8G. MFC after: 3 days
2007-10-24 04:01:31 +00:00
/*
* But, "size" is not necessarily the size of
* the file on disk; if this is a sparse file,
* the disk size may have already been set from
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
* GNU.sparse.realsize or GNU.sparse.size or
* an old GNU header field or SCHILY.realsize
* or ....
Fix reading of files that use pax 'size' attribute to store size. In particular, bsdtar uses the pax 'size' attribute for any file over 8G. MFC after: 3 days
2007-10-24 04:01:31 +00:00
*/
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
if (tar->realsize < 0) {
Fix reading of files that use pax 'size' attribute to store size. In particular, bsdtar uses the pax 'size' attribute for any file over 8G. MFC after: 3 days
2007-10-24 04:01:31 +00:00
archive_entry_set_size(entry,
tar->entry_bytes_remaining);
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
tar->realsize
Fix reading of files that use pax 'size' attribute to store size. In particular, bsdtar uses the pax 'size' attribute for any file over 8G. MFC after: 3 days
2007-10-24 04:01:31 +00:00
= tar->entry_bytes_remaining;
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
case 'u':
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (strcmp(key, "uid")==0) {
archive_entry_set_uid(entry,
tar_atol10(value, strlen(value)));
} else if (strcmp(key, "uname")==0) {
archive_strcpy(&(tar->entry_uname), value);
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
}
return (0);
}
/*
* parse a decimal time value, which may include a fractional portion
*/
static void
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
pax_time(const char *p, int64_t *ps, long *pn)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
char digit;
int64_t s;
unsigned long l;
int sign;
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
int64_t limit, last_digit_limit;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Consolidate numeric limit macros in one place; include them only on platforms that need them. FreeBSD doesn't.
2007-04-15 00:53:38 +00:00
limit = INT64_MAX / 10;
last_digit_limit = INT64_MAX % 10;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
s = 0;
sign = 1;
if (*p == '-') {
sign = -1;
p++;
}
while (*p >= '0' && *p <= '9') {
digit = *p - '0';
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
if (s > limit ||
(s == limit && digit > last_digit_limit)) {
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
s = INT64_MAX;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
}
s = (s * 10) + digit;
++p;
}
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
*ps = s * sign;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/* Calculate nanoseconds. */
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
*pn = 0;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
if (*p != '.')
return;
l = 100000000UL;
do {
++p;
if (*p >= '0' && *p <= '9')
More portability improvements, thanks to Juergen Lock. High-resolution mtime/ctime/atime is not POSIX-standard, so hide set/get of high-resolution time fields behind easily-mutable macros. That makes it easier to change how those fields are accessed.
2004-04-21 05:13:42 +00:00
*pn += (*p - '0') * l;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
else
break;
} while (l /= 10);
}
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
/*
* Parse GNU tar header
*/
static int
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_gnutar(struct archive_read *a, struct tar *tar,
struct archive_entry *entry, const void *h)
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
{
const struct archive_entry_header_gnutar *header;
(void)a;
/*
* GNU header is like POSIX ustar, except 'prefix' is
* replaced with some other fields. This also means the
* filename is stored as in old-style archives.
*/
/* Grab fields common to all tar variants. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
header_common(a, tar, entry, h);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
/* Copy filename over (to ensure null termination). */
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
header = (const struct archive_entry_header_gnutar *)h;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_strncpy(&(tar->entry_pathname), header->name,
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
sizeof(header->name));
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_pathname(entry, tar->entry_pathname.s);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
/* Fields common to ustar and GNU */
/* XXX Can the following be factored out since it's common
* to ustar and gnu tar? Is it okay to move it down into
* header_common, perhaps? */
archive_strncpy(&(tar->entry_uname),
header->uname, sizeof(header->uname));
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_uname(entry, tar->entry_uname.s);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
archive_strncpy(&(tar->entry_gname),
header->gname, sizeof(header->gname));
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
archive_entry_copy_gname(entry, tar->entry_gname.s);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
/* Parse out device numbers only for char and block specials */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
if (header->typeflag[0] == '3' || header->typeflag[0] == '4') {
archive_entry_set_rdevmajor(entry,
tar_atol(header->rdevmajor, sizeof(header->rdevmajor)));
archive_entry_set_rdevminor(entry,
Clean up some consistent confusion between "dev" and "rdev." Mostly, these were being used correctly even though a lot of variables and function names were mis-named. In the process, I found and fixed a couple of latent bugs and added a guard against adding an archive to itself.
2004-06-27 18:38:13 +00:00
tar_atol(header->rdevminor, sizeof(header->rdevminor)));
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
} else
archive_entry_set_rdev(entry, 0);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
tar->entry_padding = 0x1ff & (-tar->entry_bytes_remaining);
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
/* Grab GNU-specific fields. */
libarchive 2.2.3 * "compression_program" support uses an external program * Portability: no longer uses "struct stat" as a primary data interchange structure internally * Part of the above: refactor archive_entry to separate out copy_stat() and stat() functions * More complete tests for archive_entry * Finish archive_entry_clone() * Isolate major()/minor()/makedev() in archive_entry; remove these from everywhere else. * Bug fix: properly handle decompression look-ahead at end-of-data * Bug fixes to 'ar' support * Fix memory leak in ZIP reader * Portability: better timegm() emulation in iso9660 reader * New write_disk flags to suppress auto dir creation and not overwrite newer files (for future cpio front-end) * Simplify trailing-'/' fixup when writing tar and pax * Test enhancements: fix various compiler warnings, improve portability, add lots of new tests. * Documentation: document new functions, first draft of libarchive_internals.3 MFC after: 14 days Thanks to: Joerg Sonnenberger (compression_program) Thanks to: Kai Wang (ar) Thanks to: Colin Percival (many small fixes) Thanks to: Many others who sent me various patches and problem reports.
2007-05-29 01:00:21 +00:00
archive_entry_set_atime(entry,
tar_atol(header->atime, sizeof(header->atime)), 0);
archive_entry_set_ctime(entry,
tar_atol(header->ctime, sizeof(header->ctime)), 0);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
if (header->realsize[0] != 0) {
Update libarchive to 2.4.10. This includes a number of improvements that I've been working on but put off committing until after the RELENG_7 branch, including: * New manpages: cpio.5 mtree.5 * New archive_entry_strmode() * New archive_entry_link_resolver() * New read support: mtree format * Internal API change: read format auction only runs once * Running the auction only once allowed simplifying a lot of bid logic. * Cpio robustness: search for next header after a sync error * Support device nodes on ISO9660 images * Eliminate a lot of unnecessary copies for uncompressed archives * Corrected handling of new GNU --sparse --posix formats * Correctly handle a zero-byte write to a compressed archive * Fixed memory leaks Many of these improvements were motivated by the upcoming bsdcpio front-end. There have also been extensive improvements to the libarchive_test test harness, which I'll commit separately.
2007-12-30 04:58:22 +00:00
tar->realsize
= tar_atol(header->realsize, sizeof(header->realsize));
archive_entry_set_size(entry, tar->realsize);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
}
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
if (header->sparse[0].offset[0] != 0) {
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
gnu_sparse_old_read(a, tar, header);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
} else {
if (header->isextended[0] != 0) {
/* XXX WTF? XXX */
}
}
More work on ACLs: fix error in archive_entry's ACL parsing code, try to set ACLs even if fflag restore fails, first cut at reading Solaris tar ACLs Code improvement: merge gnu tar read support into main tar reader; this eliminates a lot of duplicate code and generalizes the tar reader to handle formats with GNU-like extensions. Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype' from archive_entry (this makes archive_entry more format-agnostic) Thanks to: David Magda for providing Solaris tar test files
2004-04-12 01:16:16 +00:00
return (0);
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static void
gnu_add_sparse_entry(struct tar *tar, off_t offset, off_t remaining)
{
struct sparse_block *p;
p = (struct sparse_block *)malloc(sizeof(*p));
if (p == NULL)
__archive_errx(1, "Out of memory");
memset(p, 0, sizeof(*p));
if (tar->sparse_last != NULL)
tar->sparse_last->next = p;
else
tar->sparse_list = p;
tar->sparse_last = p;
p->offset = offset;
p->remaining = remaining;
}
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
static void
gnu_clear_sparse_list(struct tar *tar)
{
struct sparse_block *p;
while (tar->sparse_list != NULL) {
p = tar->sparse_list;
tar->sparse_list = p->next;
free(p);
}
tar->sparse_last = NULL;
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/*
* GNU tar old-format sparse data.
*
* GNU old-format sparse data is stored in a fixed-field
* format. Offset/size values are 11-byte octal fields (same
* format as 'size' field in ustart header). These are
* stored in the header, allocating subsequent header blocks
* as needed. Extending the header in this way is a pretty
* severe POSIX violation; this design has earned GNU tar a
* lot of criticism.
*/
Make some purely internal symbols static to reduce link pollution.
2005-10-12 15:38:45 +00:00
static int
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
gnu_sparse_old_read(struct archive_read *a, struct tar *tar,
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
const struct archive_entry_header_gnutar *header)
{
ssize_t bytes_read;
const void *data;
struct extended {
struct gnu_sparse sparse[21];
char isextended[1];
char padding[7];
};
const struct extended *ext;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
gnu_sparse_old_parse(tar, header->sparse, 4);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
if (header->isextended[0] == 0)
return (ARCHIVE_OK);
do {
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
data = __archive_read_ahead(a, 512, &bytes_read);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
if (bytes_read < 0)
return (ARCHIVE_FATAL);
if (bytes_read < 512) {
libarchive 2.0 * libarchive_test program exercises many of the core features * Refactored old "read_extract" into new "archive_write_disk", which uses archive_write methods to put entries onto disk. In particular, you can now use archive_write_disk to create objects on disk without having an archive available. * Pushed some security checks from bsdtar down into libarchive, where they can be better optimized. * Rearchitected the logic for creating objects on disk to reduce the number of system calls. Several common cases now use a minimum number of system calls. * Virtualized some internal interfaces to provide a clearer separation of read and write handling and make it simpler to override key methods. * New "empty" format reader. * Corrected return types (this ABI breakage required the "2.0" version bump) * Many bug fixes.
2007-03-03 07:37:37 +00:00
archive_set_error(&a->archive, ARCHIVE_ERRNO_FILE_FORMAT,
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
"Truncated tar archive "
"detected while reading sparse file data");
return (ARCHIVE_FATAL);
}
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
__archive_read_consume(a, 512);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
ext = (const struct extended *)data;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
gnu_sparse_old_parse(tar, ext->sparse, 21);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
} while (ext->isextended[0] != 0);
if (tar->sparse_list != NULL)
tar->entry_offset = tar->sparse_list->offset;
return (ARCHIVE_OK);
}
Make some purely internal symbols static to reduce link pollution.
2005-10-12 15:38:45 +00:00
static void
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
gnu_sparse_old_parse(struct tar *tar,
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
const struct gnu_sparse *sparse, int length)
{
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
while (length > 0 && sparse->offset[0] != 0) {
gnu_add_sparse_entry(tar,
tar_atol(sparse->offset, sizeof(sparse->offset)),
tar_atol(sparse->numbytes, sizeof(sparse->numbytes)));
sparse++;
length--;
}
}
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/*
* GNU tar sparse format 0.0
*
* Beginning with GNU tar 1.15, sparse files are stored using
* information in the pax extended header. The GNU tar maintainers
* have gone through a number of variations in the process of working
* out this scheme; furtunately, they're all numbered.
*
* Sparse format 0.0 uses attribute GNU.sparse.numblocks to store the
* number of blocks, and GNU.sparse.offset/GNU.sparse.numbytes to
* store offset/size for each block. The repeated instances of these
* latter fields violate the pax specification (which frowns on
* duplicate keys), so this format was quickly replaced.
*/
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/*
* GNU tar sparse format 0.1
*
* This version replaced the offset/numbytes attributes with
* a single "map" attribute that stored a list of integers. This
* format had two problems: First, the "map" attribute could be very
* long, which caused problems for some implementations. More
* importantly, the sparse data was lost when extracted by archivers
* that didn't recognize this extension.
*/
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
static int
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
gnu_sparse_01_parse(struct tar *tar, const char *p)
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
{
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
const char *e;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
off_t offset = -1, size = -1;
for (;;) {
e = p;
while (*e != '\0' && *e != ',') {
if (*e < '0' || *e > '9')
return (ARCHIVE_WARN);
e++;
}
if (offset < 0) {
offset = tar_atol10(p, e - p);
if (offset < 0)
return (ARCHIVE_WARN);
} else {
size = tar_atol10(p, e - p);
if (size < 0)
return (ARCHIVE_WARN);
gnu_add_sparse_entry(tar, offset, size);
offset = -1;
}
if (*e == '\0')
return (ARCHIVE_OK);
p = e + 1;
}
}
/*
* GNU tar sparse format 1.0
*
* The idea: The offset/size data is stored as a series of base-10
* ASCII numbers prepended to the file data, so that dearchivers that
* don't support this format will extract the block map along with the
* data and a separate post-process can restore the sparseness.
*
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
* Unfortunately, GNU tar 1.16 had a bug that added unnecessary
* padding to the body of the file when using this format. GNU tar
* 1.17 corrected this bug without bumping the version number, so
* it's not possible to support both variants. This code supports
* the later variant at the expense of not supporting the former.
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
*
* This variant also replaced GNU.sparse.size with GNU.sparse.realsize
* and introduced the GNU.sparse.major/GNU.sparse.minor attributes.
*/
/*
* Read the next line from the input, and parse it as a decimal
* integer followed by '\n'. Returns positive integer value or
* negative on error.
*/
static int64_t
gnu_sparse_10_atol(struct archive_read *a, struct tar *tar,
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
ssize_t *remaining)
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
{
int64_t l, limit, last_digit_limit;
const char *p;
ssize_t bytes_read;
int base, digit;
base = 10;
limit = INT64_MAX / base;
last_digit_limit = INT64_MAX % base;
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
/*
* Skip any lines starting with '#'; GNU tar specs
* don't require this, but they should.
*/
do {
bytes_read = readline(a, tar, &p, tar_min(*remaining, 100));
if (bytes_read <= 0)
return (ARCHIVE_FATAL);
*remaining -= bytes_read;
} while (p[0] == '#');
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
l = 0;
while (bytes_read > 0) {
if (*p == '\n')
return (l);
if (*p < '0' || *p >= '0' + base)
return (ARCHIVE_WARN);
digit = *p - '0';
if (l > limit || (l == limit && digit > last_digit_limit))
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
l = INT64_MAX; /* Truncate on overflow. */
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
else
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
l = (l * base) + digit;
p++;
bytes_read--;
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/* TODO: Error message. */
return (ARCHIVE_WARN);
}
/*
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
* Returns length (in bytes) of the sparse data description
* that was read.
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
*/
static ssize_t
gnu_sparse_10_read(struct archive_read *a, struct tar *tar)
{
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
ssize_t remaining, bytes_read;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
int entries;
off_t offset, size, to_skip;
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
/* Clear out the existing sparse list. */
gnu_clear_sparse_list(tar);
remaining = tar->entry_bytes_remaining;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/* Parse entries. */
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
entries = gnu_sparse_10_atol(a, tar, &remaining);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (entries < 0)
return (ARCHIVE_FATAL);
/* Parse the individual entries. */
while (entries-- > 0) {
/* Parse offset/size */
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
offset = gnu_sparse_10_atol(a, tar, &remaining);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (offset < 0)
return (ARCHIVE_FATAL);
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
size = gnu_sparse_10_atol(a, tar, &remaining);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (size < 0)
return (ARCHIVE_FATAL);
/* Add a new sparse entry. */
gnu_add_sparse_entry(tar, offset, size);
}
/* Skip rest of block... */
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
bytes_read = tar->entry_bytes_remaining - remaining;
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
to_skip = 0x1ff & -bytes_read;
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
if (to_skip != __archive_read_skip(a, to_skip))
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
return (ARCHIVE_FATAL);
return (bytes_read + to_skip);
Read gtar-style sparse archives. This change also pointed out one API deficiency: the archive_read_data_into_XXX functions were originally defined to return the total bytes read. This is, of course, ambiguous when dealing with non-contiguous files. Change it to just return a status value.
2004-06-27 01:15:31 +00:00
}
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
/*-
* Convert text->integer.
*
* Traditional tar formats (including POSIX) specify base-8 for
* all of the standard numeric fields. This is a significant limitation
* in practice:
* = file size is limited to 8GB
Clean up some consistent confusion between "dev" and "rdev." Mostly, these were being used correctly even though a lot of variables and function names were mis-named. In the process, I found and fixed a couple of latent bugs and added a guard against adding an archive to itself.
2004-06-27 18:38:13 +00:00
* = rdevmajor and rdevminor are limited to 21 bits
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
* = uid/gid are limited to 21 bits
*
* There are two workarounds for this:
* = pax extended headers, which use variable-length string fields
* = GNU tar and STAR both allow either base-8 or base-256 in
* most fields. The high bit is set to indicate base-256.
*
* On read, this implementation supports both extensions.
*/
static int64_t
tar_atol(const char *p, unsigned char_cnt)
{
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
/*
* Technically, GNU tar considers a field to be in base-256
* only if the first byte is 0xff or 0x80.
*/
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
if (*p & 0x80)
return (tar_atol256(p, char_cnt));
return (tar_atol8(p, char_cnt));
}
/*
* Note that this implementation does not (and should not!) obey
* locale settings; you cannot simply substitute strtol here, since
* it does obey locale.
*/
static int64_t
tar_atol8(const char *p, unsigned char_cnt)
{
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
int64_t l, limit, last_digit_limit;
int digit, sign, base;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
base = 8;
Consolidate numeric limit macros in one place; include them only on platforms that need them. FreeBSD doesn't.
2007-04-15 00:53:38 +00:00
limit = INT64_MAX / base;
last_digit_limit = INT64_MAX % base;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
while (*p == ' ' || *p == '\t')
p++;
if (*p == '-') {
sign = -1;
p++;
} else
sign = 1;
l = 0;
digit = *p - '0';
while (digit >= 0 && digit < base && char_cnt-- > 0) {
if (l>limit || (l == limit && digit > last_digit_limit)) {
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
l = INT64_MAX; /* Truncate on overflow. */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
}
l = (l * base) + digit;
digit = *++p - '0';
}
return (sign < 0) ? -l : l;
}
/*
* Note that this implementation does not (and should not!) obey
* locale settings; you cannot simply substitute strtol here, since
* it does obey locale.
*/
static int64_t
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
tar_atol10(const char *p, unsigned char_cnt)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
int64_t l, limit, last_digit_limit;
int base, digit, sign;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
base = 10;
Consolidate numeric limit macros in one place; include them only on platforms that need them. FreeBSD doesn't.
2007-04-15 00:53:38 +00:00
limit = INT64_MAX / base;
last_digit_limit = INT64_MAX % base;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
while (*p == ' ' || *p == '\t')
p++;
if (*p == '-') {
sign = -1;
p++;
} else
sign = 1;
l = 0;
digit = *p - '0';
while (digit >= 0 && digit < base && char_cnt-- > 0) {
if (l > limit || (l == limit && digit > last_digit_limit)) {
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
l = INT64_MAX; /* Truncate on overflow. */
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
}
l = (l * base) + digit;
digit = *++p - '0';
}
return (sign < 0) ? -l : l;
}
/*
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
* Parse a base-256 integer. This is just a straight signed binary
* value in big-endian order, except that the high-order bit is
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
* ignored.
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
*/
static int64_t
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
tar_atol256(const char *_p, unsigned char_cnt)
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
{
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
int64_t l, upper_limit, lower_limit;
signed/unsigned fixes (thanks to GCC4) and a few related minor style corrections.
2005-09-24 21:15:00 +00:00
const unsigned char *p = (const unsigned char *)_p;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Consolidate numeric limit macros in one place; include them only on platforms that need them. FreeBSD doesn't.
2007-04-15 00:53:38 +00:00
upper_limit = INT64_MAX / 256;
lower_limit = INT64_MIN / 256;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
/* Pad with 1 or 0 bits, depending on sign. */
if ((0x40 & *p) == 0x40)
l = (int64_t)-1;
else
l = 0;
l = (l << 6) | (0x3f & *p++);
while (--char_cnt > 0) {
if (l > upper_limit) {
Consolidate numeric limit macros in one place; include them only on platforms that need them. FreeBSD doesn't.
2007-04-15 00:53:38 +00:00
l = INT64_MAX; /* Truncate on overflow */
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
break;
} else if (l < lower_limit) {
Consolidate numeric limit macros in one place; include them only on platforms that need them. FreeBSD doesn't.
2007-04-15 00:53:38 +00:00
l = INT64_MIN;
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
break;
}
Fix the handling of signed values when parsing base-256 header values. In particular, this means we can now correctly read gtar archives that contain timestamps prior to the start of the Epoch. Also, make the code in this area more portable. ANSI C99 headers are not yet ubiquitous (for example, FreeBSD 4 still lacks them), so be prepared for systems that don't have the INT64_MAX, INT64_MIN, and UINT64_MAX macros. This version still requires int64_t and uint64_t be defined (which can be done in archive_platform.h if necessary), but doesn't require them to be exactly 64 bits.
2004-07-24 17:46:45 +00:00
l = (l << 8) | (0xff & (int64_t)*p++);
Initial import of libarchive. What it is: A library for reading and writing various streaming archive formats, especially tar and cpio. Being a library, it should be easy to incorporate into pkg_* tools, sysinstall, and any other place that needs to read or write such archives. Features: * Full automatic detection of both compression and archive format. * Extensible internal architecture to make it easy to add new formats. * Support for "pax interchange format," a new POSIX-standard tar format that eliminates essentially all of the restrictions of historic formats. * BSD license Thanks to: jkh for pushing me to start this work, gordon for encouraging me to commit it, bde for answering endless style questions, and many others for feedback and encouragement. Status: Pretty good overall, though there are still a few rough edges and the library could always use more testing. Feedback eagerly solicited.
2004-02-09 23:22:54 +00:00
}
return (l);
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/*
* Returns length of line (including trailing newline)
* or negative on error. 'start' argument is updated to
* point to first character of line. This avoids copying
* when possible.
*/
static ssize_t
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
readline(struct archive_read *a, struct tar *tar, const char **start,
ssize_t limit)
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
{
ssize_t bytes_read;
ssize_t total_size = 0;
const void *t;
const char *s;
void *p;
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
t = __archive_read_ahead(a, 1, &bytes_read);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (bytes_read <= 0)
return (ARCHIVE_FATAL);
s = t; /* Start of line? */
p = memchr(t, '\n', bytes_read);
/* If we found '\n' in the read buffer, return pointer to that. */
if (p != NULL) {
bytes_read = 1 + ((const char *)p) - s;
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
if (bytes_read > limit) {
archive_set_error(&a->archive,
ARCHIVE_ERRNO_FILE_FORMAT,
"Line too long");
return (ARCHIVE_FATAL);
}
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
__archive_read_consume(a, bytes_read);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
*start = s;
return (bytes_read);
}
/* Otherwise, we need to accumulate in a line buffer. */
for (;;) {
This commit updates libarchive to be compatible with GNU tar 1.17's implementation of --posix --sparse, at the cost of losing compatibility with GNU tar 1.16. Fortunately, the 1.17 implementation actually makes sense, so the libarchive code is now a bit more straightforward than before. Background: GNU tar 1.16 defined a new way to store sparse files in --posix archives. Unfortunately, the implementation incorrectly inserted several blocks of null padding after each such entry. As a result, non-GNU tar implementations saw the archive as truncated after any sparse entry. This was fixed in GNU tar 1.17 at the cost of losing compatibility with GNU tar 1.16 for this new format (which is not the default, so hopefully rarely used). Libarchive recently gained support for reading the GNU tar 1.16 formats; this commit updates it to read the GNU tar 1.17 variant instead. Approved by: re (ksmith for libarchive portion) Approved by: re (blanket for libarchive_test portion) MFC after: 5 days
2007-08-18 21:53:25 +00:00
if (total_size + bytes_read > limit) {
archive_set_error(&a->archive,
ARCHIVE_ERRNO_FILE_FORMAT,
"Line too long");
return (ARCHIVE_FATAL);
}
archive_string_ensure() used to call exit(3) if it couldn't allocate more memory for a string. Change this so it returns NULL in that case, and update all of its callers to handle the error. Some of those callers can now return errors back to the client instead of calling exit(3). Approved by: re (bmah)
2007-07-15 19:13:59 +00:00
if (archive_string_ensure(&tar->line, total_size + bytes_read) == NULL) {
archive_set_error(&a->archive, ENOMEM,
"Can't allocate working buffer");
return (ARCHIVE_FATAL);
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
memcpy(tar->line.s + total_size, t, bytes_read);
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
__archive_read_consume(a, bytes_read);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
total_size += bytes_read;
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
/* If we found '\n', clean up and return. */
if (p != NULL) {
*start = tar->line.s;
return (total_size);
}
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
/* Read some more. */
MfP4: Big read filter refactoring. This is an attempt to eliminate a lot of redundant code from the read ("decompression") filters by changing them to juggle arbitrary-sized blocks and consolidate reblocking code at a single point in archive_read.c. Along the way, I've changed the internal read/consume API used by the format handlers to a slightly different style originally suggested by des@. It does seem to simplify a lot of common cases. The most dramatic change is, of course, to archive_read_support_compression_none(), which has just evaporated into a no-op as the blocking code this used to hold has all been moved up a level. There's at least one more big round of refactoring yet to come before the individual filters are as straightforward as I think they should be...
2008-12-06 06:45:15 +00:00
t = __archive_read_ahead(a, 1, &bytes_read);
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (bytes_read <= 0)
return (ARCHIVE_FATAL);
s = t; /* Start of line? */
p = memchr(t, '\n', bytes_read);
Correct multiple security issues in how libarchive handles corrupt tar archives, including a potentially exploitable buffer overflow. Approved by: re (kensmith, security blanket) Reviewed by: kientzle Security: FreeBSD-SA-07:05.libarchive
2007-07-12 15:00:28 +00:00
/* If we found '\n', trim the read. */
Read support for the new GNU tar sparse formats added in gtar 1.15 and gtar 1.16.
2007-06-13 03:35:37 +00:00
if (p != NULL) {
bytes_read = 1 + ((const char *)p) - s;
}
}
}
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static wchar_t *
utf8_decode(struct tar *tar, const char *src, size_t length)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
wchar_t *dest;
ssize_t n;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
/* Ensure pax_entry buffer is big enough. */
if (tar->pax_entry_length <= length) {
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
wchar_t *old_entry;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (tar->pax_entry_length <= 0)
tar->pax_entry_length = 1024;
while (tar->pax_entry_length <= length + 1)
tar->pax_entry_length *= 2;
old_entry = tar->pax_entry;
tar->pax_entry = (wchar_t *)realloc(tar->pax_entry,
tar->pax_entry_length * sizeof(wchar_t));
if (tar->pax_entry == NULL) {
free(old_entry);
/* TODO: Handle this error. */
return (NULL);
}
}
dest = tar->pax_entry;
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
while (length > 0) {
Bug: Standard C still requires declarations to precede statements. <sigh> Portability: Eliminate an accidental __unused, accomodate systems with non-POSIX strerror_r
2004-03-20 22:35:33 +00:00
n = UTF8_mbrtowc(dest, src, length);
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
if (n < 0)
return (NULL);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
if (n == 0)
break;
dest++;
src += n;
length -= n;
}
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
*dest = L'\0';
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
return (tar->pax_entry);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
/*
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
* Copied and simplified from FreeBSD libc/locale.
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
*/
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
static ssize_t
Bug: Standard C still requires declarations to precede statements. <sigh> Portability: Eliminate an accidental __unused, accomodate systems with non-POSIX strerror_r
2004-03-20 22:35:33 +00:00
UTF8_mbrtowc(wchar_t *pwc, const char *s, size_t n)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
{
int ch, i, len, mask;
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
unsigned long wch;
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
if (s == NULL || n == 0 || pwc == NULL)
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
return (0);
/*
* Determine the number of octets that make up this character from
* the first octet, and a mask that extracts the interesting bits of
* the first octet.
*/
ch = (unsigned char)*s;
if ((ch & 0x80) == 0) {
mask = 0x7f;
len = 1;
} else if ((ch & 0xe0) == 0xc0) {
mask = 0x1f;
len = 2;
} else if ((ch & 0xf0) == 0xe0) {
mask = 0x0f;
len = 3;
} else if ((ch & 0xf8) == 0xf0) {
mask = 0x07;
len = 4;
} else {
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
/* Invalid first byte. */
return (-1);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
if (n < (size_t)len) {
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
/* Valid first byte but truncated. */
return (-2);
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
}
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
/*
* Decode the octet sequence representing the character in chunks
* of 6 bits, most significant first.
*/
wch = (unsigned char)*s++ & mask;
i = len;
while (--i != 0) {
if ((*s & 0xc0) != 0x80) {
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
/* Invalid intermediate byte; consume one byte and
* emit '?' */
*pwc = '?';
return (1);
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
}
wch <<= 6;
wch |= *s++ & 0x3f;
}
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
/* Assign the value to the output; out-of-range values
* just get truncated. */
*pwc = (wchar_t)wch;
Eliminate reliance on non-portable <err.h> by implementing a very simple errx() function. Improve behavior when bzlib/zlib are missing by detecting and issuing an error message on attempts to read gzip/bzip2 compressed archives.
2004-08-14 03:45:45 +00:00
#ifdef WCHAR_MAX
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
/*
* If platform has WCHAR_MAX, we can do something
* more sensible with out-of-range values.
*/
if (wch >= WCHAR_MAX)
*pwc = '?';
Eliminate reliance on non-portable <err.h> by implementing a very simple errx() function. Improve behavior when bzlib/zlib are missing by detecting and issuing an error message on attempts to read gzip/bzip2 compressed archives.
2004-08-14 03:45:45 +00:00
#endif
Simplify some of the wide-character handling, inspired in part by OpenBSD's not-quite-standard-compliant standard libraries. (No loss of functionality, just minor recoding to not rely on certain "standard" facilities that weren't actually needed.)
2006-05-01 01:02:19 +00:00
/* Return number of bytes input consumed: 0 for end-of-string. */
Many fixes: * Disabled shared-library building, as some API breakage is still likely. (I didn't realize it was turned on by default.) If you have an existing /usr/lib/libarchive.so.2, I recommend deleting it. * Pax interchange format now correctly stores and reads UTF8 for extended attributes. In particular, pax format can portably handle arbitrarily long pathnames containing arbitrary characters. * Library compiles cleanly at -O2, -O3, and WARNS=6 on all FreeBSD-CURRENT platforms. * Minor portability improvements inspired by Juergen Lock and Greg Lewis. (Less reliance on stdint.h, isolating of various portability-challenged constructs.) * archive_entry transparently converts multi-byte <-> wide character strings, allowing clients and format handlers to deal with either one, as appropriate. * Support for reading 'L' and 'K' entries in standard tar archives for star compatibility. * Recognize (but don't yet handle) ACL entries from Solaris tar. * Pushed format-specific data for format readers down into format-specific storage and out of library-global storage. This should make it easier to maintain individual formats without mucking with the core library management. * Documentation updates to track the above changes. * Updates to tar.5 to correct a few mistakes and add some additional information about GNU tar and Solaris tar formats. Notes: * The basic 'tar' reader is getting more general; there's not much point in keeping the 'gnutar' reader separate. Merging the two would lose a bunch of duplicate code. * The libc ACL support is looking increasingly inadequate for my needs here. I might need to assemble some fairly significant code for parsing and building ACLs. <sigh>
2004-03-19 22:37:06 +00:00
return (wch == L'\0' ? 0 : len);
}
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
/*
* base64_decode - Base64 decode
*
* This accepts most variations of base-64 encoding, including:
* * with or without line breaks
* * with or without the final group padded with '=' or '_' characters
* (The most economical Base-64 variant does not pad the last group and
* omits line breaks; RFC1341 used for MIME requires both.)
*/
static char *
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
base64_decode(const char *s, size_t len, size_t *out_len)
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
{
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
static const unsigned char digits[64] = {
'A','B','C','D','E','F','G','H','I','J','K','L','M','N',
'O','P','Q','R','S','T','U','V','W','X','Y','Z','a','b',
'c','d','e','f','g','h','i','j','k','l','m','n','o','p',
'q','r','s','t','u','v','w','x','y','z','0','1','2','3',
'4','5','6','7','8','9','+','/' };
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
static unsigned char decode_table[128];
char *out, *d;
A subtle point: "pax interchange format" mandates that all strings (including pathname, gname, uname) be stored in UTF-8. This usually doesn't cause problems on FreeBSD because the "C" locale on FreeBSD can convert any byte to Unicode/wchar_t and from there to UTF-8. In other locales (including the "C" locale on Linux which is really ASCII), you can get into trouble with pathnames that cannot be converted to UTF-8. Libarchive's pax writer truncated pathnames and other strings at the first nonconvertible character. (ouch!) Other archivers have worked around this by storing unconvertible pathnames as raw binary, a practice which has been sanctioned by the Austin group. However, libarchive's pax reader would segfault reading headers that weren't proper UTF-8. (ouch!) Since bsdtar defaults to pax format, this affects bsdtar rather heavily. To correctly support the new "hdrcharset" header that is going into SUS and to handle conversion failures in general, libarchive's pax reader and writer have been overhauled fairly extensively. They used to do most of the pax header processing using wchar_t (Unicode); they now do most of it using char so that common logic applies to either UTF-8 or "binary" strings. As a bonus, a number of extraneous conversions to/from wchar_t have been eliminated, which should speed things up just a tad. Thanks to: Bjoern Jacke for originally reporting this to me Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this Thanks to: Gunnar Ritter for getting the standard fixed MFC after: 5 days
2008-03-15 01:43:59 +00:00
const unsigned char *src = (const unsigned char *)s;
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
/* If the decode table is not yet initialized, prepare it. */
if (decode_table[digits[1]] != 1) {
Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results.
2009-12-29 05:44:39 +00:00
unsigned i;
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
memset(decode_table, 0xff, sizeof(decode_table));
for (i = 0; i < sizeof(digits); i++)
decode_table[digits[i]] = i;
}
/* Allocate enough space to hold the entire output. */
/* Note that we may not use all of this... */
MFp4: libarchive 2.5.4b. (Still 'b' until I get a bit more feedback, but the 2.5 branch is shaping up nicely.) In addition to many small bug fixes and code improvements: * Another iteration of versioning; I think I've got it right now. * Portability: A lot of progress on Windows support (though I'm not committing all of the Windows support files to FreeBSD CVS) * Explicit tracking of MBS, WCS, and UTF-8 versions of strings in archive_entry; the archive_entry routines now correctly return NULL only when something is unset, setting NULL properly clears string values. Most charset conversions have been pushed down to archive_string. * Better handling of charset conversion failure when writing or reading UTF-8 headers in pax archives * archive_entry_linkify() provides multiple strategies for hardlink matching to suit different format expectations * More accurate bzip2 format detection * Joerg Sonnenberger's extensive improvements to mtree support * Rough support for self-extracting ZIP archives. Not an ideal approach, but it works for the archives I've tried. * New "sparsify" option in archive_write_disk converts blocks of nulls into seeks. * Better default behavior for the test harness; it now reports all failures by default instead of coredumping at the first one.
2008-05-26 17:00:24 +00:00
out = (char *)malloc(len - len / 4 + 1);
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
if (out == NULL) {
*out_len = 0;
return (NULL);
}
d = out;
while (len > 0) {
/* Collect the next group of (up to) four characters. */
int v = 0;
int group_size = 0;
while (group_size < 4 && len > 0) {
/* '=' or '_' padding indicates final group. */
if (*src == '=' || *src == '_') {
len = 0;
break;
}
/* Skip illegal characters (including line breaks) */
if (*src > 127 || *src < 32
|| decode_table[*src] == 0xff) {
len--;
src++;
continue;
}
v <<= 6;
v |= decode_table[*src++];
len --;
group_size++;
}
/* Align a short group properly. */
v <<= 6 * (4 - group_size);
/* Unpack the group we just collected. */
switch (group_size) {
case 4: d[2] = v & 0xff;
/* FALLTHROUGH */
case 3: d[1] = (v >> 8) & 0xff;
/* FALLTHROUGH */
case 2: d[0] = (v >> 16) & 0xff;
break;
case 1: /* this is invalid! */
break;
}
d += group_size * 3 / 4;
}
*out_len = d - out;
return (out);
}
static char *
url_decode(const char *in)
{
char *out, *d;
const char *s;
Portability and style fixes: * Actually use the HAVE_<header>_H macros to conditionally include system headers. They've been defined for a long time, but only used in a few places. Now they're used pretty consistently throughout. * Fill in a lot of missing casts for conversions from void*. Although Standard C doesn't require this, some people have been trying to use C++ compilers with this code, and they do require it. Bit-for-bit, the compiled object files are identical, except for one assert() whose line number changed, so I'm pretty confident I didn't break anything. ;-)
2006-11-10 06:39:46 +00:00
out = (char *)malloc(strlen(in) + 1);
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
if (out == NULL)
return (NULL);
for (s = in, d = out; *s != '\0'; ) {
Minor code hardening: Verify the final bytes of the string are actually accessible before trying to use them.
2008-05-27 04:46:12 +00:00
if (s[0] == '%' && s[1] != '\0' && s[2] != '\0') {
POSIX.1e-style Extended Attribute support This commit implements storing/reading POSIX.1e-style extended attribute information in "pax" format archives. An outline of the storage format is in the tar.5 manpage. The archive_read_extract() function has code to restore those archives to disk for Linux; FreeBSD implementation is forthcoming. Many thanks to Jaakko Heinonen for finding flaws in earlier proposals and doing the bulk of the coding in this work.
2006-03-21 16:55:46 +00:00
/* Try to convert % escape */
int digit1 = tohex(s[1]);
int digit2 = tohex(s[2]);
if (digit1 >= 0 && digit2 >= 0) {
/* Looks good, consume three chars */
s += 3;
/* Convert output */
*d++ = ((digit1 << 4) | digit2);
continue;
}
/* Else fall through and treat '%' as normal char */
}
*d++ = *s++;
}
*d = '\0';
return (out);
}
static int
tohex(int c)
{
if (c >= '0' && c <= '9')
return (c - '0');
else if (c >= 'A' && c <= 'F')
return (c - 'A' + 10);
else if (c >= 'a' && c <= 'f')
return (c - 'a' + 10);
else
return (-1);
}
Reference in New Issue Copy Permalink