Sync libarchive with vendor including important security fixes.
Issues fixed (FreeBSD):
PR #778: ACL error handling
Issue #745: Symlink check prefix optimization is too aggressive
Issue #746: Hard links with data can evade sandboxing restrictions
This update fixes the vulnerability #3 and vulnerability #4 as reported in
"non-cryptanalytic attacks against FreeBSD update components".
https://gist.github.com/anonymous/e48209b03f1dd9625a992717e7b89c4f
Fix for vulnerability #2 has already been merged in r304989.
MFC after: 1 week
Security: http://gist.github.com/anonymous/e48209b03f1dd9625a992717e7b89c4f
Sync libarchive with vendor
Vendor issues fixed:
PR #777: Multiple bugfixes for setup_acls()
This includes a bugfix for a bug that caused ACLs not to be read properly
for files and directories inside subdirectories and as a result not being
stored or being incorrectly stored in tar archives.
MFC after: 3 days
Sync libarchive with vendor including three security fixes
Vendor issues fixed:
Issue #744: Very long pathnames evade symlink checks
Issue #748: libarchive can compress, but cannot decompress zip some files
PR #750: ustar: fix out of bounds read on empty string ("") filename
PR #755: fix use of acl_get_flagset_np() on FreeBSD
MFC after: 3 days
Update libarchive to 3.2.1 (bugfix and security fix release)
List of vendor fixes:
- fix exploitable heap overflow vulnerability in Rar decompression
(vendor issue 719, CVE-2016-4302, TALOS-2016-0154)
- fix exploitable stack based buffer overflow vulnebarility in mtree
parse_device functionality (vendor PR 715, CVE-2016-4301, TALOS-2016-0153)
- fix exploitable heap overflow vulnerability in 7-zip read_SubStreamsInfo
(vendor issue 718, CVE-2016-4300, TALOS-2016-152)
- fix integer overflow when computing location of volume descriptor
(vendor issue 717)
- fix buffer overflow when reading a crafred rar archive (vendor issue 521)
- fix possible buffer overflow when reading ISO9660 archives on machines
where sizeof(int) < sizeof(size_t) (vendor issue 711)
- tar and cpio should fail if an input file named on the command line is
missing (vendor issue 708)
- fix incorrect writing of gnutar filenames that are exactly 512 bytes
long (vendor issue 682)
- allow tests to be run from paths that are equal or longer than 128
characters (vendor issue 657)
- add memory allocation errors in archive_entry_xattr.c (vendor PR 603)
- remove dead code in archive_entry_xattr_add_entry() (vendor PR 716)
- fix broken decryption of ZIP files (vendor issue 553)
- manpage style, typo and description fixes
Post-3.2.1 vendor fixes:
- fix typo in cpio version reporting (Vendor PR 725, 726)
- fix argument range of ctype functions in libarchive_fe/passphrase.c
- fix ctype use and avoid empty loop bodies in WARC reader
MFC after: 1 week
Security: CVE-2016-4300, CVE-2016-4301, CVE-2016-4302
Approved by: re (kib)
Update libarchive to 3.2.0
New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive
Patched files (fixed compiler warnings):
contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)
MFC after: 1 month
Relnotes: yes
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.
Sponsored by: EMC / Isilon Storage Division
the FreeBSD test suite
functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.
A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)
As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
We want to ensure we always use libarchive from ports in the ports tree.
It simplifies ports maintainance and anyway libarchive.pc was not reflecting the
different way libarchive can be built in base
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
header with archive_crypto_private.h, and its ARCHIVE_HASH_xxx macros
were renamed to ARCHIVE_CRYPTO_xxx.
Rename these macros in lib/libarchive/config_freebsd.h, to re-enable the
hashes for libarchive again. This affects the mtree format writer, and
the xar format reader and writer modules.
This also requires changes in the library order for statically linking
rescue, otherwise ld would complain about redefined symbols. Thanks to
jkim for pointing out the solution.
Reviewed by: kientzle
MFC after: 1 week
This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation
This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h
Discussed at: BSDcan
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.
Update libarchive to 3.1.2
Some of new features:
- support for lrzip and grzip compression
- support for writing tar v7 format
- b64encode and uuencode filters
- support for __MACOSX directory in Zip archives
- support for lzop compresion (external utility)
Some of new features:
- New readers: RAR, LHA/LZH, CAB reader, 7-Zip
- New writers: ISO9660, XAR
- Improvements to many formats, especially including ISO9660 and Zip
- Stackable write filters to write, e.g., tar.gz.uu in a single pass
- Exploit seekable input; new "seekable" Zip reader can exploit the Zip
Central Directory when it's available; the old "streamable" Zip reader
is still fully supported for cases where seeking is not possible.
Full release notes available at:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes
3730:
Fix issue 174 (Windows path names, not relevant for FreeBSD)
3734:
Merge r1989: archive_clear_error should set errno to 0.
3735:
Merge r3247 from trunk: Clear errors before returning
from archive_read_support_format_all()
3799:
Check the position before dereferencing the pointer.
This avoids dereferencing one byte past the end of a string
3824:
Merge r3823 from trunk for issue 199 (hang in iso9660 reading)
Obtained from: http://code.google.com/p/libarchive
MFC after: 2 weeks
The following additional vendor revisions are applied:
Revision 3740:
Use archive_clear_error() to clear the error markers.
Obtained from: http://code.google.com/p/libarchive
MFC after: 2 weeks
Fixes extraction of Zip entries that use length-at-end without specifying
either the compressed or uncompressed length. In particular, fixes bsdtar
extraction of such files.
Obtained from: http://code.google.com/p/libarchive
Reported by: Patrick Lamaiziere <patfbsd@davenulle.org> (freebsd-stable@)
MFC after: 1 week
ISO images with tar.
Vendor revision 3648 (merge of 3647):
Additional fix to issue 168 because the change of r3642 was not sufficient.
- Make sure "CL" entry appear after its "RE" entry which the "CL" entry
should be connected with.
- Give consideration to the case that the top level "RE" entry has
already been exposed outside before its tree.
Approved by: re (kib)
Obtained from: libarchive (release/2.8, svn rev 3648)
MFC after: 3 days
Upstream revision 3645 (merge of 3642):
Change the mechanism handling a rr_moved directory,
which is Rockridge extension that can exceed the limitation of
a maximum directory depth of ISO 9660.
- Stop reading all entries at a time.
- Connect "CL" entry to "RE" entry dynamically, which "CL" and "RE"
have information to rebuild a full directory tree.
- Tweak some related tests since we use Headsort for re-ordering
entries and it cannot make a steady order when the keies of
the entries are the same.
http://code.google.com/p/libarchive/issues/detail?id=168
Reviewed by: kientzle
Approved by: re (kib)
Obtained from: libarchive (release/2.8, svn rev 3645)
MFC after: 3 days
Partial merge of 2431 from trunk: Retry writes on EINTR.
This should fix the SIGINT handler in bsdtar.
Note: The rest of r2431 can't be merged, since it interacts
with a big write-side rearchitecture.
PR: bin/149409
Reviewed by: kientzle
Approved by: re (kib)
MFC after: 3 days
violated ECMA-119 (ISO9660): allow reserved4 to be 0x20 in PVD.
This allows tar to read FreeBSD distribution ISO images created
with makefs prior to NetBSD bin/45217 bugfix (up to 9.0-BETA1).
In addition, merge following important bugfixes from
libarchive's release/2.8 branch:
Revision 2812:
Merge 2811 from trunk: Don't try to verify that compression-level=0
produces larger results than the default compression, since this isn't
true for all versions of liblzma.
Revision 2817:
Merge 2814 from trunk: Fix Issue 121 (mtree parser error)
http://code.google.com/p/libarchive/issues/detail?id=121
Revision 2820:
Fix issue 119.
Change the file location check that a file location does not exceed
volume block. New one is that a file content does not exceed volume
block(end of an ISO image). It is better than previous check even
if the issue did not happen.
While reading an ISO image generated by an older version of mkisofs
utility, a file location indicates the end the ISO image if its file
size is zero and it is the last file of all files of the ISO image,
so it is possible that the location value is the same as the number
of the total block of the ISO image.
http://code.google.com/p/libarchive/issues/detail?id=119
Revision 2955:
Issue 134: Fix libarchive 2.8 crashing in archive_write_finish() when
the open has failed and we're trying to write Zip format.
http://code.google.com/p/libarchive/issues/detail?id=134
Revision 2958:
Followup on Issue 134:
1) Port test_open_failure to libarchive 2.8 branch to test
the problem reported in Issue 134.
This test also shows that archive_read_open() sometimes
fails to report open errors correctly.
2) Fix the bug in archive_read.c
3) Comment out the tests that close functions are invoked
promptly when open fails; that's fully fixed in libarchive 3.0,
but I don't think it's worth fixing here.
Revision 3484:
Use uintmax_t with %ju
Revision 3487:
Fix issue 163.
Correctly allocate enough memory for a input buffer saved.
http://code.google.com/p/libarchive/issues/detail?id=163
Revision 3542:
Merge 2516, 2536 from trunk: Allow path table offset values of
0 and 18, which are used by some ISO writers.
Reviewed by: kientzle
Approved by: re (kib)
MFC after: 3 days
[mixing the two can be quite bad -- they define the same context structures,
but with differing structure members (and sizes)]
Update the hash function support comments, and update config_freebsd.h
to match.
Approved by: kientzle
In particular, this check avoids a warning when
extracting directory entries from certain GNU tar
archives that store directory contents.
MFC after: 3 days
because there's inter-entry garbage, just scan forward
to find the next one. This allows us to handle a lot
of Zip archives that have been modified in-place.
Thanks to: Gleb Kurtsou for sending me a sample archive
bottom of the manpages and order them consistently.
GNU groff doesn't care about the ordering, and doesn't even mention
CAVEATS and SECURITY CONSIDERATIONS as common sections and where to put
them.
Found by: mdocml lint run
Reviewed by: ru
Adjust dependencies for programs using libarchive
Add xz and linkage against liblzma to rescue system
Approved by: kientzle, delphij (mentor)
MFC after: 2 weeks
Instead of trying to reference-count them and free them as soon
as they are no longer needed, we now just keep them around and free
them all when we release the archive object. This fixes a number
of minor memory leaks, especially when reading damaged archives.
Similar to libexec/, do the same with lib/. Make WARNS=6 the norm and
lower it when needed.
I'm setting WARNS?=0 for secure/. It seems secure/ includes the
Makefile.inc provided by lib/. I'm not going to touch that directory.
Most of the code there is contributed anyway.
from googlecode:
* Support for zisofs compressed entries
* Support for relocated deep directories
* Direct calculation of link counts for accurate nlink values
even on images that lack Rockridge extensions
* Faster handling of the internal file lists.
* Better detection of ISO variants
1<<30 bytes of memory, which is required for some streams.
Also, try to make the taster more discriminating about raw lzma streams.
The detection here is still really weak, though; please use xz instead
of lzma.
