Commit Graph

844 Commits

Author SHA1 Message Date
Martin Matuska
9f3de9e26d MFV r313071:
Sync libarchive with vendor

Vendor changes (relevant to FreeBSD):
- support extracting NFSv4 ACLs from Solaris tar archives
- bugfixes and optimizations in the ACL code
- multiple fixes in the test suite
- typo and other small bugfixes

Security fixes:
- cab reader: endless loop when parsing MSZIP signature (OSS-Fuzz 335)
- LHA reader: heap-buffer-overflow in lha_read_file_header_1()
  (CVE-2017-5601)
- LZ4 reader: null-pointer dereference in lz4_filter_read_legacy_stream()
  (OSS-Fuzz 453)
- mtree reader: heap-buffer-overflow in detect_form() (OSS-Fuzz 421, 443)
- WARC reader: heap-buffer-overflow in xstrpisotime() (OSS-Fuzz 382, 458)

Memory leak fixes:
- ACL support: free memory allocated by acl_get_qualifier()
- disk writer: missing free in create_filesystem_object()
- file reader: fd leak (Coverity 1016755)
- gnutar writer: fix free in archive_write_gnutar_header()
  (Coverity 101675)
- iso 9660 reader: missing free in parse_file_info()
  (partial Coverity 1016754)
- program reader: missing free in __archive_read_program()
- program writer: missing free in __archive_write_program_free()
- xar reader: missing free in xar_cleanup()
- xar reader: missing frees in expat_xmlattr_setup()
  (Coverity 1229979-1229981)
- xar writer: missing free in file_free()
- zip reader: missing free in zip_read_local_file_header()

MFC after:	1 week
X-MFC with:	310866, 310868, 310870, 311899
2017-02-02 00:50:46 +00:00
Enji Cooper
dd1ed99736 Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones
This implifies pathing in make/displayed output

MFC after:    3 weeks
Sponsored by: Dell EMC Isilon
2017-01-20 03:54:36 +00:00
Martin Matuska
9520841213 Build libarchive tests missing in r311899
MFC after:	1 week
X-MFC with:	r311899
2017-01-10 22:01:37 +00:00
Enji Cooper
a08cc1ee27 Look for list.h in ${.CURDIR} to unbreak the build with a ports-based copy
of llvm38 on ^/stable/11 (oh, the bugs you find when you set CC,CXX,CPP
manually and it skips the bootstrap stage for the toolchain...)
2017-01-01 04:13:24 +00:00
Martin Matuska
e9ed7ea4dc MFV r310796, r310797:
Sync libarchive with vendor.

Vendor changes (relevant to FreeBSD):
PR #771: Add NFSv4 ACL support to pax and restricted pax

NFSv4 ACL information may now be stored to and restored from tar archives.
ACL must be non-trivial and supported by the underlying filesystem, e.g.
natively by ZFS or by UFS with the NFSv4 ACL enable flag set.

MFC after:	2 weeks
Relnotes:	yes
2016-12-30 22:18:22 +00:00
Martin Matuska
7105995c64 MFV r310622:
Sync libarchive with vendor.

Vendor bugfixes (relevant to FreeBSD):
PR 846: Spelling fixes
PR 850: Fix issues with reading certain jar files
OSS-Fuzz 286: Bugfix in archive_strncat_l()
2016-12-27 01:10:28 +00:00
Martin Matuska
a2e802b76b MFV r309587:
Sync libarchive with vendor.

Vendor bugfixes:

libarchive #831:
  Spelling fixes
libarchive #832:
  Relax sanity checks of number fields in tar header even more
OSS-Fuzz #16:
  Fix possible hang in uudecode_filter_read()
OSS-Fuzz #220:
  Reject an 'ar' filename table larger than 1GB or a filename larger
  than 1MB.

MFC after:	1 week
2016-12-06 01:35:27 +00:00
Martin Matuska
007dbc1f18 MFC r309362:
Sync libarchive with vendor.
Small improvements, style fixes, bugfixes.
Restores compatibility with tar archives created with Perl Archive::Tar (1)

MFC after:	1 week
Reported by:	Matthew Seaman <matthew@freebsd.org> (1)
2016-12-01 15:46:26 +00:00
Martin Matuska
d5d08d297d MFV r309299:
Sync libarchive with vendor.

Important vendor bugfixes (relevant to FreeBSD):
#821: tar -P cannot extract hardlinks through symlinks
#825: Add sanity check of tar "uid, "gid" and "mtime" fields

PR:		213255
Reported by:	Tijl Coosemans <tilj@FreeBSD.org>
MFC after:	1 week
2016-11-29 22:14:42 +00:00
Justin Hibbits
dc9b124d66 Create a new MACHINE_ARCH for Freescale PowerPC e500v2
Summary:
The Freescale e500v2 PowerPC core does not use a standard FPU.
Instead, it uses a Signal Processing Engine (SPE)--a DSP-style vector processor
unit, which doubles as a FPU.  The PowerPC SPE ABI is incompatible with the
stock powerpc ABI, so a new MACHINE_ARCH was created to deal with this.
Additionaly, the SPE opcodes overlap with Altivec, so these are mutually
exclusive.  Taking advantage of this fact, a new file, powerpc/booke/spe.c, was
created with the same function set as in powerpc/powerpc/altivec.c, so it
becomes effectively a drop-in replacement.  setjmp/longjmp were modified to save
the upper 32-bits of the now-64-bit GPRs (upper 32-bits are only accessible by
the SPE).

Note: This does _not_ support the SPE in the e500v1, as the e500v1 SPE does not
support double-precision floating point.

Also, without a new MACHINE_ARCH it would be impossible to provide binary
packages which utilize the SPE.

Additionally, no work has been done to support ports, work is needed for this.
This also means no newer gcc can yet be used.  However, gcc's powerpc support
has been refactored which would make adding a powerpcspe-freebsd target very
easy.

Test Plan:
This was lightly tested on a RouterBoard RB800 and an AmigaOne A1222
(P1022-based) board, compiled against the new ABI.  Base system utilities
(/bin/sh, /bin/ls, etc) still function appropriately, the system is able to boot
multiuser.

Reviewed By:	bdrewery, imp
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D5683
2016-10-22 01:57:15 +00:00
Martin Matuska
f3e9b21a7a MFV r306669:
Sync libarchive with vendor including security fixes.

Important vendor bugfixes (relevant to FreeBSD):
#747: Out of bounds read in mtree parser
#761: heap-based buffer overflow in read_Header (7-zip)
#784: Invalid file on bsdtar command line results in internal errors (1)

PR:		213092 (1)
MFC after:	1 week
2016-10-04 11:56:46 +00:00
Ruslan Bukin
cda20f96df Increase timeout from 300 (default) to 600 seconds.
It takes 6-7 minutes to proceed the test on MIPS64EB.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-09-26 20:13:33 +00:00
Martin Matuska
24113d8c17 MFV r305816:
Sync libarchive with vendor including important security fixes.

Issues fixed (FreeBSD):
PR #778: ACL error handling
Issue #745: Symlink check prefix optimization is too aggressive
Issue #746: Hard links with data can evade sandboxing restrictions

This update fixes the vulnerability #3 and vulnerability #4 as reported in
"non-cryptanalytic attacks against FreeBSD update components".
https://gist.github.com/anonymous/e48209b03f1dd9625a992717e7b89c4f

Fix for vulnerability #2 has already been merged in r304989.

MFC after:	1 week
Security: http://gist.github.com/anonymous/e48209b03f1dd9625a992717e7b89c4f
2016-09-14 21:15:01 +00:00
Martin Matuska
47af42f8e7 MFV r305420:
Sync libarchive with vendor

Vendor issues fixed:
PR #777: Multiple bugfixes for setup_acls()

This includes a bugfix for a bug that caused ACLs not to be read properly
for files and directories inside subdirectories and as a result not being
stored or being incorrectly stored in tar archives.

MFC after:	3 days
2016-09-05 15:40:41 +00:00
Martin Matuska
54635f0c74 Temporarily disable two libarchive tests that have not yet been fixed by
vendor. Tests will be re-enabled after a fix has been merged.

MFC after:	3 days
2016-08-27 00:47:47 +00:00
Martin Matuska
cfa49a9b0b MFV r304060:
Sync libarchive with vendor including three security fixes

Vendor issues fixed:

Issue #744: Very long pathnames evade symlink checks
Issue #748: libarchive can compress, but cannot decompress zip some files
PR #750: ustar: fix out of bounds read on empty string ("") filename
PR #755: fix use of acl_get_flagset_np() on FreeBSD

MFC after:	3 days
2016-08-14 09:26:10 +00:00
Martin Matuska
f061a2215f MFV r302003,r302037,r302038,r302056:
Update libarchive to 3.2.1 (bugfix and security fix release)

List of vendor fixes:
- fix exploitable heap overflow vulnerability in Rar decompression
  (vendor issue 719, CVE-2016-4302, TALOS-2016-0154)
- fix exploitable stack based buffer overflow vulnebarility in mtree
  parse_device functionality (vendor PR 715, CVE-2016-4301, TALOS-2016-0153)
- fix exploitable heap overflow vulnerability in 7-zip read_SubStreamsInfo
  (vendor issue 718, CVE-2016-4300, TALOS-2016-152)
- fix integer overflow when computing location of volume descriptor
  (vendor issue 717)
- fix buffer overflow when reading a crafred rar archive (vendor issue 521)
- fix possible buffer overflow when reading ISO9660 archives on machines
  where sizeof(int) < sizeof(size_t) (vendor issue 711)
- tar and cpio should fail if an input file named on the command line is
  missing (vendor issue 708)
- fix incorrect writing of gnutar filenames that are exactly 512 bytes
  long (vendor issue 682)
- allow tests to be run from paths that are equal or longer than 128
  characters (vendor issue 657)
- add memory allocation errors in archive_entry_xattr.c (vendor PR 603)
- remove dead code in archive_entry_xattr_add_entry() (vendor PR 716)
- fix broken decryption of ZIP files (vendor issue 553)
- manpage style, typo and description fixes

Post-3.2.1 vendor fixes:
- fix typo in cpio version reporting (Vendor PR 725, 726)
- fix argument range of ctype functions in libarchive_fe/passphrase.c
- fix ctype use and avoid empty loop bodies in WARC reader

MFC after:	1 week
Security:	CVE-2016-4300, CVE-2016-4301, CVE-2016-4302
Approved by:	re (kib)
2016-06-22 07:49:59 +00:00
Martin Matuska
cdf63a700c MFV r299425:
Update libarchive to 3.2.0

New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
  similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive

Patched files (fixed compiler warnings):

contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)

MFC after:	1 month
Relnotes:	yes
2016-05-12 10:16:16 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
a70cba9582 First pass through library packaging.
Sponsored by:	The FreeBSD Foundation
2016-02-04 21:16:35 +00:00
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Bryan Drewery
3c89d6b0e3 Don't override LIB*DIR variables from src.libnames.mk.
In some cases switch to the LIB*SRCDIR value.

These recently were defined in r291327 and r291619.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 22:20:04 +00:00
Bryan Drewery
62f38e94ec Don't override LIB*DIR variables from src.libnames.mk.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 22:04:59 +00:00
Bryan Drewery
277fbb92d5 Remove redundant DPSRCS which were already in SRCS.
DPSRCS already contains all of SRCS.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 20:38:07 +00:00
Bryan Drewery
7b3ea376a2 META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:28 +00:00
Enji Cooper
59e2ff550c Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite

functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.

A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)

As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-12 18:31:21 +00:00
Bryan Drewery
75f66cde5c MFV r285970:
Apply upstream changeset bf4f6ec64e:

  Fix issue 356: properly skip a sparse file entry in a tar file.

PR:		201506
MFC after:	3 days
Relnotes:	yes
2015-07-28 18:41:28 +00:00
Baptiste Daroussin
dbfbf7ae92 Drop libarchive.pc
We want to ensure we always use libarchive from ports in the ports tree.
It simplifies ports maintainance and anyway libarchive.pc was not reflecting the
different way libarchive can be built in base
2015-07-19 21:31:52 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Jilles Tjoelker
97792364fd libarchive: Allow setting nanosecond timestamps. 2015-04-25 21:25:00 +00:00
Tijl Coosemans
1243a98e38 Remove the const qualifier from iconv(3) to comply with POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv.html

Adjust all code that calls iconv.

PR:		199099
Exp-run by:	antoine
MFC after:	2 weeks
2015-04-15 09:09:20 +00:00
Baptiste Daroussin
6b129086dc Convert libraries to use LIBADD
While here reduce a bit overlinking
2014-11-25 11:07:26 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Martin Matuska
0ac327af11 Add pkg-config file for libarchive
Requested by:	bapt
MFC after:	1 week
2014-11-02 09:37:45 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Dimitry Andric
02b6306fb0 In r232153, libarchive 3.0.3 was imported, replacing the archive_hash.h
header with archive_crypto_private.h, and its ARCHIVE_HASH_xxx macros
were renamed to ARCHIVE_CRYPTO_xxx.

Rename these macros in lib/libarchive/config_freebsd.h, to re-enable the
hashes for libarchive again.  This affects the mtree format writer, and
the xar format reader and writer modules.

This also requires changes in the library order for statically linking
rescue, otherwise ld would complain about redefined symbols.  Thanks to
jkim for pointing out the solution.

Reviewed by:	kientzle
MFC after:	1 week
2014-07-26 15:33:20 +00:00
Marcel Moolenaar
e7d939bda2 Remove ia64.
This includes:
o   All directories named *ia64*
o   All files named *ia64*
o   All ia64-specific code guarded by __ia64__
o   All ia64-specific makefile logic
o   Mention of ia64 in comments and documentation

This excludes:
o   Everything under contrib/
o   Everything under crypto/
o   sys/xen/interface
o   sys/sys/elf_common.h

Discussed at: BSDcan
2014-07-07 00:27:09 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Simon J. Gerraty
cc3f4b9965 Merge from head 2014-05-08 23:54:15 +00:00
Warner Losh
c6063d0da8 Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
Simon J. Gerraty
9d2ab4a62d Merge head 2014-04-27 08:13:43 +00:00
Warner Losh
3bdf775801 NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.
2014-04-13 05:21:56 +00:00
Simon J. Gerraty
3caf0790a8 Merge head@256284 2013-10-13 02:35:19 +00:00
Xin LI
e4dedeefae Temporarily disable iconv for non-shared library builds. The dynamic
loading of conversation table is not yet compatible with static builds.

Approved by:	re (gjb)
2013-09-26 17:55:36 +00:00
Simon J. Gerraty
69e6d7b75e sync from head 2013-04-12 20:48:55 +00:00
Martin Matuska
acc60b03c1 MFV r248590,248594:
Update libarchive to 3.1.2

Some of new features:
  - support for lrzip and grzip compression
  - support for writing tar v7 format
  - b64encode and uuencode filters
  - support for __MACOSX directory in Zip archives
  - support for lzop compresion (external utility)
2013-03-22 13:36:03 +00:00