Commit Graph

104 Commits

Author SHA1 Message Date
Simon J. Gerraty
d9a4274795 Update/fix Makefile.depend for userland 2023-04-18 17:14:23 -07:00
Dag-Erling Smørgrav
f9349d4274 gzip: Add support for decompressing zstd files.
Sponsored by:	Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37236
2022-11-04 13:42:09 +01:00
Kyle Evans
5b7adeb184 zdiff: avoid non-conformant features
`setvar` is a non-conformant feature that looks slightly neater but is
not portable to other /bin/sh implementations.  Making the script
portable is straightforward, so let's do it.

Tests are added to make sure that I didn't break anything major in the
process.

Reviewed by:	bapt (previous version), jilles
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D35275
2022-05-30 19:25:16 -05:00
Eric van Gyzen
58135fbd8b gzip: fix error handling in unxz
The result of fstat() was not checked.  Furthermore, there was a
redundant check of st.st_size.  Fix both.

Reported by:    Coverity
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2022-02-25 09:30:29 -06:00
Tom Jones
4669f23ef7 Remove SMALL conditionals from gzip
gzip has SMALL conditionals which enable building a reduced size version
of the binary. These exist as part of the introduction of BSD licensed
gzip in 2004 in NetBSD and appear to have been required to reach a size
for inclusion in their install media. For more information see commits
to gzip in the NetBSD tree on the 28th of March 2004.

SMALL doesn't appear to be hooked up to our build system and
complicates gzip quite a bit.

Reviewed by:	kevans,	imp
Sponsored by:	Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D34047
2022-01-27 17:27:21 +00:00
Tom Jones
21c966a6b9 Fix test output when gzip is run with -tlv
When run with test, verbose and list we need to parse the file otherwise
the test output is "NOT OK" even for the file is valid.

Reviewed by:	kevans, allanjude, imp
Sponsored by:	Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D34046
2022-01-27 17:20:23 +00:00
Bryan Drewery
2dfa4b66b3 fts_read: Handle error from a NULL return better.
This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set.  That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.

As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it.  Otherwise we might see
a random error from one of the iterations.

gzip is ignoring most errors and could be improved separately.

Reviewed by:	vangyzen
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D27184
2020-12-08 23:38:26 +00:00
Xin LI
a0eb385c2e Remove unneeded checks for prelen.
In order to determine the type of a compressed file, we have to read
in the first four bytes which may also be important for decompression
purposes, to do that we would pass the buffer that we have already
read in, along with the size of it.

Rename header1 to fourbytes to make that explicit, and remove all
checks for prelen.

Reported by:	cem
Reviewed by:	cem
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D24034
2020-03-14 05:57:22 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Simon J. Gerraty
5ab1c5846f Add Makefile.depend.options
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend

DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options

See share/mk/dirdeps-options.mk

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22469
2019-12-11 17:37:37 +00:00
Xin LI
197884df8a Correct documentation year.
MFC after:	2 weeks
2019-01-07 08:29:27 +00:00
Xin LI
5c4b64e66b Port NetBSD improvements:
- Add -l support for xz files
 - Add lzip support to gzip based on the example lzip decoder.

Obtained from:	NetBSD
MFC after:	2 weeks
Relnotes:	yes
2019-01-07 08:27:11 +00:00
Xin LI
34e314e794 Use endian.h le32dec() instead of rolling our own.
Suggested by:	phk
Reviewed by:	imp, pfg
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D16192
2018-07-10 01:42:28 +00:00
Conrad Meyer
e76247cd60 gzip(1): Don't shadow global 'err'
Unbreak work build on ppc due to -Werror=shadow.  Introduced in r336121.

X-MFC-With:		r336121
2018-07-09 08:37:55 +00:00
Xin LI
b393a8ace6 Don't delete outfile unconditionally.
MFC after:	1 month
2018-07-09 06:19:33 +00:00
Pedro F. Giffuni
3fcbc83d04 gzip: fix for undefined behavior.
Unportable left shift reported with MKSANITIZER=yes
USE_SANITIZER=undefined:

# progress -zf ./games.tgz  tar -xp -C "./" -f -
/public/src.git/usr.bin/gzip/gzip.c:2126:33: runtime error: left shift of
251 by 24 places cannot be represented in type 'int'
100%
|****************************************************************************************************************|
44500 KiB  119.69 MiB/s    00:00 ETA

Refactor the following code into something that is more clear
and fix signed integer shift, by casting all buf[] elements to
(unsigned int):

unsigned char buf[8];
uint32_t usize;
[...]
else {
    usize = buf[4] | buf[5] << 8 |
            buf[6] << 16 | buf[7] << 24;
[...]

New version:

    usize = buf[4];
    usize |= (unsigned int)buf[5] << 8;
    usize |= (unsigned int)buf[6] << 16;
    usize |= (unsigned int)buf[7] << 24;

Only the "<< 24" part needs explicit cast, but for consistency make the
integer promotion explicit and clear to a code reader.

Sponsored by <The NetBSD Foundation>

Obtained from:	NetBSD (CVS rev. 1.113)
MFC after:	1 week
2018-07-08 22:39:33 +00:00
Xin LI
b6f7731dba Remove "All rights reserved" from my files.
See r333391 for the rationale.

MFC after:	1 week
2018-05-10 06:41:08 +00:00
Xin LI
f5e4607265 Close the correct file descriptor.
MFC after:	2 weeks
2017-12-12 06:56:21 +00:00
Conrad Meyer
611ecc623b gzip(1): Remove duplicate close()
CID:		1383560
Reported by:	Coverity
Sponsored by:	Dell EMC Isilon
2017-12-12 01:19:08 +00:00
Xin LI
807e955710 Create links for xzdiff.
MFC after:	2 weeks
2017-12-05 07:01:10 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Xin LI
216f72f141 Set errno to EFTYPE instead of EINVAL to be more consistent with the
rest of code.

MFC after:	1 month
2017-11-25 09:03:38 +00:00
Xin LI
90f528e8d7 Support SIGINFO.
Obtained from:	NetBSD
MFC after:	2 weeks
2017-11-21 08:14:30 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Xin LI
b8fe1f690c Reflect actual NetBSD revision we already have.
MFC after:	3 days
2017-02-06 07:02:17 +00:00
Konstantin Belousov
1c32456953 Use type-independent formats for printing nlink_t and ino_t.
Extracted from:	ino64 work by gleb, mckusick
Discussed with:	mckusick
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-01-06 16:59:33 +00:00
Xin LI
104b0f4a1c style(9) indent changes, no actual code change.
MFC after:	2 weeks
2016-11-26 07:02:44 +00:00
Xin LI
64093e14eb Use printable ASCII instead of octal representation.
MFC after:	2 weeks
2016-08-27 00:56:37 +00:00
Marcelo Araujo
d61c7fc026 Use nitems() from sys/param.h.
MFC after:	2 weeks.
Sponsored by:	gandi.net (BSD Day Taiwan)
2016-07-30 07:02:33 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Xin LI
199c446625 Update NetBSD RCS IDs to reflect the changes being upstreamed.
MFC after:	13 days
X-MFC-With:	r290024
2015-10-27 21:26:05 +00:00
Xin LI
354ed04257 In gunzip(1), treat trailing garbage as a warning and not an error. This
allows scripts to distinguish it between real fatal errors, for instance a
CRC mismatch.

Update manual page for the behavior change.

PR:		bin/203873
Submitted by:	Eugene Grosbein <eugen grosbein net>
MFC after:	2 weeks
2015-10-26 22:29:58 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Xin LI
dec32474f0 r281540 was upstreamed as NetBSD r1.108 of gzip.c, note it as merged. 2015-04-16 22:30:57 +00:00
Xin LI
c1c4677aec When reading in the original file name from gzip header, we read
in PATH_MAX + 1 bytes from the file.  In r281500, strrchr() is
used to strip possible path portion of the file name to mitigate
a possible attack.  Unfortunately, strrchr() expects a buffer
that is NUL-terminated, and since we are processing potentially
untrusted data, we can not assert that be always true.

Solve this by reading in one less byte (now PATH_MAX) and
explicitly terminate the buffer after the read size with NUL.

Reported by:	Coverity
CID:		1264915
X-MFC-with:	281500
MFC after:	13 days
2015-04-15 00:07:21 +00:00
Xin LI
0eeac58930 Sync with NetBSD:
- Mention xz(1) in gzip(1).
 - Strip away path from header name when decompressing.

MFC after:	2 weeks
2015-04-13 19:46:30 +00:00
Jilles Tjoelker
d6b3ef634c compress,gzip,xz: Preserve timestamps with nanosecond precision. 2015-02-17 13:12:54 +00:00
Baptiste Daroussin
3e11bd9e2a Convert to usr.bin/ to LIBADD
Reduce overlinking
2014-11-25 14:29:10 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Xin LI
060ea80ecb Sync with NetBSD.
MFC after:	2 weeks
2014-10-23 01:22:29 +00:00
Enji Cooper
cdfd89cea1 Integrate usr.bin/gzip/tests from NetBSD into atf/kyua
Sponsored by: EMC / Isilon Storage Division
2014-10-09 02:24:34 +00:00