Commit Graph

32 Commits

Author SHA1 Message Date
Conrad Meyer
98689d0ffb libprivatezstd: Restrict symbol visibility similar to upstream
We have to relax three symbols to dynamically link zstd(1), but the rest
are the same.
2021-01-03 11:26:46 -08:00
Conrad Meyer
bcae12b591 zstd: Fix non-FreeBSD CI libzstd build
Fix non-FreeBSD CI build after v1.4.8.  This definition was only used in
zstd(1), which isn't part of non-FreeBSD CI (I guess).  The ifdef was
added in v1.4.5 import.

Upstream does not currently support shared-linked zstd(1), but I have
proposed https://github.com/facebook/zstd/pull/2450 .  If that is
adopted, we can add -DZSTD_PROGRAMS_LINK_SHARED to our libzstd build and
drop some diffs.

Reported by:	uqs
2021-01-03 07:42:00 -08:00
Conrad Meyer
f7cd7fe51c sys/contrib/zstd: Import zstd 1.4.8
Merge commit 'f6ae97673c28bdb9ae795bd235ab6f26f2536a2d' into main

Changes from 1.4.5:
  * https://github.com/facebook/zstd/releases/tag/v1.4.8
  * https://github.com/facebook/zstd/releases/tag/v1.4.7
  (and there was no public v1.4.6)

Conflicts:
	sys/contrib/zstd/lib/common/zstd_internal.h (new ZSTD_NO_INTRINSICS)
2020-12-25 07:42:41 -08:00
Conrad Meyer
b6be5f6405 Unbreak ARM64 kernel build after r361426
X-MFC-With:	r361426
2020-05-23 23:10:03 +00:00
Conrad Meyer
37f1f2684f Update to Zstandard 1.4.5
As usual, the full release notes are found on Github:

  https://github.com/facebook/zstd/releases/tag/v1.4.5

Notable changes include:

* Improved decompress performance on amd64 and arm (5-10%
  and 15-50%, respectively).
* '--patch-from' zstd(1) CLI option, which provides something like a very fast
  version of bspatch(1) with slightly worse compression.  See release notes.

In this update, I dropped the 3-year old -O0 workaround for an LLVM ARM bug;
the bug was fixed in LLVM SVN in 2017, but we didn't remove this workaround
from our tree until now.

MFC after:	I won't, but feel free
Relnotes:	yes
2020-05-23 21:23:46 +00:00
Conrad Meyer
c0f419ec56 contrib/zstd: Revise Xlist for 1.4.5 import 2020-05-23 20:39:36 +00:00
Conrad Meyer
9cbefe25d4 Update to Zstandard 1.4.4
The full release notes can be found on Github:

  https://github.com/facebook/zstd/releases/tag/v1.4.4

Notable changes in this release include improved decompression speed (about
10%).  See the Github release notes for more details.

MFC after:	I'm not going to, but feel free
Relnotes:	yes
2019-11-16 16:39:08 +00:00
Conrad Meyer
51ee1719f9 zstd: Add an Xlist and document use during import
No functional change.
2019-11-06 06:50:02 +00:00
Warner Losh
5e34c4c505 Stopgap fix for gcc platforms.
Our in-tree gcc doesn't have a no-tree-vectorize optimization knob, so we get a
warning that it's unused. This causes the build to fail on all our gcc platforms.
Add a quick version check as a stop-gap measure to get CI building again.
2019-08-08 20:09:36 +00:00
Conrad Meyer
4d3f1eafc9 Update to Zstandard 1.4.2
The full release notes for 1.4.1 (skipped) and 1.4.2 can be found on Github:

  https://github.com/facebook/zstd/releases/tag/v1.4.1
  https://github.com/facebook/zstd/releases/tag/v1.4.2

These are mostly minor updates; 1.4.1 purportedly brings something like 7%
faster decompression speed.

Relnotes:	yes
2019-08-08 16:54:22 +00:00
Conrad Meyer
2b9c00cb6b Update to Zstandard 1.4.0
The full release notes can be found on Github:

  https://github.com/facebook/zstd/releases/tag/v1.4.0

Relnotes:	yes
2019-04-19 02:54:13 +00:00
Conrad Meyer
a0483764f3 Update to Zstandard 1.3.8
This merge brings in a couple new files, which needed to be attached to the
build; a new dependency on <limits.h>, which must be stubbed; and a name
change in the Context parameter constants, from ZSTD_p_foo to ZSTD_c_foo.

Significantly, it fixes a kernel build error with GCC where floating-point
functions were included in the kernel build, by hiding them under the same
compile-time #ifdef that already covered their invocation.  That issue was
introduced to FreeBSD in the 1.3.7 update and tracked upstream here:

  https://github.com/facebook/zstd/issues/1386

The full 1.3.8 release notes can be found on Github:

  https://github.com/facebook/zstd/releases/tag/v1.3.8

Relnotes:	yes
2018-12-29 21:18:01 +00:00
Conrad Meyer
b937183e00 Revert accidentally included changes in r342108
If you're curious, please follow along in https://reviews.freebsd.org/D18537 .

Sorry for the noise.
2018-12-15 05:47:22 +00:00
Conrad Meyer
26649bb5e8 efirt: When present, attempt to use EFI runtime services to shutdown
PR:		maybe related to 233998 (inconclusive at this time)
Submitted by:	byuu <byuu AT tutanota.com> (previous version)
Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D18506
2018-12-15 05:46:04 +00:00
Conrad Meyer
0f743729ab Update to Zstandard 1.3.7
Relnotes:	yes
Sponsored by:	Dell EMC Isilon
2018-10-22 18:29:12 +00:00
Mateusz Guzik
34c538c356 malloc: try to use builtins for zeroing at the callsite
Plenty of allocation sites pass M_ZERO and sizes which are small and known
at compilation time. Handling them internally in malloc loses this information
and results in avoidable calls to memset.

Instead, let the compiler take the advantage of it whenever possible.

Discussed with:	jeff
2018-06-02 22:20:09 +00:00
Conrad Meyer
19fcbaf142 Update to Zstandard 1.3.4
Includes our local patch to conditionalize use of __builtin_clz(ll) on
Clang's __has_builtin() (which is just defined to false when building with
GCC).

The issue is tracked upstream at https://github.com/facebook/zstd/pull/884 .
Otherwise, these are vanilla Zstandard 1.3.4 files.

Reported by:	allanjude, Yann Collet
Sponsored by:	Dell EMC Isilon
2018-03-26 23:54:59 +00:00
Conrad Meyer
22aec4de9f lib(private)zstd: Fix riscv build
Link __bswap[ds]i2() intrinsics in to libzstd for riscv, where the C runtime
apparently lacks such intrinsics.

Broken in r330894.

Reported by:	asomers
Sponsored by:	Dell EMC Isilon
2018-03-18 03:42:57 +00:00
Conrad Meyer
052d3c1290 Update to Zstandard 1.3.3
Includes patch to conditionalize use of __builtin_clz(ll) on __has_builtin().
The issue is tracked upstream at https://github.com/facebook/zstd/pull/884 .
Otherwise, these are vanilla Zstandard 1.3.3 files.

Note that the 1.3.4 release should be due out soon.

Sponsored by:	Dell EMC Isilon
2018-03-14 03:00:17 +00:00
Conrad Meyer
af3415f104 MFV: zstd: FIO_addFInfo: Fully initialize output 'total' struct
Silence a Coverity warning about 'windowSize' being uninitialized.
(Yes, nothing that calls this routine actually uses the windowSize
value.  Still, appeasing Coverity is pretty harmless in this case.)

Reported by:	Coverity
Reviewed by:	Yann Collet
Obtained from:	zstd 606374269cf3485972c90b993fbb84dc20da032f
Sponsored by:	Dell EMC Isilon
2018-03-05 20:03:45 +00:00
Mark Johnston
b9625b0d3b Move zstd malloc()/free()/calloc() macros to stdlib.h.
The definitions otherwise leak into anything that includes zstd.h,
which is not desirable for native FreeBSD code.

Reviewed by:	allanjude, cem, imp
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D14352
2018-02-13 19:18:00 +00:00
Pedro F. Giffuni
86446f5610 zstd: Use mallocarray(9) for calloc macro.
This is in contrib code but since we only have mallocarray(9) in current
we will not upstream this.

This effectively brings back r327934, which was reverted to correct the
log message.
2018-01-13 19:02:51 +00:00
Pedro F. Giffuni
8c3720bfc1 Revert r327934 to fix the log message. 2018-01-13 18:56:42 +00:00
Pedro F. Giffuni
d195572171 zstd: Use memalloc(9) for calloc macro.
This is in contrib code but since we only have memalloc(9) in current we
will not upstream this.
2018-01-13 18:09:09 +00:00
Conrad Meyer
8fb95dc265 Zstd: Add bswap intrinsics for small MIPS systems (e.g., Onion Omega)
Reported by:	mizhka
Sponsored by:	Dell EMC Isilon
2018-01-11 06:30:50 +00:00
Conrad Meyer
8038068a2a Finally, fix Zstd kernel build on MIPS and RISC-V
Add an implementation of the intrinsics invoked by __builtin_ctz{,ll} and
__builtin_clz{,ll}, and include this compilation unit on platforms that lack
assembly intrinsics for those builtins (MIPS and RISC-V).

Future cleanup work might involve bringing these into a mini libcompiler-rt
for the standalone kernel environment.  Or cleaning up the approach upstream
takes for builtins in standalone environments (or just FreeBSD).  For now,
at least this builds, and doesn't require modifying the vendor code.

Reported by:	jeff, markj, mizhka
Reviewed by:	jhb (earlier version), rpokala (comment text earlier version)
Sponsored by:	Dell EMC Isilon
2018-01-10 06:30:59 +00:00
Conrad Meyer
d3692a4dee Integrate zstd into the kernel
Mock userspace headers and include mocked headers first in compilation
command to inject kernel headers and override e.g., malloc(3) with
malloc(9).

Submitted by:	allanjude
Reviewed by:	imp (earlier version), bapt (earlier version)
Differential Revision:	https://reviews.freebsd.org/D10407
2018-01-08 20:14:16 +00:00
Baptiste Daroussin
1ad8ac7a1d Revert local changes made to make zstd(1) frontend behave like gzip(1) and friends
This change was made to allow zstd(1) to be a dropin replacement for gzip(1) and
friends, allowing easy integration, in particular with newsyslog(8). At the
price of having a zstd(1) command which by default behaves differently than what
upstream default, confusing users.

newsyslog(8) has been adapted to now be more flexible in what it accepts as
compression program, so we can switch back zstd(1) to its default behaviour

Reported by:	many
2017-12-06 09:53:10 +00:00
Allan Jude
90edb2ac5b Use __has_builtin() to ensure clz and clzll builtins are available
The existing check of the GCC version number is not sufficient

This fixes the build on sparc64 in preparation for integrating ZSTD into
the kernel for ZFS and Crash Dumps.
2017-12-04 01:16:26 +00:00
Baptiste Daroussin
6247019ec2 Actually commit the right patch for r325929 2017-11-17 09:33:29 +00:00
Baptiste Daroussin
5cdf66d629 Do not remove the sources when zstd is called as zstdcat 2017-11-17 09:29:26 +00:00
Warner Losh
0c16b53773 Move zstd from contrib to sys/contrib so it can be used in the
kernel. Adjust the Makefiles that referenced it to the new path.

Sponsored by: Netflix
OK'd by: cem@ and AllanJude@
2017-11-14 05:03:38 +00:00