Add an option to enable/disable DTrace without disabling ZFS. New
architectures such as CHERI may support ZFS before they support DTrace
and the old model of WITHOUT_CDDL disabling both wasn't helpful.
For compatiblity, the CDDL option remains and WITHOUT_CDDL implies
WITHOUT_DTRACE. WITHOUT_DTRACE also implies WITHOUT_CTF.
As part of this change, largely convert cddl/*/Makefile to using the
more compact SUBDIR.${MK_<FOO>}+= form rather than using intermediate
variables.
Reviewed by: markj
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D35718
Passing "0x%p" to sprintf results in double "0x" being printed.
This causes a dtrace script compilation failure when "-d" flag
is specified.
Fix that by removing the extraneous "0x".
Reviewed by: markj
Approved by: mw(mentor)
Obtained from: Semihalf
Sponsored by: Alstom
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35690
This is a repeat of 13cee53cef35 ("libicp: Fix build for powerpc64le")
but for the rescue version of the library.
Fixes: 1f1e2261e3 ("zfs: merge openzfs/zfs@deb121309")
The C sources use the assembly routines for little-endian powerpc64, and
the assembly files have ppc64le in their names, but the guard here was
for big-endian powerpc64.
Fixes: 1f1e2261e3 ("zfs: merge openzfs/zfs@deb121309")
Notable upstream pull request merges:
#12321 Fix inflated quiesce time caused by lwb_tx during zil_commit()
#13244 zstd early abort
#13360 Verify BPs as part of spa_load_verify_cb()
#13452 More speculative prefetcher improvements
#13466 Expose zpool guids through kstats
#13476 Refactor Log Size Limit
#13484 FreeBSD: libspl: Add locking around statfs globals
#13498 Cancel in-progress rebuilds when we finish removal
#13499 zed: Take no action on scrub/resilver checksum errors
#13513 Remove wrong assertion in log spacemap
Obtained from: OpenZFS
OpenZFS commit: b9d98453f9
Notable upstream pull request merges:
#10662 zvol_wait: Ignore locked zvols
#12789 Improve log spacemap load time
#12812 Improved zpool status output, list all affected datasets
#13277 FreeBSD: Use NDFREE_PNBUF if available
#13302 Make zfs_max_recordsize default to 16M
#13311 Fix error handling in FreeBSD's get/putpages VOPs
#13345 FreeBSD: Fix translation from ABD to physical pages
#13373 zfs: holds: dequadratify
#13375 Corrected edge case in uncompressed ARC->L2ARC handling
#13388 Improve mg_aliquot math
#13405 Reduce dbuf_find() lock contention
#13406 FreeBSD: use zero_region instead of allocating a dedicated page
Obtained from: OpenZFS
OpenZFS commit: c0cf6ed679
The presence of the latter causes a link error when building a
statically linked ctfdump(1) because libc defines the same symbol.
libc's warn() is defined as a weak symbol and so does not cause the same
problem, but let's just use libc's version.
Reported by: stephane rochoy <stephane.rochoy@stormshield.eu>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
This is for marking a test case is flakey and should not be executed in
the CI environment.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34635
Notable upstream pull request merges:
#9078: log xattr=sa create/remove/update to ZIL
#11919: Cross-platform xattr user namespace compatibility
#13014: Report dnodes with faulty bonuslen
#13016: FreeBSD: Fix zvol_cdev_open locking
#13019: spl: Don't check FreeBSD rwlocks for double initialization
#13027: Fix clearing set-uid and set-gid bits on a file when
replying a write
#13031: Add enumerated vdev names to 'zpool iostat -v' and
'zpool list -v'
#13074: Enable encrypted raw sending to pools with greater ashift
#13076: Receive checks should allow unencrypted child datasets
#13098: Avoid dirtying the final TXGs when exporting a pool
#13172: Fix ENOSPC when unlinking multiple files from full pool
Obtained from: OpenZFS
OpenZFS commit: a86e089415
For some reason I can't reproduce this locally, but Jenkins complains.
Reported by: Jenkins
Fixes: bdf290cd3e ("ctf: Add v3 support to CTF tools, ctf{convert,dump,merge}")
ctfdump handles v2 and v3. ctfconvert now emits only CTFv3, whereas
ctfmerge can merge v2 and v3 containers into v3 containers.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34364
In general, the patch adds indirection to minimize the amount of code
that needs to know about differences between v2 and v3. Specifically,
some new ctf_get_ctt_* functions are added, and new LCTF_* macros are
added to use the underlying container's version to do the right thing.
CTF containers can have parent/child relationships, wherein a type ID in
one container refers to a type in the parent. It is permitted for the
parent and child to have different versions.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34363
These are based on definitions added to binutils' libctf. Specifically:
- Type IDs are now encoded in 32 bits rather than 16, changing the
layout of ctf_type_t, ctf_array_t, ctf_member_t and ctf_lmember_t.
- Type info is encoded in 32 bits rather than 16. The type "kind" is
extended from 5 bits to 6, and the type "vlen" is extended from 10
bits to 25.
The main upside is that we remove the current limit, imposed by CTFv2,
of 2^{15} distinct types in the main kernel executable. Other limits,
such as that on the number of elements in an enum, imposed by the vlen
limit, are also raised.
This change adds v2 and v3 flavours of macros and type definitions which
differ between the two versions. Compatibility is preserved for now by
having generic names refer to the v2 definitions, so, e.g., ctf_type_t
is still a v2 type.
No functional change intended.
Reviewed by: Domagoj Stolfa
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34360
Use it instead of the existing ctf.h from OpenSolaris. This makes it
easier to use CTF in the core kernel, and to extend the CTF format to
support wider type IDs.
The imported ctf.h is modified to depend only on _types.h, and also to
provide macros which use the "parent" bit of a type ID to refer to types
in a parent CTF container.
No functional change intended.
Reviewed by: Domagoj Stolfa, emaste
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34358
It is unused on FreeBSD and complicates some efforts to modify the CTF
format to permit wider type IDs, so remove it. No functional change
intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
CTFv1 was obsolete before libctf was imported into FreeBSD, and
ctfconvert/ctfmerge can emit only CTFv2. Make ctf.h a bit easier to
maintain by ripping v1 support out. No functional change intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
For consistency with other CTF toolchain code. No functional change
intended.
Fixes: 105fd928b0 ("libctf: Improve check for duplicate SOU definitions in ctf_add_type()")
MFC after: 1 week
This call was added back in the early days of dtrace porting and
no one knows why anymore. The extra flushing causes lots of
unnecessary CPU overhead when a script produces lots of output,
as well as easily losing output because the command can't keep up.
Sponsored by: Netflix
Reviewed by: imp, markj
Differential Revision: https://reviews.freebsd.org/D34216
Parts of zstd, used in openzfs and other places, trigger a new clang 14
-Werror warning:
```
sys/contrib/zstd/lib/decompress/huf_decompress.c:889:25: error: use of bitwise '&' with boolean operands [-Werror,-Wbitwise-instead-of-logical]
(BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
While the warning is benign, it should ideally be fixed upstream and
then vendor-imported, but for now silence it selectively.
MFC after: 3 days
Notable upstream pull request merges:
#12766 Fix error propagation from lzc_send_redacted
#12805 Updated the lz4 decompressor
#12851 FreeBSD: Provide correct file generation number
#12857 Verify dRAID empty sectors
#12874 FreeBSD: Update argument types for VOP_READDIR
#12896 Reduce number of arc_prune threads
#12934 FreeBSD: Fix zvol_*_open() locking
#12947 lz4: Cherrypick fix for CVE-2021-3520
#12961 FreeBSD: Fix leaked strings in libspl mnttab
#12964 Fix handling of errors from dmu_write_uio_dbuf() on FreeBSD
#12981 Introduce a flag to skip comparing the local mac when raw sending
#12985 Avoid memory allocations in the ARC eviction thread
Obtained from: OpenZFS
OpenZFS commit: 17b2ae0b24
It was set to the start of the buffer and that can be different from the
start of teh first record because of a misalignment.
This change follows the example of dt_realloc_buf().
Reviewed by: tsoome, markj
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D33649
Members with such a type will legitimately have a size of zero, so don't
emit a warning.
PR: 260818
Reviewed by: bz
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33700
getf() on FreeBSD calls _sx_slock(), _sx_sunlock() and fget_locked().
Furthermore, it does not set the per-core fault flag, meaning it
usually ends up in a double fault panic once getf() does get called,
especially from fbt.
Reviewing the DTrace Toolkit + a number of other scripts scattered
around FreeBSD, I have not been able to find one use of getf(). Given
how broken the implementation currently is, we disable it until it
can be implemented properly.
Also comment out a test in aggs/tst.subr.d for getf().
Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33378
In 9fae47666 zfsd got a libspl dependency to avoid undefined references.
However that workaround did not help external consumers of libzfs_core.
Fix all missing dependencies lld 13 and the rtld complain about.
Reviewed by: freqlabs, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32521
Notable upstream pull request merges:
#12392 Avoid panic in case of pool errors and missing L2ARC
#12448 skip snapshot in zfs_iter_mounted()
#12516 Fix NFS and large reads on older kernels
#12533 Fail invalid incremental recursive send gracefully
#12569 FreeBSD: Really zero the zero page
#12575 Reject zfs send -RI with nonexistent fromsnap
#12602 Correct refcount_add in dmu_zfetch
#12650 zpool should call zfs_nicestrtonum() with non-NULL handle
Obtained from: OpenZFS
OpenZFS commit: ec64fdb93d
When copying a struct or union from one CTF container to another,
ctf_add_type() checks whether it matches an existing type in the
destination container. It does so by looking for a type with the same
name and kind as the new type, and if one exists, it iterates over all
members of the source type and checks whether a member with matching
name and offset exists in the matched destination type. This can
produce false positives, for example because member types are not
compared, but this is not expected to arise in practice. If the match
fails, ctf_add_type() returns an error.
The procedure used for member comparison breaks down in the face of
anonymous struct and union members. ctf_member_iter() visits each
member in the source definition and looks up the corresponding member in
the desination definition by name using ctf_member_info(), but this
function will descend into anonymous members and thus fail to match.
Fix the problem by introducing a custom comparison routine which does
not assume member names are unique. This should also be faster for
types with many members; in the previous scheme, membcmp() would perform
a linear scan of the desination type's members to perform a lookup by
name. The new routine steps through the members of both types in a
single loop.
PR: 258763
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Because lld 13.0.0 is more strict about undefined references when
linking to shared libraries, it produces the following errors for zfsd:
ld: error: /home/dim/obj/home/dim/src/llvm-13-update/amd64.amd64/tmp/usr/lib/libzfs_core.so: undefined reference to libspl_assertf [--no-allow-shlib-undefined]
ld: error: /home/dim/obj/home/dim/src/llvm-13-update/amd64.amd64/tmp/usr/lib/libnvpair.so: undefined reference to libspl_assertf [--no-allow-shlib-undefined]
ld: error: /home/dim/obj/home/dim/src/llvm-13-update/amd64.amd64/tmp/usr/lib/libavl.so: undefined reference to libspl_assertf [--no-allow-shlib-undefined]
*** [zfsd.full] Error code 1
Fix this by adding libspl (where libspl_assertf lives) to zfsd's LIBADD.
MFC after: 3 days
From the zpool_influxdb.8 manual page:
zpool_influxdb produces InfluxDB-line-protocol-compatible metrics from
zpools. Like the zpool command, zpool_influxdb reads the current pool
status and statistics. Unlike the zpool command which is intended for
humans, zpool_influxdb formats the output in the InfluxDB line protocol.
The expected use is as a plugin to a metrics collector or aggregator,
such as Telegraf.
zpool_influxdb is installed into /usr/libexec/zfs/
Differential revision: https://reviews.freebsd.org/D31094
MFC after: 3 days
Onlining a vdev can fail. Log the error if it does.
Reviewed by: mav, asomers
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D30882
In dt_cc.c when the provider is an empty string, accessing
strlen(pdp->dtpd_provider) - 1 will result in a pdp->dtpd_provider[-1]
access.
Similarly, in dt_ident.c, if p2 is a NULL pointer, doing a p2++ on it is
undefined behaviour.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Google
Differential Revision: https://reviews.freebsd.org/D30778
Summary:
f20893853e made getexecname no longer guarded by BOOTSTRAPPING, but it
does not build on non-FreeBSD as it tries to use KERN_PROC_PATHNAME and
related constants.
Reviewers: mm, imp
Reviewed By: imp
Subscribers: imp
Differential Revision: https://reviews.freebsd.org/D30763
Change -Wno-error-atomic-alignment to -Wno-error=atomic-alignment
in the Makefile of libspl.
Reported by: Ed Maste <emaste@FreeBSD.org>
Fix by: Alexander Richardson <arichardson@FreeBSD.org>
