dtrace_instr_size() is needed by the forthcoming RISC-V port of kinst,
as well as by libdtrace in D38825 for both amd64 and RISC-V.
Reviewed by: markj, mhorne
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39489
If you remove an unused spare and then reinsert it, zfsd will now online
it in all pools.
Do not MFC without 2a58b312b6 (but it's ok to MFC that one without this
one).
Submitted by: Ameer Hamza <ahamza@ixsystems.com> (zfsd), Me (tests)
MFC after: 2 weeks
MFC with: 2a58b312b6
Sponsored by: iX Systems, Axcient
Pull Request: https://github.com/freebsd/freebsd-src/pull/697
The section will contain static relocations which do not need to be
preserved after linking, and moreover these relocations may reference
symbols that end up getting removed.
Do not set SHF_ALLOC and instead let the linker decide what needs to be
done.
PR: 258872
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"
Obtained from: OpenZFS
OpenZFS commit: 431083f75b
This drops function `getexecname()` redirection.
Historically `getexecname()` is a compatibility definition. Since
openzfs has its own implementation of function `getexecname()` in libspl
and has been merged into base, the compat header file stdlib.h is
no longer needed and should not be used.
Also without this fix libspl will end up an incompatible version of
`getprogname()` with libc. In particular, if zfs is enabled, programs
such as pgrep in /rescue can be wrongly statically linked with libspl
and will not function properly.
PR: 269738
Reviewed by: markj
Fixes: 9e5787d228 Merge OpenZFS support in to HEAD
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D38733
It contains the ctf tools that most users will never use.
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D38223
Largely untested, as we can't really do anything with user probes
without an implementation of fasttrap. However, this is enough to
generate an embedded dtrace program with `dtrace -G` and link the
generated ELF file.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38301
There is an extern declaration of the global variable environ (which
has no portable header that defines it) that is unused on FreeBSD due
to zfs_setproctitle_init() being a stub.
Reviewed by: emaste
Reported by: GCC
Differential Revision: https://reviews.freebsd.org/D38231
Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.
This reverts commit adc3c128c6.
Reviewed by: brooks, kevans, jhb (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33108
When dtrace starts, it tries to detect if the dtrace klds are loaded,
and if not, it loads them by loading the dtraceall kld. This module
depends on most dtrace modules, including systrace for the native
freebsd and freebsd32 ABIs. However, it does not depend on the
systrace_linux klds, as they in turn depend on the linux ABI klds, and
we don't want to load an ABI module that the user has not explicitly
requested. This can leave a naive user in a state where they think all
syscall providers have been loaded, yet linux ABI syscalls are
"invisible" to dtrace.
To fix this, check to see if the linux ABI modules are loaded. If they
are, then load their systrace klds.
Reviewed by: markj, (emaste & jhb, earlier versions)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D37986
Otherwise, if multiple object files contain references to the same
probe, newish lld will refuse to link them by default, raising a
duplicate global symbol definition error. Previously, duplicate global
symbols with identical absolute st_values were permitted by both lld and
GNU ld.
Since dtrace has no use for probe function symbols after the relocation
performed by dtrace -G, make the symbols weak as well, following a
suggestion from MaskRay.
Reported by: dim
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.
The most import one is the inpcb itself. With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa. Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that. However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort. The new intotcpcb() macro is ready for such move.
The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.
One interesting side effect is that now TCP data is allocated from
SMR-protected zone. Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.
Large part of the change was done with sed script:
s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g
Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.
Differential revision: https://reviews.freebsd.org/D37127
Previoulsy when using NO_ROOT we recorded METALOG entries for dwatch
hard links with different file modes than their link targets, which is
not permitted.
Reviewed by: bapt
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37477
Notable upstream pull request merges:
#13680 Add options to zfs redundant_metadata property
#13758 Allow mounting snapshots in .zfs/snapshot as a regular user
#13838 quota: disable quota check for ZVOL
#13839 quota: extend quota for dataset
#13973 Fix memory leaks in dmu_send()/dmu_send_obj()
#13977 Avoid unnecessary metaslab_check_free calling
#13978 PAM: Fix unchecked return value from zfs_key_config_load()
#13979 Handle possible null pointers from malloc/strdup/strndup()
#13997 zstream: allow decompress to fix metadata for uncompressed
records
#13998 zvol_wait logic may terminate prematurely
#14001 FreeBSD: Fix a pair of bugs in zfs_fhtovp()
#14003 Stop ganging due to past vdev write errors
#14039 Optimize microzaps
#14050 Fix draid2+2s metadata error on simultaneous 2 drive failures
#14062 zed: Avoid core dump if wholedisk property does not exist
#14077 Propagate extent_bytes change to autotrim thread
#14079 FreeBSD: vn_flush_cached_data: observe vnode locking contract
#14093 Fix ARC target collapse when zfs_arc_meta_limit_percent=100
#14106 Add ability to recompress send streams with new compression
algorithm
#14119 Deny receiving into encrypted datasets if the keys are not
loaded
#14120 Fix arc_p aggressive increase
#14129 zed: Prevent special vdev to be replaced by hot spare
#14133 Expose zfs_vdev_open_timeout_ms as a tunable
#14135 FreeBSD: Fix out of bounds read in zfs_ioctl_ozfs_to_legacy()
#14152 Adds the `-p` option to `zfs holds`
#14161 Handle and detect #13709's unlock regression
Obtained from: OpenZFS
OpenZFS commit: 2163cde450
It is useful to have zfs utilities and lib in a separate package as
it allow users to create image that can support ZFS (i.e. not with
WITHOUT_ZFS in src.conf set) without bloating the default image with
all zfs tools (for example for jails).
Differential Revision: https://reviews.freebsd.org/D36225
kinst does not instantiate its probes automatically, it only does so on
demand via an ioctl interface implemented by /dev/kinst. This change
modifies libdtrace to perform that work when the script references the
kinst provider, similar to the way pid provider probes are implemented.
Reviewed by: markj
MFC after: 3 months
Sponsored by: Google, Inc. (GSoC 2022)
Differential Revision: https://reviews.freebsd.org/D36852
This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers. It's not clear what
semantics "regs" should have for them anyway.
This is akin to "uregs", which nominally provides access to the
userspace registers. In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.
Usage example: print the contents of %rdi upon each call to
amd64_syscall():
fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}
Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.
Reviewed by: christos
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36799
The debug traces for reference counting in ZFS use the pointer of the
owning object as a "tag" for references to check that when an object
drops a reference it had actually held one. In a couple of places ZFS
drops references after freeing the owning object. In userland GCC
realizes this is a use after free. However, since only the value of
the pointer is used, and it isn't indirected, the use is harmless.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D36818
Notable upstream pull request merges:
#13725 Fix BLAKE3 tuneable and module loading on Linux and FreeBSD
#13756 FreeBSD: Organize sysctls
#13773 FreeBSD: add kqfilter support for zvol cdev
#13781 Importing from cachefile can trip assertion
#13794 Apply arc_shrink_shift to ARC above arc_c_min
#13798 Improve too large physical ashift handling
#13799 Revert "Avoid panic with recordsize > 128k, raw sending and
no large_blocks"
#13802 Add zfs.sync.snapshot_rename
#13831 zfs_enter rework
#13855 zfs recv hangs if max recordsize is less than received
recordsize
Obtained from: OpenZFS
OpenZFS commit: c629f0bf62
CTF integers have an explicit width and so can be used to represent
bitfields. Bitfield types emitted by ctfconvert(1) share the name of
the base integer type, so a struct field with type "unsigned int : 15"
will have a type named "unsigned int".
To avoid ambiguity when looking up types by name, add a suffix to names
of bitfield types to distinguish them from the base type. Then, if
ctfmerge happens to order bitfield types before the corresponding base
type in a CTF file, a name lookup will return the base type, which is
always going to be the desired behaviour.
PR: 265403
Reported by: cy
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Add an option to enable/disable DTrace without disabling ZFS. New
architectures such as CHERI may support ZFS before they support DTrace
and the old model of WITHOUT_CDDL disabling both wasn't helpful.
For compatiblity, the CDDL option remains and WITHOUT_CDDL implies
WITHOUT_DTRACE. WITHOUT_DTRACE also implies WITHOUT_CTF.
As part of this change, largely convert cddl/*/Makefile to using the
more compact SUBDIR.${MK_<FOO>}+= form rather than using intermediate
variables.
Reviewed by: markj
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D35718
Passing "0x%p" to sprintf results in double "0x" being printed.
This causes a dtrace script compilation failure when "-d" flag
is specified.
Fix that by removing the extraneous "0x".
Reviewed by: markj
Approved by: mw(mentor)
Obtained from: Semihalf
Sponsored by: Alstom
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35690
This is a repeat of 13cee53cef35 ("libicp: Fix build for powerpc64le")
but for the rescue version of the library.
Fixes: 1f1e2261e3 ("zfs: merge openzfs/zfs@deb121309")
The C sources use the assembly routines for little-endian powerpc64, and
the assembly files have ppc64le in their names, but the guard here was
for big-endian powerpc64.
Fixes: 1f1e2261e3 ("zfs: merge openzfs/zfs@deb121309")
Notable upstream pull request merges:
#12321 Fix inflated quiesce time caused by lwb_tx during zil_commit()
#13244 zstd early abort
#13360 Verify BPs as part of spa_load_verify_cb()
#13452 More speculative prefetcher improvements
#13466 Expose zpool guids through kstats
#13476 Refactor Log Size Limit
#13484 FreeBSD: libspl: Add locking around statfs globals
#13498 Cancel in-progress rebuilds when we finish removal
#13499 zed: Take no action on scrub/resilver checksum errors
#13513 Remove wrong assertion in log spacemap
Obtained from: OpenZFS
OpenZFS commit: b9d98453f9
Notable upstream pull request merges:
#10662 zvol_wait: Ignore locked zvols
#12789 Improve log spacemap load time
#12812 Improved zpool status output, list all affected datasets
#13277 FreeBSD: Use NDFREE_PNBUF if available
#13302 Make zfs_max_recordsize default to 16M
#13311 Fix error handling in FreeBSD's get/putpages VOPs
#13345 FreeBSD: Fix translation from ABD to physical pages
#13373 zfs: holds: dequadratify
#13375 Corrected edge case in uncompressed ARC->L2ARC handling
#13388 Improve mg_aliquot math
#13405 Reduce dbuf_find() lock contention
#13406 FreeBSD: use zero_region instead of allocating a dedicated page
Obtained from: OpenZFS
OpenZFS commit: c0cf6ed679
