The zfs_log_clone_range() function is never called from the
zfs_clone_range_replay() function, so I assumed it is safe to assert
that zil_replaying() is never TRUE here. It turns out zil_replaying()
also returns TRUE when the sync property is set to disabled.
Fix the problem by just returning if zil_replaying() returns TRUE.
Reported by: Florian Smeets
Signed-off-by: Pawel Jakub Dawidek pawel@dawidek.net
Approved by: oshogbo, mm
Fix data corruption when cloning embedded blocks
Don't overwrite blk_phys_birth, as for embedded blocks it is part of
the payload.
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Pawel Jakub Dawidek <pawel@dawidek.net>
Issue #13392Closes#14739
Approved by: oshogbo, mm
There is one data corruption problem reported and fixed upstream, not
cherry-picked here yet.
On top of it the following fires under load:
VERIFY(zil_replaying(zfsvfs->z_log, tx));
The patch which introduced the entire machinery is a revert candidate,
but as the machinery came with a dedicated feature flag, doing so would
render affected pools read-only at best. To be figured out.
As a temporary bandaid at least stop the active usage.
Note this patch does not make the feature disappear from zpool upgrade.
Sponsored by: Rubicon Communications, LLC ("Netgate")
It is not implemented and causes panics on boot.
This is a temporary measure until someone(tm) sorts it out.
Reported by: many
Sponsored by: Rubicon Communications, LLC ("Netgate")
API contract requires VOPs to handle EXDEV internally, worst case by
falling back to the generic copy routine. This broke with the recent
changes.
While here whack custom loop to lock 2 vnodes with vn_lock_pair, which
provides the same functionality internally. write start/finish around
it plays no role so got eliminated.
One difference is that vn_lock_pair always takes an exclusive lock on
both vnodes. I did not patch around it because current code takes an
exclusive lock on the target vnode. zfs supports shared-locking for
writes, so this serializes different calls to the routine as is, despite
range locking inside. At the same time you may notice the source vnode
can get some traffic if only shared-locked, thus once more this goes
the safer route of exclusive-locking. Note this should be patched to
use shared-locking for both once the feature is considered stable.
Technically the switch to vn_lock_pair should be a separate change, but
it would only introduce churn immediately whacked by the rest of the
patch.
[note: technically the review is still in progress, but so is the
active breakage]
Sponsored by: Rubicon Communications, LLC ("Netgate")
If block_cloning is disabled, or other errors from zfs_clone_range()
return an EXDEV we should fall back to vn_generic_copy_file_range().
This fixes issues when copying files on the same dataset with
block_cloning disabled.
Upstreamed as pull request to OpenZFS.
Reviewed by: Mateusz Guzik <mjguzik@gmail.com>
OpenZFS pull request: 14713
Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"
Obtained from: OpenZFS
OpenZFS commit: 431083f75b
In commit 0a5b942d4 the FreeBSD SECTION_STATIC macro was set to
".rodata". This assembler directive is supported by LLVM (as a
convenience alias for ".section .rodata") by not by GNU as.
This caused the FreeBSD builds that are done with gcc to fail.
Therefore, use ".section .rodata" instead, similar to the other
asm_linkage.h headers.
[mjg: cherry-picked from upstream zfs bf1bec394e
to unbreak gcc12 build]
Reviewed-by: Mateusz Guzik <mjguzik@gmail.com>
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Dimitry Andric <dimitry@andric.com>
Closes#14526
Most ATM support was removed prior to FreeBSD 12. The netgraph support
was kept as it was less intrusive, but it is presumed to be unused.
Reviewed by: manu
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D38879
It's possible for Xen to switch the video mode set by the boot loader,
so that the information passed in the kernel metadata is no longer
valid. Fetch the video mode used by Xen using an hypercall and update
the medatada for the kernel to use the correct video mode.
Sponsored by: Citrix Systems R&D
Summary:
This review ports mlx5 driver, kernel's OFED stack (userland is already enabled), KTLS and krping to powerpc64 and powerpc64le.
krping requires a small change since it uses assembly for amd64 / i386.
NOTE: On powerpc64le RDMA works fine in the userspace with libmlx5, but on powerpc64 it does not. The problem is that contrib/ofed/libmlx5/doorbell.h checks for SIZEOF_LONG but this macro exists on neither powerpc64* nor amd64. Thus, the file silently goes to the fallback function written for 32-bit architectures. It works fine on little-endian architectures, but causes a hard fail on big-endian. It's possible it may also cause some runtime issues on little-endian.
Thus, on powerpc64 I verified that RDMA works with krping.
Reviewers: #powerpc, hselasky
Subscribers: bdrewery, imp, emaste, jhibbits
Differential Revision: https://reviews.freebsd.org/D38786
Summary:
This review ports mlx5 driver, kernel's OFED stack (userland is already enabled), KTLS and krping to powerpc64 and powerpc64le.
krping requires a small change since it uses assembly for amd64 / i386.
NOTE: On powerpc64le RDMA works fine in the userspace with libmlx5, but on powerpc64 it does not. The problem is that contrib/ofed/libmlx5/doorbell.h checks for SIZEOF_LONG but this macro exists on neither powerpc64* nor amd64. Thus, the file silently goes to the fallback function written for 32-bit architectures. It works fine on little-endian architectures, but causes a hard fail on big-endian. It's possible it may also cause some runtime issues on little-endian.
Thus, on powerpc64 I verified that RDMA works with krping.
Reviewers: #powerpc, hselasky
Subscribers: bdrewery, imp, emaste, jhibbits
Differential Revision: https://reviews.freebsd.org/D38786
This is a variant of CK_*_FOREACH from FreeBSD queue.h which starts
iteration at the specified item. If the item pointer is NULL, iteration
starts from the beginning of the list.
Upstream commit 74366be35a6f4635f248a3c62d2d23245a4eb0f4.
MFC after: 2 weeks
Sponsored by: Klara, Inc.
When jail.conf set the nopersist flag during startup, it was
incorrectly destroying the per-jail ZFS settings.
PR: 260160
Reported by: imp (previous version), mm (upstream), freqlabs (upstream)
MFC after: immediately
Sponsored by: Modirum MDPay
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D38662
Merge commit 'b3392d84da5bf2162baf937c77e0557f3fd8a52b' into zstd_1.5.2
full changelog: https://github.com/facebook/zstd/compare/v1.4.8...v1.5.2
Updated sys/kern/subr_compressor.c to new API
MFC after: 3 days
Relnotes: yes
Sponsored by: Klara, Inc.
Notable upstream pull request merges:
#13805 Configure zed's diagnosis engine with vdev properties
#14110 zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
#14121 Batch enqueue/dequeue for bqueue
#14123 arc_read()/arc_access() refactoring and cleanup
#14159 Bypass metaslab throttle for removal allocations
#14243 Implement uncached prefetch
#14251 Cache dbuf_hash() calculation
#14253 Allow reciever to override encryption property in case of replication
#14254 Restrict visibility of per-dataset kstats inside FreeBSD jails
#14255 Zero end of embedded block buffer in dump_write_embedded()
#14263 Cleanups identified by CodeQL and Coverity
#14264 Miscellaneous fixes
#14272 Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names
#14287 FreeBSD: Remove stray debug printf
#14288 Colorize zfs diff output
#14289 deadlock between spa_errlog_lock and dp_config_rwlock
#14291 FreeBSD: Fix potential boot panic with bad label
#14292 Add tunable to allow changing micro ZAP's max size
#14293 Turn default_bs and default_ibs into ZFS_MODULE_PARAMs
#14295 zed: add hotplug support for spare vdevs
#14304 Activate filesystem features only in syncing context
#14311 zpool: do guid-based comparison in is_vdev_cb()
#14317 Pack zrlock_t by 8 bytes
#14320 Update arc_summary and arcstat outputs
#14328 FreeBSD: catch up to 1400077
#14376 Use setproctitle to report progress of zfs send
#14340 Remove some dead ARC code
#14358 Wait for txg sync if the last DRR_FREEOBJECTS might result in a hole
#14360 libzpool: fix ddi_strtoull to update nptr
#14364 Fix unprotected zfs_znode_dmu_fini
#14379 zfs_receive_one: Check for the more likely error first
#14380 Cleanup of dead code suggested by Clang Static Analyzer
#14397 Avoid passing an uninitialized index to dsl_prop_known_index
#14404 Fix reading uninitialized variable in receive_read
#14407 free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report
#14418 Introduce minimal ZIL block commit delay
#14422 x86 assembly: fix .size placement and replace .align with .balign
Obtained from: OpenZFS
OpenZFS commit: 9cd71c8604
This updated DDP is intended to be used with the forthcoming ice(4)
driver update to 1.37.7-k. (But it will still work with the current
version.)
Co-authored-by: Piotr Kubaj <pkubaj@FreeBSD.org>
Signed-off-by: Eric Joyner <erj@FreeBSD.org>
MFC after: 1 week
Sponsored by: Intel Corporation
Overview:
Intel(R) QuickAssist Technology (Intel(R) QAT) provides hardware
acceleration for offloading security, authentication and compression
services from the CPU, thus significantly increasing the performance and
efficiency of standard platform solutions.
This commit introduces:
- Intel® 4xxx Series platform support.
- QuickAssist kernel API implementation update for Generation 4 device.
Enabled services: symmetric cryptography and data compression.
- Increased default number of crypto instances in static configuration
for performance purposes.
OCF backend changes:
- changed GCM/CCM MAC validation policy to generate MAC by HW
and validate by SW due to the QAT HW limitations.
Patch co-authored by: Krzysztof Zdziarski <krzysztofx.zdziarski@intel.com>
Patch co-authored by: Michal Jaraczewski <michalx.jaraczewski@intel.com>
Patch co-authored by: Michal Gulbicki <michalx.gulbicki@intel.com>
Patch co-authored by: Julian Grajkowski <julianx.grajkowski@intel.com>
Patch co-authored by: Piotr Kasierski <piotrx.kasierski@intel.com>
Patch co-authored by: Adam Czupryna <adamx.czupryna@intel.com>
Patch co-authored by: Konrad Zelazny <konradx.zelazny@intel.com>
Patch co-authored by: Katarzyna Rucinska <katarzynax.kargol@intel.com>
Patch co-authored by: Lukasz Kolodzinski <lukaszx.kolodzinski@intel.com>
Patch co-authored by: Zbigniew Jedlinski <zbigniewx.jedlinski@intel.com>
Sponsored by: Intel Corporation
Reviewed by: markj, jhb
Differential Revision: https://reviews.freebsd.org/D36254
Use NL80211_BAND_2GHZ instead of a hard coded 0 as array index for the
band. While LinuxKPI provides a KPI compatibility some of these values
may not necessarily be KBI compatible (in this case they shoule be so
this is a NOP) and after all it is better style.
No functional change.
MFC after: 3 days
iwl_trans_pcie_send_hcmd() does not seem to exist (anymore). Mark it
as __linux__ so we can submit the cleanup with the next upstream run.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
To quote from a comment above vput_final:
<quote>
* XXX Some filesystems pass in an exclusively locked vnode and strongly depend
* on the lock being held all the way until VOP_INACTIVE. This in particular
* happens with UFS which adds half-constructed vnodes to the hash, where they
* can be found by other code.
</quote>
As is there is no mechanism which allows filesystems to denote that a
vnode is fully initialized, consequently problems like the above are
only found the hard way(tm).
Add rudimentary support for state transitions, which in particular allow
to assert the vnode is not legally unlocked until its fate is decided
(either construction finishes or vgone is called to abort it).
The new field lands in a 1-byte hole, thus it does not grow the struct.
Bump __FreeBSD_version to 1400077
Reviewed by: kib (previous version)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D37759
Notable upstream pull request merges:
#13782 Fix setting the large_block feature after receiving a snapshot
#14157 FreeBSD: stop using buffer cache-only routines on sync
#14172 zed: post a udev change event from spa_vdev_attach()
#14181 zed: unclean disk attachment faults the vdev
#14190 Bump checksum error counter before reporting to ZED
#14196 Remove atomics from zh_refcount
#14197 Don't leak packed recieved proprties
#14198 Switch dnode stats to wmsums
#14199 Remove few pointer dereferences in dbuf_read()
#14200 Micro-optimize zrl_remove()
#14204 Lua: Fix bad bitshift in lua_strx2number()
#14212 Zstd fixes
#14218 Avoid a null pointer dereference in zfs_mount() on FreeBSD
#14235 nopwrites on dmu_sync-ed blocks can result in a panic
#14236 zio can deadlock during device removal
#14247 Micro-optimize fletcher4 calculations
#14261 FreeBSD: zfs_register_callbacks() must implement error check
correctly
Obtained from: OpenZFS
OpenZFS commit: 59493b63c1
In preparation for future updates remove the budget argument from the
netif_napi_add() in drivers and update LinuxKPI to reflect that it is
gone and only set it internally. This required changes to the currently
committed wireless drivers based on LinuxKPI (iwlwifi, rtw88, rtw89).
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
f808c43ad9 introduced a FreeBSD specific
behaviour to wait for firmware load completion before returning from
loading the driver. This does no longer allow iwl_drv_stop to detect
that startup has completed and it will wait indefinitely for a
completion event that will not happen.
We could change the complete() call to a complete_all() but to avoid
confusion, future side effects, and for simplicity daisy-chain two
complete events in FreeBSD.
PR: 267869
Reported by: Peter Much (pmc citylink.dinoex.sub.org)
Tested by: Peter Much (pmc citylink.dinoex.sub.org)
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
The import of OpenZFS PR 13758 causes a panic in zfsctl_is_node()
if ZFS is mounting as root filesystem. This implements a workaround
until the issue is resolved by authors.
Notable upstream pull request merges:
#13680 Add options to zfs redundant_metadata property
#13758 Allow mounting snapshots in .zfs/snapshot as a regular user
#13838 quota: disable quota check for ZVOL
#13839 quota: extend quota for dataset
#13973 Fix memory leaks in dmu_send()/dmu_send_obj()
#13977 Avoid unnecessary metaslab_check_free calling
#13978 PAM: Fix unchecked return value from zfs_key_config_load()
#13979 Handle possible null pointers from malloc/strdup/strndup()
#13997 zstream: allow decompress to fix metadata for uncompressed
records
#13998 zvol_wait logic may terminate prematurely
#14001 FreeBSD: Fix a pair of bugs in zfs_fhtovp()
#14003 Stop ganging due to past vdev write errors
#14039 Optimize microzaps
#14050 Fix draid2+2s metadata error on simultaneous 2 drive failures
#14062 zed: Avoid core dump if wholedisk property does not exist
#14077 Propagate extent_bytes change to autotrim thread
#14079 FreeBSD: vn_flush_cached_data: observe vnode locking contract
#14093 Fix ARC target collapse when zfs_arc_meta_limit_percent=100
#14106 Add ability to recompress send streams with new compression
algorithm
#14119 Deny receiving into encrypted datasets if the keys are not
loaded
#14120 Fix arc_p aggressive increase
#14129 zed: Prevent special vdev to be replaced by hot spare
#14133 Expose zfs_vdev_open_timeout_ms as a tunable
#14135 FreeBSD: Fix out of bounds read in zfs_ioctl_ozfs_to_legacy()
#14152 Adds the `-p` option to `zfs holds`
#14161 Handle and detect #13709's unlock regression
Obtained from: OpenZFS
OpenZFS commit: 2163cde450
This cherry-picks upstream ed566bf1cd
- Add a zfs_exit() call in an error path, otherwise a lock is
leaked.
- Remove the fid_gen > 1 check. That appears to be Linux-specific:
zfsctl_snapdir_fid() sets fid_gen to 0 or 1 depending on whether
the snapshot directory is mounted. On FreeBSD it fails, making
snapshot dirs inaccessible via NFS.
PR: 266236
MFC after: 3 days
Import two files left out initially from the driver needed for debugfs
support [1]. Adjust the driver further to make it compile on FreeBSD.
This is currently turned off and needs more LinuxKPI/lindebugfs work.
Being in the tree will allow us to collaboratively work on it and
then we can enable it for good.
Obtained from: Linux wireless-testing (tag: wt-2022-10-19) [1]
2c9078b9abcb884e27360340aaa7dfd4c0de29b3
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Powerpc* doesn't support support floating point in the kernel yet,
so disable it on zfs for now.
This fixes "panic: altivec unavailable trap" when loading zfs.ko
Approved by: jhibbits
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D36894
