If there are less than two elements avoid executing the first
sorting loop. No functional change intended.
Reviewed by: kib@
MFC after: 1 week
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D39691
When sorting, both the C11 standard (ISO/IEC 9899:2011, K.3.6.3.2) and
the ISO/IEC JTC1 SC22 WG14 N1172 standard, does not define objects of
zero size as undefined behaviour. However Microsoft's cpp-docs does.
Add proper checks for this. Found while working on bsort(3).
Reviewed by: kib@ and emaste@
MFC after: 1 week
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D39687
The bsort(3) algorithm works by swapping objects, similarly to qsort(3),
and does not require any significant amount of additional memory.
The bsort(3) algorithm doesn't suffer from the processing time issues
known the plague the qsort(3) family of algorithms, and is bounded by
a complexity of O(log2(N) * log2(N) * N), where N is the number of
elements in the sorting array. The additional complexity compared to
mergesort(3) is a fair tradeoff in situations where no memory may
be allocated.
The bsort(3) APIs are identical to those of qsort(3), allowing for
easy drop-in and testing.
The design of the bsort(3) algorithm allows for future parallell CPU
execution when sorting arrays. The current version of the bsort(3)
algorithm is single threaded. This is possible because fixed areas
of the sorting data is compared at a time, and can easily be divided
among different CPU's to sort large arrays faster.
Reviewed by: gbe@, delphij@, pauamma_gundo.com (manpages)
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D36493
These will hopefully be fixed upstream eventually, but silence the
warnings until then.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D39518
veriexec_check_path_label() can be used to check if a specified
path has a label associated with it that contains the what we
want.
Obtained from: Juniper Networks, Inc.
Ensure veriexec opens the file before doing any read operations.
When the MAC_VERIEXEC_CHECK_PATH_SYSCALL syscall is requested, veriexec
needs to open the file before calling mac_veriexec_check_vp. This is to
ensure any set up is done by the file system. Most file systems do not
explicitly need an open, but some (e.g. virtfs) require initialization
of access tokens (file identifiers, etc.) before doing any read or write
operations.
The evaluate_fingerprint() function needs to ensure it has an open file
for reading in order to evaluate the fingerprint. The ideal solution is
to have a hook after the VOP_OPEN call in vn_open. For now, we open the
file for reading, envaluate the fingerprint, and close the file. While
this leaves a potential hole that could possibly be taken advantage of
by a dedicated aversary, this code path is not typically visited often
in our use cases, as we primarily encounter verified mounts and not
individual files. This should be considered a temporary workaround until
discussions about the post-open hook have concluded and the hook becomes
available.
Add MAC_VERIEXEC_GET_PARAMS_PATH_SYSCALL and
MAC_VERIEXEC_GET_PARAMS_PID_SYSCALL to mac_veriexec_syscall so we can
fetch and check label contents in an unconstrained manner.
Add a check for PRIV_VERIEXEC_CONTROL to do ioctl on /dev/veriexec
Make it clear that trusted process cannot be debugged. Attempts to debug
a trusted process already fail, but the failure path is very obscure.
Add an explicit check for VERIEXEC_TRUSTED in
mac_veriexec_proc_check_debug.
We need mac_veriexec_priv_check to not block PRIV_KMEM_WRITE if
mac_priv_gant() says it is ok.
Reviewed by: sjg
Obtained from: Juniper Networks, Inc.
For a process supervisor using the reaper API to track process subtrees,
it is very useful to know the state of the processes on the list.
Sponsored by: https://www.patreon.com/valpackett
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39585
Introduce the OpenBSD syntax of "scrub" option for "match" and "pass"
rules and the "set reassemble" flag. The patch is backward-compatible,
pf.conf can be still written in FreeBSD-style.
Obtained from: OpenBSD
MFC after: never
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38025
FREEBSD32 API is also supported on aarch64. Refer to aarch64 instead of
arm64 because it's about the architecture (powerpc64), not the kernel
interface...
Sponsored by: Netflix
f6a0dac84 modified the zfs_iter_* functions to take a new "flags"
parameter, and introduced a variety of flags to ask the kernel to limit
the results in various ways, reducing the amount of work the caller
needed to do to filter out things they didn't need.
Unfortunately this change broke the ABI for existing clients (read:
older versions of the `zfs` program), and was reverted 399b98198.
dc95911d2 reintroduced the original patch, with the understanding that a
backwards-compatible fix would be made before the 2.2 release branch was
tagged. This commit is that fix.
This introduces zfs_iter_*_v2 functions that have the new flags
argument, and reverts the existing functions to not have the flags
parameter, as they were before. The old functions are now reimplemented
in terms of the new, with flags set to 0.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <george.wilson@delphix.com>
Original-patch-by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Sponsored-by: Klara, Inc.
Closes#14597
Running `zfs list -o avail rpool` resulted in a core dump.
This commit will fix this.
Run the needed overhead only, when `use_color()` is true.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes#14712
by making it accept some open(2) flags. More precisely, only
O_CLOEXEC is supported, the flag is translated into the KQUEUE_CLOEXEC flag
for kqueuex(2), and O_NONBLOCK is silently ignored.
Reported and tested by: vishwin
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39377
If a malicious casper process sent a host or service string that was
too long, cap_getnameinfo would overrun the caller's buffer by one byte.
The backends for this function needlessly allocated one extra byte
for these buffers. This was harmless, but could be confusing to readers.
Reported by: Coverity (an internal run at Dell)
Reviewed by: oshogbo, emaste
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D39347
There is no real need to close descriptors before a process exits, but
these close calls demonstrate by example that kqueue descriptors occupy
the same namespace as other file descriptors.
Reviewed by: fernape, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39376
Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"
Obtained from: OpenZFS
OpenZFS commit: 431083f75b
Get/set commands can now choose to provide the interface name rather
than the interface index. This allows userspace to avoid a call to
if_nametoindex().
Suggested by: melifaro
Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D39359
The cap_dns tests require Internet access. Currently they fail when
that's not available, which for CI purposes is undesirable. Let's
instead skip the tests if none of the non-casper name/addr lookups
succeed.
To that end:
- Convert the tests to ATF so that skipping is easier to implement.
- Break up the tests into separate test cases.
- If one of the system (i.e., non-casper) lookup functions fails, skip
the test if all of them failed, otherwise fail the tests, since
partial failure indicates something is flaky and deserves a closer
look.
Reviewed by: oshogbo
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39241
These tests fail spuriously when run in parallel, since some of them
write a value to a global sysctl, read it back, and expect to see the
written value. These tests complete quite quickly in any case.
MFC after: 1 week
memmem started as a GNU extension but is now widely available.
Reviewed by: mhorne (slightly earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39337
This does not remove LLVM_TARGET_MIPS. Note that the only
MACHINE_ARCH values ending in 'hf' were all MIPS architectures, hence
removing the pattern matches for 'hf'.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D39331
These libraries are linked to directly by applications rather than
opened at runtime via dlopen().
Discussed with: oshogbo
Reviewed by: markj, emaste
Differential Revision: https://reviews.freebsd.org/D39245
This reverts commit ab80f0b21f. The intent
of this change was to avoid possible compilation errors when certain
.inc files were not regenerated, but the method turns out to cause way
more rebuilds than anticipated. Another method will have to be found,
and in the mean time, WITH_CLEAN is the solution that always works.
Fixes: ab80f0b21f
This flag ensures that the tblgen tools do not actually touch the
produced .inc file, if there are no changes to the contents. In turn,
this may prevent a number of rebuilds of files that include such .inc
files, saving build time.
While here, ensure that the shell invocations to locate the used tblgen
binary do not show unnecessary error messages.
Reported by: des
MFC after: 1 week
There are FAT12 and FAT16 file systems, but FAT13 of was an
unintentional invention of mine ...
Reported by: Ravi Pokala <rpokala@freebsd.org>
MFC after: 1 month
This update implements tallying of free directory entries during
create, delete, or rename operations on FAT12 and FAT16 file systems.
Prior to this change, the total number of root directory entries
was reported as number of inodes, but 0 as the number of free
inodes, causing system health monitoring software to warn about
a suspected disk full issue.
The FAT12 and FAT16 file systems provide a limited number of
root directory entries, e.g. 512 on typical hard disk formats.
The valid range of values is 1 to 65535, but the msdosfs code
will effectively round up "odd" values to the next multiple of 16
(e.g. 513 would allow for 528 root directory entries).
This update implements tracking of directory entries during create,
delete, or rename operations, with initial values determined by
scanning the directory when the file system is mounted.
Total and free directory entries are reported in the f_files and
f_ffree elements of struct statfs, despite differences in semantics
of these values:
- There is no limit on the number of files and directories that can
be created on a FAT file system. Only the root directory of FAT12
and FAT16 file systems is limited, any number of files can still be
created in sub-directories, even when 0 free "inodes" are reported.
- A single file can require 1 to 21 directory entries, depending on
the character set, structure, and length of the name. The DOS 8.3
style file name takes up 1 entry, and if the name does not comply
with the syntax of a DOS 8.3 file name, 1 additional entry is used
for each 13 characters of the file name. Since all these entries
have to be contiguous, it is possible that a file or directory with
a long name can not be created, despite a sufficient total number of
free directory entries.
- Renaming a file can require more directory entries than currently
allocated to store its long name, which may prevent an in-place
update of the name if more entries are needed. This may cause a
rename operation to fail if no contiguous range of free entries for
the new name can be found.
- The volume label is stored in a directory entry. An empty FAT file
system with a volume label will therefore show 1 used "inode" in
df.
- The perceentage of free inodes shown in df or monitoring tools does
only represent the state of the root directory of a FAT12 or FAT16
file system. Neither does a reported value of 0% free inodes does
prevent files from being created in sub-directories, nor does a
value of 50% free inodes guarantee that even a single file with
a "long" name can be created in the root directory (if every other
directory entry is occupied and there are no 2 contiguous entries).
The statfs(2) and df(1) man pages have been updated with a notice
regarding the possibly different semantics of values reported as
total and free inodes for non-Unix file systems.
PR: 270053
Reported by: Ben Woods <woodsb02@freebsd.org>
Approved by: mckusick
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D38987
Fix newlocale() overwriting the locale name in collate object
when same instance was used for those locales, and querylocale()
reporting unexpected value for LC_COLLATE_MASK.
PR: 255646, 269375
Reviewed by: markj, bapt (previous version)
Differential Revision: https://reviews.freebsd.org/D30146
This is a userland-only pointer that isn't relevant to the kernel and
doesn't belong in the ioctl structure shared between userland and the
kernel. For the kernel, the old structure for the ioctl is still
supported under COMPAT_FREEBSD13.
This changes vm_snapshot_req() in libvmmapi to accept an explicit
vmctx argument.
It also changes vm_snapshot_guest2host_addr to take an explicit vmctx
argument. As part of this change, move the declaration for this
function and its wrapper macro from vmm_snapshot.h to snapshot.h as it
is a userland-only API.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38125
This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.
struct vcpu is an opaque type managed by libvmmapi. For now it stores
a pointer to the VM context and an integer id.
As an immediate effect this removes the divergence between the kernel
and userland for the instruction emulation code introduced by the
recent vmm(4) changes.
Since this is a major change to the vmmapi API, bump VMMAPI_VERSION to
0x200 (2.0) and the shared library major version.
While here (and since the major version is bumped), remove unused
vcpu argument from vm_setup_pptdev_msi*().
Add new functions vm_suspend_all_cpus() and vm_resume_all_cpus() for
use by the debug server. The underyling ioctl (which uses a vcpuid of
-1) remains unchanged, but the userlevel API now uses separate
functions for global CPU suspend/resume.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38124
This uses the C++ compiler as the linker instead of the C compiler
letting the compiler driver pick the right libraries. This is a no-op
on main and stable/13 but matters for stable/12 where the current
logic breaks for external GCC since it tries to use a non-existent
libstdc++.
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39197
GCC 9 on stable/12 reports a -Wmaybe-uninitialized error for the call
to free in _pfctl_clear_states.
Reviewed by: mjg
Differential Revision: https://reviews.freebsd.org/D39198
These were brought in by the libbind import, but these functions were
never actually implemented anywhere, only header declarations and symbol
map entries were imported.
Fixes: 046c3635cd ("Bring final version of libbind:")
Fixes: e45764721a ("Update our stub resolver to final version of ...")
Reported by: ld.lld 16 being --no-undefined-version by default
Sponsored by: https://www.patreon.com/valpackett
Reviewed by: emaste
Pull request: https://github.com/freebsd/freebsd-src/pull/700
Differential Revision: https://reviews.freebsd.org/D38407
- This function no longer disables interrupts
- MLINK to reboot.9
- The mentions of autoconfiguration is more about shutdown_nice(),
coming in the next commit.
- Describe the RB_* flags relevant to this function
- Describe behaviour when shutdown hooks fail the reset
- Describe expected execution contexts
- Add FF copyright
- xref panic(9)
- xref this page in reboot(2)
Reviewed by: markj
Discussed with: rpokala, Pau Amma <pauamma@gundo.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39133
Allow users to configure the address to send carp messages to. This
allows carp to be used in unicast mode, which is useful in certain
virtual configurations (e.g. AWS, VMWare ESXi, ...)
Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D38940
Allow carp configuration information to be supplied and retrieved via
netlink.
Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D39048
It got disabled in 2003:
commit acb18acfec
Author: Poul-Henning Kamp <phk@FreeBSD.org>
Date: Sun Feb 23 18:09:05 2003 +0000
Bracket the kern.vnode sysctl in #ifdef notyet because it results
in massive locking issues on diskless systems.
It is also not clear that this sysctl is non-dangerous in its
requirements for locked down memory on large RAM systems.
There does not seem to be practical use for it and the disabled routine
does not work anyway.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D39127
Commit 11913870 (#14567) added cmn_err_once() by #define'ing a
compound statement but failed to consider usage in a single
statement brace-less if else.
Fix the problem by using the common "do {} while (0)" construct.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes#14629
Under some configurations, GCC didn't predefined macro 'powerpc' for
such a target. Use the guaranteed macro '__powerpc__' instead.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: WHR <msl0000023508@gmail.com>
Closes#14631
be_clone calls be_clone_cb and both call set_error on the return
error path. set_error prints the error resulting in a double print.
be_clone_cb should just return the error code and allow be_clone
to print it.
PR: 265248
Reported by: Graham Perrin
Reviewed by: imp, kevans
Pull Request: https://github.com/freebsd/freebsd-src/pull/685
After addressing coverity complaints involving `nvpair_name()`, the
compiler started complaining about dropping const. This lead to a rabbit
hole where not only `nvpair_name()` needed to be constified, but also
`nvpair_value_string()`, `fnvpair_value_string()` and a few other static
functions, plus variable pointers throughout the code. The result became
a fairly big change, so it has been split out into its own patch.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14612
discover_cached_paths() will write a NULL into a string from a nvlist to
use it as a substring, but does not restore it before return. This
corrupts the nvlist. It should be harmless unless the string is needed
again later, but we should not do this, so let us fix it.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14612
The strings returned from parsing nvlists should be immutable, but to
simplify the code when we want a substring from it, we sometimes will
write a NULL into it and then restore the value afterward. Provided
there is no concurrent access, this is okay, unless we forget to restore
the value afterward. This was caught when constifying string functions
related to nvlists.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14612
It was used by ancient GCC assert.h. Prior to 2001 GCC used to provide
its own assert.h The GCC assert.h required __eprintf to emit the error
message. FreeBSD's own assert.h never used this.
Reviewed by: ed (previously), imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2597
Currently calls to kfpu_begin() and kfpu_end() are split between
the init() and fini() functions of the particular SIMD
implementation. This was done in #14247 as an optimization measure
for the ABD adapter. Unfortunately the split complicates FPU
handling on platforms that use a local FPU state buffer, like
Windows and macOS.
To ease porting, we introduce a boolean struct member in
fletcher_4_ops_t, indicating use of the FPU, and move the FPU state
handling from the SIMD implementations to the call sites.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes#14600
Add mostly glibc and msl compatible secure_getenv. Return NULL if
issetugid() indicates the process is tainted, otherwise getenv(x). The
rational behind this is the fact that many Linux applications use this
function instead of getenv() as it's widely consider a, "best
practice".
Reviewed by: imp, mjg (feedback)
Pull Request: https://github.com/freebsd/freebsd-src/pull/686
Signed-off-by: Lucy Marsh <seafork@disroot.org>
The referenced symbols that provide init array boundaries are weak,
hidden, and undefined. The code that iterates over that arrays is not
used for the case when libc is compiled as dso.
This should fix linking with ld.bfd.
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
The current name was a historical curiosity that started when init array
support was added, and then the file appeared a convenient place for the
addition of the MI common code to csu. It is now referenced by name in
single place and the rename is easy, so do it.
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
Do not call CPUID on each ireloc, instead call it once and cache
results, similar to how it is done on powerpc64.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D37220
Why? Most trivial point, it shaves around 600 bytes from the dynamic
binaries on amd64. Less trivial, the removed code is no longer part of
the ABI, and we can ship updates to it with libc updates. Right now most
of the csu is linked into the binaries and require us to do somewhat
tricky ABI compat when it needs to change. For instance, the init_array
change would be much simpler and does not require note tagging if we
have init calling code in libc.
This could be improved more, by splitting dynamic and static
initialization. For instance, &_DYNAMIC tests can be removed then.
Such change, nonetheless, would require building libc three times.
I left this for later, after this change stabilizes, if ever.
Reviewed by: markj
Discussed with: jrtc27 (some objections, see the review), imp
Tested by: markj (aarch64)
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D37220
We may try to build ZFS inside container too,
but let's just sync them for now.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#14605
Block Cloning allows to manually clone a file (or a subset of its
blocks) into another (or the same) file by just creating additional
references to the data blocks without copying the data itself.
Those references are kept in the Block Reference Tables (BRTs).
The whole design of block cloning is documented in module/zfs/brt.c.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Christian Schwarz <christian.schwarz@nutanix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Pawel Jakub Dawidek <pawel@dawidek.net>
Closes#13392
Summary:
All cap_* system calls would fail when capability mode support is
not present.
MFC after: 2 weeks
Reviewed by: emaste, pauamma
Differential Revision: https://reviews.freebsd.org/D38976
This standalone module is the last vestage of ATM support in the tree so
send it on its way.
Reviewed by: manu, emaste
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D38880
Most ATM support was removed prior to FreeBSD 12. The netgraph support
was kept as it was less intrusive, but it is presumed to be unused.
Reviewed by: manu
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D38879
The intent is that this is like ENOTSUP, but specifically for when
something can't be done because we have no support for the requested
crypto parameters; eg unlocking a dataset or receiving a stream
encrypted with a suite we don't support.
Its not intended to be recoverable without upgrading ZFS itself.
If the request could be made to work by enabling a feature or modifying
some other configuration item, then some other code should be used.
load-key: In the future we might have more crypto suites (ie new values
for the `encryption` property. Right now trying to load a key on such
a future crypto suite will look up suite parameters off the end of the
crypto table, resulting in misbehaviour and/or crashes (or, with debug
enabled, trip the assertion in `zio_crypt_key_unwrap`).
Instead, lets check the value we got from the dataset, and if we can't
handle it, abort early.
recv: When receiving a raw stream encrypted with an unknown crypto
suite, `zfs recv` would report a generic `invalid backup stream`
(EINVAL). While technically correct, its not super helpful, so lets
ship a more specific error code and message.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes#14577
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes#14567
libcasper(3) is not used in bhyve. So move dependency to the appropriate
place.
Reviewed by: markj
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38905
The skeleton file module/icp/include/generic_impl.c can be used for
iterating over different implementations of algorithms.
It is used by SHA256, SHA512 and BLAKE3 currently.
The Solaris SHA2 implementation got replaced with a version which is
based on public domain code of cppcrypto v0.10.
These assembly files are taken from current openssl master:
- sha256-x86_64.S: x64, SSSE3, AVX, AVX2, SHA-NI (x86_64)
- sha512-x86_64.S: x64, AVX, AVX2 (x86_64)
- sha256-armv7.S: ARMv7, NEON, ARMv8-CE (arm)
- sha512-armv7.S: ARMv7, NEON (arm)
- sha256-armv8.S: ARMv7, NEON, ARMv8-CE (aarch64)
- sha512-armv8.S: ARMv7, ARMv8-CE (aarch64)
- sha256-ppc.S: Generic PPC64 LE/BE (ppc64)
- sha512-ppc.S: Generic PPC64 LE/BE (ppc64)
- sha256-p8.S: Power8 ISA Version 2.07 LE/BE (ppc64)
- sha512-p8.S: Power8 ISA Version 2.07 LE/BE (ppc64)
Tested-by: Rich Ercolani <rincebrain@gmail.com>
Tested-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes#13741
These are added via HWCAP interface:
- zfs_neon_available() for arm and aarch64
- zfs_sha256_available() for arm and aarch64
- zfs_sha512_available() for aarch64
This one via cpuid() call:
- zfs_shani_available() for x86_64
Tested-by: Rich Ercolani <rincebrain@gmail.com>
Tested-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes#13741
We had three sha2.h headers in different places.
The FreeBSD version, the Linux version and the generic solaris version.
The only assembly used for acceleration was some old x86-64 openssl
implementation for sha256 within the icp module.
For FreeBSD the whole SHA2 files of FreeBSD were copied into OpenZFS,
these files got removed also.
Tested-by: Rich Ercolani <rincebrain@gmail.com>
Tested-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes#13741
The behavior is the same as in capability mode, it does not actually
return EINVAL for absolute lookups:
openat(AT_FDCWD,"/tmp/test",O_RDONLY|O_DIRECTORY,00) = 3 (0x3)
openat(3,"../../",O_RDONLY|0x800000,00) ERR#93 'Capabilities insufficient'
openat(3,"/etc/passwd",O_RDONLY|0x800000,00) ERR#93 'Capabilities insufficient'
Fixes: 1f305be43 ("Document {O,AT}_RESOLVE_BENEATH...")
Reviewed by: kib, pauamma (manpages), emaste
Sponsored by: https://www.patreon.com/valpackett
Pull Request: https://github.com/freebsd/freebsd-src/pull/680
Differential Revision: https://reviews.freebsd.org/D38675
In preparation for the removal of the IP_RSS_LISTEN_BUCKET socket
option.
PR: 261398 (exp-run)
Reviewed by: glebius
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D38821
Rephrase double negated sentences to improve readability
OpenBSD has done the same in the past to their man 3 daemon
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/671
In preparation for the removal of the IP(V6)_BINDMULTI option.
PR: 261398 (exp-run)
Reviewed by: glebius
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D38574
When doing a fully incremental build (with WITHOUT_CLEAN enabled), from
a commit before llvm 15 was merged (3264f6b88f), to a commit after
that, a number of .inc files were not regenerated. This could lead to
unexpected compilation errors when these .inc files were included from
llvm-project sources, similar to:
In file included from /usr/src/contrib/llvm-project/clang/lib/CodeGen/CGBuiltin.cpp:8268:
/usr/obj/usr/src/amd64.amd64/lib/clang/libclang/clang/Basic/arm_mve_builtin_cg.inc:279:18: error: no matching constructor for initialization of 'clang::CodeGen::Address'
Address Val2 = Address(Val1, CharUnits::fromQuantity(2));
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work around this by making the .inc files dependent on the tblgen binary
used for generating them. E.g., we can relatively safely assume that if
the binary gets updated, the .inc files must also be updated. (Although
this is not 100% optimal, the gain by complicating things even more is
probaby not worth the effort.)
MFC after: 3 days
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D38770
- Be consistent with RFC references, so add a space after 'RFC'
- Add a LIBRARY section
- Use standard integer types in the SYNOPSIS section
Obtained from: DragonflyBSD
MFC after: 5 days
Differential Revision: https://reviews.freebsd.org/D27548
In commit 0a5b942d4 the FreeBSD SECTION_STATIC macro was set to
".rodata". This assembler directive is supported by LLVM (as a
convenience alias for ".section .rodata") by not by GNU as.
This caused the FreeBSD builds that are done with gcc to fail.
Therefore, use ".section .rodata" instead, similar to the other
asm_linkage.h headers.
Reviewed-by: Mateusz Guzik <mjguzik@gmail.com>
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Dimitry Andric <dimitry@andric.com>
Closes#14526
For compatibility with glibc. The previous code would trigger a division
by zero in roundup() and terminate. Instead, just pass through to
malloc() for align == 0.
PR: 269688
Reviewed by: imp, mjg
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/655
The prior code mistakently thought that the g_consumer that hung off the
provider we found were the right thing to use to find all the glabel
aliases for this node. However, the only way to find that is to iterate
through all the geoms that belong to the glabel geom class, looking for
those geoms with the same name as the provider with the right efimedia.
Do this in a way that caches glabel class, and allows for it to be
absent. Tighten the filter for mounted filesystems to only look
for the ones that are mounted on /dev/.. since the rest of the code
assumes that.
MFC After: 3 days
Sponsored by: Netflix
Reviewed by: corvink, asomers
Differential Revision: https://reviews.freebsd.org/D38619
In path_to_dp, allow passing in either the actual device path "eg
/dev/foo/bar" or the path where the device is mounted (say
/mnt/baz/bing). In the former case we'll assume the path within the
device is nothing (the relpath). In the latter, we'll take from the
mount point on down as the relpath.
Sponsored by: Netflix
Reviewed by: corvink, manu, asomers
Differential Revision: https://reviews.freebsd.org/D38616
Allow there to be NULL paths in buildp. This lets us return the device
path to the partition, as well as to files on the partition.
Sponsored by: Netflix
Reviewed by: corvink, manu, asomers
Differential Revision: https://reviews.freebsd.org/D38615
If there's no efimedia attribute on the provider, and the provider's a
glabel, then find the 'parent' geom. In this case, the provider's name
is label-type/name, but the geom's label will that of the underlying
device (eg ada0p1). If it is, recurisvely call find_geom_efimedia with
the geom's name, which shuold have the efimedia attribute.
Sponsored by: Netflix
Reviewed by: corvink, manu, asomers
Differential Revision: https://reviews.freebsd.org/D38614
Add a note about why we need these extra compiler flags to suppress
warnings. EDK2 upstream code is being used verbatim and as of the last
update these issuse persist, but are benign.
Sponsored by: Netflix
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D38613
profil(II) is in the scanned 3rd edition manual that we have. We don't
have the 3rd edition sources, nor do we have the 4th edition souces. We
have a mostly complete (missing pipes) 4th edition C rewrite where
profil system call number is reserved, but it's not implemented (it's in
the manx section for things that apeared to have been in 3rd edition but
weren't yet part of the reimplemented 4th edition). The 5th edition
sources we have do have it, however. For other items that have appeared
in earlier manuals, we've added the simple verbage to the manual and
relegated the rest of the data for that file to the commit message.
To allow to run a newer world on a pre-1400079 kernel a compat shims to
the sched_affinity functions has beed added.
Reported by: antoine
Tested by: antoine
Reviewed by: kib
Differential revision: https://reviews.freebsd.org/D38555
MFC after: 3 days
Use it instead of defining IN_RTLD by base sources that want P_OSREL_
defines in userspace, but are not rtld.
This allows to remove abuse of IN_RTLD from userspace.
Reviewed by: dchagin, markj, imp
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38585
strlcat() is supposed to be given the length of the destination buffer,
including the existing contents. Unfortunately, I had been overzealous
when I wrote a51288aabb, since I gave it
the length of the destination buffer, minus the existing contents. This
likely caused a regression on large strings.
On the topic of being overzealous, the use of strlcat() in
dmu_send_estimate_fast() was unnecessary because recv_clone_name is a
fixed length string. We continue using strlcat() mostly as defensive
programming, in case the string length is ever changed, even though it
is unnecessary.
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14476
Simplify the tests for 32-bit arm soft float support. For the files
included only on arm, drop the test entirely. For others, test
MACHINE_CPUARCH against arm.
No functional change intended. File lists appear the same before / after
the change.
Sponsored by: Netflix
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D38582
To avoid confusing people, rename linux_timer.h to linux_time.h,
as linux_timer.c is the implementation of timer syscalls only,
while linux_time.c contains implementation of all stuff declared
in linux_time.h.
MFC after: 2 weeks
No need to check the mode again here; we know that `iop` wraps the
correct fd.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D37987
getopt_long(3) will not allow an `optind` setting of 0 to be bug-for-bug
compatible with the GNU implementation, as some software does rely on
it. Document it as a BUG, since it affects previous declarations of
compatibility with getopt(3).
Reviewed by: pauamma (markup)
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37867
With GCC 4.2 out of the tree for a while now and no sign of it
returning, we don't really need to support older versions that don't
allow us to specify a ctor priority anymore.
Noticed by: mjg
This function has been around since 4.4BSD but was dropped upstream in 2020. This went unnoticed when tzcode was updated. Bring it back, but prepare for removing it before 14.0 is released.
PR: 269445
MFC after: 3 days
Reported by: val@packett.cool
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D38445
This updates llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and
openmp to llvmorg-15-init-17826-g1f8ae9d7e7e4, the last commit before
the upstream release/16.x branch was created.
PR: 265425
MFC after: 2 weeks
GELI allows to read a user key from a standard input.
However if user initialize multiple providers at once, the standard
input will be empty for the second and next providers.
This caused GELI to encrypt a master key with an empty key file.
This commits initialize the HMAC with the key file, and then reuse the
finalized structure to generate different encryption keys for different
providers.
Reported by: Nathan Dorfman
Tested by: philip
Security: FreeBSD-SA-23:01.geli
Security: CVE-2023-0751
Merge commit '755d9301ca89f02956fd17858b9d4d821ab5c972' from the
vendor branch. This updates us from lua 5.4.2 to 5.4.4.
In addition, it switches around how we flavor liblua for the boot loader
and flua. This is done to reduce diffs with upstream and make it easier
to import new versions (the current method has too many conflicts to
resolve by hand): we include luaconf.local.h from luaconf.h (the only
change to this file is now that #include at the end). We then define
what we need to: for flua (which does very little) and one for stand
(which creates the new FLOAT type out of int64).
Sponsored by: Netflix
This unbloat FreeBSD-utilities a bit and not everyone uses
valectl which is the only in-tree consumer of libnetmap
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D38225
It contain all the binaries and libs from the elftoolchain contrib
project except for libelf which is used everywhere.
All of those tools are never used by the average user.
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D38224
This is not exhaustive - DWARF 5 has some new enumeration types not
implemented here - but I think I caught all the ones that are extended
in DWARF 5, plus the new compilation unit type (DW_UT_*), needed when
parsing .debug_info headers.
These were useful when extending libdwarf/ctfconvert/readelf to handle
DWARF generated by gcc 12, which is version 5 by default.
Reviewed by: emaste
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38273
In https://github.com/openzfs/zfs/pull/14228 the FreeBSD
SECTION_STATIC was set to ".data" instead of ".rodata". This
commit just restores it back to .rodata.
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Jorgen Lundman <lundman@lundman.net>
Closes#14460
While there, remove .Tn from man pages.
Also remove an obsolete comment about the 80386.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans, allanjude
Differential Revision: https://reviews.freebsd.org/D38373
Casting to int truncates size on some platforms, resulting swab not
copying all the data. Cast len to size_t to avoid right shifting a
signed value: we know here it's > 0, so we can safely cast it w/o losing
precision.
In addition, be more careful with signedness of char pointers and
temporaries. Downgrade tmp from unsigned long to unsigned char since
we're only reading and writing characters.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/516
Add missing logic to allow in-kernel VFP usage for ARMv7 NEON.
The implementation is strongly based on arm64 code.
It introduces a family of fpu_kern_* functions to enable the usage
of VFP instructions in kernel.
Apart from that the existing armv7 VFP logic was modified,
taking into account that the state of the VFP registers can now
be modified in the kernel.
Co-developed by: Wojciech Macek <wma@FreeBSD.org>
Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D37419
While here, move notes about FreeBSD-specific functionality to the
COMPATIBILITY section, and document the ECAPMODE error for shm_open().
Reviewed by: pauamma, kib
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38282
In preparation for a follow-up revision wherein kvmclock may export
timekeeping info to userspace even in the absence of AMDID_RDTSCP, fall
back to using rdtsc when rdtscp isn't available. This mimics
pvclock_read_time_info() in the kernel.
Reviewed by: kib
Tested by: Shrikanth R Kamath <kshrikanth@juniper.net>
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38341
We accidentally reused variable name "i" for inner and outer loops.
Reviewed-by: Rich Ercolani <Rincebrain@gmail.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: Reno Reckling <e-github@wthack.de>
Closes#14452Closes#14445
After commit 19d3961, progress reporting (-v) with replication flag
enabled does not report the progress on the console. This commit
fixes the issue by updating the logic to check for pa->progress
instead of pa_verbosity in send_progress_thread().
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
Closes#14448
When __thr_pshared_offpage() is called for allocation, it must not use
the cached offpage for the key. Instead, the cached offpage must be
unmapped and removed from the cache, if any.
It is legitimate for the user code to unmap the shared lock object without
destroying it, and then mapping something over the freed VA to carry
another shared lock. In this case the cached offpage must be un-cached.
PR: 269277
Reported by: rau8344@gmail.com
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38345
Remove the hard-coded dependency on HYPERV being only x86. Instead, 100%
rely on MK_HYPERV. It's always right (since it's marked BROKEN (so set
to "no") on architectures we don't support).
Sponsored by: Netflix
Reviewed by: bz
Differential Revision: https://reviews.freebsd.org/D38306
Under Linux to sched_[g|s]etaffinity() functions the value returned from a call
to gettid(2) (thread id) can be passed in the argument pid. Specifying pid as 0
will set the attribute for the calling thread, and passing the value returned
from a call to getpid(2) (process id) will set the attribute for the main thread
of the thread group.
Native cpuset(2) family of system calls has "which" argument to determine how
the value of id argument is interpreted, i.e., CPU_WHICH_TID is used to pass
a thread id and CPU_WHICH_PID - to pass a process id.
For now native sched_[g|s]etaffinity() implementation is wrong as uses "which"
CPU_WHICH_PID to pass both (process and thread id) to the kernel. To fix this
adding a new "which" CPU_WHICH_TIDPID intended to handle both id's.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D38209
MFC after: 1 week
Since f35093f8 semantics of a thread affinity functions is changed to be a
compatible with Linux:
In case of getaffinity(), the minimum cpuset_t size that the kernel permits is
the maximum CPU id, present in the system, / NBBY bytes, the maximum size is not
limited.
In case of setaffinity(), the kernel does not limit the size of the user-provided
cpuset_t, internally using only the meaningful part of the set, where the upper
bound is the maximum CPU id, present in the system, no larger than the size of
the kernel cpuset_t.
To match pthread_attr_[g|s]etaffinity_np checks of the user-provided cpusets to
the kernel behavior export the minimum cpuset_t size allowed by running kernel
via new sysctl kern.sched.cpusetsizemin and use it in checks.
Reviewed by:
Differential Revision: https://reviews.freebsd.org/D38112
MFC after: 1 week
The PCBGROUP option and KPI were removed entirely in 93c67567e0.
Reviewed by: pauamma (manpages), glebius, melifaro
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38187
Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.
This reverts commit adc3c128c6.
Reviewed by: brooks, kevans, jhb (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33108
Upstream luaconf.h is contrib/lua/src/luaconf.h.dist, while userland lua
and loader lua have copies in lib/liblua/luaconf.h and
stand/liblua/luaconf.h.
Adjust whitespace, VCS tags, etc. to match upstream's version, for ease
of comparison.
Reviewed By: imp
Sponsored By: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38206
Notable upstream pull request merges:
#13805 Configure zed's diagnosis engine with vdev properties
#14110 zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
#14121 Batch enqueue/dequeue for bqueue
#14123 arc_read()/arc_access() refactoring and cleanup
#14159 Bypass metaslab throttle for removal allocations
#14243 Implement uncached prefetch
#14251 Cache dbuf_hash() calculation
#14253 Allow reciever to override encryption property in case of replication
#14254 Restrict visibility of per-dataset kstats inside FreeBSD jails
#14255 Zero end of embedded block buffer in dump_write_embedded()
#14263 Cleanups identified by CodeQL and Coverity
#14264 Miscellaneous fixes
#14272 Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names
#14287 FreeBSD: Remove stray debug printf
#14288 Colorize zfs diff output
#14289 deadlock between spa_errlog_lock and dp_config_rwlock
#14291 FreeBSD: Fix potential boot panic with bad label
#14292 Add tunable to allow changing micro ZAP's max size
#14293 Turn default_bs and default_ibs into ZFS_MODULE_PARAMs
#14295 zed: add hotplug support for spare vdevs
#14304 Activate filesystem features only in syncing context
#14311 zpool: do guid-based comparison in is_vdev_cb()
#14317 Pack zrlock_t by 8 bytes
#14320 Update arc_summary and arcstat outputs
#14328 FreeBSD: catch up to 1400077
#14376 Use setproctitle to report progress of zfs send
#14340 Remove some dead ARC code
#14358 Wait for txg sync if the last DRR_FREEOBJECTS might result in a hole
#14360 libzpool: fix ddi_strtoull to update nptr
#14364 Fix unprotected zfs_znode_dmu_fini
#14379 zfs_receive_one: Check for the more likely error first
#14380 Cleanup of dead code suggested by Clang Static Analyzer
#14397 Avoid passing an uninitialized index to dsl_prop_known_index
#14404 Fix reading uninitialized variable in receive_read
#14407 free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report
#14418 Introduce minimal ZIL block commit delay
#14422 x86 assembly: fix .size placement and replace .align with .balign
Obtained from: OpenZFS
OpenZFS commit: 9cd71c8604
The .align directive used to align storage locations is
ambiguous. On some platforms and assemblers it takes a byte count,
on others the argument is interpreted as a shift value. The current
usage expects the first interpretation.
Replace it with the unambiguous .balign directive which always
expects a byte count, regardless of platform and assembler.
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes#14422
In the zstream code, Coverity reported:
"The argument could be controlled by an attacker, who could invoke the
function with arbitrary values (for example, a very high or negative
buffer size)."
It did not report this in the kernel. This is likely because the
userspace code stored this in an int before passing it into the
allocator, while the kernel code stored it in a uint32_t.
However, this did reveal a potentially real problem. On 32-bit systems
and systems with only 4GB of physical memory or less in general, it is
possible to pass a large enough value that the system will hang. Even
worse, on Linux systems, the kernel memory allocator is not able to
support allocations up to the maximum 4GB allocation size that this
allows.
This had already been limited in userspace to 64MB by
`ZFS_SENDRECV_MAX_NVLIST`, but we need a hard limit in the kernel to
protect systems. After some discussion, we settle on 256MB as a hard
upper limit. Attempting to receive a stream that requires more memory
than that will result in E2BIG being returned to user space.
Reported-by: Coverity (CID-1529836)
Reported-by: Coverity (CID-1529837)
Reported-by: Coverity (CID-1529838)
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14285
Introduce four new vdev properties:
checksum_n
checksum_t
io_n
io_t
These properties can be used for configuring the thresholds of zed's
diagnosis engine and are interpeted as <N> events in T <seconds>.
When this property is set to a non-default value on a top-level vdev,
those thresholds will also apply to its leaf vdevs. This behavior can be
overridden by explicitly setting the property on the leaf vdev.
Note that, these properties do not persist across vdev replacement. For
this reason, it is advisable to set the property on the top-level vdev
instead of the leaf vdev.
The default values for zed's diagnosis engine (10 events, 600 seconds)
remains unchanged.
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Rob Wing <rob.wing@klarasystems.com>
Sponsored-by: Seagate Technology LLC
Closes#13805
If zfs_receive_one() gets back EINVAL, check for the more likely case,
embedded block pointers + encryption and return that error, before
falling back to the less likely case, a resumable stream when the
kernel has not been upgraded to support resume.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Sponsored-by: rsync.net
Sponsored-by: Klara Inc.
Closes#14379
Add new macro ASMABI used by Windows to change
calling API to "sysv_abi".
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Jorgen Lundman <lundman@lundman.net>
Closes#14228
This allows parsing of zfs send progress by checking the process
title.
Doing so requires some changes to the send code in libzfs_sendrecv.c;
primarily these changes move some of the accounting around, to allow
for the code to be verbose as normal, or set the process title. Unlike
BSD, setproctitle() isn't standard in Linux; thus, borrowed it from
libbsd with slight modifications.
Authored-by: Sean Eric Fagan <sef@FreeBSD.org>
Co-authored-by: Ryan Moeller <ryan@iXsystems.com>
Co-authored-by: Ameer Hamza <ahamza@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
Closes#14376
There's no reason to pass the mixer as an argument since we can fetch it
from mix_dev. No functional change intended.
Reviewed by: markj, hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D38076
This allows us to use the TAILQ_PREV and TAILQ_FOREACH_REVERSE_* macros,
useful for an out-of-tree consumer.
Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D38055
Cast char's through unsigned char before storing as an integer in
xdr_char(), this ensures that the encoded form is consistently not
sign-extended following Open Solaris's example.
Prior to this change, platforms with signed chars would sign extend
values with the high bit set but ones with unsigned chars would not
so 0xff would be stored as 0x000000ff on unsigned char platforms and
0xffffffff on signed char platforms. Decoding has the same
result for either form so this is a largely cosmetic change, but it
seems best to produce consistent output.
For more discussion, see https://github.com/openzfs/zfs/issues/14173
Reviewed by: mav, imp
Differential Revision: https://reviews.freebsd.org/D37992
* Replay 2010[acflm] which had been merged but not recorded.
* Merge 2010n.
* Reorganize (unsplit) the code to match the upstream layout.
* Merge 2022[cdefg].
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Charles Suh <charles.suh@gmail.com>
Closes#14360
This commit supports for spare vdev hotplug. The
spare vdev associated with all the pools will be
marked as "Removed" when the drive is physically
detached and will become "Available" when the
drive is reattached. Currently, the spare vdev
status does not change on the drive removal and
the same is the case with reattachment.
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
Closes#14295
These architectures fail to handle this special case, and will cause the
corresponding setjmp/_setjmp to return 0 rather than 1. Fix this and add
regression tests (also committed upstream).
PR: 268684
Reviewed by: arichardson, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D29363
Do not cache PID for a process that does not fabricate it,
calls openlog() before forking and does not call exec() thereafter.
PR: 268666
Fixes: e9ae9fa937
Tested by: kp
MFC after: 3 days
There is a lock order inversion deadlock between `spa_errlog_lock` and
`dp_config_rwlock`:
A thread in `spa_delete_dataset_errlog()` is running from a sync task.
It is holding the `dp_config_rwlock` for writer (see
`dsl_sync_task_sync()`), and waiting for the `spa_errlog_lock`.
A thread in `dsl_pool_config_enter()` is holding the `spa_errlog_lock`
(see `spa_get_errlog_size()`) and waiting for the `dp_config_rwlock` (as
reader).
Note that this was introduced by #12812.
This commit address this by defining the lock ordering to be
dp_config_rwlock first, then spa_errlog_lock / spa_errlist_lock.
spa_get_errlog() and spa_get_errlog_size() can acquire the locks in this
order, and then process_error_block() and get_head_and_birth_txg() can
verify that the dp_config_rwlock is already held.
Additionally, a buffer overrun in `spa_get_errlog()` is corrected. Many
code paths didn't check if `*count` got to zero, instead continuing to
overwrite past the beginning of the userspace buffer at `uaddr`.
Tested by having some errors in the pool (via `zinject -t data
/path/to/file`), one thread running `zpool iostat 0.001`, and another
thread runs `zfs destroy` (in a loop, although it hits the first time).
This reproduces the problem easily without the fix, and works with the
fix.
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: George Amanakis <gamanakis@gmail.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes#14239Closes#14289
- provide various missing MLINKS for library functions
- update various SEE ALSO section to include the
new linked manual pages
- add various definitions of new functions like isideogram_l(3)
- document COMPATIBILITY for some functions
- bump man page dates
Reviewed by: gbe, bcr
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/621
Differential Revision: https://reviews.freebsd.org/D37203
This adds support to color zfs diff (in the style of git diff)
conditional on the ZFS_COLOR environment variable.
Signed-off-by: Ethan Coe-Renner <coerenner1@llnl.gov>
Currently, the receiver fails to override the encryption
property for the plain replicated dataset with the error:
"cannot receive incremental stream: encryption property
'encryption' cannot be set for incremental streams.". The
problem is resolved by allowing the receiver to override
the encryption property for plain replicated send.
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
Closes#14253Closes#13533
If the fields to be listed and sorted by are constrained to those
populated by dsl_dataset_fast_stat(), then zfs list is much faster,
as it does not need to open each objset and reads its properties.
A previous optimization by Pawel Dawidek
(0cee24064a) took advantage
of this to make listing snapshot names sorted only by name much faster.
However, it was limited to `-o name -s name`, this work extends this
optimization to work with:
- name
- guid
- createtxg
- numclones
- inconsistent
- redacted
- origin
and could be further extended to any other properties supported by
dsl_dataset_fast_stat() or similar, that do not require extra locking
or reading from disk.
This was committed before (9a9e2e343dfa2af28bf7910de77ae73aa006de62),
but was reverted due to a regression when used with an older kernel.
If the kernel does not populate zc->zc_objset_stats, we now fallback
to getting the properties via the slower interface, to avoid problems
with newer userland and older kernels.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes#14110
Libarchive 3.6.2
Important bug fixes:
rar5 reader: fix possible garbled output with bsdtar -O (#1745)
mtree reader: support reading mtree files with tabs (#1783)
various small fixes for issues found by CodeQL
MFC after: 2 weeks
PR: 286306 (exp-run)
Reverting because of issue in Makefile.inc1 during native builds:
make[1]: “.../freebsd/Makefile.inc1" line 163: Unknown target aarch64:aarch64.
Since I only tested this patch with make universe on amd64, this issue wasn't caught.
This reverts commit 83bf6ab568.
On powerpc64, powerpc64le and riscv64 some software wrongly assumes that
it runs on powerpc or riscv (32-bit).
Differential revision: https://reviews.freebsd.org/D35962
Approved by: alfredo, imp
`zfs_send_cb_impl()` calls `dump_filesystems()`, which calls
`dump_filesystem()`, which will return `-1` as an error when
`zfs_open()` returns `NULL`.
This will be passed to `zfs_standard_error()`, which passes it to
`zfs_standard_error_fmt()`, which passes it to `strerror()`.
To fix this, we modify zfs_open() to set `errno` whenever it returns
NULL. Most of the cases already have `errno` set (since they pass it to
`zfs_standard_error_fmt()`, which makes this easy. Then we modify
`dump_filesystem()` to pass `errno` instead of `-1`.
Reported-by: Coverity (CID-1524598)
Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14264
* Correct some function prototypes which were documented with the wrong
pointer type.
* Clarify return values and requirements for freeing the limit handle.
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: oshogbo
Differential Revision: https://reviews.freebsd.org/D37586
GCC 12 defaults to C++17 which removes (not just deprecates)
std::auto_ptr<>. Trying to use CXXSTD of c++03 doesn't work with
libc++ headers, but c++11 does.
Reviewed by: brooks, imp, emaste
Differential Revision: https://reviews.freebsd.org/D37531
The comment indicated -Wno-deprecated-declarations was used to avoid
warnings about deprecated auto_ptr and various deprecated function
objects from <functional>. libdevdctl (now) does not use auto_ptr,
so don't mention it in the comment.
Sponsored by: The FreeBSD Foundation
libpmc used -Wno-deprecated-declarations to silence warnings about usage
of deprecated std::auto_ptr, but there is (now) now use of auto_ptr in
libpmc.
Reviewed by: mhorne
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37576
lib/googletest used -Wno-deprecated-declarations to silence warnings
about usage of deprecated std::auto_ptr, but auto_ptr is not (now) used
anywhere in googletest.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37561
Squelch false positives reported by GCC 12 with UBSan.
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes#14150
When local properties (e.g., from -o and -x) are provided, don't leak
the packed representation of the received properties due to variable
reuse.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14197
According to the UNIX standard, <pthread.h> does not include some
PTHREAD_* values which are included in <limits.h>. OpenZFS uses
some of these values in its code, and this might cause build failure on
systems that do not have these PTHREAD_* values in <pthread.h>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Minsoo Choo <minsoochoo0122@proton.me>
Closes#14225
Previously when using NO_ROOT we recorded a METALOG entry for the
pam.d/ftp hard link with a different file mode than the link target
pam.d/ftpd, which is not permitted.
This change is similar to 1dbb9994d4 for .profile
Sponsored by: The FreeBSD Foundation
It should pass `MNT_LINE_MAX`, but passes `sizeof (mntpt)`. This is
harmless because the strlen is not actually used by the helper, but
FreeBSD's Coverity scans complained about it.
This was missed in my audit of various string functions since it is not
actually passed to a string function.
Upon review, it was noticed that the helper function does not need to be
a separate function, so I have inlined it as cleanup.
Reported-by: Coverity (CID 1432079)
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: szubersk <szuberskidamian@gmail.com>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14136
FreeBSD's Coverity scans complain that we ignore the return value. There
is no need to check the return value so we cast it to (void) to suppress
further complaints by static analyzers.
Reported-by: Coverity (CID 1018175)
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: szubersk <szuberskidamian@gmail.com>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes#14136
Suppress a false positive found by new Cppcheck version.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes#14148
This passes struct vcpu down in place of struct vm and and integer
vcpu index through the in-kernel instruction emulation code. To
minimize userland disruption, helper macros are used for the vCPU
arguments passed into and through the shared instruction emulation
code.
A few other APIs used by the instruction emulation code have also been
updated to accept struct vcpu in the kernel including
vm_get/set_register and vm_inject_fault.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37161
The arguments identifying the VM and vCPU are only needed for
vm_copy_setup.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37158
GCC 12 warns that passing "" (a constant of char[1]) to a parameter of
type char[33] could potentially overread. It is not clear from the
context that c->qops can never be "auth-int" (and if it can't, then
the "auth-int" handling in DigestCalcResponse is dead code that should
be removed since this is the only place the function is called).
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D36825
Without this patch, the code in the rpcbind client forces
the use of UDP. A comment notes that some rpcbind servers
only support UDP. This makes NFSv3 mounts to Azure servers
impossible, since they require use of TCP for rpcbind.
Since the comment is very old (imported from NetBSD in 2001)
and I do not believe any UDP only rpcbind servers will
still exist, this patch comments out the code that forces
use of UDP, so that NFSv3 mounts to Azure servers can work.
For an NFSv3 mount, the "udp" mount option will still
make mount_nfs use UDP for rpcbind so that can be used
as a workaround for any old NFSv3 server that only
supports rpcbind over UDP (if any such server still exists).
I asked if doing this change is appropriate on freebsd-fs@
and I only got one reply (off list) that supported doing
the change.
PR: 267301
MFC after: 1 month
The reported bug is UFS: bad file descriptor: soft update journaling
can not be enabled on some FreeBSD-provided disk images – failed
to write updated cg.
The UFS library (libufs(3)) failed to reopen its disk descriptor
when first attempting to update a cylinder group. The error only
occurred when trying to add journaling to a filesystem whose first
cylinder group was too full to hold the journal.
PR: 259090
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Linux defaults to setting "failfast" on BIOs, so that the OS will not
retry IOs that fail, and instead report the error to ZFS.
In some cases, such as errors reported by the HBA driver, not
the device itself, we would wish to retry rather than generating
vdev errors in ZFS. This new property allows that.
This introduces a per vdev option to disable the failfast option.
This also introduces a global module parameter to define the failfast
mask value.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Mariusz Zaborski <mariusz.zaborski@klarasystems.com>
Sponsored-by: Seagate Technology LLC
Submitted-by: Klara, Inc.
Closes#14056
Most of this file was a pile of defines, apparently from Solaris that
controlled nothing in the source tree. A few things controlled the
definition of unused types or macros which I have removed.
Considerable further cleanup is possible including removal of
architectures FreeBSD never supported. This file should likely converge
with the Linux version to the extent possible.
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14127
Require that ZFS_LEGACY_SUPPORT be defined for legacy ioctl support to
be built. For now, define it in zfs_ioctl_compat.h so support is always
built. This will allow systems that need never support pre-openzfs
tools a mechanism to remove support at build time. This code should
be removed once the need for tool compatability is gone.
No functional change at this time.
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14127
This fixes -Wsingle-bit-bitfield-constant-conversion warning from
clang-16 like:
lib/libzfs/libzfs_dataset.c:4529:19: error: implicit truncation
from 'int' to a one-bit wide bit-field changes value from
1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion]
flags.nounmount = B_TRUE;
^ ~~~~~~
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14125
uu_avl and uu_list stored internal next/prev pointers and parent
pointers (unused) obfuscated (byte swapped) to hide them from a long
forgotten leak checker (No one at the 2022 OpenZFS developers meeting
could recall the history.) This would break on CHERI systems and adds
no obvious value. Rename the members, use proper types rather than
uintptr_t, and eliminate the related macros.
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14126
Avoid assuming than a uint64_t can hold a pointer.
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14131
Avoid assuming than a uint64_t can hold a pointer and reduce the
number of casts in the process.
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14131
Check __riscv_xlen == 64 rather than _LP64 and define _LP64 if missing.
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Brooks Davis <brooks.davis@sri.com>
Closes#14128
Refer to sockets rather than processes, since one can have multiple
sockets in a load-balancing group within the same process.
MFC after: 1 week
Sponsored by: Modirum MDPay
Sponsored by: Klara, Inc.
