Reduces severe performance degradation due to false-sharing. Note that this
does not account for hardware which can perform adjacent cacheline prefetch.
[mjg: massaged the commit message and the patch to use aligned_alloc
instead of malloc]
PR: 272238
MFC after: 1 week
to make __cxa_thread_call_dtors() operational for statically linked
binaries.
Noted by: andrew
Reviewed by: emaste, dim
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40748
This change removes pre-OpenSSL 1.1 supporting code and removes/adjusted
preprocessor conditionals which were tautilogically true as FreeBSD main
has shipped with OpenSSL 1.1+ for some time.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D40711
An adversary on the network can log in via ssh as any user by spoofing
the KDC. When the machine has a keytab installed the keytab is used to
verify the service ticket. However, without a keytab there is no way
for pam_krb5 to verify the KDC's response and get a TGT with the
password.
If both the password _and_ the KDC are controlled by an adversary, the
adversary can provide a password that the adversary's spoofed KDC will
return a valid tgt for. Currently, without a keytab, pam_krb5 is
vulnerable to this attack.
Reported by: Taylor R Campbell <riastradh@netbsd.org> via emaste@
Reviewed by: so
Approved by: so
Security: FreeBSD-SA-23:04.pam_krb5
Security: CVE-2023-3326
This replaces a set but unused noop variable with a more explicit
macro.
Reviewed by: jkoshy, mhorne
Differential Revision: https://reviews.freebsd.org/D40651
- Correct the document title
- This function has an _np suffix
- acl_strip_np has a manpage, xref it
- RETURN VALUES should describe acl_is_trivial_np, not acl_get_tag_type
Sponsored by: Klara, Inc.
- Make it clear that applications are not only encouraged to use the
pmc(3) library, but use of the hwpmc(4) interface directly is a
use-case we do not support
- Move the COMPATIBILITY section above PROGRAMMING API in hwpmc(4)
- Drop statements about the driver and library being "under development"
Reviewed by: jkoshy, emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40543
* Replace hand-rolled input tokenizer with openpam_readlinev() which supports line continuations and has better quoting and escaping.
* Simplify string handling by merging struct clnt_str and struct srvr_str into just struct tac_str.
* Each server entry in the configuration file can now have up to 255 AV pairs which will be appended to the ones returned by the server in response to a successful authorization request.
This allows nss_tacplus(8) to be used with servers which do not provide identity information beyond confirming the existence of the user.
This adds a dependency on libpam, however libtacplus is currently only used by pam_tacplus(8) (which is already always used with libpam) and the very recently added nss_tacplus(8) (which is extremely niche). In the longer term it might be a good idea to split this out into a separate library.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: pauamma_gundo.com, markj
Differential Revision: https://reviews.freebsd.org/D40285
Relnotes: yes
The current code silently ignores characters after the unit as long
the unit themselves were recognized. This commit makes expand_number(3)
to fail with EINVAL if buf did not terminate after the unit character.
Historically, the function accepts and ignores "B" as a SI unit, this
behavior is preserved and e.g. KB, MB are still accepted as aliases of
K and M, document this behavior in the manual page.
While I am there, also write a few test cases to validate the behavior.
Reviewed-by: emaste
MFC-after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D40482
The sincos() man page notes the function was added to msun in FreeBSD
9.0 which must have been an oversight in the review as it was commited
to 12.0 and then backported to the 11 branch.
So I have provided a diff to correct this to the first FreeBSD version
it did ship with which was 11.2.
Reviewed by: dim, imp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D40308
These are amd64-specific and so can't be used when targetting arm64, but
they don't appear to be needed.
No functional change intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
On an arm64 system that reports as a Cortex A72 r0p3, running
pmcstat -P CPU_CYCLES command
works, but
pmcstat -P cpu-cycles command
does not. This is because the former uses the PMU event from the JSON
source, resulting in pl_event in the log event being a small index
(here, 5) into the generated events table, whilst the latter does not
match any of the JSON events and falls back on PMC's own tables, mapping
it to the PMC event 0x14111, i.e. PMC_EV_ARMV8_EVENT_11H. Then, when
libpmc gets the PMCALLOCATE event, it tries to use the event as an index
into the JSON-derived table, but doing so only makes sense for the
former, whilst for the latter it will go way out of bounds and either
read junk (which may trigger the != NULL assertion) or segfault. As far
as I can tell we don't have anything lying around to tell us which of
the two cases we're in, but we can exploit the fact that the first
0x1000 PMC event codes are reserved, and that none of our PMU events
tables reach that number of entries yet.
PR: 268857
Reviewed by: mhorne
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D39592
In order to allow to add cpuset(2) functionality to more utilities than just
cpuset(1) move the parselist code into libutil
While here, make the code a little more "library" friendly, by returning a range
of various errors so that the consumer can check for them and report appropriate
error message to the users
(One of the planed usage is the jail(8) utility)
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D12873
When performing the syscall, specify the full name of the desired
operation. This improves grep-ability.
No functional change.
Reviewed by: jkoshy
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40334
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0.
A future update may migrate to use the OpenSSL 3.0 APIs.
PR: 271615
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Currently the PROCEXEC event only reports a single address, entryaddr,
which is the entry point of the interpreter in the typical dynamic case,
and used solely to calculate the base address of the interpreter. For
PDEs this is fine, since the base address is known from the program
headers, but for PIEs the base address varies at run time based on where
the kernel chooses to load it, and so pmcstat has no way of knowing the
real address ranges for the executable. This was less of an issue in the
past since PIEs were rare, but now they're on by default on 64-bit
architectures it's more of a problem.
To solve this, pass through what was picked for et_dyn_addr by the
kernel, and use that as the offset for the executable's start address
just as is done for everything in the kernel. Since we're changing this
interface, sanitise the way we determine the interpreter's base address
by passing it through directly rather than indirectly via the entry
point and having to subtract off whatever the ELF header's e_entry is
(and anything that wants the entry point in future can still add that
back on as needed; this merely changes the interface to directly provide
the underlying variables involved).
This will be followed up by a bump to the pmc major version.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D39595
This unifies the user object and kernel module paths in libpmcstat,
allows modules loaded from non-standard locations (e.g. from a user's
home directory when testing) to be found and, since buffer is what all
the warnings here use (they were never updated when buffer_modules were
added to pick based on where the file was found) has the side-effect of
ensuring the messages are correct.
This includes obsoleting the now-superfluous -k option in pmcstat.
This change breaks the hwpmc ABI and will be followed by a bump to the
pmc major version.
Reviewed by: jhb, jkoshy, mhorne
Differential Revision: https://reviews.freebsd.org/D40048
Whilst the former is not breaking, the latter is, and so this will be
followed by a bump to the pmc major version. This will allow the flags
to actually be usable in future, as otherwise we cannot distinguish
uninitialised stack junk from a deliberately-initialised value.
Reviewed by: jhb, mhorne
Differential Revision: https://reviews.freebsd.org/D40049
At a minimum, we need enough for the colllation format version string +
locale definition version string and a full collate_info definition,
rather than just the first two and a pointer.
Sponsored by: Klara, Inc.
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0.
A future update should migrate to use the OpenSSL 3.0 APIs.
PR: 271615
Reviewed by: emaste
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0.
Future work should migrate to use the OpenSSL 3.0 APIs.
PR: 271615
Reviewed by: emaste
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0.
Future work should migrate to use the OpenSSL 3.0 APIs.
PR: 271615
Reviewed by: emaste
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
As of OpenSSL 1.1 SSL_library_init() and SSL_load_error_strings() are
deprecated. There are replacement initialization functions but they do
not need to be called: "As of version 1.1.0 OpenSSL will automatically
allocate all resources that it needs so no explicit initialisation is
required."
Wrap both calls in an OPENSSL_VERSION_NUMBER block.
PR: 271615
Reviewed by: Pierre Pronchery <pierre@freebsdfoundation.org>
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40265
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0.
A future ldns update should migrate to use the OpenSSL 3.0 APIs.
This is a recommit of c6750ddec9 after the ldns update in
5afab0e5e5.
PR: 271615
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40082
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0.
A future change should migrate to use the OpenSSL 3.0 APIs, or just
remove libtelnet.
PR: 271615
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
Commit 0bda8d3e9f ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise have some leeway to change the cpuset_t for the whole
system, but we want to keep the vmm ioctl ABI stable.
Rework IPI reporting to avoid this problem. Along the way, make VM_RUN
a bit more efficient:
- Split vmexit metadata out of the main VM_RUN structure. This data is
only written by the kernel.
- Have userspace pass a cpuset_t pointer and cpusetsize in the VM_RUN
structure, as is done for cpuset syscalls.
- Have the destination CPU mask for VM_EXITCODE_IPIs live outside the
vmexit info structure, and make VM_RUN copy it out separately. Zero
out any extra bytes in the CPU mask, like cpuset syscalls do.
- Modify the vmexit handler prototype to take a full VM_RUN structure.
PR: 271330
Reviewed by: corvink, jhb (previous versions)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40113
Since there is only the current thread in the child, no pending readers
exist. Clear the bit, since it confuses future attempts to acquire
write ownership of the rtld locks, due to URWLOCK_PREFER_READERS flag.
To be future-proof, clear all state about pending writers and readers.
PR: 271490
Reported and tested by: KJ Tsanaktsidis <kj@kjtsanaktsidis.id.au>
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40178
Moving the APIs from OpenSSL 1.1 supporting APIs to 3.x supporting APIs
is a non-trivial effort. Require 1.1 API compatibility to unblock
updating OpenSSL in base to 3.x.
This mirrors what upstream has done in their configure.ac file.
Submitted by: Pierre Pronchery <pierre@freebsdfoundation.org>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40082
The SPDX folks have obsoleted the BSD-2-Clause-NetBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
OPENSSL_API_COMPAT can be used to specify the OpenSSL API version in
use for the purpose of hiding deprecated interfaces and enabling
the appropriate deprecation notices.
This change is a NFC while we're still using OpenSSL 1.1.1 but will
avoid deprecation warnings upon the switch to OpenSSL 3.0. A future
libfido2 update may switch to use OpenSSL 3.0 APIs.
Sponsored by: The FreeBSD Foundation
- Mention it in acl(3) as an available function, xref
- Mention it in acl_get_permset(3), as acl_get_perm_np(3) is a natural
follow-up to acl_get_permset(3)
Sponsored by: Klara, Inc.
Some highlights from NEWS:
** bio: fix CTAP2 canonical CBOR encoding in fido_bio_dev_enroll_*();
gh#480.
** New API calls:
- fido_dev_info_set;
- fido_dev_io_handle;
- fido_dev_new_with_info;
- fido_dev_open_with_info.
** Documentation and reliability fixes.
** Support for TPM 2.0 attestation of COSE_ES256 credentials.
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Some highlights from NEWS:
** Added OpenSSL 3.0 compatibility.
** Removed OpenSSL 1.0 compatibility.
** Support for FIDO 2.1 "minPinLength" extension.
** Support for COSE_EDDSA, COSE_ES256, and COSE_RS1 attestation.
** Support for TPM 2.0 attestation.
** Support for device timeouts; see fido_dev_set_timeout().
** New API calls:
- es256_pk_from_EVP_PKEY;
- fido_cred_attstmt_len;
- fido_cred_attstmt_ptr;
- fido_cred_pin_minlen;
- fido_cred_set_attstmt;
- fido_cred_set_pin_minlen;
- fido_dev_set_pin_minlen_rpid;
- fido_dev_set_timeout;
- rs256_pk_from_EVP_PKEY.
** Reliability and portability fixes.
** Better handling of HID devices without identification strings; gh#381.
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
The N_host_libs dance only makes sense if host is FreeBSD.
Even then, if MK_host_egacy is yes we need libmd
libnetbsd does not need libutil when using libegacy
Sponsored by: Juniper Networks, Inc.
Notable upstream pull request merges:
#11680 Add support for zpool user properties
#14145 Storage device expansion "silently" fails on degraded vdev
#14405 Create zap for root vdev
#14659 Allow MMP to bypass waiting for other threads
#14674 Miscellaneous FreBSD compilation bugfixes
#14692 Fix some signedness issues in arc_evict()
#14702 Fix typo in check_clones()
#14715 module: small fixes for FreeBSD/aarch64
#14716 Trim needless zeroes from checksum events
#14719 vdev: expose zfs_vdev_max_ms_shift as a module parameter
#14722 Fix "Detach spare vdev in case if resilvering does not happen"
#14723 freebsd clone range fixes
#14728 Fix BLAKE3 aarch64 assembly for FreeBSD and macOS
#14735 Fix in check_filesystem()
#14739 Fix data corruption when cloning embedded blocks
#14758 Fix VERIFY(!zil_replaying(zilog, tx)) panic
#14761 Revert "ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()"
#14774 FreeBSD .zfs fixups
#14776 FreeBSD: make zfs_vfs_held() definition consistent with declaration
#14779 powerpc64: Support ELFv2 asm on Big Endian
#14788 FreeBSD: add missing vop_fplookup assignments
#14789 PAM: support the authentication facility
#14790 Revert "Fix data race between zil_commit() and zil_suspend()"
#14795 Fix positive ABD size assertion in abd_verify()
#14798 Mark TX_COMMIT transaction with TXG_NOTHROTTLE
#14804 Correct ABD size for split block ZIOs
#14806 Use correct block pointer in block cloning case.
#14808 blake3: fix up bogus checksums in face of cpu migration
Obtained from: OpenZFS
OpenZFS commit: d96e29576c
A user may use a tool chain from a package or just use an existing
tool chain from a previous installation. There is no reason for this
to disable the installation of lib${LIB}_pic.a.
This also means we don't need to force MK_TOOLCHAIN=yes in lib/libc.
This reverts part of commit c0f5aeb032.
Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39917
As documented in listen.2 manual page, the kernel emits a LOG_DEBUG
syslog message if a socket listen queue overflows. For some appliances,
it may be desirable to change the priority to some higher value
like LOG_INFO while keeping other debugging suppressed.
OTOH there are cases when such overflows are normal and expected.
Then it may be desirable to suppress overflow logging altogether,
so that dmesg buffer is not flooded over long run.
In addition to existing sysctl kern.ipc.sooverinterval,
introduce new sysctl kern.ipc.sooverprio that defaults to 7 (LOG_DEBUG)
to preserve current behavior. It may be changed to any value
in a range of 0..7 for corresponding priority or to -1 to suppress logging.
Document it in the listen.2 manual page.
MFC after: 1 month
There were two issues with the carp key configuration in the new netlink
code.
The first is that userspace failed to actually pass the CARP_NL_KEY
attribute to the kernel, so a key was never set.
The second issue is that snl_attr_get_string() returns a pointer to the
string inside the netlink message. It does not copy the string to the
target buffer. That's somewhat inconvenient to work with in libifconfig
where we have a static buffer for the key.
Introduce snl_attr_copy_string() which can copy a string to a target
buffer and uses the 'arg' parameter to pass the buffer size, so it
doesn't accidentally exceed the available space.
Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D39874
If one interface has a carp address configured on it ifconfig would show
this information on all interfaces.
That's because the kernel does (as one would expect) not return any carp
data for interface without carp configuration. However, this wound up
not overwriting the data passed in by the caller, and leaving the (now
stale) information in pace. As a result the caller thought carp was
configured on the interface.
Zero out the input structure before making the call to the kernel, so
that we correctly return 0 if there is no carp configuration on the
queried interface.
Sponsored by: Rubicon Communications, LLC ("Netgate")
FreeBSD/powerpc64 is all ELFv2 since FreeBSD 13, even big endian. The
existing sha256 and sha512 asm code assumes that BE is all ELFv1, and LE
is ELFv2. Minor changes to add ELFv2 in the BE side gets this working
correctly on FreeBSD with latest OpenZFS import.
Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Justin Hibbits <chmeeedalf@gmail.com>
Closes#14779
When testing cap_connect() and name/addr lookup functions, skip tests if
we fail and the error is not ENOTCAPABLE. This makes the tests amenable
to running in CI without Internet connectivity.
Reviewed by: oshogbo
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D39242
Make Ethernet rules more similar to the usual layer 3 rules by also
allowing ridentifier and labels to be set on them.
Reviewed by: kp
Sponsored by: Rubicon Communications, LLC ("Netgate")
Add loongarch64 definitions & lua module setjmp asm
LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Han Gao <gaohan@uniontech.com>
Signed-off-by: WANG Xuerui <xen0n@gentoo.org>
Closes#13422
When building tar for linux, just disable HAVE_READPASSPHRASE
in config_freebsd.h and libarchive will provide for readpassphrase
Otherwise the two conflict.
For DIRDEPS_BUILD we need Makefile.depend.options to
force libegacy to be built on older FreeBSD and non-FreeBSD hosts.
Add readpassphrase to libegacy to avoid the need for libbsd on Linux
src.opts.mk disable TESTS for host if MK_host_egacy is yes
Reviewed by: imp
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39766
Add Makefile.depend.options to libnetbsd, mkimg and makefs
to ensure libegacy is build if needed.
targets/pseudo/stage/Makefile avoid the need for mtree
the staging process creates target dirs as needed anyway.
Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39757
Usage:
zpool set org.freebsd:comment="this is my pool" poolname
Tests are based on zfs_set's user property tests.
Also stop truncating property values at MAXNAMELEN, use ZFS_MAXPROPLEN.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com>
Sponsored-by: Beckhoff Automation GmbH & Co. KG.
Sponsored-by: Klara Inc.
Closes#11680
Some points for the future:
- libc is not the right place for sorting algorithms.
Probably libutil is better suited for this purpose or
a dedicated libsort. Should move all sorting algorithms
away from libc eventually.
- CheriBSD uses capabilities for memory access, and could
benefit from a standard memswap() function.
- Do something about qsort() in FreeBSD's libc like:
- Mark it deprecated on FreeBSD, as a first step,
due to missing limits on CPU time.
- Audit the use of qsort() in the FreeBSD base system
and consider swapping to other existing sorting
algorithms.
Discussed with: brooks@
Differential Revision: https://reviews.freebsd.org/D36493
This reverts commit a7469c9c0a.
This reverts commit 7d65a450cd.
This reverts commit 8dcf3a82c5.
And add it to the AVZ, this is not backwards compatible with older pools
due to an assertion in spa_sync() that verifies the number of ZAPs of
all vdevs matches the number of ZAPs in the AVZ.
Granted, the assertion only applies to #DEBUG builds - still, a feature
flag is introduced to avoid the assertion, com.klarasystems:vdev_zaps_v2
Notably, this allows to get/set properties on the root vdev:
% zpool set user:prop=value <pool> root-0
Before this commit, it was already possible to get/set properties on
top-level vdevs with the syntax <type>-<vdev_id> (e.g. mirror-0):
% zpool set user:prop=value <pool> mirror-0
This syntax also applies to the root vdev as it is is of type 'root'
with a vdev_id of 0, root-0. The keyword 'root' as an alias for
'root-0'.
The following tests have been added:
- zpool get all properties from root vdev
- zpool set a property on root vdev
- verify root vdev ZAP is created
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rob Wing <rob.wing@klarasystems.com>
Sponsored-by: Seagate Technology
Submitted-by: Klara, Inc.
Closes#14405
Several makefile depend on tools built for host.
At least when using DIRDEPS_BUILD we can build these for the
pseudo machine "host" to facilitate building on older host versions.
Ideally we would build these tools in their own directories to avoid
building more than needed.
For now, setting an appropriate default for BTOOLSPATH will suffice
Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39708
If there are less than two elements avoid executing the first
sorting loop. No functional change intended.
Reviewed by: kib@
MFC after: 1 week
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D39691
When sorting, both the C11 standard (ISO/IEC 9899:2011, K.3.6.3.2) and
the ISO/IEC JTC1 SC22 WG14 N1172 standard, does not define objects of
zero size as undefined behaviour. However Microsoft's cpp-docs does.
Add proper checks for this. Found while working on bsort(3).
Reviewed by: kib@ and emaste@
MFC after: 1 week
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D39687
The bsort(3) algorithm works by swapping objects, similarly to qsort(3),
and does not require any significant amount of additional memory.
The bsort(3) algorithm doesn't suffer from the processing time issues
known the plague the qsort(3) family of algorithms, and is bounded by
a complexity of O(log2(N) * log2(N) * N), where N is the number of
elements in the sorting array. The additional complexity compared to
mergesort(3) is a fair tradeoff in situations where no memory may
be allocated.
The bsort(3) APIs are identical to those of qsort(3), allowing for
easy drop-in and testing.
The design of the bsort(3) algorithm allows for future parallell CPU
execution when sorting arrays. The current version of the bsort(3)
algorithm is single threaded. This is possible because fixed areas
of the sorting data is compared at a time, and can easily be divided
among different CPU's to sort large arrays faster.
Reviewed by: gbe@, delphij@, pauamma_gundo.com (manpages)
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D36493
These will hopefully be fixed upstream eventually, but silence the
warnings until then.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D39518
veriexec_check_path_label() can be used to check if a specified
path has a label associated with it that contains the what we
want.
Obtained from: Juniper Networks, Inc.
Ensure veriexec opens the file before doing any read operations.
When the MAC_VERIEXEC_CHECK_PATH_SYSCALL syscall is requested, veriexec
needs to open the file before calling mac_veriexec_check_vp. This is to
ensure any set up is done by the file system. Most file systems do not
explicitly need an open, but some (e.g. virtfs) require initialization
of access tokens (file identifiers, etc.) before doing any read or write
operations.
The evaluate_fingerprint() function needs to ensure it has an open file
for reading in order to evaluate the fingerprint. The ideal solution is
to have a hook after the VOP_OPEN call in vn_open. For now, we open the
file for reading, envaluate the fingerprint, and close the file. While
this leaves a potential hole that could possibly be taken advantage of
by a dedicated aversary, this code path is not typically visited often
in our use cases, as we primarily encounter verified mounts and not
individual files. This should be considered a temporary workaround until
discussions about the post-open hook have concluded and the hook becomes
available.
Add MAC_VERIEXEC_GET_PARAMS_PATH_SYSCALL and
MAC_VERIEXEC_GET_PARAMS_PID_SYSCALL to mac_veriexec_syscall so we can
fetch and check label contents in an unconstrained manner.
Add a check for PRIV_VERIEXEC_CONTROL to do ioctl on /dev/veriexec
Make it clear that trusted process cannot be debugged. Attempts to debug
a trusted process already fail, but the failure path is very obscure.
Add an explicit check for VERIEXEC_TRUSTED in
mac_veriexec_proc_check_debug.
We need mac_veriexec_priv_check to not block PRIV_KMEM_WRITE if
mac_priv_gant() says it is ok.
Reviewed by: sjg
Obtained from: Juniper Networks, Inc.
For a process supervisor using the reaper API to track process subtrees,
it is very useful to know the state of the processes on the list.
Sponsored by: https://www.patreon.com/valpackett
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39585
Introduce the OpenBSD syntax of "scrub" option for "match" and "pass"
rules and the "set reassemble" flag. The patch is backward-compatible,
pf.conf can be still written in FreeBSD-style.
Obtained from: OpenBSD
MFC after: never
Sponsored by: InnoGames GmbH
Differential Revision: https://reviews.freebsd.org/D38025
FREEBSD32 API is also supported on aarch64. Refer to aarch64 instead of
arm64 because it's about the architecture (powerpc64), not the kernel
interface...
Sponsored by: Netflix
f6a0dac84 modified the zfs_iter_* functions to take a new "flags"
parameter, and introduced a variety of flags to ask the kernel to limit
the results in various ways, reducing the amount of work the caller
needed to do to filter out things they didn't need.
Unfortunately this change broke the ABI for existing clients (read:
older versions of the `zfs` program), and was reverted 399b98198.
dc95911d2 reintroduced the original patch, with the understanding that a
backwards-compatible fix would be made before the 2.2 release branch was
tagged. This commit is that fix.
This introduces zfs_iter_*_v2 functions that have the new flags
argument, and reverts the existing functions to not have the flags
parameter, as they were before. The old functions are now reimplemented
in terms of the new, with flags set to 0.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <george.wilson@delphix.com>
Original-patch-by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Sponsored-by: Klara, Inc.
Closes#14597
Running `zfs list -o avail rpool` resulted in a core dump.
This commit will fix this.
Run the needed overhead only, when `use_color()` is true.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes#14712
by making it accept some open(2) flags. More precisely, only
O_CLOEXEC is supported, the flag is translated into the KQUEUE_CLOEXEC flag
for kqueuex(2), and O_NONBLOCK is silently ignored.
Reported and tested by: vishwin
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39377
If a malicious casper process sent a host or service string that was
too long, cap_getnameinfo would overrun the caller's buffer by one byte.
The backends for this function needlessly allocated one extra byte
for these buffers. This was harmless, but could be confusing to readers.
Reported by: Coverity (an internal run at Dell)
Reviewed by: oshogbo, emaste
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D39347
There is no real need to close descriptors before a process exits, but
these close calls demonstrate by example that kqueue descriptors occupy
the same namespace as other file descriptors.
Reviewed by: fernape, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39376
Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"
Obtained from: OpenZFS
OpenZFS commit: 431083f75b
Get/set commands can now choose to provide the interface name rather
than the interface index. This allows userspace to avoid a call to
if_nametoindex().
Suggested by: melifaro
Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D39359
The cap_dns tests require Internet access. Currently they fail when
that's not available, which for CI purposes is undesirable. Let's
instead skip the tests if none of the non-casper name/addr lookups
succeed.
To that end:
- Convert the tests to ATF so that skipping is easier to implement.
- Break up the tests into separate test cases.
- If one of the system (i.e., non-casper) lookup functions fails, skip
the test if all of them failed, otherwise fail the tests, since
partial failure indicates something is flaky and deserves a closer
look.
Reviewed by: oshogbo
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39241
These tests fail spuriously when run in parallel, since some of them
write a value to a global sysctl, read it back, and expect to see the
written value. These tests complete quite quickly in any case.
MFC after: 1 week
memmem started as a GNU extension but is now widely available.
Reviewed by: mhorne (slightly earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39337
This does not remove LLVM_TARGET_MIPS. Note that the only
MACHINE_ARCH values ending in 'hf' were all MIPS architectures, hence
removing the pattern matches for 'hf'.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D39331
These libraries are linked to directly by applications rather than
opened at runtime via dlopen().
Discussed with: oshogbo
Reviewed by: markj, emaste
Differential Revision: https://reviews.freebsd.org/D39245
This reverts commit ab80f0b21f. The intent
of this change was to avoid possible compilation errors when certain
.inc files were not regenerated, but the method turns out to cause way
more rebuilds than anticipated. Another method will have to be found,
and in the mean time, WITH_CLEAN is the solution that always works.
Fixes: ab80f0b21f
This flag ensures that the tblgen tools do not actually touch the
produced .inc file, if there are no changes to the contents. In turn,
this may prevent a number of rebuilds of files that include such .inc
files, saving build time.
While here, ensure that the shell invocations to locate the used tblgen
binary do not show unnecessary error messages.
Reported by: des
MFC after: 1 week
There are FAT12 and FAT16 file systems, but FAT13 of was an
unintentional invention of mine ...
Reported by: Ravi Pokala <rpokala@freebsd.org>
MFC after: 1 month
This update implements tallying of free directory entries during
create, delete, or rename operations on FAT12 and FAT16 file systems.
Prior to this change, the total number of root directory entries
was reported as number of inodes, but 0 as the number of free
inodes, causing system health monitoring software to warn about
a suspected disk full issue.
The FAT12 and FAT16 file systems provide a limited number of
root directory entries, e.g. 512 on typical hard disk formats.
The valid range of values is 1 to 65535, but the msdosfs code
will effectively round up "odd" values to the next multiple of 16
(e.g. 513 would allow for 528 root directory entries).
This update implements tracking of directory entries during create,
delete, or rename operations, with initial values determined by
scanning the directory when the file system is mounted.
Total and free directory entries are reported in the f_files and
f_ffree elements of struct statfs, despite differences in semantics
of these values:
- There is no limit on the number of files and directories that can
be created on a FAT file system. Only the root directory of FAT12
and FAT16 file systems is limited, any number of files can still be
created in sub-directories, even when 0 free "inodes" are reported.
- A single file can require 1 to 21 directory entries, depending on
the character set, structure, and length of the name. The DOS 8.3
style file name takes up 1 entry, and if the name does not comply
with the syntax of a DOS 8.3 file name, 1 additional entry is used
for each 13 characters of the file name. Since all these entries
have to be contiguous, it is possible that a file or directory with
a long name can not be created, despite a sufficient total number of
free directory entries.
- Renaming a file can require more directory entries than currently
allocated to store its long name, which may prevent an in-place
update of the name if more entries are needed. This may cause a
rename operation to fail if no contiguous range of free entries for
the new name can be found.
- The volume label is stored in a directory entry. An empty FAT file
system with a volume label will therefore show 1 used "inode" in
df.
- The perceentage of free inodes shown in df or monitoring tools does
only represent the state of the root directory of a FAT12 or FAT16
file system. Neither does a reported value of 0% free inodes does
prevent files from being created in sub-directories, nor does a
value of 50% free inodes guarantee that even a single file with
a "long" name can be created in the root directory (if every other
directory entry is occupied and there are no 2 contiguous entries).
The statfs(2) and df(1) man pages have been updated with a notice
regarding the possibly different semantics of values reported as
total and free inodes for non-Unix file systems.
PR: 270053
Reported by: Ben Woods <woodsb02@freebsd.org>
Approved by: mckusick
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D38987
Fix newlocale() overwriting the locale name in collate object
when same instance was used for those locales, and querylocale()
reporting unexpected value for LC_COLLATE_MASK.
PR: 255646, 269375
Reviewed by: markj, bapt (previous version)
Differential Revision: https://reviews.freebsd.org/D30146
