In preparation for clone3 system call add struct clone_args and use it in
clone implementation.
Move all of clone related bits to the newly created linux_fork.h header.
Differential revision: https://reviews.freebsd.org/D31474
MFC after: 2 weeks
At least Linux x86 ABI's does not use carry bit and expects that the dx register
is preserved. For this add a new sv_set_fork_retval hook and call it from cpu_fork().
Add a short comment about touching dx in x86_set_fork_retval(), for more details
see phab comments from kib@ and imp@.
Reviewed by: kib
Differential revision: https://reviews.freebsd.org/D31472
MFC after: 2 weeks
As no more NetBSD code in futexes exists replace NetBSD copyrights by
standard FreeBSD 2 clause license.
Add Roman Divacky's copyrights as an author of the robust futexes.
Differential revision: https://reviews.freebsd.org/D31347
MFC after: 2 weeks
The correct condition is to check the number of ivhd entries fit into
the array.
Reported by: bz
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D31514
The Xen kernel has no symbol tables, so calling lookup_symbol against
it triggers the following Divide by Zero fault:
Loading Xen kernel...
/boot/xen data=0x2809c8+0x149638 |
!!!! X64 Exception Type - 00(#DE - Divide Error) CPU Apic ID - 00000000 !!!!
Fix lookup_symbol to prevent the #DE fault from happening if the
symbol table is not loaded and also fix loadfile_raw to mark multiboot
kernels as relocatable, since the only multiboot kernel supported is
Xen and was already unconditionally booted as relocatable.
Fixes: f75caed644 ('amd64 UEFI loader: stop copying staging area to 2M physical')
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D31507
bus_activate_resource maps memory regions as uncacheable on x86, which
is more strict than required for regions allocated using xenmem_alloc,
so don't rely on bus_activate_resource and instead map the region
using pmap_mapdev_attr and VM_MEMATTR_XEN as the cache attribute.
Sponsored by: Citrix Systems R&D
This patch adds a Lookup+Open compound RPC to the NFSv4.1/4.2
NFS client, which can be used by nfs_lookup() so that a
subsequent Open RPC is not required.
It uses the cn_flags OPENREAD, OPENWRITE added by commit c18c74a87c.
This reduced the number of RPCs by about 15% for a kernel
build over NFS.
For now, use of Lookup+Open is only done when the "oneopenown"
mount option is used. It may be possible for Lookup+Open to
be used for non-oneopenown NFSv4.1/4.2 mounts, but that will
require extensive further testing to determine if it works.
While here, I've added the changes to the nfscommon module
that are needed to implement the Deallocate NFSv4.2 operation.
This avoids needing another cycle of changes to the internal
KAPI between the NFS modules.
This commit has changed the internal KAPI between the NFS
modules and, as such, all need to be rebuilt from sources.
I have not bumped __FreeBSD_version, since it was bumped a
few days ago.
ISA 3.0 allows for nested radix translations with minimal to no
involvement of the hypervisor. This should make pseries signficantly
faster on POWER9 pseries instances, as fewer hypercalls are needed to
manage pmap now.
MFC after: 2 weeks
Relnotes: yes
Summary:
Open file list is currently created as statically allocated array (64 items).
Once this array is filled up, loader will not be able to operate with files.
In most cases, this mechanism is good enough, but the problem appears, when
we have many disks with zfs pool(s). In current loader implementation, all
discovered zfs pool configurations are kept in memory and disk devices open -
consuming the open file array. Rewrite the open file mechanism to use
dynamically allocated list.
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31364
Previously, if an encrypted netdump failed, such as due to a timeout or
network failure, the key was not saved, so a partial dump was
completely useless.
Send the key first, so the partial dump can be decrypted, because even a
partial dump can be useful.
Reviewed by: bdrewery, markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D31453
That commit moved key generation into a child process, including
a memory allocation referenced by a structure. The child wrote
the structure to the parent over a pipe, but did not write the
referenced allocation. The parent read the structure from the
child and used its pointer, which was bogus in the parent.
In the child, send both chunks of data to the parent. In the
parent, make a corresponding allocation and read both chunks.
Fixes: 372557d8c3
Reviewed by: bdrewery, markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D31452
- In g_disk_start(), verify that the data to be written is initialized
according to KMSAN shadow state.
- In g_disk_done(), verify that the block driver updated shadow state as
expected, so as to catch sources of false positives early.
Sponsored by: The FreeBSD Foundation
When cloning a ktls session (which is needed when we need to
switch output NICs for a NIC TLS session), we need to also
init the reset task, like we do when creating a new tls session.
Reviewed by: jhb
Sponsored by: Netflix
Make kdb_thr_first() and kdb_thr_next() return sane values if the
allproc list and pidhashtbl haven't been initialized yet. This can
happen if the debugger is entered very early on, for example with the
'-d' boot flag.
This allows remote gdb to attach at such a time, and fixes some ddb
commands like 'show threads'.
Be explicit about the static initialization of these variables. This
part has no functional change.
Reviewed by: markj, imp (previous version)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D31495
A number of warnings passed an exit status code to bsdar_warnc, but it
does not take exit status (as a warning, it does not exit).
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
From the (substantially larger) upstream commit:
+ call delay_output_sp to handle BSD-style padding when tputs_sp is
called, whether directly or internally, to ensure that the SCREEN
pointer is passed correctly (reports by Henric Jungheim, Juraj
Lutter).
This fixes bison segfaults observed when colourized output is enabled.
Thanks to jrtc27@ for identifying the upstream fix.
PR: 256731
MFC after: 3 days
There's no need to build both pie and non-pie .o's for stand. There's
some other build thing with MK_BEAR_SSL=yes and/or MK_LOADER_VERIEXEC=yes
that causes the pie build to fail that the 'ar' stage now. Since we don't
need the PIE stuff and the non-PIE stuff, disable PIE for the boot loader.
Reviewed by: emaste
Sponsored by: Netflix
We use the midr_el1 register to decode which CPU type we are booting
from. Read it on the secondary CPUs before waiting for the boot CPU
to release us as it will need to use it before the release.
Sponsored by: The FreeBSD Foundation
Summary:
When running on a CPU that supports the arm64 sha256 intrinsics use them
to improve perfromance of sha256 calculations.
With this changethe following improvement has been seen on an Apple M1
with FreeBS running under Parallels, with similar results on a
Neoverse-N1 r3p1.
x sha256.orig
+ sha256.arm64
+--------------------------------------------------------------------+
|++ x x|
|+++ xxx|
||A |A||
+--------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 5 3.41 3.5 3.46 3.458 0.042661458
+ 5 0.47 0.54 0.5 0.504 0.027018512
Difference at 95.0% confidence
-2.954 +/- 0.0520768
-85.4251% +/- 0.826831%
(Student's t, pooled s = 0.0357071)
Reviewed by: cem
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31284
Some simulators don't implement arbitrary sized memory access to the
virtio PCI registers. Follow Linux and use single byte accesses to read
and write to these registers.
Reviewed by: bryanv, emaste (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31424
This includes a style fix around ioflag checking as well.
Sponsored by: The FreeBSD Foundation
Reviewed by: kib, bcr
Differential Revision: https://reviews.freebsd.org/D31505
This does not appear to change generated code with the default
toolchain. However, KMSAN makes use of output operand specifications to
instrument inline asm, and with incorrect specifications we get false
positives in code that uses the CK_(S)LIST macros.
This was submitted upstream:
https://github.com/concurrencykit/ck/pull/175
The commit applies the same change locally to make KMSAN usable until
something equivalent is merged upstream.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
For now, just hook the allocation path: upon allocation, items are
marked as initialized (absent M_ZERO). Some zones are exempted from
this when it would otherwise raise false positives.
Use kmsan_orig() to update the origin map for UMA and malloc(9)
allocations. This allows KMSAN to print the return address when an
uninitialized UMA item is implicated in a report. For example:
panic: MSan: Uninitialized UMA memory from m_getm2+0x7fe
Sponsored by: The FreeBSD Foundation
Sanitizer instrumentation of course cannot automatically update shadow
state when devices write to host memory. KMSAN thus hooks into busdma,
both to update shadow state after a device write, and to verify that the
kernel does not publish uninitalized bytes to devices.
To implement this, when KMSAN is configured, each dmamap embeds a memory
descriptor describing the region currently loaded into the map.
bus_dmamap_sync() uses the operation flags to determine whether to
validate the loaded region or to mark it as initialized in the shadow
map.
Note that in cases where the amount of data written is less than the
buffer size, the entire buffer is marked initialized even when it is
not. For example, if a NIC writes a 128B packet into a 2KB buffer, the
entire buffer will be marked initialized, but subsequent accesses past
the first 128 bytes are likely caused by bugs.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31338
Use this flag to indicate that busdma should allocate a map structure
even no bouncing is required to satisfy the tag's constraints. This
will be used for KMSAN.
Also fix a memory leak that can occur if the kernel fails to allocate
bounce pages in bounce_bus_dmamap_create().
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31338
Interrupt and exception handlers must call kmsan_intr_enter() prior to
calling any C code. This is because the KMSAN runtime maintains some
TLS in order to track initialization state of function parameters and
return values across function calls. Then, to ensure that this state is
kept consistent in the face of asynchronous kernel-mode excpeptions, the
runtime uses a stack of TLS blocks, and kmsan_intr_enter() and
kmsan_intr_leave() push and pop that stack, respectively.
Use these functions in amd64 interrupt and exception handlers. Note
that handlers for user->kernel transitions need not be annotated.
Also ensure that trap frames pushed by the CPU and by handlers are
marked as initialized before they are used.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31467
- During boot, allocate PDP pages for the shadow maps. The region above
KERNBASE is currently not shadowed.
- Create a dummy shadow for the vm page array. For now, this array is
not protected by the shadow map to help reduce kernel memory usage.
- Grow shadows when growing the kernel map.
- Increase the default kernel stack size when KMSAN is enabled. As with
KASAN, sanitizer instrumentation appears to create stack frames large
enough that the default value is not sufficient.
- Disable UMA's use of the direct map when KMSAN is configured. KMSAN
cannot validate the direct map.
- Disable unmapped I/O when KMSAN configured.
- Lower the limit on paging buffers when KMSAN is configured. Each
buffer has a static MAXPHYS-sized allocation of KVA, which in turn
eats 2*MAXPHYS of space in the shadow map.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31295
KMSAN enables the use of LLVM's MemorySanitizer in the kernel. This
enables precise detection of uses of uninitialized memory. As with
KASAN, this feature has substantial runtime overhead and is intended to
be used as part of some automated testing regime.
The runtime maintains a pair of shadow maps. One is used to track the
state of memory in the kernel map at bit-granularity: a bit in the
kernel map is initialized when the corresponding shadow bit is clear,
and is uninitialized otherwise. The second shadow map stores
information about the origin of uninitialized regions of the kernel map,
simplifying debugging.
KMSAN relies on being able to intercept certain functions which cannot
be instrumented by the compiler. KMSAN thus implements interceptors
which manually update shadow state and in some cases explicitly check
for uninitialized bytes. For instance, all calls to copyout() are
subject to such checks.
The runtime exports several functions which can be used to verify the
shadow map for a given buffer. Helpers provide the same functionality
for a few structures commonly used for I/O, such as CAM CCBs, BIOs and
mbufs. These are handy when debugging a KMSAN report whose
proximate and root causes are far away from each other.
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
KMSAN requires two shadow maps, each one-to-one with the kernel map.
Allocate regions of the kernels PML4 page for them. Add functions to
create mappings in the shadow map regions, these will be used by the
KMSAN runtime.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31295
