Use PCID to avoid complete TLB shootdown when switching between user
and kernel mode with PTI enabled.
I use the model close to what I read about KAISER, user-mode PCID has
1:1 correspondence to the kernel-mode PCID, by setting bit 11 in PCID.
Full kernel-mode TLB shootdown is performed on context switches, since
KVA TLB invalidation only works in the current pmap. User-mode part of
TLB is flushed on the pmap activations as well.
Similarly, IPI TLB shootdowns must handle both kernel and user address
spaces for each address. Note that machines which implement PCID but
do not have INVPCID instructions, cause the usual complications in the
IPI handlers, due to the need to switch to the target PCID temporary.
This is racy, but because for PCID/no-INVPCID we disable the
interrupts in pmap_activate_sw(), IPI handler cannot see inconsistent
state of CPU PCID vs PCPU pmap/kcr3/ucr3 pointers.
On the other hand, on kernel/user switches, CR3_PCID_SAVE bit is set
and we do not clear TLB.
I can imagine alternative use of PCID, where there is only one PCID
allocated for the kernel pmap. Then, there is no need to shootdown
kernel TLB entries on context switch. But copyout(3) would need to
either use method similar to proc_rwmem() to access the userspace
data, or (in reverse) provide a temporal mapping for the kernel buffer
into user mode PCID and use trampoline for copy.
Reviewed by: markj (previous version)
Tested by: pho
Discussed with: alc (some aspects)
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D13985
When PTI is enabled, empty IDT slots point to rsvd_pti.
Reported by: Dexuan-BSD Cui <dexuan.bsd@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 5 days
Similarly as we already do for arm64, for mitigation is necessary to
flush branch predictor when we:
- do task switch
- receive prefetch abort on non-userspace address
The user can disable this mitigation by setting 'machdep.disable_bp_hardening'
sysctl variable, or it can check actual system status by reading
'machdep.spectre_v2_safe'
The situation is complicated by fact that:
- for Cortex-A8, the BPIALL instruction is effectively NOP until the IBE bit
in ACTLR is set.
- for Cortex-A15, the BPIALL is always NOP. The branch predictor can be
only flushed by doing ICIALLU with special bit (Enable invalidates of BTB)
set in ACTLR.
Since access to the ACTLR register is locked to secure monitor/firmware on
most boards, they will also need update of firmware / U-boot.
In worst case, when secure monitor is on-chip ROM (e.g. PandaBoard),
the board is unfixable.
MFC after: 2 weeks
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D13931
- special fault handling for break-before-make mechanism should be also
applied for instruction translation faults, not only for data translation
faults.
- since arm64_address_translate_...() functions are not atomic,
use these with disabled interrupts.
Apply r328361 to duplicate copy of ccr_gcm_soft in ccp(4).
Properly honor the lack of the CRD_F_IV_PRESENT flag in the GCM software
fallback case for encryption requests.
- N32 and N64 do not have a $a0-3 gap.
- Use 'sp += 4' to skip over the gap for O32 rather than '+= i'. It
doesn't make a functional change, but makes the code match the comment.
Sponsored by: DARPA / AFRL
Create a struct cryptop_data which contains state needed for a single
symmetric crypto operation and move that state out of the session. This
closes a race with the CRYPTO_F_DONE flag that can result in use after
free.
While here, remove the 'cse->error' member. It was just a copy of
'crp->crp_etype' and cryptodev_op() and cryptodev_aead() checked both
'crp->crp_etype' and 'cse->error'. Similarly, do not check for an
error from mtx_sleep() since it is not used with PCATCH or a timeout
so cannot fail with an error.
PR: 218597
Reviewed by: kib
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D13928
here. Return ENOMEM when we can't malloc a buffer for the DSM
TRIM. This should fix the WITNESS warnings similar to the following:
uma_zalloc_arg: zone "16" with the following non-sleepable locks held:
exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff800080c34d0) locked @ /usr/src/sys/cam/nvme/nvme_da.c:351
Reviewed by: scottl@
Sponsored by: Netflix
I suppose it should make this code NUMA-aware with recent NUMA drop-in,
trying to allocate shared memory buffers from domain closer to NT-bridge.
MFC after: 2 weeks
appeared on UFS/FFS filesystems. In some cases it was promptly followed
by a panic of "softdep_deallocate_dependencies: dangling deps". This fix
should eliminate both of these occurences.
Submitted by: Andreas Longwitz <longwitz at incore.de>
Reviewed by: kib
Tested by: Peter Holm (pho)
PR: 225423
MFC after: 1 week
type is int64. While lua is setup for the representation, it's not
setup to properly print the numbers as ints. This is the least-gross
way around that, and won't affect the bootloader where we do this.
the "power down" watchdog used by the ROM boot code is still active when the
regular watchdog is activated, turn off the power-down watchdog.
This adds support for the "fsl,ext-reset-output" FDT property. When
present, that property indicates that a chip reset is accomplished by
asserting the WDOG1_B external signal, which is supposed to trigger some
external component such as a PMIC to ready the hardware for reset (for
example, adjusting voltages from idle to full-power levels), and assert the
POR signal to SoC when ready. To guard against misconfiguation leading to a
non-rebootable system, the external reset signal is backstopped by code
that asserts a normal internal chip reset if nothing responds to the
external reset signal within one second.
in the LinuxKPI. This is done by calling finit() just before returning a magic
value of ENXIO in the "linux_dev_fdopen" function.
The Linux file structure should mimic the BSD file structure as much as
possible. This patch decouples the Linux file structure from the belonging
character device right after the "linux_dev_fdopen" function has returned.
This fixes an issue which allows a Linux file handle to exist after a
character device has been destroyed and removed from the directory index
of /dev. Only when the reference count of the BSD file handle reaches zero,
the Linux file handle is destroyed. This fixes use-after-free issues related
to accessing the Linux file structure after the character device has been
destroyed.
While at it add a missing NULL check for non-present file operation.
Calling a NULL pointer will result in a segmentation fault.
Reviewed by: kib @
MFC after: 1 week
Sponsored by: Mellanox Technologies
In a corner case we could fall into OOB error.
Authored by: Patryk Duda <pdk@semihalf.com>
Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf
Sponsored by: IBM, QCM Technologies
Specifically reading is done if ffs_sbget() and writing is done
in ffs_sbput(). These functions are exported to libufs via the
sbget() and sbput() functions which then used in the various
filesystem utilities. This work is in preparation for adding
subperblock check hashes.
No functional change intended.
Reviewed by: kib
virtual address sizes
Summary:
Some architectures use physical addresses larger than virtual. This is the
minimal changeset needed to get CAM/CTL to build on these targets. No
functional changes. More changes would likely be needed for this to be fully
functional on said platforms, but they can be made when needed.
Reviewed By: mav, chuck
Differential Revision: https://reviews.freebsd.org/D14041
than virtual
Summary:
Some architectures have physical/bus addresses that are much larger
than virtual addresses. This change just quiets a warning, as DMAP is not used
on those architectures, and on 64-bit platforms uintptr_t is the same size as
vm_paddr_t and void *.
Reviewed By: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14043
Because there was an extra declaration in the vendor version, we locally
removed the second one in r238405 with 1.0.1c. Later, upstream fixed it in
1.0.2d but they removed the first one. Therefore, both were removed in our
version unfortunately. Now we revert to the vendor one to re-add it.
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D10525
Mechanically replace uses of MALLOC/FREE with appropriate invocations of
malloc(9) / free(9) (a series of sed expressions). Something like:
* MALLOC(a, b, ... -> a = malloc(...
* FREE( -> free(
* free((caddr_t) -> free(
No functional change.
For now, punt on modifying contrib ipfilter code, leaving a definition of
the macro in its KMALLOC().
Reported by: jhb
Reviewed by: cy, imp, markj, rmacklem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14035
leaks. We assume each source can be taken / dropped only once and
don't recurse. These are only enabled via DA_TRACK_REFS or
INVARIANTS. There appreas to be a reference leak under extreme load,
and these should help us colaberatively work it out. It also documents
better the reference / holding protocol better.
Reviewed by: ken@, scottl@
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D14040
device still wind up in xpt_done after the path has been
invalidated. Since we don't always need sim or devq, add some guard
rails to only fail if we have to use them.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D14040
